[Feature Request]: Show pipeline steps and ability to enable/disable

[marctorsoc]

What needs to be done?

A little bit like what spacy offers (https://spacy.io/usage/processing-pipelines), for clarity to the user it'd be good to:

• Show what steps will be done e.g. fetch -> scrape -> extract
• Give the user the ability to stop the pipeline when desired
• Idk if would make sense like in spaCy to bypass steps but still continue

What problem does this solve?

My use case (but in general I just think it's convenient to show a pipeline and being able to stop it when desired) is: it's convenient to use crawl4ai to get the HTML, but then I won't use the markdown generated. It seems like the library is forcing me to extract markdown if I want to get the HTML.

Target users/beneficiaries

No response

Current alternatives/workarounds

class NoMarkdownGenerator(MarkdownGenerationStrategy):
    """Avoid generating markdown after fetching."""

    def __init__(self):
        pass

    def generate_markdown(self, *args, **kwargs) -> MarkdownGenerationResult:
        """Generate markdown from cleaned HTML."""
        return MarkdownGenerationResult(
            raw_markdown="",
            references_markdown="",
            markdown_with_citations="",
        )

and

config = CrawlerRunConfig(
       markdown_generator=NoMarkdownGenerator()
)

Proposed approach

No response