FirecrawlTools enable an Agent to perform web crawling and scraping tasks.

​
Prerequisites

The following example requires the firecrawl-py library and an API key which can be obtained from Firecrawl.

pip install -U firecrawl-py

export FIRECRAWL_API_KEY=***

​
Example

The following agent will scrape the content from https://finance.yahoo.com/ and return a summary of the content:

cookbook/tools/firecrawl_tools.py
from phi.agent import Agent
from phi.tools.firecrawl import FirecrawlTools

agent = Agent(tools=[FirecrawlTools(scrape=False, crawl=True)], show_tool_calls=True, markdown=True)
agent.print_response("Summarize this https://finance.yahoo.com/")

​
Toolkit Params

ParameterTypeDefaultDescription
api_keystrNoneOptional API key for authentication purposes.
formatsList[str]NoneOptional list of formats to be used for the operation.
limitint10Maximum number of items to retrieve. The default value is 10.
scrapeboolTrueEnables the scraping functionality. Default is True.
crawlboolFalseEnables the crawling functionality. Default is False.

​
Toolkit Functions

FunctionDescription
scrape_websiteScrapes a website using Firecrawl. Parameters include url to specify the URL to scrape. The function supports optional formats if specified. Returns the results of the scraping in JSON format.
crawl_websiteCrawls a website using Firecrawl. Parameters include url to specify the URL to crawl, and an optional limit to define the maximum number of pages to crawl. The function supports optional formats and returns the crawling results in JSON format.

​
Information

Was this page helpful?

FileGiphy