Crawl4AI

Crawl4aiTools enable an Agent to perform web crawling and scraping tasks using the Crawl4ai library.

Prerequisites

The following example requires the crawl4ai library.

pip install -U crawl4ai

Example

The following agent will scrape the content from the https://github.com/agno-agi/phidata webpage:

cookbook/tools/crawl4ai_tools.py

from phi.agent import Agent
from phi.tools.crawl4ai_tools import Crawl4aiTools

agent = Agent(tools=[Crawl4aiTools(max_length=None)], show_tool_calls=True)
agent.print_response("Tell me about https://github.com/agno-agi/phidata.")

Toolkit Params

Parameter	Type	Default	Description
`max_length`	`int`	`1000`	Specifies the maximum length of the text from the webpage to be returned.

Toolkit Functions

Function	Description
`web_crawler`	Crawls a website using crawl4ai’s WebCrawler. Parameters include ‘url’ for the URL to crawl and an optional ‘max_length’ to limit the length of extracted content. The default value for ‘max_length’ is 1000.

Information

View on Github

Composio CSV

Getting Started

Documentation

How To

Prerequisites

Example

Toolkit Params

Toolkit Functions

Information

​Prerequisites

​Example

​Toolkit Params

​Toolkit Functions

​Information

Prerequisites

Example

Toolkit Params

Toolkit Functions

Information