Prerequisites
The following example requires thecrawl4ai
library.
Example
The following agent will scrape the content from the https://github.com/agno-agi/phidata webpage:cookbook/tools/crawl4ai_tools.py
Toolkit Params
Parameter | Type | Default | Description |
---|---|---|---|
max_length | int | 1000 | Specifies the maximum length of the text from the webpage to be returned. |
Toolkit Functions
Function | Description |
---|---|
web_crawler | Crawls a website using crawl4ai’s WebCrawler. Parameters include ‘url’ for the URL to crawl and an optional ‘max_length’ to limit the length of extracted content. The default value for ‘max_length’ is 1000. |
Information
- View on Github