Tools
Spider
Example
Spider is the fastest open source web Scraper & Crawler that returns LLM-ready data. To start using Spider, you need an API key from the Spider dashboard.
The following assistant will run a search query to get the latest news in USA and scrape the first search result. The assistant will return the scraped data in markdown format.
cookbook/tools/spider_tools.py
from phi.assistant import Assistant
from phi.tools.spider import SpiderTools
assistant = Assistant(
tools=[SpiderTools()],
show_tool_calls=True,
debug_mode=True,
)
assistant.print_response('Can you scrape the first search result from a search on "news in USA"?', markdown=True)
Toolkit Params
max_results
int
The maximum number of search results to return
url
str
The url to be scraped or crawled
Toolkit Functions
search
Searches the web for the given query.
scrape
Scrapes the given url.
crawl
Crawls the given url.