​
Example

Spider is the fastest open source web Scraper & Crawler that returns LLM-ready data. To start using Spider, you need an API key from the Spider dashboard.

The following assistant will run a search query to get the latest news in USA and scrape the first search result. The assistant will return the scraped data in markdown format.

cookbook/tools/spider_tools.py
from phi.assistant import Assistant
from phi.tools.spider import SpiderTools

assistant = Assistant(
    tools=[SpiderTools()],
    show_tool_calls=True,
    debug_mode=True,
)

assistant.print_response('Can you scrape the first search result from a search on "news in USA"?', markdown=True)

​
Toolkit Params

max_results
int

The maximum number of search results to return

url
str

The url to be scraped or crawled

​
Toolkit Functions

search

Searches the web for the given query.

scrape

Scrapes the given url.

crawl

Crawls the given url.

Was this page helpful?

ShellSQL