> ## Documentation Index
> Fetch the complete documentation index at: https://docs.phidata.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Spider

**SpiderTools** is an open source web Scraper & Crawler that returns LLM-ready data. To start using Spider, you need an API key from the [Spider dashboard](https://spider.cloud).

## Prerequisites

The following example requires the `spider-client` library.

```shell theme={null}
pip install -U spider-client
```

## Example

The following agent will run a search query to get the latest news in USA and scrape the first search result. The agent will return the scraped data in markdown format.

```python cookbook/tools/spider_tools.py theme={null}
from phi.agent import Agent
from phi.tools.spider import SpiderTools

agent = Agent(tools=[SpiderTools()])
agent.print_response('Can you scrape the first search result from a search on "news in USA"?', markdown=True)
```

## Toolkit Params

| Parameter     | Type  | Default | Description                                    |
| ------------- | ----- | ------- | ---------------------------------------------- |
| `max_results` | `int` | -       | The maximum number of search results to return |
| `url`         | `str` | -       | The url to be scraped or crawled               |

## Toolkit Functions

| Function | Description                           |
| -------- | ------------------------------------- |
| `search` | Searches the web for the given query. |
| `scrape` | Scrapes the given url.                |
| `crawl`  | Crawls the given url.                 |

## Information

* View on [Github](https://github.com/agno-agi/phidata/blob/main/phi/tools/spider.py)