Ollama is a fantastic tool for running LLMs locally. Install ollama and run a model using

ollama run llama2

After you have the local model running, use the Ollama LLM to access them

​
Usage

from phi.assistant import Assistant
from phi.llm.ollama import Ollama

assistant = Assistant(
    llm=Ollama(model="llama2"),
    description="You help people with their health and fitness goals.",
)
assistant.print_response("Share a quick healthy breakfast recipe.", markdown=True)

​
Params

model
str

Model name

host
str

Host url

format
str
default: ""

Response format, "" or "json"

timeout
Any
default: "None"

Timeout for requests

options
Dict[str, Any]
default: "None"

Dictionary of options to send with the request, example: {temperature: 0.1, stop: ['\n']}

keep_alive
Union[float, str]
default: "None"
client_kwargs
Dict[str, Any]
default: "None"

Additional {key: value} dict sent when initalizing the Ollama() client.

ollama_client
ollama.Client()
default: "None"

Provide your own ollama.Client()

Message us on discord if you need help.

Was this page helpful?

OpenAI LikeAWS Bedrock Claude