Anyscale is a great platform for providing endpoints for Open Source models.

​
Authentication

Set your ANYSCALE_API_KEY environment variable. You can get one from Anyscale here.

export ANYSCALE_API_KEY=***

​
Usage

Use Anyscale with your Assistant:

from phi.assistant import Assistant
from phi.llm.anyscale import Anyscale

assistant = Assistant(
    llm=Anyscale(model="mistralai/Mixtral-8x7B-Instruct-v0.1"),
    description="You help people with their health and fitness goals.",
)
assistant.print_response("Share a quick healthy breakfast recipe.", markdown=True)

​
Params

name
str
default: "Anyscale"

The name identifier for the assistant.

model
str
default: "mistralai/Mixtral-8x7B-Instruct-v0.1"

The specific model ID used for generating responses.

api_key
str

The API key for authenticating requests to the Anyscale service.

base_url
str
default: "https://api.endpoints.anyscale.com/v1"

The base URL for making API requests to the Anyscale service.

Was this page helpful?

TogetherFireworks