Models
OpenAI
Example
OpenAI Params
| Name | Type | Default | Description |
|---|---|---|---|
name | str | "OpenAIChat" | |
model | str | "gpt-4-1106-preview" | OpenAI model ID. |
frequency_penalty | float | - | Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim. |
logit_bias | Any | - | Modify the likelihood of specified tokens appearing in the completion. Accepts a json object that maps tokens (specified by their token ID in the tokenizer) to an associated bias value from -100 to 100. |
logprobs | bool | - | Include the log probabilities on the logprobs most likely tokens, as well as the chosen tokens. |
max_tokens | int | - | The maximum number of tokens to generate in the chat completion. |
presence_penalty | float | - | Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics. |
response_format | Dict[str, Any] | - | An object specifying the format that the model must output. Setting to { "type": "json_object" } enables JSON mode, which guarantees the message the model generates is valid JSON. |
seed | int | - | If specified, openai system will make a best effort to sample deterministically, such that repeated requests with the same seed and parameters should return the same result. |
stop | Union[str, List[str]] | - | Up to 4 sequences where the API will stop generating further tokens. |
temperature | float | - | What sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. |
top_logprobs | int | - | An integer between 0 and 5 specifying the number of most likely tokens to return at each token position, each with an associated log probability. logprobs must be set to true if this parameter is used. |
user | str | - | A unique identifier representing your end-user, which can help OpenAI to monitor and detect abuse. |
top_p | float | - | An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. |
extra_headers | Any | - | Additional headers to be added to the OpenAI request. |
extra_query | Any | - | Additional query parameters to be added to the OpenAI request. |
api_key | str | - | OpenAI API Key |
organization | str | - | OpenAI organization |
base_url | Union[str, httpx.URL] | - | OpenAI Base URL |
timeout | float | - | Timeout for the OpenAI request |
max_retries | int | - | Maximum number of retries for the OpenAI request |
default_headers | Any | - | Default headers to be added to the OpenAI request. |
default_query | Any | - | Default query parameters to be added to the OpenAI request. |
client_kwargs | Dict[str, Any] | - | Additional kwargs added when creating the OpenAI() client. |
openai_client | OpenAIClient | - | Provide your own OpenAI() client to use |
For more information, please refer to the OpenAI docs as well.
LLM Params
OpenAIChat is a subclass of the LLM class and has access to the same params
| Parameter | Type | Default | Description |
|---|---|---|---|
model | str | - | ID of the model to use. |
name | str | - | Name for this LLM. Note: This is not sent to the LLM API. |
metrics | Dict[str, Any] | - | Metrics collected for this LLM. Note: This is not sent to the LLM API. |
response_format | Any | - | Format of the response. |
tools | List[Union[Tool, Dict]] | - | A list of tools provided to the LLM. Tools are functions the model may generate JSON inputs for. If you provide a dict, it is not called by the model. Always add tools using the add_tool() method. |
tool_choice | Union[str, Dict[str, Any]] | - | Controls which (if any) function is called by the model. "none" means the model will not call a function and instead generates a message. "auto" means the model can pick between generating a message or calling a function. Specifying a particular function via {"type": "function", "function": {"name": "my_function"}} forces the model to call that function. "none" is the default when no functions are present. "auto" is the default if functions are present. |
run_tools | bool | True | If True, runs tools. |
show_tool_calls | bool | - | If True, shows tool calls in the response. |
functions | Dict[str, Function] | - | Functions extracted from the tools. Note: These are not sent to the LLM API and are only used for execution. |
function_call_limit | int | 20 | Maximum number of function calls allowed. |
function_call_stack | List[FunctionCall] | - | Stack of function calls. |
system_prompt | str | - | System prompt provided to the LLM. |
instructions | str | - | Instructions provided to the LLM. |
Was this page helpful?