Groq
Groq offers blazing-fast API endpoints for large language models
Authentication
Set your GROQ_API_KEY environment variable.
Example
Use Groq with your Assistant:
Params
The name identifier for the assistant.
The specific model ID used for generating responses.
A number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model’s likelihood to repeat the same line verbatim.
A JSON object that modifies the likelihood of specified tokens appearing in the completion by mapping token IDs to bias values between -100 and 100.
The number of log probabilities to return for each generated token.
The maximum number of tokens to generate in the chat completion.
A number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model’s likelihood to talk about new topics.
Specifies the format that the model must output. Setting to { "type": "json_object" } enables JSON mode, ensuring the message generated is valid JSON.
A seed value for deterministic sampling, ensuring repeated requests with the same seed and parameters return the same result.
Up to 4 sequences where the API will stop generating further tokens.
The sampling temperature to use, between 0 and 2. Higher values like 0.8 make the output more random, while lower values like 0.2 make it more focused and deterministic.
The number of top log probabilities to return for each generated token.
Nucleus sampling parameter. The model considers the results of the tokens with top_p probability mass.
A unique identifier representing your end-user, helping to monitor and detect abuse.
Additional headers to include in the API request.
Additional query parameters to include in the API request.
The API key for authenticating requests to the service.
The organization associated with the API key.
The base URL for making API requests to the service.
The timeout duration for requests, specified in seconds.
The maximum number of retry attempts for failed requests.
Default headers to include in all API requests.
Default query parameters to include in all API requests.
An instance of GroqClient provided for making API requests.
Was this page helpful?