Models

Model Usage

In this section we will go over the inner workings of the API models usage. You can use any model listed here.

By understanding the various parameters it accepts and fine-tuning them, you can control the behavior and quality of the model's responses.

(GPT-3.5, GPT-3.5-turbo, GPT-4) Model Parameters

temperature

The temperature parameter influences the randomness of the generated responses. A higher value, such as 0.8, makes the answers more diverse, while a lower value, like 0.2, makes them more focused and deterministic. temperature value range is [0,2]

top_p

The top_p parameter, also known as nucleus sampling or "penalty" in the API, controls the diversity and quality of the responses. It limits the cumulative probability of the most likely tokens. Higher values like 0.9 allow more tokens, leading to diverse responses, while lower values like 0.2 provide more focused and constrained answers. top_p value range is [0,1]

frequency_penalty

The frequency_penalty parameter allows you to control the model's tendency to generate repetitive responses. Higher values, like 1.0, encourage the model to explore more diverse and novel responses, while lower values, such as 0.2, make the model more likely to repeat information. frequency_penalty value range is [-2,2]

presence_penalty

The presence_penalty parameter allows you to influence the model's avoidance of specific topics in its responses. Higher values, such as 1.0, make the model more likely to avoid mentioning particular topics provided in the user messages, while lower values, like 0.2, make the model less concerned about preventing those topics. presence_penalty value range is [-2,2]

(flan-t5-base-samsum) Model Parameters

repetition_penalty

repetition_penalty (optional) — The parameter for repetition penalty. A value of 1.0 means no penalty. repetition_penalty value has to be >= 0. Defaults to None.

top_k

top_k (optional) — The number of highest probability vocabulary tokens to keep for top-k-filtering. top_k value has to be <= 1. Defaults to None.

max_length

max_length (optional). Value has to <= 512. Defaults to None.

min_length

min_length (optional). Value has to be <= 512. Defaults to None.

Previous
Get Sentiment