Model Usage

In this section we will go over the inner workings of the API models usage. You can use any model listed here.

By understanding the various parameters it accepts and fine-tuning them, you can control the behavior and quality of the model's responses.

(GPT-3.5, GPT-3.5-turbo, GPT-4) Model Parameters


The temperature parameter influences the randomness of the generated responses. A higher value, such as 0.8, makes the answers more diverse, while a lower value, like 0.2, makes them more focused and deterministic. temperature value range is [0,2]


The top_p parameter, also known as nucleus sampling or "penalty" in the API, controls the diversity and quality of the responses. It limits the cumulative probability of the most likely tokens. Higher values like 0.9 allow more tokens, leading to diverse responses, while lower values like 0.2 provide more focused and constrained answers. top_p value range is [0,1]


The frequency_penalty parameter allows you to control the model's tendency to generate repetitive responses. Higher values, like 1.0, encourage the model to explore more diverse and novel responses, while lower values, such as 0.2, make the model more likely to repeat information. frequency_penalty value range is [-2,2]


The presence_penalty parameter allows you to influence the model's avoidance of specific topics in its responses. Higher values, such as 1.0, make the model more likely to avoid mentioning particular topics provided in the user messages, while lower values, like 0.2, make the model less concerned about preventing those topics. presence_penalty value range is [-2,2]

(flan-t5-base-samsum) Model Parameters


repetition_penalty (optional) — The parameter for repetition penalty. A value of 1.0 means no penalty. repetition_penalty value has to be >= 0. Defaults to None.


top_k (optional) — The number of highest probability vocabulary tokens to keep for top-k-filtering. top_k value has to be <= 1. Defaults to None.


max_length (optional). Value has to <= 512. Defaults to None.


min_length (optional). Value has to be <= 512. Defaults to None.

Get Sentiment