Skip to content

LLM Configurations

Elluminate allows you to customize the behavior of language models through LLM configurations. This is particularly useful when you need:

  • Different Configurations: Control configuration parameters like temperature, top_p, max_tokens, etc.
  • Multiple Model Variants: Test different models or model versions
  • Custom API Endpoints: Use different API endpoints or providers

The following example demonstrates how to use different LLM configurations:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
import os

from dotenv import load_dotenv
from elluminate import Client

load_dotenv(override=True)

# Initialize the client
client = Client()

# Create a low temperature LLM config for more deterministic responses
low_temp_config, _ = client.llm_configs.get_or_create(
    name="Low Temperature GPT-4o mini",
    llm_model_name="gpt-4o-mini",
    temperature=0.1,
    api_key=os.getenv("AZURE_OPENAI_API_KEY"),
    llm_base_url=os.getenv("AZURE_OPENAI_ENDPOINT"),
    api_version=os.getenv("OPENAI_API_VERSION"),
)  # (1)!

# Create a prompt template
prompt_template, _ = client.prompt_templates.get_or_create(
    "Create a {{length}} story about a programmer solving the {{problem}} problem.",
    name="Programming Stories",
)

# Generate evaluation criteria
client.criteria.generate_many(prompt_template, delete_existing=True)

template_variables = client.template_variables.add_to_collection(
    template_variables={"length": "short", "problem": "fizzbuzz"},
    collection=prompt_template.default_template_variables_collection,
)

# Generate response using default configuration
response = client.responses.generate(
    prompt_template,
    template_variables=template_variables,
)

# Generate response using the low temperature configuration
low_temp_response = client.responses.generate(
    prompt_template,
    template_variables=template_variables,
    llm_config=low_temp_config,
)  # (2)!

# Rate responses
print("\nRating default response:")
ratings = client.ratings.rate(response)
print("Response: ", response.response, "\n")
for rating in ratings:
    print(f"Criterion: {rating.criterion.criterion_str}")
    print(f"Rating: {rating.rating}")

print("\nRating low temperature response:")
ratings = client.ratings.rate(low_temp_response)
print("Response: ", low_temp_response.response, "\n")
for rating in ratings:
    print(f"Criterion: {rating.criterion.criterion_str}")
    print(f"Rating: {rating.rating}")

1. Creating an LLM configuration with specific parameters allows you to customize the model's behavior. You can pass any credentials for a LLM provider like Azure OpenAI, OpenAI, Anthropic, etc.

2. When generating responses, you can pass the LLM configuration as an optional parameter to use these custom settings instead of the default configuration.

The following are the key configuration parameters:

  • name: A unique identifier for your configuration
  • llm_model_name: The specific model to use (e.g., "gpt-4", "gpt-3.5-turbo")
  • api_key: Authentication key for the LLM provider
  • llm_base_url: API endpoint URL
  • api_version: API version to use

Usage in the Frontend

In LLM Configurations tag you can view, search and manage your LLM configurations.

LLM Configurations

You can also create a new LLM configuration by clicking the New Configuration button. Here you can select the LLM provider and enter the required credentials.

LLM Configurations

Settings can be adjusted both during creation and afterwards for the configurations.

LLM Configurations