Skip to content

Quick Start

Start evaluating prompts with Elluminate in just a few minutes. For a deeper understanding of the platform, visit our Key Concepts and Guides.

Prerequisites

First, install the Elluminate package:

pip install elluminate

Next, you'll need to set up your API key. Visit your project's "Keys" dashboard to create a new API key. For detailed information about API key management and security best practices, see our API Key Management Guide.

Once you have your API key, export it and the service address as environment variables:

export ELLUMINATE_API_KEY=<your_api_key>
export ELLUMINATE_BASE_URL=<your_elluminate_service_address>

Remember to store your API key securely and never commit it to version control. You're now ready to begin evaluating prompts with Elluminate!

Evaluating Your First Prompt

Create a new Python file and implement your first evaluation with the following code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
from dotenv import load_dotenv
from elluminate import Client

load_dotenv(override=True)

client = Client()  # (1)!

prompt_template, _ = client.prompt_templates.get_or_create(
    "Explain how {{concept}} works in Scheme, providing a short but illustrative code example.",
    name="Scheme Concepts",
)  # (2)!

client.criteria.get_or_generate_many(prompt_template)  # (3)!


collection, _ = client.collections.get_or_create(
    name="Scheme Concepts",
)  # (4)!

template_variables = client.template_variables.add_to_collection(
    template_variables={"concept": "recursion"},
    collection=collection,
)  # (5)!

response = client.responses.generate(
    prompt_template,
    template_variables=template_variables,
)  # (6)!

# Generate ratings for a given `response`
ratings = client.ratings.rate(response)  # (7)!
for rating in ratings:
    print(f"Criteria: {rating.criterion.criterion_str}")
    print(f"Rating: {rating.rating}\n")

  1. Initializes the Elluminate client using your configured environment variables from the setup phase.

  2. Creates a prompt template using mustache syntax, incorporating template variables (like concept in this example). If the template already exists, it just gets returned.

  3. Generates evaluation criteria automatically for your prompt template or gets the existing criteria.

  4. Creates a template variables collection. This will be used to collect the template variables for a prompt template.

  5. Adds a template variable to the collection. This will be used to fill in the template variable (replacing concept with recursion).

  6. Creates a response by using your prompt template and filling in the template variable.

  7. Evaluates the response against the generated criteria, returning detailed ratings for each criterion.

View the Ratings

Access your ratings through the web interface by logging in. Upon viewing your ratings dashboard, you'll find a comprehensive overview that includes key statistics such as:

  • Overall score
  • Word count for both prompt templates and responses
  • Number of evaluation criteria used

Each rating entry can be expanded to reveal detailed information, including specific evaluation criteria and whether each criterion was successfully met. This detailed view helps you understand exactly how your responses were assessed.

Ratings Overview

Next Steps