Quick Start (SDK)

Start evaluating prompts with Elluminate in just a few minutes. For a deeper understanding of the platform, visit our Key Concepts and Guides.

Prerequisites

First, install the Elluminate package and an additional package required by the quickstart code:

pip install elluminate python-dotenv

Next, you'll need to set up your API key. Visit your project's "API Keys" page to create a new API key. For detailed information about API key management and security best practices, see our API Key Management Guide.

Once you have your API key, export it and the service address as environment variables:

export ELLUMINATE_API_KEY=<your_api_key>
export ELLUMINATE_BASE_URL=<your_elluminate_service_address>

Remember to store your API key securely and never commit it to version control. You're now ready to begin evaluating prompts with Elluminate!

Jupyter Notebook

If you're using a Jupyter Notebook, install the required packages and define the environment variables as follows:

!pip install elluminate nest-asyncio python-dotenv

%env ELLUMINATE_API_KEY=<your_api_key>
%env ELLUMINATE_BASE_URL=<your_elluminate_service_address>

Before running the quickstart code, run these lines for Jupyter Notebook compatibility:

import nest_asyncio
nest_asyncio.apply()

Evaluating Your First Prompt

Implement your first evaluation with the following Python code:

from dotenv import load_dotenv
from elluminate import Client
from elluminate.schemas import RatingMode

load_dotenv(override=True)

client = Client()  # (1)!

prompt_template, _ = client.prompt_templates.get_or_create(
    "Explain how {{concept}} works in Scheme, providing a short but illustrative code example.",
    name="Scheme Concepts",
)  # (2)!

client.criteria.get_or_generate_many(prompt_template)  # (3)!


collection, _ = client.collections.get_or_create(
    name="Scheme Concepts",
)  # (4)!

template_variables = client.template_variables.add_to_collection(
    template_variables={"concept": "recursion"},
    collection=collection,
)  # (5)!

response = client.responses.generate(
    prompt_template,
    template_variables=template_variables,
)  # (6)!

# Generate ratings for a given `response`
ratings = client.ratings.rate(response, rating_mode=RatingMode.FAST)  # (7)!
for rating in ratings:
    print(f"Criteria: {rating.criterion.criterion_str}")
    print(f"Rating: {rating.rating}\n")

1. Initializes the Elluminate client using your configured environment variables from the setup phase.

2. Creates a prompt template using mustache syntax, incorporating template variables (like concept in this example). If the template already exists, it just gets returned.

3. Generates evaluation criteria automatically for your prompt template or gets the existing criteria.

4. Creates a template variables collection. This will be used to collect the template variables for a prompt template.

5. Adds a template variable to the collection. This will be used to fill in the template variable (replacing concept with recursion).

6. Creates a response by using your prompt template and filling in the template variable.

7. Evaluates the response against the generated criteria, returning detailed ratings for each criterion.

View the Ratings

Access your ratings through the web interface by logging in. Upon viewing your ratings dashboard, you'll find a comprehensive overview that includes key statistics such as:

• Overall score
• Word count for both prompt templates and responses
• Number of evaluation criteria used

Each rating entry can be expanded to reveal detailed information, including specific evaluation criteria and whether each criterion was successfully met. This detailed view helps you understand exactly how your responses were assessed.

Next Steps

• Explore the Key Concepts of Elluminate.