Async Operations

Elluminate supports asynchronous operations for improved performance when working with multiple prompts or evaluations. This is particularly useful when:

• Batch Processing: Handling multiple prompts or responses simultaneously
• Performance Optimization: Reducing overall execution time for large-scale evaluations
• Resource Efficiency: Better utilization of system resources during I/O operations

The following example demonstrates how to use async operations with collections in Elluminate:

import asyncio

from dotenv import load_dotenv
from elluminate import Client

load_dotenv(override=True)


async def run_eval() -> None:
    client = Client()

    prompt_template, _ = await client.prompt_templates.aget_or_create(
        "Please write a short summary of the life of {{person}} from {{state}}.",
        name="Summary of Person",
    )

    values = [
        {"person": "Paul Revere", "state": "Massachusetts"},
        {"person": "George Washington", "state": "Virginia"},
    ]
    collection, _ = await client.collections.aget_or_create(
        name="Founding Fathers", description="A collection of Founding Fathers"
    )  # (1)!

    await asyncio.gather(
        *[
            client.template_variables.aadd_to_collection(template_variables, collection)
            for template_variables in values
        ]
    )  # (2)!

    await client.criteria.aget_or_generate_many(prompt_template)
    responses = await client.responses.agenerate_many(prompt_template, collection=collection)
    criteria_ratings = await asyncio.gather(*(client.ratings.arate(response) for response in responses))

    for ratings in criteria_ratings:
        for rating in ratings:
            print(f"Criteria: {rating.criterion.criterion_str}")
            print(f"Rating: {rating.rating}\n")


asyncio.run(run_eval())

1. Async methods in Elluminate are prefixed with a. For example, aget_or_create_collection instead of get_or_create_collection.

2. Using asyncio.gather allows for concurrent execution of multiple async operations, significantly improving performance when adding multiple entries.

Most Elluminate SDK operations have async counterparts, making it easy to convert existing synchronous code to async. Simply:

1. Prefix method calls with a (e.g., arate instead of rate)
2. Use await with async method calls
3. Wrap your code in an async function
4. Use asyncio.run() to execute the async function

When working with multiple operations, consider using asyncio.gather() to run them concurrently, as shown in the example above.