Prompt Templates¶

Master the foundation of systematic AI evaluation with versioned, reusable templates that enable comprehensive testing across different inputs and scenarios

Prompt Templates are the foundation of elluminate's evaluation system - versioned, reusable templates with variable placeholders that enable systematic testing of AI systems across different inputs and scenarios.

What Are Prompt Templates?¶

A Prompt Template is a reusable template containing variables ({{placeholders}}) that can be replaced with specific values to create concrete prompts. The values will be pulled from the Collection (TemplateVariable) to generate the prompt for the LLM.

Template Structure¶

Template Details¶

The first part of the Prompt Template are the Template Details

The name of your Prompt Template will help you select it later for the experiments.

The simplest version of a Prompt Template contains merely an User Message with a {{placeholder}} in double brackets.

A system message will allow you to include instructions for the model to behave in a certain way. You may also use {{placehoders}} in the system message.

For details on Structured Output and Tools, see the Advanced Features section below.

Response Generator¶

In the Response Generator you can test the output of the chosen model to see if your Prompt Template would work:

Criteria¶

Finally, you need to add some criteria to your template. You can link an existing Criterion Set or manually add each criterion. Criteria need a label and a criterion description

The button Generate Criteria automatically creates several criteria that match your existing Prompt Template:

Version Management¶

By editing and saving a Prompt Template, you will create a new version that will help you track evolution and changes in your Evaluations

Advanced Features¶

Structured Outputs¶

With Structured Outputs you can ask the prompt to generate a response in a json format. It is possible to define the desired format directly or to let the system generate it for you:

Structured outputs are specially important for evaluating agentic applications.

Tool Calling¶

You can access tools for testing the output of your agentic applications. Refer to our examples in the SDK Guide for more details.

The tools can be defined in the Tool Calling area. You may define your own tools or use one of the examples provided.

Tool Choice can also be defined.

SDK approach¶

Basic Template Creation¶

from elluminate import Client

client = Client()

# Create a simple template
template = client.prompt_templates.create(
    name="Customer Support Response",
    messages=[
        {
            "role": "system",
            "content": "You are a {{company_type}} customer support representative."
        },
        {
            "role": "user", 
            "content": "Customer inquiry: {{customer_message}}"
        }
    ]
)

Messages Format¶

Prompt templates use a messages array structure, supporting different message roles:

[
  {
    "role": "system",
    "content": "You are a helpful assistant specialized in {{domain}}."
  },
  {
    "role": "user", 
    "content": "Please help me with: {{user_request}}"
  }
]

Supported Roles¶

system: Sets behavior and context for the AI
user: Represents user input and requests
assistant: Represents AI responses (for multi-turn conversations)

Best Practices¶

When designing prompt templates, use clear and descriptive placeholder names that make the template's intent obvious (for example, {{customer_inquiry}} instead of {{input}}). Consistency in naming and format helps maintain clarity, especially when templates evolve or are reused.

Structure your prompts thoughtfully: begin with system messages to set the context, and write user messages that are specific and actionable. When planning variables, aim for realistic and diverse values that reflect the range of scenarios you want to test. Consider both typical and edge cases to ensure your template is robust and effective.

Troubleshooting¶

Common Issues¶

Missing Variables: Ensure Collections contain all template placeholders, also the ones in the criteria.

Version Conflicts: Use consistent template versions across related experiments

Placeholder Syntax: Verify correct {{variable_name}} format

Validation¶

Template Testing: Generate sample prompts before running full experiments

Variable Coverage: Verify all placeholders have corresponding Collection values

Content Review: Validate generated prompts match intended test scenarios