Personas

Evaluate your chatbot end-to-end by letting reusable, simulated users pursue a goal across real, multi-turn conversations.

A Persona is a reusable definition of a simulated user. In a persona experiment, an LLM simulates each persona while pursuing a goal, the bot or LLM endpoint under test responds turn by turn, and the resulting transcript is rated as one unit by your criterion set. Personas live in a project-wide library, are organized into persona groups, and are run against a set of scenarios so that every persona is evaluated against every scenario in a single experiment.

What It Is

Define personas once in the library and reuse them across experiments:

• The Persona Library holds your reusable, versioned personas.
• A Persona Group is a named pool of personas you select for an experiment.
• A Scenario Set holds the scenarios your personas act out. Each scenario has the user's intention and an expected outcome, plus any fields you add.
• A persona experiment runs every persona in the group against every scenario in a scenario set — this fan-out produces one simulated conversation per persona and scenario.

Scenarios and Personas & Groups appear in the sidebar when the Evaluation switcher is set to Persona (see Navigation).

Quick Start

1. In the Evaluation switcher, choose Persona, open Personas & Groups and create a new persona with the New button.
2. Create a persona group and add the personas you want to evaluate together.
3. Open Scenarios and create a scenario set with New Scenario Set. Add scenarios, each with the User's intention and an Expected outcome, and any fields of your own.
4. Under Criteria Library, click New Criterion Set and pick your scenario set as the collection, so the criteria assistant can propose criteria from its fields.
5. Click New Experiment. With Evaluation set to Persona, the Persona setup opens directly.
6. Choose your scenarios and persona group, then the LLM config for the bot and a criterion set.
7. Run the experiment. Every persona is run against every scenario, and each transcript is rated by the criterion set.

Key Concepts

Concept	Description
Persona	A reusable, versioned simulated user. Has a name, a persona description (the personality and behavior), and optional domain knowledge (background knowledge the simulated user has). The description and domain knowledge may contain {{placeholders}} filled from the scenario's fields.
Persona Library	The project-wide collection of personas. Personas can be shared org-wide (read-only in other projects).
Persona Group	A named, mutable pool of personas. Experiments reference a group; its members are pinned into the experiment at launch.
Scenario	One sample of a persona experiment. Has the User's intention (what the simulated user wants from the conversation) and an Expected outcome (how the conversation should end), plus the values of your own fields.
Scenario Set	A named set of scenarios that an experiment runs on. Every set has the fixed fields User's intention and Expected outcome; you can add your own fields, which can be referenced as {{placeholders}} in persona descriptions and criteria.
Simulator	The LLM that plays the user, driven by the persona. The simulator is a deployment-level setting and is not user-configurable.
Bot / LLM endpoint under test	The LLM that plays your assistant, defined by the experiment's LLM config. Through the SDK it can also get a system-only prompt template.
Termination	The simulator can call an end_conversation tool with one of three reasons — goal_met, goal_failed, or stuck. If it never calls the tool, the loop stops at a hard cap of MAX_TURNS = 20 (reported as Turn limit reached).

The Persona Library

With Evaluation set to Persona, open Personas & Groups in the sidebar. The page is a two-pane workbench: the Persona Library on the left and Persona Groups on the right.

Creating and editing personas

1. Click New at the top of the library to open the inline editor.
2. Fill in:
   • Name — a short label (e.g. Impatient first-time user).
   • Persona description — the personality and behavior the simulator should adopt. The simulated user writes in the language of this description, even when the user's intention or your bot's replies are in another language, unless the description says otherwise.
   • Domain knowledge (optional) — knowledge or context the simulated user has, e.g. a collection of insurance policies.
3. Click Save. Click any persona row later to edit it in place.

Placeholders such as {{customer_code}} are highlighted in each field, and the editor lists every variable the persona uses. Each one must exist as a column in the scenario set you run the persona against — see One Persona, Many Customers. To see which columns are available, pick a scenario set under Use variables from, as in the prompt template editor: its dropdown marks which of the persona's variables the scenario set has, and its fields are listed under Add variable next to each field. Variables that are not fields of that scenario set are highlighted in red; hovering any other variable shows its value in the set's first scenario. This only affects the view; the persona stays independent of any scenario set.

Sharing personas across projects

Personas live in a single project, but you can make a standard persona available everywhere in your organization instead of duplicating it per project.

• Flip the Share with organization toggle on a persona on the Personas & Groups page.
• Once shared, it becomes visible and usable read-only in every other project of the same organization — no need to recreate standard personas per project.
• Only the owning project can edit, un-share, or delete a shared persona. In other projects it shows a Shared badge and its fields are read-only.
• Sharing is lineage-wide: it applies to all versions of that persona, and can be turned off again from the owning project.

When a project needs to diverge from the shared original, clone/duplicate the persona into that project. This creates an independent, editable copy that starts un-shared, so you can adapt it without affecting the shared version.

Persona Groups

A persona group is a named pool of personas that an experiment runs together.

1. In the right pane, use the group switcher to create a New group — give it a name and an optional description.
2. Add personas to the group using the + / ✓ toggle on each library row. Members appear as cards in the group roster.
3. Remove a member with the ✗ on its card.

Groups are mutable and not versioned. When an experiment is created, it captures an immutable snapshot of the exact persona versions in the group at that moment, so later edits to the group or its personas never change a finished experiment.

Scenarios

A persona experiment runs against a scenario set — a set of scenarios, each the intention of one simulated user in a conversation. With Evaluation set to Persona, open Scenarios in the sidebar to manage them. The page lists your scenario sets.

1. Click New Scenario Set and give it a name, an optional description, and optionally your own fields (for example channel or customer_tier). A field name is letters, digits and underscores, not starting with a digit, and at most 100 characters, so it can be used as a {{placeholder}}.
2. Add scenarios with Add Scenario, or import a CSV or Excel file with a goal column, an expected_conversation_outcome column, and one column per field.

Every scenario has two fixed fields that cannot be renamed or removed:

• User's intention — what the simulated user wants from the conversation, in their own words. The simulator pursues it. In files and the API this is the goal field.
• Expected outcome — how the conversation should end. It is not shown to the simulated user; criteria can read it as {{expected_conversation_outcome}}. Both texts are required: the intention drives the conversation, the outcome its evaluation.

New scenarios are written step by step: the user's intention, then the expected outcome, then the set's fields. Test this scenario runs one conversation with a persona of your choice against a model before anything is saved, so you can see how the conversation would go. A test gives no verdict; that is what experiments and criteria are for. Saved scenarios offer the same Test on their card. A test stores nothing; its tokens count toward the project.

Any fields you add can be referenced as {{placeholders}} in a persona's description or domain knowledge, and in criteria. For example, the criterion "Does the conversation end the way the scenario expects: {{expected_conversation_outcome}}?" compares each transcript with the outcome you defined for its scenario.

One Persona, Many Customers

Placeholders let one persona serve many tenants of the same bot. Instead of writing a persona per customer, put the customer-specific values in collection columns and reference them from the persona.

Example: one support bot serves several health insurers. Every conversation must identify the insurer and a test user. Add the columns customer_code and user_id to the scenario set as fields:

goal,customer_code,user_id
"Change my postal address after moving.",DAK,dak-test-001
"Change my postal address after moving.",TK,tk-test-042
"Ask whether a dental cleaning is covered.",DAK,dak-test-001

Then write the persona once:

Field	Value
Name	Recently moved policyholder
Persona description	Insured with {{customer_code}}, just moved house, slightly impatient.
Domain knowledge	Your test user ID is {{user_id}}. You are insured with {{customer_code}}.

Each row renders the persona with its own values, so the three rows above produce three conversations from a single persona. Adding a customer means adding rows, not personas.

Missing columns

Every placeholder in a persona must be a field of the selected scenario set. The New Experiment page lists unmatched placeholders and blocks creation; the API rejects them with HTTP 400.

The Bot's System Prompt (SDK only)

In the UI, the bot under test is its LLM config: its system prompt, if any, comes from that config. Through the SDK, create_experiment additionally accepts a prompt_template whose system message is sent to the bot on every turn. Such a template must be system-only:

• Exactly one system message.
• No user or assistant messages.
• No user-side placeholders — the simulator drives the user turns.

Running a Persona Experiment

1. Set Evaluation to Persona and click New Experiment. There is no type chooser; the Persona setup opens directly.
2. In Choose scenarios, select the scenario set, and in Choose a persona group, select the group to evaluate.
3. Select the LLM config for the bot under test and the criterion set that will rate the transcripts.
4. Review the conversation count and start the experiment.

The experiment fans out across both dimensions: it runs one conversation per persona and per scenario. With P personas and S scenarios, the experiment generates P × S conversations, which the form shows as a live count before you start.

How a Run Works

For each (persona, scenario) pair:

1. Render the persona's description and domain knowledge against the scenario's field values, and read the intention from the scenario's User's intention.
2. The simulator generates the next user message. It sees the full conversation history with roles flipped (your bot's replies appear as user messages to the simulator, and vice versa).
3. The bot under test replies, using the experiment's LLM config plus the system prompt, if one was set through the SDK.
4. Steps 2 and 3 loop until the simulator calls end_conversation or MAX_TURNS is reached.
5. The final transcript is stored as the response and rated by the criterion set.

Stateless vs. stateful providers

Most providers (OpenAI, Custom API, Mock) are stateless: the full conversation history is replayed on every bot turn. Botario and Custom API configs with "stateful": true keep server-side session state: elluminate uses one session ID per conversation and sends only the current user turn on each call. Because only that turn is sent, the bot's system prompt never reaches a stateful bot — it is still kept in the stored transcript.

Reading the Results

• The full dialogue between simulator and bot is stored on each response and visible in the response viewer.
• Expand Persona as simulated in the Sample Navigator to see the persona description and domain knowledge filled with the row's current values. If the row was edited after the run, a warning icon marks that the values may differ from what the simulator received.
• Criterion ratings apply to the entire transcript, not to a single turn.
• The judge knows it is rating a conversation: it is told the persona description and the user's intention, rates the bot's replies across all turns, and reads the simulated user's messages as context only.
• Each response shows how the conversation ended: Goal met, Goal failed, Stuck, or Turn limit reached. In the SDK this is response.termination_reason (goal_met, goal_failed, stuck, max_turns, or None).
• Because each response is tied to the persona that produced it, you can filter results by persona to compare how different personas fared on the same scenarios.

SDK Reference

Scenario sets are managed in the UI or with the SDK. Personas and persona groups are managed in the UI; the SDK references an existing persona group when launching a multi-turn experiment by passing its persona_group_id, which is required for MULTI_TURN experiments.

from elluminate import Client

client = Client()

scenario_set = client.create_scenario_set(
    name="Billing support",
    fields=["channel"],               # declared fields, usable as {{channel}}
)
scenario_set.add_scenarios([
    {
        "goal": "Get a refund for a double charge",
        "expected_conversation_outcome": "The assistant confirms the refund and names the timeline.",
        "field_values": {"channel": "chat"},
    },
])

experiment = client.create_experiment(
    name="Support quality — multi-turn",
    prompt_template=system_only_template,
    scenario_set=scenario_set,        # instead of collection=
    evaluation_mode="MULTI_TURN",     # required with scenario_set
    persona_group_id=group_id,        # required for MULTI_TURN
)
experiment.run()

• client.create_scenario_set(), client.get_scenario_set(name=... | id=...), client.list_scenario_sets() and client.delete_scenario_set() manage scenario sets; the async client has the same methods.
• On a scenario set, add_field(), rename_field(), delete_field(), add_scenarios(), list_scenarios(), get_scenario(), update_scenario() and delete_scenario() change and read its content; the async variants are prefixed with a (aadd_field(), alist_scenarios(), and so on). Each Scenario has goal, expected_conversation_outcome and field_values.
• Pass scenario_set= (a ScenarioSet) instead of collection= to client.create_experiment() and its async equivalent. It requires evaluation_mode="MULTI_TURN" and cannot be combined with collection=. Passing a collection of type PERSONA as collection= still works.

Managing personas via the SDK

The SDK does not create or edit personas or persona groups — author them in the UI first, then reference the group's id. persona_group_id is a parameter of client.create_experiment() and its async equivalent.

Related

• Conversations — static, pre-recorded conversation histories
• Agent Evaluations — tool-using agent evaluations
• Criterion Sets — rule collections used to rate transcripts
• Experiments — running evaluations end-to-end