CriterionSetsResource

CriterionSetsResource ¶

CriterionSetsResource(client: Client)

Bases: BaseResource

add_prompt_template ¶

add_prompt_template(criterion_set: CriterionSet, prompt_template: PromptTemplate) -> CriterionSet

Add a prompt template to an existing criterion set.

Parameters:

Name	Type	Description	Default
`criterion_set`	`CriterionSet`	The criterion set.	required
`prompt_template`	`PromptTemplate`	The prompt template to add.	required

Returns:

Name	Type	Description
`CriterionSet`	`CriterionSet`	The updated criterion set.

aremove_prompt_template `async` ¶

aremove_prompt_template(criterion_set: CriterionSet, prompt_template: PromptTemplate) -> None

Async version of remove_prompt_template.

create ¶

create(name: str, prompt_template: PromptTemplate | None = None, criteria: List[CriterionIn] | None = None) -> CriterionSet

Create a new criterion set and optionally associate it with a prompt template.

Parameters:

Name	Type	Description	Default
`name`	`str`	The name of the criterion set.	required
`prompt_template`	`PromptTemplate`	The prompt template to associate with the criterion set. If None, creates a criterion set not linked to any template.	`None`
`criteria`	`list[CriterionIn]`	List of criterion objects.	`None`

Returns:

Name	Type	Description
`CriterionSet`	`CriterionSet`	The created criterion set.

delete ¶

delete(criterion_set: CriterionSet) -> None

Delete a criterion set.

This will also delete all associated criteria.

Parameters:

Name	Type	Description	Default
`criterion_set`	`CriterionSet`	The criterion set to delete.	required

get ¶

get(name: str, prompt_template: PromptTemplate) -> CriterionSet

Get a specific criterion set by name and prompt template.

Parameters:

Name	Type	Description	Default
`name`	`str`	The name of the criterion set.	required
`prompt_template`	`PromptTemplate`	The prompt template associated with the criterion set.	required

Returns:

Name	Type	Description
`CriterionSet`	`CriterionSet`	The requested criterion set.

Raises:

Type	Description
`ValueError`	If no criterion set is found with the specified name and prompt template.

get_or_create ¶

get_or_create(name: str, prompt_template: PromptTemplate, criteria: List[CriterionIn] | None = None) -> Tuple[CriterionSet, bool]

Get or create a criterion set for a prompt template.

Attempts to create a criterion set. If it already exists, retrieves the existing one. Relies on 409 conflict response to determine if the criterion set already exists.

Parameters:

Name	Type	Description	Default
`name`	`str`	The name of the criterion set.	required
`prompt_template`	`PromptTemplate`	The prompt template to associate with the criterion set.	required
`criteria`	`list[CriterionIn]`	List of criterion objects. if creation is needed.	`None`

Returns:

Type	Description
`Tuple[CriterionSet, bool]`	Tuple[CriterionSet, bool]: A tuple containing: - The criterion set - Boolean indicating if a new criterion set was created (True) or existing one returned (False)

list ¶

list() -> List[CriterionSet]

List all criterion sets in the project.

Returns:

Type	Description
`List[CriterionSet]`	list[CriterionSet]: List of criterion set objects.

remove_prompt_template ¶

remove_prompt_template(criterion_set: CriterionSet, prompt_template: PromptTemplate) -> None

Remove a prompt template from a criterion set.

Parameters:

Name	Type	Description	Default
`criterion_set`	`CriterionSet`	The criterion set.	required
`prompt_template`	`PromptTemplate`	The prompt template to remove.	required

CriterionSetsResource