RatingsResource

RatingsResource ¶

RatingsResource(client: Client)

Bases: BaseResource

delete ¶

delete(rating: Rating) -> None

Deletes a rating.

Parameters:

Name	Type	Description	Default
`rating`	`Rating`	The rating to delete.	required

delete_all ¶

delete_all(prompt_response: PromptResponse) -> None

Deletes all ratings for a prompt response.

Parameters:

Name	Type	Description	Default
`prompt_response`	`PromptResponse`	The prompt response to delete ratings for.	required

Raises:

Type	Description
`HTTPStatusError`	If the prompt response doesn't exist or belongs to a different project.

list ¶

list(prompt_response: PromptResponse) -> List[Rating]

Gets the ratings for a prompt response.

Parameters:

Name	Type	Description	Default
`prompt_response`	`PromptResponse`	The prompt response to get ratings for.	required

Returns:

Type	Description
`List[Rating]`	list[Rating]: List of rating objects for the prompt response.

Raises:

Type	Description
`HTTPStatusError`	If the prompt response doesn't exist or belongs to a different project.

rate ¶

rate(prompt_response: PromptResponse, experiment: Experiment | None = None, rating_mode: RatingMode = DETAILED) -> List[Rating]

Rates a response against its prompt template's criteria using an LLM.

This method evaluates a prompt response against all applicable criteria associated with its prompt template. If template variables were used for the response, it will consider both general criteria and criteria specific to those variables.

Parameters:

Name	Type	Description	Default
`prompt_response`	`PromptResponse`	The response to rate.	required
`experiment`	`Experiment \| None`	Optional experiment to associate ratings with. If provided, the method will verify that the response matches the experiment's prompt template, collection, and LLM configuration before rating.	`None`
`rating_mode`	`RatingMode`	Mode for rating generation: - FAST: Quick evaluation without detailed reasoning - DETAILED: Includes explanations for each rating	`DETAILED`

Returns:

Type	Description
`List[Rating]`	list[Rating]: List of rating objects, one per criterion.

Raises:

Type	Description
`HTTPStatusError`	If no criteria exist for the prompt template

rate_many ¶

rate_many(prompt_responses: List[PromptResponse], experiment: Experiment | None = None, rating_mode: RatingMode = DETAILED, timeout: float | None = None) -> List[List[Rating]]

Batch version of rate.

Parameters:

Name	Type	Description	Default
`prompt_responses`	`list[PromptResponse]`	List of prompt responses to rate.	required
`experiment`	`Experiment \| None`	Optional experiment to associate ratings with.	`None`
`rating_mode`	`RatingMode`	Mode for rating generation (FAST or DETAILED). If DETAILED a reasoning is added to the rating.	`DETAILED`
`timeout`	`float`	Timeout in seconds for API requests. Defaults to no timeout.	`None`

Returns:

Type	Description
`List[List[Rating]]`	List[List[Rating]]: List of lists of rating objects, one per criterion for each prompt response.