RatingsResource

RatingsResource ¶

RatingsResource(client: Union[Client, AsyncClient])

Bases: BaseResource

list ¶

list(prompt_response: PromptResponse) -> List[Rating]

Gets the ratings for a prompt response.

Parameters:

Name	Type	Description	Default
`prompt_response`	`PromptResponse`	The prompt response to get ratings for.	required

Returns:

Type	Description
`List[Rating]`	list[Rating]: List of rating objects for the prompt response.

Raises:

Type	Description
`NotFoundError`	If the prompt response doesn't exist or belongs to a different project.

rate ¶

rate(prompt_response: PromptResponse, rating_mode: RatingMode = DETAILED) -> List[Rating]

Rates a response against its prompt template's criteria using an LLM.

This method evaluates a prompt response against all applicable criteria associated with its prompt template. If template variables were used for the response, it will consider both general criteria and criteria specific to those variables.

Parameters:

Name	Type	Description	Default
`prompt_response`	`PromptResponse`	The response to rate.	required
`rating_mode`	`RatingMode`	Judge reasoning effort: - FAST: Quicker evaluation, less judge reasoning effort - DETAILED: More judge reasoning effort Both return explanations. Some rating versions pin their effort and ignore this.	`DETAILED`

Returns:

Type	Description
`List[Rating]`	list[Rating]: List of rating objects, one per criterion.

Raises:

Type	Description
`APIError`	If no criteria exist for the prompt template

rate_many ¶

rate_many(prompt_responses: List[PromptResponse], rating_mode: RatingMode = DETAILED, timeout: float | None = None, polling_interval: float = 3.0) -> List[List[Rating]]

Batch version of rate.

Parameters:

Name	Type	Description	Default
`prompt_responses`	`list[PromptResponse]`	List of prompt responses to rate.	required
`rating_mode`	`RatingMode`	Judge reasoning effort (FAST or DETAILED). Both return reasoning; some rating versions pin their effort and ignore this.	`DETAILED`
`timeout`	`float`	Timeout in seconds for API requests. Defaults to no timeout.	`None`
`polling_interval`	`float`	Time between status checks in seconds. Defaults to 3.0.	`3.0`

Returns:

Type	Description
`List[List[Rating]]`	List[List[Rating]]: List of lists of rating objects, one per criterion for each prompt response.