Project

class mixedvoices.core.project.Project(project_id: str, metrics: List[Metric] | None = None, success_criteria: str | None = None, evals: Dict[str, Evaluator] | None = None, _metrics: Dict[str, Metric] | None = None)[source]

Bases: object

add_metrics(metrics: List[Metric]) → None[source]: Add new metrics to the project.

create_evaluator(test_cases: List[str], metric_names: List[str] | None = None) → Evaluator[source]

Create a new evaluator for the project

Parameters:

test_cases (List[str]) – List of test cases to evaluate the agent on.
metrics (Optional[List[str]]) – List of metric names to be evaluated, or None to use all project metrics.

Returns:

The newly created evaluator

Return type:

Evaluator

create_version(version_id: str, prompt: str, metadata: Dict[str, Any] | None = None)[source]

Create a new version in the project

Parameters:

version_id (str) – Name of the version
prompt (str) – Prompt used by the voice agent
metadata (Optional[Dict[str, Any]]) – Metadata to be associated with the version. Defaults to None.

get_metric(metric_name: str) → Metric[source]

Get a metric by name.

Parameters:: metric_name (str) – The name of the metric to get
Returns:: The metric
Return type:: Metric

get_metrics_by_names(metric_names: List[str]) → List[Metric][source]

Get multiple metrics by their names.

Parameters:: metric_names (List[str]) – The names of the metrics to get
Returns:: The metrics
Return type:: List[Metric]

property id: str: Get the name of the project

list_evaluators() → List[Evaluator][source]: Get all evaluators in the project

list_metric_names() → List[str][source]: Get all metric names.

load_evaluator(eval_id: str) → Evaluator[source]

Load an evaluator from the project

Parameters:: eval_id (str) – ID of the evaluator to load
Returns:: The loaded evaluator
Return type:: Evaluator

load_version(version_id: str) → Version[source]

Load a version from the project

Parameters:: version_id (str) – ID of the version to load

property metrics: List[Metric]: Get all metrics in the project

remove_metric(metric_name: str) → None[source]

Remove a metric by name.

Parameters:: metric_name (str) – The name of the metric to remove

property success_criteria: Get the success criteria of the project

update_metric(metric: Metric) → None[source]

Update an existing metric.

Parameters:: metric (Metric) – The metric to update

update_success_criteria(success_criteria: str | None) → None[source]

Update the success criteria of the project

Parameters:: success_criteria (Optional[str]) – The new success criteria. If it is None, the success criteria will be removed

property version_ids: Get all version names in the project

mixedvoices.core.project.create_project(project_id: str, metrics: List[Metric], success_criteria: str | None = None)[source]

Create a new project

Parameters:

project_id (str) – Name of the project
metrics (List[Metric]) – List of metrics to be added to the project
success_criteria (Optional[str]) – Success criteria for the version. Used to automatically determine if a recording is successful or not. Defaults to None.

mixedvoices.core.project.load_project(project_id: str)[source]: Load an existing project