Measurements

vigilant.measurements.learning(data: DataFrame, **kwargs) → DataFrame

Measures the model’s learning with respect to each version.

Parameters:

  • data – performance data with indicated dataset and model versions.

  • **kwargs – optional additional arguments passed to clean().

Returns:

A pandas DataFrame containing the calculated learning for each indicated version.

vigilant.measurements.potential(data: DataFrame, **kwargs) → DataFrame

Measures the model’s potential learning with respect to each version.

Parameters:

  • data – performance data with indicated dataset and model versions.

  • **kwargs – optional additional arguments passed to clean().

Returns:

A pandas DataFrame containing the calculated potential for each indicated version.

vigilant.measurements.retention(data: DataFrame, decay: int | float | None = None, **kwargs) → DataFrame

Measures the model’s knowledge retention with respect to each version.

Parameters:

  • data – performance data with indicated dataset and model versions.

  • decay – exponential decay term for calculating the weighted average.

  • **kwargs – optional additional arguments passed to clean().

Returns:

A pandas DataFrame containing the calculated retention for each indicated version.

Configuration

class vigilant.config.Configuration(**kwargs)

Stores user configuration options; config is the global instance.

dataset_key

Column containing dataset version information.

model_key

Column containing model version information.

performance_key

Column containing model performance.

decay

exponential decay term used in retention().

Utilities

vigilant.utilities.clean(data: DataFrame, dataset_key: str | None = None, model_key: str | None = None, performance_key=None, verbose: bool = True) → DataFrame

Cleans data to only contain versions with both dataset and model information.

Parameters:

  • data – performance data with indicated dataset and model versions.

  • dataset_key – Column containing dataset version information; if None, uses the value in Configuration.

  • model_key – Column containing model version information; if None, uses the value in Configuration.

  • performance_key – Column containing model performance; if None, uses the value in Configuration.

Returns:

The cleaned data.