streamsight.evaluators.MetricAccumulator

class streamsight.evaluators.MetricAccumulator

Bases: object

__init__()

Methods

__init__()
add(metric, algorithm_name)	Add a metric to the accumulator
df_macro_level_metric()	Macro metric across all timestamps
df_metric([filter_timestamp, filter_algo, level])	Dataframe representation of the metric
df_micro_level_metric()	Micro metric across all timestamps
df_user_level_metric()	User metric across all timestamps
df_window_level_metric()

Attributes

metrics
user_level_metrics
window_level_metrics

add(metric: Metric, algorithm_name: str) → None

Add a metric to the accumulator

Takes a Metric object and adds it under the algorithm name. If the specified metric already exists for the algorithm, it will be overwritten with the new metric.

Parameters:

• metric (Metric) – Metric to store

• algorithm_name (str) – Name of the algorithm

df_macro_level_metric() → DataFrame

Macro metric across all timestamps

Returns:

_description_

Return type:

pd.DataFrame

df_metric(filter_timestamp: int | None = None, filter_algo: str | None = None, level: MetricLevelEnum = MetricLevelEnum.MACRO) → DataFrame

Dataframe representation of the metric

Returns a dataframe representation of the metric. The dataframe can be filtered based on the algorithm name and the timestamp.

Parameters:

• filter_timestamp (Optional[int], optional) – Timestamp value to filter on, defaults to None

• filter_algo (Optional[str], optional) – Algorithm name to filter on, defaults to None

• level (MetricLevelEnum, optional) – Level of the metric to compute, defaults to MetricLevelEnum.MACRO

Returns:

Dataframe representation of the metric

Return type:

pd.DataFrame

df_micro_level_metric() → DataFrame

Micro metric across all timestamps

Returns:

_description_

Return type:

pd.DataFrame

df_user_level_metric() → DataFrame

User metric across all timestamps

Computation of metrics evaluated on the user level

Returns:

_description_

Return type:

pd.DataFrame

df_window_level_metric() → DataFrame

metrics

Deprecated since version Use: window_level_metrics instead

property user_level_metrics

property window_level_metrics: defaultdict