base
logger = logging.getLogger(__name__) module-attribute ¶
Builder dataclass ¶
Bases: ABC
Base class for Builder objects.
Provides methods to set specific values for the builder and enforce checks such that the builder can be constructed correctly and to avoid possible errors when the builder is executed.
Source code in src/recnexteval/evaluators/builder/base.py
18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 | |
setting = field(init=False) class-attribute instance-attribute ¶
Setting to evaluate the algorithms on
metric_k = 10 class-attribute instance-attribute ¶
K value for metrics
ignore_unknown_user = True class-attribute instance-attribute ¶
Ignore unknown user in the evaluation
ignore_unknown_item = True class-attribute instance-attribute ¶
Ignore unknown item in the evaluation
seed = 42 class-attribute instance-attribute ¶
Random seed for reproducibility
metric_entries = field(default_factory=dict) class-attribute instance-attribute ¶
dict of metrics to evaluate algorithm on. Using dict instead of list for fast lookup
set_metric_k(K) ¶
Set K value for all metrics.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
K | int | K value to set for all metrics. | required |
Source code in src/recnexteval/evaluators/builder/base.py
49 50 51 52 53 54 55 | |
add_metric(metric) ¶
Add metric to evaluate algorithm on.
Metric will be added to the metric_entries dict where it will later be converted to a list when the evaluator is constructed.
Note
If K is not yet specified, the setting's top_K value will be used. This requires the setting to be set before adding the metric.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
metric | str | type | Metric to evaluate algorithm on. | required |
Raises:
| Type | Description |
|---|---|
ValueError | If metric is not found in METRIC_REGISTRY. |
RuntimeError | If setting is not set. |
Source code in src/recnexteval/evaluators/builder/base.py
57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 | |
add_setting(setting) ¶
Add setting to the evaluator builder.
Note
The setting should be set before adding metrics or algorithms to the evaluator.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
setting | Setting | Setting to evaluate the algorithms on. | required |
Raises:
| Type | Description |
|---|---|
ValueError | If setting is not of instance Setting. |
Source code in src/recnexteval/evaluators/builder/base.py
100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 | |
clear_metrics() ¶
Clear all metrics from the builder.
Source code in src/recnexteval/evaluators/builder/base.py
120 121 122 | |
build() abstractmethod ¶
Build object.
Raises:
| Type | Description |
|---|---|
NotImplementedError | If the method is not implemented. |
Source code in src/recnexteval/evaluators/builder/base.py
150 151 152 153 154 155 156 157 | |