orion.evaluate_pipeline

orion.evaluate_pipeline(data, truth, pipeline, hyperparameters=None, metrics=None, train_data=None)

Evaluate the performance of a pipeline.

The pipeline is evaluated by executing it on a signal for which anomalies are known and then applying one or more metrics to it to compute scores.

The pipeline can be passed as:

• An str with a path to a JSON file.

• An str with the path to a pickle file.

• An str with the name of a registered Orion pipeline.

• An MLPipeline instance.

• A dict with an MLPipeline specification.

If the pipeline is not fitted, it is possible to pass separated learning data can be passed to fit the pipeline to it before using it to detect anomalies.

Parameters

• data (str or DataFrame) – Data to analyze searching for anomalies. It can be passed as a path to a CSV file or as a DataFrame.

• truth (str or DataFrame) – Table of known anomalies to use as the ground truth for scoring. It can be passed as a path to a CSV file or as a DataFrame.

• pipeline (str or Pipeline or dict) –

  Pipeline to use. It can be passed as:

  • An str with a path to a JSON file.

  • An str with the name of a registered pipeline.

  • An str with the path to a pickle file.

  • An MLPipeline instance.

  • A dict with an MLPipeline specification.

• hyperparameters (str or dict) – Hyperparameters to set to the pipeline. It can be passed as a hyperparameters dict in the mlblocks format or as a path to the corresponding JSON file. Defaults to None.

• metrics (list[str]) – List of metrics to use. If not passed, all the Orion metrics are applied.

• train_data (str or DataFrame) – Data to which the pipeline should be fitted. It can be passed as a path to a CSV file or as a DataFrame. If not given, the pipeline is used without fitting it first.

orion.detect_anomalies orion.benchmark.benchmark