src.pipeline.verbose_transformation_pipeline

Module containing the VerboseTransformationPipeline class.

Classes

VerboseTransformationPipeline([steps, title])

A verbose transformation pipeline that logs to the terminal.
class src.pipeline.verbose_transformation_pipeline.VerboseTransformationPipeline(steps=<factory>, title='Transformation Pipeline')[source]

A verbose transformation pipeline that logs to the terminal.

get_cache_exists(cache_args=None)[source]

Check if some cache exists.

Parameters:

cache_args (CacheArgs | None) – The cache arguments.

Return type:

bool

Returns:

Whether the cache exists.

run_dossier_pipeline(dossier_id, raw_data_path, processed_data_path, final_data_path)[source]

Run the pipeline on the data.

It also creates a cache after each step in the pipeline.

Parameters:

  • dossier_id (str) – The dossier ID.

  • raw_data_path (Path) – The path to the raw data.

  • processed_data_path (Path) – The path to the processed data.

  • final_data_path (Path) – The path to the final data.

Return type:

DataFrame

Returns:

The processed data.