Store Helper Functions#

expectmine.storage.utils.validate_adapter_init(persistent_path: Path, working_directory: Path)[source]#

Validates that the init arguments are of valid format. Throws an error if not.

Parameters:

persistent_path (Path) – The path which the storage can use to persist data.
working_directory (Path) – The path which the storage can use to create temporary files in for the user or pipeline to access.

Example:

>>> validate_adapter_init(Path("path1"), Path("path2"))

>>> validate_adapter_init(Path("path2"))
ValueError("Storage paths can not be empty.")

Raises:

expectmine.storage.utils.validate_key(key: str)[source]#

Validates that the given key is of valid format. Throws an error if not.

Parameters:: key (str) – The key to associate with the value. A key cannot be empty. Keys have a maximum length of 255 characters.
Example:

>>> validate_key("hello")

>>> validate_key("")
ValueError("Key can not be empty")

Raises:

expectmine.storage.utils.validate_pipeline(key: str, steps: list[BaseStep], io: list[BaseIo], input_files: list[Path])[source]#

Validates that the given key and pipeline are valid and can be stored in the persistent store.

Parameters:

key (str) – The key to associate with the pipeline. A key cannot be empty. Keys have a maximum length of 255 characters.
steps (list[BaseStep]) – List of BaseSteps that make up the pipeline.
io (list[BaseIo]) – List of io objects that where used to configure each individual step.
input_files (list[Path]) – List of input files to the pipeline.

Example:

>>> validate_pipeline("Hello", [Step(...)], [Io(...)], [Path('text.txt')])

>>> validate_pipeline("")
ValueError("Key can not be empty")

Raises:

TypeError – If the arguments have the wrong type.
ValueError – If the key is empty or longer than 255 characters. If Steps and volatile_stores do not match in length or input files are not files but contain directories.

expectmine.storage.utils.validate_pipeline_store_init(persistent_path: Path)[source]#

Validates that the init arguments are of valid format. Throws an error if not.

Parameters:: persistent_path (Path) – The path which the storage can use to persist data.
Example:

>>> validate_pipeline_store_init(Path("path1"))

>>> validate_pipeline_store_init()
TypeError("Persistent path needs to be of type Path")

expectmine.storage.utils.validate_step_name(step_name: str)[source]#

Validates that the given step_name is of valid format. Throws an error if not.

Parameters:: step_name (str) – The key to associate with the value. A key cannot be empty. Keys have a maximum length of 255 characters.
Example:

>>> validate_step_name("hello")

>>> validate_step_name("")
ValueError("step_name can not be empty")

Raises:

expectmine.storage.utils.validate_storage_init(step_name: str, persistent_path: Path, working_directory: Path)[source]#

Validates that the init arguments are of valid format. Throws an error if not.

Parameters:

step_name (str) – Step name to which the storage should be scoped
persistent_path (Path) – The path which the storage can use to persist data.
working_directory (Path) – The path which the storage can use to create temporary files in for the user or pipeline to access.

Example:

>>> validate_storage_init("hello", Path("path1"), Path("path2"))

>>> validate_storage_init("", Path("path1"), Path("path2"))
ValueError("Step name can not be empty.")

Raises:

expectmine.storage.utils.validate_value(value: object)[source]#

Validates that the given object is of valid format. Throws an error if not.

Parameters:: value (object) – The value to store. The maximum size of a value is 25 MiB. The object needs to be Pickled.
Example:

>>> validate_value("hello")

>>> validate_value("")
ValueError("Key can not be empty")

Raises:

TypeError – If the arguments have the wrong type.
ValueError – If the object or file is too big, the wrong size or can’t be pickled.