Policy-Based Task orchestration

Manages and coordinates data movement tasks based on predefined policies or rules. This allows the management of an unlimited number of tasks, including their execution, but also monitoring and reporting aspects.

Type of Workflows

Workflow Manager allows the creation of different data movement tasks:

• Copy

• Move

• Erase

• Scan

Different type of policies are available:

• Data Migration

• Data Archiving (Active Archive)

• Data Protection (Offline Archive)

Currently, the two workflows Data Archiving and Data Protection are available in the , when the Data Migration is available in the .

Each task includes the definition of the source storage, with its dataset and the destination storage.

Execution Modes

It is possible to define multiple tasks, they are executed in parallel mode in a multi-threaded system.

Each task will execute the movement of a list of files (1 or n) and segment the processing into 3 main parts:

• Preparation phase

• Execution phase

• Finalization phase

For each task execution process, specific jobs are created. The default segmentation is defined to optimize the data processing (example of a default setting: create a job of 1 GB or 100 files maximum).

Each job will be sent into a different queue system. A dispatching engine will coordinate the processing of each job in regard to the defined priority.

Then, different workers are created to do the copy of files and to do the reporting to Finalization process about the status of the copy.

The workflow manager can execute these 3 phases in two main different modes:

• sequential processing

• parallel processing

The Data Mover always takes into account the remaining space of the destination storage. This means that if the available capacity is insufficient to store the file being copied, the operation will either result in an error message such as "stopped by system" (in the case of NAS and file systems), or it will request a new tape (for tape storage). In any case, files are never split across multiple media.

Task Action

Each running task can be paused or stopped. If a task is paused, it can always be restarted. Pressing the stop button will stop the task. Once the stop button has been pressed, it may take a few seconds or minutes before the task is stopped. This is because the task always finishes the current job under processing.

Extended Data Set Selector

An unlimited data set selector brings granularity to the data you have to select. Movement can be executed between any type of storage from storage where hot data are stored to cold storage.

Path Management

For any type of data movement, the Workflow Manager always maintains the original path where the contents were originally stored.

Business applications are configured to access the same folders, even if data have been moved. It is why it is very important that when data are retrieved, they are recopied into their original folder.

Meanwhile, Nodeum offers different options to change this behavior and provides solutions to relocate the contents into another folder structure.

Result of Task execution

Each task finishes with a status, each task status determines the result of the processing that has been executed. This following table describes each type of status.