Task Management

Task management capabilities allow users to control and monitor data movement workflows.

Unlimited number of tasks can be defined simultaneously, this to move, archive and protect data between different storage tiers.

In addition, a centralized dashboard allows users to monitor task progress and get real-time updates on the status of their data.

A list of (non exhaustive) options is available for each defined task:

• Task Name

• Type of workflow (Offline Archive, Active Archive)

• Action (Copy, Move)

• Source (Pool of Storage)

Allow the selection of the overall storage or get the option to browse the storage and include specific contents.

• Destination (Pool of Storage, Specify Specific Folder destination and conflict resolution)

• Filtering with inclusion / exclusion of files

• Incremental or full type selection depending on the content of the storage pool selected at destination

• Priority setting

• Schedule (Manually, Periodically, Automatically)

Task Creation

Behind the task creation mechanism, different type of workflows can defined.

Data Protection / Offline Archive

Definition

Data Protection or Offline Archiving task secures selected data on a dedicated storage tier. Data accessibility is managed by the administrator behind the workflow. It allows only the data to be read or retrieved online.

Depending on the medium chosen, data can be physically moved off-site. This requires human intervention to retrieve the data online.

Users have the option of running an offline workflow that will copy or move the data to a secondary storage pool.

Why to use this workflow?

1. Protect stored content that is online (online means "modifiable" by users or applications). Workflow allows these data to be copied according to defined criteria.

2. Archive data on a disconnected media, with the possibility of get it online - offsite.

Traceability

A complete traceability is kept.

User/administrator have an overview of their data mobility even if the data has been secure offline.

In case of tape library, data can even be move out of the library (off site). All contents are stored in open format, under common standards.

Available Criteria

Source	Destination	Type of action Mover
Primary Storage : Nas Object Storage	Secondary Storage : Tape Pools Cloud Pools Nas Pools	Copy / Move
Container	Secondary Storage : Tape Pools Cloud Pools Nas Pools	Copy / Move
Restore Source	Destination	Type of action Mover
Tape Pools	Container (cache) Primary Storage (NAS or Object Storage)	Copy / Move
Cloud Pools	Container (cache) Primary Storage (NAS or Object Storage) Cloud Pool	Copy / Move
Nas Pools	Container (cache) Primary Storage (NAS or Object Storage) Nas Pool	Copy / Move
Task	Container (cache) Primary Storage (NAS or Object Storage)	Copy / Move
From Reference File	Container (cache) Primary Storage (NAS or Object Storage)	Copy / Move

Data Archiving / Active Archive

Before to use this workflow, make sure to configure the container and cache. Learn more to Container configuration section

Definition

This makes it possible to archive a collection of data that can be moved from an expensive primary storage tier to a secondary storage tier. The data may or may not be removed from primary storage and must remain accessible from time to time. Data archiving allows data to be stored that does not require intensive reading and writing, but can be modified or read from time to time.

Why to use this workflow?

Data is stored on less expensive storage media, such as object storage, tape storage or even low-cost disks. An active archive includes functionality to seamlessly move data between different storage tiers (disk - tape - cloud) so that applications or users can access it directly.

Scan an LTFS Media

This workflow makes it possible to re-import tapes that have already been written.

To import LTFS Tapes into Nodeum, follow these steps :

Add the LTFS TAPE 000001L6 in the tape library connected to Nodeum

Define a workflow to scan the LTFS TAPE and then execute it

• Select Scan an LTFS Media

• Select as the source the tape need to scan "Tape “000001L6"

• Launch the task

Get the result after the task execution

• Data are indexed and recognized by Nodeum without reading the complete tape

• Metadata are injected in the catalog

• End user can see and retrieve easily the contents (files, images, video)

• End user accesses contents straight away; no need to make new set of tapes

Task Monitoring

To aid in monitoring task execution, the Console offers a comprehensive monitoring feature. This feature provides a global overview of all task activities.

Three different sections are available :

Upcoming task

This section shows the task(s) scheduled. For each task, you can see detailed information including the date - time when the task will be executed.

It is possible to skip an execution in clicking in the right cross icon.

If a task hasn't been executed properly, then you will an information saying that "task execution missed for one or multiple occurence".

Running task

This section provides the list of the task(s) which are currently running.

For each of these tasks, you can

1. See the detail and the status.

2. Pause the task execution.

3. Resume a paused task .

4. Stop the task execution

History

In this section you can find all the tasks that have been completed.

List of tasks history

Task Report

For each task running or finished, you can open to see the global view about the task.

To see the full informations click on the arrow beside the task:

List of Nodeum's tasks

And have a look on the task information:

Screenshot about a task execution

Definitions of task informations :

Informations	Definition
Timing	Task starts the x date and at x hours Elapsed time for the task Estimated remaining time to process the task
Type	Offline Archive Active Archive Data Exchange	Move or Copy to a destination: Network Shared Folder named Container Nas Tape
Processed size	size of data processed
Processed items	Number of items of data processed	To see the good process click on the tab processed items*
Logs	Details of task's log

Each task report contains all files processed during the task, a downloaded copy can be exported.

Task Log

The following table exposes the list of messages available in the logs, the messages are mostly sorted in the displayed' sequence.

Message	Alternative value	Definition
Task is starting	n/a	The task starts.
Parallel Processing mode is Enabled	Parallel Processing mode is Disabled	Enabled means that all processes are running in // at the opposite of Disabled where the different processes are executed in serial. The Disabled mode is used for troubleshooting a task.
Initialization Process is starting	n/a	The process in charged of the inventory is started.
License is valid	License is not valid ('description of the error'), task cannot be executed	It checks the validity of the Nodeum licence is valid.
Task Job Limits : 100 files/job - 2 parallel job(s) - 200 parallel DBFile/job - 1000000000 B/job	# parallel job(s) # parallel DBFile/job # B/job	It defines the number of files to be processed per batch. This example means that the task will included 2 jobs; each of them will contains maximum 200 files or a size of maximum 1GB. When one of these values is reached, then another batch will be defined.
Incremental Mode is Configured. The process will select only new content	Full Mode is Configured.	The process will select only new content or will select all files to be processed.
Processing Order is Configured. (Ascending)	Processing Order is Configured. (Descending)	This manages the order of files processing. Best performance is Ascending.
Check integrity of data transfers: MD5	Check integrity of data transfers: CRC32 Check integrity of data transfers: XXHAS64	The type of integrity check is displayed.
Pool pool name: Tape tape barcode: VALID	n/a	This verifies if the associated pool contains a valid tape.
Pool pool name:1 valid tape(s)	n/a	This summarizes the number of valid tapes in the associated pool.
Main Process is starting	n/a	The copy process starts.
Finalization Process is starting	n/a	The process responsible of managing the catalog changes starts.
Tape tape barcode: is Mounted on the drive drive s/n	n/a	Indicates a request to mount a tape in a tape drive.
Tape tape barcodeloaded on the drive drive s/n is Dismounted	n/a	Indicates a request to dismount a tape in a tape drive.
LTFS File System of the Tape tape barcodeloaded on the drive drive s/n is Mounted	n/a	The LTFS partition is mounted for the specified tape in the specified tape drive.
LTFS File System of the Tape tape barcodeloaded on the drive drive s/nis Umounted	n/a	The LTFS partition is unmounted for the specified tape in the specified tape drive.
Unable to Mount LTFS File System of the Tape tape barcodeloaded on the drive drive s/n	n/a	The tape cannot mount its LTFS file system properly.
Unable to Copy a File. (Read-Only LTFS FileSystem, Try to mount it again)	n/a	Indicates that the tape is mounted with Read-Only LTFS file system and can't copy the file. The task will remount another tape. It is highly recommended to check the tape manually.
Total of items/size parsed: xxxx(yyyTB)	n/a	This list the result of the Initialization process and list the number of files and size to be processed by the task.
Total of items/size already up to date on the destination: xxxx(yyyTB)	n/a	This is the result of the verification of the number of files and the size already available at the destination.
Total of items/size to process on this schedule: xxxx(yyyTB)	n/a	This is the summary of what will be processed during this task execution.
Initialization Process is finished	n/a	This confirms that the Initialization process is finished and then files inventory is available.
All Files were processed on this Rule	n/a	This confirms that all files have been processed.
Main Process is finished	n/a	The Main process is finalized.
Retry made on volume tape barcode: xx	n/a	Shows the number of mount retries for all the defined tape barcode.
Finalization Process is finished	n/a	The Finalization process is finalized.
Task is finished with Success	n/a	The Task is finalized.