Task Management
Task management capabilities allow users to control and monitor data movement workflows.
Unlimited number of tasks can be defined simultaneously, this to move, archive and protect data between different storage tiers.
In addition, a centralized dashboard allows users to monitor task progress and get real-time updates on the status of their data.
A list of (non exhaustive) options is available for each defined task:
Task Name
Type of workflow (Offline Archive, Active Archive)
Action (Copy, Move)
Source (Pool of Storage)
Allow the selection of the overall storage or get the option to browse the storage and include specific contents.
Destination (Pool of Storage, Specify Specific Folder destination and conflict resolution)
Filtering with inclusion / exclusion of files
Incremental or full type selection depending on the content of the storage pool selected at destination
Priority setting
Schedule (Manually, Periodically, Automatically)
Task Creation
Behind the task creation mechanism, different type of workflows can defined.
Data Protection / Offline Archive
Definition
Data Protection or Offline Archiving task secures selected data on a dedicated storage tier. Data accessibility is managed by the administrator behind the workflow. It allows only the data to be read or retrieved online.
Depending on the medium chosen, data can be physically moved off-site. This requires human intervention to retrieve the data online.
Users have the option of running an offline workflow that will copy or move the data to a secondary storage pool.
Why to use this workflow?
Protect stored content that is online (online means "modifiable" by users or applications). Workflow allows these data to be copied according to defined criteria.
Archive data on a disconnected media, with the possibility of get it online - offsite.
Traceability
A complete traceability is kept.
User/administrator have an overview of their data mobility even if the data has been secure offline.
In case of tape library, data can even be move out of the library (off site). All contents are stored in open format, under common standards.
Available Criteria
Primary Storage :
Nas
Object Storage
Secondary Storage :
Tape Pools
Cloud Pools
Nas Pools
Copy / Move
Container
Secondary Storage :
Tape Pools
Cloud Pools
Nas Pools
Copy / Move
Restore Source
Destination
Type of action Mover
Tape Pools
Container (cache)
Primary Storage (NAS or Object Storage)
Copy / Move
Cloud Pools
Container (cache)
Primary Storage (NAS or Object Storage)
Cloud Pool
Copy / Move
Nas Pools
Container (cache)
Primary Storage (NAS or Object Storage)
Nas Pool
Copy / Move
Task
Container (cache)
Primary Storage (NAS or Object Storage)
Copy / Move
From Reference File
Container (cache)
Primary Storage (NAS or Object Storage)
Copy / Move
Data Archiving / Active Archive
Before to use this workflow, make sure to configure the container and cache. Learn more to Container configuration section
Definition
To archive a collection of data that can be moved from an expensive primary storage tier to a secondary storage tier. The data may or may not be removed from primary storage and must remain accessible from time to time. Data archiving allows data to be stored that does not require intensive reading and writing, but can be modified or read from time to time.
Why to use this workflow?
Data is stored on less expensive storage media, such as object storage, tape storage or even low-cost disks. An active archive includes functionality to seamlessly move data between different storage tiers (disk - tape - cloud) so that applications or users can access it directly.
Scan an LTFS Media
This workflow makes it possible to re-import tapes that have already been written.
To import LTFS Tapes into Nodeum, follow these steps :
Add the LTFS TAPE 000001L6 in the tape library connected to Nodeum
Define a workflow to scan the LTFS TAPE and then execute it
Select Scan an LTFS Media
Select as the source the tape need to scan "Tape “000001L6"
Launch the task
Get the result after the task execution
Data are indexed and recognized by Nodeum without reading the complete tape
Metadata are injected in the catalog
End user can see and retrieve easily the contents (files, images, video)
End user accesses contents straight away; no need to make new set of tapes
Task Monitoring
To aid in monitoring task execution, the Console offers a comprehensive monitoring feature. This feature provides a global overview of all task activities.
Three different sections are available :
Upcoming task
This section shows the task(s) scheduled. For each task, you can see detailed information including the date - time when the task will be executed.
It is possible to skip an execution in clicking in the right cross icon.
If a task hasn't been executed properly, then you will an information saying that "task execution missed for one or multiple occurence".
Running task
This section provides the list of the task(s) which are currently running.
For each of these tasks, you can
See the detail and the status.
Pause the task execution.
Resume a paused task .
Stop the task execution
History
In this section you can find all the tasks that have been completed.
Task Report
For each task running or finished, you can open to see the global view about the task.
To see the full informations click on the arrow beside the task:
And have a look on the task information:
Definitions of task informations :
Timing
Task starts the x date and at x hours
Elapsed time for the task
Estimated remaining time to process the task
Type
Offline Archive
Active Archive
Data Exchange
Move or Copy to a destination:
Processed size
size of data processed
Processed items
Number of items of data processed
To see the good process click on the tab processed items*
Logs
Details of task's log
Each task report contains all files processed during the task, a downloaded copy can be exported.
Task Log
The following table exposes the list of messages available in the logs, the messages are mostly sorted in the displayed' sequence.
Task is starting
n/a
The task starts.
Parallel Processing mode is Enabled
Parallel Processing mode is Disabled
Enabled means that all processes are running in // at the opposite of Disabled where the different processes are executed in serial. The Disabled mode is used for troubleshooting a task.
Initialization Process is starting
n/a
The process in charged of the inventory is started.
License is valid
License is not valid ('description of the error'), task cannot be executed
It checks the validity of the Nodeum licence is valid.
Task Job Limits : 100 files/job - 2 parallel job(s) - 200 parallel DBFile/job - 1000000000 B/job
# parallel job(s) # parallel DBFile/job # B/job
It defines the number of files to be processed per batch. This example means that the task will included 2 jobs; each of them will contains maximum 200 files or a size of maximum 1GB. When one of these values is reached, then another batch will be defined.
Incremental Mode is Configured. The process will select only new content
Full Mode is Configured.
The process will select only new content or will select all files to be processed.
Processing Order is Configured. (Ascending)
Processing Order is Configured. (Descending)
This manages the order of files processing. Best performance is Ascending.
Check integrity of data transfers: MD5
Check integrity of data transfers: CRC32 Check integrity of data transfers: XXHAS64
The type of integrity check is displayed.
Pool pool name: Tape tape barcode: VALID
n/a
This verifies if the associated pool contains a valid tape.
Pool pool name:1 valid tape(s)
n/a
This summarizes the number of valid tapes in the associated pool.
Main Process is starting
n/a
The copy process starts.
Finalization Process is starting
n/a
The process responsible of managing the catalog changes starts.
Tape tape barcode: is Mounted on the drive drive s/n
n/a
Indicates a request to mount a tape in a tape drive.
Tape tape barcodeloaded on the drive drive s/n is Dismounted
n/a
Indicates a request to dismount a tape in a tape drive.
LTFS File System of the Tape tape barcodeloaded on the drive drive s/n is Mounted
n/a
The LTFS partition is mounted for the specified tape in the specified tape drive.
LTFS File System of the Tape tape barcodeloaded on the drive drive s/nis Umounted
n/a
The LTFS partition is unmounted for the specified tape in the specified tape drive.
Unable to Mount LTFS File System of the Tape tape barcodeloaded on the drive drive s/n
n/a
The tape cannot mount its LTFS file system properly.
Unable to Copy a File. (Read-Only LTFS FileSystem, Try to mount it again)
n/a
Indicates that the tape is mounted with Read-Only LTFS file system and can't copy the file. The task will remount another tape. It is highly recommended to check the tape manually.
Total of items/size parsed: xxxx(yyyTB)
n/a
This list the result of the Initialization process and list the number of files and size to be processed by the task.
Total of items/size already up to date on the destination: xxxx(yyyTB)
n/a
This is the result of the verification of the number of files and the size already available at the destination.
Total of items/size to process on this schedule: xxxx(yyyTB)
n/a
This is the summary of what will be processed during this task execution.
Initialization Process is finished
n/a
This confirms that the Initialization process is finished and then files inventory is available.
All Files were processed on this Rule
n/a
This confirms that all files have been processed.
Main Process is finished
n/a
The Main process is finalized.
Retry made on volume tape barcode: xx
n/a
Shows the number of mount retries for all the defined tape barcode.
Finalization Process is finished
n/a
The Finalization process is finalized.
Task is finished with Success
n/a
The Task is finalized.
Last updated
