Hook service (callback)

About Callback

Callbacks are codes which are executed at a specific stage of each data movement.

Each task can contains multiple callbacks. For each callback, the code has to be defined and when it has to be triggered.

Callbacks are executed into a 'safe' - sandbox environment, which prevent any border effect with the overall system and attached storage.

The executed task information are available into each block of code, and can be manipulated.

Plugins oriented

This feature supports multiple language, each of them is defined into a specific plugins. For now, two languages are supported: python and JS.

This means that two plugins are available and have to be loaded:

Extract of the Nodeum Ansible inventory/group_vars/all/options.yml file

...
###! Install the plugin for handling callbacks in JavaScript
# dispatcher_plugin_javascript_hooks_enabled: true
###! Install the plugin for handling callbacks in Python
# dispatcher_plugin_python_hooks_enabled: true
...

Python Code Structure

This is the standard block code of a callback written in Python. The code logic has to be included into the def run(req)

import utils
import sys

def run(req):
  ...
  return True

The req structure can then be manipulated where there are information to be retrieved from the task itself. The list of standard information is the following:

This provides the ID of the task: req.id

This retrieve all information about the task

info = utils.get_node_info(req)

then following information are available:

• File path: info.path

• File Mode: info.mode

• File UID: info.uid

• File GID: info.gid

• File size: info.size

• File access time: info.atime.ToDatetime()

• File modification time: info.mtime.ToDatetime()

• File creation time: info.ctime.ToDatetime()

• File additional metadata: metadata = info.metadata and then metadataKey = metadata['key']

JS Code Structure

The logic is the same in JS, with a different logic b

const utils = require('./utils')

module.exports = function(req) {
  ....
  return true
}

The req structure can still be manipulated where there are information to be retrieved from the task itself. The list of standard information is the following:

This provides the ID of the task: const id = req.getId()

And then it is possible to retrieve all information about the task:

const info = utils.getNodeInfo(req)

then following information are available:

• File path: info?.getPath()

• File Mode: info?.getMode()

• File UID: info?.getUid()

• File GID: info?.getGid()

• File size: info?.getSize()

• File access time: info?.getAtime()?.toDate()

• File modification time: info?.getMtime()?.toDate()

• File creation time: info?.getCtime()?.toDate()

• File additional metadata: metadataMap = info?.getMetadataMap() and then metadataKey = metadataMap.get('key')

Callback definition

Console

The Console allows the creation and definition of callback. It has to be done per task creation:

Terminal

The callbacks are stored into separated folders: /opt/nodeum/plugins/javascript-hooks/ and /opt/nodeum/plugins/python-hooks/. These folders are present into the Nodeum Data Mover nodes.

Tips & Trick

Task Status Verification

Previous task status can be checked in the callback "Before task is executed".

Example in js:

task.getLastExecution()?.getProgress()

export enum Progress {
  PROGRESS_UNSPECIFIED = 0,
  PROGRESS_INIT_STARTED = 1,
  PROGRESS_INIT_DONE = 2,
  PROGRESS_PROCESS_STARTED = 4,
  PROGRESS_PROCESS_DONE = 8,
  PROGRESS_FINAL_STARTED = 16,
  PROGRESS_FINAL_DONE = 32,
  PROGRESS_HAS_ERROR = 64,
  PROGRESS_STOPPED = 128,
  PROGRESS_STOPPED_BY_USER = 256,
  PROGRESS_PAUSED = 512,
}