Prerequisites

Cache Configuration

Cache is required to fulfil these two main features:

Data Archiving
Scan LTFS Tape

Caching system has to be a block device which is seen behind /dev/ device or any supported mounted filesystem.

Here are some examples:

Internal RAID on local disks
Internal RAID on JBOD
SCSI Attached Storage
iSCSI Attached Storage
Linux based block replication (e.g., DRBD, GlusterFS, …).

How to proceed?

In case the disk used for the cache is not ready, you need to initialize it by executing the following commands.

First, run /opt/nodeum/bin/core/cache_disk_format.sh /dev/sdx (replace sdx by the correct device, likely sdb) to create the CACHE Disk.

The second step is to select the CACHE Disk with this command:

/opt/nodeum/bin/core/cache_disk_select.sh /dev/sdx (where sdx is the same device as the previous step).

Then the process will start.

Once done, the disk is configured as a cache and you can see its status:

Extend your cache

Based on the server type, you have multiple the solution to increase the cache size.

Backup your data

Before to extend the cache, copy all data of the cache into a secondary storage

Increase the disk volume capacity

RAID

Add new disk(s) into the available RAID volume used as Nodeum Cache.
Wait till the RAID reconstruction is finished (this can take couple of days depending of the disk size).

Virtual Machine Datastore

Shutdown the Nodeum server
Edit the Nodeum server setting, pick the correct disk and extent its size
Save the configurations

Extend the volume partition

If the new disk partition will be greater than 16TB, the resize2fs command (version <1.43) will not work.

Error : "resize2fs: New size too large to be expressed in 32 bits"

To fix use a latest version Gparted ; this tool will provide this command in 64bits version. A procedure to use Gparted is decribed below.

Gparted procedure :

Use a Live ISO and edit the disk size graphically :

Download the Gparted ISO (https://gparted.org/download.php)
Creation of an USB key with Gparted ISO (in following Gparted Live CD/USB creation procedure : https://gparted.org/livecd.php).
Boot on the Live ISO
1. For Physical Server, this can be done with a CD/USB key or via Remote Management Server interface (where ISO image will be mapped as a CD).
2. For Virtual Server, you have to boot the VM on the downloaded Gparted ISO.
When the server boot, you will get the Gparted graphical interface.
If you have a disk partition higher than 16 TB, you need to have a 64 bits file system. To convert your file system into 64 bits, you need to execute this command with th Gparted terminal : sudo resize2fs -b /dev/sdb1
Select the Nodeum cache disk (most of cases, /dev/sdb1), the 'resize/move' option should be available, click on it.
Drag the vertical bar to extend the partition size
Once done, click on Resize/Move
Once done, click on apply to save the changes to the disk
Once the resize have finished, reboot the server and enjoy your new bigger Cache.

Command line interface procedure (available starting Nodeum 1.8 and higher)

Stop all Nodeum services

$ /usr/mtc/bin>./core_stop all
[core_stop] core_watchdog has stopped successfully (CORE_STOP/cstop_stop_watchdog)
[core_stop] Trying umount (CORE_STOP/cstop_stop_fuse).
[core_stop] Trying umount force (CORE_STOP/cstop_stop_fuse).
[core_stop] Trying umount lazy (CORE_STOP/cstop_stop_fuse).
umount: /mnt/FUSE: not mounted
[core_stop] core_fuse has stopped successfully (CORE_STOP/cstop_stop_fuse).
[core_stop] data_mining has stopped successfully (CORE_STOP/cstop_stop_data_mining)
[core_stop] core_manager has stopped successfully (CORE_STOP/cstop_stop_manager)
[core_stop] library_manager has stopped successfully (CORE_STOP/cstop_stop_library)
[core_stop] core_superv has stopped successfully (CORE_STOP/cstop_stop_superv)

2. Check which disk must be extended, the disk is the one mounted on /mnt/CACHE

$ /usr/mtc/bin>df -lh
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda2       146G   88G   52G  64% /
tmpfs           1,9G   40K  1,9G   1% /dev/shm
/dev/sda1       465M   39M  402M   9% /boot
/dev/sdb1       886G  7,4G  834G   1% /mnt/CACHE

3. Umount the Cache disk

$ /usr/mtc/bin>umount /mnt/CACHE
$ /usr/mtc/bin>mount
/dev/sda2 on / type ext4 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw)
/dev/sda1 on /boot type ext4 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)

4. Check the disk

sanity, if there are errors that are not fixed automatically, stop here (fix them before going to the next step)

$ /usr/mtc/bin>e2fsck -f -y -v -C 0 /dev/sdb1
e2fsck 1.41.12 (17-May-2010)
Le bitmap de bloc du groupe 707 n'est pas initialisé
alors que le bitmap d'i-noeud est en cours d'utilisation.
Corriger ? oui
 
Le bitmap de bloc du groupe 708 n'est pas initialisé
alors que le bitmap d'i-noeud est en cours d'utilisation.
Corriger ? oui
 
Le bitmap de bloc du groupe 709 n'est pas initialisé
alors que le bitmap d'i-noeud est en cours d'utilisation.
Corriger ? oui
 
Le bitmap de bloc du groupe 710 n'est pas initialisé
alors que le bitmap d'i-noeud est en cours d'utilisation.
Corriger ? oui
 
Le bitmap de bloc du groupe 711 n'est pas initialisé
alors que le bitmap d'i-noeud est en cours d'utilisation.
Corriger ? oui
 
Passe 1 : vérification des i-noeuds, des blocs et des tailles
Passe 2 : vérification de la structure des répertoires
Passe 3 : vérification de la connectivité des répertoires
Passe 4 : vérification des compteurs de référence
Passe 5 : vérification de l'information du sommaire de groupe
 
   63944 inodes used (0.11%)
      98 non-contiguous files (0.2%)
       3 non-contiguous directories (0.0%)
         nombre d'i-noeuds avec des blocs ind/dind/tind : 0/0/0
         Histogramme des profondeurs d'extents : 63863/71
 5658831 blocks used (2.40%)
       0 bad blocks
       1 large file
 
   63702 regular files
     233 directories
       0 character device files
       0 block device files
       0 fifos
       0 links
       0 symbolic links (0 fast symbolic links)
       0 sockets

5. use parted to extend the partition size

$ /usr/mtc/bin>parted /dev/sdb
(parted) unit s
(parted) print free
Modèle: VMware Virtual disk (scsi)
Disque /dev/sdb : 2097152000s
Taille des secteurs (logiques/physiques): 512B/512B
Table de partitions : gpt
 
Numéro  Début        Fin          Taille       Système de fichiers  Nom      Fanions
        34s          2047s        2014s        Espace libre
 1      2048s        1887434751s  1887432704s  ext4                 primary
        1887434752s  2097151966s  209717215s   Espace libre
 
(parted) rm 1
(parted) mkpart primary 2048s 2097151966s
(parted) print free
Modèle: VMware Virtual disk (scsi)
Disque /dev/sdb : 2097152000s
Taille des secteurs (logiques/physiques): 512B/512B
Table de partitions : gpt
 
Numéro  Début  Fin          Taille       Système de fichiers  Nom      Fanions
        34s    2047s        2014s        Espace libre
 1      2048s  2097151966s  2097149919s  ext4                 primary
 
(parted) quit

6. Check the disk again

$ /usr/mtc/bin>e2fsck -f -y -v -C 0 /dev/sdb1
e2fsck 1.41.12 (17-May-2010)
Passe 1 : vérification des i-noeuds, des blocs et des tailles
Passe 2 : vérification de la structure des répertoires
Passe 3 : vérification de la connectivité des répertoires
Passe 4 : vérification des compteurs de référence
Passe 5 : vérification de l'information du sommaire de groupe
 
   63944 inodes used (0.11%)
      98 non-contiguous files (0.2%)
       3 non-contiguous directories (0.0%)
         nombre d'i-noeuds avec des blocs ind/dind/tind : 0/0/0
         Histogramme des profondeurs d'extents : 63863/71
 5658831 blocks used (2.40%)
       0 bad blocks
       1 large file
 
   63702 regular files
     233 directories
       0 character device files
       0 block device files
       0 fifos
       0 links
       0 symbolic links (0 fast symbolic links)
       0 sockets
--------
   63935 files

7. Extend the disk size

$ /usr/mtc/bin>resize2fs -p /dev/sdb1
resize2fs 1.41.12 (17-May-2010)
En train de retailler le système de fichiers sur /dev/sdb1 à 262143739 (4k) blocs.
Début de la passe 1 (max = 800)
Extension de la table d'i-noeudsXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Le système de fichiers /dev/sdb1 a maintenant une taille de 262143739 blocs.

If the operation fails, see Cautions 1 above.

8. Mount the disk

$ /usr/mtc/bin>mount /dev/sdb1 /mnt/CACHE/ -o user_xattr
$ /usr/mtc/bin>df -lh
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda2       146G   88G   52G  64% /
tmpfs           1,9G   32K  1,9G   1% /dev/shm
/dev/sda1       465M   39M  402M   9% /boot
/dev/sdb1       985G  7,4G  927G   1% /mnt/CACHE

9. Start the Nodeum services

$ /usr/mtc/bin>./core_start all
[core_start] core_superv has started successfully (CORE_START/cstart_start_superv)
[core_start] library_manager has started successfully (CORE_START/cstart_start_library)
[core_start] core_manager has started successfully (CORE_START/cstart_start_manager)
[core_start] data_mining has started successfully (CORE_START/cstart_start_data_mining
[core_start] core_fuse has started successfully (CORE_START/cstart_start_fuse).
[core_start] core_watchdog has started successfully (CORE_START/cstart_start_watchdog)

Last updated 4 months ago