Using Ultrium 7 WORM tapes

[badgerfruit]

Good morning all

I have just installed a WORM drive (Ultrium LTO8HH) into one of our servers and have previously only used RW media with it via LTFS. So for example, to access and use a RW tape, I would simply call:

Code: Select all

/usr/local/bin/mkltfs -d /dev/st0
/usr/local/bin/ltfs /mnt/tape -o devname=/dev/st0

This would then allow me to write into /mnt/tape as if it were a regular folder and then umount the drive when done.

Please note, due to the required capacity for storage, I am using LTO7 WORM tapes but in an LTO8 drive (so that we can expand into the future without needing to buy a new tape drive in X months' time) and according to the compatibility datasheet https://www.ibm.com/support/knowledgece ... udrca.html, the LTO 7 WORM tapes are compatible with the LTO8 drive.

Okay so if I insert an LTO7 WORM tape into the drive and then issue the command:

Code: Select all

/usr/local/bin/mkltfs -d /dev/st0

I get the following message:

Code: Select all

LTFS15000I Starting mkltfs, QUANTUMLTFS Standalone version 2.2.2, log level 2
LTFS15042I This binary is built for Linux (x86_64)
LTFS17087I Kernel version: Linux version 3.10.0-693.11.1.el7.x86_64 (builder@kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC) ) #1 SMP Mon Dec 4 23:52:40 UTC 2017 i386
LTFS17089I Distribution: CentOS Linux release 7.4.1708 (Core)
LTFS15003I Formatting device '/dev/st0'
LTFS15004I LTFS volume blocksize: 524288
LTFS15005I Index partition placement policy: None
LTFS17085I Plugin: Loading "ltotape" driver
LTFS20013I Drive type is IBM ULTRIUM-HH8, serial number is xxxxxxxxxx, firmware revision is H9E3
LTFS17160I Maximum device block size is 1048576
LTFS20062E Unsupported cartridge type (LTO7WORM)
LTFS11299E Cannot format: unsupported medium
LTFS15023I Formatting failed

Okay so, "unsupported cartridge type" and "unsupported medium" are pretty clear, it doesn't like the tape. Sure enough, from Quantum's documentation http://downloads.quantum.com/ltfs/2.2/6 ... e_RevA.pdf and page 27:

Q: Can I Use WORM Media with Quantum LTFS?
A : No, WORM Media cannot be partitioned and so is not usable with Quantum LTFS.

Okay so that (to me at least) says that in order to use a WORM tape, I should use something other than LTFS. It just doesn't specify what. I have googled the heck out of this and had no joy at all so I am reaching out to the community for some help as I guess there will be someone out there who has used a WORM tape on a Centos box - surely I am not the first person on the planet to have done so.

Thank you for reading and I look forward to any suggestions, advice, ideas etc!

[avij]

I guess the method I used back in stone age with QIC and DAT tapes still work, tar cvfz /dev/st0 /path/to/backup

star and cpio may also be used.

[badgerfruit]

I guess the method I used back in stone age with QIC and DAT tapes still work, tar cvfz /dev/st0 /path/to/backup

star and cpio may also be used.

Hi there, thank you for your prompt reply but sadly, this has not worked:

Code: Select all

[~]# tar cvfz /dev/st0 /path/to/files
tar: Removing leading `/' from member names
/path/to/files/
/path/to/files/mailtemplate.txt
/path/to/files/file1.php
/path/to/files/files2.php
tar (child): /dev/st0: Cannot write: Input/output error
tar (child): Error is not recoverable: exiting now

[avij]

I investigated this a bit and you may be hitting the minimum speed requirement. Our friends over at Gentoo forums had a suggestion to use mbuffer for buffering. You can find mbuffer from the EPEL repository.

You could also try to use tar without compression (the z flag) and with a single large file (like .iso images or disk images), and see if it works any better. Smaller files tend to be slower to process.

Unrelated to this, but also note that some tar archive formats have limitations for total archive size. You may need to do some research on this, and most importantly, try to restore the files from the tape. I have seen some cases where tar wrote the archive happily, but extracting the files from the same archive did not work due to the tar archive format limitation.

[badgerfruit]

Sadly, I have to write many thousand individual small files to the tape (and then be able to read them from the tape afterwards at a later date) so I can't just create a zip/ISO and drop that over using tar/star/something-else.

I checked out that thread (thanks for that) and the last reply on there starts with a comment "just load the 'st' driver and write to the drive with tar", you'll have to excuse my noobishness here but wtf lol?

I've *heard of* st and mt but never used them, I can of course rtfm but before I go off and do that, am I right in thinking that to WRITE to the WORM tape, you simply send the files to the /dev/xxx ?

How would you go about reading them once they've been written, mount the device somewhere and read it like a normal folder?

I'm sorry for being thick here but I've never had to use WORM before.

[TrevorH]

If the 'st' driver isn't loaded then you don't have any /dev/st* devices.

[badgerfruit]

If the 'st' driver isn't loaded then you don't have any /dev/st* devices.

Ah har, well, in that case, I can confirm that the ST driver IS loaded!

Code: Select all

[root@myserver ~]# ls -l /dev/st*
crw-rw---- 1 root tape 9,  0 Jan 30 10:07 /dev/st0
crw-rw---- 1 root tape 9, 96 Jan 30 10:07 /dev/st0a
crw-rw---- 1 root tape 9, 32 Jan 30 10:07 /dev/st0l
crw-rw---- 1 root tape 9, 64 Jan 30 10:07 /dev/st0m

EDIT: So I had another look at that Gentoo thread linked to earlier and thought it was worth a try using mbuffer as suggested, whacked up the command:

Code: Select all

tar -c /mnt/myfiles | mbuffer -m 800M -p 95 -s 65536 > /dev/st0

and strangely enough, no errors ...

Code: Select all

tar: Removing leading `/' from member names
in @  0.0 KiB/s, out @  0.0 KiB/s, 2178 MiB total, buffer   0% full
summary: 2178 MiByte in 29.8sec - average of 73.1 MiB/s, 25x full

... now, can anyone offer an idea on how I can see if this actually wrote to the tape please?

[TrevorH]

Yes, read it back using tar -vtf /dev/st0

[avij]

At some point it might be beneficial to run man tar and read how tar works.

[badgerfruit]

Good morning all
So first off, thank you for all the help so far, I'm pleased to report some progress!

Code: Select all

[root@myserverrestore]# tar -vtf /dev/st0
tar: /dev/st0: Cannot read: Cannot allocate memory
tar: At beginning of tape, quitting now
tar: Error is not recoverable: exiting now

Google'd the error and came across https://unix.stackexchange.com/question ... -scsi-tape which explains (in detail) the issue. From there, they suggested using a slightly different tar command which I have done ...

Code: Select all

[root@myserver~]# dd if=/dev/st0 bs=256k | tar tvf -
drwxr-xr-x root/root         0 2018-02-13 10:30 mnt/bigdisk/quantum-backup/
-rw-r--r-- root/root       144 2018-02-12 10:23 mnt/bigdisk/quantum-backup/mailtemplate.txt
-rwxr-xr-x root/root      4639 2016-12-29 09:27 mnt/bigdisk/quantum-backup/tf-insite.php
-rwxr-xr-x root/root  28119040 2018-02-13 09:30 mnt/bigdisk/quantum-backup/Patbase.adp
-rw-r--r-- root/root     12288 2018-01-19 11:37 mnt/bigdisk/quantum-backup/.oldbackup.old.swp
-rwxrwxr-x root/root      1507 2018-02-13 10:29 mnt/bigdisk/quantum-backup/LTFSbackup.sh
-rwxr-xr-x root/root  25260032 2018-02-13 09:30 mnt/bigdisk/quantum-backup/Patbase.ade
-rw-r--r-- root/root 2178769994 2018-02-12 23:00 mnt/bigdisk/quantum-backup/docservbackup.sql
-rw-r--r-- root/root   51243625 2018-02-13 09:30 mnt/bigdisk/quantum-backup/mydocs.tgz
-rw-r--r-- root/root       1044 2018-02-13 10:30 mnt/bigdisk/quantum-backup/backup.sh
0+34843 records in
0+34843 records out
2283427840 bytes (2.3 GB) copied, 9.5264 s, 240 MB/s

So it's read all the files I wrote to the tape (yay) but, it doesn't seem to have actually done anything with them (in that they're not restored on to the filesystem - well, the ORIGINAL files are but I'll need to restore the files from tape (ideally to a specified location)).

I shall endeavour to read more about tar and perhaps this may hold the solution!
Thank you again (and of course, you're welcome to write the solution for me hahaha)