What does it try to accomplish? The normal DMF dump script uses one tape/day for dumps. With a 4 week retention period at least 28 tapes will be needed for backups. Level 0 dumps, and in particular incremental dumps, are likely to only use a fraction of the space on each tape which is pretty wasteful. This dump script only uses one tape/week for incremental dumps (as long as they all fit on one tape, which is likely) so a minimum of 8 tapes are needed with a 4 week retention period. The script dumps the configured filesystems on a set of tapes external to DMF that it manages in its own database. The documentation below was written for our site and assumes a IBM 3494a library, the dump script should probably work for other configurations too but for the tape category check. --- Usage sysdump.pl [ -fe ] [ -d date ] [ -l level ] -f Fake run. A xfsdump "simulator" is used instead of performing a real dump. The simulator keeps virtual tapes as files in /tmp containing the length of data "written" and returns EOT when 500MB has been "written". The fake mode expects a tape inventory listing from the tape library to be found as /tmp/tapes.txt. This can be created like so "mtlib -l3494a -qI > /tmp/tapes.txt". The internal tape database is copied to a file with same name and the extention ".fake" if no such file already exists. This "fake" database is then updated each time the script is run in fake mode, including expiry processing. -e Run expiry processing on the internal tape database but don't dump. -d Set the date to be used as todays date. This enables premature expiry in conjunction with the -e option, or debugging of the internal database with the -f option. -l Dump at the specified level no matter what day it is. --- How does it work? The backups on a level 0 day use at least one tape. Each tape that is selected as a dump target is overwritten for level 0 backups. Multiple filesystems are dumped to the same tape if space allows. Incremental dumps fill tapes one at a time. Each incremental dump session the tape with the latest dump is used first. If more tapes are needed free tapes are selected. If a level 0 dump with a date later than the latest incremental dump exists a free tape is selected for the dump session (that is, this is the first incremental based on the latest level 0 dump. We try not to mix incrementals based on different base dumps on a tape). This way incremental dumps are always based on the latest completed level 0 dump and incremental dumps based on the same level 0 dump are kept together on as few tapes as possible. The process goes something like this : - Read internal tape database - If no level 0 dump tape can be found and this is an incremental dump session the dump level reverts to level 0 - Select a tape to begin dumping to (possibly a non-full incremental dump tape on non-level 0 days, otherwise a tape marked with FREETAPE) - Check that the tape is in the right category in the robot - Use the DMF support program dm_tape to mount the selected tape (using OpenVault for the mount service) and run xfsdump on each configured filesystem - If xfsdump returns end of tape, a new tape is selected, mounted and the dump is restarted. - If the backup level is 0, the tape database is expired according to the retention date set (expired tapes are marked with FREETAPE). - The internal tape database is written to disk The tape database is an ASCII file with lines on the form VSN Base date Incr date ( ) The first date is the base dump date and the second is the date of the latest incremental dump on this tape (always zero for level 0 tapes). The date string "FREETAPE" marks the tape as eligible for reuse (that is it can be overwritten). Each level 0 day expired incremental tapes are marked for reuse. The date string "FULLTAPE" marks the tape as full. It cannot be used for further dumps until it expires. FULLTAPE can never appear with level 0 tapes (the incremental dump date needs to be NULLDATE to identify the tape as a level 0 dump tape). The FREETAPE and FULLTAPE date strings can only appear in the incremental date field (the second date field) in the database. The states a tape can be in are as follows : - Free, a lone VSN on one line is considered a free tape - Free, a VSN with the string "FREETAPE" in the Incr field is a free tape - Base dump, if the Base field is a valid date and the Incr field is 0 the tape contains level 0 dumps from the Base date - Incremental dump, if the Base and the Incr field are both valid dates the tape contains incremental dumps from the Incr date, based on the level 0 dumps from the Base date. - Full incremental dump, if the Base field is a valid date and the Incr field contains the string "FULLTAPE" the tape contains incremental dumps based on the level 0 dumps from the Base date and it is full. xfsdump keeps track of dumps, their labels and ids and the tape volumes they use in its own inventory. Since the labels and ids of a dump session are necessary to restore that session the xfsdump inventory is dumped as text and stored on another machine at the end of each dump session. --- Restore procedures Cases : 1. Restore a full filesystem when the system disk is ok, OpenVault works and the local xfsdump inventory is available. - Find the session label and id of the dump in the xfsdump inventory > xfsdump -I depth=2,mnt=hostname:/file_system_mount_point Example ... time: Mon Sep 2 17:54:44 2002 session label: "/hsm_lvl0_20020902" session id: f9bbf447-52d5-1026-8cad-08006913d8f2 level: 0 ... media label: "PDD080" - Restore filesystem > dmxfsrestore -I -o '-D -L /hsm_lvl0_20020902 -S f9bbf447-52d5-1026-8cad-08006913d8f2 /hsm' /hsm_lvl0_20020902 drives dmxfsrestore will mount the tape specified in the inventory and perform the restore operation. Note that OpenVault must be up for this to work. 2. Restoring a few files from a level 0 dump interactively : - Find the session label and id of the dump (see 1.) - Restore files : > dmxfsrestore -I -o '-i -L root_lvl0_20020902 -S f9bbf443-52d5-1026-8cad-08006913d8f2 /' root_lvl0_20020902 drives Do you want to mount PDD080 for xfsrestore - y|n : y Mounting PDD080 - please wait... ... If the dump spans multiple tapes, xfsrestore will ask please change media in drive 1: media change declined (timeout in 3600 sec) 2: display media inventory status 3: list needed media objects Unmount the current tape : > ov_stat -d Drive Name Group Access Broken Disabled SoftState HardState DCP SoftState DCP HardState Occupied Cartridge PCL 114630 drives true false false inuse loaded ready ready true PDD080 > ov_unmount -d 114630 -V PDD080 -A dmf Unmounted PDD080 > ov_stat -d Drive Name Group Access Broken Disabled SoftState HardState DCP SoftState DCP HardState Occupied Cartridge PCL 114630 drives true false false inuse unloaded ready ready false Then select option 4 (dmxfsrestore will mount the new tape automatically) -> 4 examining new media ... Do you want to mount PDD081 for xfsrestore - y|n : y Mounting PDD081 - please wait... ... ========================== subtree selection dialog ========================== the following commands are available: pwd ls [ ] cd [ ] add [ ] delete [ ] extract quit help -> ls tmp 9140358 A007814 9140357 tapes.txt 9140356 tkt_pek_to_root_7410 9140355 A007400 9140354 .3494sock 9140353 foo.txt 9140349 A000753 9140348 A000695 29791271 .X11-pipe/ 27903040 .X11-unix/ 9140347 .rtmond_socket 9140346 .rtmond_pid 22139379 .arraysvcs/ 17410623 .ps_data/ 13708007 .slip-rendezvous/ 9140345 .mkpd_socket -> cd tmp -> add foo.txt -> extract --------------------------------- end dialog --------------------------------- ... 3. No OpenVault, no local xfsdump inventory - Find the session label and id of the dump in the xfsdump inventory Look in the latest xfsdump inventory that was saved on the remote host (as specified in the backup script). In this case, dmxfsrestore cannot be used, the tapes need to be manually mounted. This can be done from any machine connected to the robot. If the drive 114630 is connected to the restoring machine : Irix > mtlib -l3494a -m -VPDD080 -x114630 AIX > mtlib -l /dev/lmcp0 -m -VPDD080 -x114630 - Restore filesystem If drive 114630 is device 4 on bus 3 : > xfsrestore -D -f /dev/rmt/tps3d4 -L /hsm_lvl0_20020902 -S f9bbf447-52d5-1026-8cad-08006913d8f2 /hsm xfsrestore will prompt for tape changes, they will have to be effected manually Irix > mtlib -l3494a -d -VPDD080 -x114630 > mtlib -l3494a -m -VPDD081 -x114630 AIX > mtlib -l /dev/lmcp0 -d -VPDD080 -x114630 > mtlib -l /dev/lmcp0 -m -VPDD081 -x114630 -- --- Notes Multiple dump runs on the same day is ok. Incremental dumps are always run at level 5. --- Internals Unified tape database format base date incr date ( ) Lvl-0 tapes always have the incr date set to "00000000" ($NULLDATE). Lvl-5 tapes always have a valid base date AND a valid incr date (or FULLTAPE). The base date is always NULLDATE or a valid date. The incr date can be FULLTAPE or FREETAPE also. Common : Read tape database, find latest base date ($basedate) and latest incremental date ($lastincr). The vsn corresponding to the latest incremental date is saved in $vsn (not removed from %vsnhash). lvl0 : Get an empty tape for todays dump, save used tape in @usedtapes dump get further tapes if needed, mark old tapes as FULLTAPE, save used tapes in @usedtapes Set the incr date for all tapes with a base date older than the retention date to $FREETAPE (check that some bases remain). Save the contents of %vsnhash, append the contents of @usedtapes with todays date as base date and $NULLDATE as incr date. lvl5 : If we don't have a tape with the latest incremental, or if there is a newer base dump ($basedate gt $lastincr), mark vsn as FULLTAPE in vsnhash (if $vsn is set), get a fresh tape and save it in @usedtapes. dump get further tapes if needed, mark old tapes as FULLTAPE, save used tapes in @usedtapes Save the contents of %vsnhash, append the contents of @usedtapes with todays date as incr date and $basedate as base date.