Amanda: Difference between revisions
m (→Welcome) |
|||
(15 intermediate revisions by the same user not shown) | |||
Line 7: | Line 7: | ||
Guaranteed data retention time is 10 days, typical data retention time is 1 month. | Guaranteed data retention time is 10 days, typical data retention time is 1 month. | ||
Periodically, amanda backups are archived to offline media (SDLT tapes, USB hard drives) stored in secure location and kept permanently. | |||
== AMANDA information == | == AMANDA information == | ||
Line 14: | Line 16: | ||
* http://www.zmanda.com - Amanda on zmanda.com | * http://www.zmanda.com - Amanda on zmanda.com | ||
* http://wiki.zmanda.com - Amanda wiki | * http://wiki.zmanda.com - Amanda wiki | ||
* https://github.com/zmanda/amanda - Amanda git repository on github | |||
== AMANDA status (automatically updated) == | == AMANDA status (automatically updated) == | ||
Line 35: | Line 38: | ||
* [https://daq.triumf.ca/~daqweb/amanda/amadmin_find.txt amadmin find] | * [https://daq.triumf.ca/~daqweb/amanda/amadmin_find.txt amadmin find] | ||
* [https://daq.triumf.ca/~daqweb/amanda/amadmin_info.txt amadmin info] | * [https://daq.triumf.ca/~daqweb/amanda/amadmin_info.txt amadmin info] | ||
== Adding backup clients == | == Adding backup clients == | ||
Line 43: | Line 44: | ||
* prepare backup client machine: | * prepare backup client machine: | ||
* RHEL7/SL7/CentOS7 | * Ubuntu LTS 20.04 - [[Ubuntu#Install_amanda_client]] | ||
* RHEL7/SL7/CentOS7 - [[SLinstall#Enable_AMANDA_backups_.28CentOS7.29]] | |||
* RHEL/SL/4/5/6: | * RHEL/SL/4/5/6: | ||
<pre> | <pre> | ||
Line 67: | Line 69: | ||
NOTE: once a client is removed from the amanda backup list, normal data recycling will eventually permanently delete the backup data from amanda. Backup data saved on amanda archive tapes is preserved forever. | NOTE: once a client is removed from the amanda backup list, normal data recycling will eventually permanently delete the backup data from amanda. Backup data saved on amanda archive tapes is preserved forever. | ||
== Troubleshooting failing clients == | == Troubleshooting failing clients == | ||
Line 95: | Line 85: | ||
</ul> | </ul> | ||
== Restoring data from amanda == | |||
To recover files from amanda: | |||
* examine the list of all available backup files: https://daq.triumf.ca/~daqweb/amanda/all_files.txt | |||
* identify the backup files to be restored. For restoration of deleted files, this would be the latest full backup (level 0), the latest level 1 backup, the latest level 2 backup, etc. | |||
* if this is an emergency (you need the files right away, or you need files from a backup older than the latest full backup), contact [http://www.triumf.ca/profiles/3103 Konstantin Olchanski] by email or SMS to the listed cell phone number. | |||
* send a request to the DAQ group asking to retrieve the amanda backup files. Typically with rsync or scp these files from amanda to disk storage on some client machine. | |||
* unpack the amanda backup files: (for full restoration, untar the amanda incremental backups files consecutively) | |||
<pre> | |||
dd if=00004.titan00._home1.0 bs=32k skip=1 | tar xzvf - | |||
dd if=00055.titan00._home1.1 bs=32k skip=1 | tar xzvf - | |||
dd if=00114.titan00._home1.2 bs=32k skip=1 | tar xzvf - | |||
etc... | |||
</pre> | |||
== Adding a new client on the amanda server == | |||
* ssh root@amanda | |||
* edit ~amanda/daily/disklist, | |||
* add entry "clienthostname /home1 comp-user-tar # clientusername@triumf.ca" | |||
* su - amanda | |||
* run "amcheck -c daily" | |||
* if amcheck is not happy, follow instructions for troubleshooting failing client | |||
== Instructions for using bsdtcp authentication (Obsolete) == | |||
NOTE: Information in this section is obsolete | |||
The default "bsd" authentication uses the UDP transport and sometimes has problems | The default "bsd" authentication uses the UDP transport and sometimes has problems | ||
Line 147: | Line 153: | ||
</ul> | </ul> | ||
== Restoring data from archive USB HDDs == | |||
Archive USB HDDs contain an rsync copy of amanda system files and of the latest full backups. Incremental backups are not archived. | |||
To restore the data, mount the USB disk *read only*, identify the backups file (use find . | grep clienthostname), copy the backup file to local storage, unmount, disconnect the archive disk, recover individual files per instructions above (dd if=00004.titan00._home1.0 bs=32k skip=1 | tar xzvf -). | |||
( | |||
dd if= | |||
== Restoring data from archive SDLT tapes == | |||
<ul> | <ul> | ||
<li>AMANDA backup tapes are made using "tar -M" from amanda:~root/backup.sh. | <li>AMANDA backup tapes are made using "tar -M" from amanda:~root/backup.sh. | ||
Line 254: | Line 171: | ||
<li>This restores the amanda compressed backup file. Unpack it by running "dd if=00257.dork._home1.0 bs=32k skip=1 | tar xzvf -" | <li>This restores the amanda compressed backup file. Unpack it by running "dd if=00257.dork._home1.0 bs=32k skip=1 | tar xzvf -" | ||
</ul> | </ul> | ||
Latest revision as of 17:19, 27 January 2021
Welcome
The AMANDA disk backup server is operated by the TRIUMF DAQ group as a backup and archiving service for data acquisition, detector facility and other computers systems in the ST department, Science division and other TRIUMF users.
Backups are stored on a 20TB disk array (RAID6/XFS). Backups are done on a 10 day schedule - full backup, level 1 incremental, level 2 incremental, etc, full backup, cycle repeats. Generally the system stores 2 full backups (the last full backup and the one before it) plus the incremental backups between them. Older full backups and older incrementals are stored space permitting.
Guaranteed data retention time is 10 days, typical data retention time is 1 month.
Periodically, amanda backups are archived to offline media (SDLT tapes, USB hard drives) stored in secure location and kept permanently.
AMANDA information
- http://www.amanda.org - Amanda project home page
- http://sourceforge.net/projects/amanda - Amanda on sourceforge
- http://www.zmanda.com - Amanda on zmanda.com
- http://wiki.zmanda.com - Amanda wiki
- https://github.com/zmanda/amanda - Amanda git repository on github
AMANDA status (automatically updated)
Use username: amanda, password: amanda
- summary of virtual tape usage
- latest backup reports: summary, last run, last amreport, errors, summary of errors
- reports: per host, per day, amreport per day
- all backup files: all files
- top users: files, file systems, hosts
- erased virtual tapes
- amcheck
- amstatus
- amoverview
- last amdump
- last amflush
- virtual tape slots
- amadmin balance
- amadmin disklist
- amadmin find
- amadmin info
Adding backup clients
Note: currently only linux backup clients are documented. MacOS is known to work but amanda client package does not seem to be generally available.
- prepare backup client machine:
- Ubuntu LTS 20.04 - Ubuntu#Install_amanda_client
- RHEL7/SL7/CentOS7 - SLinstall#Enable_AMANDA_backups_.28CentOS7.29
- RHEL/SL/4/5/6:
ssh root@client edit ~amanda/.amandahosts, add line "amanda.triumf.ca amanda" edit /etc/hosts.allow, add line "amandad: amanda.triumf.ca" chkconfig xinetd on chkconfig amanda on service xinetd reload
# if local filewall is running (before RHEL/SL/CentOS7) # Allow input from the backup server over UDP to the amanda port and on any unprivileged TCP port iptables -I INPUT 1 -m state --state NEW -s 142.90.100.196 -p udp --dport amanda -j ACCEPT iptables -I INPUT 1 -m state --state NEW -s 142.90.100.196 -p tcp --dport 1024:65535 -j ACCEPT service iptables save
- send a request to the DAQ group with following information: machine full hostname (i.e. titan00.triumf.ca), list of filesystems to backup (i.e. /home1, /etc) and a contact email address who will receive any error messages from failed backups.
Removing backup clients
Send a request to the DAQ group with the name of machine or filesystem to be removed, stop the amanda services on the client machine.
NOTE: once a client is removed from the amanda backup list, normal data recycling will eventually permanently delete the backup data from amanda. Backup data saved on amanda archive tapes is preserved forever.
Troubleshooting failing clients
If an amanda client is failing backups, login to the client as root and try this:
- Run "df", it should not hang
- The system disk should have some free space
- There should be no stuck amanda processes: "ps -efw | grep amanda" should return nothing. If you see any running amanda pro cesses, kill them.
- xinetd should be running, do "service xinetd restart" if unsure
- (on Solaris) inetd should be running, do "inetd -s" if unsure
- Examine the log files in /var/log/amanda. Do "ls -ltr" to see the latest files. If any log files have size zero, the system disk may have been 100% full at that time, preventing amanda from working.
- On the amanda server, as amada, run "amcheck -c daily host"
Restoring data from amanda
To recover files from amanda:
- examine the list of all available backup files: https://daq.triumf.ca/~daqweb/amanda/all_files.txt
- identify the backup files to be restored. For restoration of deleted files, this would be the latest full backup (level 0), the latest level 1 backup, the latest level 2 backup, etc.
- if this is an emergency (you need the files right away, or you need files from a backup older than the latest full backup), contact Konstantin Olchanski by email or SMS to the listed cell phone number.
- send a request to the DAQ group asking to retrieve the amanda backup files. Typically with rsync or scp these files from amanda to disk storage on some client machine.
- unpack the amanda backup files: (for full restoration, untar the amanda incremental backups files consecutively)
dd if=00004.titan00._home1.0 bs=32k skip=1 | tar xzvf - dd if=00055.titan00._home1.1 bs=32k skip=1 | tar xzvf - dd if=00114.titan00._home1.2 bs=32k skip=1 | tar xzvf - etc...
Adding a new client on the amanda server
- ssh root@amanda
- edit ~amanda/daily/disklist,
- add entry "clienthostname /home1 comp-user-tar # clientusername@triumf.ca"
- su - amanda
- run "amcheck -c daily"
- if amcheck is not happy, follow instructions for troubleshooting failing client
Instructions for using bsdtcp authentication (Obsolete)
NOTE: Information in this section is obsolete
The default "bsd" authentication uses the UDP transport and sometimes has problems in the presence of packet loss on the network. In this case, try the "bsdtcp" authentication that uses the TCP transport.
- On the client:
- create file /etc/xinetd.d/amanda-bsdtcp:
# default: off # description: The client for the Amanda backup system.\ # This must be on for systems being backed up\ # by Amanda. service amanda { disable = no socket_type = stream protocol = tcp wait = no user = amandabackup group = disk server = /usr/sbin/amandad # Configure server_args for the authentication type you will be using, # and the services you wish to allow the amanda server and/or recovery # clients to use. # # Change the -auth= entry to reflect the authentication type you use. # Add amindexd to allow recovery clients to access the index database. # Add amidxtaped to allow recovery clients to access the tape device. server_args = -auth=bsdtcp amdump }
- chkconfig amanda off
- chkconfig amanda-bsdtcp on
- service xinetd reload
- On the server:
- in the disklist, replace dump type "comp-user-tar" with "bsdtcp-comp-user-tar"
- su - amanda
- amcheck -c daily ladd03
Restoring data from archive USB HDDs
Archive USB HDDs contain an rsync copy of amanda system files and of the latest full backups. Incremental backups are not archived.
To restore the data, mount the USB disk *read only*, identify the backups file (use find . | grep clienthostname), copy the backup file to local storage, unmount, disconnect the archive disk, recover individual files per instructions above (dd if=00004.titan00._home1.0 bs=32k skip=1 | tar xzvf -).
Restoring data from archive SDLT tapes
- AMANDA backup tapes are made using "tar -M" from amanda:~root/backup.sh.
- The catalog of these tapes, the output of "tar tvf", is created by running "verify.sh" and manually renaming it to amanda-backup- yyyy-mm-dd.txt.
- To restore any file, look in the catalog to find out which tape(s) contains it.
- a) If the file is contained on a single tape, restore it using "tar -b 2048 -xvf /dev/nst0 file1..." as usual.
- b) If the file is split between several tapes, load both tapes into the tape robot into consecutive slots, then follow the exampl e in ~root/example-restore.sh (run "tar -C /home1/restore -M -b 2048 --new-volume-script="/root/loadNextTape.perl" --checkpoint -xvf /dev/nst0 amanda/daily/vtapes/79/data/00257.dork._home1.0")
- This restores the amanda compressed backup file. Unpack it by running "dd if=00257.dork._home1.0 bs=32k skip=1 | tar xzvf -"