Recovery from AIX operating system failure

Recover from AIX OS failure:
———————————

Recover from OS failure.

Contents:
1. How to view the bootlist:
2. How to change the bootlist:
3. How to make a device bootable:
4. How to make a backup of the OS:
5. Shutdown a pSeries AIX system in the most secure way:
6. How to restore specific files from a mksysb tape:
7. Recovery of rootvg

1. How to view the bootlist:

At boottime, once the POST is completed, the system will search the boot list for a
bootable image. The system will attempt to boot from the first entry in the bootlist.
Its always a good idea to see what the OS thinks are the bootable devices and the order of what the OS
thinks it should use. Use the bootlist command to view the order:

# bootlist -m normal -o

As the first item returned, you will see hdisk0, the bootable harddisk.

If you need to check the bootlist in “service mode”, for example if you want to boot from tape to restore the rootvg, use

# bootlist -m service -o

2. How to change the bootlist:

The bootlist, in normal operations, can be changed using the same command as used in section 1, for example

# bootlist -m normal hdisk0 cd0

This command makes sure the hdisk0 is the first device used to boot the system.

If you want to change the bootlist for the system in service mode, you can change the list in order to use rmt0
if you need to restore the rootvg.

# bootlist -m service rmt0

3. How to make a device bootable:

To make a device bootable, use the bosboot command:

# bosboot -ad /dev/ipldevice

So, if hdisk0 must be bootable, or you want to be sure its bootable, use

# bosboot -ad /dev/hdisk0

4. How to make a backup of the OS:

The mksysb command creates an installable image of the rootvg. This is synonym to say that mksysb creates
a backup of the operating system (that is, the root volume group).

You can use this backup to reinstall a system to its original state after it has been corrupted.
If you create the backup on tape, the tape is bootable and includes the installation programs
needed to install from the backup.

To generate a system backup and create an /image.data file (generated by the mkszfile command) to a tape device
named /dev/rmt0, type:

# mksysb -i /dev/rmt0

If a backup tape was created with the -e switch, like in:

# mksysb -i -e /dev/rmt0

then a number of directories are NOT included in the backup. These exclusions are listed in the “/etc/exclude.rootvg” file.

The mksysb command should be used regularly. It must certainly be done after installing apps or devices.
In normal conditions, the OS does not change, and a bootable tape should be created at some frequency.

5. Shutdown a pSeries AIX system in the most secure way:

1. Shut down all applications in a controlled way.
2. Make sure no users are on the system.
3. Use the shutdown command:

shutdown -r        to reboot the system
shutdown -m        to reboot in maintenance mode

6. How to restore specific files from a mksysb tape:

$ tctl fsf 3
$ restore -xvf /dev/rmt0.1 ./your/file/name

For example, if you need to get the vi command back, put the mksysb tape in the tape drive (in this case, /dev/rmt0)
and do the following:

cd /                         # get to the root directory
tctl -f /dev/rmt0 rewind     # rewind the tape
tctl -f /dev/rmt0.1 fsf 3    # move the tape to the third file, no rewind
restore -xqf /dev/rmt0.1 -s 1 ./usr/bin/vi    # extract the vi binary, no rewind

Further explanation why you must use the fsf 3 (fast forward skip file 3):
The format of the tape is as follows:
1. A BOS boot image
2. A BOS install image
3. A dummy Table Of Contents
4. The system backup of the rootvg

So if you just need to restore some files, first forward the tape pointer to position 3, counting from 0.

7. Recovery of rootvg

7.1 Check if the system can boot from tape:
# bootinfo -e

If a 1 is returned, the system can boot from tape, if a 0 is returned a boot from tape is not supported.

7.2 Recover the rootvg:

One possible method is the following:
1. Check whether the tape is in front of the disk with the bootlist command:
# bootlist -m normal -o
2. Insert the mksysb tape
3. Power on the machine. The system will boot from the tape.
4. The Installation and Maintenance Menu will be displayed.

Welcome to Base Operating System
Installation and Maintenance

Type the number of your choice and press Enter.  Choice is indicated by >>>.

>>> 1 Start Install Now with Default Settings

2 Change/Show Installation Settings and Install

3 Start Maintenance Mode for System Recovery

Type 3 and press enter to start maintenance mode.
The next screen you should see is :-

Maintenance

Type the number of your choice and press Enter.

>>> 1 Access a Root Volume Group
2 Copy a System Dump to Removable Media
3 Access Advanced Maintenance Functions
4 Install from a System Backup

>>> Choice [1]:

Type 4 and press enter to install from a system backup.
The next screen you should see is :-

Choose Tape Drive

Type the number of the tape drive containing the system backup to be
installed and press Enter.

Tape Drive                     Path Name

>>> 1 tape/scsi/ost                  /dev/rmt0

>>> Choice [1]:

Type the number that corresponds to the tape drive that the mysysb tape
is in and press enter.
The next screen you should see is :-

Welcome to Base Operating System
Installation and Maintenance

Type the number of your choice and press Enter.  Choice is indicated by >>>.

>>> 1 Start Install Now with Default Settings

2 Change/Show Installation Settings and Install

3 Start Maintenance Mode for System Recovery

+—————————————————–
88  Help ?         |Select 1 or 2 to install from tape device /dev/rmt0
99  Previous Menu  |
|
>>> Choice [1]:

You can now follow your normal mksysb restore procedures.

Leave a comment