If you have issues getting to the computer room easily, and you have to update an HMC on the raised floor, then you can also do that upgrade remotely. IBM describes two methods on their website: by using the update media and using the recoverable media. Using the update media method, you may end up with a corrupted HMC. The only way to solve this, us accessing the HMC in the computer room (*sigh*).
Therefore, use the recoverable media option. That one works better. A link to the documentation and software can be found here.
If your AIX server level is below 5.3 TL06, the easiest way is just to upgrade your current OS to TL 06 at minimum (take note it will depend of configurations for Power6 processors) then clone your server and install it on the new p6.
But if you want to avoid an outage on your account, you can do the next using a NIM server (this is not official procedure for IBM, so they do not support this):
- Create your mksysb resource and do not create a spot from mksysb.
- Create an lppsource and spot with minimum TL required (I used TL08).
- Once you do nim_bosinst, choose the mksysb, and the created spot. It will send a warning message about spot is not at same level as mksysb, just ignore it.
- Do all necessary to boot from NIM.
- Once restoring the mksysb, there's some point where it is not able to create the bootlist because it detects the OS level is not supported on p6. So It will ask to continue and fix it later via SMS or fix it right now.
- Choose to fix it right now (it will open a shell). You will notice oslevel is as the same as mksysb.
- Create a NFS from NIM server or another server where you have the necessary TL and mount it on the p6.
- Proceed to do an upgrade, change the bootlist, exit the shell. Server will boot with new TL over the p6.
Let's say you want to know what process is tying up port 25000:
# netstat -aAn | grep 25000
f100060020cf1398 tcp4 0 0 *.25000 *.* LISTEN
f10006000d490c08 stream 0 0 f1df487f8 0 0 0 /tmp/.sapicm25000
So, now let's see what the process is:
# rmsock f100060020cf1398 tcpcb
The socket 0x20cf1008 is being held by proccess 1806748 (icman).
If you have lsof installed, you can get the same result with the lsof command:
# lsof -i :[PORT]
Example:
# lsof -i :5710
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
oracle 2638066 oracle 18u IPv4 0xf1b3f398 0t1716253 TCP host:5710
The "Integrated Virtual Ethernet" or IVE adapter is an adapter directly on the GX+ bus, and thus up to 3 times faster dan a regular PCI card. You can order Power6 frames with different kinds of IVE adapters, up to 10GB ports.
The IVE adapter acts as a layer-2 switch. You can create port groups. In each port group up to 16 logical ports can be defined. Every port group requires at least 1 physical port (but 2 is also possible). Each logical port can have a MAC address assigned. These MAC addresses are located in the VPD chip of the IVE. When you replace an IVE adapters, LPARS will get new new MAC addresses.
Each LPAR can only use 1 logical port per physical port. Different LPARs that use logical ports from the same port group can communicate without any external hardware needed, and thus communicate very fast.
The IVE is not hot-swappable. It can and may only be replaced by certified IBM service personnel.
First you need to configure an HAE adapter; not in promiscues mode, because that is meant to be used if you wish to assign a physical port dedicated to an LPAR. After that, you need to assign a LHAE (logical host ethernet adapter) to an LPAR. The HAE needs to be configured, and the frame needs to be restarted, in order to function correctly (because of the setting of multi-core scaling on the HAE itself).
So, to conclude: You can assign physical ports of the IVE adapter to separate LPARS (promiscues mode). If you have an IVE with two ports, up to two LPARS can use these ports. But you can also configure it as an HAE and have up to 16 LPARS per physical port in a port group using the same interface (10Gb ports are recommended). There are different kinds of IVE adapters; some allow to create more port groups and thus more network connectivity. The IVE is a method of virtualizing ethernet without the need for VIOS.
When you encounter an issue where ssh through a firewall works perfectly, but when doing scp of large files (for example mksysb images) the scp connection stalls, then there's a solution to this problem: Add "-l 8192" to the scp command.
The reason for scp to stall, is because scp greedily grabs as much bandwith of the network as possible when it transfers files, any delay caused by the network switch of the firewall can easily make the TCP connection stalled.
Adding the option "-l 8192" limits the scp session bandwith up to 8192 Kbit/second, which seems to work safe and fast enough (up to 1 MB/second):
# scp -l 8192 SOURCE DESTINATION
An adapter that has previously been added to a LPAR and now needs to be removed, usually doesn't want to be removed from the LPAR, because it is in use by the LPAR. Here's how you find and remove the involved devices on the LPAR:
First, run:
# lsslot -c pci
This will find the adapter involved.
Then, find the parent device of a slot, by running:
# lsdev -Cl [adapter] -F parent
(Fill in the correct adapter, e.g. fcs0).
Now, remove the parent device and all its children:
# rmdev -Rl [parentdevice] -d
For example:
# rmdev -Rl pci8 -d
Now you should be able to remove the adapter via the HMC from the LPAR.
If you need to replace the adapter because it is broken and needs to be replaced, then you need to power down the PCI slot in which the adapter is placed:
After issuing the "rmdev" command, run
diag and go into "Task Selection", "Hot Plug Task", "PCI Hot Plug Manager", "Replace/Remove a PCI Hot Plug Adapter". Select the adapter and choose "remove".
After the adapter has been replaced (usually by an IBM technician), run
cfgmgr again to make the adapter known to the LPAR.
It is possible to stop and start an LPAR from the HMC prompt:
# lssycfg -r lpar
This command will list all partitions known to this HMC.
# chsysstate -o osshutdown -r lpar -n [partition name]
This command will send a shutdown OS command to the lpar.
# chsysstate -o on -r lpar -n [partition name]
This command will activate the partition.
# lsrefcode -r lpar -F lpar_name,refcode
This command will show the LED code.
The AIX kernel has an "enter_dbg" variable in it that can be set at the beginning of the boot processing which will cause all boot process output to be sent to the system console. In some cases, this data can be useful in debugging boot issues. The procedure for setting the boot debugger is as follows:
First: Preparing the system.
Set up KDB to present an initial debugger screen
# bosboot -ad /dev/ipldevice -I
Reboot the server:
# shutdown -Fr
Setting up for Kernel boot trace:
When the debugger screen appears, set
enter_dbg to the value we want to use:
************* Welcome to KDB *************
Call gimmeabreak...
Static breakpoint:
.gimmeabreak+000000 tweq r8,r8 r8=0000000A
.gimmeabreak+000004 blr
<.kdb_init+0002C0> r3=0
KDB(0)> mw enter_dbg
enter_dbg+000000: 00000000 = 42
xmdbg+000000: 00000000 = .
KDB(0)> g
Now, detailed boot output will be displayed on the console.
If your system completes booting, you will want to turn
enter_dbg off:
************* Welcome to KDB *************
Call gimmeabreak...
Static breakpoint:
.gimmeabreak+000000 tweq r8,r8 r8=0000000A
.gimmeabreak+000004 blr
<.kdb_init+0002C0> r3=0
KDB(0)> mw enter_dbg
enter_dbg+000000: 00000042 = 0
xmdbg+000000: 00000000 = .
KDB(0)> g
When finished using the boot debugger, disable it by running:
# bosdebug -o
# bosboot -ad /dev/ipldevice
It is very easy to clone your rootvg to another disk, for example for testing purposes. For example: If you wish to install a piece of software, without modifying the current rootvg, you can clone a rootvg disk to a new disk; start your system from that disk and do the installation there. If it succeeds, you can keep using this new rootvg disk; If it doesn't, you can revert back to the old rootvg disk, like nothing ever happened.
First, make sure every logical volume in the rootvg has a name that consists of 11 characters or less (if not, the alt_disk_copy command will fail).
To create a copy on hdisk1, type:
alt_disk_copy -d hdisk1
If you now restart your system from hdisk1, you will notice, that the original rootvg has been renamed to old_rootvg. To delete this volume group (in case you're satisfied with the new rootvg), type:
# alt_rootvg_op -X old_rootvg
A very good article about alternate disk installs can be found on
developerWorks.
If you wish to copy a mirrored rootvg to two other disks, make sure to use quotes around the target disks, e.g. if you wish to create a copy on disks hdisk4 and hdisk5, run:
# alt_disk_copy -d "hdisk4 hdisk5"
A very easy way to see what was installed recently on your system:
# lslpp -h
Number of results found: 469.
Displaying results: 251 - 260.