I’m sure that like me you do not have a physical access to your Hardware Management Consoles, or even if you have this access, some of your HMC are so far away from your working site (even in a foreign country) that you can’t afford to physically move to this place to update it. Even worse if -like me- you are working in a big place (who says too big ?) this job is often performed by IBM Inspectors and you do not have to worry about your Hardware Management Consoles and just have to ask IBM guys for anything about HMC. For some reasons I had to update an old Hardware Management Console from v7r7.3.0 to v7r7.7.0 SP3. Everybody is confused about the differences between updating an HMC and upgrading an HMC. I know really good bloggers : Anthony English and Rob McNelly have already post about this particular subject but I have to write this post as a reminder and to clarify some points which are not tell in Anthony’s and Rob’s posts :
- Rob’s post : Remote HMC upgrades
- Anthony’s post : Update HMC with sunscreen
To finish this post I’ll talk about a feature nobody is using : the HMC is coming with an Integrated Management Module, this one allows you to have more control and to be autonomous with you HMC.
The difference between updating, upgrading and migrating
There is a lot of confusion when people are trying “update” their HMC. When do I have to update using the updhmc command, when do I have to upgrade using saveupgdata, getupdfiles and chhmc command, and finally when do I have to migrate using HMC Recovery CD/DVD ? All of this three operations are not well described by IBM. Here is what I’m doing in for each case, and this is the result of my own experience (don’t take this as an official document). Here is a little reminder it can be useful for a lot of people : an HMC version number looks like this : v7r7.7.0 SP3. v7 is the VERSION. 7.7.0 is the RELEASE and SP3 is the SERVICE PACK.
- Updating : You have to update your HMC if your are applying a service pack, or a corrective fix update on the HMC, this operation can only be performed by the updhmc command. Use this method if fix central gives you an iso named “HMC_Update_*.iso” or a zip files named “MHxxxxx.zip”. These fixes can be applied to a minor version of the HMC.
- Upgrading : You have to upgrade your HMC if your are moving from one minor version to another (from one release to another), for instance if your are moving from v7r7.7.0 to v7r7.8.0, this operation can be made by using HMC Recovery DVD (fix central will gives you two isos named “HMC_Recovery_*.iso”), or by using the images provided for a network upgrade (I’ll explain this in this post).
- Migrating : You have to migrate your HMC by using HMC Recovery DVD when your are moving from one major version to another. For example when your are moving from an HMC v6 to and HMC v7 (for instance from any v6 version to v7r7.8.0). In this case you have no other choice than burning DVDs and moving in front of the HMC to perform the operation by yourself.
Upgrading
You can upgrade your HMC from its local storage by using the network images provided by IBM on a public FTP server, once connected to the public FTP server get the version you want to upgrade to and download all the files, bzImage and initrd.gz included :
# ftp://ftp.software.ibm.com/software/server/hmc/network/v7770/ # ls FTP Listing of /software/server/hmc/network/v7770/ at ftp.software.ibm.com [..] Feb 26 2013 00:00 2708320 bzImage Feb 26 2013 00:00 808497152 disk1.img Feb 26 2013 00:00 1142493184 disk2.img Feb 26 2013 00:00 1205121024 disk3.img Feb 26 2013 00:00 78 hmcnetworkfiles.sum Feb 26 2013 00:00 34160044 initrd.gz # mget *.*
Put all the files on a server where you have an FTP server running (the HMC getupgfiles is using FTP to get the files) and download all the files with the getupgfiles command directly form the HMC (if your HMC has a direct access to the internet you can specify it the command):
hscroot@gaff:~> getupgfiles -h 192.168.0.99 -u root -d /export/HMC/network_ugrade/v7770 Enter the current password for user root:
While images are downloading the HMC is mounting a temporary filesystem called /hmcdump and put the images in it. Once the images are downloaded the filesystem /hmcdump is unmounted. You can check the download progression with a loop looking on the /hmcdump filesystem :
hscroot@gaff:~> while true ; do date; ls -la /hmcdump; sleep 60; done [..] drwxr-xr-x 3 root root 4096 2013-12-24 16:26 . drwxr-xr-x 30 root root 4096 2013-12-19 14:52 .. -rw-r--r-- 1 root hmc 824223312 2013-12-24 16:32 disk3.img -rw-r--r-- 1 root hmc 78 2013-12-24 16:26 hmcnetworkfiles.sum drwx------ 2 root root 16384 2007-12-19 03:24 lost+found Tue Apr 1 08:10:30 CEST 2014 total 3121248 drwxr-xr-x 3 root root 4096 2013-12-24 16:52 . drwxr-xr-x 30 root root 4096 2013-12-19 14:52 .. -rw-r--r-- 1 root hmc 2708320 2013-12-24 16:52 bzImage -rw-r--r-- 1 root hmc 808497152 2013-12-24 16:52 disk1.img -rw-r--r-- 1 root hmc 1142493184 2013-12-24 16:45 disk2.img -rw-r--r-- 1 root hmc 1205121024 2013-12-24 16:36 disk3.img -rw-r--r-- 1 root hmc 78 2013-12-24 16:26 hmcnetworkfiles.sum -rw-r--r-- 1 root hmc 34160044 2013-12-24 16:52 initrd.gz drwx------ 2 root root 16384 2007-12-19 03:24 lost+found
Please note that this filesystem is only mounted while the getupgfile command is running and can’t be mounted after the command execution … :
hscroot@gaff:~> mount /hmcdump mount: only root can mount /dev/sda6 on /hmcdump
Before launching the upgrade save all the data needed for the upgrade to disk, close all HMC events and clear all the filesystems :
- Save all HMC upgrade data to disk. This command is MANDATORY, it save all the partition profile data, and the user data and the whole HMC configuration, if you forget this command you have to reconfigure the HMC by hand, so be careful with this one
![:-)]()
:
hscroot@gaff:~> saveupgdata -r disk
hscroot@gaff:~> chsvcevent -o closeall
hscroot@gaff:~> chhmcfs -o f -d 0
The images are now downloaded to the HMC, to upgrade the HMC you just have to tell the HMC to boot on its alternate disk and to use the files you’ve just download for the upgrade :
- To set the alternate disk partition on the HMC as a startup device on the next HMC boot and enable the upgrade on the alternate disk use the chhmc command :
hscroot@gaff:~> chhmc -c altdiskboot -s enable --mode upgrade
hscroot@gaff:~> lshmc -r ssh=enable,sshprotocol=,remotewebui=enable,xntp=enable,"xntpserver=127.127.1.0,kronosnet1.fr.net.intra,kronosnet2.fr.net.intra,kronosnet3.fr.net.intra",syslogserver=,netboot=disable,altdiskboot=enable,ldap=enable,kerberos=disable,kerberos_default_realm=,kerberos_realm_kdc=,kerberos_clockskew=,kerberos_ticket_lifetime=,kerberos_keyfile_present=,"sol=disabled "
:hscroot@gaff:~> hmcshutdown -t now -r
Depending on the HMC model and on the version of the HMC the upgrade can takes 10 minutes to 40 minutes, you’ll have to be patient and to cross your finger and pray everything is going well. But don’t worry I never had an issue with this method. Once the HMC is rebooted and upgraded, you can check that the altdiskboot attribute is now set to disable :
hscroot@gaff:~> lshmc -r ssh=enable,sshprotocol=,remotewebui=enable,xntp=enable,"xntpserver=127.127.1.0,kronosnet1.fr.net.intra,kronosnet2.fr.net.intra,kronosnet3.fr.net.intra",syslogserver=,syslogtcpserver=,syslogtlsserver=,netboot=disable,altdiskboot=disable,ldap=enable,kerberos=disable,kerberos_default_realm=,kerberos_realm_kdc=,kerberos_clockskew=,kerberos_ticket_lifetime=,kpasswd_admin=,trace=,kerberos_keyfile_present=,legacyhmccomm=enable,sol=disabled
Updating
Once the HMC is upgraded you have to update it. Unfortunately updates files (often ISO files) are only available on fix central and not on the public FTP. Get the ISO updates file from fix central and put it on your FTP (once again) server, then use the updhmc command to update the HMC, repeat the operation for each updates, and then reboot the HMC (in the example below I’m using sftp) :
hscroot@gaff:~> updhmc -t s -i -h 192.168.0.99 -u root -f /export/HMC/v7r770/HMC_Update_V7R770_SP1.iso Password: iptables: Chain already exists. ip6tables: Chain already exists. [..] The corrective service file was successfully applied. A mandatory reboot is required but was not specified on the command syntax. hscroot@gaff:~> updhmc -t s -i -h 192.168.0.99 -u root -f /export/HMC/v7r770/HMC_Update_V7R770_SP2.iso Password: ip6tables: Chain already exists. ACCEPT tcp opt -- in eth3 out * 0.0.0.0/0 -> 0.0.0.0/0 tcp dpt:5989 ACCEPT udp opt -- in eth3 out * 0.0.0.0/0 -> 0.0.0.0/0 udp dpt:657 [..] The corrective service file was successfully applied. A mandatory reboot is required but was not specified on the command syntax. hscroot@gaff:~> hmcshutdown -t now -r
After upgrading and updating the HMC check the version is ok with the lshmc command :
hscroot@gaff:~> lshmc -V "version= Version: 7 Release: 7.7.0 Service Pack: 3 HMC Build level 20131113.1 ","base_version=V7R7.7.0 "
Using and configuring the Integrated Management Module
I like to be autonomous and do things on my own. Who had never been stuck on a problem with an HMC and was forced to called an IBM inspector to reboot the HMC or even to insert a CD in the CDRom reader. A few people know this but the HMC is based on an IBM Xserie server (Who said Lenovo ?) and is shipped with an Integrated Management Module allowing you to boot, start, and stop the HMC without the need to have someone in the data-center. Unfortunately this method seems not to be supported by IBM so do it at your own risk.
Use the dedicated port for the Integrated Management Console (the Red port)
From the HMC command line using the chhmc command configure the Integrated Management Module IP address :
hscroot@gaff:~> chhmc -c imm -s modify -a 10.10.20.4 -nm 255.255.255.0 -g 10.10.20.254
Restart the Integrated Management Module to commit the changes. The IMM will not be pingable before restart :
hscroot@gaff:~> chhmc -c imm -s restart
The Integrated Management Module is now pingable and you can check its configuration :
hscroot@gaff:~> lshmc -i ipv4addr=10.10.20.4,networkmask=255.255.255.0,gateway=10.10.20.254,username=USERID,mode=Dedicated
By default the username is USERID and the password is PASSW0RD (with a zero), you can change it to fit your needs :
hscroot@gaff:~> chhmc -c imm -s modify -u immusername --passwd "abc123"
The Integrated Management Module is now configured and can be accessed from the web interface of from SSH :
I will not detail all the actions you can do with the Integrated Management Module but here is a screen showing the Hardware Health of the HMC :
One thing you can do for free (without IMM license) is to control the Power of the HMC, choosing to stop/start/restart or reboot. This feature can be very useful when the HMC is stucked :
If you choose to restart the HMC the Integrated Management Module will warn you before restarting :
You can access the HMC Integrated Management Module by using the SSH command line :
- Use the power command to control the power of the HMC :
system> help power usage: power on [-options] - power on server power off [-options] - power off server power cycle [-options] - power off, then on power state - display power state power -rp [alwayson|alwaysoff|restore] - host power restore policy options: -s - shut down OS first -every day - daily or weekly on,off or cycle commands [Sun|Mon|Tue|Wed|Thu|Fri|Sat|Day|clear] -t time - time (hh:mm) additional options for on. -d date - date (mm/dd/yyyy) -clear - clear on date
system> power cycle -s ok
system> power state power on State:Booting OS or in unsupported OS
The Integrated Management Module is a licensed product and unfortunately IBM does not support the Integrated Management Module on the HMC.It seems that the IMM license can’t be acquired for the HMC. I have checked on the trial licenses page and the HMC Hardware does not even exists when you have to choose the Hardware model for the trial license. This a shame because the licensed IMM allows to remote control the HMC, and to manage Virtual CDrom ….. useful for migration. So if an IBMer is reading this and have an explanation about this feel free to tell me what I’ve missed in the comments :
I hope this post will let you manage your HMC alone and to be autonomous