Topics: Hardware, Installation, System Admin

Automating microcode discovery

You can run invscout to do a microcode discovery on your system, that will generate a hostname.mup file. Then you go upload this hostname.mup file at this page on the IBM website and you get a nice overview of the status of all firmware on your system.

So far, so good. What if you have plenty of systems and you want to automate this? Here's a script to do this. This script first does a webget to collect the latest catalog.mic file from the IBM website. Then it distributes this catalog file to all the hosts you want to check. Then, it runs invscout on all these hosts, and collects the hostname.mup files. It will concatenate all these files into 1 large file and do an HTTP POST through curl to upload the file to the IBM website and have a report generated from it.

So, what do you need?

  • You should have an AIX jump server that allows you to access the other hosts as user root through SSH. So you should have setup your SSH keys for user root.
  • This jump server must have access to the Internet.
  • You need to have wget and curl installed. Get it from the Linux Toolbox.
  • Your servers should be AIX 5 or higher. It doesn't really work with AIX 4.
  • Optional: a web server, like Apache 2, would be nice, so you can drop the resulting HTML file on your website every day.
  • An entry in the root crontab to run this script every day.
  • A list of servers you want to check.
Here's the script:
#!/bin/ksh

# script:  generate_survey.ksh
# purpose: To generate a microcode survey html file

# where is my list of servers located?
SERVERS=/usr/local/etc/servers

# what temporary folder will I use?
TEMP=/tmp/mup

# what is the invscout folder
INV=/var/adm/invscout

# what is the catalog.mic file location for invscout?
MIC=${INV}/microcode/catalog.mic

# if you have a webserver,
# where shall I put a copy of survey.html?
APA=/usr/local/apache2/htdocs

# who's the sender of the email?
FROM=microcode_survey@ibm.com

# who's the receiver of the email?
TO="your.email@address.com"

# what's the title of the email?
SUBJ="Microcode Survey"

# user check
USER=`whoami`
if [ "$USER" != "root" ];
then
    echo "Only root can run this script."
    exit 1;
fi

# create a temporary directory
rm -rf $TEMP 2>/dev/null
mkdir $TEMP 2>/dev/null
cd $TEMP

# get the latest catalog.mic file from IBM
# you need to have wget installed 
# and accessible in $PATH
# you can download this on:
# www-03.ibm.com
# /systems/power/software/aix/linux/toolbox/download.html
wget techsupport.services.ibm.com/server/mdownload/catalog.mic
# You could also use curl here, e.g.:
#curl techsupport.services.ibm.com/server/mdownload/catalog.mic -LO

# move the catalog.mic file to this servers invscout directory
mv $TEMP/catalog.mic $MIC

# remove any old mup files
echo Remove any old mup files from hosts.
for server in `cat $SERVERS` ; do
   echo "${server}"
   ssh $server "rm -f $INV/*.mup"
done

# distribute this file to all other hosts
for server in `cat $SERVERS` ; do
   echo "${server}"
   scp -p $MIC $server:$MIC
done

# run invscout on all these hosts
# this will create a hostname.mup file
for server in `cat $SERVERS` ; do
   echo "${server}"
   ssh $server invscout
done

# collect the hostname.mup files
for server in `cat $SERVERS` ; do
   echo "${server}"
   scp -p $server:$INV/*.mup $TEMP
done

# concatenate all hostname.mup files to one file
cat ${TEMP}/*mup > ${TEMP}/muppet.$$

# delete all the hostname.mup files
rm $TEMP/*mup

# upload the remaining file to IBM.
# you need to have curl installed for this
# you can download this on:
# www-03.ibm.com
# /systems/power/software/aix/linux/toolbox/download.html
# you can install it like this:
# rpm -ihv 
#    curl-7.9.3-2.aix4.3.ppc.rpm curl-devel-7.9.3-2.aix4.3.ppc.rpm
# more info on using curl can be found on: 
# http://curl.haxx.se/docs/httpscripting.html
# more info on uploading survey files can be found on:
# www14.software.ibm.com/webapp/set2/mds/fetch?pop=progUpload.html

# Sometimes, the IBM website will respond with an
# "Expectation Failed" error message. Loop the curl command until
# we get valid output.

stop="false"

while [ $stop = "false" ] ; do

curl -H Expect: -F mdsData=@${TEMP}/muppet.$$ -F sendfile="Upload file" \ 
   http://www14.software.ibm.com/webapp/set2/mds/mds \
   > ${TEMP}/survey.html

#
# Test if we see Expectation Failed in the output
#

unset mytest
mytest=`grep "Expectation Failed" ${TEMP}/survey.html`

if [ -z "${mytest}" ] ; then
        stop="true"
fi

sleep 10

done

# now it is very useful to have an apache2 webserver running
# so you can access the survey file
mv $TEMP/survey.html $APA

# tip: put in the crontab daily like this:
# 45 9 * * * /usr/local/sbin/generate_survey.ksh 1>/dev/null 2>&1

# mail the output
# need to make sure this is sent in html format
cat - ${APA}/survey.html <<HERE | sendmail -oi -t
From: ${FROM}
To: ${TO}
Subject: ${SUBJ}
Mime-Version: 1.0
Content-type: text/html
Content-transfer-encoding: 8bit

HERE

# clean up the mess
cd /tmp
rm -rf $TEMP




If you found this useful, here's more on the same topic(s) in our blog:


UNIX Health Check delivers software to scan Linux and AIX systems for potential issues. Run our software on your system, and receive a report in just a few minutes. UNIX Health Check is an automated check list. It will report on perfomance, capacity, stability and security issues. It will alert on configurations that can be improved per best practices, or items that should be improved per audit guidelines. A report will be generated in the format you wish, and the report includes the issues discovered and information on how to solve the issues as well.

Interested in learning more?