UNIX Health Check documentation

Index
  1. Obtaining Support
  2. Using UNIX Health Check
  3. Reports generated by UNIX Health Check
  4. Versions of UNIX Health Check
  5. Installing UNIX Health Check
  6. Determining the version of UNIX Health Check
  7. Running checks individually
  8. Using the checkall master script
  9. Using options with the checkall master script
  10. Return codes
  11. Running categories of scripts
  12. Automating the download of UNIX Health Check
  13. Enabling password-less access through SSH on cluster nodes
  14. Resolving email issues
  15. Resolving issues found by UNIX Health Check

↑
1. Obtaining Support
Getting support for UNIX Health Check is easy: When you ask for support, please include the following information:
  • Your contact information.
  • A description of the issue you've encountered.
  • If appropriate, the output file generated by UNIX Health Check that includes the issue.
For more information, visit our support page:
↑
2. Using UNIX Health Check
We provide UNIX Health Check software to scan UNIX systems. When we talk about UNIX systems, we mean Red Hat Enterprise Linux based systems, and also AIX. It is like going to a doctor for a full check-up. You'll get a report of all features of your system, and how the system is doing. If the system is properly installed, configured and maintained, an all green report with a score of 100% will be given. If for some reason some things are off, we'll let you know in the report what is wrong, and how you can fix it.

The goal should be to get the system(s) on a score of 95% or higher. There is no need to aim for a 100% score. After all, there are always reasons why a certain system is configured (slightly) differently than how best practices indicate how a system should be configured.

UNIX Health Check can be beneficial in several stages of a system's lifecycle. Most of our customers simply run the entire health check daily, to provide them with a report of their server status. But doing a UNIX Health Check is also very useful at certain other important points in the life cycle of a UNIX system:
  • Before putting a new system into production - to make sure the system was installed and configured correctly.
  • Before and after doing a change on a server - to make sure no issues exist before applying patches/changes, and that no adverse situations were introduced as part of a change.
  • Before or during audits - to make sure a system is compliant with company and federal regulations.
  • During yearly performance reviews - to determine if the system administrators have taken good care of the servers.
  • Before or during any Bare Metal Restore or Disaster Recovery exercises - to validate that a server has been recovered correctly.
  • When doing a security review of a system - ensuring your system is protected against outside and inside vulnerabilities, such as hackers of malicious software.
  • When taking over support of systems from others or other accounts to perform an initial system scan.
    • ... and of course:
  • As part of the daily routine to check and monitor systems, because not daily reviewing systems, can have costly consequences.

↑
3. Reports generated by UNIX Health Check
Reports can be generated by UNIX Health Check in different formats. By default, it will generate a report in TEXT format (also known as "log" format). You can also select to have a report generated in web page (HTML) format, or comma separated (CSV) format, or in Extended Markup Language (XML) format. Use one of these options to select a specific format:

Option
Extension
Description
-l
.log
TEXT format (log) (default)
-h
.html
Web page format (HTML)
-c
.csv
Comma separated format (CSV)
-x
.xml
Extended Markup Language format (XML)

If you combine any of the options above, the order of the options used determines which format is used to generate the report. The option that is mentioned last, will determine the format. For example, if you select options "-l", "-x" and "-h", the format of the report is HTML, because the "-h" option (for HTML output) was mentioned last.

A report generated by UNIX Health Check consists of multiple parts:

  1. The UNIX Health Check copyright message.

    A sample of the copyright message.

    This copyright message states that the use of UNIX Health Check is subject to license agreements, along with displaying information on how to make changes based on issues discovered by UNIX Health Check, and information about how to get additional support. You can suppress displaying this information by using the "-b" option.

  2. An overview section of the options selected when running UNIX Health Check.

    A sample of the section that shows the options selected.

    This section will display the options selected when UNIX Health Check was run; when the report was created; what version of UNIX Health Check was used; the name of the output file and the number of check scripts that were run. You can suppress this output by using the "-b" option.

  3. A system configuration section.

    A sample of the system configuration section.

    This is a short overview of many configuration items of the server, which includes items like the serial number of the server, the amount of memory, the CPU clock rate, and the OS level installed. The system configuration section is not displayed when the "-b" option is used. The system configuration section looks different depending on the version of UNIX Health Check used (either the AIX version or the Red Hat Enterprise Linux version).

  4. The output of individual check scripts.


    Two samples of the output of individual check scripts.

    In this part of the report you'll see the output of individual check scripts. For each check script, it will show the name of the check script that was run, a description of the check script (if the "-d" option was used; the "-d" option adds descriptions of check scripts), the output of the check script, and if applicable, any errors that were generated when the check script was run.

    At the end, you'll also see the return code of the script (More information about return codes in Return codes), along with the amount of time in seconds that it took to run the check script. If the "-g" option was used, then only the output of individual check scripts is shown in this section, when any of the check scripts returns a non-zero return code (meaning either an ERROR or WARNING status occurred, and ignoring any successful check scripts).

    In the HTML output version of UNIX Health Check different colors are used to indicate the result for each individual check script. Check scripts that completed successfully are listed as green. Check scripts that ended with a WARNING status (return code 2), are listed in yellow. And check scripts that have ended with an error, are listed as red. This helps to identify the severity level of each individual check script.

  5. A results summary.

    A sample of the results section.

    This section displays the total run time for all check scripts, the number of check scripts that were run, the number of checks that returned an OK (return code 0), WARNING (return code 1) and ERROR (return code 2), and a total score for the system. It will finally point to the location of the output file that was generated by UNIX Health Check.

↑
4. Versions of UNIX Health Check
Two versions of UNIX Health Check are available:
  • UNIX Health Check for AIX
    This version can be used on AIX systems only. This includes VIOS (Virtual I/O Servers) and WPARs (Workload Partitions).
  • UNIX Health Check for Red Hat Enterprise Linux
    This version can be used on Red Hat Enterprise Linux, and its derivatives, CentOS, CentOS Stream, AlmaLinux, Rocky Linux, Oracle Linux and Scientific Linux.
The default shell used in AIX is the Korn Shell, and to indicate that, all the check scripts in the AIX version of UNIX Health Check use extension ".ksh". The default shell in Red Hat Enterprise Linux and all its derivatives is the Bash shell, and to indicate that, all the check scripts in the Red Hat Enterprise Linux version of UNIX Health Check use extension ".sh". The Korn shell style check scripts of UNIX Health Check cannot be used on Red Hat Enterprise Linux (and its derivatives such as CentOS, CentOS Stream, AlmaLinux, Rocky Linux, Oracle Linux and Scientific Linux). The Bash shell style scripts of UNIX Health Check for Red Hat Enterprise Linux cannot be used on AIX.

When downloading UNIX Health Check software, you have several options:

First, decide the format of the UNIX Health Check software you wish to use. The software is available in two formats:
  • As a TAR image; which is a regular TAR file that you can un-pack in any folder on your system.
  • As a RPM package, which can be used to install using the "rpm" command.
The software is exactly the same in either format, so you only have to download one of the two formats.

If you have licensed our software for both AIX and Red Hat Enterprise Linux, you will find a TAR image file for both versions, as well as a RPM package for both versions on our download page.

When using the TAR image file(s), the name of the TAR image file is as follows:
  • ahc_<date>.tar

    This file is for AIX. AHC is short for AIX Health Check. This version can be used both on AIX and VIOS systems. And it can also be used on WPARs. It can NOT be used on Red Hat Enterprise Linux (or similar) systems.

  • rhc_<date>.tar

    This file is for Red Hat Enterprise Linux, CentOS, CentOS Stream, AlmaLinux, Rocky Linux, Oracle Linux and Scientific Linux. RHC is short for Red Hat Health Check. This TAR image can NOT be used on AIX based systems.

When using the RPM package file(s), the name of the RPM package file is as follows:
  • uhc-<date>-1.aix.noarch.rpm.tar

    This RPM file is for AIX. UHC is short for UNIX Health Check. The OS release (aix) is indicated in the RPM file name. This version can be used both on AIX and VIOS systems. And it can also be used on WPARs. It can NOT be used on Red Hat Enterprise Linux (or similar) systems.

  • uhc-<date>-1.rhel.noarch.rpm.tar

    This file is for Red Hat Enterprise Linux, CentOS, CentOS Stream, AlmaLinux, Rocky Linux, Oracle Linux and Scientific Linux. The OS release (rhel, short for Red Hat Enterprise Linux) is indicated in the RPM file name. This TAR image can NOT be used on AIX based systems.

UNIX Health Check software will be provided to you once a purchase order has been received, or when you've completed a UNIX Health Check license purchase online. The software will be made available to you through download from our website. As soon as we've received a purchase order or your online payment, we'll send you a login and password, so you can download the software, and get started right away.

The software can be downloaded from our download page: On the download page, download the appropriate version(s) for your system(s), and transfer the UNIX Health Check TAR image file or RPM package file onto the system you wish to run it on.

The download page may look similar like this:

A sample of the download page.


You may see a section for AIX, for Red Hat Enterprise Linux, or both, depending on the versions of UNIX Health Check that you have licensed.

In this download section, you will find a link to download the file, the size of the file, the check sum (Sum) of the file, a link to a file that contains the SHA-512 hash of the file, and for RPM packages, the GPG key that was used to digitally sign the RPM package. You can use that information to verify the correct download of UNIX Health Check software. More on using this information later, in the next chapter.

↑
5. Installing UNIX Health Check
This chapter describes the installation of UNIX Health Check software.

5a. Verifying the integrity of downloaded files

Once you've downloaded UNIX Health Check software, it is best practice to verify that you've downloaded the correct file and that it has not been tampered with. In the previous chapter you've seen an example of what the download section may look like when downloading UNIX Health Check software. Various information is provided on the download page to assist in the verification of the correctness of downloaded files:
  • The file size: The file size indicated on the download page has to match exactly to the file you have downloaded. Run the "ls" command on the file, to check the file size. For example: 

    # cd /uhc
# ls -als ahc_latest.tar
6712 -rw-r--r-- 1 root system 6871040 Apr 26 13:53 ahc_latest.tar
    This is just an example. When running the "ls" command, please use the filename, and the matching file size as indicated on the download page, to verify that the downloaded file has the correct size. Also, please do not use the picture of the download section shown in chapter 4, as this is just an example, and file sizes may change on a daily basis. Please use the actual information from your download page instead to verify the correct file size.
  • The check sum (Sum) of the file: The check sum of the UNIX Health Check software file can be determined by using the "sum" command (both available on AIX and Red Hat Enterprise Linux). For example: 
    # cd /uhc
# sum ahc_latest.tar
46389  6760 ahc_latest.tar
    The first number indicated in the output of the "sum" command (in the example above listed as 46389) is the checksum of the file. The second number (6760 in the example above) indicates the block count).

    Please note that a similar command, "cksum", is also available, but this command will generate different check sums for files, and should not be used for verifying the downloaded software files of UNIX Health Check.
  • The SHA-512 hash: The SHA-512 hash is by far the best method to confirm the correctness of the package. On the download page you will find links to text files for each available file (both TAR image files and RPM package files), that contain the SHA-512 hash of each of the UNIX Health Check software files. To calculate the SHA-512 has yourself, use the "sha512sum" command on Red Hat Enterprise Linux: 
    # cd /uhc
# sha512sum rhc_2019-03-26.tar
5a1eeaf65843032d3f50e2e54dc5958b544df...74766a2c24a459de8a8d4  rhc_2019-03-26.tar
    The first long string contains the SHA-512 hash which can be compared with the text file found in the download section.

    On AIX, use the "shasum" command to verify the MD5 hash, for example: 
    # cd /uhc
# shasum -a 512 ahc_2019-03-26.tar
2e49f14435f71d9a2aa0bd7045b9e66dfb7db...0c44581c707f86b15dca9  ahc_2019-03-26.tar
  • The GPG key: This applies to RPM package files only. When we create RPM packages, we will digitally sign them with our GPG key. You can use the GPG key file (UHC-GPG-KEY) provided on the download page to verify the signature. More on that later in this documentation.
For Windows systems, Microsoft provides the Microsoft File Checksum Integrity Verifier tool, available at https://www.microsoft.com/en-us/download/details.aspx?id=11533. This tool may also be used to validate the integrity of downloaded files from the UNIX Health Check website.

5b. Installing UNIX Health Check using a TAR file

If you've chosen to use the TAR image file as the download of your choice, then follow the procedure in this paragraph to install the software.

When using the TAR image file, it is recommended to un-pack the file in a separate folder. Any folder will do as long as it's an empty folder; you can transfer the TAR image file into the folder of your preference and un-pack it there. The example below assumes that you will be installing the software in folder "/uhc". We recommend that you create a separate file system of at least 256 MB, and use that file system for storing the UNIX Health Check software and any reports generated by it.

The TAR image file can either be the TAR image file for AIX (starting with "ahc_") or for Red Hat Enterprise Linux (starting with "rhc_"). In the procedure below, the filename "ahc_latest.tar" is used. Please exchange this name for the correct TAR image file name for your system.

Create the installation folder. In the procedure below, folder "/uhc" is used, but any folder you wish can be used. Just make sure to use an empty folder. 
# mkdir /uhc
# cd /uhc
Move the UNIX Health Check TAR image file into the folder (assuming you have placed the TAR image file in the /tmp folder on the system when you've downloaded the file): 
# mv /tmp/ahc_latest.tar /uhc
Next, un-pack the TAR image file: 
# tar -xvf ahc_latest.tar
x checkactivatedrpcservices.ksh, 1625 bytes, 4 media blocks.
x checkadaptersdefined.ksh, 297 bytes, 1 media blocks.
x checkadapters.ksh, 319 bytes, 1 media blocks.
x checkaiooa.ksh, 334 bytes, 1 media blocks.
x checkaiostatus.ksh, 1765 bytes, 4 media blocks.
x checkall.ksh, 26286 bytes, 52 media blocks.
x checkaudit.ksh, 235 bytes, 1 media blocks.
...
...
[lines removed]
...
...
x checkxntpd.ksh, 2557 bytes, 5 media blocks.
x checkzombies.ksh, 407 bytes, 1 media blocks.
x COPYRIGHT, 930 bytes, 2 media blocks.
x DESCRIPTIONS, 124444 bytes, 244 media blocks.
When upgrading to a new version of UNIX Health Check, empty the folder that the UNIX Health Check TAR image file was un-packed into, and then un-pack the new version into the same folder: 
# cd /uhc
# rm -rf /uhc/*
# mv /tmp/ahc_latest.tar /uhc
# tar -xvf ahc_latest.tar
When the ahc_latest.tar or rhc_latest.tar image file has been un-packed, you can safely remove the tar image: 
# cd /uhc
# rm -f *.tar
To query the currently installed version of the software, look at the VERSION file: 
# cat /uhc/VERSION
19.03.26
To remove the software, simply remove the folder that the UNIX Health Check software was installed into: 
# rm -rf /uhc


5c. Installing UNIX Health Check using an RPM package

If you wish to use the RPM package file to install UNIX Health Check on your system, then use the procedure in this paragrapht.

You can use the "rpm" command on both AIX and Red Hat Enterprise Linux to install, remove or upgrade UNIX Health Check software. The default install location is folder, when using the RPM package file, is "/opt/uhc".

To install, run the "rpm" command using the "-ihv" options (the "-i" option stands for "install"): 
# rpm -ihv uhc-2019.03.25-1.rhel.noarch.rpm
Preparing...                          ################################# [100%]
Updating / installing...
   1:uhc-2019.03.25-1.rhel            ################################# [100%]
To upgrade to a newer version, run the "rpm" command using the "-Uhv" options (the "-U" option stands for "upgrade"): 
# rpm -Uhv uhc-2019.03.26-1.rhel.noarch.rpm
Preparing...                          ################################# [100%]
Updating / installing...
   1:uhc-2019.03.26-1.rhel            ################################# [ 50%]
Cleaning up / removing...
   2:uhc-2019.03.25-1.rhel            ################################# [100%]
To query the currently installed version of the UNIX Health Check software, use the "rpm" command using the "-q" (for "query") option: 
# rpm -q uhc
uhc-2019.03.26-1.rhel.noarch
To un-install the software, run the "rpm" command using the "-e" (for "erase") option: 
# rpm -e uhc
When you install the RPM package, you may see a NOKEY error. This means that the "rpm" command was unable to verify the digital signature for the RPM package file. To avoid seeing this error, import the GPG key first: 
# cd /tmp
# wget https://unixhealthcheck.com/UHC-GPG-KEY
# rpm --import /tmp/UHC-GPG-KEY
By importing the key first, the GPG key is used by the "rpm" command to verify the digital signature of the RPM package file, and when it matches, no error is displayed.

5d. Additional comments on installing UNIX Health Check

We recommend that you run UNIX Health Check initially on one of your test servers, before running it on any production server, even though it is perfectly safe to run UNIX Health Check on production systems.

All the commands shown in this documentation are to be run as user root; UNIX Health Check requires root, and no support is available if UNIX Health Check is run using any other user account.

UNIX Health Check requires 64 MB to be free in the /var and /tmp file systems. If that space is not available, either try clearing out these file systems by deleting old files, or you may try extending the file systems.

UNIX Health Check for AIX requires that the ncargs setting on the AIX system is set to 256 or higher. This setting determines the maximum number of command line arguments that can be used on the system, and due to the large number of check scripts available in UNIX Health Check for AIX, this value should be 256 or higher. The default value for ncargs in AIX 6 and AIX 7 is 256. On older AIX versions, you may encounter that it is still set to 6. Also, on some older AIX versions, ncargs can't be configured higher than 128.

To check the current ncargs setting, run:
# lsattr -El sys0 -a ncargs
To determine the maximum value ncargs can be configured to, run:
# lsattr -R -l sys0 -a ncargs
Set ncargs to either 256, or if 256 is not supported, set it to the highest available value, such as 128, by running:
# chdev -l sys0 -a ncargs=256


5e. Files in the installation folder

Once the TAR image file is unpacked, or the RPM package installed, you will find many check scripts in the installation folder, along with the master script. The master script is called "checkall.ksh" on AIX and "checkall.sh" on Red Hat Enterprise Linux. Note the difference in filename extenstion as indicated earlier in this documenation, based on the Operating System that is being used. You will also find some other files like CHANGES, COPYRIGHT, COUNT, DOCUMENTATION, FAQ, SUPPORT, VERSION, DESCRIPTIONS, LICENSE, and perhaps a few more, all in uppercase names.

Here's an overview of all the files included:

File checkall.ksh or checkall.sh is the script that can be used to run more than one check script, and is referred to as the master script. You will generally use it to run all the check scripts, or groups of check scripts. Throughout this documentation, the master check script is either referred to as checkall, checkall.ksh (for AIX) or checkall.sh (for Red Hat Enterprise Linux, CentOS, CentOS Stream, AlmaLinux, Rocky Linux, Oracle Linux and Scientific Linux). Please ensure that you use the correct version, depending on the Operating System used.

Files that start with check and end with sh (except for checkall.ksh or checkall.sh) are the actual check scripts that are run by UNIX Health Check. You should find several hundreds of those, or even more than a thousand, depending on the version downloaded. Each check script checks a single function or a single item on the system. You can run each check script individually if you like, but it is more common and less time-consuming to run all or groups of check scripts through the checkall master script.

No check script will ever change anything on your system. It is safe to run them at any time, without affecting the system in any way. However, due to the nature of the commands run by some check scripts, it may be possible that some harmless errors may occur in log files or in the case of AIX, in the error report (for example when unavailable disks are attempted to be accessed by a check script). Please do not be alarmed by this; it is normal behavior. Again, it's perfectly safe to run any script on any system at any time without affecting the system.

File APARS is a file containing all high severity APARs available for AIX, as released by IBM. The source of this file is http://www-304.ibm.com/webapp/set2/flrt/doc?page=aparCSV and is used by UNIX Health Check for AIX to check for the correct APARs installed on the AIX system. This file is not included on the UNIX Health Check version for Red Hat Enterprise Linux.

File CHANGES is an overview of the most recent changes to UNIX Health Check. You can review it to determine what was changed recently in the UNIX Health Check software.

File COPYRIGHT is the copyright message that is part of UNIX Health Check.

File COUNT shows the number of checks that is available in the full version of UNIX Health Check.

File DEMO provides additional information about the demo version of UNIX Health Check. This file is only present if the demo version of UNIX Health Check is being used. The demo version can be found at www.unixhealthcheck.com/demo.

File DESCRIPTIONS contains a description for each check script available in the UNIX Health Check software package. It will provide you information of what each check script does. Additionally, each check script has a description in the header of the file. By simply viewing a check script in an editor such as VI, you will be able to see what the check script is checking. When the master check script checkall is used, UNIX Health Check will include the description for each check in the report, if the "-d" option is used; we'll discuss that later in this documentation.

File DOCUMENTATION provides a link to the UNIX Health Check documentation (this web page), so you will never have to search for the latest documentation for UNIX Health Check.

File FAQ provides a link to our frequently asked questions, also on the UNIX Health Check website.

File LICENSE contains the license information of UNIX Health Check.

File SUPPORT provides a link to our support page, in case you need additional support when using UNIX Health Check.

File TOOLBOX provides a list of all the available RPM packages and their latest versions, as distilled from the AIX Toolbox for Linux Applications on the IBM website at https://www.ibm.com/developerworks/aix/library/aix-toolbox/alpha.html. This file is used by UNIX Health Check for AIX to check for the latest versions of various RPMs installed on the AIX system. This file is not included on the UNIX Health Check version for Red Hat Enterprise Linux.

File VERSION contains the version number of UNIX Health Check. The version is an inverted date. For example, version 17.05.24 indicates the version of UNIX Health Check as released on May 24, 2017.

The mode for all the check scripts should already be set to 755 after installation. Please verify that this is the case. If not, please run:
# chmod 755 check*sh
Please repeat the installation of UNIX Health Check on every UNIX system, that you wish to check with UNIX Health Check. You may do this manually, or automate this task through a configuration management system, such as Ansible or Puppet.

↑
6. Determining the version of UNIX Health Check
To determine the version of UNIX Health Check that you are using, run the checkall.ksh script (on AIX) or checkall.sh script (on Red Hat Enterprise Linux, CentOS, CentOS Stream, AlmaLinux, Rocky Linux, Oracle Linux and Scientific Linux) using the "-u" option. The "-u" option can be used to show all the different options for UNIX Health Check, along with the version number: 
# ./checkall.sh -u

Copyright (c) 2004-2017 UNIX Health Check - All Rights Reserved

www.unixhealthcheck.com

This is confidential and unpublished work of authorship subject to limited
use license agreements and is a trade secret, which is the property of 
UNIX Health Check (www.unixhealthcheck.com). All use, disclosure and/or 
reproduction not specifically authorized in writing by UNIX Health Check 
is strictly prohibited.

Any expressed or implied warranties are disclaimed. In no event shall UNIX 
Health Check be liable for any direct, indirect, incidental, special, 
exemplary, or consequential damages (including, but not limited to, loss 
of use, data, profits, or business interruption) however caused and on any 
theory of liability, whether in contract, strict liability, or tort 
(including negligence or otherwise) arising in any way out of the use of 
these scripts, even if advised of the possibility of such damage.

Version: 17.05.24

Usage : checkall.sh [ -b ] [ -c ] [ -C ] [ -d ] [ -D ] [ -e ]
                    [ -E script1,script2,... ] [ -f file ] [ -g ] [ -G ]
		    [ -h ] [ -i ] [ -l ] 
		    [ -m emailaddress1,emailaddress2,... ]
                    [ -n ] [ -s script1,script2,... ] [ -u ] [ -v ]
                    [ -w # ] [ -x ]

 -b : Basics: no system configuration, end results and copyright message.
 -c : Write comma separated output (CSV).
 -C : Comma separated list of categories of scripts to run.
 -d : Add descriptions of checks to output.
 -D : Enable the debug option - for UNIX Health Check support use only.
 -e : Exclude ALL check scripts (will only run system configuration).
 -E : Comma-separated list of check scripts to exclude from the run.
 -f : Specify location of output.
 -g : Suppress showing output of check scripts that complete successfully.
 -G : Display check scripts that complete with ERROR; no WARNINGs.
 -h : Write web page output (HTML).
 -i : Inventory. Deprecated option. Use "-C inventory" instead.
 -l : Write text output to log file (TEXT) (default).
 -m : Comma-separated list of email addresses to send output to.
 -n : Only send email when result is less than 100% (errors only).
 -s : Comma-separated list of check scripts to run.
 -t : Specify the maximum allowed run time for check scripts.
 -u : Display usage information.
 -v : Verbose output to screen.
 -w : Specify the width of the output.
 -x : Write Extended Markup Language output (XML).
The example above shows the version number after the copyright message. Another way to determine the version of UNIX Health Check is to read the VERSION file: 
# cat VERSION
17.05.24
The version of UNIX Health Check is a reversed date. Version 17.05.24 is the release of UNIX Health check on May 24, 2017.

The example above also shows the different options that can be used with UNIX Health Check. These are discussed in more detail in Using options with the checkall master script.

Be sure to check regularly for updated versions of UNIX Health Check, as they are released often. The latest version is always available on our Support page (Click on the "download the latest version of UNIX Health Check" link in the download section, and log in with the user credentials that were provided to you upon purchase of a license for UNIX Health Check).

There's also a possibility to automate the download of UNIX Health Check directly to your UNIX systems. For more information see: Automating the download of UNIX Health Check.

↑
7. Running checks individually
Once the software has been installed, you will notice a lot of files within the chosen folder. Most of the files are individual check scripts that start with "check". Each individual script (except for script checkall.sh or checkall.ksh) will check a certain function or configuration of the system. You can run each script individually, if you like. For example, to check if wget is installed, run checkwget.ksh (on AIX) or checkwget.sh (on Red Hat Enterprise Linux). To determine the model name of the UNIX server, run checkmodelname.ksh or checkomodelname.sh.

For example, on AIX: 
# ./checkwget.ksh
wget-1.9.1-1
# ./checkmodelname.ksh
9117-MMB
And an example on Red Hat Enterprise Linux: 
# ./checkwget.sh
wget-1.14-13.el7.x86_64
# ./checkmodelname.sh
Dell Inc. PowerEdge R820
Please note that UNIX Health Check is designed to run as user root only. Many scripts will still run using a different user account, but UNIX Health Check is only supported when run via the root account. Root access is required, because UNIX Health Check runs several root-level commands, and not using the root user account may result in unwanted errors generated by the check scripts.

UNIX Health Check never makes any change to the UNIX system; it only reports. UNIX Health Check is not designed to automatically resolve any issues found, because the configuration can depend on many different factors within your environment or infrastructure. From the output of the check script(s), you can determine what issue was found (that is, if an issue was found), and what possible action can or should be taken to remediate the issue.

↑
8. Using the checkall master script
As described earlier, most users will usually use the checkall.ksh (for AIX) or checkall.sh (for Red Hat Enterprise Linux, CentOS, CentOS Stream, AlmaLinux, Rocky Linux, Oracle Linux and Scientific Linux) master script to run either a selection or all the check scripts to generate a report. Running check scripts individually can be a tedious task with hundreds and hundreds of different check scripts. You can use the checkall master script to run a combination of different check scripts. For example, if you wish to run check scripts checkwget.ksh and checkmodelname.ksh on your AIX server, you can use the "-s" (for "Script") option:
# ./checkall.ksh -s checkwget.ksh,checkmodelname.ksh
Multiple check scripts can be specified using the "-s" option. Scripts need to be separated using the comma (",") only.

Please note that when you run the above command, that no output will be shown on the screen. When the checkall.ksh script completes running all or the selected scripts, it will simply return the command prompt. UNIX Health Check will log its output by default to a log file. After running the above command, you will be able to find a log file within the folder in which checkall.ksh is located: 
# ls *log
checkall_hostname.log
You can open the log file with an editor and review the output. If you rather have UNIX Health Check both log its output to a log file and display the output on the screen while it is running, then add the "-v" (for "Verbose output") option: 
# ./checkall.ksh -s checkwget.ksh,checkmodelname.ksh -v
You can determine the width of the output using the -w (for "Width") option. For example, if you wish UNIX Health Check to generate a report that is only 70 characters wide, run: 
# ./checkall.ksh -s checkwget.ksh -v -w 70
######################################################################

2017-05-24 14:41:45: UNIX HEALTH CHECK FOR AIX

Copyright (c) 2004-2017 UNIX Health Check - All Rights Reserved

www.unixhealthcheck.com

This is confidential and unpublished work of authorship subject to
limited use license agreements and is a trade secret, which is the
property of UNIX Health Check (www.unixhealthcheck.com). All use,
disclosure and/or reproduction not specifically authorized in writing
by UNIX Health Check is strictly prohibited.

Any expressed or implied warranties are disclaimed. In no event shall
UNIX Health Check be liable for any direct, indirect, incidental,
special, exemplary, or consequential damages (including, but not
limited to, loss of use, data, profits, or business interruption)
however caused and on any theory of liability, whether in contract,
strict liability, or tort (including negligence or otherwise) arising
in any way out of the use of these scripts, even if advised of the
possibility of such damage.

This report is generated by UNIX Health Check for AIX. It is an
overview of check scripts run on an AIX system, and depending on the
options selected when the checkall.ksh script was run, it will list
each check script, the returncode of the check script, the output of
the check script and a description. At the end of this report is an
overview of all scripts run and a score report.

Any individual implementing changes should completely understand the
change and deem each change appropriate before it is applied to the
system. As a standard rule, please take into consideration the impact
on other components before implementing the change. Also, we
encourage all to conduct a peer review of all changes before
implementation. Most importantly, if the effect of a change is not
fully understood, do not implement that change until a satisfactory
explanation can be given as to what the effects of the change are. We
recommend implementation of one change at a time. The application of
many changes all at once will increase the likelihood of confusion,
if issues arise.

For more information, check website https://www.unixhealthcheck.com.
For support, email to support@unixhealthcheck.com.

######################################################################

2017-05-24 14:41:45: OPTIONS SELECTED
-------------------------------------

2017-05-24 14:41:45: Version:         17.05.19
2017-05-24 14:41:45: Start at:        05/24/2017 14:41:44 PDT
2017-05-24 14:41:45: Options:         -s checkwget.ksh -v -w 70
2017-05-24 14:41:45: Output file:     checkall_server01.log
2017-05-24 14:41:45: Width:           70
2017-05-24 14:41:45: Display:         All checks
2017-05-24 14:41:45: Descriptions:    No
2017-05-24 14:41:45: Script time-out: 180 seconds
2017-05-24 14:41:45: Output type:     TEXT
2017-05-24 14:41:45: # Checks:        1
2017-05-24 14:41:45: Script:          checkwget.ksh

######################################################################

2017-05-24 14:41:50: SYSTEM CONFIGURATION
-----------------------------------------

2017-05-24 14:41:50: Hostname:        server01
(server01.unixhealthcheck.com)
2017-05-24 14:41:50: IP Address:      172.16.42.43
2017-05-24 14:41:50: IP Assignment:   Static
2017-05-24 14:41:50: Subnet Mask:     255.255.255.0
2017-05-24 14:41:50: Default Gateway: 172.16.42.1
2017-05-24 14:41:50: Name Server(s):  172.16.42.56 172.16.42.57
2017-05-24 14:41:50: LPAR / VM:       1 SERVER01
2017-05-24 14:41:50: OS Level:        AIX 7.1.4.32 7100-04
2017-05-24 14:41:50: PowerHA Level:   7.1.3.6
2017-05-24 14:41:50: Model:           IBM,8286-42A IBM Power S824
2017-05-24 14:41:50: Serial Number:   21B096W
2017-05-24 14:41:50: Firmware Level:  SV860_082
2017-05-24 14:41:50: Kernel:          64 bit
2017-05-24 14:41:50: Hardware:        64 bit
2017-05-24 14:41:50: Processor Type:  PowerPC_POWER8
2017-05-24 14:41:50: CPU Clock Rate:  3525 MHz
2017-05-24 14:41:50: rPerf:           421.90 rPerf  based on 24.00
Virtual CPU cores
2017-05-24 14:41:50: CPUs:            24
2017-05-24 14:41:50: Logical CPUs:    192
2017-05-24 14:41:50: Capacity:        Min: 1.00 Entitled: 24.00 Max:
24.00 Increment: 1.00
2017-05-24 14:41:50: Physical CPUs:   24
2017-05-24 14:41:50: System type:     mode=Capped
type=Dedicated-SMT-8 weight=0 smt=8
2017-05-24 14:41:50: Virtual CPUs:    Desired: 24 Min: 1 Online: 24
Max: 24
2017-05-24 14:41:50: Memory:          255744MB
2017-05-24 14:41:50: Memory Settings: min: 256MB, desired: 255744MB,
max: 262144MB, online: 255744MB
2017-05-24 14:41:50: Paging Space:    32768MB (1% in use)
2017-05-24 14:41:50: Uptime:          02:41PM up 9 days, 2:41, 1
user, load average: 2.28, 2.29, 2.47

######################################################################

2017-05-24 14:41:51: CHECK SCRIPTS
----------------------------------

----------------------------------------------------------------------
2017-05-24 14:41:50: Running check script 1 of 1: checkwget.ksh
2017-05-24 14:41:51: Check checkwget.ksh completed successfully:
returncode 0
2017-05-24 14:41:51: Runtime: 1 second(s)
2017-05-24 14:41:51: 100% complete - 0 checks to go.
----------------------------------------------------------------------

######################################################################

2017-05-24 14:41:51: RESULTS
----------------------------

2017-05-24 14:41:51: Run time for all checks              : 7 seconds
2017-05-24 14:41:51: Total number of checks               : 1
2017-05-24 14:41:51: # Checks with result OK              : 1
2017-05-24 14:41:51: # Checks with result WARNING         : 0
2017-05-24 14:41:51: # Checks with result ERROR           : 0
2017-05-24 14:41:51: Score [Percentage OK/WARNING]        : 100.00 %

2017-05-24 14:41:51: For details see logfile              :
/uhc/checkall_server01.log

######################################################################
Note that the width setting should be between 70 or 1024 characters. By default, UNIX Health Check will produce output with a width that matches the current terminal, or if that cannot be determined, a width of 130 characters wide. This width only applies to regular text output, and does not apply to different output formats, such as CSV, XML and HTML (discussed later in this documentation).

To simply run all the check scripts, for example on AIX, run:
# ./checkall.ksh
By not specifying the "-s" option, UNIX Health Check's master script, checkall, will run all the check scripts, and generate an output file. Please note that when run without the "-s" option, the checkall script will determine which scripts should be run on your system, and may not run all scripts available. For example, if your AIX system is not a VIOS, or is not part of a PowerHA/HACMP cluster, checkall.ksh will skip any checks related to VIOS and/or PowerHA/HACMP systems, reducing the number of scripts that it should run, and thus shortening the time needed to run all the scripts. Therefore, you may notice that checkall runs less check scripts than what is included in the full UNIX Health Check software image.

↑
9. Using options with the checkall master script
Shown below are all the options you can use with the UNIX Health Check master script checkall:

OptionDescription
-b
Shows only the basic output. It will not show any of the end results, the configuration section will be skipped and no copyright message will be shown.
-c
This option will write a CSV file output instead of the default log file ("-l") output. CSV is comma separated output, which can be useful if you wish to use the information generated by UNIX Health Check in Microsoft Windows Excel or in a database.

Note: The order of the options determine which type of output is written. If you specify both the "-l" and "-c" options, the last option provided will determine the type of output written. This also applies to the "-h" (for HTML output) and "-x" (for XML output) options.

By default, UNIX Health Check will write CSV output to a file called checkall_hostname.csv in the same folder where UNIX Health Check is run from.

When using the CSV output format, no descriptions will be added to the output, even when option "-d" has been specified, as this does not fit in the comma separated output format.
-C
This option allows running scripts of a certain category. Specify a single category, or a comma separated list of categories of scripts to be run. For example, to run all security scripts, use the "-C" option as follows:
-C security
Or to run both memory and storage related scripts, use the "-C" option as follows:
-C memory,storage
You can choose from the following categories:

backupRun all scripts that check the backup of the system, including TSM, EMC Avamar/Networker, OSSV Snapvault, Commvault Simpana, and Veritas Netbackup.
bootingRun all check scripts that check items of the boot sequence, including inittab, rc.d files, autorestart, and start scripts.
capacityRun all check scripts that check the capacity of the system, including storage, CPU, network and memory.
clusterRun all scripts that check cluster configuration and status, including PowerHA/HACMP (on AIX), Red Hat Cluster (on Red Hat Enterprise Linux) and Veritas clustering.
daemonsRun all check scripts that check daemons and services on the system, including inetd.
datetimeRun all check scripts that check date and time settings on the server, including ntp and chrony configurations.
debuggingRun all check scripts that check debug items, such as debug filesets, core dump settings, error logging and system dumps.
hardwareRun all check scripts that check the physical hardware of the system, including adapters, devices, power supplies, firmware, and LPAR configuration.
inventoryRun all check scripts that generate inventory information and only display output of commands for inventory purposes.
loggingRun all check scripts for checking log files, alog and (r)syslog configuration.
mailRun all check scripts that check mail related items, such as the mail configuration and aliases, Sendmail and Postfix.
memoryRun all check scripts that check memory usage, vmo options (on AIX), swap space and memory cache.
monitoringRun all check scripts that check the monitoring of the system, including BMC Patrol, Ganglia, Marimba and Nimsoft.
networkRun all check scripts that check network settings, DHCP, DNS, and NFS.
performanceRun all check scripts that check the performance of the system, including those that check Active Memory Expansion (on AIX).
powervmRun all check scripts that check PowerVM configuration and settings, including dlpar, VIOS, and WPARs (AIX version only).
printingRun all check scripts that check printing and printer queues, including Cups.
redundancyRun all check scripts that check redundancy features and configuration on the server (does not include cluster items).
schedulingRun all check scripts that check the scheduling of jobs on the system, including at and cron.
securityRun all check scripts that check permissions/owner of files, the ssh configuration, old files, user accounts and user limits.
softwareRun all check scripts that check the software installed on the system, such as filesets (lslpp) and rpm packages, and additional software installed on the server.
storageRun all check scripts that check the storage of the system, including SDD, SDDPCM, multi-pathing, disks, and EMC Powerpath.
virtualizationRun all check scripts that check any of the virtualization features (Red Hat version only).

Please note that check scripts can belong to multiple categories. For example, a script that checks if xmdaily on AIX is disabled in the root crontab, checkxmdaily.ksh, belongs to both the category "scheduling" and the category "monitoring", as the xmdaily entry in the root crontab controls running xmwlm/topasrec processes for monitoring purposes on the system.
-d
This option will add descriptions of each individual check script to the output. This is very useful if you wish to know how to resolve certain issues. By default, no descriptions are written to the output. The descriptions for check scripts can also be found in the DESCRIPTIONS file.
-D
This option will enable the debugging feature of the checkall.ksh master check script. Use this option only when instructed to do so by the UNIX Health Check technical support team, for the purpose of gathering information regarding an issue with UNIX Health Check.
-e
This option will exclude running all scripts. This may sound weird, but this can be useful to only generate the system configuration section.

The "-e" option is overruled by the "-s" option. If one or more check scripts to run were specified using the "-s" option, these check scripts will still be run, even if the "-e" option was used as well.

The "-e" option however, overrules the "-C" and "-E" options. If you choose to run a certain category of scripts using the "-C" option, or if you exclude specific scripts using the "-E" option, then including the "-e" option will result in all the scripts being excluded.
-E
This option can be used to exclude specific scripts from being run. This option is useful if one or more check scripts generate a warning or error, and you have made sure that the warning or error does not apply to your environment. In that case, you can use the -E option to exclude specific scripts. Provide a comma-separated list of scripts that you would like to have excluded. For example:
# ./checkall.ksh -v -E checkbit.ksh,checkuptime.ksh
Multiple check scripts may only be separated by a single comma. Do not specify any folder names when specifying any check scripts to exclude. Only the name of a check script including the .ksh extension (on AIX) or the .sh extension (on Red Hat Enterprise Linux, CentOS, CentOS Stream, AlmaLinux, Rocky Linux, Oracle Linux and Scientific Linux) is allowed. UNIX Health Check assumes to find all the check scripts in the same folder where checkall.ksh or checkall.sh is run from. UNIX Health Check will not exclude any scripts it can't find in the same folder where checkall.ksh or checkall.sh is run from.

Additionaly, you may also supply a list of check scripts in a text file that need to be excluded.

For example, create a text file called /tmp/excludelist, and add the check scripts that need to be excluded, each on a seperate line: 
# cat /tmp/excludelist
checkzip.ksh
checkzombies.ksh
Next, use the -E option to specify this file:
# ./checkall.ksh -v -E /tmp/excludelist
You may also combine check scripts and text files listing check scripts. For example:
# ./checkall.ksh -v -E /tmp/excludelist,/tmp/excludelist2,checkchpuspeed.ksh
The -e option (lowercase) overrules the -E option (uppercase). If both are specified, the -e option will exclude all check scripts, and any scripts excluded specifically using the -E option will be ignored.
-f
You can use this option to specify a different location to write the output to. By default, the output will be written into the same folder from which the checkall script is run. You need to supply the filename, or the full path to the filename for this option, for example: 
# ./checkall.ksh -f filename
# ./checkall.ksh -f /full/path/to/filename
-g
This option suppresses the output of any checks that complete successfully. This is specifically very useful, when running all scripts through the master check script checkall.ksh (AIX) or checkall.sh (Red Hat Enterprise Linux, CentOS, CentOS Stream, AlmaLinux, Rocky Linux, Oracle Linux and Scientific Linux). Many of the check scripts will most likely return a zero error code, indicating they have completed successfully, and thus a report of those check scripts is usually not worth reviewing, because they complete successfully and no issue is identified. Using the "-g" option will make sure the checks that complete successfully will not end up in the report. By default, the output of all scripts is shown.
-G
This option suppresses the output of any checks that complete successfully - and check scripts that return a WARNING (return code 2). Only check scripts that return an ERROR (return code 1) will be displayed. The "-G" option supersedes the "-g" option. This option is useful if you only wish to review the most critical items identified on the system, and when you do not care about any WARNING messages.
-h
This option will write an HTML file output instead of the default log file ("-l") output. By generating HTML output, it is easier to see which check scripts return an error or warning, and which check scripts complete successfully. This is accomplished by using different colors for each script. A script that completes successfully (or ends with return code 0) will be listed as green. Check scripts that end with a WARNING (or return code 2) will be listed as yellow. Check scripts that end with an ERROR (or return code 1) will be listed as red.

Note: The order of the options determine which type of output is written. If you specify both the "-l" and "-h" options, the last option provided will determine the type of output written. This also applies to the "-c" (for CSV output) and "-x" (for XML output) options.

By default, UNIX Health Check will write HTML output to file called checkall_hostname.html in the same folder where UNIX Health Check is run from.

A sample of HTML output can be seen on the Sample reports page.

To view the HTML reports that UNIX Health Check generates, one of the following browsers is required:
  • Microsoft Internet Explorer 8 or higher
  • Microsoft Edge
  • Mozilla Firefox
  • Google Chrome
Other browsers may or may not work.
-i
This is an old option, and its use is deprecated. It was used to generate an inventory section. If you wish to run any scripts of the inventory category, you should use option "-C inventory" instead.
-l
This option will write a text log file output. This is the default output type if neither of the options "-l", "-c", "-x" or "-h" is provided. By default, UNIX Health Check will write a text log file to file name checkall_hostname.log in the same folder where UNIX Health Check is run from.

Note: The order of the options determine which type of output is written. If you specify both the "-h" and "-l" option, the last option provided will determine the type of output written. This also applies to the "-c" (for CSV output) and "-x" (for XML output) options.
-m
This option can be used to provide a comma-separated list of email addresses to send output to, for example:
# checkall.ksh -m info@redhathealthcheck.com,info@aixhealthcheck.com
Mail is sent through the mailx command. Before you attempt to use this option, please make sure that it is possible to send email by running:
# cat /etc/hosts | mailx -v -s subject my@emailaddress.com
Email will be sent in the type of output that you've specified. If you didn't specify any of the "-c" (for CSV format), "-h" (for HTML format), "-x" (for XML format) or "-l" (for TEXT) options, a text file output (by default, the "-l" option) will be sent through email. If you specified "-c", a CSV style output will be sent through email. If you specified "-h", an HTML type report will be sent through email. If you specified "-x", an XML type report will be sent through email.

To view the HTML reports by email, only Microsoft Outlook 2003/2010/2016 and Windows Live Mail are supported. Other email clients may or may not be able to view the HTML report generated by UNIX Health Check properly.

Multiple email addresses may only be separated by a single comma.
-n
The "-n" option will ensure that only output is sent by email if there are any check scripts that didn't complete successfully (return code 0).

This is very useful to use in combination with the "-m" option. For example, if you have several crontab entries set up to monitor your systems, you can use the "-n" option to only alert you by email if there is indeed a check that generates an ERROR message (return code 1). No email is sent if all the check scripts complete successfully (return code 0) or if only check scripts completed with a WARNING message (return code 2).

For example, you could add the following to the root user's crontab on an AIX system, to alert you if there's an issue on the UNIX system: 
# Check every hour:  Free space in file systems, 
# paging space usage, memory utilization, 
# high CPU using processes, printer queues, 
# error report, zombie processes
0 * * * *   /uhc/checkall.ksh -gdhbnm email@address.com 
	    -s checkfreespaceinfs.ksh,checkpgspusage.ksh,
            checkmemoryutilization.ksh,checkhighcpu.ksh
5 * * * *   /uhc/checkall.ksh -gdhbnm email@address.com 
            -s checkenq.ksh,checkerrpt.ksh,checkzombies.ksh
# Check once a day: TSM backups
9 9 * * *   /uhc/checkall.ksh -gdhbnm email@address.com 
	    -s checktsmsched.ksh
# Check once a week: mksysb backups, logical volume/file 
# system structure
0 8 * * 1,5 /uhc/checkall.ksh -gdhbnm email@address.com 
            -s checkmksysb.ksh,checklvfscreate.ksh
A useful combination for the "-n" option, is to also use the "-b" option. This way, also any end results and copyright messages are eliminated, thus further reducing the emails sent out.
-s
You can use the "-s" option to provide a comma-separated list of check scripts to run. The default action of checkall.ksh is (without supplying the "-s" option) to run all the check scripts. With the "-s" option specified, you can specify which check scripts to run. For example:
# ./checkall.ksh -s checkbit.ksh,checkpgspusage.ksh
Multiple check scripts may only be separated by a single comma. Do not specify any folder names when specifying any check scripts to run. Only the name of a check script including the .ksh extension (on AIX) or the .sh extension (on Red Hat Enterprise Linux, CentOS, CentOS Stream, AlmaLinux, Rocky Linux, Oracle Linux and Scientific Linux) is allowed. For example: "-s /tmp/checkbit.ksh" will not work, but "-s checkbit.ksh" will work. UNIX Health Check assumes to find all the check scripts in the same folder where checkall.ksh or checkall.sh is run from. UNIX Health Check will skip running any scripts it can't find in the same folder where checkall.ksh or checkall.sh is run from.

Additionaly, you may also supply a list of check scripts in a text file that need to be included.

For example, create a text file called /tmp/includelist, and add the check scripts that need to be included, each on a seperate line: 
# cat /tmp/includelist
checkzip.ksh
checkzombies.ksh
Next, use the -s option to specify this file:
# ./checkall.ksh -v -s /tmp/includelist
You may also combine check scripts and text files listing check scripts. For example:
# ./checkall.ksh -v -s /tmp/includelist,/tmp/includelist2,checkchpuspeed.ksh
The "-s" option overrules the "-e" (for excluding check scripts) option. Scripts specified with the "-s" option will be run, even if the same scripts were excluded using the "-e" option. The "-s" option also overrules the "-E" option. If specific scripts were excluded using "-E" option, and still included using the "-s" option, then the scripts will be run. Finally, the "-s" option also overrules the "-C" option (for specifying categories of check scripts). If certain scripts were specified using the "-s" option, then the "-C" option (for specifying categories of check scripts) will be ignored.
-t
The "-t" option allows you to specify a maximum allowed run time for check scripts.

The default allowed run time for a check script is 180 seconds.

A check script will NOT be cancelled after exceeding the allowed run time. Each check script is allowed to complete, no matter how long it takes to run. However, a check script result is marked with status ERROR if the default allowed runtime is exceeded. To avoid reporting an ERROR status for check scripts that take a long time to run, for example, on slower systems, you can utilize the "-t" option to increase the maximum allowed runtime.
-u
The "-u" option displays the usage information, and then exits the checkall master script. Using the question mark ("-?") instead of the "-u" option has the same effect.
-v
The "-v" option displays verbose output information on screen, while the checkall master script is running.
-w
With the "-w" option you can specify the width of the output. By default, UNIX Health Check will generate output of 130 characters wide or at the same width of the terminal that it is run in. You can specify any value with "-w" from 70 to 1024 characters:
# ./checkall.sh -w 512
Specifying the width of the output has no effect if HTML output ("-h") is selected.
-x
This option will write an XML file output instead of the default TEXT log file ("-l") output. By generating XML output, output of UNIX Health Check can be loaded into various other systems/applications or can be viewed in tree-like form with the use of a XML reader.

Note: The order of the options determine which type of output is written. If you specify both the "-l" and "-x" options, the last option provided will determine the type of output written. This also applies to the "-c" (for CSV output) and "-h" (for HTML output) options.

By default, UNIX Health Check will write XML output to file checkall_hostname.xml in the same folder where UNIX Health Check is run from.

To view the XML reports that UNIX Health Check produces, an XML reader is required, for example the free XML editor, Foxe, from First Object, or the free MindFusion XML Viewer from MindFusion.
-?
Using the question mark ("-?") has the same effect as using the "-u" option, and will provide the usage information.

↑
10. Return codes
Each check script ends with one of the following three exit/return codes:

0
The script completed successfully.
1
The script found an ERROR.
2
The script ended with a WARNING.

Errors are usually something that requires some form of remediation, and should be resolved, if possible. Warnings are just that, warnings. These may be ignored, but it might still be useful to consider looking into the warnings.

You can see the return code of a single check script by checking the $? value. For example: 
# ./checkpgspsize.ksh
# echo $?
0
This particular script ended with return code 0, and thus ended successfully (no ERROR or WARNING is generated) and thus all it checked is okay.

The following is an example of a check script that ends with an ERROR status: 
# ./checkbosadtdebug.ksh
Fileset bos.adt.debug not installed.
# echo $?
1
As you can see, a script that ends with an ERROR (or a WARNING) status, will also output a message about the error found.

↑
11. Running categories of scripts
UNIX Health Check includes a feature to run certain categories of scripts. Instead of running all the check scripts that are included in UNIX Health Check, this option allows you to focus on those features deemed relevant to you at that time. For example, you may want to check all the security items of the UNIX system, and not be bothered by anything else. Or sometimes, you may wish to generate only an inventory, by running only those scripts that provide inventory type information.

You can use the "-C" option available in UNIX Health Check, to run a category of scripts. For example, to run all the security type scripts, run:
# ./checkall.ksh -C security
Likewise, if you wish to generate the inventory information of the UNIX system, run:
# ./checkall.ksh -C inventory
You can also combine several categories of scripts by providing a comma-separated list of categories to be run. For example, if you wish to run all the storage and security type scripts, you can run:
# ./checkall.ksh -C storage,security
A detailed list of available categories can be found in section Using options with the checkall master script.

Please note that scripts may belong to multiple categories. For example, a script that checks if xmdaily is disabled in the root crontab, checkxmdaily.ksh, belongs to both the category "scheduling" and the category "monitoring", as the xmdaily entry in the root crontab controls running xmwlm/topasrec processes for monitoring purposes on the system.

Also keep in mind that by selecting one or more categories, that this determines how many scripts will be run. For example, the security category is a large category of scripts, that includes over 300 scripts to be run, while the datetime category is a much smaller category, that includes only a few scripts.

↑
12. Automating the download of UNIX Health Check
It is easy to automate the download of the latest version of UNIX Health Check to your UNIX system. Doing so, removes the need for downloading UNIX Health Check manually each time a new version of UNIX Health Check is released, and removes the need to transfer the downloaded file onto your UNIX systems manually.

12a. What do you need?
  • Install either curl or wget on your UNIX system, if not already installed. For AIX: These tools are not part of a regular AIX installation. You can download the RPM for these tools from the AIX Toolbox for Linux Applications. For Red Hat Enterprise Linux: You can use the familiar yum command to install the packages.
  • Your UNIX system needs Internet access, and needs to be able to resolve the "unixhealthcheck.com" domain through DNS.

    For example, test with a simple ping:
    # ping unixhealthcheck.com
  • You need a valid license for UNIX Health Check, which provides you a login (in the format of an email address) and a password to access the UNIX Health Check download page. If you are unable to download the UNIX Health Check software manually from the download page, then you won't be able to automate the download as well.
12b. Examples of using curl and wget to download UNIX Health Check

Here's an example of how you can use curl to download the latest version of UNIX Health Check (assuming your login/email address is "user@email.com" and your password is "mypassword"):

For AIX: 
curl -d "emailaddress=user@email.com&password=mypassword" \
   --referer https://www.unixhealthcheck.com \
   https://www.unixhealthcheck.com/downloadauto.php \
   -o ahc_latest.tar
For Red Hat Enterprise Linux, CentOS, CentOS Stream, AlmaLinux, Rocky Linux, Oracle Linux and Scientific Linux: 
curl -d "emailaddress=user@email.com&password=mypassword" \
   --referer https://www.unixhealthcheck.com \
   https://www.unixhealthcheck.com/downloadauto.php \
   -o rhc_latest.tar
This command will download file ahc_latest.tar or rhc_latest.tar in your current folder. Please note that providing the email address, password and referer is required. Without it, the download will fail.

The same download can be accomplished with wget:

For AIX: 
wget --post-data 'emailaddress=user@email.com&password=mypassword' \
   --referer=https://www.unixhealthcheck.com \
   https://www.unixhealthcheck.com/downloadauto.php \
   -O ahc_latest.tar
For Red Hat Enterprise Linux, CentOS, CentOS Stream, AlmaLinux, Rocky Linux, Oracle Linux and Scientific Linux: 
wget --post-data 'emailaddress=user@email.com&password=mypassword' \
   --referer=https://www.unixhealthcheck.com \
   https://www.unixhealthcheck.com/downloadauto.php \
   -O rhc_latest.tar
Once you have downloaded the ahc_latest.tar or rhc_latest.tar file, you will be able to un-pack the TAR image file and use UNIX Health Check.

12c. Automating download with a script

You can automate downloading UNIX Health Check easily with the use of a shell script. The script that is shown below downloads UNIX Health Check with the use of wget, will un-pack the downloaded file, run UNIX Health Check, send an HTML style report through email, and clean up afterwards.

For AIX: 
#!/usr/bin/ksh
 
# VARIABLES
emailaddress="user@email.com"
password="mypassword"
folder="/uhc"
wget="/usr/bin/wget"
myfile="ahc_latest.tar"
 
# Test if folder already exists.
if [ -d ${folder} ] ; then
        echo "Folder ${folder} already exists. Aborting."
        exit
fi
 
# Test if wget is executable.
if [ ! -x ${wget} ] ; then
        echo "Wget does not exist or is not executable. Aborting."
        exit
fi
 
# Create a folder to store UNIX Health Check:
echo "Creating folder ${folder}..."
mkdir -p ${folder}
 
# Download the file.
echo "Downloading UNIX Health Check..."
${wget} --post-data "emailaddress=$emailaddress&password=$password" \
   --referer=https://www.unixhealthcheck.com \
   https://www.unixhealthcheck.com/downloadauto.php \
   -O ${folder}/${myfile} >/dev/null 2>&1
 
# Un-pack the downloaded file.
if [ -s ${folder}/${myfile} ] ; then
        cd ${folder}
        echo "Un-packing the downloaded file..."
        tar -xvf ${folder}/${myfile} >/dev/null 2>&1
        cd - >/dev/null 2>&1
        echo "Removing downloaded file..."
        rm -f ${folder}/${myfile}
 
        # Run UNIX Health Check.
        if [ -x ${folder}/checkall.ksh ] ; then
                echo "Running UNIX Health Check..."
		${folder}/checkall.ksh -hdm ${emailaddress}
                echo "Removing folder ${folder}..."
                rm -rf ${folder}
        else
                echo "Encountered an error with checkall."
                echo "Removing folder ${folder}..."
                rm -rf ${folder}
                exit
        fi
else
        echo "Error downloading UNIX Health Check."
        echo "Removing folder ${folder}..."
        rm -rf ${folder}
        exit
fi
For Red Hat Enterprise Linux, CentOS, CentOS Stream, AlmaLinux, Rocky Linux, Oracle Linux and Scientific Linux: 
#!/bin/bash
 
# VARIABLES
emailaddress="user@email.com"
password="mypassword"
folder="/uhc"
wget="/bin/wget"
myfile="rhc_latest.tar"
 
# Test if folder already exists.
if [ -d ${folder} ] ; then
        echo "Folder ${folder} already exists. Aborting."
        exit
fi
 
# Test if wget is executable.
if [ ! -x ${wget} ] ; then
        echo "Wget does not exist or is not executable. Aborting."
        exit
fi
 
# Create a folder to store UNIX Health Check:
echo "Creating folder ${folder}..."
mkdir -p ${folder}
 
# Download the file.
echo "Downloading UNIX Health Check..."
${wget} --post-data "emailaddress=$emailaddress&password=$password" \
   --referer=https://www.unixhealthcheck.com \
   https://www.unixhealthcheck.com/downloadauto.php \
   -O ${folder}/${myfile} >/dev/null 2>&1
 
# Un-pack the downloaded file.
if [ -s ${folder}/${myfile} ] ; then
        cd ${folder}
        echo "Un-packing the downloaded file..."
        tar -xvf ${folder}/${myfile} >/dev/null 2>&1
        cd - >/dev/null 2>&1
        echo "Removing downloaded file..."
        rm -f ${folder}/${myfile}
 
        # Run UNIX Health Check.
        if [ -x ${folder}/checkall.sh ] ; then
                echo "Running UNIX Health Check..."
		${folder}/checkall.sh -hdm ${emailaddress}
                echo "Removing folder ${folder}..."
                rm -rf ${folder}
        else
                echo "Encountered an error with checkall."
                echo "Removing folder ${folder}..."
                rm -rf ${folder}
                exit
        fi
else
        echo "Error downloading UNIX Health Check."
        echo "Removing folder ${folder}..."
        rm -rf ${folder}
        exit
fi
The script above can be used to download UNIX Health Check to your UNIX system in folder "/uhc". Please make sure to provide the correct login/email address and password combination in the beginning of the script for variables "emailaddress" and "password", and if you wish to specify a different location for storing UNIX Health Check, you can change this in the script as well, by updating the "folder" variable.

The output of the script above, will look like this (assuming you have named the script "run-uhc"): 
# ./run-uhc
Creating folder /uhc...
Downloading UNIX Health Check...
Un-packing the downloaded file...
Removing downloaded file...
Running UNIX Health Check...
Removing folder /uhc...

↑
13. Enabling password-less access through SSH on cluster nodes
When running UNIX Health Check on cluster nodes, such as PowerHA/HACMP or Red Hat Cluster, it is important that password-less access for user root is available from one cluster node to another. If this is not set up correctly, some scripts may fail.

Here are the steps to perform on each cluster node:

First, make sure OpenSSH and OpenSSL are installed on the two clustered UNIX servers, serverA and serverB.

On each server, as user root, type:
# ssh-keygen -t rsa
This will prompt you for a secret passphrase. If this is your primary identity key, use an empty passphrase (which is not secure, but the easiest to work with). You will get two files called id_rsa and id_rsa.pub in the .ssh sub-folder in the root user's home directory.

Copy the id_rsa.pub file to the other host's .ssh sub-folder with the name authorized_keys. For example, on serverA:
# scp ~root/.ssh/id_rsa.pub serverB:~root/.ssh/authorized_keys
Now serverB should be ready to accept your SSH key from serverA. Please note that if the authorized_keys file already exists on serverB, that instead of copying over the file, you probably rather want to append the contents of id_rsa.pub from serverA to the authorized_keys file on serverB, to avoid overwriting any existing keys already present in the authorized_keys file.

On Red Hat Enterprise Linux, you can use the "ssh-copy-id" command to copy over the the contents of the id_rsa.pub file and append it to the authorized_keys file on the target host, by running: 
# ssh-copy-id serverB
To test, type:
# ssh serverB
This should allow you to log in without typing a password or passphrase.

↑
14. Resolving email issues
When sending email from UNIX Health Check's master script (checkall.ksh or checkall.sh) using the "-m" option, you may run into issues while sending email.

Before you attempt to use the email feature of UNIX Health Check, please make sure to verify that you can send email from your system, by sending a test email as follows:
# cat /etc/hosts | mailx -v -s test email@domain.com
Please replace "email@domain.com" with a valid email address within your organization.

By using the "-v" option with the mailx command, you will see verbose output of the email you're sending. If mail does not arrive at the designated mailbox, check the mail queue on the UNIX system to determine if mail has been sent or not:
# mailq
If mail is not being sent, this is usually due to two common issues:
  • DNS is not properly configured
    For the mail function of your system, it is important that your system can resolve the fully qualified domain name of your system. Make sure that the short hostname, the fully qualified domain name, e.g. host.domain.com, and the IP address of your server can be resolved, by testing with the nslookup command.

    For example: 
    # nslookup myserver
Name:   myserver.domain.com
Address: 10.43.242.82

# nslookup myserver.domain.com
Name:   myserver.domain.com
Address: 10.43.242.82

nslookup 10.43.242.82
82.242.43.10.in-addr.arpa name = myserver.domain.com.
    If this does not work in the example above, please check with your network administrator to correct the DNS settings for your server.
  • Missing SMTP server entry in /etc/sendmail.cf or /etc/postfix/main.cf file
    Mail can also not be sent properly, if the name of the SMTP server is missing in /etc/sendmail.cf file. Look for an entry in this file that starts with "DS", for example: 
    # "Smart" relay host (may be null)
DSrelay.domain.com
    If no entry is present, discuss with your email administrator which SMTP relay host can be used to relay email from the UNIX system to your desired mailbox.

    On many Linux distributions, postfix is used for sending email, and the main configuration file for postfix can be found in /etc/postfix/main.cf, in which the target SMTP server should be defined using the relayhost entry, such as:
    relayhost = relay.domain.com
  • Issues on the SMTP server
    Issues on the SMTP server side may also prevent email from being received. An SMTP server may not allow email being sent from the client server, or there may be other issues on the SMTP server preventing email from being transmitted. Please consult with your SMTP server administrator if that is the case. If email remains in the mail queue (as seen by running the mailq command), especially when "read errors" are displayed when the mailq command is run, this may be an indication of issues on the SMTP server side.

↑
15. Resolving issues discovered by UNIX Health Check
When reviewing reports generated by UNIX Health Check, you may see issues that need to be remediated.

Please take note of the warning in the report:

"Any individual implementing changes should completely understand the change and deem each change appropriate before it is applied to the system. As a standard rule, please take into consideration the impact on other components before implementing the change. Also, we encourage all to conduct a peer review of all changes before implementation. Most importantly, if the effect of a change is not fully understood, do not implement that change until a satisfactory explanation can be given as to what the effects of the change are. We recommend implementation of one change at a time. The application of many changes all at once will increase the likelihood of confusion, if issues arise."

It is important to follow change procedures properly while resolving any issues found by UNIX Health Check.

We recommend starting out by running UNIX Health Check on a single system first. And when an issue is found, attempt to resolve the issue not only on the one system, but if it is an issue that applies to multiple systems in your organization, apply the solution to multiple servers, obviously by following proper change procedures, and doing changes on non-production systems first, and when successful, followed by applying the same changes to the production systems. Repeat that for each issue found, until you've exhausted the issues to be remediated. Then move forward by running UNIX Health Check on the next system. You'll most likely find that many issues have already been resolved, by resolving issues and applying appropriate changes on all the systems, from running UNIX Health Check on the previous system. Quite often, UNIX administrators configure systems more or less the same way, or also, may make similar mistakes on multiple systems. As such it is good practice, when resolving an issue, to determine if the same issue is also present on other systems within your organization, and resolving them on all the systems. That helps to standardize your environment. To aid in this, configuration systems, such as Puppet and/or Ansible can be used to configure systems appropriately.