The following procedure describes how you can continuously monitor a log file through the use of SystemD on Red Hat Enterprise Linux (or similar operating systems).

Let's say you want to receive an email when a certain string occurs in a log file. For example, if the string "error" occurs in file /var/log/messages.

First, create a script that does a tail of /var/log/messages and searches for that string:

# cat /usr/local/bin/monitor.bash
#!/bin/bash

tail -fn0 /var/log/messages | while read line ; do
   echo "${line}" | grep -i "error" > /dev/null
   if [ $? = 0 ] ; then
      echo "${line}" | mailx -s "error in messages file" your@emailaddress.com
   fi
done

You can run that script, and be done with it. But if that script somehow gets cancelled or killed, for example when the system is rebooted, then the monitoring of the log file stops as well. That's where the use of Systemd comes in.

Create a file in folder /etc/systemd/system, such as "monitor.service", and add the following:

[Unit]
Description=My monitor script
After=network.target

[Service]
Type=simple
ExecStart=/bin/bash /usr/local/bin/monitor.bash
TimeoutStartSec=0
Restart=always
StartLimitInterval=0

[Install]
WantedBy=default.target

This file is a description of a service managed by Systemd, and it basically tells SystemD to run the script that we created earlier, and to restart it, in case it fails (that's what "Restart=always" is for).

Next, you'll have to tell Systemd that you made some changes:

# systemctl daemon-reload

Now you can start the newly defined service:

# systemctl start monitor.service

And after starting it, you can query the status:

# systemctl status monitor.service
monitor.service - My monitor script
   Loaded: loaded (/etc/systemd/system/monitor.service; enabled; vendor preset: disabled)
   Active: active (running) since Wed 2018-10-03 12:12:28 CDT; 2s ago
 Main PID: 1832 (bash)
   CGroup: /system.slice/monitor.service
           ??1832 /bin/bash /usr/local/bin/monitor.bash
           ??1833 tail -fn0 /var/log/messages
           ??1834 /bin/bash /usr/local/bin/monitor.bash

Oct 03 12:12:28 enemigo systemd[1]: Started My monitor script.
Oct 03 12:12:28 enemigo systemd[1]: Starting My monitor script...

As you can see in the output above, both the monitor.bash script and the tail command are running. To test if the service is actually restarted, you can try killing the tail process or the monitor.bash script, and then check for the status again. You'll see it is restarted.

You may also want to test that the new monitor script indeed is working. Considering that /var/log/messages is a file written to through rsyslog, you can log an entry with the string "error" to that file as follows:

# logger error

Next, you should receive an email saying that an "error" occurrence was found in the messages file.

Finally, you'll want to make sure this new monitor service is restarted also when the system boots:

# systemctl enable monitor.service

If you found this useful, here's more on the same topic(s) in our blog:

• Using tmp.mount
• ILO: Using Virtual Media with the Command-Line Interface
• Common items to install under Red Hat on Dell hardware
• Monitor SSH logins
• VIM Swap and backup files