Topics: Red Hat / Linux, Security

Renew self-signed SSL Certificate

Should the root user receive emails from certwatch about expiring self-signed certificates, like these:

 ################# SSL Certificate Warning ################

  Certificate for hostname 'yourhost', in file (or by nickname):

  The certificate needs to be renewed; this can be done
  using the 'genkey' program.

  Browsers will not be able to correctly connect to this
  web site using SSL until the certificate is renewed.

                                  Generated by certwatch(1)

Then, you can run the following command to renew this self-signed certificate for a new year:
# openssl req -new -days 365 -x509 -nodes -out /etc/pki/tls/certs/localhost.crt \ 
-keyout /etc/pki/tls/private/localhost.key

More complete instructions can be found here:

Topics: Performance, Red Hat / Linux

How to view threads of a process on Linux

Threads are a popular programming abstraction for parallel execution on modern operating systems. When threads are forked inside a program for multiple flows of execution, these threads share certain resources (e.g., memory address space, open files) among themselves to minimize forking overhead and avoid expensive IPC (inter-process communication) channel. These properties make threads an efficient mechanism for concurrent execution.

In Linux, threads (also called Lightweight Processes (LWP)) created within a program will have the same "thread group ID" as the program's PID. Each thread will then have its own thread ID (TID). To the Linux kernel's scheduler, threads are nothing more than standard processes which happen to share certain resources. Classic command-line tools such as ps or top, which display process-level information by default, can be instructed to display thread-level information.

Here are several ways to show threads for a process on Linux:

Using the ps command

The "-T" option for the ps command enables thread views. The following command list all threads created by a process with :

# ps -T -p <pid>
For example to list the threads for the following java process:
# ps -ef | grep 97947
deploy   97947 97942  1 00:51 ?       00:13:51 java
Run the following command:
# ps -T -p 97947
97947 97947 ?    00:00:00 java
97947 97948 ?    00:00:00 java
97947 97949 ?    00:00:00 java
The "SID" column represents thread IDs, and "CMD" column shows thread names.

Using the top command

The top command can show a real-time view of individual threads. To enable thread views in the top output, invoke top with "-H" option. This will list all Linux threads. You can also toggle on or off thread view mode while top is running, by pressing 'H' key.
top - 14:43:25 up 6 days,  5:40,  2 users,  load average: 0.87, 0.33, 0.22
Threads: 684 total,   1 running, 683 sleeping,   0 stopped,   0 zombie
%Cpu(s):  6.3 us,  4.0 sy,  0.0 ni, 89.6 id,  0.1 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem :  7910136 total,   384812 free,  1603096 used,  5922228 buff/cache
KiB Swap:  8388604 total,  8239100 free,   149504 used.  5514264 avail Mem
Note how in the example above the number of threads on the system is listed.

To restrict the top output to a particular process and check all threads running inside the process:
# top -H -p <pid>
Using htop

A more user-friendly way to view threads per process is via htop, an ncurses-based interactive process viewer. This program allows you to monitor individual threads in tree views.

To enable thread views in htop, launch htop, and press F2 to enter htop setup menu. Choose "Display option" under "Setup" column, and toggle on "Tree view" and "Show custom thread names" options. Presss F10 to exit the setup.

Topics: Red Hat / Linux

Linux strace command

The strace command in Linux lets you trace system calls and signals. Or in other words: it allows you to see what a program or process is doing.

Following is its syntax:

strace [OPTIONS] command
In the simplest case, strace runs the specified command until it exits. It intercepts and records the system calls which are called by a process and the signals which are received by a process. The name of each system call, its arguments and its return value are printed on standard error or to the file specified with the -o option. For example:
# strace ls
If a process is already running, you can trace it by simply passing its PID as follows; this will fill your screen with continued output that shows system calls being made by the process. To end it, press [Ctrl + C].
# strace -p 3569

Topics: Red Hat / Linux, Security

Discovering and joining identity domains on RHEL

The "realm discover" command returns complete domain configuration and a list of packages that must be installed for the system to enrolled in the domain.

The "realm join" command then sets up the local machine for use with a specified domain by configuring both the local system services and the entries in the identity domain. the process run by "realm join" follows these steps:

  • Running a discovery scan for the specified domain.
  • Automatic installation of the packages required to join the system to the domain.
    This includes SSSD and the PAM home directory job packages. Note that the automatic installation of packages requires the PackageKit suite to be running. If PackageKit is disabled, the system prompts you for the missing packages, and you will be required to install them manually using the "yum" utility.
  • Joining the domain by creating an account entry for the system in the directory.
  • Creating the /etc/krb5.keytab host keytab file.
  • Configuring the domain in SSSD and restarting the service.
  • Enabling domain users for the system services in PAM configuration and the /etc/nsswicht.conf file.
To install the software manually, run:
# yum install realmd oddjob oddjob-mkhomedir sssd 
# yum install adcli krb5-workstation samba-common-tools
When run without any options, the "realm discover" command displays information about the default DNS domain. It is also possible to run a discovery for a specific domain, such as:
# realm discover mydomain.local
Before this works, make sure the system can access the domain controllers, such as the AD servers. You may have to add those to /etc/resolv.conf, as realmd will use DNS SRV lookups to find the domain controllers in the domain automatically.

To join the domain use the "realm join" command, for example:
# realm join -v - U user mydomain.local
By default, the join is performed as the domain administrator. For AD, the administrator is called "Administrator"; for IdM, it is called "admin". To connect as a different user, use the -U option, as was shown in the example above. When prompted for a password, type it in.

Once the join has been completed, use a separate user account to login to the system, to ensure that domain accounts work.

If you run into issue, make sure that the following ports are opened in the firewall, if present: 53, 389, 636, 88, 464, 3268, 3269 and 123.

For more details on joining a RHEL system to a domain, see:

Topics: Red Hat / Linux, Security

Resetting the Root Password of RHEL-7

In case you've ever forgotten the root password of a RHEL 7 system, here's a good description of how to change it:

Topics: Red Hat / Linux, Storage

RHEL: Recovering a corrupt file system in emergency mode

If you have a RHEL system that boots in emergency mode due to a corrupt file system, here are some steps to perform to resolve the issue.

Once booted in emergency mode, connect to the system on the console. Try running:

# journalctl -xe
To help determine why the system booted in emergency mode. For example, you may discover that the /var file system has an issue.

Once you do, shut down the system, and boot it from the boot ISO or DVD (depending if the system is a virtual or a physical system).

When using VMware, you'll have to edit the settings of the VM, and enable the "Force BIOS setup" setting, and mount/connect the boot ISO image. Then start it up, and open the console. Once in the BIOS, make sure CDROM is high in the boot order.

Once the system is booting RHEL, select "Troubleshooting", and "Rescue a Red Hat Enterprise Linux system". The system will boot up. Select "Continue" when prompted.

Run "df -h" to list the mounted directories. Since /var is the file system having the issue, unmount it:
# unmount /mnt/sysimage/var
Once /var is unmounted, run fsck to fix any issues:
# fsck /mnt/sysimage/var
Once completed, you can reboot the system. If you connected an ISO image earlier, make sure to disconnect the ISO image.

Topics: LVM, Red Hat / Linux

Removing a LVM based file system

If you need to remove a non-root LVM based file system, it can be removed using the following steps. If the file system contains any data, then please make sure to take a backup of that data before proceeding with the following steps.

In this example, we will be deleting "u01lv" from volume group "oracle". The logical volume is mounted on mount point /u01.

# df /u01 | grep u01
/dev/ampper/oracle/u01lv   209612800  80946812  128665988  39%  /u01
# lvs | grep u01
  u01lv oracle -wi-ao---- 200.00g
As you can see, mount point /u01 is define don logical volume u01lv, and is 200 GB.

Start off, by deleting the corresponding entry in /etc/fstab.
# grep u01 /etc/fstab 
/dev/oracle/u01lv    /u01    xfs  defaults,noatime 0 0
Then unmount the file system:
# umount /u01
Disable the logical volume:
# lvchange -an /dev/oracle/u01lv
Delete the logical volume:
# lvremove /dev/oracle/u01lv
At this point, depending if this logical volume is defined on a disk within a volume group, you can, if the disk is now empty after removing the logical volume, also remove the disk from the volume group. First run the "pvs" command and check for an empty disk, such as:
# pvs
  PV         VG       Fmt   Attr  PSize   PFree
  /dev/sdb   oracle   lvm2  a--   <5.00t  <5.00t
If the disk is empty, as seen as disk /dev/sdb in the example above, you can then remove it from the volume group:
# vgreduce oracle /dev/sdb
Or, if the logical volume removed from the volume group was the very last logical volume to remove, and the volume group is empty, you can also remove the volume group. Do so by disabling the volume group and removing it:
# vgchange -an oracle
# vgremove oracle
And then finally remove any physical volumes used, such as /dev/sdb:
# pvremove /dev/sdb

Topics: AIX, Networking

Etherchannel failover testing

When an Etherchannel has been configured on AIX, using a primary and a backup adapter for failover purposes, it is possible to force a failover between these adapters.

To do this, first check what the currently active adapter is within the Etherchannel. For example, if the Etherchannel is called ent4, run:

# entstat -d ent4 | grep "Active channel"
Active channel: backup adapter
As you can see, the Etherchannel is currently active on the backup adapter. To see the defined adapters for both primary and backup, run:
# lsattr -El ent4
Now, force it to fail over to the primary adapter:
# /usr/lib/methods/ethchan_config -f ent4
Please note that it is best to run this command from the console, as you may temporarily lose network connectivity when the Etherhannel is failing over. You may also notice messages being logged in the error report by running the errpt command in the form of "ETHERCHANNEL FAILOVER".

Now run the entstat command again to determine the active channel:
# entstat -d ent4 | grep "Active channel"
Active channel: primary channel

Topics: Red Hat / Linux, System Admin


XRDP is an Open Source Remote Desktop Protocol server, very similar to what is used on Windows Server systems, but XRDP is meant for Linux. Once installed on Linux, you can set up a RDP (or Remote Desktop Connection) session from a Windows system directly to a Linux system.

Here's how you install and configure it on RHEL or CentOS 7:

First of all, we need to install the EPEL repository and XRDP server:

# yum -y install
# yum -y install xrdp
Next, we need to start and enable the service:
# systemctl start xrdp.service 
# systemctl enable xrdp.service
To check if its running, run:
# netstat -an | grep 3389 
tcp   0    0*   LISTEN
That's all. Now you can connect to your server from any Windows machine using RDP.

Topics: Security

Setting up an Access Point using hostapd

This is the fifth article in a series of articles on security awareness, focusing on WiFi security.

In this article, we'll be setting a (rogue) Access Point, which can be very useful for man-in-the-middle attacks.

What's needed?

  • A computer running Kali Linux. See: ali Linux Bootable USB drive.
  • A few wireless interfaces, preferably three. One will be used for regular Internet access. One will be used for monitoring WiFi traffic. And the last one will be used for the Access Point. For the purposes of having a good signal, a wireless device that has sufficient power and one that can be switched into managed mode is required. A device that is typically used for this purposes is the Alfa AWUS036NH (2.4 GHz only) or Alpha AWUS052NH (both 2.4 and 5 GHz bands).
  • Good knowledge on WiFi networks, as described in Scanning WiFi networks.
In this article we'll be setting up an Access Point. As you know, Access Points or routers transmit their WiFi network(s) with a SSID or network name. Some of these names can be very common. For example, in the USA, very common network names are "Google Starbucks" or "attwifi". Many people use these open networks provided by Starbucks and AT&T, and often have these WiFi networks saved onto their devices (phones and laptops) and set to connect automatically. We're going to exploit that to our advantage. Any device such as a phone or laptop that knows about a WiFi network, and has it set to connect automatically, will simply indeed connect automatically to the WiFi, if WiFi is turned on on the device, and if a WiFi network with that name is in range.

So, if you've once visited the local Starbucks and used their open and free WiFi hotspot, and saved that network onto your device and have it set to connect automatically (which quite often is the default behavior), and you then visit another Starbucks later on, your device will connect automatically. And that happens only based on the SSID or network name, which in the case of Starbucks is known "Google Starbucks". A hacker can exploit that by setting his/her own Access Point with the same name.

First step: Install the required software

For the rogue Access Point to work, we'll need 2 items:
  • hostapd - the Access Point software
  • dnsmasq - a DHCP server that will provide IP address to any clients
Start up the Kali Linux system, and once logged in as user root, connect the system using one of the WiFi interfaces to the Internet using an existing Wifi Network. You can do so by navigating to the top right of the screen and connect a WiFi network. You can of course, also use a cabled Internet connection.

Then, install the required software:
# apt-get update
# apt-get install dnsmasq hostapd

Step 2: Set up the Access Point

Fist figure out which wireless network device you will be using for the Access Point. Run the "iwconfig" command to see the available wireless network devices. Use a wireless network device that can be switched into Managed mode, and one that has sufficient power. You may even attempt to increase the transmit power, as described here. For this article, we'll assume the wireless network device we'll use for Access Point is wlan1. If you have trouble determining which wireless network device maps to which physical device, try unplugging any USB wireless network devices, and plugging them in one by one, and running "iwconfig" in between.

Set up the configuration file for "hostapd", which defines the Access Point:
ssid=Google Starbucks
In the example above, an Access Point is defined with the network name "Google Starbucks", and it will run on channel 4, using wireless network device wlan1.

Save the file as hostapd.conf.

Within the same folder, start the Access Point:
# hostapd hostapd.conf
If, at this point, you see any errors, then most likely, another process is keeping the wireless network device occupied, usually the Network Manager process. In this article it was already described how to exclude a wireless network device from being managed by NetworkManager. Follow these steps, and you'll see that hostapd can be run without any issues.

When hostapd is running, check on another computer or phone if you can indeed discover a wireless network with the SSID "Google Starbucks".

Setting up a DHCP server

For our Access Point to work, we'll need it to act as a DHCP server as well, by handing out IP addresses to clients that will connect to the Access Point. Without that, the clients won't be able to do anything. For this purpose, we'll be using dnsmasq.

Again, we're going to create a configuration file, this time called dnsmasq.conf. Open a new terminal window and create the file:
This configuration file will tell dnsmasq to use the wlan1 device, and start handing out IP addresses in the range of through It will set the subnet mask to It will provide a DHCP lease of 12 hours (12h). We're setting up DNS servers as well by using Google's DNS servers and And we're going to log everything (which by default will be through rsyslog in /var/log/messages).

Save the file. Next, we'll need to configure an IP address on the wireless interface wlan1, so clients can actually communicate to it:
# ifconfig wlan1 up netmask
# route add -net netmask gw
Now start the DHCP server:
# dnsmasq -C dnsmasq.conf -d
You may be able to see what the DHCP server is doing by monitoring the screen, or in a separate terminal window, run a command like:
# tail -f /var/log/messages | grep dnsmasq

Next step: Forward traffic

We're not quite there yet. Clients can now discover a WiFI network, and can get an IP address assigned by our Access Point. But in order for the clients to not suspect anything, we'll need to provide Internet access. If you've configured Internet access on one of your wireless network devices, as described in the beginning of this article, then you can do that. Let's assume the wireless network device begin used for the Internet access is wlan0. So wlan0 provides the regular Internet connectivity and wlan1 is being used by the Access Point. If you need to identify which wireless network is used for the Internet access, run "iwconfig". That command will show you if a wireless network device is associated to any SSID/WiFi network, and if so, which one.

Now forward the traffic:
# iptables --table nat --append POSTROUTING --out-interface wlan0 -j MASQUERADE
# iptables --append FORWARD --in-interface wlan1 -j ACCEPT
# echo 1 > /proc/sys/net/ipv4/ip_forward
This will accept any network traffic from wlan1, and forward it to wlan0.

Next: Connect!

Now connect another device such as a phone or laptop to the wireless network called "Google Starbucks". It should connect normally and without any issues. In fact, you may even discover, if that device already knew about the Google Starbucks network, that it connected all by itself.

Next: Start monitoring

The Access Point has now been set up. The DHCP server is handing out IP addresses to any clients that connect, and Internet access is provided to any clients. Effectively, the Access Point now becomes the man-in-the-middle, and all network traffic will flow through the Access Point, and can be monitored. In the terminal screen running "hostapd", you can see if any clients connect. In the terminal window running "dnsmasq" you can see any clients that receive an IP address from the DHCP server, and any DNS queries that these clients do (these are also logged in /var/log/messages). Viewing the DNS queries of any clients will give you some idea of what the user is doing on their device. If you want to see what DHCP IP addresses have been assigned (or "leased") to clients, then run:
# cat /var/lib/misc/dnsmasq.leases
Now sit and wait for any clients to connect. Preferably do this in a location where many people stop by, such as your local coffee shop or a work location. You'll see more and more devices connect by themselves, especially if the location doesn't have any other open WiFi hotspots nearby. If you really want to start discovering what clients are doing, you can monitor their network traffic by running Wireshark, which can provide information about all network packets, but that's beyond the scope of this article.

Also, hostapd allows for multiple Access Points to be set up at the same time. So, if you wish to run two WiFi networks, such as "Google Starbucks" and "attwifi", you can do that too. But again, that's beyond the scope of this article. Should you need to discover what are common WiFi network names in your area, then in yet another terminal window, run the airodump-ng command as described in Scanning WiFi networks. In the bottom section of the output of airodump-ng you can see any client (or "station") information, and also see what names of WiFi networks these clients are probing for. This gives you a good idea about which wireless network names are commonly known on clients, and you can use that information when setting up the Access Point, by listing that network name (SSID) in the hostapd.conf configuration file for hostapd.

Finally: How to protect yourself from all this

You've now gotten a good idea of why using open networks might allow others to see what you're doing. The next obvious question will be: How do you protect yourself from this? Well, there are some options:
  • Do not use open WiFi networks/hotspots.
  • If you do use an open WiFi network/hotspot, then remain aware that others can see what you're doing, so hold off on checking your bank account or entering password anywhere.
  • Visit only websites that are secure - that are using "https:". This means any website traffic is encrypted, and can't be viewed, even when using a packet sniffer like Wireshark. Do not use any websites that are not secure (that are using "http:" instead of "https:".
  • If you do use an open WiFi network/hotspot, then make sure to disconnect afterwards, and to not have it set to connect automatically when in range.
  • Use a VPN when connected to a open WiFi network/hotspot. This will encrypt all your network traffic. Keep in mind though that DNS queries are never encrypted, and can still be seen by someone else.

Number of results found: 454.
Displaying results: 1 - 10.