Useful command line little tricks (part 4)

It's been a while, but I think it's time for yet another instance of these series. Last time I used an entire post to talk about lshw. Today I want to talk about commands that help in understanding what the system is doing and running diagnostics.

FINGER

Very nice command which displays who logged in to the system.

finger -l

This would return something like the following:

Login: user1   Name: John
Directory: /home/user1     Shell: /bin/bash
On since Fri Mar 12 12:40 (CET) on tty1   4 seconds idle
(messages off)
No mail.
No Plan.

Login: user2        Name: Jane
Directory: /home/user2     Shell: /bin/bash
On since Fri Mar 12 08:21 (CET) on tty7 from :0
4 hours 20 minutes idle
On since Fri Mar 12 11:26 (CET) on pts/0 from :0.0
No mail.
No Plan.

DMESG

Print Kernel information from its ring buffer. Specially interesting for diagnostics and understanding boot problems.

dmesg > boot.log

dmesg returns loads of information, most of which may not be interesting if you are trying to troubleshoot something specific. We can try the usual filtering to narrow down the content. For example, if I was interested in saving the full command output into a log file for later reviewing, but wanted to display on screen information about my wireless connection activity (wlan0 in my case), that would go something like this:

dmesg | tee boot.log | grep wlan0

As usual, creativity is key as you will need to use different command options and filters depending on what you are trying to achieve.

LOGS

As with most systems, the use of logs is frequently the best way to understand what is not working correctly. As we learned in the first article from these series, Linux stores log messages under /VAR/LOG. You can take a quick look to see what you can find in there:

ls -lh /var/log | less

By listing contents this way we can get a better understanding of them, as we see owners, groups, masks, sizes, permissions, etc. Many applications log their messages in here: dpkg, xorg, aptitude, etc. Even more importantly, you will see some logs which are system logs: syslog, boot, dmesg, kern and others display information related to system processes and services. You may have noticed that there are folders as well: apt, samba, cups, gdm...

Probably this is a bit of an overkill, so let's try to cover some of the most important or commonly used:

SYSLOG

Pretty much self explanatory, this log contains information about system activity. It is constantly login data while the system is up and running. You may have noticed how the system keeps a history of these logs, which are eventually compressed to save space (thus the *.gz extension).

As I mentioned, this system log is constantly adding data as system events occur. If we opened the file, we would only get a snapshot of what was happening at that very moment.

less /var/log/syslog

This allows us to scroll through the contents of that snapshot at command line level. If the system has been up and running for some time, you will surely realise that there is a lot of information in there!

less /var/log/syslog > system.log

Therefore, you may want to store that snapshot into a file and then open it with your text editor of choice, which should provide more of a friendly interface, as well as making the content a bit easier to read and work with.

tail -n20 -f /var/log/syslog

A very nice thing we can do is monitor this log in real time, as managed by the command right above. This is specially interesting when you are troubleshooting something you can control (like plugin a device, reproducing an application problem, etc), for you can see exactly what the system logs when the problem happens.

Let's say we are having issues with a external USB drive that is underperforming, maybe taking too long for read/write operations. Because PCs nowadays usually have several USB ports, and some are not as fast as others, it may be interesting to find out how the system recognises our external device. Now, let's run the same command before we plug in our device:

tail -n20 -f /var/log/syslog

Once we see its output in real time, let's plug in the USB drive. Here's the output I got:

Mar 19 16:54:52 KarmicKoala kernel: [17846.861024] usb 1-4: new high speed USB device using ehci_hcd and address 2
Mar 19 16:54:52 KarmicKoala kernel: [17847.328590] usb 1-4: configuration #1 chosen from 1 choice
Mar 19 16:54:52 KarmicKoala kernel: [17847.328865] scsi5 : SCSI emulation for USB Mass Storage devices
Mar 19 16:54:52 KarmicKoala kernel: [17847.328939] usb-storage: device found at 2
Mar 19 16:54:52 KarmicKoala kernel: [17847.328942] usb-storage: waiting for device to settle before scanning
Mar 19 16:54:57 KarmicKoala kernel: [17852.328183] usb-storage: device scan complete
Mar 19 16:54:57 KarmicKoala kernel: [17852.328818] scsi 5:0:0:0: Direct-Access     MEM      Drive Mini Metal 0.00 PQ: 0 ANSI: 2
Mar 19 16:54:57 KarmicKoala kernel: [17852.329346] sd 5:0:0:0: Attached scsi generic sg5 type 0
Mar 19 16:54:57 KarmicKoala kernel: [17852.333553] sd 5:0:0:0: [sdd] 31588352 512-byte logical blocks: (16.1 GB/15.0 GiB)
Mar 19 16:54:57 KarmicKoala kernel: [17852.334162] sd 5:0:0:0: [sdd] Write Protect is off

Once again, lots of interesting information here. We can see how my USB drive was recognized as a high speed device, the available disk space capacity, information about write protection, etc.

Obviously, this is an example, there is nothing here that looks concerning about this USB drive. If we had a problem though, this would be a good way to find out lots of info that could potentially help in troubleshooting.

CUPS

This folder contains logs detailing printer access and/or errors.

GDM

This folder stores logs from the GNOME display manager, which manages our login screen themes among other things.

XORG

The Xorg logs show information from this server, which would give us a start in troubleshooting problems with video (resolution, crashes, etc).

APT & APTITUDE

Having issues with any of these package managers? Look here first.

DIST-UPGRADE

As already discussed in other posts, upgrading to a new release could get painful. If the case, the information found here may prove useful.

CONCLUSION

I think you will agree about the relevancy of this logging feature. Moreover, this is a very important element in the system when we are trying to troubleshoot potential issues. /VAR/LOG certainly does not store every single log in existence, so you may need to look elsewhere for a specific application you have downloaded, but it offers tons of good information you may find very useful.

In fact, if you come from the Windows world, you may get a kick out of being able to get so much information from your system. Being able to actually "get" why something is not working as it should and eventually get to fix it is nice for a change.

Finally it feels like you (not your computer) are the one in control!

Enjoy!