Doesn’t matter if you’re a system administrator or a normal user, as it is very important to keep your computer infrastructure and network running smoothly. Hence you need a reliable system monitoring tool that will help you keep track of all the system activities like CPU performance, Memory Usage, Network Monitoring, and status of all the connected devices.

There are many choices available on the internet for system monitoring tools, and we have crafted a list of the best system monitoring tools for you by testing each of them in different circumstances. So, sit back and enjoy the ride to find the best system monitoring tool for Ubuntu, which matches your requirements.

1. Htop

Htop is a cross-platform system monitor, process viewer and manager, and reliable alternative to top, which is also a system monitoring tool for Linux and its distros. It is specially designed and developed for consoles and terminals; hence it supports text mode.`

It is a feature-rich system monitoring tool that can be used on Linux, FreeBCD, OpenBCD, and macOS. Talking about features, it offers information based on various parameters such as tasks, load average, and uptime. You can change color preferences on its UI to match your requirements.

For Linux and its distros, it provides a delay account matrix and offers support for custom scripts and real-time signals. As it is open-source and free, it makes it one of the best system monitoring tools out there for Linux systems.

2. Glances

Written in Python, Glances is another cross-platform system monitoring tool on our list. It uses a web-based interface to give you the maximum system information in the minimum possible space. Depending on terminal size, it automatically adjusts itself and displays all the information in a single window.

It can also be used in client/server mode and remote system monitoring, which could be done through a web interface or terminal. You get all the important information in one place is one of the positives of this tool.

The thing I like most about this system monitoring tool is that you can keep track of using its web interface, which allows remote monitoring. Linux running on low-end or older computers might find it tough to run this tool smoothly as it demands higher CPU resources.

Download Here

3. Stacer

Stacer is an open-source system monitor and optimization tool which helps system administrators manage system resources and task under on roof. It is a modern tool with an excellent user interface that makes you feel at home even on first use.

It is a feature-rich tool that lets you manage startup apps, clean unnecessary package caches, crash reports, application logs, application caches, and trash under the system cleaner tab. It starts or stops services easily, sort processes based on process id (PID), CPU, and memory usage, and find a particular process easily using its name in the search bar and uninstall applications, which are not required anymore.

Resource tab displays CPU, RAM, Disk, CPU load average, and network activity for the last 60 seconds, and it also comes with APT repository manager, which you can use to activate, disable, or delete any repository. Ubuntu users can use this feature to edit the package repositories.

4. BashTOP

BashTOP is another cool and reliable system monitoring tool for Linux and its distros such as Ubuntu. It displays the usage stats for processor, memory, disks, network, and other resources.

It is an excellent tool for desktop and computer users who are generally personal users. However, the system administrators and server users won’t find this tool that useful as their demands will be higher. Also, it is a little bit slower as compared to other system monitoring tools such as htop.

It is easy to use tool and sports a beautiful user interface with everything placed perfectly.

5. GNOME System Monitor

It is a simple system monitoring tool that comes pre-installed on various Linux distros running GNOME desktop environment. This tool shows which programs are running, how much processor time, memory, and disk space being used.

As you can see in the screenshot, it has a clean and simple user interface. Every information and stats are placed perfectly in the user interface, which makes it easy to read and understand.

The CPU history tab shows how much processor capacity is being used for each CPU, the Memory and history tab shows how much of your computer’s memory (RAM) is being used, and under the network tab, you see the download and upload speed of the network over the last 60 seconds.

6. Netdata

The Netdata is a cross-platform system monitoring tool that has a very attractive and modern user interface, which is eye-catching. It is a great tool for personal as well as enterprise use for managing large computer infrastructures.

It offers timely insightful health alarms and interactive visualizations you can troubleshoot slowdowns and anomalies in your server infrastructures. It is a simple to use tool that can be deployed in minutes. Despite being completely free to use, this tool is quite reliable and powerful system monitoring tool.

Download Here

So, these are the best system monitoring tools you can use on your computers running Linux and its distros. There are some other tools available for Linux, but the ones listed above are deeply tested and then presented to you. Feel free to share your views at @linuxhint and @SwapTirthakar. ]]> https://linuxhint.com/swap_memory_linux/ Sun, 21 Jun 2020 17:55:02 +0000 https://linuxhint.com/?p=61865 For every operating system, there is a dedicated amount of RAM available that makes the processing of a program possible. However, the amount of this RAM is limited which is why RAM cannot hold a bulk of data in it. Therefore, there should be a backup option available which can support RAM whenever it runs out of memory.

This concept holds for the Windows operating system as well as for Linux. In Windows OS, whenever RAM has an insufficient amount of memory to hold a process, it borrows some amount of memory from the secondary storage. This borrowed memory is known as Virtual Memory. Similarly, whenever RAM runs out of memory in Linux, it borrows some memory from the secondary storage to store its inactive content.

In this way, the RAM finds sufficient space to hold a new process within it. Here, the borrowed space from the hard disk is called Swap Memory. In this article, we will try to learn the concept of swap memory in detail.

Working of Swap Memory:

As explained above, swap memory is the dedicated amount of hard drive that is used whenever RAM runs out of memory. There is a memory management program in Linux that takes care of this process. Whenever RAM is short of memory, the memory management program looks for all those inactive blocks of data present in RAM that have not been used for a long time.

When it successfully finds those blocks, it shifts them into the swap memory. In this way, the space of RAM is freed up and hence it can be utilized for some other programs that need processing on an urgent basis. The concept of swapping is very much similar to the concept of paging used in the Windows operating system.

Types of Swap Memory:

Typically there are two different types of swap memory which are mentioned below:

• Swap Partition- This is the default type of swap memory which is in fact, a hard drive partition that is dedicated to swapping.
• Swap File- This is a self-created type of swap memory. Whenever there is no sufficient amount of space left in the hard drive to create a swap partition, a swap file is manually created for swapping the inactive contents of RAM into it.

What should be the Ideal Frequency of Swapping?

Linux allows us to set the frequency of swapping on our own i.e. how frequently the process of swapping should take place. You can set the value of swapping between 0 and 100 depending upon your requirements. A low-frequency value of swapping means that the process of swapping will take place very rarely only when it is needed whereas a high-frequency value of swapping means that the swapping process will occur quite often. However, the default and recommended value of swapping frequency is 60.

Benefits of using Swap Memory:

By learning the working of swap memory, we can easily perceive the benefits of using it. However, some of the main benefits of using swap memory are listed below:

• It can easily hold those inactive blocks of RAM that are hardly used once or twice and then they are never used. The freed up RAM can then be used to hold more programs that have a higher priority.
• It prevents RAM from running out of space.
• It acts as a backup for enhancing the actual space of RAM.
• It allows you to run heavy applications more conveniently that require a large amount of RAM.
• During the process of hibernation, all the contents of RAM are written on the swap memory. Therefore, it is essentially required for the hibernation process to take place successfully.
• It enhances the overall performance of your system.

Conclusion:

In this article, we have learned the usage and working swap memory along with its numerous benefits. Swap memory acts as a backup option for RAM when it runs short of space. We all know that we cannot have an infinite amount of RAM however; we do realize that today’s high-end applications require a large amount of RAM to operate smoothly. Therefore, we must have a sufficient amount of RAM to avoid our applications from crashing.

Also, there is a cost associated with adding in more RAM whereas there is no cost of using swap memory. Moreover, additional RAM can also be plugged in up to a certain limit depending upon your hardware. Hence, the only option we are left with is to use swap memory which can make our system work very efficiently without any cost.

How vm.min_free_kbytes works

Memory allocations may be needed by the system in order to ensure proper functioning of the system itself.  If the kernel allows all memory to be allocated it might struggle when needing memory for regular operations to keep the OS running smoothly.  That is why the kernel provides the tunable vm.min_free_kbytes.  The tunable will force the kernel’s memory manager to keep at least X amount of free memory.   Here is the official definition from the linux kernel documentation: “This is used to force the Linux VM to keep a minimum number of kilobytes free.  The VM uses this number to compute a watermark[WMARK_MIN] value for each lowmem zone in the system. Each lowmem zone gets a number of reserved free pages based proportionally on its size. Some minimal amount of memory is needed to satisfy PF_MEMALLOC allocations; if you set this to lower than 1024KB, your system will become subtly broken, and prone to deadlock under high loads. Setting this too high will OOM your machine instantly.“

Validating vm.min_free_kbytes Works

In order to test that the setting of min_free_kbytes is working as designed, I have created a linux virtual instance with only 3.75 GB of RAM.  Use the free command below to analyze the system:

Looking at the free memory utility above using the -m flag to have the values printed in MB.  The total memory is 3.5 to 3.75 GB of memory.  121 MB of memory is used, 3.3 GB of memory is free, 251 MB is used by the buffer cache.  And 3.3 GB of memory is available.

Now we are going to change the value of vm.min_free_kbytes and see what the impact is on the system memory.  We will echo the new value to the proc virtual filesystem to change the kernel parameter value as per below:

You can see that the parameter was changed to 1.5 GB approximately and has taken effect.  Now let’s use the free command again to see any changes recognized by the system.

The free memory and the buffer cache are unchanged by the command, but the amount of memory displayed as available has been reduced from 3327 to 1222 MB.  Which is an approximate reduction of the change in the parameter to 1.5 GB min free memory.

Now let’s create a 2GB data file and then see what reading that file into the buffer cache does to the values.  Here is how to create a 2GB data file in 2 lines of bash script below.  The script will generate a 35MB random file using the dd command and then copy it 70 times into a new data_file output:

Let’s read the file and ignore the contents by reading and redirecting the file to /dev/null as per below:

Ok, what has happened to our system memory with this set of maneuvers, let’s check it now:

Analyzing the results above.  We still have 1.8 GB of free memory so the kernel has protected a large chunk of memory as reserved because of our min_free_kbytes setting.  The buffer cache has used 1691 MB, which is less than the total size of our data file which is 2.3 GB.  Apparently the entire data_file could not be stored in cache due to the lack of available memory to use for the buffer cache.  We can validate that the entire file is not stored in cache but timing the repeated attempts to read the file. If it was cached, it would take a fraction of a second to read the file.  Let’s try it.

The file read took almost 20 seconds which implies its almost certainly not all cached.

As one final validation let’s reduce the vm.min_free_kbytes to allow the page cache to have more room to operate and we can expect to see the cache working and the file read getting much faster.

With the extra memory available for caching the file read time dropped from 20 seconds before to .364 seconds with it all in cache.

I am curious to do another experiment.  What happens with malloc calls to allocate memory from a C program in the face of this really high vm.min_free_kbytes setting.  Will it fail the malloc?  Will the system die?  First reset the the vm.min_free_kbytes setting to the really high value to resume our experiments:

Let’s look again at our free memory:

Theoretically we have 1.9 GB free and 515 MB available.  Let’s use a stress test program called stress-ng in order to use some memory and see where we fail.  We will use the vm tester and try to allocate 1 GB of memory.  Since we have only reserved 1.5 GB on a 3.75 GB system, i guess this should work.

Let’s try it again with more workers, we can try 1, 2, 3, 4 workers and at some point it should fail.  In my test it passed with 1 and 2 workers but failed with 3 workers.

Let’s reset the vm.min_free_kbytes to a low number and see if that helps us run 3 memory stressors with 1GB each on a 3.75GB system.

This time it ran successfully without error, i tried it two times without problems.  So I can conclude there is a behavioral difference of having more memory available for malloc, when the vm.min_free_kbytes value is set to a lower value.

Default setting for vm.min_free_kbytes

The default value for the setting on my system is 67584 which is about 1.8% of RAM on the system or 64 MB. For safety reasons on a heavily thrashed system i would tend to increase it a bit perhaps to 128MB to allow for more reserved free memory, however for average usage the default value seems sensible enough.  The official documentation warns about making the value too high.  Setting it to 5 or 10% of the system RAM is probably not the intended usage of the setting, and is too high.

Setting vm.min_free_kbytes to survive reboots

In order to ensure the setting can survive reboots and is not restored to the default values when rebooting be sure to make the sysctl setting persistent by by putting the desired new value in the /etc/sysctl.conf file.

Conclusion

We have seen that the vm.min_free_kbytes linux kernel tunable can be modified and can reserve memory on the system in order to ensure the system is more stable especially during heavy usage and heavy memory allocations.  The default settings might be a little too low, especially on high memory systems and should be considered to be increased carefully.  We have seen that the memory reserved by this tunable prevents the OS cache from using all the memory and also prevents some malloc operations from using all the memory too.

How Linux File System Cache Works

The kernel reserves a certain amount of system memory for caching the file system disk accesses in order to make overall performance faster.  The cache in linux is called the Page Cache. The size of the page cache is configurable with generous defaults enabled to cache large amounts of disk blocks.  The max size of the cache and the policies of when to evict data from the cache are adjustable with kernel parameters.  The linux cache approach is called a write-back cache.  This means if data is written to disk it is written to memory into the cache and marked as dirty in the cache until it is synchronized to disk.  The kernel maintains internal data structures to optimize which data to evict from cache when more space is needed in the cache.

During Linux read system calls, the kernel will check if the data requested is stored in blocks of data in the cache, that would be a successful cache hit and the data will be returned from the cache without doing any IO to the disk system.  For a cache miss the data will be fetched from IO system and the cache updated based on the caching policies as this same data is likely to be requested again.

When certain thresholds of memory usage are reached background tasks will start writing dirty data to disk to ensure it is clearing the memory cache.  These can have an impact on performance of memory and CPU intensive applications and require tuning by administrators and or developers.

Using Free command to view Cache Usage

We can use the free command from the command line in order to analyze the system memory and the amount of memory allocated to caching.  See command below:

What we see from the free command above is that there is 7.5 GB of RAM on this system.  Of this only 209 MB is used and 6.5 MB is free.  667 MB is used in the buffer cache.  Now let’s try to increase that number by running a command to generate a file of 1 Gigabyte and reading the file.  The command below will generate approximately 100MB of random data and then append 10 copies of the file together into one large_file.

Now we will make sure to read this 1 Gig file and then check the free command again:

We can see the buffer cache usage has gone up from 667 to 1735 Megabytes a roughly 1 Gigabyte increase in the usage of the buffer cache.

Proc Sys VM Drop Caches Command

The linux kernel provides an interface to drop the cache let’s try out these commands and see the impact on the free setting.

We can see above that the majority of the buffer cache allocation was freed with this command.

Experimental Verification that Drop Caches Works

Can we do a performance validation of using the cache to read the file? Let’s read the file and write it back to /dev/null in order to test how long it takes to read the file from disk.  We will time it with the time command.  We do this command immediately after clearing the cache with the commands above.

It took 8.4 seconds to read the file.  Let’s read it again now that the file should be in the filesystem cache and see how long it takes now.

Boom!  It took only .2 seconds compared to 8.4 seconds to read it when the file was not cached.  To verify let’s repeat this again by first clearing the cache and then reading the file 2 times.

It worked perfectly as expected.  8.5 seconds for the non-cached read and .2 seconds for the cached read.

Conclusion

The page cache is automatically enabled on Linux systems and will transparently make IO faster by storing recently used data in the cache.  If you want to manually clear the cache that can be done easily by sending an echo command to the /proc filesystem indicating to the kernel to drop the cache and free the memory used for the cache.  The instructions for running the command were shown above in this article and the experimental validation of the cache behavior before and after flushing were also shown.

Moreover, Linux being open-source has allowed there to be much fewer vulnerabilities, bugs and threats to look out for. However, what if we could even improve it more? We’ll be looking at some of these steps today that will surely let you have a smoother and faster Ubuntu system.

Step 1: Keeping your Ubuntu updated

One of the most important things to a speed up is to have the latest Ubuntu version installed. This is because Canonical, the developers of Ubuntu, continually release new updates for Ubuntu which are designed to fix common bugs as well as improve the overall system performance and efficiency. With the latest update, Ubuntu will have most of the issues in the old version sorted out and will be able to run much more smoothly. To have the latest Ubuntu updates, first open the terminal through either Ubuntu Dash or Ctrl+Alt+T shortcut and enter the following command:

Moreover, to check for any major upgrades available, run the following command:

It is always important to back up your files before proceeding with this task.

Step 2: Managing Startup Applications

Linux uses your system memory as sparingly as possible. However, over time, as you install more and more applications, your system can slow down due to a huge number of these applications starting and taking huge resources as soon as your system boots up. In Ubuntu, handling these startup applications is very easy.

Go to the Ubuntu Dash and search for Startup Applications. Over here you’ll have the options to Add, Remove and Edit as you like. Click on any option that seems unnecessary to load every time you boot up. If you’re unsure, you can simply remove the check mark next to the option instead of removing it. Please see the image below for a better understanding:

Step 3: Setting the best Mirrors for Software Updates

To get faster updates from Ubuntu Center, it is always required to use the best server. Since Ubuntu software repositories are mirrored across the world, the best server is usually the one closest to your location. To select this option, go to Ubuntu Dash and search for Software and Update. Next click on the Ubuntu Software tab, then click on the drop down option on Download from and choose Other.

Now click on Select the Best Server.

Now this will start a test to find out which is the server closest to you so that it updates the files much faster.

Step 4: Enabling Proprietary Drivers

One of the best things about Ubuntu is that it itself identifies and installs the drivers that are needed on your system. It is to be noted that there are two types of drivers available: open source ones and the proprietary ones. By default, Ubuntu installs the open source drivers and in some cases, that causes problems. This is where Ubuntu excels as it is aware of these issues and allows you to install additional proprietary drivers. To set this up, go to Ubuntu Dash and search for Software and Update. Next click on the Additional Drivers tab and from here, you will be able to see all the devices for which drivers can be installed.

As shown in the image above, you will also get the option to either use the driver or to not use the device at all. To install the drivers, select it and click on Apply Changes. After the installation is complete, you will get a restart option. Click on it to restart your system and finish driver installation. The driver will be in use after the reboot.

Step 5: Install Preload

Preload is a tool that runs in the background and keeps track of the user’s most frequently used applications. And using this data, it tries to predict what applications the user might run next and ensures that they’re the first to load on login, which reduces your start-up time significantly. To install preload, simply run the following command in the terminal:

Once installation is complete, simply reboot your system and it will automatically start running in the background and improve your startup speed.

Step 6: Using Apt-fast Instead of Apt-get

Apt-get is one of the most important Ubuntu commands. It is basically a command line tool for installing or updating any packages in your system. Apt-fast is something of a packet that attaches itself to apt-get and accelerates the upgrade of the system and downloading of packages from the network. For installation of apt-fast, we run the following commands:

Step 7: Reducing Overheating of the System

One of the major problems that systems have to face is the overheating problem. When your system gets hot, it runs slower and gives a poor performance. A very effective tool in the Ubuntu repository for this problem is TLP which helps in cooling down your system and making it run faster and smoother. In order to install TLP, run the following commands in your terminal:

When the installation is done, run the following command to start it.

TLP will start running in the background afterwards.

Step 8: Cleaning Up Apt-Cache

Ubuntu uses Apt for installing, managing & removing software’s on the system which results in cache of previously downloaded and installed packages being stored, even after their uninstallation. This package cache can quickly become large and eat up your space. To clean the apt cache, simply run the following command:

Step 9: Removing Unnecessary Packages

When you install software on Ubuntu, it also automatically installs libraries and packages that may need to satisfy the dependencies of the installed package. However, if that package is uninstalled, these dependencies still exist on your system and aren’t needed anymore. To remove them, simply run the command:

This basically removes all the unnecessary packages or unwanted dependencies from your system.

Step 10: Cleaning up the Thumbnail Cache

Ubuntu automatically creates a thumbnail and stores these in a hidden directory at ~/.cache/thumbnails. With time, these thumbnails increase dramatically and so will the thumbnail cache. You can check the size of the thumbnail cache with the command below:

As seen by the image above, my system has a thumbnail cache about 2 Mb in size. To remove these, simply enter the command:

A Faster, Efficient Ubuntu

Linux has slowly garnered a name for it being a fast and efficient operating system. But it’s not flawless. Even it can slow down or become sluggish over time. Using any of the steps that we’ve covered, either applied in isolation or all of them combined, can surely help to speed up your system and save resources for the tasks that matter most to you.

Fast-cli

Fast-cli is a command line utility to check download and upload speeds of your network. It is based on Fast.com website made by Netflix and a series of tests are run on Netflix’s own production servers.

To install Fast-cli on Ubuntu, run the commands below:

To check network speed using Fast-cli, run the command below:

Speedtest-cli

Speedtest-cli is a command line utility that uses speedtest.net to check network bandwidth speed. It can check both download and upload speeds as well as server response time.

To install Speedtest-cli in Ubuntu, run the commands below:

To perform a speed test using speedtest-cli, run the command below:

Wget

Wget is a command line download manager that can fetch files using HTTP, HTTPS and FTP protocols.

To install wget in Ubuntu, run the command below:

To perform a download speed test (without upload), run the command below:

The command above uses free speed test service provided by Linode for testing their servers. You can replace “newark” part in the command above with any other server listed on Linode’s speed test page available here.

Youtube-dl

Youtube-dl is a command line utility to download YouTube files in various audio and video formats. It is possible to check download speed by downloading a YouTube video using youtube-dl.

To install youtube-dl in Ubuntu, run the command below:

To perform a speedtest using youtube-dl, run the following command:

The command above downloads a video from KDE Community YouTube channel. You can replace the YouTube URL with your own. The “/dev/null” part ensures that video is not saved anywhere on the filesystem.

The biggest advantage of using youtube-dl to perform a speedtest is its built-in option to bypass geo restrictions. To download a video from a different country, use the command below:

Replace the “US” part with any other ISO 3166-2 country code.

Curl

Curl is a command line utility that allows data transfer using URLs. These URLs are not limited to HTTP protocol only, as curl supports many other standards. Curl is used a lot for testing and interacting with RESTful APIs.

To install curl in Ubuntu, run the command below:

To perform a speedtest using curl, use the following command:

The command above uses Digital Ocean servers to test the network speed. You can replace the URL with any other Digital Ocean server listed here.

Note that curl displays speed in KB/sec. To convert it to MB/sec (similar to wget’s output), you have to multiply the result with 0.001. The speed in the screenshot above would be 6794 KB/sec * 0.001 = 6.794 MB/sec.

Aria2

Aria2 is a command line download manager for Linux. It can pause and resume downloads from servers that support resuming. It also provides an option for multi-threaded downloading.

To install aria2 in Ubuntu, run the command below:

To perform a speedtest using aria2, run the command below:

Ignore the error shown in the screenshot above. The command above uses Digital Ocean’s server to do a speedtest (as explained earlier). You can replace the URL with any other Digital Ocean server listed here.

Conclusion

These are some of the methods you can use to test network speed using command line apps. If you are just looking to test download speed, wget is pretty straightforward to use. For testing both download and upload speeds, use fast-cli or speedtest-cli app.

This article will list various utilities to perform CPU benchmarks and stress tests on your system. While many of these apps provide options to test other hardware components as well, this article will focus on CPU tests only.

HardInfo (Benchmark Only)

HardInfo is a graphical utility for Linux that displays information about various hardware components in your PC like processor, RAM modules, sensors, hard drives and so on. It also supports exporting a detailed report about your system in txt or html format so that you can share it with your friends, compare it with other systems or use it for debugging purposes.

Besides showing general information about your system, HardInfo provides a few benchmark tools to measure the performance of your hardware. As far as CPU benchmarking is concerned, it includes five methods: CPU Blowfish, CPU CryptoHash, CPU Fibonacci, CPU N-Queens and CPU Zlib.

To install HardInfo in Ubuntu, run the command below:

To run a CPU benchmark, click on any benchmark method under “Benchmarks” section, as shown in the screenshot below.

Once the benchmark has finished, HardInfo will tell you if a lower or higher score is better. It also displays scores for CPUs made by other vendors for easy comparison.

Sysbench (Benchmark Only)

Sysbench, as the name suggests, is a command line app to run benchmarks on your system. Written in Lua, Sysbench is mainly intended for doing database benchmarking. However it includes options to test CPU, memory and file throughput as well.

To install Sysbench in Ubuntu, run the command below:

To start a benchmark using Sysbench, run the following command:

Where:

• “cpu” is the test type
• “threads” is the number of threads created by sysbench to perform multi-threaded benchmarking

You can use other command line switches to customize the benchmark behavior. More details about these options can be found here.

Once the test has finished, you can use the “events per second” variable to gauge the performance of your CPU. Unfortunately unlike HardInfo, Sysbench does not provide benchmark results of other CPU models made by various hardware vendors out there. If you have multiple computers, you can run the test on both and then compare the numbers. Higher is the number of events per second, better is the CPU performance.

Stress (Stress Test Only)

Stress is a simple command line utility to conduct CPU, memory and disk input-output stress tests.

You can install stress in Ubuntu by running the following command:

Note that Stress will not run any benchmark test on your system, instead it will consistently hammer the CPU and make it run at 100%. You are advised to close all the running apps before doing a test. Running Stress for a longer period can crash the system and you may need to do a hard reboot to recover the system.

You can start the stress test by running the command below where “cpu” refers to the number of threads used for performing the test. To stop the test anytime, close the terminal or press <CTRL + C> to make it stop.

I used Stress extensively when I built a new PC for myself a year back. It helped me greatly in tweaking fan speeds in BIOS for improved cooling.

Stress-ng (Stress Test Only)

Stress-ng is an updated implementation of Stress tool mentioned above. It is much more advanced and includes hundreds of tests to put a load on your system.

You can install Stress-ng in Ubuntu by running the command below:

Note that the warning mentioned above applies to this tool as well. Running stress-ng for a long time can freeze your system making it recoverable through a hard reboot only.

You can start the stress-ng test by running the command below where “cpu” refers to the number of threads used for performing the test. To stop the test anytime, close the terminal or press <CTRL + C> to make it stop.

Running the command above will run around 70 CPU tests sequentially. However, you can run all of these tests in parallel by running the command below:

It is possible to customize the command using many other command line switches, but explaining all of them here is beyond the scope of this article. You can explore these options from here.

S-tui (Stress Test Only)

S-tui is a CPU resource monitor that uses a terminal based graphical interface to display data and graphs.

It supports CPU stress testing using the Stress app listed above while giving you a nice bird’s eye view on the various CPU parameters like frequency, temperature and utilization.

S-tui can be installed in Ubuntu by running the command below:

As it is just a terminal based frontend for Stress command line app, be vary of system lock up during high usage.

Run the following command to launch it. To stop the test anytime, close the terminal or press <CTRL + C> to make it stop.

7-Zip (Benchmark Only)

7-Zip is a file compression tool that can do extreme levels of compression on files and store them in a reduced size 7z archive format. 7-Zip comes with a built-in option to run LZMA compression benchmarks for measuring CPU performance.

To install 7-Zip in Ubuntu, run command:

To run single threaded benchmark, use the command below:

Multi-threaded benchmarking can be run by using the following command:

The results are in million instructions per second (MIPS). You can compare these results with other CPU models from here.

Conclusion

These are some of the free and open source utilities that can run CPU benchmarks on your system. In the good old days, running Flash Player in Firefox was an excellent way to stress test your CPU. These days, however, all the apps mentioned above do a pretty good job.

We can use htop command to see a list of all the running processes of our Linux server. But what if we want to monitor how much disk bandwidth each of these processes are consuming? Well, we can use iotop.

iotop is an interactive I/O monitoring tool on Linux just like htop. With iotop, you can easily monitor disk read and write bandwidth usage for each running processed on Linux.

In this article, I am going to show you how to install iotop on popular Linux distributions and how to use iotop to monitor disk read and write bandwidth usage per running process. I will be using Ubuntu 18.04 LTS for the demonstration, but the same commands should work on any modern Linux distributions. So, let’s get started.

Installing iotop on Ubuntu:

iotop is available in the official package repository of Ubuntu. So, you can easily download iotop using the APT package manager.

First, update the APT package repository cache with the following command:

Now, install iotop with the following command:

iotop should be installed.

Now, check whether iotop is working with the following command:

iotop is working as you can see.

Installing iotop on CentOS 7:

iotop is available in the official package repository of CentOS 7. You can easily install it with the YUM package manager as follows:

Basic Usage of iotop:

To monitor the disk usage of each running processes with iotop, run iotop as follows:

As you can see, the iotop interactive window is opened. Here, you can see which process is using the disk.

As you can see on the screenshot below, on the top left side, the total disk read speed/bandwidth is displayed. The same way, on the top right side, the total disk write speed/bandwidth is displayed.

As you can see, iotop displays columns for,

• Thread ID (TID).
• I/O Priority class/level (PRIO).
• the owner of the thread with TID (USER).
• the disk read per second (DISK READ).
• the disk write per second (DISK WRITE).
• the percentage of time the thread spent while swapping in (SWAPIN).
• the percentage of time the thread spent waiting on I/O (IO>).
• the command the thread is running (COMMAND).

For most of the cases, the Thread ID (TID) is equivalent to Process ID (PID).

Displaying Processes Doing I/O Operations Only:

By default, iotop shows all the running processes whether they are doing I/O operations or not. So, the list is very long. It is harder to find the processes we need and monitor them.

Luckily, iotop lets you display only the processes that are doing I/O operations. To do that, you can use the -o or –only option of iotop.

To display only the processes doing I/O operations, run iotop as follows:

Or

As you can see, the process with TID 1345 and 1957 are doing I/O operations right now.

This is the command that I am running to simulate I/O operations in this article.

NOTE: If you haven’t started iotop with the -o or –only option, you can still switch to this mode by pressing the o key on your keyboard. You can toggle between these 2 modes of iotop using the o key.

Displaying Total I/O Usage Per Process:

iotop also lets you monitor how much disk reads and disk writes in total each processes have done after iotop started as well. To do that, you have to use the -a or –accumulated option. You can also use it with the -o or –only option as well.

For example,

As you can see, the total disk reads and disk writes per process is displayed.

Displaying PID instead of TID:

As I’ve stated earlier, Thread ID (TID) is the same as Process ID (PID) most of the time. You can use these interchangeably. But if you really want to be sure that you’re working with exactly process ID (PID), then iotop has the -P or –processes option that you can use to change the default TID column to PID column.

To display PID column instead of TID column, run iotop as follows:

Or

As you can see, the TID column is replaced by the PID column.

Filtering iotop Processes:

You can filter iotop processes based on process ID (PID), thread ID (TID) and the processed owner (USER).

For example, if you want to monitor the disk I/O of the processes with the PID 2024 and 2035 only, you would run iotop as follows:

As you can see, only the processes with the PID 2024 and 2035 are being monitored.

If you wanted to monitor the disk I/O of the threads with the TID 2024 and 2035 only, you would run iotop as follows:

Again, let’s say, you want to monitor the disk I/O of the processes running as the user shovon. To do that, run iotop as follows:

As you can see, only the processes running as the user shovon is displayed.

If you want to monitor multiple users at once, you can do that as well.

For example, to monitor disk I/O of all the processes that the users shovon and lily are running, run iotop as follows:

So, that’s basically how you use iotop to monitor disk I/O in Linux. Thanks for reading this article.

Given how involved the process of benchmarking is, and how important it is when making a decision. We need some standard set of tools that we can use to benchmark our systems, get a simple to understand result and use it to compare different hardware components and configurations effectively.

Here are a few free benchmarking tools that you cover a wide array of hardware and use cases.

1. Indigo Benchmark — For rendering and content creation

Now that the PC and desktop computing war is waging at an all time high between AMD and Intel and also AMD and Nvidia, this benchmark is strongly recommended. This benchmark can be used to test both your CPU and GPU to the utmost for certain workloads like video rendering and content creation.

The reason it is first on the list is because it is cross-platform. You can install it on macOSX, Windows and, of course, Linux. The crossplatform nature of the software can also help you choose the best operating system for your rig, on top of letting you compare various hardware options.

2. Phoronix Test Suite

Phoronix offers a more complete set of tools for benchmarking nearly any aspect of your system. Moreover, it is completely open source and not just free to use. The framework it offers is extensible and can accomodate any number of different tests that you may want to see your system perform. It is extremely powerful, flexible and useful for both sysadmins as well as desktop enthusiasts.

Moreover, the official website for Phoronix offers a very in depth analysis of the benchmarking procedures, in case you are new to this field. Their latest post detailing the impact of spectre and meltdown mitigation patches on your system’s performance is something I can personally recommend.

3. IOzone

Perhaps not the foremost consideration while building a PC or a server, your SSDs are important. Faster SSDs lead to snappier systems. The reason is quite simple. Modern CPUs and memory are fast enough that once a program or a data reaches them it can then quickly be read or executed.

Secondary storage, like your SSDs, are major bottlenecks. The longer it takes for information to reach your CPU, the slower your experience will be. IOzone lets you have a really close peek at how your storage is doing. Sequential reads, sequential writes as well as random IOPs are to be considered to select your perfect SSD.

Workloads like video streaming can benefit from higher sequential reads whereas databases can really benefit from higher random IOPs. So storage benchmarking is never isn’t as simple as running dd to a disk.

4. Web Latency Benchmark

We have talked a lot about storage and compute, that leaves out one thing and that is networking. While there are a ton of tools for network engineers to benchmark and monitor their networking their 10G interfaces, I wanted to talk about a different layer of networking altogether.

The Web Latency Benchmark is a benchmark for your web browser from Google. This cross-platform benchmark is quite useful when comparing the real-world performance of your web browser. Things like delay between keystrokes and browser responses, scroll latency and jank and a few other things are measured by the benchmark.

Browsers are something we spend a lot of time working at, if the performance between Firefox and Chrome differs even in the slightest, it is worth the time to benchmark them and pick the better one.

5. 7-zip

Yes, the archival tool 7-zip comes with its own benchmarking tool built into it. If your workload involves a lot of compressing and uncompressing. Then this is benchmark is really worth considering.

You can take this tool even further, things like running password brute force attack or dictionary attack are all possible using 7-zip. If you want to see the difference between your CPU and GPU when handling these kind of workloads (which can be multithreaded easily), 7-zip has a lot to offer.

Conclusion

Before you start running benchmarks on your own system, I would highly encourage you to check out PassMark software’s website and just try and infer what the different CPU benchmarks show and reflect. There’s multithreaded score, single threaded score and different CPUs work at different clock-speeds. In short, there is quite a bit of variation.

Try and picture yourself as someone trying to pick one of the CPUs for their own build, how would you decide which one is better for you? Good benchmarks should answer these questions for you.

In this article, I will show you some of the common usages of the dd command in Linux. I will be using Debian 9 Stretch for the demonstration. But it should work on any other Linux distribution. Let’s get started.

Making a Bootable USB Drive with dd:

Making a bootable USB drive of your favorite operating system with the dd command is very easy. All you need is a USB drive and an ISO or IMG image of the operating system that you want to make a bootable USB of.

Let’s assume, you have downloaded an ISO image of Alpine Linux and the file is saved to the ~/Downloads directory as alpine-standard-3.8.0-x86_64.iso

Now, you can list all the connected storage or block devices with the following command:

As you can see, all the connected storage or block devices are listed. Here, /dev/sdb is my USB drive. It has two partitions, /dev/sdb1 and /dev/sdb2. But when you make USB bootable drives, you have to use the /dev/sdb, the whole block device, not any partitions.

Now make bootable USB drive of Alpine Linux with the following command:

Here, if=~/Downloads/alpine-standard-3.8.0-x86_64.iso option is used to tell dd that the input file is in the path ~/Downloads/alpine-standard-3.8.0-x86_64.iso and of=/dev/sdb option is used to tell dd that the output file is in the path /dev/sdb. The bs=1M tells dd to read from ~/Downloads/alpine-standard-3.8.0-x86_64.iso and write to /dev/sdb 1 Megabytes of data at a time.

As you can see, the ISO file is copied to the block device /dev/sdb. Now you can use it to install Alpine Linux.

This command is very destructive. The dd command wipes the partition table and other metadata, flags from the block device. So you must be careful.

Displaying Progress Bar:

By default, the dd command does not show any progress bar. But you can tell dd to show it with the status=progress option.

For example, to copy data from /dev/sda to /dev/sdb 1 Megabytes at a time and also show the progress bar, run the following command:

As you can see, the progress bar is displayed. You can see how much of the data is copied and the rate at which it is being copied.

Measuring Read and Write Performance of a Storage Device with dd:

You can measure the read and write speed of a storage device with dd very easily. Of course, there are many graphical softwares that do provide this information, but the command line lovers would find this very interesting.

First, you have to mount the partition or the storage device on your filesystem. If you don’t have a partition on your storage device, you can always create it with the fdisk command and format it to your desired filesystem (such as FAT32, EXT4, NTFS, XFS etc). Here, I assume that you do have a partition /dev/sdb1 and it is formatted as EXT4.

Let’s say, you want to mount /dev/sdb1 partition to /mnt directory, then run the following command:

As you can see, the partition /dev/sdb1 is mounted on /mnt directory.

Now let’s create a 1 GB file testrw in the /mnt directory with dd:

Here, count=1 means, read bs=1G which is 1 Gigabyte from /dev/zero, and write it to /mnt/testrw file.

The oflag=direct option is used to disable disk caching. If disk caching is enabled, you will not get very accurate results.

NOTE: Remember, for this operation, you must have at least 1 GB of free memory or RAM on your computer. If you can’t afford that much free RAM, then reduce the bs size. For example, set bs=128M or even less, bs=64M.

As you can see, the write speed of my USB drive is about is about 6.1 MB per second.

You can also test the read speed of your storage device with the following command:

As you can see, I can read at 4.3 MB per second.

Testing Storage Device Latency with dd:

The latency of a storage device is the time it takes to access the device. It is an important parameter that we can determine with the help of the dd command.

To test for latency, we can write or read small chunks of data (about 512 bytes at a time) X times and see how long it takes. Then we can calculate how long it takes to read or write a single chunk of data very easily. This is called the latency of the storage device.

For example, let’s say you want to calculate the write latency. Now run the following command to write 512 bytes chunk about 1000 times:

As you can see, it takes about 16.4541 seconds to write 1000 chunks of 512 bytes data. Now, to write a single chunk of data, it takes about (16.4541s / 1000 = 0.0164 s) 0.0164 seconds. So the write latency is about 0.0164 seconds for this storage device.

You can calculate the read latency the same way.

So that’s how you use dd and test performance of an I/O device with it. Thanks for reading this article.

You might wonder why you would need a tool like this as there are plenty of good performance analysis tools available by default on most Linux distributions. Every install includes tools like top and vmstat, and tracing utilities like strace are usually just an apt-get away. Where does OProfile fit in?

The tools previously mentioned are excellent at obtaining a snapshot of a Linux system in real time. Tools like top or htop show all running processes, their current memory consumption, and processor usage. But knowing what processes and system calls are consuming the most resources becomes problematic.

That’s where OProfile comes in. This utility suite not only performs its analysis at a deeper level, but also saves data and allows you to produce performance reports that offer a wealth of information that can help you debug even the most elusive performance issue.

OProfile is not just for developers. In a desktop environment, OProfile can help you track down CPU-intensive background tasks or I/O calls that are slowing you down and aren’t immediately evident. On a busy system with shifting process priorities, this data can be hard to collect, let alone interpret. The multi-process nature of a server environment makes this task even more difficult with traditional tools.

That said, developers will no doubt get the most use out of OProfile. The information I’ll present will cover the basics of both use cases so you can dig into the performance metrics of any Linux program.

Installation

There is a very important note that must be made before diving deeply into OProfile – you may not be able to install it in a virtualized environment. If you are running Linux inside a VirtualBox, VMWare, or similar VM environment, OProfile may not be able to access the necessary performance counters to collect data. Furthermore, even if you are able to use it in a virtual environment, precise timing may be somewhat distorted based on host system load, so please keep this in mind if you aren’t running on native hardware.

Several Linux distributions have OProfile in their package management systems, making installation easy:

• Debian / Ubuntu / Linux Mint – sudo apt-get install oprofile
• Fedora / CentOS – sudo yum install oprofile
• Arch – sudo pacman -S oprofile

A Simple Example

Once the program is installed, let’s get our feet wet with a trivial yet useful example. The program “ls” is a command you probably use all the time. It simply displays a list of files and folders in the current directory. Let’s trace its output:

sudo operf ls

You’ll see something similar to the above screen shot. Once the profiler is finished, it will announce “Profiling done.” It has saved it’s data in a folder called oprofile_data which can be used to generate a report.

Running the command opreport (without sudo in this case) produces a report similar to this:

In this example, the default report shows the number of samples when the CPU was not in a HALT state (in other words, was actively doing something). Kallsyms provides symbol lookup used by the profiler, and the ld.so and libc.so are part of the glibc package, a common library linked into nearly all Linux executables that provides basic functionality developers can use to keep from reinventing the wheel and provide a generic level of compatibility between various systems. You can see that the actual program ls had far less non-HALT time – the bulk of the heavy lifting was done by the standard libraries.

Once we’re done with the report, it’s a good idea to either remove the data folder or save it for future analysis. In this example, we’ll just remove it since we’re running sample exercises. Since we ran the command with sudo, we must remove the folder with sudo. Be careful!

sudo rm -Rf oprofile_data

A More Complex Example

In this next example, we’ll run a program that actually does something more complex than just list files in the current folder. Let’s download WordPress with wget.

sudo operf wget http://wordpress.org/latest.tar.gz

After this example, we can generate a report with the “opreport” command:

You’ll see a lot more activity after this one. The wget command had to do a lot of work behind the scenes to obtain the latest copy of WordPress. Though it’s not necessary to examine each item, the interesting points of interest are:

• ath9k and ath9k_hw – These modules are responsible for the WiFi connection on this laptop.
• mac80211 and cfg80211 – These libraries were instrumental in performing the network connection required by wget.
• libnss_dns and libresolv were used in resolving the wordpress.org domain into an IP address so wget could make an HTTP connection.
• libcrypto and libssl – These libraries are part of the OpenSSL library. This performed the work to decode the received data from the https:// url. Note that even though we specified a URL with http://, the WordPress server redirected us to https:// and wget followed this redirect.
• libpthread – This library performs threading operations which allow programs to do multiple things at once. In this case, wget started a thread to download the program and also provide an ASCII-based download progress indicator on the screen.

This kind of data can provide a wealth of information for a developer. But how is this important to a system administrator of a server or a power user on a desktop? By knowing which parts of a program are taking the most CPU time, we can find out what needs optimization or where the slowdown is occurring, allowing us to make better decisions about how to optimize our system.

In this example, the most CPU time was taken by the crypto/SSL routines. This is understandable because cryptography is a time consuming task. Had the wordpress.org website not redirected us to https:// this library would not have been used, saving us CPU time. The network layer would still have been used, but using a wired connection instead of a wireless connection would likely have been less taxing. Disabling the progress indicator on the wget program (via the -nv switch) would have saved CPU time in displaying download progress.

Digging Into Symbols

Even though the default report provides valuable and useful information, we can dig further. By running this:

opreport --demangle=smart --symbols

We can find out exactly how much CPU time functions in the libraries consumed:

In this example, I used the wget command above but used an http:// URL (one that doesn’t redirect to https://) and you can see the absence of OpenSSL libraries in the trace. However, instead of just the library name, we now have a full listing of the functions involved. As you can see, the network layer consumed most of the CPU non-HALT time.

Taking it to the Next Level

In the previous examples we’ve used OProfile to take a look at one program at a time. You can examine your entire system at once using the –system-wide switch:

sudo operf --system-wide

Using this technique, OProfile will gather statistics in the same manner and stop when you hit CTRL+C. Afterwards, you can run the opreport command. Since the profiler will likely generate much more data (especially on a desktop or busy server).

opreport > report.txt

The report is now viewable in a file called report.txt

Low Overhead

It is important to note that while OProfile shouldn’t interfere with the operation of your programs, it will create a bit of overhead and thus slow down execution. In our simple examples above it didn’t create a problem, but on a program with long execution and extensive function calls you will likely notice a difference. Because of this, I wouldn’t recommend using this program in a production server environment unless faced with a critical performance problem that must be solved with live usage. Even then, I would use it just long enough to find the issue.

Conclusion

OProfile is a powerful performance profiling tool. It taps into the lowest level available in Linux to obtain performance counters and metrics that give you valuable information about your programs.

Gone are the days of guesswork in performance debugging – you now have the power to know precisely what your system is doing and how to improve it. By studying the reports generated by OProfile, you can make informed, data-driven decisions on optimizing your system.