20 Linux System Monitoring Tools

Every SysAdmin Should Know

Need to monitor Linux server performance? Try these built-in command and a few add-on tools. Most Linux distributions are equipped with tons of monitoring. These tools provide metrics which can be used to get information about system activities. You can use these tools to find the possible causes of a performance problem. The commands discussed below are some of the most basic commands when it comes to system analysis and debugging server issues such as:

1. Finding out bottlenecks.
2. Disk (storage) bottlenecks.
3. CPU and memory bottlenecks.
4. Network bottlenecks.

#1: top - Process Activity Command

The top program provides a dynamic real-time view of a running system i.e. actual process activity. By default, it displays the most CPU-intensive tasks running on the server and updates the list every five seconds.

Fig.01: Linux top command

Commonly Used Hot Keys
The top command provides several useful hot keys:

Hot Key           Usage
t                 Displays summary information off and on.
m                 Displays memory information off and on.
A                 Sorts the display by top consumers of various system resources. Useful for quick identification of performance-hungry tasks on a system.
f                 Enters an interactive configuration screen for top. Helpful for setting up top for a specific task.
o                 Enables you to interactively select the ordering within top.
r                 Issues renice command.
k                 Issues kill command.
z                 Turn on or off color/mono

=> Related: How do I Find Out Linux CPU Utilization?

#2: vmstat - System Activity, Hardware and System Information

The command vmstat reports information about processes, memory, paging, block IO, traps, and cpu activity.

# vmstat 3

Sample Outputs:

procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0      0 2540988 522188 5130400    0    0     2    32    4    2  4  1 96  0  0
 1  0      0 2540988 522188 5130400    0    0     0   720 1199  665  1  0 99  0  0
 0  0      0 2540956 522188 5130400    0    0     0     0 1151 1569  4  1 95  0  0
 0  0      0 2540956 522188 5130500    0    0     0     6 1117  439  1  0 99  0  0
 0  0      0 2540940 522188 5130512    0    0     0   536 1189  932  1  0 98  0  0
 0  0      0 2538444 522188 5130588    0    0     0     0 1187 1417  4  1 96  0  0
 0  0      0 2490060 522188 5130640    0    0     0    18 1253 1123  5  1 94  0  0

Display Memory Utilization Slabinfo

# vmstat -m

Get Information About Active / Inactive Memory Pages

# vmstat -a

=> Related: How do I find out Linux Resource utilization to detect system bottlenecks?

#3: w - Find Out Who Is Logged on And What They Are Doing

w command displays information about the users currently on the machine, and their processes.

# w username

# w vivek

Sample Outputs:

 17:58:47 up 5 days, 20:28,  2 users,  load average: 0.36, 0.26, 0.24
USER     TTY      FROM              LOGIN@   IDLE   JCPU   PCPU WHAT
root     pts/0    10.1.3.145       14:55    5.00s  0.04s  0.02s vim /etc/resolv.conf
root     pts/1    10.1.3.145       17:43    0.00s  0.03s  0.00s w

#4: uptime - Tell How Long The System Has Been Running

The uptime command can be used to see how long the server has been running. The current time, how long the system has been running, how many users are currently logged on, and the system load averages for the past 1, 5, and 15 minutes.

# uptime

Output:

 18:02:41 up 41 days, 23:42,  1 user,  load average: 0.00, 0.00, 0.00

1 can be considered as optimal load value. The load can change from system to system. For a single CPU system 1 - 3 and SMP systems 6-10 load value might be acceptable.

#5: ps - Displays The Processes

ps command will report a snapshot of the current processes. To select all processes use the -A or -e option:

# ps -A

Sample Outputs:

  PID TTY          TIME CMD
    1 ?        00:00:02 init
    2 ?        00:00:02 migration/0
    3 ?        00:00:01 ksoftirqd/0
    4 ?        00:00:00 watchdog/0
    5 ?        00:00:00 migration/1
    6 ?        00:00:15 ksoftirqd/1
....
.....
 4881 ?        00:53:28 java
 4885 tty1     00:00:00 mingetty
 4886 tty2     00:00:00 mingetty
 4887 tty3     00:00:00 mingetty
 4888 tty4     00:00:00 mingetty
 4891 tty5     00:00:00 mingetty
 4892 tty6     00:00:00 mingetty
 4893 ttyS1    00:00:00 agetty
12853 ?        00:00:00 cifsoplockd
12854 ?        00:00:00 cifsdnotifyd
14231 ?        00:10:34 lighttpd
14232 ?        00:00:00 php-cgi
54981 pts/0    00:00:00 vim
55465 ?        00:00:00 php-cgi
55546 ?        00:00:00 bind9-snmp-stat
55704 pts/1    00:00:00 ps

ps is just like top but provides more information.

Show Long Format Output

# ps -Al

To turn on extra full mode (it will show command line arguments passed to process):

# ps -AlF

To See Threads ( LWP and NLWP)

# ps -AlFH

To See Threads After Processes

# ps -AlLm

Print All Process On The Server

# ps ax

# ps axu

Print A Process Tree

# ps -ejH

# ps axjf

# pstree

Print Security Information

# ps -eo euser,ruser,suser,fuser,f,comm,label

# ps axZ

# ps -eM

See Every Process Running As User Vivek

# ps -U vivek -u vivek u

Set Output In a User-Defined Format

# ps -eo pid,tid,class,rtprio,ni,pri,psr,pcpu,stat,wchan:14,comm

# ps axo stat,euid,ruid,tty,tpgid,sess,pgrp,ppid,pid,pcpu,comm

# ps -eopid,tt,user,fname,tmout,f,wchan

Display Only The Process IDs of Lighttpd

# ps -C lighttpd -o pid=

OR

# pgrep lighttpd

OR

# pgrep -u vivek php-cgi

Display The Name of PID 55977

# ps -p 55977 -o comm=

Find Out The Top 10 Memory Consuming Process

# ps -auxf sort -nr -k 4 head -10

Find Out top 10 CPU Consuming Process

# ps -auxf sort -nr -k 3 head -10

#6: free - Memory Usage

The command free displays the total amount of free and used physical and swap memory in the system, as well as the buffers used by the kernel.

# free

Sample Output:

            total       used       free     shared    buffers     cached
Mem:      12302896    9739664    2563232          0     523124    5154740
-/+ buffers/cache:    4061800    8241096
Swap:      1052248          0    1052248

=> Related: :

1. Linux Find Out Virtual Memory PAGESIZE


2. Linux Limit CPU Usage Per Process


3. How much RAM does my Ubuntu / Fedora Linux desktop PC have?

#7: iostat - Average CPU Load, Disk Activity

The command iostat report Central Processing Unit (CPU) statistics and input/output statistics for devices, partitions and network filesystems (NFS).

# iostat

Sample Outputs:

Linux 2.6.18-128.1.14.el5 (www03.nixcraft.in)  06/26/2009

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           3.50    0.09    0.51    0.03    0.00   95.86

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              22.04        31.88       512.03   16193351  260102868
sda1              0.00         0.00         0.00       2166        180
sda2             22.04        31.87       512.03   16189010  260102688
sda3              0.00         0.00         0.00       1615          0

=> Related: : Linux Track NFS Directory / Disk I/O Stats

#8: sar - Collect and Report System Activity

The sar command is used to collect, report, and save system activity information. To see network counter, enter:

# sar -n DEV more

To display the network counters from the 24th:

# sar -n DEV -f /var/log/sa/sa24 more

You can also display real time usage using sar:

# sar 4 5

Sample Outputs:

Linux 2.6.18-128.1.14.el5 (www03.nixcraft.in)   06/26/2009

06:45:12 PM       CPU     %user     %nice   %system   %iowait    %steal     %idle
06:45:16 PM       all      2.00      0.00      0.22      0.00      0.00     97.78
06:45:20 PM       all      2.07      0.00      0.38      0.03      0.00     97.52
06:45:24 PM       all      0.94      0.00      0.28      0.00      0.00     98.78
06:45:28 PM       all      1.56      0.00      0.22      0.00      0.00     98.22
06:45:32 PM       all      3.53      0.00      0.25      0.03      0.00     96.19
Average:          all      2.02      0.00      0.27      0.01      0.00     97.70

=> Related: : How to collect Linux system utilization data into a file

#9: mpstat - Multiprocessor Usage

The mpstat command displays activities for each available processor, processor 0 being the first one. mpstat -P ALL to display average CPU utilization per processor:

# mpstat -P ALL

Sample Output:

Linux 2.6.18-128.1.14.el5 (www03.nixcraft.in)   06/26/2009

06:48:11 PM  CPU   %user   %nice    %sys %iowait    %irq   %soft  %steal   %idle    intr/s
06:48:11 PM  all    3.50    0.09    0.34    0.03    0.01    0.17    0.00   95.86   1218.04
06:48:11 PM    0    3.44    0.08    0.31    0.02    0.00    0.12    0.00   96.04   1000.31
06:48:11 PM    1    3.10    0.08    0.32    0.09    0.02    0.11    0.00   96.28     34.93
06:48:11 PM    2    4.16    0.11    0.36    0.02    0.00    0.11    0.00   95.25      0.00
06:48:11 PM    3    3.77    0.11    0.38    0.03    0.01    0.24    0.00   95.46     44.80
06:48:11 PM    4    2.96    0.07    0.29    0.04    0.02    0.10    0.00   96.52     25.91
06:48:11 PM    5    3.26    0.08    0.28    0.03    0.01    0.10    0.00   96.23     14.98
06:48:11 PM    6    4.00    0.10    0.34    0.01    0.00    0.13    0.00   95.42      3.75
06:48:11 PM    7    3.30    0.11    0.39    0.03    0.01    0.46    0.00   95.69     76.89

=> Related: : Linux display each multiple SMP CPU processors utilization individually.

#10: pmap - Process Memory Usage

The command pmap report memory map of a process. Use this command to find out causes of memory bottlenecks.

# pmap -d PID

To display process memory information for pid # 47394, enter:

# pmap -d 47394

Sample Outputs:

47394:   /usr/bin/php-cgi
Address           Kbytes Mode  Offset           Device    Mapping
0000000000400000    2584 r-x-- 0000000000000000 008:00002 php-cgi
0000000000886000     140 rw--- 0000000000286000 008:00002 php-cgi
00000000008a9000      52 rw--- 00000000008a9000 000:00000   [ anon ]
0000000000aa8000      76 rw--- 00000000002a8000 008:00002 php-cgi
000000000f678000    1980 rw--- 000000000f678000 000:00000   [ anon ]
000000314a600000     112 r-x-- 0000000000000000 008:00002 ld-2.5.so
000000314a81b000       4 r---- 000000000001b000 008:00002 ld-2.5.so
000000314a81c000       4 rw--- 000000000001c000 008:00002 ld-2.5.so
000000314aa00000    1328 r-x-- 0000000000000000 008:00002 libc-2.5.so
000000314ab4c000    2048 ----- 000000000014c000 008:00002 libc-2.5.so
.....
......
..
00002af8d48fd000       4 rw--- 0000000000006000 008:00002 xsl.so
00002af8d490c000      40 r-x-- 0000000000000000 008:00002 libnss_files-2.5.so
00002af8d4916000    2044 ----- 000000000000a000 008:00002 libnss_files-2.5.so
00002af8d4b15000       4 r---- 0000000000009000 008:00002 libnss_files-2.5.so
00002af8d4b16000       4 rw--- 000000000000a000 008:00002 libnss_files-2.5.so
00002af8d4b17000  768000 rw-s- 0000000000000000 000:00009 zero (deleted)
00007fffc95fe000      84 rw--- 00007ffffffea000 000:00000   [ stack ]
ffffffffff600000    8192 ----- 0000000000000000 000:00000   [ anon ]
mapped: 933712K    writeable/private: 4304K    shared: 768000K

The last line is very important:

• mapped: 933712K total amount of memory mapped to files


• writeable/private: 4304K the amount of private address space


• shared: 768000K the amount of address space this process is sharing with others

=> Related: : Linux find the memory used by a program / process using pmap command

#11 and #12: netstat and ss - Network Statistics

The command netstat displays network connections, routing tables, interface statistics, masquerade connections, and multicast memberships. ss command is used to dump socket statistics. It allows showing information similar to netstat. See the following resources about ss and netstat commands:

• ss: Display Linux TCP / UDP Network and Socket Information


• Get Detailed Information About Particular IP address Connections Using netstat Command

#13: iptraf - Real-time Network Statistics

The iptraf command is interactive colorful IP LAN monitor. It is an ncurses-based IP LAN monitor that generates various network statistics including TCP info, UDP counts, ICMP and OSPF information, Ethernet load info, node stats, IP checksum errors, and others. It can provide the following info in easy to read format:

• Network traffic statistics by TCP connection


• IP traffic statistics by network interface


• Network traffic statistics by protocol


• Network traffic statistics by TCP/UDP port and by packet size


• Network traffic statistics by Layer2 address

Fig.02: General interface statistics: IP traffic statistics by network interface

Fig.03 Network traffic statistics by TCP connection

#14: tcpdump - Detailed Network Traffic Analysis

The tcpdump is simple command that dump traffic on a network. However, you need good understanding of TCP/IP protocol to utilize this tool. For.e.g to display traffic info about DNS, enter:

# tcpdump -i eth1 'udp port 53'

To display all IPv4 HTTP packets to and from port 80, i.e. print only packets that contain data, not, for example, SYN and FIN packets and ACK-only packets, enter:

# tcpdump 'tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'

To display all FTP session to 202.54.1.5, enter:

# tcpdump -i eth1 'dst 202.54.1.5 and (port 21 or 20'

To display all HTTP session to 192.168.1.5:

# tcpdump -ni eth0 'dst 192.168.1.5 and tcp and port http'

Use wireshark to view detailed information about files, enter:

# tcpdump -n -i eth1 -s 0 -w output.txt src or dst port 80

#15: strace - System Calls

Trace system calls and signals. This is useful for debugging webserver and other server problems. See how to use to trace the process and see What it is doing.

#16: /Proc file system - Various Kernel Statistics

/proc file system provides detailed information about various hardware devices and other Linux kernel information. See Linux kernel /proc documentations for further details. Common /proc examples:

# cat /proc/cpuinfo

# cat /proc/meminfo

# cat /proc/zoneinfo

# cat /proc/mounts

17#: Nagios - Server And Network Monitoring

Nagios is a popular open source computer system and network monitoring application software. You can easily monitor all your hosts, network equipment and services. It can send alert when things go wrong and again when they get better. FAN is "Fully Automated Nagios". FAN goals are to provide a Nagios installation including most tools provided by the Nagios Community. FAN provides a CDRom image in the standard ISO format, making it easy to easilly install a Nagios server. Added to this, a wide bunch of tools are including to the distribution, in order to improve the user experience around Nagios.

18#: Cacti - Web-based Monitoring Tool

Cacti is a complete network graphing solution designed to harness the power of RRDTool's data storage and graphing functionality. Cacti provides a fast poller, advanced graph templating, multiple data acquisition methods, and user management features out of the box. All of this is wrapped in an intuitive, easy to use interface that makes sense for LAN-sized installations up to complex networks with hundreds of devices. It can provide data about network, CPU, memory, logged in users, Apache, DNS servers and much more. See how to install and configure Cacti network graphing tool under CentOS / RHEL.

#19: KDE System Guard - Real-time Systems Reporting and Graphing

KSysguard is a network enabled task and system monitor application for KDE desktop. This tool can be run over ssh session. It provides lots of features such as a client/server architecture that enables monitoring of local and remote hosts. The graphical front end uses so-called sensors to retrieve the information it displays. A sensor can return simple values or more complex information like tables. For each type of information, one or more displays are provided. Displays are organized in worksheets that can be saved and loaded independently from each other. So, KSysguard is not only a simple task manager but also a very powerful tool to control large server farms.

Fig.05 KDE System Guard {Image credit: Wikipedia}

See the KSysguard handbook for detailed usage.

#20: Gnome System Monitor - Real-time Systems Reporting and Graphing

The System Monitor application enables you to display basic system information and monitor system processes, usage of system resources, and file systems. You can also use System Monitor to modify the behavior of your system. Although not as powerful as the KDE System Guard, it provides the basic information which may be useful for new users:

• Displays various basic information about the computer's hardware and software.


• Linux Kernel version


• GNOME version


• Hardware


• Installed memory


• Processors and speeds


• System Status


• Currently available disk space


• Processes


• Memory and swap space


• Network usage


• File Systems


• Lists all mounted filesystems along with basic information about each.

Fig.06 The Gnome System Monitor application

Bounce: Additional Tools

A few more tools:

MySQL Server Tuning Tips and Tricks




Fine-tuning the settings of servers in general requires a strong grasp of the inner workings of the servers and lots of patience. Did we mention patience? Yup, we did. It?s really important to understand that there are no tips that work for all kinds of setups. But thankfully we can talk about some widely known guidelines that can be followed to get the most out of your MySQL servers.

You are reading the fourth and final part of this MySQL performance optimization multi-part article series. Before we begin, don't forget to check out the earlier parts. Within the first article the most important benchmarking concepts and strategies were pointed out, while the second segment presented benchmarking applications and other useful tools to apply what we have learned and test our own performance. In the previous third article we talked about optimal schema design and query optimizations.

Since this is the last article, we'd actually like to finish the series by covering some of the most powerful server tuning tips and tricks that may affect the performance of your server by far. Consider everything that you're going to read about as hints, and don't take anything for granted. By now you should know how to benchmark your server, notice what works or not, apply tweaks, find slow query bottlenecks, and so forth.

Now that you can't wait to read about the promised server tuning tips, let's begin!

<
MySQL Server Tuning Tips and Tricks - Hardware Specs, Common Mistakes
(

Before we get into the tuning of server settings and discussions of various tips and tricks regarding performance, we should cover briefly the importance of hardware settings-and by this we mean the hardware components of the server on which MySQL service is running. First of all, always try to aim for 64-bit processors along with 64-bit compatible operating systems. x64 is crucial in order to maximize memory efficiency.

As always, having high bandwidth memories can give a powerful boost. The formula of "the more, the better" applies to CPU cache sizes as well. Multiple CPU platforms, SMP, and even multi-core systems are preferred due to their increased scalability (serving multiple queries at the same time on a parallel processing basis).

RAID gives a ton of benefits, but be careful when deciding which kind of level you choose. For example, RAID1 (or RAID10) seems to be an okay choice, while RAID5 tends to become frustrating and too expensive to manage; in that case, the costs exceed the benefits by far. If you pick RAID then set medium-to-high RAID chunk size. Something along the lines of 512KB-1MB should be enough, but as always-test for yourself!

Lastly, when it comes to server performance, some people neglect the quality and throughput of network devices, such as network cards for example. Nowadays gigabit is the trend and they are really worth the effort. It's important that you check whether they are configured and working properly (full duplex?).

Moreover, keep in mind that the operating system is also responsible for the way your MySQL server performs. The server instance per se is a single process and, thus, the OS should be configured to permit really large process sizes. Additionally, enable the --large-pages option in MySQL if the OS you are running has this kind of capability. Also, since tables are quite large (usually) consider using a larger file system block size.

Finally, do various benchmarks on Disk I/O and memory performance because these two strongly affect the performance of your MySQL server. Also, enable and try different schemes of caching, re-benchmark the performance, and compare results. Fine-tune the read-ahead kernel technique. Check this documentation and read more.

Now let's talk a little bit about common mistakes that should be avoided. First of all, don't try to search for an "optimal" or very best MySQL configuration file because, who knows? You may find one that is claimed to be the best, and then you're going to set yourself up for nothing but frustration. However, check their content and see the values of the variables; try to search for the meaning of each in the official documentation.

As soon as you have "demystified" the content of the configuration file, then you can grab only the meaningful ones-here we refer to those that are useful for your own needs and server specs. Oh, and do not even attempt to use the default MySQL configuration. It won't be enough; the default config file is really weak and small.

There are a few commands that you should use to monitor the load and performance of the OS. For example, vmstat gives an overview of processes, virtual memory information, paging, block I/O, CPU usage, and all that; mpstat reports CPU-related statistics and also gives you global averages for each processor/core. Finally, you may want to check out iostat. This command gives you lots of input/output statistics...

Even though MySQL is cross-platform during the aforementioned commands, we've somewhat focused on Linux as the main operating system. If you are using Windows, then try to find third-party applications and/or utilities that report the same information to you. The bottom line is that you should be able to monitor the disk I/O and CPU performance.

MySQL Server Tuning Tips and Tricks - More MySQL Server Tuning
(

On the previous page we've laid the foundation of what is going to be required here.

Here we'll do our best to present some of the configuration variables that you should fiddle around with (meaning try different values until you find one that matches your needs and performs accordingly), a few general guidelines that may serve as "rules of thumb," and ultimately some swapping- and caching- related hints.

First of all, the common scenario is where both Apache and MySQL daemons are being run from the same server. This is not necessarily bad, since having dedicated servers for each daemon sometimes isn't worth the extra investment. However, if you are required to serve lots of simultaneous clients dynamically on a per-connection basis, then you need to configure MySQL accordingly to accommodate the load spikes.

In order to do this we need to grab our access_log Apache log since it contains the ratio of dynamic requests to static requests. The former depend son MySQL queries (such as a web page that works with a MySQL database via PHP commands; actually, it establishes connections, requests the queries, and grabs the data), while the latter has nothing to do with MySQL (no connections need to be opened). They are static.

Let's assume the following real-world scenario. Your Apache daemon is configured to serve a total number of 128 maximum simultaneous connections, of which the ratio of dynamic versus static pages is, say, 1:9. Now let's do the math: 128 / 10 = 12,8. Rounding the amount to 13, we end up with about 13 dynamic pages that "work with" MySQL database(s).

Now comes the fun part. If you are really sure that the ratio is usually around that, then to maintain the overall safety of your server configuration, we can truly assume that doubling 13, meaning 2*13=26, would be a "safe" count of simultaneous database connections to be opened, right? If so, then you can set that amount of max_connections and max_user_connections located in the my.cnf file.

Speaking of the previous two variables, there are a few possibilities regarding the my.cnf configuration file. You need to know that those variables should be located within the [mysqld] section in the conf file. Check that section, since you may already have these variables declared. If so, replace their values with your amount. Make sure they aren't commented; if so-remove comments. If they aren't present at all, then just add them as independent and new entries (set-variable=var_name=X).

After you've become familiar with the my.cnf config file, it's time to talk about the rest of the variables. In this part we won't be so detailed anymore, since there may not be any general guidelines whatsoever. You just need to understand their meaning (what each variable refers to) and then you can predict some "all right" values already.

Memory settings are important, since they have the greatest effect on overall performance. Look for the following variables: key_buffer_size (if you run MyISAM only databases, then setting 30% of total RAM is usually recommended; 4GB is the max limit per key), innodb_buffer_pool_size (if you run InnoDB only databases then setting 70%-80% of total memory is advocated), query_cache_size (32M-512M), read_buffer_size, read_rnd_buffer_size (try values around 1-6MB for the latter two buffer sizes).

Now think a little bit. How frequently do you sort incredibly large sets? If quite often then you may want to increase the value of sort_buffer_size globally right from the config file. If you sort really huge sets only rarely, then it is advisable to leave a smaller buffer size in the config file (global definition) and just change the "sort_buffer_size" right before executing those large sorts.

The aforementioned is important because you will be able to re-set the DEFAULT amount specified globally within the configuration file right after the sorting of those huge sets is done. Having an insanely high buffer size without a real reason will hurt performance.

Finally, set table_cache to at least 1024, and thread_cache to something around 16-64. These values are measured in number of units (number of tables, and how many threads). Pay attention when measurement units are in bytes (M, G, etc.) or just count. If InnoDB is your thing, then check out innodb_log_file_size (512M should be enough but hopefully you can get away with less), and innodb_log_buffer_size (a few megabytes).

A few paragraphs above we mentioned query_cache_size. That variable, if it is set, enables query caching. If you've enabled it, then you should monitor its usage. The command show status like 'qcache%' reports the default values of query-caching-related variables (such as Qcache_<something_goes_here>). You may want to flush status before running the previous command to report the variables.

Now that you know most of the query-caching-related variables, check out list table below to see their usual meaning and some hints that you can find spread over the Web.

Qcache_free_blocks -> number of contiguous memory blocks in the cache. If this value is high, then blocks are fragmented; flush query cache defragments the query cache.

Qcache_free_memory- > free memory in the cache.

Qcache_hits -> incremented each time a query is served from the cache.

Qcache_inserts -> incremented each time a query is inserted. Divide the number of inserts by the hits to get your miss rate; subtract this value from 1 to get your hit rate.

Qcache_lowmem_prunes -> number of times the cache ran out of memory and was purged in order to free it up. Check this value over time; if it's increasing, then chances are either fragmentation is high(er) or memory is quite low.

Qcache_not_cached -> number of queries that weren't candidates for caching.

Qcache_queries_in_cache -> number of queries and responses currently cached.

Qcache_total_blocks -> number of blocks in the cache.

As a final piece of advice, check out this official documentation. It covers lots of variables.

MySQL Server Tuning Tips and Tricks - Final Thoughts
(

Now that you've reached this final page, it means that we're finishing up this series. Hopefully you have found its segments educational and informative. At the least, you should have developed a sense of what high performance MySQL tuning and optimization is about, and know how to imagine optimization "sessions" and such.

We can't stress enough that optimizing MySQL server instances and databases (and generally any kind of server) is a sensitive task since there are just way too many factors. All of them highly depend on your setup, requirements, and data. Different organizations have different purposes and database needs, thus totally different queries are to be answered as responsively as possible, on a variety of server specifications.

Fortunately for us, since MySQL is the most popular open-source cross-platform database solution, there are literally tons of discussions circulating around tweaks and optimizations. Conferences are done all over the world where hundreds of DBAs and experts in their field gather together to share their findings in order to achieve bleeding edge performance from their MySQL runs.

Moreover, this means that there is a lot of information out there spread over the Internet within various blog entries and forum posts, but let's not exclude the dozens of books and tutorials that have been written on the topic. Lots of them are also frequently updated (new tweaks, new releases, etc.).

What you need to develop is the skill to get the most out of that information, adapt everything to your own needs, and you'll reap astonishing rewards. Keep in mind, just because some say that such and such improves performance by a great deal, don't take that statement for granted. Also, be careful because there are tweaks that can alter the behavior of MySQL (are they safe?). Others give performance boosts only randomly.

Summing these up, we truly think that by reading this multi-part article series we have opened a new world of endless possibilities for you. You should learn to love tweaking!

In closing, I'd like to invite you to join our experienced community of technology professionals on all areas of IT&C starting from software and hardware up to consumer electronics at Dev Hardware Forums. As well, be sure to check out the community of our sister site at Dev Shed Forums. We are friendly and we'll do our best to help you.

<

sshpass: Login To SSH Server / Provide SSH Password Using A Shell Script

Q. How do I login over ssh without using password less RSA / DSA public keys? How do I use ssh in a shell script? How do I login non-interactivly performing password authentication with SSH and shell scripts?

A. You can use sshpass command to provide password for ssh based login. From the man page:

sshpass is a utility designed for running ssh using the mode referred to as "keyboard-interactive" password authentication, but in non-interactive mode.

ssh uses direct TTY access to make sure that the password is indeed issued by an interactive keyboard user. Sshpass runs ssh in a dedicated tty, fooling it into thinking it is getting the password from an interactive user.

The command to run is specified after sshpass' own options. Typically it will be "ssh" with arguments, but it can just as well be any other command. The password prompt used by ssh is, however, currently hardcoded into sshpass.

WARNING! These examples considered the least secure as simple ps command can expose password to all users on the same host. I highly recommend using ssh's public key authentication.

Install sshpass under Debian / Ubuntu Linux

Type the following command: $ sudo apt-get install sshpass

How do I use sshpass?

Login to ssh server called server.example.com with password called t@uyM59bQ: $ sshpass -p 't@uyM59bQ' ssh username@server.example.com Under shell script you may need to disable host key checking: $ sshpass -p 't@uyM59bQ' ssh -o StrictHostKeyChecking=no username@server.example.com

How do I backup /var/www/html using rsync?

Run rsync over SSH using password authentication, passing the password on the command line: $ rsync --rsh="sshpass -p myPassword ssh -l username" server.example.com:/var/www/html/ /backup/

Further readings:

• sshpass man page

Below is the URL to download the sshpass source code

...sshpass download

How To Download a RPM Package Using yum Command Without Installing On Linux

Q I would like to only download the packages via yum and not install/update them. How do I download a RPM package using yum command under CentOS Enterprise Linux server 5.x or RHEL 5.x systems?

A. You need to install plugin called yum-downloadonly. This plugin adds a --downloadonly flag to yum so that yum will only download the packages and not install/update them. Following options supported by this plugin:

[a] --downloadonly : don't update, just download a rpm file
[b] --downloaddir=/path/to/dir : specifies an alternate directory to store packages such as /tmp

Please note following instructions are only tested on CentOS server but should work with RHN and RHEL without any problem.

How do I install yum-downloadonly plugin?
Type the following command to install plugin, enter:
# yum install yum-downloadonly

Refer to below link for original post

http://www.cyberciti.biz

Here is the URL for Yum repository RHEL5.X

[root@host test]# cat /etc/yum.repos.d/rhel-debuginfo.repo
# CentOS-Base.repo
#
# This file uses a new mirrorlist system developed by Lance Davis for CentOS.
# The mirror system uses the connecting IP address of the client and the
# update status of each mirror to pick mirrors that are updated to and
# geographically close to the client. You should use this for CentOS updates
# unless you are manually picking other mirrors.
#
# If the mirrorlist= does not work for you, as a fall back you can try the
# remarked out baseurl= line instead.
#
#

[base]
name=CentOS-$releasever - Base
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os
baseurl=http://mirror.centos.org/centos/5.3/os/$basearch/
gpgcheck=0
gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-5

#released updates
[updates]
name=CentOS-$releasever - Updates
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updates
baseurl=http://mirror.centos.org/centos/5.3/updates/$basearch/
gpgcheck=0
gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-5

[extras]
name=CentOS-$releasever - Extras
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=extras
baseurl=http://mirror.centos.org/centos/5.3/extras/$basearch/
gpgcheck=0
gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-5

Transfer a file / folders, compress it midstrem over the network, uncompress the file on the recieving end:

ssh -l root 00.00.000.000 '(cd /opt/uploads/ && tar -czf - . -C /opt/uploads)' | tar -xzf -

command to compare two directories in linux recursively

[user@host tmp]$ cat compare.sh

#!/bin/bash

prefix1='/tmp/directory1/messaging' # First directory without trailing /

prefix2='/tmp/directory2/messaging' # Second directory without trailing /

find -L $prefix1 -type f | while read filename; do

name="${filename#$prefix1*}"

sum1="$(md5sum -b $prefix1$name)"

sum2="$(md5sum -b "$prefix2$name")"

if [ "${sum1% *}" = "${sum2% *}" ]; then

echo "ok: $prefix1$name" >> /dev/null

#echo ""

else

echo "not ok: $prefix1$name"

diff $prefix1$name $prefix2$name

fi

done