Monitoring page faults (page out)

Page-outs can be a sign of trouble. When the kernel detects that memory is running low, it attempts to free up memory by paging out. Though this may happen briefly from time to time, if page-outs are plentiful and constant, the kernel can reach a point where it's actually spending more time managing paging activity than running the applications, and system performance suffers. To monitor page-outs in the system, we can use vmstat. Below script display the amount of free memory for page-out. Ideally it should be zero.

#!/bin/bash
/usr/bin/vmstat | head -3 | tail -n +3 | awk {'print $8'}

[root@fc21 ~]# pageout
0

Monitoring swap space

To monitor the swap space available in the system, we can use free command. Below script display the swap space available in the system.

[root@fc21 ~]# cat /usr/local/sbin/swap
#!/bin/bash
/usr/bin/free | head -3 | tail -n +3 | awk {'print $3'}

[root@fc21 ~]# swap
1648

Monitoring ram

To monitor the free ram available in the system, we can use free command. Below script display the available ram.

#!/bin/bash
/usr/bin/free | head -2 | tail -n +2 | awk {'print $4'}

[root@fc21 ~]# ram
135720

Monitoring active tcp connections

Active tcp connections number brings us light on whether the system can serve the request fast. A high number indicate queuing/processing of requests. This number can be retrieved from netstat.

#!/bin/bash
tcp=$(/bin/netstat -nt | tail -n +3 | grep ESTABLISHED | wc -l)
printf "%s \n" "$tcp"

[root@fc21 ~]# tcp
5

Monitoring diskspace

To monitor the amount of hdd space used in the system, we can use df. Below script display the percentage the disk is used.

[root@fc21 ~]# cat /usr/local/sbin/disk
#!/bin/bash
/bin/df -t ext4 -m | awk {'print $5'} | tail -1

[root@fc21 ~]# disk
10%

Monitoring system load

System load can be obtained from Uptime command. Below script is used to display the last min system load number.

[root@fc21 ~]# cat load
#!/bin/bash
uptime | awk '{print $10}' | awk -F, '{print $1}'

[root@fc21 ~]# load
1.43

httpd: segmentation fault (GDB)

During one of my php programming, I encountered a httpd segmentation fault. The error was shown in error_log.

[root@fc21 ~]# tail -f /var/log/httpd/error_log

[Fri Jan 09 08:31:50.859976 2015] [core:notice] [pid 6757] AH00052: child pid 6775 exit signal Segmentation fault (11)

I decided to use GDB to see where the segmentation fault occur.

[root@fc21 ~]# systemctl stop httpd
[root@fc21 ~]# systemctl status httpd

To start GDB.

[root@fc21 ~]# gdb httpd
(gdb) r -DONE_PROCESS
Program received signal SIGSEGV, Segmentation fault.
0x00007fffe7f431e8 in _zend_mm_free_int () from /etc/httpd/modules/libphp5.so

And it displayed the fault occured in zend_mm_free_int. Proceed with a "where".

(gdb) where
#0  0x00007fffe7f431e8 in _zend_mm_free_int ()
   from /etc/httpd/modules/libphp5.so
#1  0x00007fffe7f629ce in convert_to_double ()
   from /etc/httpd/modules/libphp5.so
#2  0x00007fffe0d20ea1 in zif_trader_stoch (ht=,
    return_value=0x7ffff7f257f8, return_value_ptr=,
    this_ptr=, return_value_used=)
    at /root/trader-0.3.0/functions/trader_stoch.c:67
#3  0x00007fffe7f5bf2a in dtrace_execute_internal ()
   from /etc/httpd/modules/libphp5.so
#4  0x00007fffe8019800 in zend_do_fcall_common_helper_SPEC ()
   from /etc/httpd/modules/libphp5.so
#5  0x00007fffe7fa9dd0 in execute_ex () from /etc/httpd/modules/libphp5.so
#6  0x00007fffe7f5bdc8 in dtrace_execute_ex ()
   from /etc/httpd/modules/libphp5.so
#7  0x00007fffe7f6e7c0 in zend_execute_scripts ()
   from /etc/httpd/modules/libphp5.so
#8  0x00007fffe7f0a6cb in php_execute_script ()
   from /etc/httpd/modules/libphp5.so
#9  0x00007fffe801b3c2 in php_handler () from /etc/httpd/modules/libphp5.so
#10 0x00005555555942f0 in ap_run_handler ()
#11 0x0000555555594839 in ap_invoke_handler ()
#12 0x00005555555a966a in ap_process_async_request ()

To continue, enter "cont".

(gdb) cont
Continuing.

Program terminated with signal SIGSEGV, Segmentation fault.

GDB is a useful tool to identify where fault lies. Maybe you could be one to raise a bug in php/httpd/linux with this. And for this problem, I actually found out that I need to cast a decimal in a php variable.

array_push($high,number_format($obj["high"],2,'.',''));