Check disk drive for badblock,errors

badblocks is used to search for bad blocks on a device (usually a disk partition). Device is the special file corresponding to the device (e.g /dev/sda1). It can be a good idea to periodically check for bad blocks. This is done with the badblocks command. It outputs a list of the numbers of all bad blocks it can find. This list can be fed to fsck to be recorded in the filesystem data structures so that the operating system won’t try to use the bad blocks for storing data. The following example will show how this could be done. The command:

badblocks -v /dev/sda1 > badblocks.txt 

The above command will generate the file badblocks.txt. You can pass this file to the fsck command to record these bad blocks. Do make sure you type in the correct filesystem, etc ext3,ext4,xfs.

fsck -t ext3 -l badblocks.txt /dev/sda1

Reference: Link1

Setup SVN (Subversion) on Fedora / RHEL

Apache Subversion (often abbreviated SVN) is a software versioning and revision control system distributed as free software under the Apache License. Developers use Subversion to maintain current and historical versions of files such as source code, web pages, and documentation. Its goal is to be a mostly compatible successor to the widely used Concurrent Versions System (CVS).

This instructions will help you installing SVN server.

Install PHP and Apache Packages

yum install httpd php php-devel php-cli php-pear

Start Apache web server and setup for it to autostart on system boot

yum install httpd php php-devel php-cli php-pear
service httpd restart
chkconfig httpd on

Install svn using yum

yum install mod_dav_svn subversion

Configure Subversion (subversion.conf)

vi /etc/httpd/conf.d/subversion.conf

LoadModule dav_svn_module modules/mod_dav_svn.so
LoadModule authz_svn_module modules/mod_authz_svn.so

<Location /svn>
DAV svn
SVNParentPath /svn/repos/projectX
AuthType Basic
AuthName "Subversion Repository"
AuthUserFile /etc/svn-auth-file
Require valid-user
</Location>

Create SVN Repository

cd /svn/repos/projectX
svnadmin create svnrepo
chown -R apache:apache svnrepo

Create SVN Users
Following commands will create two users for svn.

htpasswd -cb /etc/svn-auth-file admin9 pass9
htpasswd -b /etc/svn-auth-file user8 pass8

If you are wondering why I display the password on commandline. I actually hit a bug. My httpd version is 2.4.4. See this Link1 Link2

If you have httpd 2.4.6 and above you can use below command.

htpasswd /etc/svn-auth-file user7

Access Your Repository in Browser

Open using a browser to access the repository.
http://192.168.0.5/svn/svnrepo/

Enter user name and password in browser.

Checkout Files to Your Repository

svn co http://192.168.0.5/svn/svnrepo/

Add and Checkin Files to Your Repository

cd svnrepo
vi fileA.txt 
vi fileB.txt
svn add fileA.txt fileB.txt
svn ci fileA.txt fileB.txt -m "First commit"

Access http://192.168.0.5/svn/svnrepo/ url in browser, you will see your new files there.

Convert html to pdf on Fedora

wkhtmltopdf is a linux program that convert a webpage(html) to pdf format. The installation instruction of wkhtmltopdf are as below.

yum install wkhtmltopdf

To convert, simply enter as below:

wkhtmltopdf http://google.com test.pdf

However it will errored as it needs a X server:

wkhtmltopdf: cannot connect to X server

To solve the problem, we can install:

yum install xorg-x11-server-Xvfb

Create a shell script as below:

vi wkhtmltopdf.sh

And save with these parameters:

xvfb-run -a -s "-screen 0 640x480x16" wkhtmltopdf $*

And this is the way to run in commandline

wkhtmltopdf.sh http://www.google.com test.pdf

ZEN Loadbalancer

Recently I just came across an article on Zen loadbalancer. Its a Debian ISO, ready to install. The installation and configuration is easy, and straight forward. Below is the feature list taken from its website. After installation, you can configure the LB from its https gui page. Config and testing can be done within minutes after installation.

Features
 Advanced Layer7 load balancing
 Max 30000 concurrent connections in TCP. *
 sNAT load balancing load balancing
 Balance TCP or UDP services
 Balance dataline communications
 HTTP and HTTPS services special options
 HTTP/S persistence client session enabled through cookie, header, basic, ip, url
 SSL wrapper / offload
 Wide range of load balance algorithms like: round robin, weight, priority or hash
 Persistence client sessions
 VLAN Tagging (802.1Q)
 Advanced network configuration for physical, virtual or VLAN interfaces
 Independent route tables for every physical or VLAN NICs
 Advanced checking for backend servers through FarmGuardian
 High availability load balancer service through an active-pasive cluster
 Optional configuration backups system
 Advanced global status with graphs
 Easy administration over https GUI and ssh
 Virtual service configurations can be edited and tuned on-the-fly
 Use NTP sync
 Easy and free updates over APT repositories
 Configure virtual servers and farms as your hardware allows
 Advanced system monitoring with graphs
 Management of SSL certificates
 Real Time syncronization between cluster nodes

*hardware depend

mysqlslap benchmark ext3,ext4,xfs on CENTOS6

Following previous post of Bonnie Benchmark for ext3,ext4,xfs, we noticed a better performance of ext4. This time, we will look into the performance difference using mysqlslap on the 3 filesystems. The partition steps will be the same as previous: For ext3 fs:

fdisk /dev/sda
mke2fs -t ext3 /dev/sda4
mount /dev/sda4 /bench -t ext3
bonnie++ -d /bench/ -c 5 -s 1G -n 32 -m ext3

For ext4 fs:

fdisk /dev/sda
mke2fs -t ext4 /dev/sda4
mount /dev/sda4 /bench -t ext4
bonnie++ -d /bench/ -c 5 -s 1G -n 32 -m ext4

For xfs fs:

fdisk /dev/sda
/sbin/mkfs.xfs /dev/sda4
mount /dev/sda4 /bench -t xfs
bonnie++ -d /bench/ -c 5 -s 1G -n 32 -m xfs

To move the mysql datadir from "/var/lib/mysql" to "/bench", by editing mysql config file("/etc/my.cnf"). Replace

#datadir                         = /var/lib/mysql
datadir                         = /bench

Copy mysql datafiles to new location,change the owner,start the mysql server.

cp -fr /var/lib/mysql/* /bench
chown mysql:mysql /bench -R
service mysqld start

Create create_table.sql scripts:

vi create_table.sql

Paste below codes.

create table testTable (text1 varchar(50),number1 int)
insert into testTable (text1,number1) values ('ERf56768ZSDR4567343',7894)

Create queries.sql scripts:

vi queries.sql

Paste below codes.

select * from testTable

Run the mysqlslap

mysqlslap --user=root --password --concurrency=50 --iterations=500 --create=create_table.sql --query=queries.sql

Repeat above steps for different filesystems. The results:
EXT3 and EXT4 were good and quite comparable, followed by XFS.

Setup Load-Balancing Cluster with LVS and Piranha on Centos 6

This setup guide is about building web cluster with a pair of Linux loadbalancers as frontend. It uses Linux Virtual Server(LVS) and Piranha. Piranha is a web-based gui installed in LVS Routers primarily to generating a valid /etc/lvs.cf file. Start by installing LVS on LVS Router.

[root@lvsrouter ~]# yum groupinstall "Load Balancer"
================================================================================
 Package         Arch        Version                         Repository    Size
================================================================================
Installing:
 ipvsadm         x86_64      1.25-10.el6                     base          41 k
 piranha         x86_64      0.8.6-2.el6_4.1                 updates      623 k
Installing for dependencies:
 libedit         x86_64      2.11-4.20080712cvs.1.el6        base          74 k
 libnl           x86_64      1.1-14.el6                      base         121 k
 php             x86_64      5.3.3-22.el6                    base         1.1 M
 php-cli         x86_64      5.3.3-22.el6                    base         2.2 M
 php-common      x86_64      5.3.3-22.el6                    base         524 k

Transaction Summary
================================================================================
Install       7 Package(s)

Total download size: 4.7 M
Installed size: 18 M
Is this ok [y/N]: y

Start piranha and pulse services on reboot.

chkconfig piranha-gui on
chkconfig pulse on

Set a password for piranha web

/usr/sbin/piranha-passwd

Allow ports in iptables

vi /etc/sysconfig/iptables
-A INPUT -m state --state NEW -m tcp -p tcp --dport 3636 -j ACCEPT

Edit piranha config

vi  /etc/sysconfig/ha/conf/httpd.conf

Start the piranha gui service

service piranha-gui start

For LVS router to forward network packets properly to real servers, each LVS router node must have IP forwarding turned on. Turn on packet forwarding by editing sysctl.conf:

vi /etc/sysctl.conf
net.ipv4.ip_forward = 1

Reload sysctl

sysctl -p /etc/sysctl.conf

Start http services on the Real Servers

service httpd start

On the LVS Router, log in to the Piranha web ui to begin configuration. (eg http://(loadbalancer-ip):3636). Using "piranha" as user name, and key in the newly setup password. Enter Primary Router's IP address in "Global Settings"->"Primary server public IP:".

Enter Redundant Router's IP address in "Redundancy"->"Redundant server public IP:".

Enter Virtual Server settings in "Virtual Servers"->"Virtual Server".

Include Real Server settings in "Virtual Servers"->"Real Server".

On each time the Piranha Gui was changed, have to sync the settings to Backup Routers, and restart the pulse service on both routers.

service pulse restart

To see the virtual server's statistics, use "watch ipvsadm".

[root@lvsrouter ~]# watch ipvsadm 

Create arptables entry for each Virtual IP address on each Real Server(eg Webserver). You can add below command to /etc/rc.local to start on every reboot. If the network adaptor on Real Server is eth0, refer to below, if not change as according:

ip addr add (virtual ip) dev eth0:1

Direct Routing with arptables_jf. To configure each real server to ignore ARP requests for each of the virtual IP addresses the Piranha cluster services:

yum install arptables_jf
arptables -A IN -d (virtual_ip) -j DROP
arptables -A OUT -d (virtual_ip) -j mangle --mangle-ip-s (real_ip)
chkconfig arptables_jf on
service arptables_jf save
service arptables_jf restart

Create a loopback on each Real Server for monitoring Virtual IP.

vi /etc/sysconfig/network-scripts/ifcfg-lo:0

DEVICE=lo:0
IPADDR=(Virtual IP)
NETMASK=255.255.255.255
NETWORK=192.168.0.0
ONBOOT=yes
NAME=loopback

PHP: Enable zlib output compression

You can enable php compression as it have a lot of benefits. It reduces the web traffic from host provider to your web. It speeds up the loading of your pages on clients PC. And it is supported by all newer web browser. All you have to do is to set "zlib.output_compression=on" in php.ini:

zlib.output_compression=on