squid之过期判断refresh_pattern

       usage: refresh_pattern [-i] regex min percent max [options]
       Basically a cached object is:
               FRESH if expires < now, else STALE

               STALE if age > max

               FRESH if lm-factor < percent, else STALE

               FRESH if age < min

               else STALE

vmware-server on archlinux

archlinux这个东西升级太频繁。如果一些软件不兼容。如果把内核升至最新的话,有一些信赖于内核的软件就不能用了。。。。

我的这个升级到了2.6.37.结果装了一天vmware-server。愣是没装上。补丁包都是针对2.6.36以下的。没有最新的2.6.37的。。

 

安装好后,提示vmmon模块版本号不正确。。。脑袋瓜。。我对这个东东没有好感。。折腾了一天。。give up。。。换virtual box吧。。。

 

郁闷。。

pv计算并发连接数公式

6台Web服务器(除开FLV视频存储服务器),达到了可承受4000万PV(页面访问量)的性能
每台Web服务器能够处理3万并发连接数


另外说明一下pv和并发连接数的计算公式:这个会影响今后并发连接数技术指标:
保证pv/day的并发连接数的计算公式是:
并发连接数=pv / seconds(one day=86400) * (para connect per a page) * (time to react) * (factor=5) / (web hosts)

Shell中命令行选项和参数的处理【转】

转自:

http://www.51testing.com/?uid-225738-action-viewspace-itemid-225325

LinuxShell中怎样处理tail -n 10 access.log这样的命令行选项呢?这是被别人问起的一个问题,好好学习了一下,进行总结如下:
在bash中,可以用以下三种方式来处理命令行参数,每种方式都有自己的应用场景。
1.直接处理,依次对$1,$2,…,$n进行解析,分别手工处理;
2.getopts来处理,单个字符选项的情况(如:-n 10 -f file.txt等选项);
3.getopt,可以处理单个字符选项,也可以处理长选项long-option(如:–prefix=/home等)。
总结:一般小脚本手工处理也许就够了,getopts能处理绝大多数的情况,getopt较复杂、功能也更强大。


下面分别进行简单的说明:

1.直接手工处理位置参数
必须要要知道几个变量,
    *    $0 :即命令本身,相当于C/C++中的argv[0]
    *    $1 :第一个参数.
    *    $2, $3, $4 … :第2、3、4个参数,依次类推。
    *    $#  参数的个数,不包括命令本身
    *    $@ :参数本身的列表,也不包括命令本身
    *    $* :和$@相同,但”$*” 和 “$@”(加引号)并不同,”$*”将所有的参数解释成一个字符串,而”$@”是一个参数数组。
手工处理方式能满足多数的简单需求,配合shift使用也能构造出强大的功能,但处理复杂选项的时候建议用下面的两种方法。
给个实例吧(getargs.sh):

#!/bin/bash
if [ $# -lt 1 ]; then
    echo “error.. need args”
    exit 1
fi
echo “commond is $0″
echo “args are:”
for arg in “$@”
do
    echo $arg
done

运行命令:./getargs.sh 11 22 cc
commond is ./getargs.sh
args are:
11
22
cc

2.getopts (Shell内置命令)
处理命令行参数是一个相似而又复杂的事情,为此,C提供了getopt/getopt_long等函数,C++的boost提供了Options库,在shell中,处理此事的是getopts和getopt.
先说一下getopts/getopt的区别吧,getopt是个外部binary文件,而getopts是shell builtin。

[admin@intlqa142055x ~]$ type getopt
getopt is /usr/bin/getopt
[admin@intlqa142055x ~]$ type getopts
getopts is a shell builtin

getopts不能直接处理长的选项(如:–prefix=/home等)
关于getopts的使用方法,可以man bash  搜索getopts
getopts有两个参数,第一个参数是一个字符串,包括字符和“:”,每一个字符都是一个有效的选项,如果字符后面带有“:”,表示这个字符有自己的参数。getopts从命令中获取这些参数,并且删去了“-”,并将其赋值在第二个参数中,如果带有自己参数,这个参数赋值在“OPTARG”中。提供getopts的shell内置了OPTARG这个变变,getopts修改了这个变量。
这里变量$OPTARG存储相应选项的参数,而$OPTIND总是存储原始$*中下一个要处理的元素位置。
while getopts “:a:bc” opt  #第一个冒号表示忽略错误;字符后面的冒号表示该选项必须有自己的参数
代码实例(getopts.sh):

echo $*
while getopts “:a:bc” opt
do
        case $opt in
                a ) echo $OPTARG
                    echo $OPTIND;;
                b ) echo “b $OPTIND”;;
                c ) echo “c $OPTIND”;;
                ? ) echo “error”
                    exit 1;;
        esac
done
echo $OPTIND
shift $(($OPTIND – 1))
#通过shift $(($OPTIND – 1))的处理,$*中就只保留了除去选项内容的参数,可以在其后进行正常的shell编程处理了。
echo $0
echo $*

执行命令:./getopts.sh -a 11 -b -c
-a 11 -b -c
11
3
b 4
c 5
5
./getopts.sh

3.getopt(一个外部工具)
具体用用法可以 man getopt
#-o表示短选项,两个冒号表示该选项有一个可选参数,可选参数必须紧贴选项,如-carg 而不能是-c arg
#–long表示长选项

简单举个例子吧(getopt.sh):


#!/bin/bash

# A small example program for using the new getopt(1) program.
# This program will only work with bash(1)
# An similar program using the tcsh(1) script. language can be found
# as parse.tcsh

# Example input and output (from the bash prompt):
# ./parse.bash -a par1 ‘another arg’ –c-long ‘wow!*?’ -cmore -b ” very long “
# Option a
# Option c, no argument
# Option c, argument `more’
# Option b, argument ` very long ‘
# Remaining arguments:
# –> `par1′
# –> `another arg’
# –> `wow!*?’

# Note that we use `”$@”‘ to let each command-line parameter expand to a
# separate word. The quotes around `$@’ are essential!
# We need TEMP as the `eval set –‘ would nuke the return value of getopt.

#-o表示短选项,两个冒号表示该选项有一个可选参数,可选参数必须紧贴选项
#如-carg 而不能是-c arg
#–long表示长选项
#”$@”在上面解释过
# -n:出错时的信息
# — :举一个例子比较好理解:
#我们要创建一个名字为 “-f”的目录你会怎么办?
# mkdir -f #不成功,因为-f会被mkdir当作选项来解析,这时就可以使用
# mkdir — -f 这样-f就不会被作为选项。

TEMP=`getopt -o ab:c:: –long a-long,b-long:,c-long::
     -n ‘example.bash’ — “$@”`

if [ $? != 0 ] ; then echo “Terminating…” >&2 ; exit 1 ; fi

# Note the quotes around `$TEMP': they are essential!
#set 会重新排列参数的顺序,也就是改变$1,$2…$n的值,这些值在getopt中重新排列过了
eval set — “$TEMP”

#经过getopt的处理,下面处理具体选项。

while true ; do
        case “$1″ in
                -a|–a-long) echo “Option a” ; shift ;;
                -b|–b-long) echo “Option b, argument `$2′” ; shift 2 ;;
                -c|–c-long)
                        # c has an optional argument. As we are in quoted mode,
                        # an empty parameter will be generated if its optional
                        # argument is not found.
                        case “$2″ in
                                “”) echo “Option c, no argument”; shift 2 ;;
                                *)  echo “Option c, argument `$2′” ; shift 2 ;;
                        esac ;;
                –) shift ; break ;;
                *) echo “Internal error!” ; exit 1 ;;
        esac
done
echo “Remaining arguments:”
for arg do
   echo ‘–> ‘”`$arg'” ;
done

运行命令:./getopt.sh –b-long abc -a -c33 remain
Option b, argument `abc’
Option a
Option c, argument `33′
Remaining arguments:
–> `remain’

LVS问题小记

两台WEB服务器上跑着hypy的搜索服务,已经把WEB、MYSQL放到了LVS上,现在要把这个搜索服务也放在LVS上,搜索是用apache+mod_wsgi支撑,端口为3321

前台用户点击搜索后,PHP程序将需要搜索的内容及相关参数以http的方式访问3321,apache返回的结果经PHP整理后显示给用户

直接将两台搜索放在的VIP后面,在前台点击搜索始终看不到LVS上有活动链接

问题:

既是客户端又充当RealServer的情形。。。。。。。这种情形下会出现问题

 

由于设置了RealServer的lo:0为VIP,导致 RealServer 请求到自己本身,根本没有经过LVS

解决方案:

1.由于WEB已经做了负载均衡,如果请求本机的话。自各请求自己也会出现负载均衡的结果,其它WEB服务器访问VIP将会根据权重请求到对应的服务器上

2.将搜索服务从WEB服务器上独立出来,就不会出现这种问题了。这是一种比较好的方案,不过根据实际情况还是选择1为好

 

总结:

其实这是一种低级的错误,在实施之前就可以通过理论来避免,在实施之前没有认真思考,直接配置,导致了时间的浪费

以后在实施一个配置之前要好好思考一下工作会不会正常,减少时间的浪费 ,提高工作效率

 

 

PS:LVS是用DR的方式做的

服务器优化参数

net.ipv4.tcp_max_tw_buckets = 6000
net.ipv4.ip_local_port_range = 10000 65000
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_syn_retries = 1
net.ipv4.tcp_keepalive_time = 30
net.ipv4.tcp_fin_timeout = 10
net.core.somaxconn = 8192
net.ipv4.tcp_max_syn_backlog = 819200
net.ipv4.tcp_timestamps = 0

 

net.core.rmem_max=16777216

net.core.wmem_max=16777216

net.ipv4.tcp_rmem=4096 87380 16777216

net.ipv4.tcp_wmem=4096 65536 16777216

net.core.netdev_max_backlog=3000

深入讨论proc,内核参数调优(转载)

Disabling the TCP options reduces the
overhead of each TCP packet and might help to get the last few percent
of performance out of the server. Be aware that disabling these options
most likely decreases performance for high-latency and lossy links.

* net.ipv4.tcp_sack = 0

* net.ipv4.tcp_timestamps = 0


Increasing the TCP send and receive buffers will increase the
performance a lot if (and only if) you have a lot of large files to
send.

* net.ipv4.tcp_wmem = 4096 65536 524288

* net.core.wmem_max = 1048576

If you have a lot of large file uploads, increasing the receive buffers will help.

* net.ipv4.tcp_rmem = 4096 87380 524288

* net.core.rmem_max = 1048576

# These ensure that TIME_WAIT ports either get reused or closed fast.

net.ipv4.tcp_fin_timeout = 1

net.ipv4.tcp_tw_recycle = 1

# TCP memory

net.core.rmem_max = 16777216

net.core.rmem_default = 16777216

net.core.netdev_max_backlog = 262144

net.core.somaxconn = 262144

net.ipv4.tcp_syncookies = 1

net.ipv4.tcp_max_orphans = 262144

net.ipv4.tcp_max_syn_backlog = 262144

net.ipv4.tcp_synack_retries = 2

net.ipv4.tcp_syn_retries = 2

# you shouldn’t be using conntrack on a heavily loaded server anyway, but these are

# suitably high for our uses, insuring that if conntrack gets turned on, the box doesn’t die

net.ipv4.ip_conntrack_max = 1048576

net.nf_conntrack_max = 1048576

# increase Linux TCP buffer limits

echo 8388608 > /proc/sys/net/core/rmem_max

echo 8388608 > /proc/sys/net/core/wmem_max

# increase Linux autotuning TCP buffer limits

echo “4096 87380 8388608″ > /proc/sys/net/ipv4/tcp_rmem

echo “4096 65536 8388608″ > /proc/sys/net/ipv4/tcp_wmem

#echo 65536 > /proc/sys/fs/file-max # physical RAM * 256/4

echo “1024 65000″ > /proc/sys/net/ipv4/ip_local_port_range

#echo 1 > /proc/sys/net/ipv4/tcp_syncookies

echo 8192 > /proc/sys/net/ipv4/tcp_max_syn_backlog

# Decrease the time default value for tcp_fin_timeout connection

#echo 30 > /proc/sys/net/ipv4/tcp_fin_timeout

#echo 3 > /proc/sys/net/ipv4/tcp_syn_retries

#echo 2 > /proc/sys/net/ipv4/tcp_retries1

# Decrease the time default value for tcp_keepalive_time connection

#echo 1800 >/proc/sys/net/ipv4/tcp_keepalive_time

# Turn off tcp_window_scaling

echo 0 >/proc/sys/net/ipv4/tcp_window_scaling

#echo “67108864” > /proc/sys/kernel/shmmax

# Turn off the tcp_sack

echo 0 >/proc/sys/net/ipv4/tcp_sack # This disables RFC2018 TCP Selective Acknowledgements

#Turn off tcp_timestamps

echo 0 >/proc/sys/net/ipv4/tcp_timestamps # This disables RFC1323 TCP timestamps

echo 5 > /proc/sys/kernel/panic # reboot 5 minutes later then kernel panic

the third:

net.ipv4.tcp_window_scaling = 1

net.ipv4.tcp_syncookies = 1

net.core.rmem_max = 16777216

net.core.wmem_max = 16777216

net.ipv4.tcp_rmem = 4096 87380 16777216

net.ipv4.tcp_wmem = 4096 65536 16777216

Lot of tuning

# Disables packet forwarding

net.ipv4.ip_forward = 0

# Enables source route verification

net.ipv4.conf.default.rp_filter = 1

# Disables the magic-sysrq key

kernel.sysrq = 0

# Decrease the time default value for tcp_fin_timeout connection

net.ipv4.tcp_fin_timeout = 25

# Decrease the time default value for tcp_keepalive_time connection

net.ipv4.tcp_keepalive_time = 3600

# Turn on the tcp_window_scaling

net.ipv4.tcp_window_scaling = 1

# Turn on the tcp_sack

net.ipv4.tcp_sack = 1

# tcp_fack should be on because of sack

net.ipv4.tcp_fack = 1

# Turn on the tcp_timestamps

net.ipv4.tcp_timestamps = 1

# Enable TCP SYN Cookie Protection

net.ipv4.tcp_syncookies = 1

# Enable ignoring broadcasts request

net.ipv4.icmp_echo_ignore_broadcasts = 1

# Disable ICMP Redirect Acceptance

net.ipv4.conf.all.accept_redirects = 0

# Enable bad error message Protection

net.ipv4.icmp_ignore_bogus_error_responses = 1

# Don’t Log Spoofed Packets, Source Routed Packets, Redirect Packets

net.ipv4.conf.all.log_martians = 0

# Make more local ports available

net.ipv4.ip_local_port_range = 1024 65000

# Increase maximum amount of memory allocated to shm

kernel.shmmax = 1073741824

# Improve file system performance

vm.bdflush = 100 1200 128 512 15 5000 500 1884 2

# This will increase the amount of memory available for socket input/output queues

net.ipv4.tcp_rmem = 4096 25165824 25165824

net.core.rmem_max = 25165824

net.core.rmem_default = 25165824

net.ipv4.tcp_wmem = 4096 65536 25165824

net.core.wmem_max = 25165824

net.core.wmem_default = 65536

net.core.optmem_max = 25165824

# If you are feeling daring, you can also use these settings below, otherwise just remove them. (Should increase performance)

net.core.netdev_max_backlog = 2500

net.ipv4.tcp_tw_recycle = 1

net.ipv4.tcp_tw_reuse = 1

net.ipv4.tcp_rmem = 4096 25165824 25165824

net.core.rmem_max = 25165824

net.core.rmem_default = 25165824

net.ipv4.tcp_wmem = 4096 65536 25165824

net.core.wmem_max = 25165824

net.core.wmem_default = 65536

# Disables packet forwarding

net.ipv4.ip_forward=0

# Disables IP source routing

net.ipv4.conf.all.accept_source_route = 0

net.ipv4.conf.lo.accept_source_route = 0

net.ipv4.conf.eth0.accept_source_route = 0

net.ipv4.conf.default.accept_source_route = 0

# Enable IP spoofing protection, turn on source route verification

net.ipv4.conf.all.rp_filter = 1

net.ipv4.conf.lo.rp_filter = 1

net.ipv4.conf.eth0.rp_filter = 1

net.ipv4.conf.default.rp_filter = 1

# Disable ICMP Redirect Acceptance

net.ipv4.conf.all.accept_redirects = 0

net.ipv4.conf.lo.accept_redirects = 0

net.ipv4.conf.eth0.accept_redirects = 0

net.ipv4.conf.default.accept_redirects = 0

# Enable Log Spoofed Packets, Source Routed Packets, Redirect Packets

net.ipv4.conf.all.log_martians = 0

net.ipv4.conf.lo.log_martians = 0

net.ipv4.conf.eth0.log_martians = 0

# Disables IP source routing

net.ipv4.conf.all.accept_source_route = 0

net.ipv4.conf.lo.accept_source_route = 0

net.ipv4.conf.eth0.accept_source_route = 0

net.ipv4.conf.default.accept_source_route = 0

# Enable IP spoofing protection, turn on source route verification

net.ipv4.conf.all.rp_filter = 1

net.ipv4.conf.lo.rp_filter = 1

net.ipv4.conf.eth0.rp_filter = 1

net.ipv4.conf.default.rp_filter = 1

# Disable ICMP Redirect Acceptance

net.ipv4.conf.all.accept_redirects = 0

net.ipv4.conf.lo.accept_redirects = 0

net.ipv4.conf.eth0.accept_redirects = 0

net.ipv4.conf.default.accept_redirects = 0

# Disables the magic-sysrq key

kernel.sysrq = 0

# Decrease the time default value for tcp_fin_timeout connection

net.ipv4.tcp_fin_timeout = 15

# Decrease the time default value for tcp_keepalive_time connection

net.ipv4.tcp_keepalive_time = 1800

# Turn off the tcp_window_scaling

net.ipv4.tcp_window_scaling = 0

# Turn off the tcp_sack

net.ipv4.tcp_sack = 0

# Turn off the tcp_timestamps

net.ipv4.tcp_timestamps = 0

# Enable TCP SYN Cookie Protection

net.ipv4.tcp_syncookies = 1

# Enable ignoring broadcasts request

net.ipv4.icmp_echo_ignore_broadcasts = 1

# Enable bad error message Protection

net.ipv4.icmp_ignore_bogus_error_responses = 1

# Log Spoofed Packets, Source Routed Packets, Redirect Packets

net.ipv4.conf.all.log_martians = 1

# Increases the size of the socket queue (effectively, q0).

net.ipv4.tcp_max_syn_backlog = 1024

# Increase the tcp-time-wait buckets pool size

net.ipv4.tcp_max_tw_buckets = 1440000

# Allowed local port range

net.ipv4.ip_local_port_range = 16384 65536