Varnish在企业中的应用

一、Varnish的安装、命令使用及配置文件介绍

二、以案例方式介绍如何配置Varnish

三、服务验证

一、Varnish的安装、命令使用及配置文件介绍

1、配置YUM源，使用yum安装Varnish

12	`# rpm --nosignature -i http://repo.varnish-cache.org/redhat/varnish-3.0/el5/noarch/varnish-release-3.0-1.el5.centos.noarch.rpm# yum -y install varnish`

2、查看Varnish生成的文件

1234567891011121314151617 # rpm -ql varnish/etc/rc.d/init.d/varnish #Varnish服务启动脚本/etc/rc.d/init.d/varnishlog #Varnish日志服务启动脚本/etc/rc.d/init.d/varnishncsa #Varnish日志服务启动脚本/etc/sysconfig/varnish #Varnish启动服务的主配置文件/etc/varnish #配置文件目录/etc/varnish/default.vcl #默认VCL文件/usr/bin/varnish_reload_vcl #Varnish的VCL管理工具/usr/bin/varnishadm #Varnish管理工具/usr/bin/varnishhist/usr/bin/varnishlog #Varnish日志管理工具/usr/bin/varnishncsa #Varnish日志管理工具/usr/bin/varnishreplay/usr/bin/varnishsizes/usr/bin/varnishstat #Varnish状态查看工具/usr/bin/varnishtest/usr/bin/varnishtop

3、Varnish服务配置文件介绍

123456789101112131415161718192021222324252627 NFILES=131072 MEMLOCK=82000NPROCS="unlimited"# DAEMON_COREFILE_LIMIT="unlimited" #内核最大打开的文件数RELOAD_VCL=1 #是否自动加载VCLVARNISH_VCL_CONF=/etc/varnish/default.vcl #默认加载的VCL文件VARNISH_LISTEN_PORT=80 #监听端口，默认为6081VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1 #管理的IP地址VARNISH_ADMIN_LISTEN_PORT=6082 #管理端口VARNISH_SECRET_FILE=/etc/varnish/secret #密钥文件VARNISH_MIN_THREADS=50 #最小线程数量VARNISH_MAX_THREADS=1000 #最大线程数量VARNISH_THREAD_TIMEOUT=120 #线程超时时间VARNISH_STORAGE_FILE=/var/lib/varnish/varnish_storage.bin #缓存文件位置VARNISH_STORAGE_SIZE=1G #设置文件缓存大小变量VARNISH_MEMORY_SIZE=64M #设置内存缓存大小变量#VARNISH_STORAGE="file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}" #默认存储到文件中，这里可以修改存储位置VARNISH_STORAGE="malloc,${VARNISH_MEMORY_SIZE}" #设置缓存位置为内存VARNISH_TTL=120 DAEMON_OPTS="-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \ -f ${VARNISH_VCL_CONF} \ -T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \ -t ${VARNISH_TTL} \ -w ${VARNISH_MIN_THREADS},${VARNISH_MAX_THREADS},${VARNISH_THREAD_TIMEOUT} \ -u varnish -g varnish \ -S ${VARNISH_SECRET_FILE} \ -s ${VARNISH_STORAGE}" #所有启动加载选项

4、启动Varnish

12345 [root@varnish ~]# service varnish start[root@varnish ~]# netstat -anput|grep varnishtcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 20303/varnishdtcp 0 0 127.0.0.1:6082 0.0.0.0:* LISTEN 20301/varnishdtcp 0 0 :::80 :::* LISTEN 20303/varnishd

5、Varnishd命令介绍

123456789101112131415 # varnishd -h-a address:port #表示varnish对httpd的监听地址及其端口-b address:port #表示后端服务器地址及其端口-d #表示使用debug调试模式-f file #指定varnish服务器的配置文件-p param=value #指定服务器参数，用来优化varnish性能-P file #Varnish进程PID文件存放路径-n dir #指定varnish的工作目录-s kind[,storageoptions] #指定varnish缓存内容的存放方式，常用的方式有：“-s file,<dir_or_file>,<size>”；其中“<dir_or_file>”指定缓存文件的存放路径，“<size>”指定缓存文件的大小-t #指定默认的TTL值-T address:port #设定varnish的telnet管理地址及其端口-V #显示varnish版本号和版权信息-w int[,int[,int]] #设定varnish的工作线程数，常用的方式有： -w min,max -w min,max,timeout如：-w3,25600,50 #这里最小启动的线程数不能设定过大，设置过大，会导致varnish运行异常缓慢

6、Varnishadm命令使用介绍

12345678910111213141516171819202122232425262728293031 # varnishadm -hvarnishadm: invalid option -- 'h'usage: varnishadm [-n ident] [-t timeout] [-S secretfile] -T [address]:port command [...] -n is mutually exlusive with -S and -T注释：常用选项 -S：指定密钥文件 -T：指定服务器地址与端口；如管理一个Varnish服务器，连接到Varnish服务器：# varnishadm -S /etc/varnish/secret -T 127.0.0.1:6082varnish> helphelp [command]ping [timestamp]auth responsequit #退出bannerstatus #查看状态start #启动Varnishstop #关闭Varnishvcl.load <configname> <filename> #加载一个VCL文件vcl.inline <configname> <quoted_VCLstring>vcl.use <configname> #使用VCL文件vcl.discard <configname>vcl.list #查看VCL文件列表vcl.show <configname> #查看VCL文件配置param.show [-l] [<param>]param.set <param> <value>panic.showpanic.clearstorage.listbackend.listbackend.set_health matcher stateban.url <regexp>ban <field> <operator> <arg> [&& <field> <oper> <arg>]...ban.list

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

二、以案例方式介绍如何配置Varnish

A、环境介绍

架构简单描述：安装一个论坛程序，在Varnish服务器上做动静分离，将动态页面的请求都转发到"lamp"服务器组，"lamp"服务器组的数据库安装在"lamp1"服务器上面；将静态页面转发到"web"服务器组，对请求的静态内容做缓存

------------------------------------------------------------------------------------------------

B、安装lamp服务器并架设好论坛更多详细安装请看前面博客...

1、在lamp1与lamp2服务器上安装服务并测试

12345678 ######在Lamp1服务器上安装Httpd、Php、Mysql，启动服务[root@lamp1 ~]# yum -y install httpd php mysql-server php-mysql[root@lamp1 ~]# service httpd start[root@lamp1 ~]# service mysqld start------------------------------------------------------------------------######在Lamp2服务器上安装Httpd、Php，启动服务[root@lamp2 ~]# yum -y install httpd php php-mysql[root@lamp2 ~]# service httpd start

2、在Lamp1与Lamp2服务器创建php测试页面并访问验证，做法相同，这里只介绍一次

1	`[root@lamp1 ~]# echo "<?php phpinfo();" > /var/www/html/index.php`

3、安装论坛程序点此下载

1234567891011 [root@lamp1 ~]# mysqlmysql> create database bbs;mysql> grant all on bbs.* to 'bbsuser'@'172.16.%.%' identified by 'bbspass';mysql> flush privileges;注释：为论坛创建一个数据库并授权用户访问------------------------------------------------------------------------######安装论坛程序[root@lamp1 ~]# unzip Discuz_X3.0_SC_UTF8.zip[root@lamp1 ~]# cp -rf upload/* /var/www/html/[root@lamp1 ~]# chmod -R +w /var/www/html/{config,data,uc_server,uc_client} #添加可写权限[root@lamp1 ~]# chown -R apache /var/www/html/* #修改属主权限

4、访问"lamp1"服务器地址安装论坛

5、点击我同意-->下一步-->(全新安装 Discuz! X)下一步-->安装数据库界面

6、下一步-->安装完成-->访问主页并登录测试

7、将论坛程序拷贝到lamp2服务器上一份并访问测试

12	`[root@lamp1 ~]# scp -rp /var/www/html/* 172.16.14.3:/var/www/html/[root@lamp2 ~]# service httpd restart #重启lamp2服务器的WEB服务`

------------------------------------------------------------------------------------------------

C、安装web服务器并创建测试页访问

123	`[root@web ~]# yum -y install httpd[root@web ~]# service httpd start[root@web ~]# echo "<h1>WEB</h1>" > /var/www/html/index.html #创建测试页`

将Lamp1服务器上的论坛程序拷贝到WEB服务器一份，因为需要论坛中的一些静态文件如：（.jpg|.html）结尾的文件等

1	`[root@lamp1 ~]# scp -rp /var/www/html/* 172.16.14.3:/var/www/html/`

D、Varnish安装及配置，这里不再介绍如何安装，在第一部分已经介绍了如何安装

1、在第一部分中我们已经修改过默认监听端口为"80",接下来为Varnish提供一个VCL配置文件，建议基于默认的配置文件基础上修改，修改前备份一下文件

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183 [root@varnish ~]# cd /etc/varnish/[root@varnish varnish]# cp default.vcl default.vcl.bak[root@varnish varnish]# vim default.vcl######定义ACLacl purgers { #定义acl，实现IP地址过滤 "127.0.0.1"; "172.16.0.0"/16;}######定义健康状态检测probe dynamic { #设置动态网站服务器健康状态检测 .url = "/index.html"; .interval = 5s; .timeout = 1s; .expected_response = 200;} #这里设置了两个健康状态检测主要是为了区分动、静网站probe static { #设置动态网站服务器健康状态检测 .url = "/index.html"; #定义检测的页面 .interval = 5s; #探测请求的发送周期，默认为5秒 .timeout = 1s; #每次探测请求的过期时间 .expected_response = 200;}######定义后端服务器backend app1 { #定义一个后端服务器 .host = "172.16.14.2"; #服务器地址 .port = "80"; #服务器监听端口 .probe = dynamic; #健康状态检测}backend app2 { .host = "172.16.14.3"; .port = "80"; .probe = dynamic;}backend web { .host = "172.16.14.4"; .port = "80"; .probe = static;}######定义后端服务器组，实现负载均衡效果director apps random { #定义一个后端服务器组，实现负载均衡效果 { .backend = app1; #调用前面已定义过的后端主机 .weight = 2; #设置权重 } { .backend = app2; .weight = 2; }}######定义vcl_recv函数，实现请求到达并成功接收后调用此函数中定义的规则sub vcl_recv {######定义动、静分离,以".php"或".php?后面跟所有文件"结尾的请求都发送到动态服务器，其他请求都发送到静态服务器 if (req.url ~ "\.php(\?\.*|$)") { set req.backend = apps; } else { set req.backend = web; } return(lookup);######定义允许清除缓存的IP地址,调用的是前面定义的ACL if (req.request == "PURGE") { if (!client.ip ~ purgers) { error 405 "Method not allowed"; } return(lookup); }######重新定义http请求首部，让后端服务器可以记录请求客户端的真实IP地址 if (req.restarts == 0) { if (req.http.x-forwarded-for) { set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip; } else { set req.http.X-Forwarded-For = client.ip; } }######除了定义的请求方法外，其他请求都到后端服务器 if (req.request != "GET" && req.request != "HEAD" && req.request != "PUT" && req.request != "POST" && req.request != "TRACE" && req.request != "OPTIONS" && req.request != "DELETE") { return (pipe); } if (req.request != "GET" && req.request != "HEAD") { return (pass); }######定义不缓存认证与Cookie信息 if (req.http.Authorization || req.http.Cookie) { return (pass); }######定义压缩功能 if (req.http.Accept-Enconding) { if (req.url ~ "\.(jpg|jpeg|gif|bmp|png|flv|gz|tgz|tbz|mp3)$") { remove req.http.Accept-Encoding; remove req.http.Cookie; } else if (req.http.Accept-Encoding ~ "gzip") { set req.http.Accept-Encoding = "gzip"; } else if (req.http.Accept-Encoding ~ "deflate") { set req.http.Accept-Encoding = "deflate"; } else { remove req.http.Accept-Encoding; } }######定义指定格式结尾的文件去除Cookie信息 if (req.request == "GET" && req.url ~ "\.(jpeg|jpg|gif|png|bmp|swf)$") { unset req.http.cookie; }######定义防盗链设置 if (req.http.referer ~ "http://.*") { if (!(req.http.referer ~ "http://.*\.baidu\.com" || req.http.referer ~"http://.*\.google\.com.*")) { set req.http.host = "www.allen.com"; set req.url = "http://172.16.14.4/error.html"; } }}######定义vcl_hash函数sub vcl_hash { hash_data(req.url); if (req.http.host) { hash_data(req.http.host); } else { hash_data(server.ip); } return(hash);}######定义vcl_hit函数sub vcl_hit { if (req.request == "PURGE") { #语法方法为"PURGE" purge; #清除缓存 error 200 "Purged."; #返回错误状态码为"200" } return(deliver);}######定义vcl_miss函数sub vcl_miss { if (req.request == "PURGE") { purge; error 404 "Not In Cache."; } return(fetch);}######定义vcl_psss函数sub vcl_pass { if (req.request == "PURGE") { error 502 "Purged On A Passed Object."; } return(pass);}######定义vcl_fetch函数sub vcl_fetch {######定义缓存，如果匹配到已定义文件结尾的缓存1天，其他则缓存1小时 if (req.request == "GET" && req.url ~ "\.(html|jpg|png|bmp|jpeg|gif|js|ico|swf|css)$") { set beresp.ttl = 1d; set beresp.http.expires = beresp.ttl; } else { set beresp.ttl = 1h; } return(deliver);}######定义在http首部中，如果请求命中显示"HIT",未命中则显示"MISS"sub vcl_deliver { if (obj.hits > 0) { set resp.http.X-Cache = "HIT"; } else { set resp.http.X-Cache = "MISS"; }}----------------------------------------------------------------------[root@varnish ~]# service varnish restart #重启服务生效，重启服务器后所有缓存将被清除，当然也可以不用重启服务使其生效，如下：----------------------------------------------------------------------[root@varnish ~]# varnishadm -S /etc/varnish/secret -T 127.0.0.1:6082varnish> help #获取帮助varnish> vcl.load acl_1 default.vcl #加载acl文件,acl_1为配置名称200 VCL compiled.varnish> vcl.list #查看加载的acl文件列表200 active 7 bootavailable 0 acl_1varnish> vcl.use acl_1 #应用acl文件200varnish> quit #退出------------------------------------------------------------------------注释： -S：指定varnish的密钥文件 -T：指定varnish服务器地址及管理端口，默认端口为"6082"

2、如果要想让后端服务器记录客户端的真实IP地址需要修改Apache记录日志格式，如下：

1	`LogFormat` `"%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\""combined`

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

三、服务验证

1、对服务器做压力测试，如下：

12345678910111213141516171819202122232425262728293031 ######后端服务器不经过缓存测试[root@localhost ~]# ab -c 100 -n 1000 http://172.16.14.2/index.phpConcurrency Level: 1000Time taken for tests: 6.812 secondsComplete requests: 10000Failed requests: 0Write errors: 0Non-2xx responses: 10051Total transferred: 2281577 bytesHTML transferred: 0 bytesRequests per second: 1468.04 [#/sec] (mean) #每秒请求并发Time per request: 681.179 [ms] (mean)Time per request: 0.681 [ms] (mean, across all concurrent requests)Transfer rate: 327.10 [Kbytes/sec] received----------------------------------------------------------------------######经过缓存测试[root@localhost ~]# ab -c 1000 -n 10000 http://172.16.14.1/index.phpConcurrency Level: 1000Time taken for tests: 2.594 secondsComplete requests: 10000Failed requests: 0Write errors: 0Non-2xx responses: 10056Total transferred: 3117360 bytesHTML transferred: 0 bytesRequests per second: 3855.05 [#/sec] (mean)Time per request: 259.400 [ms] (mean)Time per request: 0.259 [ms] (mean, across all concurrent requests)Transfer rate: 1173.59 [Kbytes/sec] received----------------------------------------------------------------------注释：从上面数据中可以看出，经过缓存做压力测试并发量高

2、测试缓存是否能命中

1234567891011121314151617181920212223242526272829 [root@lamp2 ~]# curl -I http://172.16.14.1/index.phpHTTP/1.1 301 Moved PermanentlyServer: Apache/2.2.15 (CentOS)X-Powered-By: PHP/5.3.3location: forum.phpContent-Type: text/html; charset=UTF-8Content-Length: 0Accept-Ranges: bytesDate: Wed, 09 Oct 2013 01:09:01 GMTX-Varnish: 2142028839Age: 0Via: 1.1 varnishConnection: keep-aliveX-Cache: MISS #第一次请求，未命中显示"MISS"------------------------------------------------------------------------[root@lamp2 ~]# curl -I http://172.16.14.1/index.phpHTTP/1.1 301 Moved PermanentlyServer: Apache/2.2.15 (CentOS)X-Powered-By: PHP/5.3.3location: forum.phpContent-Type: text/html; charset=UTF-8Content-Length: 0Accept-Ranges: bytesDate: Wed, 09 Oct 2013 01:09:08 GMTX-Varnish: 2142028841 2142028839Age: 7Via: 1.1 varnishConnection: keep-aliveX-Cache: HIT #第二次请求，命中则显示"HIT"

3、验证动、静分离的效果

12	`[root@web ~]# service httpd stop #停止提供静态页面的httpd服务Stopping httpd: [ OK ]`

注释：从上图中可以看出，提供静态页面的服务停止后，所有图片都不能显示，当然把服务再启动起来就可以访问正常了，这里就不在测试了...

4、验证健康状态检测

5、查看缓存命中率状态；命中率的高低

6、验证手动清除缓存

123456789101112131415161718192021222324252627282930313233 [root@lamp2 ~]# curl -X PURGE http://172.16.14.1/index.php<?xml version="1.0" encoding="utf-8"?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html> <head> <title>200 Purged.</title> </head> <body> <h1>Error 200 Purged.</h1> <p>Purged.</p> <h3>Guru Meditation:</h3> <p>XID: 580728625</p> <hr> <p>Varnish cache server</p> </body></html> #注释：已经成功清除缓存----------------------------------------------------------------------[root@lamp2 ~]# curl -I http://172.16.14.1/index.phpHTTP/1.1 301 Moved PermanentlyServer: Apache/2.2.15 (CentOS)X-Powered-By: PHP/5.3.3location: forum.phpContent-Type: text/html; charset=UTF-8Content-Length: 0Accept-Ranges: bytesDate: Wed, 09 Oct 2013 02:03:03 GMTX-Varnish: 580728626Age: 0Via: 1.1 varnishConnection: keep-aliveX-Cache: MISS注释：由于上面清除了缓存，这里第一次请求为"MISS"

到此，Varnish的配置及验证已全部完成，

秒客网

Varnish在企业中的应用

相关文章