新建立的网站，未必需要流量，各个方面还在优化，也为了节省资源和网站安全，需要屏蔽一些东西的
location / {
if ($http_user_agent ~* "robot|qihoobot|Baiduspider|Googlebot|Googlebot-Mobile|Googlebot-Image|Mediapartners-Google|Adsbot-Google|Feedfetcher-Google|Yahoo! Slurp|Yahoo! Slurp China|YoudaoBot|Sosospider|Sogou spider|Sogou web spider|MSNBot|ia_archiver|Tomato Bot") {
return 403;
}
}
测试 ...
curl -I -A "qihoobot" www.xxx.com