使用AWSTATS自动分析Nginx日志
环境说明： Nginx安装目录：/usr/local/nginx Nginx配置文件存放目录：/usr/local/nginx/conf/nginx.conf Nginx日志目录：/usr/local/nginx/logs Awstats安装目录：/usr/local/awstats Awstats配置文件存放目录：/etc/awstats 日志切割脚本存放目录：/usr/local/nginx/sbin/cut_nginx_log.sh 一、Nginx日志切割 1、修改nginx.conf配置文件，使AWSTATS支持分析日志格式 # vim /usr/local/nginx/conf/nginx.conf
| log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; access_log logs/access.log main; |
2、编辑Nginx日志切割脚本 # vim /usr/local/nginx/sbin/cut_nginx_log.sh
| #!/bin/bash mv /usr/local/nginx/logs/access.log /usr/local/nginx/logs/access_`date +%Y%m%d`.log mv /usr/local/nginx/logs/error.log /usr/local/nginx/logs/error_`date +%Y%m%d`.log killall -s USER1 nginx |
# chmod +x /usr/local/nginx/sbin/cut_nginx_log.sh 3、 二、下载安装Awstats 1、下载安装
| # wget http://awstats.sourceforge.net/files/awstats-7.0.tar.gz # tar -zxvf awstats-7.0.tar.gz # mv awstats-7.0 /usr/local/awstats # mkdir -p /var/lib/awstats |
2、配置
| # mkdir - p /tmp/src/wwwroot/cgi-bin/ # cp /usr/local/awstats/wwwroot/cgi-bin/awstats.model.conf /tmp/src/wwwroot/cgi-bin/ # /usr/local/awstats/tools/awstats_configure.pl -----> Running OS detected: Linux, BSD or Unix Warning: AWStats standard directory on Linux OS is '/usr/local/awstats'. If you want to use standard directory, you should first move all content of AWStats distribution from current directory: /tmp/src to standard directory: /usr/local/awstats And then, run configure.pl from this location. Do you want to continue setup from this NON standard directory [yN] ? 输入y |
回车
| -----> Check for web server install Enter full config file path of your Web server. Example: /etc/httpd/httpd.conf Example: /usr/local/apache2/conf/httpd.conf Example: c:\Program files\apache group\apache\conf\httpd.conf Config file path ('none' to skip web server setup): > none 这里不是apache，写none，跳过# |
回车
| -----> Update model config file '/tmp/src/wwwroot/cgi-bin/awstats.model.conf' File awstats.model.conf updated. -----> Need to create a new config file ? Do you want me to build a new AWStats config/profile file (required if first install) [y/N] ? 输入 y #创建一个全新的统计 |
回车
| -----> Define config file name to create What is the name of your web site or profile analysis ? Example: www.mysite.com Example: demo Your web site, virtual server or profile name: > 192.168.2.143被统计网站的域名 # |
回车
| -----> Define config file path In which directory do you plan to store your config file(s) ? Default: /etc/awstats Directory path to store config file(s) (Enter for default): > #回车继续 -----> Create config file '/etc/awstats/awstats.192.168.2.143.conf' Config file /etc/awstats/awstats.192.168.2.143.conf created. -----> Add update process inside a scheduler Sorry, configure.pl does not support automatic add to cron yet. You can do it manually by adding the following command to your cron: /tmp/src/wwwroot/cgi-bin/awstats.pl -update -config=192.168.2.143 Or if you have several config files and prefer having only one command: /tmp/src/tools/awstats_updateall.pl now Press ENTER to continue... A SIMPLE config file has been created: /etc/awstats/awstats.192.168.2.143.conf You should have a look inside to check and change manually main parameters. You can then manually update your statistics for '192.168.2.143' with command: > perl awstats.pl -update -config=192.168.2.143 You can also build static report pages for '192.168.2.143' with command: > perl awstats.pl -output=pagetype -config=192.168.2.143 Press ENTER to finish... |
三、修改统计日志文件的路径
| # vim /etc/awstats/awstats.192.168.2.143.conf - LogFile="/var/log/httpd/mylog.log"修改成---> LogFile="/usr/local/nginx/logs/access_%YYYY-0%MM-0%DD-0.log" |
这里是对应上面Nginx日志切割所生成的目录存放位置，注意awstats的年月日格式，分析的执行顺序是： Nginx 产生日志 –> 日志切割 –> Nginx 继续产生日志 –> 另存切割日志 –> 交由Awstats统计 –> 生成结果 四、执行Awstats 日志更新程序开始统计分析
| #/usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=192.168.2.143 Create/Update database for config "/etc/awstats/awstats.192.168.2.143.conf" by AWStats version 7.0 (build 1.971) From data in log file "/usr/local/nginx/logs/access_20121207.log"... Phase 1 : First bypass old records, searching new record... Searching new records from beginning of log file... Phase 2 : Now process new records (Flush history on disk after 20000 hosts)... Jumped lines in file: 0 Parsed lines in file: 66466 Found 0 dropped records, Found 0 comments, Found 1 blank records, Found 58472 corrupted records, Found 0 old records, Found 7993 new qualified records. |
看到以上显示，证明日志切割和 Awstats 都已经运行无误了。统计分析完成后，结果还在 Awstats 的数据库中。在 Apache 上，可以直接打开 Perl 程序的网页查看统计。但本文开始时已经提到，Nginx 对 Perl 支持并不好，所以我们要换个方法，利用 awstats 的工具将统计的结果生成静态文件。 五、Awstats生成静态文件
| # cp -rf /usr/local/awstats/wwwroot/ /usr/local/nginx/logs/ # mkdir /usr/local/nginx/logs/wwwroot/awstats # /usr/local/awstats/tools/awstats_buildstaticpages.pl -update -config=192.168.2.143 -lang=cn -dir=/usr/local/nginx/logs/wwwroot/awstats/ -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl |
/usr/local/awstats/tools/awstats_buildstaticpages.pl #Awstats 静态页面生成脚本 -update -config=192.168.2.143 #更新配置选项 -lang=cn #语音中文 -dir=/usr/local/nginx/logs/wwwroot/awstats/ #统计结果输出目录 -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl #日志更新程序路径 六、修改nginx配置文件
| # vim /usr/local/nginx/conf/nginx.conf server { listen 8080; server_name localhost; location ~ ^/awstats/ { # html 静态页面目录 root /usr/local/nginx/logs/wwwroot/awstats; index index.html; access_log off; error_log off; charset gb2312; } location ~ ^/icon/ { # 图标目录 root /usr/local/nginx/logs/wwwroot; index index.html; access_log off; error_log off; charset gb2312; } } |
七、配置Awstats自动运行
| # vim /usr/local/nginx/sbin/cut_nginx_log.sh 23 59 * * * root /usr/local/nginx/sbin/cut_nginx_log.sh 00 01 * * * /usr/local/awstats/tools/awstats_buildstaticpages.pl -update -config=192.168.2.143 -lang=cn -dir=/usr/local/nginx/logs/wwwroot/awstats/ -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl |
八、结果验证，URL输入 http://192.168.2.143:8080/awstats/awstats.192.168.2.143.html
本文出自 “Darrenpan” 博客，请务必保留此出处http://darren.blog.51cto.com/1081720/1081974
[背景]
给公司同事装了个nginx的平台，想监控下来自那里的人访问，就想到了awstats。可也让我头疼了一会,awstats跑的是perl cgi，而我按照张宴的配置是不可以跑perl的cgi的。就有了现在这篇文章。本文章是借助coolerfeng兄弟的大作 [url]http://coolerfeng.blog.51cto.com/133059/97723[/url]的延续，也可以说是完整版吧!呵呵! [过程] Perl的cgi配置按照coolerfeng大哥的就可以了。不过有几点注意，在这里要说明下 (1)
location ~ ^/cgi-bin/.*\.cgi$ {
gzip off; #gzip makes scripts feel slower since they have to complete before getting gzipped
fastcgi_pass unix:/var/run/nginx/cgi.sock;
fastcgi_index index.cgi;
fastcgi_param SCRIPT_FILENAME /var/www/cgi-bin$fastcgi_script_name; 此处要修改为 ： fastcgi_param SCRIPT_FILENAME $document_root $fastcgi_script_name;
fastcgi_param QUERY_STRING $query_string;
fastcgi_param REQUEST_METHOD $request_method;
fastcgi_param CONTENT_TYPE $content_type;
fastcgi_param CONTENT_LENGTH $content_length;
fastcgi_param GATEWAY_INTERFACE CGI/1.1;
fastcgi_param SERVER_SOFTWARE nginx;
fastcgi_param SCRIPT_NAME $fastcgi_script_name;
fastcgi_param REQUEST_URI $request_uri;
fastcgi_param DOCUMENT_URI $document_uri;
fastcgi_param DOCUMENT_ROOT $document_root;
fastcgi_param SERVER_PROTOCOL $server_protocol;
fastcgi_param REMOTE_ADDR $remote_addr;
fastcgi_param REMOTE_PORT $remote_port;
fastcgi_param SERVER_ADDR $server_addr;
fastcgi_param SERVER_PORT $server_port;
fastcgi_param SERVER_NAME $server_name;
(2) ./cgiwrap-fcgi.pl >/dev/null 2>&1 &执行后，需要再执行 Chown –R www:www /usr/local/webserver/perl/nginx/fcgi/cgi.sock 每次运行一个后台进程后，都要再执行下 chown (3) /var/log/nginx { # 说真的，我一开始就被这个搞晕了，修改为 /home/wiki/www/logs/wikilogs.log, 这样就可以轮替 wikilogs.log 了。 /home/wiki/www/logs/wikilogs.log {
daily
missingok
rotate 7
compress
delaycompress notifempty # 如果日志为空，延时压缩
create 644 nginx root # 注意，这个要修改与你的 nginx 运行的用户权限一样，如我的是 www, 那么就要改成 create 644 www root sharedscripts
prerotate
/usr/local/awstats/wwwroot/cgi-bin/awstats.pl --config=www.mysite.com -update # 改成你在执行 awstats_configure.pl 时配置的 website 名称，如我的 wikiob.xxxx.com ，那么就是 /usr/local/awstats/wwwroot/cgi-bin/awstats.pl --config=wikiob.xxxx.com -update
endscript
postrotate
if [ -f /usr/local/nginx/nginx.pid ]; then
kill -USR1 `cat /usr/local/nginx/nginx.pid` # 要修改成自己的 nginx.pid 的路径，如我的 /usr/local/webserver/nginx/nginx.pid
fi
endscript
} 注意：可以用 logrotate -vf /etc/logrotate.d/nginx调试，用这个来排除错误! (4) 只需要修改 nginx 的 logformat ，不需要更改 awstats 的 logformat ，保留为 1 便可，如果按照 cooler 兄的更改，我这便报这个问题， Found 6 corrupted records, == 》显示不了数据 . 几点注意后，我们来安装awstats （1）下载，解压 如：/usr/local/awstats 这个路径很重要，因为awstats的配置默认路径就是这个，我曾将tools下的工具移到其他目录下，再去执行配置时，会报一些问题给你的，并且告诉你，他的默认路径就是这个。 （2）cp -r /usr/local/awstats/wwwroot5 * * * * /usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=wikiob.xxxx.com [完成] [url]http://wikiob.xxxx.com/awstats/cgi-bin/awstats.pl[/url] 便可以正常访问了。 [总结] 朋友的力量是无穷的…….天助自助者。嘻嘻。。。。本文仅供参考.
本文出自 “坏男孩” 博客，请务必保留此出处http://5ydycm.blog.51cto.com/115934/140029
![LVS-DR模式案例[图]](data:image/gif;base64,R0lGODlhAQABAIAAAP///wAAACwAAAAAAQABAAACAkQBADs=)