AWStats Installation, Configuration and Reporting
There are 3 steps to begin using AWStats:
I. Setup: Installation and configuration
II. Process logs: Building/updating statistics database
III. Run Reports: Building and reading reports
I. Setup: Installation and configuration using awstats_configure.pl
A) Setup for an Apache or compatible web server (on Unix/Linux, Windows, MacOS...)
* Step 1:
(if you use a package provided with a Linux distribution or Windows installer, step 1 might have already been done; if you don't know, you can run this step again)
After downloading and extracting the AWStats package, you should run the awstats_configure.pl script to do several setup actions. You will find it in the AWStats tools directory (If using the Windows installer, the script is automatically launched):
perl awstats_configure.pl
This is what the script does/asks (you can do all these steps manually instead of running awstats_configure.pl if you prefer):
A) awstats_configure.pl tries to determine your current log format from your Apache web server configuration file httpd.conf (it asks for the path if not found). If you use a common log, awstats_configure.pl will suggest changing it to the NCSA combined/XLF/ELF format (you can use your own custom log format but this predefined log format is often the best choice and makes setup easier).
If you answer yes, awstats_configure.pl will modify your httpd.conf, changing the following directive:
from
CustomLog /yourlogpath/yourlogfile common
to
CustomLog /yourlogpath/yourlogfile combined
See the Apache manual for more information on this directive (possibly installed on your server as www.mysite.com/manual).
B) awstats_configure.pl will then add, if not already present, the following directives to your Apache configuration file (note that the "/usr/local/awstats/wwwroot" path might differ according to your distribution or OS:
#
# Directives to add to your Apache conf file to allow use of AWStats as a CGI.
# Note that path "/usr/local/awstats/" must reflect your AWStats Installation path.
#
Alias /awstatsclasses "/usr/local/awstats/wwwroot/classes/"
Alias /awstatscss "/usr/local/awstats/wwwroot/css/"
Alias /awstatsicons "/usr/local/awstats/wwwroot/icon/"
ScriptAlias /awstats/ "/usr/local/awstats/wwwroot/cgi-bin/"
#
# This is to permit URL access to scripts/files in AWStats directory.
#
<Directory "/usr/local/awstats/wwwroot">
Options None
AllowOverride None
Order allow,deny
Allow from all
</Directory>
C) if changes were made as indicated in parts A and B, awstats_configure.pl restarts Apache to apply the changes. To be sure the log format change is effective, go to your homepage. This is an example of the type of records you should see inserted in your new log file after Apache was restarted:
62.161.78.75 - - [dd/mmm/yyyy:hh:mm:ss +0000] "GET / HTTP/1.1" 200 1234 "http://www.from.com/from.html" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)"
D) awstats_configure.pl will ask you for a name for the configuration profile file. Enter an appropriate name such as that of your web server or the virtual domain to be analyzed, i.e. mysite.
awstats_configure.pl will create a new file called awstats.mysite.conf by copying the template file awstats.model.conf. The new file location is:
- For Linux/BSD/Unix users: /etc/awstats.
- For Mac OS X, Windows and other operating systems: the same directory as awstats.pl (cgi-bin).
E) awstats_configure.pl ends.
* Step 2:
Once a configuration file has been created (by awstats_configure.pl, by your package installer or just by a manual copy of awstats.model.conf), it's important to verify that the "MAIN PARAMETERS" match your needs. Open awstats.mysite.conf in your favorite text editor (i.e. notepad.exe, vi, gedit, etc) - don´t use a word processor - and make changes as required.
Particular attention should be given to these parameters:
- Verify the LogFile value. It should be the full path of your server log file (You can also use a relative path from your awstats.pl directory, but a full path avoids errors).
- Verify the LogType value. It should be "W" for analyzing web log files.
- Check if LogFormat is set to "1" (for "NCSA apache combined/ELF/XLF log format") or use a custom log format if you don't use the combined log format.
- Set the SiteDomain parameter to the main domain name or the intranet web server name used to reach the web site to analyze (Example: www.mysite.com). If you have several possible names for same site, use the main domain name and add the others to the list in the HostAlias parameter.
- You can also change other parameters if you want. The full list is described in Configurations/Directives options page.
Installation and configuration is finished. You can jump to the Process logs: Building/updating statistics database section.
B) Setup for Microsoft's IIS server
* Step 1:
Configure IIS to create logs in the "Extended W3C log format" (You can still use your own custom log format but setup is easier if you use the standard extended format). To do so, start the IIS management console snap-in, select the appropriate web site and open its Properties. Choose "W3C Extended Log Format", then Properties, then the Tab "Extended Properties" and uncheck everything under Extended Properties. Once they are all cleared, check just the following fields:
date
time
c-ip
cs-username
cs-method
cs-uri-stem
cs-uri-query
sc-status
sc-bytes
cs-version
cs(User-Agent)
cs(Referer)
To be sure the log format change is effective, you must stop IIS, backup it up (if you desire) and remove all of the old log files, restart IIS and go to your homepage. This is an example of the type of records you should find in the new log file:
2000-07-19 14:14:14 62.161.78.73 - GET / 200 1234 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+5.01;+Windows+NT+5.0) http://www.from.com/from.htm
* Step 2:
Copy the contents of the AWStats provided cgi-bin folder, from where the AWStats package put it on your local hard drive, to your server's cgi-bin directory (this includes awstats.pl, awstats.model.conf, and the lang, lib and plugins sub-directories).
* Step 3:
Move AWStats icon sub-directories and its content into a directory readable by your web server, for example C:\yourwwwroot\icon.
* Step 4:
Create a configuration file by copying awstats.model.conf to a new file named awstats.mysite.conf where "mysite" is a value of your choice but usually is the domain or virtual host name. This new file must be saved in the same directory as awstats.pl (i.e. cgi-bin).
* Step 5:
Edit your new awstats.mysite.conf file to match your specific environment:
- Change the LogFile value to the full path of your web server log file (You can also use a relative path from your awstats.pl (cgi-bin) directory).
- Change the LogType value to "W" for analyzing web log files.
- Change the LogFormat to 2 if you are using the "Extended W3C log format" described in step 1; in the case of a custom format, list the IIS fields being logged, for example:
LogFormat="date time c-ip cs-username cs-method cs-uri-stem cs-uri-query sc-status sc-bytes cs-version cs(User-Agent) cs(Referer)"
- Change the DirIcons parameter to reflect relative path of icon directory.
- Set the SiteDomain parameter to the main domain name or the intranet web server name used to reach the web site being analyzed (Example: www.mydomain.com).
- Set the AllowToUpdateStatsFromBrowser parameter to 1 if you don't have command line access and have only cgi access.
- Review and change other parameters if appropriate.
Installation and configuration is finished. You can jump to the Process logs: Building/Updating statistics database section.
C) Setup for other web servers
The setup process is similar to the setup for Apache or IIS.
Use LogFormat to value "3" if you have WebStar native log format. Use a personalized LogFormat if your log format is other.
D) Setup for other Internet servers, i.e. FTP, Mail, Streaming media
The setup process for other file formats is described in the relevant FAQ topics: FAQ-COM090: FTP FAQ-COM100: Mail and FAQ-COM110: Streaming media.
II. Process logs: Building/updating statistics database
* Update from command line (recommanded):
The first log analysis should be done manually from the command line since the process may be long and it's easier to solve problems when you can see the command output (if you don't have Command Line access, skip to Step 2). The AWStats create (and update) statistics database command is:
perl awstats.pl -config=mysite -update
where mysite must be substituted with the domain/virtual host name you selected earlier during AWStats configuration.
AWStats will read the configuration file awstats.mysite.conf (or if not found, awstats.conf) and create/update its database with all summary information issued from analyzed log file.
AWStats statistics database files are saved in directory defined by the DirData parameter in configuration file.
When the create/update is finished, you should see a similar result on your screen:
Update for config "/etc/awstats/awstats.mysite.conf"
With data in log file "/pathtoyourlog/yourlog.log"...
Phase 1 : First bypass old records, searching new record...
Searching new records from beginning of log file...
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...
Jumped lines in file: 0
Parsed lines in file: 225730
Found 122 dropped records,
Found 87 corrupted records,
Found 0 old records,
Found 225521 new qualified records.
Dropped records are records discarded because they were not "user HTTP requests" or were requests matching AWStats filters (See the SkipHosts, SkipUserAgents, SkipFiles, OnlyHosts, OnlyUserAgents and OnlyFiles parameters). If you want to see which lines were dropped, you can add the -showdropped option on the command line.
Corrupted records are records that do not match the log format defined by the "LogFormat" parameter in the AWStats configuration file. All web servers will typically have a few corrupted records (<5%) even when everything works correctly. This can result for several reasons: 1) Web server internal bugs, 2) bad requests made by buggy browsers, 3) a dirty web server shutdown, such as unplugging the server...
If all of your lines are corrupted and the LogFormat parameter in AWStats configuration file is correct, then there may be a setup problem with your web server log format. Don't forget that your LogFormat parameter in the AWStats configuration file MUST match the log file format you analyze. If you want to see which lines are corrupted, you can add the -showcorrupted option on the command line.
Old records are simply records that were already processed by a previous update session. Although it is not necessary to purge your log file after each update process, it is highly recommended that you do so as often as possible.
New records are records in your log file that were successfully used to build/update the statistics database.
Note: A log analysis process might be slow (one second for each 4500 lines of your logfile with an Athlon 1Ghz, plus DNS resolution time for each different IP address in your logfile if DNSLookup is set to 1 and not already done in your log file). See the Benchmarks page for more detailed information.
* Update from a browser:
AWStats statistics can also be updated from a browser, providing real-time statistics, by clicking the "Update now" link that appears when AWStats is used as a CGI (The URL is described in the next section 'Run reports: Building and reading reports').
Warning!!
To enable this link, the configuration file parameter AllowToUpdateStatsFromBrowser must be set to 1 (The link is not enabled by default).
Using the on-line update does not prevent you from running the update process automatically on a scheduled basis (the command is same as that of the first update process above).
For this, you have two choices:
- Include the update command in your logrotate process. See FAQ-COM120 for details.
- Or add instructions in your crontab (Unix/Linux) or your task scheduler (Windows), to regularly launch the Awstats update process. See FAQ-COM130 for details.
See the AWStats Benchmarks page for the recommended update/logrotate frequency.
III. Run reports: Building and reading reports
To see the analysis results, you have several options depending on your security policy.
Note: you must have created a statistics data base for the analysis period by processing your log files before you try to create reports. See the previous section.
1. The first option is to build the main reports, in a static HTML page, from the command line, using the following syntax (skip to the second option if you only have CGI access):
perl awstats.pl -config=mysite -output -staticlinks > awstats.mysite.html
where mysite must be substituted with the domain/virtual host name you selected earlier during AWStats configuration.
To create specific individual reports, specify the report name on the command line as follows¹:
perl awstats.pl -config=mysite -output=alldomains -staticlinks > awstats.mysite.alldomains.html
perl awstats.pl -config=mysite -output=allhosts -staticlinks > awstats.mysite.allhosts.html
perl awstats.pl -config=mysite -output=lasthosts -staticlinks > awstats.mysite.lasthosts.html
perl awstats.pl -config=mysite -output=unknownip -staticlinks > awstats.mysite.unknownip.html
perl awstats.pl -config=mysite -output=alllogins -staticlinks > awstats.mysite.alllogins.html
perl awstats.pl -config=mysite -output=lastlogins -staticlinks > awstats.mysite.lastlogins.html
perl awstats.pl -config=mysite -output=allrobots -staticlinks > awstats.mysite.allrobots.html
perl awstats.pl -config=mysite -output=lastrobots -staticlinks > awstats.mysite.lastrobots.html
perl awstats.pl -config=mysite -output=urldetail -staticlinks > awstats.mysite.urldetail.html
perl awstats.pl -config=mysite -output=urlentry -staticlinks > awstats.mysite.urlentry.html
perl awstats.pl -config=mysite -output=urlexit -staticlinks > awstats.mysite.urlexit.html
perl awstats.pl -config=mysite -output=browserdetail -staticlinks > awstats.mysite.browserdetail.html
perl awstats.pl -config=mysite -output=osdetail -staticlinks > awstats.mysite.osdetail.html
perl awstats.pl -config=mysite -output=unknownbrowser -staticlinks > awstats.mysite.unknownbrowser.html
perl awstats.pl -config=mysite -output=unknownos -staticlinks > awstats.mysite.unknownos.html
perl awstats.pl -config=mysite -output=refererse -staticlinks > awstats.mysite.refererse.html
perl awstats.pl -config=mysite -output=refererpages -staticlinks > awstats.mysite.refererpages.html
perl awstats.pl -config=mysite -output=keyphrases -staticlinks > awstats.mysite.keyphrases.html
perl awstats.pl -config=mysite -output=keywords -staticlinks > awstats.mysite.keywords.html
perl awstats.pl -config=mysite -output=errors404 -staticlinks > awstats.mysite.errors404.html
¹If you prefer, you can use the awstats_buildstaticpages tool to build all these pages in one command, or to generate PDF files.
Notes:
a) You can also add a filter on the following reports: urldetail, urlentry, urlexit, allhosts, refererpages. The filter can be a regexp (regular expression) on the full key you want AWStats to report on and is appended to the output parameter separated by a ":".
For example, to output the urldetail report, including only pages which contain /news in their URL, you can use the following command line:
perl awstats.pl -config=mysite -output=urldetail:/news -staticlinks > awstats.mysite.urldetailwithfilter.html
b) If you want to build a report for a particular month, add the options -month=MM -year=YYYY where MM is the month expressed as two digits, i.e. 03, and year is the four digit year. To build a report for a full year, add the options -month=all -year=YYYY (warning: this is often resource intensive and might use a lot of memory and CPU. Unix/Linux like operating systems might benefit from use of the "nice" command.)
2) The second option is to dynamically view your statistics from a browser. To do this, use the URL:
http://www.myserver.mydomain/awstats/awstats.pl?config=mysite
where mysite specifies the configuration file to use (AWStats will use the file awstats.mysite.conf).
All output command line options (except -staticlinks and -logfile) are also available when using AWStats with a browser. Just use them as URL parameters: change "-option" to "&option", i.e. http://www.myserver.mydomain/awstats/awstats.pl?month=MM&year=YYYY&output=unknownos
Reports are generated in real time from the statistics data base. If this is slow, or putting too much load on your server, consider generating static reports instead.
If the AllowToUpdateStatsFromBrowser parameter is set to 1 in AWStats configuration file, you will also be able to run the update process from your browser. Just click on the link "Update now".
CVS Install on Linux
1 Create a cvs user and a cvs group to own repository files. We won't give any password to this user, so login will be disabled. We'll later add real users to the cvs group.
$ su
Password: {enter password}
# /usr/sbin/groupadd cvs
# /usr/sbin/useradd �CM �Cg cvs cvsadmin
#/usr/sbin/useradd �CM �Cg cvs cvscommon
This command normally creates both a user named cvs and a group named cvs.
2 Create cvs repository directory in a location of your choice
# mkdir /var/cvsroot
# chown cvsadmin.cvs /var/cvsroot
3 Set the 'group sticky bit' on all directories used by CVS
# chmod -R g+ws /var/cvsroot
4 Initialize the repository using cvs init
# su cvsadmin
$ cvs �Cd /var/cvsroot init
$ ls -l /var/cvsroot
total 4
drwxrwsr-x 3 cvs cvs 4096 Mar 5 05:32 CVSROOT
5 Setting up config
$ cd /var/cvsroot/CVSROOT
$ chmod u+w config
$vi config
... uncomment or change the line that defines auth...
for example: SystemAutho=no
6 Add virtual cvs users
$ htpasswd -c �Cd /var/cvsroot/CVSROOT/passwd admin
Input password for admin
$ htpasswd �Cd /var/cvsroot/CVSROOT/passwd tester
Input password for tester
Map the virtual cvs users to system user, and the format is "cvsuser:password:systemuser".
$ vi /var/cvsroot/CVSROOT/passwd
Append ":cvsadmin" to admin, and append ":cvscommon" to tester.
$ exit
7 Start CVS Server
Check if the cvspserver exists in the path "/etc/xinetd.d", if not please create one under Linux as following.
#vi /etc/xinetd.d/cvspserver
service cvspserver
{
id = cvspserver
flags = REUSE
socket_type = stream
protocol = tcp
user = root
wait = no
server = /usr/bin/cvs
server_args = -f --allow-root=/var/cvsroot pserver
disable = no
}
#service xinetd restart
#netstat �Cna |grep 2401
The port for cvs server is 2401, and check the server status.
9 Check the CVS Server status
# export CVSROOT=:pserver:tester@localhost:2401/var/cvsroot
#cvs login
Input your password. All is ok if no problem.
#cvs logout
10 Update cvs version
#cd /usr/local/src
#wget https://ccvs.cvshome.org/files/documents/19/874/cvs-1.12.12.tar.gz
#tar zxpvf cvs-1.12.12.tar.gz
#cd cvs-1.12.12
#./configure
#make
#make install
Install is finished now, please do check again.
#cvs �Cversion
#cvs login
Input your password for tester
#cvs logout
/**
* Created by IntelliJ IDEA.
* User: Henry Zhu
* Date: 2007-5-14
* Time: 9:28:40
* To change this template use File | Settings | File Templates.
*/
import java.util.*;
public class random {
public static void main(String[] args)
{int[] num={1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32};
Random r=new Random();
//StringBuffer str1=new StringBuffer();
int com=0;
System.out.println("The common six number are:");
for(int i = 0; i < 6; i ++) {
com = r.nextInt(32 - i);
System.out.println(num[com]);
for(int j = com; j < 32 - i - 1; j ++) {
num[j] = num[j + 1];
}
}
int[] num2={1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16};
Random r2=new Random();
int spe=0;
spe=r2.nextInt(16);
System.out.println("The special number is: " + num2[spe]);
}
}
KsharP
2007-05-14
Henry Zhu
QQ：284841013
MSN:ksharp1229@hotmail.com
Email:zhuheng1229@gmail.com
Homepage:http://ksharp.anyp.cn
| 用awk查询用户登录次数 | |
| last |awk '{print $1}'|awk '{ for (i = 1; i<= NF; i++) freq[$i]++ } END { for (word in freq) printf "%s\t%d\n", word, freq[word]}' | |
