Initialisation
Running in this mode could be quite long
if your logfile are huge, but you'll need to run it only once. So after a
successfull installation, you just have to run the 'cron-w3perl.pl -a'
command (or use the admin interface) which will compute everything from scratch.
If you make important changes to your configuration file, you maybe need
to re-init your stats or any changes you have made will only occur for the
next incremental run.
Incremental
This is the update mode. The scripts are designed to be run once a
day.
To avoid unecessary CPU use, incremental
stats are computed between the last run and today. By default, the maximum
number of days between two incremental run is 7 days but it could be
change. Time taken to update all your stats is a few minutes depending
on your average logfile increase. Running 'cron-w3perl.pl -e' will
update all your stats.
A day took typically 3 seconds to compute for an average 6000 hits/day website and 15-20 seconds for a 40000 hits/day website.
Initialisation benchmark
To give an idea how long it could be, this is the time taken to
complete the stats on my XP2500+, 512 Mb RAM.
Website |
Intervalle |
Logfile size |
Hits / day |
CPU |
Speed |
W3Perl |
1216 days (40 months) |
1.7 Gb (124 Mb compressed) |
6 000 |
41 min 50 |
3550 lines/sec |
INRP |
129 days (3 months) |
1.2 Gb (154 Mb compressed) |
61 000 |
39 min 26 |
3970 lines/sec |
|
(Logfile 1.7 Gb) |
CPU |
Speed |
Relative speed |
reverse dns off |
41 min 50 |
3550 lines/sec |
100 % |
geoip_free module |
1 h 02 min 55 |
2280 lines/sec |
65 % |
netgeo module |
1 h 04 min 15 |
2280 lines/sec |
65 % |
reverse dns on |
9 h 12 min 20 |
270 lines/sec |
7 % |
|
Level of precision is by default three and reverse dns is off.
Compression rate is around 90% of raw logfiles.
To get speed increase, disable reverse dns (very slow) but you won't
be able to get country stats. If you really need country stats, use the
Geo::IP perl module (which is a reverse dns file stored locally) or
use the fastresolve tool to translate IP to hostname before running
W3Perl. Turning off robot detection or referer spammer filtering can increase speed also.
Avoid using the highest level of precision with huge logfiles as it will require a lots of RAM
to process.
|