Viewing traffic details via logs

[sammiedawg]

I read somewhere online that you may view traffic details (Where people are coming from, which keywords they searched for, etc) in your log files.

Where are these log files located? I went into OPS and I only saw "error logs" under services.

Thanks!

[pjeutr]

I'm also searching for the access_logs to see some more detailed information.
Anyone knows where to find them?

To see statistics you can go to http://yourdomain.com/stats/
Maybe you have to enable them in OPS under services>HTTP

[Neat Pete]

Sign on to OPS.

Select Services, then HTTP then Log Files.

Under the paragraph Copy Traffic Statisics there is a combo box and a button marked Submit.

Set the box (I use the daily setting) and click on Submit.

The log files will be copied to your account area at the interval you have selected.

You can then download your log files using FTP. They are in the root directory, not in /htdocs, and are not visible to the internet.

Mine appears most days a bit after midnight (Boston time) and I download it.

My experience is that PowWeb often delete their own copy of the log files in error (making the Webalizer stats totally wrong), it pays to download your own copy promptly and analyse it yourself. Don't get into all this analysis stuff unless you have a clear plan of what you want to do, otherwise just download a couple of log files and search through them. They are written to the standard "extended" log format.

[pjeutr]

thnx, guys

@keyplyr
I'm really interested in having logs for different subdomains.
But when I Iook in the log, that I downloaded, I only see stuff like "GET /index.html HTTP/1.0"
I don't see any information about subdomains. Do I have to configure something else?

[sammiedawg]

I downloaded my "log" file & the "analog program" - However, I don't really quite understand how to install this Analog program. I unzipped the Analog zip file into a folder and received 400+ images and .html files.

Any help would be greatly appreciated - THanks

[Neat Pete]

All my sites (three URLS, and several more specialities, at least in my mind) an come in the one lot of Webalizer stats. The first thing I did was increase the top file names list to 200 and the top search strings to 100. This means the normal Webalizer stats are sufficient for many purposes.

For specialist analysis per-site, I download the log files and have written a fairly complex set of routines to work out the "web site" for each line in the log file.

I have my own list of sites plus Redirected (files which no longer exist or are on-redirect; search engines keep requesting files years after you delete, rename or rearrange them), Errors (calls for rubbish you have never heard of), Worms (typically malicious calls trying to detect Windows servers) and Combined (where the analysis below has failed)

In brief, here is how the analysis works to decide the web site, given the request to the server and the rest of the log line. The analysis stops as soon as you get an answer...

(1) Decide the web site from the directory structure.
(2) Decide the web site from the actual file name, if the directory structure is shared or ambiguous.
(3) If the two file names are the same, try to decide the web site from the size of the file served.
(4) Decide the web site from a few obvious cases in the search string.
(5) Otherwise you can't nail it, so it has to be "Combined".

The code can be regularly improved by analysing the files that went to "Combined". There is no "one line" solution to all this.

[sammiedawg]

Thanks for the info

I clicked on Analog.exe and I saw a Dos prompt pop-up and then close by itself

I then changed the path in the analog.cfg file. I tried the regular "access_log" and ".gz" file.

I then clicked on reports.html, but I still see the default information in that file.

Did I miss any steps?

Thanks

[sammiedawg]

Keyplyr mentioned that there should be a Report.html file, but I only see reports.html. Are they the same file?

In analog.cfg, there is a " .* " after the path name. Do I delete that?

Thanks!

[Neat Pete]

Whats the difference between Report.html file and reports.html?

[sammiedawg]

I got it to work finally :) Thanks for the help!

What is the difference between "downloading the logs, using Analog, and viewing the stats" versus just going into OPS "Services ---> HTTP ---> Visitors Statistics"?

Thanks

[sammiedawg]

This may be a dumb question, but what are robotic requests?

[Builder]

Probably search engine bots, i.e. googlebot et al.

Kevin

[Neat Pete]

Definitely, robotic requests means robots visiting your site. Apart from occasional errors in the log, there are only two types of visitors to a web site - Humans and Robots. It's important to think very differently about these two cases. What do they want from your site for starters??? Who told them to visit you??? Are they only looking at the pictures??? Have they visited before??? Are they criminals and vandals, for whom the guillotine is surely inadequate.

Here is a link to an interview with Dr Stephen Turner, the author of Analog....

http://www.kaushik.net/avinash/2006/07/ten-minutes-with-dr-stephen-turner.html

[digitalkidz]

i used Awastats and noticed under "origins - referring sites," there are a bunch of links to porn sites. Why would porn sites refer to my site?

How can I stop them?

[entrecon]

I think those are self promoting links. I have heard of them doing this in case you click on them and/or if your stats are public it helps with search engines as it is a site linking to them.

[digitalkidz]

is there a way that i can stop porn sites from doing this?

[sammiedawg]

What are some "must-have" add-on's when customizing the Analog log?

Currently, I have the following:

General Summary
Quick Summary
Monthly Report
Daily Summary
Hourly Summary
Domain Report
Organization Report
Search Word Report
Operating System Report
Status Code Report
File Size Report
File Type Report
Directory Report
Request Report

Thanks again for the info! :)

[sammiedawg]

thanks for the links