View Full Version : error_log and access_log questions
I was looking at my error_log file tonight and noticed 2 strange things. I see the same IP hit 116 different pages that do exist on the server, all in 4 minutes time. Any ideas what that is?
Also I see an error to a page and directory that has never existed on my website...
[Sun Apr 20 23:01:18 2003] [error] [client 212.179.247.161] File does not exist: /www/x/xxx/htdocs/eyeblaster/addineye.html
I've never had a 'eyeblaster' directory or .html file with that name. Any ideas what this is too?
I know the access_log file rolls over once a week but I thought I read something that it might be going to once a day. Is there any word on that, or am I remembering wrong? I ask because my access_log is huge! Right now it's at 38Mb....how do you read a file that big? ;)
Thanks for any help :)
Deb
G'Day Deb,
Don't know about the error log, probably someone just typed the wrong url in and kept trying.
To split the log file up have a look at this info.
You would need a phpscript to do this.
<? system ( 'cd /www/u/username/logs; split -l1000 access_log' ) ?>
This splits the log file into 1000 line files. It of course doubles the access_log usage to 38MB*2.
HalfaBee
SPLIT(1) FreeBSD General Commands Manual SPLIT(1)
NAME
split - split a file into pieces
SYNOPSIS
split [-a suffix_length] [-b byte_count[k|m]] [-l line_count]
[-p pattern] [file [name]]
DESCRIPTION
The split utility reads the given file and breaks it up into files of
1000 lines each. If file is a single dash (`-') or absent, split reads
from the standard input.
The options are as follows:
-a Use suffix_length letters to form the suffix of the file name.
-b Create smaller files byte_count bytes in length. If ``k'' is
appended to the number, the file is split into byte_count kilo-
byte pieces. If ``m'' is appended to the number, the file is
split into byte_count megabyte pieces.
-l Create smaller files n lines in length.
-p pattern
The file is split whenever an input line matches pattern, which
is interpreted as an extended regular expression. The matching
line will be the first line of the next output file. This option
is incompatible with the -b and -l options.
If additional arguments are specified, the first is used as the name of
the input file which is to be split. If a second additional argument is
specified, it is used as a prefix for the names of the files into which
the file is split. In this case, each file into which the file is split
is named by the prefix followed by a lexically ordered suffix using
suffix_length characters in the range ``a-z''. If -a is not specified,
two letters are used as the suffix.
If the name argument is not specified, the file is split into lexically
ordered files named with prefixes in the range of ``x-z'' and with suf-
fixes as above.
Originally posted by cdnmama
I was looking at my error_log file tonight and noticed 2 strange things. I see the same IP hit 116 different pages that do exist on the server, all in 4 minutes time. Any ideas what that is?
Deb
It could've been a spider of some sort. Maybe a search engine, maybe a spammer looking for email addresses. It may also have been someone storing your entire site for offline browsing, or someone trying to just copy your site for other reasons. Or maybe a 4-year old clicking on links. :)
Spiders and spammers only follow links from valid webpages.
Hackers try and find links to folders that are not on homepages or search engine pages.
I always try and change the default folders to avoid this problem.
HalfaBee
Originally posted by HalfaBee
To split the log file up have a look at this info.
You would need a phpscript to do this.
<? system ( 'cd /www/u/username/logs; split -l1000 access_log' ) ?>
Hi HalfaBee....thanks for replying :) Your suggestion for splitting the log went over my head unfortunately. No idea what that all means ;)
Originally posted by satis
It could've been a spider of some sort. Maybe a search engine, maybe a spammer looking for email addresses. It may also have been someone storing your entire site for offline browsing, or someone trying to just copy your site for other reasons. Or maybe a 4-year old clicking on links.
LOL I'm thinking you might be right about some of these possibilities. If someone was trying to take the entire site, they only took one third. Can't see someone clicking on 116 links in 4 mins though ;)
Deb
Sorry Deb :)
Cut and paste the code and save it as split.php then upload to htdocs and run from your browser.
You may want to change 1000 to 10000 as it will create a lot of files all with 1000 lines in them.
HalfaBee
Originally posted by HalfaBee
Sorry Deb :)
Cut and paste the code and save it as split.php then upload to htdocs and run from your browser.
You may want to change 1000 to 10000 as it will create a lot of files all with 1000 lines in them.
HalfaBee
Hi.....I tried this line as you suggested and it didn't work for me
<? system ( 'cd /www/x/xxx/logs; split -|10000 access_log' ) ?>
I did replace the x's with the proper username. I uploaded the split.php and called on it with the browser but the page was blank?
http://dreamworkdesigns.com/split.php
I must be doing something wrong....any ideas? thanks......Deb
It has probably done it.
check your logs file with ftp and there should be lots of files.
HalfaBee
All I see in the LOGS folder are the same 4 files as there was before....hmmmmm
Deb
Try again with this, make sure you cut and paste, not type it. I think you put a | not a l.
<? system ( 'cd /www/x/xxx/logs; split -l10000 access_log;la' ) ?>
HalfaBee
I got it working now :) Silly me replaced the l with the | thinking it was right ;)
When I pasted it over to Notepad, I thought it was a one...hehe
Thanks for your help HalfaBee :)
I hate this font for programming and scripts.
How many files did it create?
HalfaBee
it created 15 files about 2Mb each.....much better for reading if I need to.
Would I have to run this every week when the access_log rolls over? or will it automatically create all the files now?
Deb
It only creates the files when you run the script.
If you want to look at the previous weeks data change the file name to access_log.txt
Halfabee
Thanks again for your help......it's good to learn something new everyday :)
Deb
Not a problem Deb.
If we stop learning we are dead.
The split command is a really useful unix command. It can used for all sorts of splitting. I used it to fix up an old DB I had. Luckily it was a fixed length one and I was able to split it into 512byte chunks and restore what I needed.
A 38MB log file would be so hard to do anything with. Even downloading would be a pest.
Good luck with it.
HalfaBee
vBulletin v3.6.0, Copyright ©2000-2009, Jelsoft Enterprises Ltd.