robot.txt??!#?!#?!

[powerkor]

i uploaded formail.pl and the dir's are chmod'd right, and i uploaded the file in ascii mode..

but i get this error

[Fri Feb 8 20:55:23 2002] [error] [client 198.139.155.86] File does not exist: /www/p/powerkor/htdocs/robots.txt

im wondering y, please help thanx

[powerkor]

im thinking it has to do with powweb, im not sure...

Im trying to use my own FormMail.pl 1.9, the new version

[sophiespo]

robots.txt has nothing to do with powweb.

When a search engine goes through your site, it looks at everything and anything in your space. You can create a file called robots.txt which contains a list of files/folders which you want the search engine to ignore. I dont know much about this, I dont know how the robots.txt is supposed to be set out and Im also not sure if you have to add something to your .htaccess file for it to work, but I do know that not all search engines take any notice of it.

In short, a search engine has gone through your site, and it looked for a robots.txt file to see if there was anything you wanted excluded from the search. Because you dont have a robots.txt file, it showed up in your error_log. Its nothing to worry about. It happens a lot.

sophie

[Bud]

If you would like your site to be found in the search engines, then the error you received is a good sign.

The search engine has indexed your site and asked for your robots.txt file to see if any files/directories should NOT be indexed.

Not all search engines abide by the robots.txt file, but most legit ones do.

The robots.txt file format is something like:

User-agent: *

Disallow: /admin/

Disallow: /images/


It should be placed in your htdocs directory and contain all files and directories you do not want to show in search engines. It does not utilize the .htaccess file.

~bud :)

[SBGlasby]

seeing a listing in you'r error logs... or anything else that says...

cannot find robots.txt file can be a good thing or a bad thing...

depending on who is trying to access it....

you can use the robots .txt file to more or less control... whcih... if any or all searchengines see on your site....

a good place to start is:

http://www.searchengineworld.com/robots/robots_tutorial.htm

[SimCoWeb]

I believe powerkor was looking for help with the formmail script.... not the SE robots! :D

powerkor, did you place the 'formmail.pl' script inside your cgi-bin? If so, did you create its own sub-directory inside the cgi-bin and chmod it to 755 as well as the script itself?

I haven't viewed the new version of formmail (1.9) but based on older versions there's a part of the script you need to change regarding 'referrers'. This is a security setup to make sure that outsiders aren't using your script on your server to parse their forms. It should say something like '@referrers {'www.blahblah.com', 'blahblah.com'}

But don't look for THAT specific line. It's just an example :) There will be line with something to do with 'referrers'. You need to set those to your URL's.

In your form, there is one hidden fields that HAS to be there. It's 'recipient' .... which is your email address. You can also have 'subject', and a 'redirect' which will take the user to the page your designate after they complete the form. If you don't use the 'redirect' then it will show the default thank you/confirmation page from the script...which is kinda ugly :D

[Bud]

Sim, we were trying to clairify the error, I believe...and let powerkor know it has nothing to do with the problem he/she is having ;)

~bud

[powerkor]

Thanks to all that replied, it seemed that the server or soemthing was lagged and just needed alittle time to update or something cus the formail works fine now.

Also, thanx for the tip of robots.txt ill be sure to utilize that :)