Search Engine Friendly osC site at PowWeb

[zbasic]

I have concluded, by a combination of trial and error and research, that the "Force Cookie Use' configuration option cannot be employed when you are using a shared SSL certificate. I was trying to use this option, along with some URL rewriting rules, to make my site more search engine friendly.

It appears that the only good option is to use the 'Prevent Spider Session' configuration option together with 'Search Engine Safe URLs' and try to keep spiders.txt up to date. Of course, any spider whose User-Agent string isn't matched by spiders.txt will still get a session ID.

I'd be interested in hearing what others hosting at PowWeb have done in this regard.

[pureconcepts]

Sorry, I may not actually provide specific help. Nonethless, I say there have been a large number of post concerning SEF-ness and SEO this past week. Was there a new release concerning these topics? Did Santa promise a new toy for Christmas? What is with the frienzy?

I understand the need to bury session vars for reasons beyond just SEF-URL and SEO. However, I am a firm believer that in the end writting valid code with correct markup, as well as focusing on content go a long way. Okay, so maybe I pump some title or alt attributes here and there. But really though, this is a black-hole topic that unless you work for a large SE, you could get caught up in developing around or for something that may not even make a difference. Furthermore, you would never know because maybe your ranking went up or down one position. Which, of course, could be for a number of reasons and should not be considered evidence a certain method worked or did not work. This just fuels the fire.

I think SE's are black magic. Damn crazy, damn cool, black magic.

[tacimala]

I have developed two osC sites so far, but neither is focused too much on SEO. From what I have read though there are two things that may be able help you. One is by looking at the contributions available for SEO and to implement some of those properly. Specifically, the one for mod_rewrite to make search engine friendly URL's and another to create dynamic title tags for you based on the item category and name. A stock install of osC only uses one title tag throughout the entire site AFAIK.

The second way to increase search engine spider traffic and their reluctance to index your site is to create a better page for the cookies not enabled page that the spiders reach when they can't create a session on your site. From what I've understood the default page says something to the effect of "please turn on cookies to view this site" and some more of that jargon, but it doesn't have any good content. Find out where this page is at, leave some of that text at the top, but then make that page as search engine friendly as possible. Make some good anchor links with your keywords and create a site map of sorts. That way when the search engines end their visit on that page, they have some good content and good links to move on to instead of ending there.

A good way to test all this would be to measure your rankings for 8-10 keywords on googlerankings.com as you stand one day, make some changes, check on Google/Yahoo/MSN/etc. to make sure their cache updated for your new content/implementation and then check your rankings again. If there is a noticeable difference one way or another, make the appropriate changes/additions and move forward from there. Don't forget to follow up to help others!

[stevel]

The "Search Engine Friendly URL" feature of osC is worthless. Worse than worthless, in fact. It will harm your search engine rankings. You are correct that you cannot use Force Cookie Use here, but you don't want to, as it drives away customers. Using Prevent Spider Sessions and keeping your spiders.txt up to date is an important step. I maintain a spiders.txt as a contribution.

There is a good SEO contribution - this one (http://www.oscommerce.com/community/contributions,2823). I use this on my Cheshire Garden site and I have found it does improve search engine traffic. I didn't have much trouble installing this, some others do.

Other tips:

- Use robots.txt to keep search engines away from pages they shouldn't try for, such as shopping cart and login
- Add code that tests $session_started and displays links such as "buy now" only if a session is started. It's especially important to do this to suppress the product listing column sort links.
- Modify the product_info.php to display the product name in a H1 tag rather than just text
- Use the "All Products" contributiion
- Add code to prevent cPath= from appearing in product URLs

[zbasic]

My original post was focused on one aspect of SEO - structuring the site to produce URLs that are search engine friendly. There are much broader issues to contend with like choosing good keywords and using them properly in relavent copy, and properly titling pages. All of that is for naught if the search engines won't crawl your site because they don't like your URLs or if they have a session id in them.

So far, I have installed the Header Tags Controller (http://www.oscommerce.com/community/contributions,207) and I've modified robots.txt to try to keep spiders away from pages that are useless to it (login, shopping_cart, etc.).

[zbasic]

The "Search Engine Friendly URL" feature of osC is worthless. Worse than worthless, in fact. It will harm your search engine rankings.

In what way? Please elaborate.

[stevel]

Perhaps several years ago some search engines had problems with "dynamic URLs". Today this is not an issue. Google and the other major search engines can handle them just fine.

Keeping the session ID out of the URL is indeed important, for many reasons. That's where Prevent Spider Sessions comes into play.

The "SEF URL" feature of osC adds fake levels of directory nesting to your URL. So what was, say, example.com?cPath=24&product_id=56 turns into example.com/cPath/24/product_id/56 or some such. Google and perhaps some other search engines look at "how far away" from the home page an indexed page is, with a higher weighting given to pages that appear closer to the home page. It isn't a big effect, but it is there, and the SEF URL feature can result in pages deemed less important by the spiders.

The other problem is that the SEF URL feature breaks some contributions.

Now the SEO contribution I use puts the product and/or category name in the URL itself. I did not realize it earlier, but search engines consider "more important" a URL which contains the keyword people are searching for. Since implementing this, I have found the Google ranking of many of my pages much higher than before, many ending up on the first page for certain keyword searches. I can see the results in my incoming link stats.

I forgot to mention Header Tags Controller. That is indeed a must. You also want to make sure that the spiders don't see similar content on pages with different URLs, hence my suggestion to remove the cPath= from product URLs and to hide sort links from spiders.

[zbasic]

Excellent information, thanks.

I just finished modifying my site to remove certain links/buttons when $spider_flag is set. For example, "Add to Cart", "Login", "My Account", and "Cart Contents". Also, I modified the left column content so that the Shopping Cart box is not there when a spider is detected.

The exact edits that you would make depends on whether you have one of the templating systems installed; other modifications probably change things as well.

As a side note, a useful tool for testing changes like this is the User Agent Switcher (http://chrispederick.com/work/useragentswitcher) extension for FireFox/Mozilla. It lets you select which User Agent you want your browser to act like so you can see what a particular spider will see.