My prediction: rel = "nofollow" will not prevent comment spam
January 19, 2005 2:19 PM
Google, MSN, Yahoo, and others have decided that the new best way to prevent comment spam is to use an HTML tag that will prevent Google (and other search engines) from following links that could be spam links.
That new tag works as so:
<a href="url" rel="nofollow">Link Text</a>
When indexing sites, this will prevent Google and others from "following" these links. Normally following those links helps a site's PageRank, which makes those sites appear higher in search results. The theory is that if spammers aren't getting their PageRank improved, then they will stop spamming blogs. Ha!
My prediction is that this change will not prevent comment spam because of the following issues:
- The text of the spam messages will still be indexed by search engines.
- Humans will still see the the spam messages.
- Humans will still be able to follow the spam links.
- Most importantly: Spammers can still post spam messages.
In short, all this does is make life a little easier for Google. Their PageRank system, which might just be flawed, has been abused and blamed for the onslaught of comment spam. Now they have an out - an excuse to to say it's not their fault anymore. I'm all for better search results in Google (ever try to search for a specific hotel?), but their solution simply does not prevent comment spam.
What do I recommend?
- For automated spam bots: To prevent bots from posting spam comments, I require JavaScript. When a human user clicks the Submit button, JavaScript to renames field names before the comment is submitted to the server. Fields are unique named every time the page loads, and the server will only post comments when it gets the field name it is expecting. This prevents the automated spam attacks because the software spammers are not able to predict the field names.
- For manually entered spam comments: I have basic spam filtering mechanism similar to many email spam filters. It looks for common spam words, URLs, and topics, and prevents those messages from being posted.
- Either way, I have a RSS feed which shows me whenever a spam comment is attempted, along with IP address and other information so I can track the progress, and watch for false positives (real comments that the system thought were spam), and easily ban IP's of known spammers.
Something important to me is that my solutions stop comment spam without requiring any extra effort from my users, Some sites now require registration or a CAPTCHA input to add a comment. I feel that this is just an unnecessary pain which prevents people many busy people adding their feedback. I also feel that the links in comments are often important enough that search engines should follow them, therefor always putting a nofollow tag will ultimately be unhelpful to those small sites that should get a higher PageRank.
And most importantly, unlike the nofollow "solution" from Google, my recommendations can actually prevent spam comments from appearing on sites, and that's what we all want, right?
Comments
Related Posts
Category: Google
- First match on Google in just a couple days September 13, 2006 2:02:00 PM
- Google vs Yahoo June 8, 2005 11:59:00 PM
- MSN Virtual Earth to take on Google Maps May 24, 2005 11:59:00 AM
- Google suspends downloads of Google Web Accelerator May 12, 2005 1:34:00 AM
- Google Web Accelerator - Let Google Watch You Do Everything... May 5, 2005 3:20:00 AM
- 15 more in Google...
Category: My Blog
- Gnomedex is two weeks away! June 9, 2005 6:25:00 PM
- My site mentioned on 98 Rock this morning May 18, 2005 4:14:00 PM
- 300 people reading my feed via RSS April 22, 2005 3:35:00 AM
- I burned my RSS feed April 21, 2005 1:14:00 AM
- Tag clouds aren't cool anymore? April 20, 2005 1:24:00 AM
- 55 more in My Blog...
Category: My thoughts
- Hardware manufactures - stop overstuffing the Start Menu! March 23, 2006 5:16:00 PM
- Automatic Bull-Sh*t detector extension for email, blogs, and browsers March 3, 2006 4:55:00 PM
- Need a better Podcast/Blogcast downloader June 20, 2005 3:56:00 PM
- A possible future of operating systems June 9, 2005 12:25:00 AM
- 1 more in My thoughts...
Category: Ideas
- Problems with Web 2.0, and how to fix them for Web 3.0 March 29, 2006 5:08:00 PM
- Automatic Bull-Sh*t detector extension for email, blogs, and browsers March 3, 2006 4:55:00 PM
- IE and Firefox feature request: download visible images first April 12, 2005 2:09:00 PM
- Shift-click needs more power April 12, 2005 12:25:00 AM
- Fun ways to do your taxes April 6, 2005 7:59:00 PM
- 32 more in Ideas...
Posted January 19, 2005 3:43 PM
In reply to your first suggestion, it's a sound suggestion. However JavaScript wouldnt work. Remember JavaScript is client side; if the JavaScript generates a random field ID the server will have no idea what it is and will not be able to validate.
Instead you could use a server side script on the "post comment" page that would generate the random id and store it in a session or cookie. Then the server would be able to find out what the random ids are supposed to be.
http://www.DylanGreene.com
Posted January 19, 2005 4:15 PM
Actually a public key is also included as a hidden field. The JavaScript uses the public key to compute the field names. The server knows what fields to expect based on the public key.
http://www.DylanGreene.com
Posted January 19, 2005 4:17 PM
I decided not to use cookies because some users disable them for privacy reasons, and I didn't use session variables in case you are browsing multiple pages at once.
The one problem with requiring JavaScript is that I can't post comments from my phone. :(Posted January 19, 2005 7:14 PM
Well, my knowledge of crypto is very limited, but would that actually work? As I see it, the JavaScript would need two values, the key to encrypt it with, and some other piece of data. Now the other piece of data would have to be diferent everytime otherwise the same result would be generated and spammers would just compensate for this. You could use the date and time, but the time between the user submitting the form and the next page generating might be nominally diferent...
...unless I'm misunderstanding you?
http://netinstitute.com
Posted January 19, 2005 11:08 PM
If Google wanted to stop the spamming of blogs, they would have proposed a real solution that stopped spamming, (like yours). It seem that, from Google's perspective, reducing the search engine rankings of every blogger who has ever posted a comment or used trackback is either a good thing or at least not bad. That so many bloggers are applauding a move that will dramatically reduce their voice while doing very little to reduce spam is puzzling to me.
http://www.DylanGreene.com
Posted January 20, 2005 1:48 AM
Nathan - I'm explaining it without giving away all the secrets. The public key and the public field name are the input for an algorithm that produces the field name that the server is looking for. That algorithm is easily viewable by viewing my source code, but my assumption is that the tools that comment spammers use do not and will not support JavaScript any time soon.
http://www.dylangreene.com/albums/473
Posted January 20, 2005 1:55 AM
Bloggingpro.com agrees!
http://www.bloggingpro.com/archives/2005/01/19/rel-nofollow-will-not-prevent-comment-spam/http://www.dylangreene.com/albums/473
Posted January 20, 2005 1:56 AM
The Unofficially Yahoo Weblog agrees!
http://yahoo.weblogsinc.com/entry/1234000803028085/Posted January 20, 2005 3:24 AM
Aha, I see. ;)
http://www.2-downloads.com
Posted January 20, 2005 9:44 AM
So in my opinion, the Blog software developers must include "random text image" validation in the forms. That will kill spam bots for 100%. I newer heard about workaround of this feature. For human it will increase the time for comment, but it will be ok I think.
I other hand javascript pop up pages will not be crawled in any way...
This 2 simple techniques will reduce spam comments a lot.
http://www.dylangreene.com
Posted January 20, 2005 10:10 AM
It seems like this person agrees too, but it's in German and automatic translation tools are only so good...
http://segert.net/weblog/item.php?i=109http://richardathome.no-ip.com
Posted January 20, 2005 11:08 AM
Couldn't agree more. Infact, I blogged more or less an identical article to this yesterday: The Perils of rel='nofollow'
You offer some sane advise on cutting down comment spam. Although its now pretty trivial to develope a spam bot that is Javascript enabled.
I'm currently developing a PHP script that is pretty effective at filtering out referal/trackback spam: An Automatic Solution to Referral/Track-Back Spam. Any feedback is always apreciated :-)
http://richardathome.no-ip.com
Posted January 20, 2005 11:09 AM
hmmm... my links got lost somewhere in the post.
The perils of rel='nofollow' : http://richardathome.no-ip.com/index.php?article_id=416
An Automatic Solution to Referral/Track-Back Spam : http://richardathome.no-ip.com/index.php?article_id=415
http://www.einfach-persoenlich.de/
Posted January 22, 2005 3:24 AM
>> rel = "nofollow" will not prevent comment spam
Correct!nofollow - noLinks, noComments, noTrackbacks, noPageRank?
http://www.sarkis-webdesign.com/
Posted May 3, 2005 9:12 AM
Hi Joerg,
usually I used nofollow - noLinkshttp://www.internet-marketing.cd
Posted July 24, 2005 5:51 PM
I think there are better possibilities than nofollow. I have different directories without nofollow. Some of them are free for nearly all users. But I have written some strong rules for the users and I control everytime what they write in. I delete non- quality- comments and sites. O.K. that`s a lot of work. But in end I have a quality directory and good PR- Links for quality websites and descriptions inside.
http://usefulresources.co.uk/
Posted September 21, 2005 2:03 AM
Greets all,
MSN Bot is still crawling my site's bits which were rel="nofollow" :S
anybody know what the hell is going on?
<a href="index.php?do=disclaimer" rel="nofollow" target="_self">Disclaimer</a>
65.54.188.61 - - [21/Sep/2005:04:49:09 +0100] "GET /index.php?do=disclaimer HTTP/1.0" 200 9873 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)"
http://www.alah.hu
Posted October 18, 2005 10:20 AM
"PREDICTION: Dear Readers, Do you want that Palestine will be free? If your answer is yes, I recommend you the following home page: http://www.alah.hu Please send this letter to your friends! Sincerely, Zoltan Biro
http://www.pdesigner.net
Posted November 29, 2005 9:37 PM
No follow is to prevents search enhines from scrwling, however you can use robot.txt in your folder where prevented pages are.
http://www.suchmaschinen-optimierungen.info
Posted January 20, 2006 7:08 AM
Yes, I agree with Your prediction. In my opinion You can use this tag for sites like contact. So no pagerank goes to sites without real content.
Posted January 24, 2006 2:44 PM
Google PageRank carefully explained and what you can do with it - written by top SEO experts.
http://www.pagerank-prediction.com/
http://www.pagerank-prediction.com
Posted January 24, 2006 2:45 PM
Google PageRank carefully explained and what you can do with it - written by top SEO experts.
http://www.pagerank-prediction.com/
http://www.control4server.de
Posted February 22, 2006 6:55 PM
check your site free seo tools
http://www.stoorm.free.fr
Posted March 7, 2006 8:40 PM
another site with good tools
http://www.creativewed.co.uk
Posted March 17, 2006 5:49 AM
I don't think it will stop spam either. What it may do is put off normal users from posting comments.
http://www.usefulresources.co.uk/
Posted March 29, 2006 12:45 PM
Any views on the fact that msn bot ignores rel="nofollow"
http://www.singermachines.co.uk
Posted March 30, 2006 3:17 PM
We dont use rel=nofollow, however there are times when it is handy and you always have the option, either to use it or not, depending on the situation required.
http://www.nivid.com
Posted May 23, 2006 1:10 PM
even rel=nofollow dosnt stops SEBots to visit spamming sites.
http://www.adamfreelance.org
Posted October 11, 2006 1:44 AM
rel=nofollow can stop seBots to visit spamming sites, but it must be coinsided first by the programming. Which depends on the language.
http://www.standardwarfare.com/
Posted October 15, 2006 5:37 PM
I have seen this explaination in other places and I have to say I agree, it won't prevent spam.
http://www.millr.net/
Posted October 30, 2006 2:41 PM
I think having this added to comments on your own blog will only put people off wanting to post on your blog in the first place, not just put off spammers.
http://www.designerpk.com
Posted December 4, 2006 7:16 PM
156 SEO Tools
This SEO Tools page has links to the best SEO Tools on the internet and these tools will help you to optimize your website and move your search engine position higher.
http://www.seocompany.ca/tool/seo-tools.html
http://andybeard.eu
Posted December 5, 2006 5:07 PM
It is 22 months on, and your prediction was correct
What is worse, nofollow has unbalanced the natural linking of the net with many sites maintaining an unnatural amount of pagerank, especially when they benefit from unreciprocated trackbacks.
Comments on blogs have some extremely useful benefits, and this is partially how I actually arrived at this site.
A fresh comment on a blog for some reason makes it appear back on the radar for Google, and more specifically Google Alert. I have a Google alert setup for "nofollow" discussion, because I am in the middle of building a community site that supports people who don't use nofollow.
For some strange reason, today I received an alert because of the previous comment.
http://www.freetube.us.tc
Posted December 19, 2006 7:48 PM
the nofollow tags are basically pointless, because outbound links from your site do not affect your sites pagerank or rating - so most sites are just paranoid and scared and use these tags thinking it will increase their pagerank in the long run. The rel no follow tags should only be used in dire cases, when your receiving spam site submissions.
Basically most blogs should not use nofollow tags, because it penalizes your commenter's - if you let commenter's leave their link without the rel = nofollow, you reward them for commenting on your site and thus you'll actually get more users posting on your blog. People see it as a way to help the blogger, as well as be rewarded for commenting on a bloggers site. But using the nofollow tags is discouraging and some commenters may leave because it seems like a blog owner is being paranoid or too selfish. Remember some outbound links are needed to increase PR.
http://www.affiliatebestprograms.com
Posted May 11, 2007 9:03 PM
I agree with Daringer, using the nofollow tag penalizes legitimate commentors. If you really want to eliminate comment spam, the best way to do it is through moderation. I have all my blogs set so that each comments must be approved before it is posted. I've also found that setting the blog so that people can post without signing up attracts more legit posters.
btw, I found this post while researching for a similar article regarding using the nofollow tag, but I was searching for a way to add a nofollow tag to certain links in my blogroll while leaving other links with natural flow. If you're using Wordpress and are interested in adding code to your blog enable this feature, I've posted a solution here..
http://www.affiliatebestprograms.com/2007/04/28/seo-tips-the-nofollow-attribute-and-your-blog/
http://www.sanfranciscogiants1.info
Posted May 25, 2007 7:38 PM
Im debating on whether its worth it or not because how many people actually check before they comment if there is a nofollow or not? Its gotta be very low. However if you have a website that is only for SEO then I guess it is a wise move.
http://www.seegeorgiarealestate.com
Posted June 29, 2007 10:21 PM
I think it still needs some tweeking according to our friends at <a href = "http://www.seegeorgiarealestate.com">Atlanta Real Estate</a>
http://www.dollarsblog.com
Posted July 31, 2007 1:42 AM
Your right it won't increase blog spam. It will increase comments though on blog I think. Ill change it to dofollow on my site once I figure out how to do it.
http://webgrrrl.net/
Posted July 31, 2007 11:02 AM
I'm using nofollow solely because I hope my PR will increase and that my web site visitors will reciprocate by removing nofollow from their links as well.
http://www.pmi-fl.com
Posted November 7, 2007 4:42 PM
I agree completely. I've read an interview with a known spammer (as I'm sure at least a few of us have) who didn't seem phazed at all by this weak attempt at comment spam control. He basically stated that as long as the text is still there and people can still click through, he is accomplishing his mission. The best spam protection is human moderation.
Posted December 13, 2007 11:00 PM
Canton Georgia Real Estate
Atlanta Real Estate agents
http://www.01interactive.com
Posted February 19, 2008 2:16 AM
Comment spamming in a blog about comment spamming :)
http://www.xn--ingilizcetercman-vzb.com
Posted February 24, 2008 4:25 PM
Your comment contains very useful information about all thank you
http://sftco.com/
Posted March 7, 2008 8:38 AM
Ya thats true, but it will control spam upto some extent.Like if no tag then we get lot more spam comments.
Posted March 30, 2008 3:58 AM
<a href="http://www.somgraphic.com/">Somali Web Design</a>
http://www.bestfeatherbeds.com/
Posted March 30, 2008 4:01 AM
Comments? what are they ;-)
http://www.winningponies.com/results/picks/tips/Kentucky-Derby-2007.html
Posted April 13, 2008 3:03 PM
Here is my share
http://www.ugg-boot-australia.com/
Posted April 23, 2008 9:22 AM
I also dofollow on my site.