The people relying on these methods of analysis aren't the SEOs to be concerned about.

For those interested in a competitive intelligence, check out this blog post which contains 21 tools:

http://www.scrappybusiness.com/competitive-intelligence-tools.htm

"NOARCHIVE your pages to block the search cache"

I don't think I understand what this does. Could someone explain it? I assume it doesn't actually prevent the page from being cahced as that would be stupid.

The NOARCHIVE directive tells search engines not to display a public cache for the page. NOARCHIVING a page is a sign of competitive or stealth optimization (but it may be done for other reasons -- some people use it to protect intellectual property rights, for example).

The SEO tools that crawl the site and crank out pretty reports are just a small list of reasons to do these things but you need to get people's attention to get the word out.

Basically, my post just discussed bot blocking techniques from an SEO point of view, repacked to address a different audience, but the methods described also provide security in various ways and protect a site from some lame botnets, scrapers and data mining operations.

Keep in mind it's not about messing with your competitors mind, it's about securing your intellectual property away from prying eyes and those that would use it for their own purposes which are often detrimental to the health of your site in the search engines.

Competitor or scraper, once it's indexed and competing with you the end results are the same.