Yahoo has introduced a new tag (technically, an “attribute”) that will allow you to tell their spider, “Hey! Don’t bother with this part of the page!” Yes, that’s correct — this part of the page. It’s kinda like using a robots.txt file to skip an entire page, but now you can tell the spider to skip parts of a page. What it does is help you tell the spider exactly where the “meat” of the page content is.
It should look like this in your code:
Yahoo explains the new attribute on their blog and shares a few examples of how it might be used:
- to skip a page’s navigational menu
- to skip a site header and/or footer
- to skip legal disclaimers and other “forced” content
- to skip advertising blocks
You can argue whether or not this is really needed. Search engine algorithms are quite sophisticated and can use block-level analysis to determine what a header, footer, menu bar, etc. is on most pages. But perhaps not all.
Off the top of my head, this seems like it would be most useful on:
1.) Deep content pages
2.) Pages with a high code-to-text ratio
3.) Site map pages
But … (there’s always a “but”)
I would like some more clarification on the exact impact of using this attribute. For example, if I wrap my main site menu bar with it, does that mean those links to my other pages aren’t even going to be crawled? I believe the answer is “no,” but it would be nice to see that in writing. What about the anchor text in those links? We need more information about exactly what happens to the code and content inside the new attribute.
And one other impact for the link-buying crowd: Just as you’ve had to check the source code to make sure links aren’t being no-followed, now you also have to check the source code to make sure the links aren’t being robots-nocontented.
Update: Melissa at Yahoo PR tells me that Priyank G. updated the original blog post with some more information that addresses the questions I (and others) asked about links:
“…the ‘robots-nocontent’ does not in any way affect how links are treated. All links will continue to be used to find targets and will carry attribution to the target if they do not have the ‘rel=nofollow’ tag on them, whether or not they are inside a ‘robots-nocontent’ section.”>
So, there ya have it.