I wrote an SEO Tutorial some years ago (before using WP), the older search engine optimization tutorial is still very relevant, but like the rest of the world I moved on and now use WordPress as an SEO CMS. Hence this free WordPress SEO guide. WordPress SEO WordPress out the box (vanilla WordPress with no WordPress SEO plugins or WordPress SEO theme) is pretty good from an SEO perspective, WordPress is a really good base SEO CMS for building optimized websites. However, it is far from search engine optimized and it’s very easy to make on-page SEO mistakes within WordPress especially when using popular WordPress SEO plugins like Yoast WordPress SEO and All In One SEO Pack. All the popular […]
Continue Reading WordPress SEO Guide
Google Panda Update Noindex and Nofollow Meta Tag on Categories
Hi Dave
One of the key things in the Google Panda update was the issues with too many site’s pages that do not have SEO value (such as category/tag pages – pages 2 to ~) and lower the overall site ranking.
Is there a php tricky way to add a NOINDEX, FOLLOW meta tag to all the tags and category pages except the first ones – i.e. except page no. 1.
Also, to prevent indexing the [domain]/page2, [doamin]/page3 and so on.
Please read this – http://www.labnol.org/internet/google-traffic-after-panda/18914/
Google Panda Update Noindex and Nofollow Meta Tag on Categories
Google Panda Update and Noindex, Nofollow SEO Advise
Your first assumption is what you’ve read online in forums etc… about the Google Panda update is verified/true. Without that verification that takes time to test it’s guess work.
Google changes it’s search engine algorithm all the time, sometimes it’s a small change and other times it’s big. Either way what worked that was SEO whitehat a few years ago generally still works for the most part today, Google does not throw the baby out with the bath water, it tweaks the search algorithm over time for better results for it’s users.
The domains the author of the article you linked to are pretty much duplicate content sites (they are content farms). Google is renowned for hating thin content sites like these, and though those particular sites do add value (tiny amount) to the duplicate content it still is duplicate content. Why should a copied version of an article rank higher than the original?
Since Google is always trying to improve it’s results I can see a very strong argument for downgrading that type of site (have you seen http://googleblog.blogspot.com/2011/02/finding-more-high-quality-sites-in.html ) and just a small downgrade for finding duplicate content can have a major impact on SERPs (dropping from top 5 to top 10 is a big deal). You also find with the bigger Google updates if a change has resulted in a lot of good sites loosing their rankings the algorithm is presumably re-tweaked so the negative impacts aren’t as severe (sites go down in rankings with the first change and back up with the re-tweak). Occasionally Google is going to go too severe in an algo change and have to dial it back a bit.
The reason for those sites being downgraded is because they are content farms and that was what the Panda update was targeting. Do you run content farms? I don’t, so shouldn’t be affected negatively by Panda (which my network wasn’t).
Some of the SEO advice within the article is bad SEO advice.
Nofollow deletes link benefit.
Noindex wastes link benefit.
Never use nofollow or noindex unless you absolutely have to. The general rule should be if you have a page you don’t want indexing it really shouldn’t exist or at least avoid linking to it. Monthly archive pages for example, what do they add to a website?
Fortunately for the three domains he analyses they are not short of backlinks, so though it will have a negative impact it’s to some degree mitigated by the number of backlinks.
Deleting tags/categories/archives that are not useful is damn good SEO advice. You should aim for a site that every page on the site is targeting one or more SERPs so the link benefit flowing to those pages are doing something useful. In practice this isn’t always practical, but if you keep it in mind you’ll do far better in Google than if you create pages for the sake of it. If you follow this advice there should be no need to nofollow/noindex categories/tags etc… Never nofollow links to pages on your own sites, that’s dumb because it deletes the link benefit, I’d rather have the link benefit flow through a monthly archive page than delete the link benefit completely.
Unfortunately there’s no easy way to have multiple archive pages (page 2, 3, 4… of a category/tag) and not pass link benefit through them (Update: Stallion Responsive deals with this without using noindex or nofollow). If you nofollow them the link benefit is deleted (that’s nuts) and if you noindex them the link benefit though not deleted isn’t working on the pages that are noindexed (the link benefit is wasted). Why noindex a page that potentially could rank for something? Google doesn’t penalize sites for having good navigation and without good navigation how will Google naturally find your deeper content? However it doesn’t reward sites for have multiple archives that add nothing to a site, I never use monthly archives and generally if there’s an argument for having a tag there’s a better argument for having a category (my sites tend to only have category archives : there’s no real difference between a tag and a category).
Fixing broken links and redirecting pages that are no longer present but have backlinks to them, very good SEO advice, it stops the link benefit from being wasted.
Blocking the indexing of WordPress login pages not really necessary with the current blocking options available in all WordPress themes except mine. The choice is do you leave the link as is and it’s wasted link benefit or do you nofollow/noindex the login page and the link benefit is wasted? Not a lot to gain SEO wise either way for most WordPress users, though some login pages do link back to the home page so not all the link benefit is wasted: note for my theme users I’ve hidden the login/admin links from search engines, so not an issue for those users.
Although I don’t recommend using nofollow/noindex, I do have a way to block the wasting of SEO benefit through archive pages, but need to find the time to make it into an SEO plugin or part of Stallion (Update: part of Stallion Responsive). My plans are along the lines of noindex, but the link benefit isn’t wasted on the archives that are not indexed.
Regarding the three domains the author of the article analysed can you think of any reason why deleting link benefit (nofollow) and wasting link benefit (noindex) would result in better Google rankings?
BTW Traffic to my network of sites went up after the Panda update, so the SEO techniques I use and recommend appear to work better after the Panda update. I have never used noindex and have been removing nofollow links from my sites, had loads of nofollow links when Google didn’t delete link benefit, takes time to remove them from around 100 domains. One domain had a fair number of nofollow links, removed them all recently and traffic is up a bit since.
David
Google Panda Update and Noindex, Nofollow SEO Advise
Google Panda Update and Noindex, Nofollow
Thank you Dave
I must read your explanation once more to fully understand your apporoach.
I agree with what you’ve added that “The general rule should be if you have a page you don’t want indexing it really shouldn’t exist or at least avoid linking to it.”
BTW – the author of the site I refer to made a meta tag of NOINDEX, FOLLOW not a NOINDEX, NOFOLLOW to pages/2 to ~.
I think (am not sure yet) is that the best bet could be to check the WordPress admin option of Settings/Reading settings of ‘Blog pages show at most’ and opt for ‘high number’ (some hundreds posts) in order that all posts filed in a category will be listed on the same category page thus ensure indexing a single cat/tag page rather than several (with the same title and meta description).
Of course, archives pages and author pages are not useful pages and I recall that Talian theme eliminates these to be created in the first place.
Google Panda Update and Noindex, Nofollow
Noindex Always Wastes Link Benefit
Doesn’t matter how you use noindex it wastes link benefit.
Here’s an extreme for you, you noindex but follow links from your home page. All those hard earned backlinks do ZERO SEO work on your home page, does that sound like a good SEO tip?
Is there much difference between your home page and a category page, do you not want all your link benefit as it flows through your site (through links) to work on every page it passes through?
I have plenty of SERPs from category pages, I’d never noindex them.
Each page link benefit passes through uses link benefit: if you are familiar with the PageRank formula it’s the dampening factor and it’s estimated 15% of the link benefit is ‘used’ on each page the link benefit passes through. If you noindex a page 15% of the link benefit passing though that page does no SEO work (that’s bad SEO wise).
From a link benefit perspective it makes no sense at all to ever use noindex, there’s no SEO benefit from a link benefit perspective to use noindex since it wastes a sites most valuable SEO resource.
There are arguments for not wanting some pages of a site indexed, the best way to achieve this is not create pages you don’t want indexed. Second best solution is don’t link to them as there’s no easy way to stop link benefit passing through a standard link without deleting (nofollow deletes it) or wasting it (noindex wastes it).
I’m working on a WordPress SEO plugin that will do what noindex does, but does not waste link benefit. The SEO plugin stops archive pages from being indexed (like noindex), but rather than waste the link benefit it redirects it back to the home page.
So far with the SEO plugin all archive types can be independently not indexed in their entirety (so all date archives, all categories, all tags, all search etc…) and working on the ability to have only the first page of an archive set (first page of category for example) indexed with the paged archives redirected back to the first page (currently have it working with it redirecting the link benefit back to home, want to redirect back to the first page of an archive set).
Also almost got it working with individual static pages and blog posts, you select a list of page/post IDs you don’t want indexing (got a problem to solve in the code to get it fully working).
In my tests (not with this exact plugin setup, but same concept: built into a part of Stallion) it works well. Surprised no one has thought of an SEO plugin like this before as noindex and nofollow are anti-SEO and should be avoided.
Currently plan to give the SEO plugin away for free to support Stallion sales (it will work with most WordPress themes).
David
Noindex Always Wastes Link Benefit
Robots Meta Plugin SEO Value?
Thank you,
I fully understand why noindex (or nofollow) is a waste/delete of link juice.
The labnol.org guy was so convinced that I believed he was right..
Have you used Robots Meta Plugin? https://wordpress.org/plugins/robots-meta/ I think it provides some of the features you’ve mentioned.
I’ve decided to file (present) all posts related to a specific category in a single page (in the specific category page).
It just make sense to me that readers will see all the ingredients at once (scrolling down the page) rather than sitting and browsing pages after pages.
Robots Meta Plugin SEO Value?
WordPress SEO Tutorial : Robots Meta WordPress Plugin Review
I’m familiar with the Robots Meta WordPress Plugin, it uses noindex and nofollow, so not a WordPress SEO plugin I’d recommend.
I’m close to finishing a true WordPress SEO Plugin, you can read about it at Stallion WordPress SEO Plugin. I believe it does everything you mentioned in your first comment.
With a bit of luck will have it released in a week or two.
BTW you might be mixing up rel=”nofollow” and nofollow added to a robots meta tag. rel=”nofollow” is basically what I’ve been talking about (mostly) when talking about nofollow, you can add this attribute to individual text links and the link benefit that would pass through the link is deleted.
A nofollow added to a robots meta tag results in all links on that page not being followed, this means the link benefit hits a dead end and is wasted (like a page with no links off it at all, basically deleted). A nofollow robots meta tag is worse than a rel=”nofollow” attribute.
When talking about noindex I’d been referring to a “noindex, follow” robots meta tag combination. The noindex part wastes the link benefit (as described earlier) that should be used on the noindexed page, but the remaining link benefit (it’s believed 85% of link benefit passes through a page via it’s links) continues on it’s way, so link benefit is wasted not deleted, you could argue 15% of the link benefit is deleted because it does no SEO work, but because it’s a follow tag the remaining 85% passes through the links.
Clearly a “noindex, nofollow” robots meta tag combination is the worst SEO choice, all the link benefit that should pass through the noindexed page is deleted.
David
WordPress SEO Tutorial : Robots Meta WordPress Plugin Review
WordPress SEO Tutorial robot.txt
If not Meta Robot plugin that generates robot.txt what robot.txt file your would recommend using, if at all (meanwhile)?
There is an example robots.txt file available here: http://www.askapache.com/seo/updated-robotstxt-wordpress.html
And I got that link from SEO wordpress codex page: http://codex.wordpress.org/Search_Engine_Optimization_for_WordPress
In addition, I am still a Talian 03 user and there’s an issue that the theme files can’t be edited thru the dashboard.
Have you managed to locate the problem and fix it?
Thanks
WordPress SEO Tutorial robot.txt
WordPress Search Engine Optimization of robots.txt File
When using a robots.txt file the first question to ask is what are you trying to achieve?
Lets go through the example robots.txt file you found:
User-agent: * means all bots should obey this set of rules, doesn’t mean they will (all my robots.txt files include “Crawl-Delay: 20” but Google ignores it).
Anything starting disallow means you don’t want it spidered. Are you having specific problems with some sections of a site being indexed you don’t want indexed (not much point using “Disallow: /cgi-bin” if you don’t even have a cgi-bin directory for example)?
If not there’s no need to have anything here. I will add never use a robots.txt file to hide files you don’t want visitors to see, it’s real easy to load https://stallion-theme.co.uk/robots.txt and if I were dumb enough to add files that I wanted to keep secret, not much of a secret :-)
As a side note most of my robots.txt files are identical to the one for this site, just the crawl delay (supposed to slow down spidering : my sites are constantly hammered by bots) and I only have a robots.txt file to stop a 404 error code when bots look for a robots.txt file.
If you were trying to stop WordPress tags from being indexed adding “Disallow: /tag” would stop them from being spidered, BUT it would waste a lot of link benefit and mean your tags won’t be indexed (is that what you want?). The new WordPress SEO plugin I mentioned above that I was working on has been released at Stallion WordPress SEO Plugin and it can achieve the equivalent of the WordPress relevant disallows above without wasting link benefit.
Specifying a specific user-agent and allowing it is a complete waste of time. By default they are allowed to spider your site and adding the above won’t increase or decrease the number of visits a user agent will make.
Specifying a user-agent and disallowing it can be useful. If you are having a problem with a spider you can set this to stop it spidering completely, but be aware if you are having a problem with a bot it probably isn’t a good one one that will follow the rule!
Useful if you have a sitemap file.
As you can see the robots.txt file is basically useful for stopping spiders doing things you don’t want them to do, but they don’t all follow the rules and when you do disallow a section of a WordPress site it can come at the cost of link benefit.
BTW some of the information at http://codex.wordpress.org/Search_Engine_Optimization_for_WordPress is wrong.
Search Engine Site Submissions complete BS, total waste of time submitting a site to a search engine like Google, only way to get a site indexed long term and increase rankings is through backlinks that are not rel=”nofollow” or on a noindex page.
Meta Tags have zero ranking value.
Robots.txt Optimization, the example robots.txt file is awful, that could seriously damage a sites SEO, would damage my sites for certain!!!
Talian 3 was never broken, that’s a permissions issue you have. If you want to edit your files online they have to have the correct permissions, depending on your server you’ll want the files set to 666 for full access, but I would strongly advise setting them back to 644 for added security after editing. I find it more secure to edit the files offline and upload using FTP (tendency to forget to change the permissions back).
As a Talian 3 customer you’ll be entitled to both a Talian 5 upgrade and Stallion 6 upgrade. Drop me an email from the email address you used to order and I’ll give you the upgrade details.
David
WordPress Search Engine Optimization of robots.txt File
Allow WP-Includes in Robots.txt? (Google Search Console Warning)
Hi Dave, I know this is kind of an old comment, but just wanted to just confirm that you are saying it is okay to not disallow wp-includes in robots.txt?
I know a lot of people have had issues with getting warnings from Google Search Console (formerly Webmaster Tools) about blocked resources, and it seems that blocking wp-includes is the culprit in some cases.
I know that some suggest blocking wp-includes, but allowing certain sections of it to be crawled.
But I don’t really understand why some think it is necessary to disallow this file, perhaps it is just an old practice that is now outdated?
Any thoughts you might have are appreciated.
Allow WP-Includes in Robots.txt? (Google Search Console Warning)
Caching Plugins SEO Value - WP Super Cache or W3 Total Cache?
Hello David
What is your view regarding caching plugins such as – WP Super cache or W3 total cache?
Do we really need them to speed up the site?
Do you think these two plugin are necessary?
Thanks
WordPress SEO Image Alt Text Optimization
I have a website. But the visitors are poor.
Someone advised to SEO optimize my image and use alt tags to get more traffic.
But how shall i do that.
Can you give me a guide line or web link from where i can be benefited.