SEO Disaster: The Cost of Enabling ‘Discourage Search Engines’ in WordPress

Inside the WordPress settings, one option stands out due to its significant impact on search engine optimization (SEO) – the ‘Discourage search engines from indexing this site’ setting. When enabled, this function essentially tells search engines to ignore the website, leading to drastic SEO consequences.

The Purpose and Proper Use of this Function

The ‘Discourage Search Engines’ function is a WordPress feature that essentially tells search engines to ignore your website. In other words, it’s like hanging a “do not disturb” sign for search engines.

While this might initially seem counterproductive, especially when most businesses strive for visibility in search engine results, there are scenarios where this feature can be highly beneficial.

Website Development and Testing

During the website development and testing phase, developers often use dummy content or incomplete pages. These are not meant for the public eye, and certainly not something you’d want search engines to index. In this case, enabling ‘Discourage search engines from indexing this site’ is an effective way to prevent premature indexing.

It’s crucial to remember that when your site is ready for public viewing, you should immediately deactivate this setting. This way, search engines can find your site and display it in their search results.

Private Blogs or Websites

If your WordPress site is intended for a select group of users, like a personal blog or an internal company website, you may not want it to show up in public search results. This function can keep your content exclusive to your intended audience.

Sensitive Information

Sites handling sensitive or confidential information that should not be publicly available can use this setting to discourage search engines from indexing the site.

SEO Consequences of Enabling this Setting

When ‘Discourage Search Engines’ is enabled, the site’s SEO health starts to plummet. The website becomes invisible to search engines, leading to zero visibility on search engine result pages (SERPs), resulting in no organic traffic.

Example: the website of Hotel am Stern, Brühl, Germany.
The robots meta tag has the noindex, nofollow value.
Example: the website of Hotel am Stern, Brühl, Germany.
Because of its robots meta tag, it has zero indexed pages in Google.
Example: the website of Hotel am Stern, Brühl, Germany.
According to Ahrefs, the website has been getting zero organic traffic since April 2021.

It’s concerning to think about how many WordPress users may have unknowingly enabled this setting, perhaps even for years, without understanding its severe impact on their site’s visibility in search engine results.

It’s equally worrying to think about how many have struggled with low site traffic, not realizing that a simple checkbox in their settings is the culprit.

Industry Breakdown: The Unexpected Reach of NOINDEX

Various industries, surprisingly, have a notable proportion of websites that have activated the ‘Discourage Search Engines’ function. This lack of visibility due to the NOINDEX tag is not confined to a single sector but rather a widespread problem.

Lodging Websites

Hospitality is an industry that heavily relies on online visibility. Potential guests often use search engines to research and book accommodations.

My analysis shows that out of 205,000 WordPress-powered lodging websites, 3.3% are NOINDEXED. This amounts to nearly 6,765 accommodation providers who may be inadvertently turning away potential guests because their websites are invisible in search engine results.

In Switzerland, 5.3% of lodging sites have this issue, followed by Norway with 4.3%, Spain with 4.16%, Germany with 3.85%, and the USA with 3.66%.

These statistics show the global reach of this problem and emphasize the need for regular SEO audits and proper WordPress setting checks to ensure website visibility in search engine results.

Podcast Websites

Podcasters may be unknowingly limiting their audience growth due to a simple SEO oversight. Out of 21,000 WordPress-powered podcast websites I examined, approximately 4% are NOINDEXED.

That’s about 840 podcast websites missing out on potential listeners who might be searching for their content through search engines.

City Hall Websites

City hall websites are often the go-to resource for residents seeking information on local services, policies, and news.

I discovered that out of 12,000 WordPress-powered city hall websites, 4% are currently NOINDEXED.

Business Owners are Confused by the Google Knowledge Panel

The Google Knowledge Panel is like a mini-website that shows up in Google search results. It provides quick information about a business, such as opening hours, website links, and customer reviews.

This panel can make a business more visible online, and it’s often seen as a sign of good performance.

A branded search for Hotel am Stern displays the Knowledge Panel, thus creating the impression that the website is doing well.

However, it’s important to understand that having a Google Knowledge Panel doesn’t necessarily mean your actual website is doing well in terms of being found in search results. This is because the panel is tied more to your Google business account, not the health of your website’s visibility to search engines.

During the research phase for this article, I reached out to around 500 lodging businesses whose websites were found to have the NOINDEX setting enabled. I hoped to alert them to the potential issues they were unknowingly facing.

Regrettably, the majority did not respond to my outreach. However, the few who did, interestingly, directed me towards their Google Knowledge Panel. They thought everything was okay because they saw their information in this panel.

So, this confirmed for me that the Google Knowledge Panel can inadvertently create a sense of false security for business owners. They may see their business information in this panel and assume their website is equally accessible in search results, which, unfortunately, may not be the case.

So, even if your business has a Google Knowledge Panel, you still need to make sure your actual website is easily found in search results.

How to Recover from this SEO Mistake

Start by turning off the ‘Discourage Search Engines’ option in your WordPress settings.

Then, submit your site to search engines using the ‘URL Inspection’ tool in Google Search Console or the ‘Submit URLs’ feature in Bing Webmaster Tools.

The URL Inspection screen in Google Search Console.
The URL Submission screen in Bing Webmaster Tools.

Fixing your website and getting it back in the good books of search engines like Google and Bing can take some time. How long? It varies, depending on the size of your site, how bad the issue was, and how quickly search engines notice your fixes.

As a rough estimate, it could take a few weeks or even a few months to start seeing improvements. It’s important to stay patient and consistent during this time.

This recovery period provides a perfect opportunity to analyze and revamp your site’s page structure. Ensure that your site is easy to navigate, with a clear hierarchy and an effective internal linking structure. Updating your site structure not only improves user experience but can also boost your SEO performance.

Importance of Regular SEO Audits

Just as a car needs regular maintenance checks to prevent breakdowns, your website needs regular SEO audits to prevent search engine disasters.

An SEO audit is essentially a health check for your website to identify areas that need improvement. These audits examine various aspects of your website, such as loading speed, proper use of keywords, quality of content, ease of navigation, and much more.

For those unfamiliar with SEO practices or those who find this process time-consuming, consider hiring a professional WordPress developer.

A professional WordPress developer understands the technical aspects of SEO and the WordPress platform in-depth. They can effectively conduct SEO audits and promptly identify potential problems. They can also provide you with valuable insights and recommendations to improve your SEO strategy continuously.

More Warnings Needed for ‘Discourage Search Engines’ Setting

While WordPress has made building and managing a website accessible to many, some of its features can lead to significant issues if not used correctly.

For this reason, it’s imperative that the team behind WordPress places greater emphasis on educating users about the potential consequences of this setting.

A few suggestions to improve this situation:

  1. Clearer Warnings:
    The WordPress dashboard could provide clearer warnings about the implications of enabling this setting. For example, a pop-up warning could appear when a user checks the ‘Discourage Search Engines’ box, outlining the potential SEO consequences.
  2. Better Documentation:
    While WordPress does provide documentation on this setting, it could be improved with more detailed explanations and real-world examples of when to use and not to use this setting.
  3. Periodic Checks and Alerts:
    WordPress could implement a feature that periodically checks if the ‘Discourage Search Engines’ setting has been activated and sends an alert to the website owner, reminding them of the setting’s implications.

As the world’s leading content management system, WordPress has a responsibility to ensure its users are well-informed about the potential consequences of its features.

Additional Resources

Links to Advanced SEO Guides and Tools

There are plenty of resources online to help you with your SEO journey.

Recommendations for WordPress SEO Plugins

WordPress plugins can greatly simplify the SEO process by helping with meta tags, XML sitemaps, and overall site optimization. However, always remember that plugins are tools to assist you, not a replacement for sound SEO practices.

About the Author

Dumitru Brinzan is a professional WordPress developer from Germany. He assists small and medium businesses with WordPress website creation, website support, technical SEO, WordPress coaching, and consulting. With his expertise, business owners can focus on what they do best while knowing their site is in good hands.

Be the first to leave a comment

Leave a Reply

Your email address will not be published. Required fields are marked *