Google On How To Block Your Staging Server

Aug 15, 2017 - 8:03 am 5 by

Google Stagging Crawl

Google's John Mueller posted on Google+ an overview of how to remove your staging server from Google and then the best way to block it going forward. Some call it a staging server, some a development server, some a testing server or some have all of these three and more. Heck, we even have pentest servers here, so we need to block them carefully from search engines and open them to those that need access.

First John explained that if these servers do find their content indexed by Google, the fastest way to remove the content is via Google Search Console. Of course that means you need to verify the staging URL with Google Search Console, but that is only long enough until you do the remove content method. After that, you then want to close the site from the outside world.

John wrote:

The fastest way to get the staging site removed from search is remove it via Search Console. For that, you need to verify ownership via Search Console [1] (ironically, this means you'll likely have to make it accessible to search engines again, or figure out DNS verification, which isn't that common but also not that hard). From there, you can do a site-removal request [2], which will take the whole hostname out of Google's search for ca 90 days. During this time, you can figure out and implement your general plan to block the staging site from search.

Then going forward, you want to lock down access to the server. John shared these tips, i.e. blocking on the server side with HTTP authentication or IP address whitelisting. He said all the other alternatives he does not like. John wrote:

My recommendation for staging sites is to block access on the server side, either with server-side / HTTP authentication [3] or IP address whitelisting (IP addresses can change, and this would block you from using tools from home, etc, so it's worth being cautious there and whitelisting rather than blacklisting).

He shared more details on Google+ adding that it's normal for URLs to not be recrawled in months, so if you add any block on the URL level, it can easily take a half year or longer to be fully processed for all URLs." "The site-removal request gives you most of that time, and you can submit another one should you need to extend it," he said.

Forum discussion at Google+.

 

Popular Categories

The Pulse of the search community

Follow

Search Video Recaps

 
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: March 27, 2024

Mar 27, 2024 - 4:00 pm
Google Ads

Google Local Service Ads Sends Email Asking You To Upload Photos

Mar 27, 2024 - 7:51 am
Google Maps

European Hotels Can Remove Pricing Information From Google Hotel Search

Mar 27, 2024 - 7:41 am
Google Search Engine Optimization

SEOs, Please Don't Remove Contact Us & About Us Pages

Mar 27, 2024 - 7:31 am
Misc Google

Google Search Developer Docs Gain AI Generated Help Features

Mar 27, 2024 - 7:21 am
Google Maps

Google Local Reviews Reactions Notice

Mar 27, 2024 - 7:11 am
Previous Story: Google Search Console Structured Data Report Gets More Details