How To Exclude Googlebot From Indexing Your Website

Simply put, Googlebot is the automated web crawler that Google uses to scan websites and blogs. When websites update or add posts to their sites, those posts need to be indexed in search engines so that people can find them. This is where Googlebot comes in. If your website or blog has been submitted to Google, then Googlebot will occasionally scan your site. There are different options when controlling Googlebot's scanning. For example, under Google Webmaster Tools, you can choose to have the bot search your website more often or less frequently based upon your own needs and how often you update your site.

However, some people may have either a more private blog or a website that they do not wish to be included into Google search results. If you want to exclude Googlebot from indexing your website or blog, the process is very simple.

The simplest way to exclude Googlebot from indexing your website or blog is to make use of Google Webmaster Tools, which you can find here. You will need to have a Google account to access Webmaster Tools.

After you have Webmaster Tools, you will be able to remove your site from Google and prevent indexing from Googlebot. The first step in this process is to click the + sign for Site configuration. It is located along the left navigation pane within Webmaster Tools, directly under the "Dashboard" link. After you have done this, you will see a link appear that says "Crawler Access". Click it.

You should now be in the Crawler access menu. At the very top of the page you will see a removal requirements link. Click to read it. After you have done that, you will see 3 link tabs toward the top of the page: Test Robots.txt, Generate Robots.txt, and Remove URL. Select the tab that says Remove URL.

After you have done this it will bring you to a page with 4 removal options: individual URLs, site directories and sub directories, entire site, and cached pages. Select the one that meets your requirements. More then likely, if you do not want Google to index your page at all, then select "entire site". Next, it will bring you to a page where you must agree and put a checkmark in a box stating that you want to remove your site.

Click submit removal request.

That's all there is to it. Googlebot should now avoid any indexing of your website or blog.