Search engine spiders will crawl your whole website to cache your website’s pages for their index. In general, most website owners are happy for search engines to crawl and index any page they want; however there are situations where you would not want pages to be indexed.
For example, if you are developing a new website, it is usually best if you block search engines from indexing your website so that your incomplete website is not listed on search engines. This can be done easily through the reading settings page at http://www.yourwebsite.com/wp-admin/options-reading.php.
Hide a WordPress Page from Search Engines Using Yoast SEO:
This method is easier and recommended for beginners.
First thing you need to do is install and activate the Yoast SEO plugin. For more details, see our step by step guide on how to install a WordPress plugin.
If you are new to Yoast SEO, then you may want to see our guide on how to install and set up Yoast SEO plugin.
Next, you need to edit the post or page that you want to hide from search engines. Scroll down to the Yoast SEO meta box below the post editor and click on the advanced settings button.
The advanced settings section allows you to add meta tag robots to your blog posts or pages. Using the robots meta tag, you can tell search engines not to index or follow a page.
First you need to select ‘noindex’ from the drop down menu next to ‘Meta robots index’ option. After that, click on ‘nofollow’ next to ‘Meta robots follow’ option.
You can now save/publish your post or page.
Yoast SEO will now add this line of code to your post or page:
This line simply tells search engines not to follow or index this page.