Meta robots tag is the most crucial and slightly complex topic in Seo. But with little bit concentration, you can understand their usage and can see implementation in search engines. Do you know? What it does and why we use it? Probably you don’t know. So let’s start to read about meta robots tag.
What is Meta Robots Tag:
Meta robots tag is used to control website pages in search engines. Through meta robots tag, you can index or not index your particular pages, it’s up to you that what you want to do. Meta robots tag write in website <head>…</head> section. Review the below meta robots tag code, which you will utilize behind your web pages.
<meta name="robots" content="noindex"/>
Above tag use for all search engines and NoIndex giving an instruction that do not crawl or index pages in all search engines. If we replace Noindex to Index then see, what will happen.
<meta name="robots" content="index"/>
Above tag allow all search engines to index this particular page. Trivial change can create huge difference.
Here many numerous directives, which you can use for multiple purposes. Check it out below list of directives and their usage guide.
| Directives | Usage Guide |
| Index | Use to allow search engine crawler to index this pages. |
| Noindex | Use to disallow search engine do not crawl this page. |
| DoFollow | Use to instruct crawler to index all links on that page. |
| Nofollow | Use to restrict crawler do not crawl any link on that page. |
| Noarchive | A instruction for search engine that do not show cache link in search results. |
| NoSnippet | With this directive, search engine do not show this page snippet in search results. |
| NoImageIndex | This directive stop search engine crawler to index this page images. |
| NoTranslate | This directive will stop to translate this page in search engine. |
| NoODP | This directive stop DMOZ directory to use their own meta data like title and description for your website. |
| NOYDIR | This directive stop Yahoo directory to use their own meta data like title and description for your website. |
Above directives playing major and prominent role in search engine. If you want to use multiple directives in a single meta robot tag then review the below code.
<meta name="robots" content="noindex,follow"/>
Above tag contains two directives but working in single tag. Similar above tag will work for every search engine but let’s think about that, if we need to give a particular command/instruction only for Google search engine crawler then see the below tag.
<meta name="googlebot" content="noindex"/>
Above meta robots tag work only for Google search bot not for else crawlers.
Let us know, if you find this article useful and got your answer. We suggest you to read What is nofollow and dofollow and what are What are the Noodp and Noydir Meta tags. For more discussion and your opinions, leave your comment below.
Siraj bro I read your blog everyday and get some new idea everytime. Thanks for posting this valuable article. Knowledgeable for SEO experts.
Thanks again
“Trivial change can create huge difference” as you say if we not use meta robots tags then does google require for this. If we leave this then google crawl pages automatically?
Yes, if you do not use these meta-tags especially “noindex” then Google will crawl your pages automatically whether you submit your sitemap via webmaster or not. But depends on various strategies. 🙂