Welcome to the Treehouse Community

Want to collaborate on code errors? Have bugs you need feedback on? Looking for an extra set of eyes on your latest project? Get support with fellow developers, designers, and programmers of all backgrounds and skill levels here with the Treehouse Community! While you're at it, check out some resources Treehouse students have shared here.

Looking to learn something new?

Treehouse offers a seven day free trial for new students. Get access to thousands of hours of content and join thousands of Treehouse students and alumni in the community today.

Start your free trial

Business SEO Basics Better SEO Through Code Noindex and Nofollow

Lucas Krause
Lucas Krause
19,924 Points
Posted by Lucas Krause
Lucas Krause
19,924 Points

Using <meta name="robots" content="noindex"> to prevent access

I'm not quite sure whether I understand Dan correctly when he says

So, don't think that robots.txt file is where you
can hide content that you don't want getting seen.
Therefore, to prevent users from accessing pages that they should not
access, use that no index meta tag on the page instead.

  1. As far as I know directories as well as their subdirectories and files inside of these directories won't get indexed by search engines. It's exactly the same as using the appropriate meta tag in a HTML file. Therefore using such meta tag inside of a file located in a directory disallowed by index.txt would be redundant.
  2. Am I right that he is wrong when he says that you can prevent people from accessing a specific file using the robots meta tag inside of that file? Or did I misunderstand him entirely?

1 Answer

Philip G
Philip G
14,600 Points

Hi Lucas,

  1. If you disallowed the page in your robots.txt(not index.txt), you're right.

  2. With the <meta> Tag you can't prevent people seeing your page. If you want to do so, the easiest way is to set this in your .htaccess.

Lucas Krause
Lucas Krause
19,924 Points
Lucas Krause
Lucas Krause
19,924 Points

Thanks for your response. Of course I meant robots.txt, not index.txt. :D You confirmed me in each point. I think this should be fixed since it misleads students. I'm going to forward this to the support.

Philip G
Philip G
14,600 Points

Yeah, you should do that