Pages blocked in robots.txt showing in google index

Publishing help General
1

Hi

My robots file looks like this:
http://leicestermortgagebroker.co.uk/robots.txt

but when I run site://leicestermortgagebroker.co.uk in google, the contact and privacy pages are showing in google’s index, which I don’t want.

Any idea why this is? Is the only way around it to remove these urls in google webmaster tools?

Webflow Support, any idea if/when the ability to noindex individual pages will be available as a feature? It would be really useful for seo purposes.

Thanks a lot

Graham

2

robots.txt Disallow does not guarantee that a page will not appear in results: Google may still decide, based on external information such as incoming links, that it is relevant. If you wish to explicitly block a page from being indexed, you should instead use the noindex robots meta tag or X-Robots-Tag HTTP header.

3

Hi Vincent

Thanks very much for your reply.

How do I use the noindex robots meta tag in webflow on an individual page please. I thought this wasn’t possible in webflow yet?

Thanks

Graham

4

I am not sure you can for individual pages as the custom header code (where Meta goes), impact the whole site. Maybe ask Webflow support about that.

5

Was there any resolution to this question? I have the same need.

6

I too, have the same question.

7

Seems like you should be able to throw <META NAME="ROBOTS" CONTENT="NOINDEX"> into the Head tag section with no problem … as noted in the UI “code included here will only apply to this page”

1 Like
8

I really don’t know why I was under the impression that was not the case… must have been raelly tired :—) Thanks for clearing things up.

1 Like
9

No problem! Easy enough to do, especially when you’re tired : )