Option to prevent search engines from indexing site

Community resources Product feedback
1

Hi,

Is there a way to prevent the search engines indexing web pages?

Thanks

2

Does this help?

3

maybe this is what you are looking for.

research / google robots.txt - disallow

https://www.google.com/search?q=robots.txt+disallow&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a&channel=sb

or click here: robots.txt disallow - Google Search

4

LOL @revolution that’s exactly what my link was for. Just with actual instructions from the webflow team :stuck_out_tongue:

5

oops… I didn’t see your response.
Yours is better anyways. Instructions are always good :smile:

6

haha no worries :stuck_out_tongue:

7

This would be a good feature to have built in Webflow @callmevlad.
How about a checkbox on the page settings? “[_] Hide from search engines”

8

yes - this would be a good feature to have.

If they add this features… here’s another one that would added in the same place.

WebFlow currently allows system-wide Keywords / Keyphrases.

But; basically OP wants to add custom meta tags (not just keywords) per page.

I do this all the time using php… but from what I see - it cannot currently be done in WF.

9

Hi all,

Exactly what i was looking for and agreed, it would be a good feature.

Thanks again

10

Hey guys, adding the following meta tag to the Head section in the Custom Code tab from your Site Settings could prevent search engines from indexing your site.

Of course, this is not a perfect solution and no one (and I mean NO ONE) tells Google what to do :slight_smile: hehe

<meta name="robots" content="noindex">

Hope this helps!

11

I’m pasting here my thoughts as shared with Webflow support over a few emails…

Something just hit me about published sites. The following site is set to private. I have published it on numerous occasions to test it across various devices. But I just noticed that if I’m not logged into Google Chrome, that I still have access to the site. If I’m not logged in, then I shouldn’t be able to see the site, otherwise it’s not private. Can you clarify this?

But is a Published site indexable by Google? I would sincerely hope not with regards the negative SEO implication on duplicate content, amongst other things.

You ought to include that somewhere within the control panel. I’m off to Unpublish the sites, if that’s even possible.

There really should be a way to not index a site because by publishing we allow our sites, which are in development, to be accessed by clients for review without giving them control panel access. No way do I want Google or any other search engine indexing it. This just seems like a major, major oversight.

Please reconsider how this is handled.

Also, if someone buys into a Pro account, they’re most likely doing it for the unbranded aspects. I mean, that’s a major feature of going with Pro. So by Google indexing, you’ve circumvented the “white-label” aspect of Webflow which kind of sucks.