How to disable search indexing on custom domain

I’m publishing a wedding website that I’d like to remain off of search engines, for obvious reasons. I’ve checked “Disable Webflow subdomain indexing” and added the appropriate robots.txt file, but according to other posts in this forum, that only disables indexing of the thissite.webflow.io site, not the custom domain.

How can I disable indexing for the custom domain as well?


Here is my site Read-Only: LINK
(how to share your site Read-Only link)

Once you’ve published the site you can create a custom robots.txt file to exclude bots.

You can also visit the static page settings for each static page, and configure it to exclude from the sitemap. This will also add a META noindex tag which is great.

CMS pages don’t have that feature, so if you’re using the CMS you can add that META manually, or lean on the robots exclusion approach if you’ve done it before search engines discover you.

As stated in my OP, I’ve already created a custom robots.txt but I’m told this will only work on the webflow.io staging site, not the custom domain.

I think you’re a bit confused. They are separate features.

The “disable Webflow subdomain indexing” creates the robots.txt exclusion automatically for you on your webflow.io site. only.

Making a custom robots.txt file is a different thing, and works on your public site.

@djv5010 I agree with @memetican, robots.txt will be enough to keep crawlers out of your website:

User-agent: *
Disallow: /

if your website can be found on site:https://yoursite.com on Google. Then, I will manually remove it by following Google recommendations

Congratulations for your wedding :slight_smile:

Sofian

Oh interesting! Well then it sounds like I’m all set.

The only other thread on this forum about this exact topic, the OP was told that the custom robots.txt only applied to the webflow.io not the custom domain. I suppose they were just wrong.