Here’s the official response I got from Webflow support regarding partial site hosting via reverse proxy. Note that Webflow now allows for href prefixing in the custom code advanced options.
The way to setup the connection in Webflow is to create a subdomain using a cname record to point to proxy-ssl.webflow.com.
For example, if you create a subdomain of the root domain called “connect” then point “connect.yourdomain.com” at proxy-ssl.webflow.com and add that domain to the hosting tab of project settings in Webflow.
Next, in your reverse proxy, the url that you will use to pull the pages from Webflow will be: https://connect.yourdomain.com.
Here is an article on setting the href prefix for the Webflow site: Href prefix - Webflow University Documentation