Create Webflow project programatically

Hello All!

What I need: I need to programmatically setup Webflow pilots. One example use case would be having new pilots being setup after admins send a form submission to a custom endpoint I have.

I’m about to start looking into this and try to find ways I can make that happen, but I’m wondering if any of you have ever done something along those lines? If any of you have any pointers to save me some research time, they would be greatly appreciated.

@webdev pinging you here just in case you know of something? Thank you!

Thank you all!

EDIT 1: Provisional solution on this reply

Yep, I’m quite familiarized with their API… But the word “impossible” doesn’t really work for me, so I will do my best to find a way. I’m just trying to explore if the community has already done some exploration on this subject.

Even if I need to setup selenium or puppeteer on a backend so I can mock an actual person logging in to Webflow, I just know that there’s always a solution to pretty much everything :joy::joy:

1 Like

I appreciate the input… However, I just did a test run using Puppeteer and had no issues: Webflow + Puppeteer Test - YouTube

I don’t know if the same result would happen on a cloud environment, but if that becomes an issue, I can circumvent that by providing a locally executable app (maybe using Electron for this) so admins can run this on their own machines.

Again, I’m now just wondering if someone in the community has a solution that does not rely on automation, since automation software are pretty heavyweight and annoying to work with, (if compared to regular API calls for example)

Thanks!

The team I work with builds several sites using the same internal template we have in our team. Sometimes we need to setup 3 new projects on the same day using the same base project. And webflow is only a small part in the ecosystem that new projects interact with…we also need to individually setup firebase and google analytics for example. So the whole setup of a single project takes around 15 minutes and it’s currently quite technical.

The reasoning for me to start automating things is to have code take care of the boring part of setting things up for every new project. This will also allow non-technical people to setup new projects without needing to wait a dev. to have bandwidth to set things up. The admin will then be able to get started working on the CMS right away.

Does this make sense to you @flashsites ?

That sounds like a really cool use case.

You should also look at how Webflow supports copy & pasting components. You could use the same internal format to paste your design components into each new project. That would make your puppeteer script much simpler.

1 Like

Thanks for the input @JudoHacker !! I also appreciate the positiveness from your end!!

Our designs don’t really change that much between different projects, it’s mostly content changes between each project.

However, the point you raised is actually a very interesting one for other projects that might need something similar. I recently did some explorations on top of that exact subject (moving components around projects), and you might find this topic a nice read: How does component webflow libraries like flowbase, library.relume.io, incorporate copy button to copy components and paste it to webflow canvas? - Design help / General - Forum | Webflow

1 Like

Ok, to anyone currently following this thread, or to anyone in the future that faces the same need, here’s the initial solution I implemented on my end:

It uses puppeteer to mimic an actual person performing those steps, and it uses your session cookie to authenticate to webflow (Think of it as a self-inflicted session hijacking based on the “wfsession” cookie that gets added to your browser once you login to Webflow)

The code below has some TODOs for myself, but please don’t mind those, since everything major that is related to this forum topic is there. You can then simply adjust the code below to match your needs.

import { runWith } from "firebase-functions";
import * as yup from "yup";
import puppeteer from "puppeteer";

import WebflowAPI from "@services/Webflow";
import PayloadValidator from "@services/validators/Payload";

interface IApiReq {
  siteName: string;
  siteSlug: string;
  folder: string;

  usesFirebase: boolean;
  multiDirectionalFeedbackEnabled: boolean;
  wantsEmailOnSignIn: boolean;
  shouldAlwaysShowLikesCounter: boolean;
  siteLanguage: string;

  wfSessionVal: string;
}

const cssSelectors = {
  dashboardNewSiteBtn: '[data-automation-id="new-site-button"]',
  dashboardSelectTeamTemplateBtn:
    '.templates-category [data-automation-id="select-blank-template-link"]',
  dashboardSiteNameInput: '[data-automation-id="site-name-input"]',
  dashboardCreateSiteBtn: '[data-automation-id="create-site-button"]',

  designerLogoBtn: '[data-automation-id="DesignerLogo-button"]',
  designerLogoSettingsBtn:
    '[data-automation-id="DesignerLogo-settings-button"]',

  settingsFavicon: "img.favicon", // Only element in settings that holds the site id

  settingsSaveBtn: "[data-automation-id='dash-site-save']",
  settingsSlugInput: "[data-automation-id='dash-site-short-name']",
  settingsFolderSelect: "[ng-model='site.dashboardFolder']",

  settingsCodeLinkItem: "[data-automation-id='CustomCode-menu-item']",
  settingsCodeHeadEditor: "form > div:nth-child(4)",
  settingsCodeSave: "form > div:nth-child(3) > button",
};

export default runWith({ timeoutSeconds: 540, memory: "4GB" }).https.onRequest(
  async (req, res) => {
    /**
     * ! START - Payload Validation
     */
    await PayloadValidator.validate(
      yup.object().strict().shape({
        wfSessionVal: yup.string().required(),
        siteName: yup.string().required(),
        siteSlug: yup.string().required(),
        folder: yup.string().required(),
        usesFirebase: yup.boolean().required(),
        multiDirectionalFeedbackEnabled: yup.boolean().required(),
        wantsEmailOnSignIn: yup.boolean().required(),
        shouldAlwaysShowLikesCounter: yup.boolean().required(),
        siteLanguage: yup.string().required(),
      }),
      req.body
    );

    const {
      siteName,
      siteSlug,
      folder,
      usesFirebase,
      multiDirectionalFeedbackEnabled,
      wantsEmailOnSignIn,
      shouldAlwaysShowLikesCounter,
      siteLanguage,
      wfSessionVal,
    } = req.body as IApiReq;

    // TODO: Setup site in Google Analytics

    const {
      dashboardNewSiteBtn,
      dashboardCreateSiteBtn,
      dashboardSelectTeamTemplateBtn,
      dashboardSiteNameInput,
      designerLogoBtn,
      designerLogoSettingsBtn,
      settingsSlugInput,
      settingsFavicon,
      settingsSaveBtn,
      settingsFolderSelect,
      settingsCodeLinkItem,
      settingsCodeHeadEditor,
      settingsCodeSave,
    } = cssSelectors;

    /**
     * ! START - Webflow Authentication
     */
    const browser = await puppeteer.launch({
      // headless: false,
      defaultViewport: { height: 1080, width: 1400 },
    });
    const page = await browser.newPage();

    // Authentication
    await page.setCookie({
      name: "wfsession",
      value: wfSessionVal,
      domain: "webflow.com",
      path: "/",
      expires: Date.now() + 1 * 24 * 60 * 60 * 1000, // 1 day
      httpOnly: true,
      secure: true,
      priority: "Medium",
      sameSite: "None",
    });

    await page.goto("https://webflow.com/dashboard?org=acme-corp");
    await page.waitForNetworkIdle();

    /**
     * ! START - Site Setup
     */
    await page.waitForSelector(dashboardNewSiteBtn);
    await page.click(dashboardNewSiteBtn);
    await page.waitForSelector(dashboardSelectTeamTemplateBtn);
    await page.click(dashboardSelectTeamTemplateBtn);
    await page.type(dashboardSiteNameInput, siteName);
    await page.click(dashboardCreateSiteBtn);

    // Wait for webflow to finish setup of project
    await page.waitForSelector(designerLogoBtn, {
      timeout: 60 * 1000,
    });
    await page.waitForTimeout(10 * 1000); // wait for webflow overlay to be gone so we can click on settings

    // Go to settings
    await page.click(designerLogoBtn);
    await page.waitForSelector(designerLogoSettingsBtn);
    await page.click(designerLogoSettingsBtn);

    // Get Site ID
    await page.waitForNetworkIdle();
    await page.waitForSelector(settingsFavicon);
    const siteId = await page.evaluate(
      `$("${settingsFavicon}").attr("src").split("/")[3]`
    );

    // Set site slug
    await page.waitForSelector(settingsSlugInput);
    await page.evaluate(
      `$("${settingsSlugInput}").val("${siteSlug}").trigger("change")`
    );

    // Save settings
    await page.click(settingsSaveBtn);
    await page.waitForNetworkIdle();

    // Move site to appropriate folder
    await page.evaluate(
      `$("${settingsFolderSelect} [label='${folder}']")[0].selected=true;`
    );
    await page.evaluate(`$("${settingsFolderSelect}").trigger("change");`);

    // Save settings
    await page.click(settingsSaveBtn);
    await page.waitForNetworkIdle();

    // Setup site configuration on custom code
    await page.click(settingsCodeLinkItem);
    await page.waitForNetworkIdle();

    // Get data inside Angular Component Scope
    await page.evaluate(
      `scope = angular.element($("${settingsCodeHeadEditor}")[0]).scope();`
    );

    // Update data in Angular Component Scope
    await page.evaluate(
      `scope.meta.head = scope.meta.head.replace(/usesFirebase =(.*);/,'usesFirebase = ${usesFirebase};')`
    );
    await page.evaluate(
      `scope.meta.head = scope.meta.head.replace(/multiDirectionalFeedbackEnabled =(.*) &&/,'multiDirectionalFeedbackEnabled = ${multiDirectionalFeedbackEnabled} &&')`
    );
    await page.evaluate(
      `scope.meta.head = scope.meta.head.replace(/wantsEmailOnSignIn = (true|false)/,'wantsEmailOnSignIn = ${wantsEmailOnSignIn}')`
    );
    await page.evaluate(
      `scope.meta.head = scope.meta.head.replace(/shouldAlwaysShowLikesCounter =(.*) &&/,'shouldAlwaysShowLikesCounter = ${shouldAlwaysShowLikesCounter} &&')`
    );
    await page.evaluate(
      `scope.meta.head = scope.meta.head.replace(/window.siteLanguage = "(.+)";/,'window.siteLanguage = "${siteLanguage}";')`
    );
    // await page.evaluate(
    //   `scope.meta.head = scope.meta.head.replace(/ga: "(.+)"/,'ga: "${gaId}"')`
    // ); // TODO: Implement
    // await page.evaluate(
    //   `scope.meta.head = scope.meta.head.replace(/gtm: "(.+)"/,'gtm: "${gtmId}"')`
    // ); // TODO: Implement

    // Save settings
    await page.click(settingsCodeSave);
    await page.waitForNetworkIdle();

    // Close browser
    await browser.close();

    // Delete all items of CMS
    await WebflowAPI.deleteAllItemsOfSite(siteId);

    // TODO: Create site in Firebase

    // TODO: Publish Website

    // TODO: Update Firebase allowed domains

    // TODO: Setup site in Google Analytics

    res.status(200).json({ message: `Site setup. ID: ${siteId}` });
  }
);

2 Likes

Yo this is so interesting to read about.

Being a non-coder it’s difficult for me to visualise this code happening. Would you mind taking a video of this script running and showing the results?

Hello @Jose_Urbano , Sure thing! Let me try to get some time from work and then I can record a video so you can actually see the script doings its magic

@Jeandcc - Personally I use UIPath Studio to automate anything on my Windows desktop. Really no limit to what it can do. Automation Anywhere is another option. Of course there is a learning curve but it is low code.

Once my automations are built I can just fire them off as needed.

1 Like

This is all very fascinating. Thank you for sharing!

1 Like

Very interesting feedback and great software suggestions @webdev . They might not work for my use-case (which needs to cover a situation of me not being able to even trigger the automation), but they’re definitely something to keep in mind for the future. The provisional solution I came up with does work well given my needs of allowing ‘anyone’ to send a form submission and trigger the setup process entirely by themselves.

(Maybe I’ll use one of those to setup Universal Analytics, that doesn’t offer an API and it’s tricker to ‘hack’ around if compared to Webflow).