CSV Import With Multi-Reference Fields

EOC · May 17, 2017, 5:56pm

Hello All,

I have a .csv that I am trying to import that has two columns for multi-reference fields (category and department).

These multi-reference fields are used to filter collection items based on the department and then by category to organize the dynamic lists into sections on a page.

I can’t see a way to “map” which collection item relates to which department or category on import as the system has the multi-reference fields grayed out and won’t let me map my .csv fields to them.

Am I missing something?

cyberdave · May 17, 2017, 6:03pm

Hi @EOC, Mapping for those fields is not available at this time. However, that would be an awesome improvement to have for that feature, and is something you could definitely post about in our Webflow Wishlist.

I hope this helps

EOC · May 17, 2017, 6:12pm

Hello @cyberdave thanks for the quick reply, that’s a bummer to say the least.

We were thinking about migrating some WordPress clients in the near future, but how is this a truly viable option for migrating blog content if you can’t pull multi-reference fields? All the categories would be left behind for manual input later… Doesn’t the defeat the purpose?

Also, you may want to make a note to someone to update the how to article on .csv import to mention this… I wasted a lot of time creating a .csv that won’t work.

EOC · May 17, 2017, 6:27pm

Hello @cyberdave just read my last reply and hope it didn’t come across as snarky, wasn’t meant to

cyberdave · May 17, 2017, 6:35pm

Hi @EOC, no worries, you raise a valid point about the multi-reference fields. More support for this will likely come, but I do not have the timeline yet for that.

Multi-reference fields were not included to the first phase of CSV import and now that this feature is in production, it will continue to be updated and improved.

Many of the features now in Webflow have been requested in the past by customers who have trusted Webflow to build the features they need. Stay tuned!

EOC · May 17, 2017, 6:37pm

Hello @cyberdave for the record, I think you guys are doing a great job and I look forward to new features and updates!

Lux · October 8, 2017, 6:27am

Yep. Hitting this wall as well. To the Wishlist!

Fagerlund · April 17, 2018, 6:55pm

Wonder if there are any updates on this. I am trying to import 150 films with all data for a festival, and would need to map the films to Tag and Series collections. So for instance one film would come with three tags, that would already be in another collection in webflow.

Can I multi refer these automatically? Or do we need a couple of days extra budget to map them manually

cc/ @cyberdave

andre9000 · July 25, 2018, 11:53am

We solved the issue by importing without the references and afterwards using the Webflow API to update the reference fields. Was a bit of manual work, but worked just fine.

UprControl · July 25, 2018, 1:00pm

@andre9000 thanks for the news of a workaround on this. It’s been a major issue for us too when creating CMS content from external files. Can I ask how you did it?? You did one pass that imported the .csv with the reference fields left blank - then what? Did you use zapper or any other automation in your process? Any guidance on this would be much appreciated. Cheers Dylan

andre9000 · July 25, 2018, 1:15pm

I only used the webflow api with node:
https://developers.webflow.com/

You’ll need to fetch the item id of the item you want to reference.

we migrated several webflow collections into 1. the steps were the following:

export collections into 1 standardized csv using the api. the reference id was an extra field “reference id”
import the csv to the new collection
setting references using the api and the id value stored in “reference id”

Hope that helps

AMG_Blomat · June 25, 2019, 9:49am

Hi @andre9000
I’m trying to do something like this, I have about more 1,000 items in a collection, each has 4 multi-reference field, I’ve already loaded my csv without the multireference.

I don’t understand how to download the csv from api, because I can only get a json file from that, non un csv.

I already can see the id’s of my multirefence from the API.

I need to understand how to download rewrite the file and reload it.

You could better explain the process you have used? could be the only solution to my problem.

thanks for your help in advance
Matt

andre9000 · June 25, 2019, 10:08am

@AMG_Blomat
Hi Matt,

do I understand correctly: You already imported all 1000 items to a webflow collection, and now just need to set the multireferences?

Or do you want to export it, for a non-webflow CMS?

AMG_Blomat · June 25, 2019, 10:10am

first one, already imported all 1000 items to a webflow collection, and now just need to set the multireferences

andre9000 · June 25, 2019, 10:28am

I would probably not reupload everything via CSV, but just write a script that updates all existing items via webflow API.

Steps would go roughly like this:

authorize api
fetch an array of ids, one for every item in the collection
iterate through the array
- define the fields and values that should change (the references)
- update item via patch method

keep the api’s rate limit in mind.

AMG_Blomat · July 1, 2019, 10:43am

Sorry but I have a lot of problem with update the data,
Now my workflow in node is:

1 - Download all the items with the API in one big json file.

2 - convert the json in csv and edit in excel

3 - convert to JSON and update via API (but I have a big problem with the rate limit)

The problem is the updateItem() method work single item, one by one,
it’s possible to give all the items of the JSON in one single API call?

Probably the patch method you mentioned but how it’s work?

andre9000 · July 1, 2019, 12:02pm

one possibility would be to reattempt the request every few seconds, until it succeeds. We recently did a bulk update for several collections with this code, it dealt with the rate limit just fine:

// map: obj with changes
async function updateAllItems({ webflow, collectionId, map }) {
  const ids = await getAllItemIds({ webflow, collectionId });
  for (const [i, itemId] of ids.entries()) {
    try {
      await updateItem({
        webflow,
        collectionId,
        itemId,
        map
      });
    } catch (err) {
      console.error(`Error at index ${i}:`);
      console.error(err);
    }
  }
}

async function updateItem({
  webflow, collectionId, itemId, map
}) {
  try {
    // currently not implemented in webflow-api package
    const url = `https://api.webflow.com/collections/${collectionId}/items/${itemId}`;
    const options = {
      body: JSON.stringify({ fields: map }),
      method: 'PATCH',
      headers: {
        Authorization: `Bearer ${webflow.token}`,
        'Accept-Version': '1.0.0',
        'Content-Type': 'application/json'
      }
    };
    const response = await fetch(url, options);
    checkStatus(response);
    console.log(`✔ Updated ${itemId}`);
  } catch (err) {
    if (err.message === 'Too Many Requests') {
      console.log('too many requests');
      const waitingTime = 1000 * 5; // 5s
      wait(waitingTime);
      console.log('retry...');
      updateItem(...arguments);
    } else {
      throw err;
    }
  }
}

andre9000 · July 1, 2019, 12:53pm

(obiously there are some other functions being used, but they are either not important, or it is obvious what they are supposed to do)

AMG_Blomat · July 1, 2019, 2:32pm

thanks, seems clear what that function has to do, but I’m a real noob with javascript and json,
can I see how you declare this variables?

webflow,
collectionId,
itemId,
map ← this is the json of all field I want to change correct?

I’m trying to define that variable, but I still get always this error

(node:4297) UnhandledPromiseRejectionWarning: TypeError: Cannot destructure property webflow of 'undefined' or 'null'. at updateAllItems (/Users/mattia/Downloads/cms/update-all.js:14:30) at Object.<anonymous> (/Users/mattia/Downloads/cms/update-all.js:62:1) at Module._compile (internal/modules/cjs/loader.js:701:30) at Object.Module._extensions..js (internal/modules/cjs/loader.js:712:10) at Module.load (internal/modules/cjs/loader.js:600:32) at tryModuleLoad (internal/modules/cjs/loader.js:539:12) at Function.Module._load (internal/modules/cjs/loader.js:531:3) at Function.Module.runMain (internal/modules/cjs/loader.js:754:12) at startup (internal/bootstrap/node.js:283:19) at bootstrapNodeJSCore (internal/bootstrap/node.js:622:3) (node:4297) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1) (node:4297) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

andre9000 · July 1, 2019, 3:00pm

The file I copied this from is much bigger, so this is all just parts of it, probably needs some changes to be able to run

map ← this is the json of all field I want to change correct?

Yes!

For ‘webflow’, i use webflows npm package

you can call the functions from above like this:

import Webflow from 'webflow-api';
async function run() {
  try {
    const webflow = new Webflow({ token: YOUR_WEBFLOW_API_TOKEN });
    const collectionId = YOUR_COLLECTION_ID;
    const map = {
      'heading': 'My new heading'
    };
    await updateAllItems({
      webflow,
      collectionId,
      map
     });
   } catch (err) {
      console.error(err);
   }
}

run();

Topic		Replies	Views
Importing items that reference another collection CMS	24	7056	October 28, 2020
CSV import now supports reference and multi-reference fields Updates webflowcommunitynews , cms	10	3103	April 18, 2022
CSV Imports + (Multi)Reference Fields Webflow Tips	14	3169	January 25, 2021
Multi-reference field not imported for most of the CMS items General	11	1272	August 18, 2024
CSV upload to multi reference field CMS	4	928	January 15, 2023

CSV Import With Multi-Reference Fields

Related topics