Webflow and Web Scraping

Hello,

My client is in contact with Indeed in an attempt to set up web scraping for their website.

Basically the idea is that Indeed would pull info from the “careers” page of my client’s website to add job listings to indeed.com.

However, Indeed has told him that they are not able to access the page due to geoblock settings on the website.

Is web scraping possible with Webflow? and if it is, what do I need to do to get it working?

Any help would be much appreciated.

Thanks,

Reid

Hi Reid,

Is your clients website hosted on Webflow?

Omar

Yes, I wouldn’t be here otherwise lol.

I designed and built the site myself using Webflow.

I’m not aware of Webflow having geo restrictions as an offered feature, and if they did I’d imagine you would remember setting it up for your client. First thing I’d do is ask Indeed what country their scraping IPs are in. I’d assume they’re using U.S. based IPs to scrape, being a U.S. company, so them being geoblocked is odd to me. Webflow certainly wouldn’t be geoblocking the U.S.

Okay, I’ll have my client reach out regarding this.

Will let you know what they say.

Thanks!

In the dynamic arena of the digital era, a robust WebScraping.AI emerges as a catalyst for profound transformation, weaving an indispensable link between the expansive expanse of the internet and the vital insights that propel innovation and drive strategic choices for developers and enterprises. This API serves as an ever-vigilant sentinel, tirelessly navigating the labyrinthine corridors of websites, extracting precious data with unerring precision, and delivering it into the eager hands of users. Its role is pivotal in an epoch where data stands as the crown jewel, facilitating the harvesting of a wide spectrum of information, from market dynamics and consumer sentiments to competitive intelligence and pricing trends. For developers, it is the quintessential toolbox, enabling the crafting of dynamic applications, intelligent bots, and data-driven solutions. This bridge between the raw fabric of the web and structured data becomes the pivot point for innovation, driving efficiency across various sectors.