Hands-Free Web Scraping for Hands-On Marketers

Subtitle: Tips for dealing with “Load More”

Quick Tips

Visit your target website
Click Listly’s extension icon and select LISTLY WHOLE / LISTLY PART
Copy CSS Selector of “Load More” button
Click Edit Setting on Databoard
Paste CSS Selector to auto-click field
Refresh data sources

Marketers spend hours on routine tasks from market research to competitor price monitoring. Well, there’s no getting around the need for marketing automation. Above all, we cannot talk about it without web scraping. By the end of this read, you’ll be ready to gather data from any website using “Load More” buttons.

In this tutorial, we’ll collect women’s new product information from the H&M website.

The website displays only 36 products at a time, and you can load more items once you click the “LOAD MORE PRODUCTS” button.

Normally you should repeat mouse clicks until there are no new products left, but you can automate this process in just 10 seconds with Listly.

Click LISTLY Whole and you’ll see that only 36 items were collected.

Move your mouse to the “LOAD MORE PRODUCTS” button, right-click, and select Inspect.

You’ll be able to see some of the code is highlighted in blue, right-click again and select Copy and Copy selector.

Go back to your Databoard and click add setting.

There you go, paste the element CSS selector to the following field. You can also change the number of times to repeat the auto-click action.

Repeat N Times: The number of times to repeat the auto-click action
Wait after clicking: The amount of time that is given to Listly’s web scraping service after clicking a “Load More” button

Here, we set the auto-click action to repeat 23 times and changed the wait time to 5 seconds.

All right, now it’s time to hit the refresh data button!

What do you say? Every single new item on the H&M website was collected in just one page.

You know what? You can also download the data by simply clicking the EXCEL button on your Databoard!