How to troubleshoot web scraping errors – Part Selection Error
The Part Selection Error occurs when you attempt to scrape a specific part of a website using Parts and repeat this scraping task across multiple pages. This error may be caused by one or more reasons, such as
(1) Incorrect Part selection
(2) Unexpected pop-up windows on your destination website
(3) Bot detection by your destination website
(4) Failed connection to your destination website
(1) To avoid incorrect Part selection, make sure to click on a specific part you want to scrape accurately. If you're unsure, click the upward arrow (↑) in Listly's extension until the size of the green selection box remains unchanged.
(2) & (3) Otherwise, the error might be on your destination website's end. As many websites take actions to prevent bots from scraping their content by limiting the number of requests from a single IP address or popping up a promo window. In this case, you might want to set up a proxy server or adjust your extraction speed.
(4) If you're automating Part scraping tasks by grouping more than 1,000 pages, the destination website might flag your IP address for suspicious activity and block you from accessing its pages. It is highly recommended to group no more than 300 pages at a time and set the speed to 1.
