Is your price monitoring solution flawed?

Why high-frequency data scraping spiders undermine the accuracy of reseller price monitoring

Price monitoring technology can be a powerful commercial tool for brands selling their products through third parties–no matter what sales channels you’re using. However, the challenges with selling online are the sheer size of the digital shelf and the fact that prices constantly change. This makes effective human monitoring virtually impossible.

But there is now an answer–AI-powered tools that can automatically analyze hundreds of your resellers’ pricing strategies in real-time right across the digital shelf.  These tools also allow you to monitor your competitors. And observe price fluctuations in specific product groups.

With a new product launch, price monitoring enables you to analyze how other brands price similar products. You can use this information to pitch your product at a competitive level. This wealth of data-driven insight allows you to maximize your profit margins. While creating an interesting offer for your resellers.

Choosing a price-monitoring tool isn’t without risk. Many solutions on the market gather and process pricing information in a way that undermines data accuracy, reliability, and scalability. An excellent example of this is the solutions that use high-frequency data-scraping spiders.

The problem with high-frequency data-scraping spiders

Price-monitoring technology harvests data from online reseller channels similar to how Google’s search engines crawl the web. A software called spiders ‘visit’ eCommerce sites and collects or ‘scrape’ pricing data. What is less well known is that there are good and bad spiders.

High-frequency spiders scrape data from reseller websites multiple times a day to keep KPIs updated. This may sound like a good idea. But this approach causes problems for resellers and brand manufacturers alike.

That’s because high-frequency spiders operate at peak trading times. They arrive in such large numbers that they slow down website speed and cut site conversion rates. In response, resellers often install protection software that produces a torrent of false data to confuse spiders.

The result is that these high-frequency spiders gather inaccurate and untrustworthy data–precisely the opposite of what brand manufacturers want. Brands that condone high-frequency data scraping also risk upsetting valuable reseller relationships.

Compare the effectiveness of high-frequency spiders with the ethical alternatives used by market-leading price monitoring solutions such as Detail Online.

The benefits of high-quality, ethical spiders

Ethical spiders may be more expensive. But that’s because their algorithms are more accurate and sophisticated. For example, you can individually customize them for each reseller website to ensure greater accuracy. You can also program them to follow the sun worldwide. Operating in the early morning and during low-traffic periods, so they don’t slow retailer websites and impact conversion rates.

They also use local IP addresses, so the end-user benefits from a local customer’s eye view. It’s safe to say that for price monitoring technology, you get what you pay for. The most advanced ethical spiders don’t just monitor the price. They also analyze data such as:

  • Product content correctness.
  • Stock availability,
  • And customer reviews.

They also dig down into primary, secondary, and even tertiary content to provide the high-quality insights brands need. They can analyze first pages, category pages, product pages, descriptions, prices, images, and out-of-stock statuses.

Ethical spiders can also check whether products appear in the relevant internal and external searches and categories. Leading price monitoring solutions can double-check data collected by their spiders using AI-powered validation engines. If any data seems out of place after this validation loop, they can automatically flag it for a manual check by a human. This rigorous approach means leading price-monitoring technology can achieve an accuracy level of up to 98%.

AI-powered price monitoring–a universal solution

The digital shelf spans many countries, so the best price-monitoring solutions must also be truly global. They can audit reseller channels in any retail segment, region, time zone, or language. Scalability is another crucial factor to keep in mind. Price-monitoring software built on cloud platforms, such as Microsoft Azure, is fully scalable, so your solution will grow as your company grows.

Monitoring your resellers’ and your competitors’ prices is a legitimate way to help ensure you preserve your profit margins while remaining competitive. While brand manufacturers can use price monitoring to set reseller prices in the US, in the EU, and most countries, brands must ensure the data they collect isn’t used to influence how retailers price their products.

Savvy consumers now use price monitoring and product comparison tools at many stages of their online journey when shopping. So it makes sense that brand manufacturers adopt a similar strategy to ensure they remain competitive.

Originally published Apr. 21, 2021, updated Apr. 22, 2021

Get a Digital Shelf Analytics demo

Get a hands-on, personalized demo and see for yourself why global brands use our tool every day to monitor, analyze, and improve product visibility to succeed in digital commerce.
Book a demo