Last year, in the evening hours of Black Friday, brands and agencies on Amazon began noticing a peculiar problem: Amazon was reporting inaccurate advertising data.
The numbers associated with bids, budgets, and revenues on Amazon all seemed confusingly off the mark. In some cases, Amazon told advertisers that their ad spend was half of what they had expected.
In the world of Amazon advertising, it was a high-profile example of what those of us who work with Amazon data every day already know: Amazon’s reporting tools are not infallible. Errors and incomplete data sets are part of the price of doing business in e-commerce. And brands that make advertising and other spending decisions based on Amazon’s reports have to prepare themselves to identify and neutralize these errors.
Preparedness is especially important because the typical error isn’t a highly visible reporting blackout on the scale of Amazon’s Black Friday issue. Rather, we usually encounter much smaller issues—an incomplete data set, say, or a late-arriving report—that can more easily slip through the cracks.
What do we mean by data errors?
Many companies, including us here at Intentwise, collect data from a network of Amazon APIs and from Amazon Marketing Stream, Amazon’s hour-by-hour look into a brand’s ad performance and bid management strategy.
These tools can tell you how much you spend on advertising, how much revenue those ads drove, and how your cost-per-click fluctuate throughout the day, among many other insights. They are powerful data feeds—but they come with challenges.
First, Amazon’s APIs can be difficult to integrate into your business, as they require constant updating and fine-tuning. Last year alone, for instance, Amazon released five different versions of its Sponsored Ads API.
Second, these APIs sometimes have errors and inconsistencies. Yes, in the aggregate, these issues are rare—but if you’re not proactive about identifying them, even a small mistake in Amazon’s data can have outsized repercussions.
Broadly, we find that data errors tend to fall into one of three major buckets:
Data not being available on time. Amazon usually estimates when the next rendition of a report is expected to become available—say, at 6 a.m. eastern on Friday. But once in a while, Amazon will miss its own deadline, and brands will find that they don’t have the data they expected at their fingertips.
Incomplete data. Once in a while, you might download an Amazon advertising report and—whether you notice it right away or not—realize it has some key data points missing. It’s rare, but not totally uncommon, for a report from Amazon to be missing multiple rows or columns of data.
Delay on the API end. By far the most common data reporting issue we see is the lag time between reporting on the Amazon Advertising console and on Amazon’s APIs.
The console is what you see when you log into your Amazon Advertising dashboard, and it tends to have the most up-to-the-minute reporting. The APIs, meanwhile, allow you to migrate that ad data into other platforms and data warehouses—but sometimes there is a lag associated. In the worst case scenario, these lags can stretch on for hours or even days. If you’re not prepared, these discrepancies in reporting are incredibly confusing.
What problems can data errors cause?
At Intentwise, we encounter data errors all the time. We have seen data errors from Amazon significant enough that they skew reported ad spend or revenue by as much as 3% to 4%.
Often, the impact of an error varies a lot based on both the specifics of the mistake and the needs of the brand or agency that is impacted. As Raghu Kashyap, CTO of Intentwise, put it recently, “It really depends on the scale of a customer.”
While a major brand might not notice a few missing data points, a smaller one definitely will. We recently worked with a smaller agency, for instance, which found that Amazon underreported the sales generated from an ad campaign. Even though the difference amounted to only a small number of sales, those underreported sales made a big impact on how the agency evaluated the success of the ad campaign.
How do you resolve a data error?
Our answer is simple: Build in systems that constantly check for data issues.
At Intentwise, we collect massive amounts of data from Amazon’s network of APIs and data streams every day. We download every report that Amazon publishes, and we pull each one into our various platforms, including Intentwise Analytics Cloud.
To ensure accuracy, we are constantly running automatic checks and balances on each new report from Amazon to make sure everything looks right.
Let’s take one of the above cases as an example. A brand downloads its latest ad spend report, and it looks shorter than usual. We know, for instance, that the same report last year had X number of rows, and we are able to automatically detect any discrepancies. If a metric looks amiss, or if a report seems incomplete, we will automatically flag it and investigate.
Whether you opt to partner with an outside data manager like Intentwise or you build a data engine internally, you should be sure to implement a similar preparedness system. When Amazon’s APIs are informing your business decisions, make sure you are prepared for delayed reports, incomplete data sets, and anything else that might crop up.
