If you sell on Amazon, you’re probably inundated with data. Every day, you need information about your ad budgets, clicks, impressions, sales velocity, inventory levels, and so on.
Managing that data might not be a big deal for a brand that is just starting out. But in my experience, as they grow in size, most brands realize they need better processes for collecting and analyzing all of that data if they wish to remain competitive.
Eventually, they’re faced with a question: Do they hire a group of in-house programmers and data analysts to build their own data infrastructure? Or do they work with an outside partner to do it?
It’s the classic build vs. buy dilemma. I’m asked about this a lot, and I think there are a few key points for brands to consider.
What are the upsides of in-house data management?
Obviously, I’m a little biased here: I’m the CEO of a company that is able to make API connections, enrich that data, and enable analysis, all in a single platform.
There are definitely some upsides to having an in-house data team. For example, in-house teams:
- Know your business inside and out, and can run analyses on your data that are tailored toward creating actionable insights unique to your needs.
- Understand the metrics you are most interested in tracking and utilizing, including metrics that might be either obscure or brand-specific.
In-house data teams are well positioned to implement insights from your data. That’s what they do best—and they shouldn’t get bogged down in the messy, painstaking process of building up your data infrastructure from scratch.
External solutions will establish these foundational API connections better, faster, and cheaper than any in-house option. When you choose an external partner, you free your data teams up to focus instead on extracting actionable insights.
The risks: Building a data collection process takes time
So what happens if you choose to create these connections in-house?
Building in-house API connections is not simple. A typical brand might pull in information from several of a number of data sources:
- Amazon Sponsored Ads API
- Amazon DSP API
- Amazon Vendor Central API
- Amazon Seller Central API
- Amazon Marketing Cloud API
- Amazon Marketing Steam API
- Walmart APIs
To connect to an API, you need to write software programming that requests the specific data sets you want directly from the API. You then need to store this data somewhere.
Building these connections might sound simple, but it’s not. According to brands I’ve spoken with, it can take an experienced team of software developers 3-4 months.
Just building the API connections isn’t enough
I often find that this first step—linking and warehousing your API data—is actually the easy part. For brands, the real challenge is in organizing this data effectively to allow for actionable insights.
For example, once you create that firehose of information to your data warehouse, you then have to:
- Create mapping tables. Remember, just because you pull API data into your warehouse does not mean that data is organized according to your needs. You have to build a vast network of mapping tables that group the inputs for you. Let’s say you want to track your ad campaigns across both your brand and non-brand keywords. You’ll need to build a mapping table that identifies certain keywords as brand keywords (“Nike sneakers”) and some as non-brand (“sneakers”). There are countless examples of this across keywords, campaigns, groups of products, etc.
- Add reference data. If you want, say, a view of your profitability by product, you have to attach an always-updating table of your profit margins to the data warehouse. You have to set that margin data to frequently refresh, since it shifts often.
- Convert currencies. If you sell in multiple geographies, in order to track your ad spend or profitability numbers, you need to build workflows that automatically convert currencies.
Some notes of caution for the in-house solution
When deciding whether to hire someone in-house, you should also keep in mind:
To build a working data store, you need distinct types of expertise. That usually means multiple hires. For instance, the person you hire to build the API connections to your data warehouse needs software skills and experience with Amazon’s APIs.
Meanwhile, you would rely more on a data analyst to organize the data and build the dashboards. The data analyst needs e-commerce knowledge, and should be comfortable with Amazon’s reports and metrics to ensure they can make this data actionable.
Be prepared for future API updates. Amazon is constantly fine-tuning its APIs. In 2022, Amazon released 5 different versions of the Sponsored Ads API alone.
That means more data, but it’s also a lot of work to keep up with. Every time there’s an API update, you have to adjust your data connectors. Sometimes these updates are easy to incorporate. Other times, the outcome is more complicated.
One recent update to the Sponsored Products API, for example, allowed us to download reports for multiple days at a time, rather than for just one day.
That’s a huge benefit—but rewriting your data connectors to include that capability is a big lift for a small team.
