“A data warehouse serves as a source of truth for your company.”
Author: Martin Schneider
– Former Big Data Engineer at Customlytics
Every big company has it and is proud of having it, and speaks about it: a data warehouse. Such as Lyft Engineering Blog, Netflix, Spotify and Airbnb.
The big question is, why do you need a data warehouse? It’s a time saver and makes you efficient and allows you to scale analytics. E-commerce companies such as Amazon, Netflix or Lyft are “trendsetter” when it comes to using data as a competitive advantage. While doing so they put pressure on their competitors to keep up with them.
Something similar is happening in the mobile app industry, independent from the vertical. The market has matured over the last years and data has become the competitive edge for many developers and marketers alike.
1. The evolution of analytics
All of them have gone a long way from using (many) Excel files until they had their own data warehouse and BI-Tool (see Ben Weber’s Medium post series or his book). The demand for data grows as the business does. It allows stakeholders across all functions to make informed decisions to further grow the business.
What is an analytics evolution? Well, it’s a scenario where a developer or a marketing manager takes care of ad-hoc data requests issuing a CSV-file for C- or middle-management. When a top-level question displays problems or surprising outcomes it results in an even higher need for data. Management wants to understand the unexpected result and request more data.
There will be a point when the time a developer has to spend or a marketer has to wait with data requests has become his full-time job. This would be the time to consider building a data warehouse.
2. Benefits of a dedicated Business Intelligence team
A typical argument against establishing a BI/A-team is: “Why should I invest in a data warehouse and a BI/A-team, which doesn’t bring any value?” Well, this is a valid question. Let me break it to you: a Business Intelligence/Analytics-team is truly not generating any direct profits or return on investment. But if you consider it as an investment into work efficiency and time, it pays off.
Having dedicated analytic resources has the following benefits:
- Free time for developers pulling data and build reports
- Easier and timelier access to data for all stakeholders
- Building a scalable and adaptable data structure in order to prevent expensive code refactoring to get the data needed
- Minimizing negative impacts on the production server, since data is replicated in the data warehouse
3. Head to the cloud or not?
After you decided to follow the path of building your data warehouse you will be faced with the decision of having a cloud or on-premise solution.
Well, for me the answer is quite simple: a cloud solution is the answer. Why? Because it scales with your demand and doesn’t require a lot of staff to operate.
Another good argument for a cloud solution is the development and further competition in the cloud market. It’s becoming easier and cheaper to build a scalable and maintainable data warehouse. Which makes it a viable solution even for small companies.
Cloud providers also offer credits, which allow you to build and test your setup during the initial phase:
Now let’s get back to the efficiency argument from earlier. As your company grows you have to incorporate more and more data sources to perform the requested analytics or enrich your data.
This quickly becomes a time-consuming and tedious task to do it for every requested report.
The natural next step is to automate these tasks. Also, making the data available on demand opposed to on-request, brings you naturally to a data warehouse.
A data warehouse serves as a source of truth for your company. It’s the place where all first and third party data is combined and waiting to be analyzed and consumed to generate more insights.
An example of third party data is listed below:
- production server logs
- Mobile Measurement Partner (MMP) data &
- marketing data
- iTunes/ Google Play stats
- Finance data
- mapping IP to Geo
4. The basis matters
All data in the world doesn’t help if the data doesn’t provide the answer to your business questions. So defining the right KPIs for your business is crucial as deciding for the right technologies.
Across a company, different stakeholders require different information at different frequencies or level of detail. All these thoughts should go into the architecture of your data warehouse. It will enhance the latency a stakeholder has to wait until he gets the data required to make a decision.
To sum it up: from a technical perspective, a data warehouse is a valuable investment into efficiency.
For Marketers, a data warehouse is the source of knowledge where they find all information easily they need to make data-driven decisions.
Curious how data warehousing can help your organization reach its marketing goals? Find more information in this article or drop us a note at [email protected].
