In the previous post, we briefly mentioned the “Grail – data lakehouse” solution, which is Dynatrace’s response to data challenges such as: their rapid growth, building data logic, automating the process of data collection, analysis and security, or ensuring BIZDEVSECOPS in the “as a code” model. Below, we will introduce the topic in a bit more detail.
Grail is a Dynatrace database specifically designed for observability data. Works as one unified solution for storing data, logs, metrics, traces, events, and more. All data stored in Grail is interconnected within a real-time model that reflects the topology and mutual relations in the monitored environment.
Grail fundamentally improves upon the technical core of the Dynatrace Software Intelligence Platform. In the case of multicloud and cloud-native architectures, the amount of data and the complexity of application environments and the dependencies between their components increase dramatically. Businesses need an effective way to store data in the right context and search for it for instant insights and process automation.
Grail was built with one primary goal in mind: to make all your data — and therefore their value — available for accurate, real-time answers. This purpose is not limited to observability activities. Grail supports security data as well as business intelligence data. With this in mind, Grail needs to achieve three main goals with minimal cost impact:
- Dealing with and managing huge amounts of data – both during acquisition and analysis;
- Work with different and independent data types;
- Put data into context and enrich it with topology metadata.
The above principles affect its architecture. Grail is a tool that combines the advantages of a data warehouse with the advantages of a data lakehouse. The data warehouse is specifically built and optimized for specific use cases, providing valuable insights into structured data and capable of handling large data volumes. In contrast, data lakehouses can handle various types of unstructured and semi-structured data to an unknown extent, thus introducing openness and flexibility.
To enable this architecture and data lakehouse, Dynatrace separated storage from analytics and computation. This separation keeps data and storage formats open while keeping data in context. In addition, it creates a rich analytics layer powered by Dynatrace’s causal AI, Davis® AI. Additionally, it creates a new DQL query engine that offers insights at unmatched speed.
As a result, Grail was created from three different building blocks, each with a specific purpose:
1) Acquisition and processing: High performance, automated data collection and processing.
2) Storage: A storage solution specifically designed for observability and data security.
3) Analytics: sifting through petabytes of data in real time to extract real value from it.
You can read more about each of the above aspects on the Dynatrace blog >> here
