Real-time data visualization. What requirements does fast data impose on business intelligence (BI) platforms?
Real-Time Business Intelligence
Real-time business intelligence should not be confused with real-time activity monitors. Real-time BI is a richly visual, human-oriented data exploration experience that uses fresh, constantly developing data. Like the best traditional BI solutions, it allows you to explore and interact with data by creating and reconfiguring charts and graphs, zooming in, filtering, drilling down to detail, and creating custom calculations.
But, unlike old-school BI, real-time BI can handle massive amounts of data and process user queries, while fresh data continues to roll in. That means that organizations can use their live data to do new things, such as increase sales through event-based marketing, better manage financial portfolios in real-time, and respond more quickly and accurately to public health crises.
The Limitations of Traditional BI
Traditional BI is designed for data that is modeled and packaged in batches — not high volume, high velocity data. Traditional BI also have significant performance problems with very large data sets. Most traditional BI expects an arbitrary query extract before users can visualize their data. Getting that initial query (and then dealing with changes!) can take forever and prove very frustrating to users.
IT professionals try to compensate for this shortcoming by extracting data periodically from big data systems and transforming it in batches to make it more consumable by the older tools.
But this approach only skirts the issue. As more types of data come online, more resources are necessary to manage the transformation and batch process. Batch-oriented cannot deliver real-time business intelligence.
The new way forward in BI is to think of data in streams rather than as a collection of data snapshots. When you think about it, even historical records can be represented as a stream. For example, sales transactions are made at points in time, and with real-time BI platforms you can visualize those transactions over time while filtering and enriching the data.
Logi Composer was built streaming-first. It’s designed to operate with low-latency, high-bandwidth networks and modern streaming platforms.
It starts with the architecture, with a messaging and queue management system that is resilient. And it can access data from any high-performance big data store.
Every feature of Logi Composer, from the user-specified refresh rate and the use of microqueries to our Data DVR and data Fusion capabilities, originates from our dedication to the core principle of streaming data visualization at scale.
And it’s why no amount of retrofitting a traditional BI system can really cope with the demands of streaming data. Every modification that old-school BI vendors make to accommodate streaming data requires a compromise that will eventually affect performance, scale, and/or complexity.