Logi News

What’s New in Zoomdata 4.9: Microservices for Elastic Scalability

By Mary Flynn
Share on LinkedIn Tweet about this on Twitter Share on Facebook

Logi Analytics is proud to introduce Zoomdata 4.9, the latest Long Term Service (LTS) release of Logi’s embedded data exploration solution for modern software applications. Zoomdata 4.9 continues its commitment to delivering great user experiences, a smarter and faster query engine, and first-of-its-kind elastic scalability for embedded analytics software.

Cloud-Ready Embedded Analytics for Modern Software Applications

It’s well understood that the preferred software delivery model is now SaaS (Software as a Service), which allows organizations or consumers to subscribe to software hosted in the cloud. Application teams that deliver software through a download-and-install model still expect their software to be deployed in a public or private cloud.

One of the many benefits of cloud services such as AWS (Amazon Web Services), GCP (Google Cloud Platform) and Microsoft Azure is that you can spin up and down compute resources on-demand. This elastic scalability lets you maximize performance under heavy load and minimize costs because you only deploy resources when they’re needed. If you’re not familiar with elastic computing, read that sentence again. It’s DevOps nirvana.

To take full advantage of cloud capabilities, modern software applications are developed using a microservices architecture (MSA). Properly architected, a mature MSA isolates key capabilities into self-registering microservices. Upon deployment, microservices are allocated the compute resources they individually require for optimal performance.

High Availability and Horizontal Scalability: Monolithic vs Microservices

Not too surprisingly, a lot of embedded software products are still developed as monolithic applications. While feature-rich and mature, they don’t deploy or scale the same way as their modern parent applications. Traditional web application architecture offers only two scalability models:

  • Scale vertically by increasing compute resources on a single node — this can provide more capacity but no failover
  • Scale horizontally by replicating the full software stack over and over again behind a load-balancer — this provides more compute power and adds high availability (HA), but at great expense and effort

Monolithic applications simply cannot scale quickly enough to meet unanticipated demand, and traditional HA deployments are expensive and aggravating to maintain.

Zoomdata 4.9 from Logi Analytics lets you deploy microservices where and when they make sense.

Elastic Scale Using Distributed Microservices

In the diagram below, all Zoomdata microservices are deployed at least two times to avoid single points of failure (SPOFs) and assure high availability. In large deployments, you may find that some microservices need more compute resources than others, such as the zEngine, or a Smart Data Connector for Cloudera Impala, Elasticsearch, Snowflake, or the like. In these scenarios, you can pick and choose how many instances you want to deploy of each microservice, without wasting compute resources where they are not needed.

In the sample diagram below, note that microservices can be replicated any number of times, and run independently of each other.

Microservices HA and elastic scalability

When more resources are needed to meet demand, you can elastically spin microservices up and down at any time. They self-register with each other and gracefully shut down, and your customers will never know the difference. In the diagram above, additional zEngine and Smart Data Connector microservices were deployed to meet demand, and then shut down when demand subsided.

Zoomdata’s microservices architecture delivers high availability and horizontal scalability:

  • Fully redundant microservices expose no Single Points of Failure (SPOFs)
  • Container ready
  • Scale where you need it
  • Scale elastically up and down when you need
  • Optimize compute resources such as CPU processing power and RAM

What does this all mean for your SaaS or cloud-based application? For one, it means your DevOps team will love and respect you. Why? Because the microservices architecture in your embedded analytics solution will be resilient, scalable, and optimized to meet demand and optimize budget. And that leads to the real benefit: a profoundly better user experience that keeps your customers engaged in your application and coming back for more.

Enhanced User Experience

Zoomdata has always been easy to use, and with version 4.9 it keeps getting better. Usability enhancements include a cleaner, faster, and more productive data exploration experience, a more developer-friendly and user-friendly expression editor, and an upgraded Data DVR that makes it more intuitive to analyze time-based data.

In Zoomdata 4.9, the Data DVR time bar now displays the full date range by default. The selected range is highlighted in blue (rather than yellow), and the date ranges that are not selected appear in grey. This makes it easier to filter, navigate, and understand time series data.

Data DVR & Timebar

The expression editor is now easier to use, with common developer hints such as color-coding, type-ahead / auto-complete, and contextual help.

Zoomdata 4.9 expression editor

Zoomdata standardized and updated its charting library for smoother redraws when playing data, and to make charts easier to extend and customize.

Zoomdata 4.9 ECharts 4

Raw data tables and map markers can now refresh with smooth, up-to-the-second live data updates.

Zoomdata 4.9 Raw Data and Map live mode

You can now drill down to visualize data at the millisecond (1/1000 of a second) level. This level of detail is important for many IoT, telemetrics, and many clinical drug and healthcare use cases.

Zoomdata 4.9 millisecond granularity

Advances in Modern Data Sources

Zoomdata continues to develop new Smart Data Connectors (SDCs) in response to market demand. In addition to our existing SDCs for modern data sources, we now offer real-time and interactive visualizations for:

  • Couchbase, a distributed NoSQL document-oriented database that is optimized for interactive applications and high concurrency
  • TIBCO Data Virtualization, an enterprise-class data virtualization solution that combines disparate data sources on-demand

Zoomdata 4.9 also enhanced its support for Google BigQuery, and added expanded security capabilities for the following SDCs:

  • Solr: Kerberos support
  • Oracle : SSL
  • Apache Drill: user impersonation / delegation

Smarter Data Engine

One of the benefits of API-first development and investing in a microservices architecture is that you can refactor and even replace components in ways that you simply can’t do with monolithic applications. And that’s exactly what we did in Zoomdata 4.9. When we developed the query engine a few years ago, it was tightly coupled to an embedded Spark server. Over time our development team saw some weaknesses in that approach, such as concurrency and microservice-compatible scalability. So we built a new engine that exposes the same APIs but is more powerful, versatile, and easier to maintain and extend.

Risky? Not really. This “new” engine, which we call zEngine, has been used since version 3.6 for Data Fusion. With Zoomdata 4.9, the zEngine is now the default query engine for all user requests. If for some reason you need to fall back to the classic query engine, contact our support team and we’ll walk you through the process.

Finally, with the new zEngine, Data Fusion now supports chart types that were difficult or too resource-intensive to compute on the classic engine: histogram, box plot/quartiles, and raw data tables.

Boxplots help you evaluate outliers, data symmetry, and how tightly your data is grouped:

Zoomdata 4.9 boxplot charts with Data Fusion

Histograms are very good at displaying the distribution of numerical data:

Zoomdata 4.9 Data Fusion - histogram

Additional Resources

You can learn more about our Microservices Architecture: A Brief History At Zoomdata on the Zoomdata blog.

Zoomdata documentation is now on Logi DEVNET, where you can find the LTS 4.9 Release Summary and notes for every version 4.x Rapid Release. To learn more about our agile delivery of Rapid Release and Long Term Supported software, see Zoomdata Release Vehicles.

Originally published November 14, 2019

About the Author

Mary is the Director of Product Marketing at Logi Analytics, where she joined through Zoomdata. Prior to her role at Zoomdata, she kept herself busy as the mayor of the Town of Chevy Chase in Maryland, and a community advocate in Montgomery County. Mary's professional career spans twenty years in software product marketing, sales engineering, and management roles, starting early on at Brio Technology and more recently at Jaspersoft.