Evaluating an Embedded Analytics Solution

Great! You’ve decided to invest in embedded analytics. Now what?

Picking the right solution involves thoroughly evaluating the technology, understanding the expertise offered by the vendor, and implementing a process to ensure success.
First, let’s examine the evaluation criteria that are critical to embedded analytics implementations. These include both the technical and non-technical requirements that are common to most evaluations.

  • Self-Service Capabilities: These are the core capabilities you will make available to your users. These may include dashboards and reports as well as the interactive and analytical functions they can perform.
  • Data Environment: While the solution you choose will connect to your current data environment and meet your data security needs, it should also be flexible enough to meet future demands as your data tier evolves.
  • Embeddability, Customization, and Integration: One of the major ways embedded analytics initiatives differ from standalone analytics projects is the need to integrate with the application environment. This means providing a look and feel aligned to your brand as well as the extensibility to meet evolving business requirements.
  • Development and Deployment: Since time-to-value is so critical to the success of the project, having a development environment where you can create, style, embed, deploy, and iterate on embedded analytics will enable your team to deliver the functionality the business demands.
  • Licensing, Services, and Company Expertise: Choosing the right partner is not simply about the technology; it’s also about finding the level of expertise you require for training, support, and services, as well as agreeing on the business terms that ensure shared success.

After detailing the common evaluation criteria in each of these categories, we will walk through the entire process for a successful evaluation.

Self-Service Capabilities

These are the core capabilities for all the end users of your application. During your evaluation, make sure that the capabilities important to your project are demonstrated and understand how you will deliver and iterate upon these capabilities inside your application.

Category Strategic Objective Requirement
The Spectrum of Self-Service Personas Increase the adoption of embedded analytics by providing a broad range of users with a tailored experience that matches their needs and skills. Users will typically fall into one or more self-service personas.
  • Information Consumers prefer a defined experience where they can access core business metrics through dashboards and reports that have been prepared for them.
  • Content Creators are more knowledgeable information workers who respond to ad hoc requests for new dashboards and reports.
  • Analysts need an exploratory environment to discover insights and create new metrics that drive the business forward.
  • Presentation and Information Delivery Empower everyone to quickly draw conclusions, monitor key performance indicators, and obtain a complete view of the business through visualization.
  • Data Visualizations include bar charts, gauges, heat maps, spark lines, and geographic maps.
  • Dashboards, both static and interactive, present multiple visualizations in a single view.
  • Reports, both static and interactive, present tabular views of data.
  • Provide an optimal user experience regardless of where and how users prefer to access information. Evaluate the compatibility of solutions across different devices and formats.
  • Web Browser: Users should be able to access all content and capabilities on standard web browsers.
  • Mobile: Users should also be able to easily access and interact with analytics on mobile devices and utilize mobile features such as touch input.
  • Exports: Content should be available in non-web formats for printing and offline access, such as PDF and Excel spreadsheets.
  • Interactivity and Automation Create an engaging experience where users can explore the data and interact with the information presented.
  • Filtering: Users can choose the data that is important to them and get more specific in their analysis.
  • Drilling: Users can dig deeper and gain greater insights into the underlying data.
  • Personalization: Users choose the visualizations and reports most important to them, and re-arrange content into their preferred view.
  • Grow user adoption by embedding analytics into everyday work.
  • Workflow Actions: Users can perform actions on selected data, such as initiating a workflow process on specific records or updating the data, without having to leave the application.
  • Alerts: Users receive automated notifications when specific actions are taken or thresholds are met.
  • Scheduling: Content can be scheduled for delivery on a one-time or recurring basis.
  • Analysis and Authoring Empower users by giving them greater flexibility in their analysis and the ability to create and format the desired content on their own.
  • Data Query: Users choose the data sources, tables, and columns they are interested in – without having to write SQL.
  • Data Analysis and Visualization: Users can intuitively see, understand, and visualize the data. Important capabilities include filtering data, creating new calculations and metrics, and creating visualizations.
  • Dashboard and Report Authoring: Users lay out dashboards and reports and share what they’ve created with colleagues.
  • Extend the value of the data in your application by providing deeper insights into business trends.
  • Benchmarking: Users can compare their performance against industry benchmarks and identify areas for improvement.
  • Advanced Analytics: Provide a unique value proposition in your applications by developing and incorporating advanced (and often proprietary) statistical models into the analysis.
  • Data Environment

    The solutions you evaluate should be compatible with your current data environment, while at the same time be flexible enough to meet future demands as your data architecture evolves. We outline the diverse data requirements commonly evaluated by application providers.

    Category Strategic Objective Requirement
    Data Sources Leverage native connectivity optimized for the data source. Ideally, your primary data source should belong in this group.
  • Databases such as SQL Server, Oracle, MySQL, and DB2
  • Big Data including columnar/analytic data stores (e.g., Vertica, Redshift, Infobright); Hadoop (e.g., Cloudera, Hortonworks, MAPR); NoSQL (e.g., MongoDB)
  • Cloud Applications such as Salesforce.com, Netsuite, MS Dynamics, and Google Analytics
  • OLAP cubes for multi-dimensional analysis
  • Benefit from data flexibility with generic connectors when a vendor-specific connector is not available.
  • ODBC/JDBC connectivity
  • Web Services such as REST and SOAP APIs
  • Files such as XLS, CSV, and XML
  • Enjoy ultimate data source flexibility through APIs or plug-ins to connect to uncommon or proprietary data sources. Data APIs and Plug-Ins coded in your language of choice provide customized data access.
    Data Management Balance your needs for real-time reporting and interactive self-service analysis with a solution that enables you to connect directly to underlying data sources and cache data from transactional systems.
  • Direct Connect: Directly query the data source to facilitate real-time reporting and leverage the analytical capabilities of the underlying data source.
  • Data Caching: Unlike “direct connect,” data is extracted from the underlying sources into a high-performance data store to optimize reporting and analysis from transactional systems.
  • Create a complete view of the data that users can easily work with by preparing the data for analysis.
  • Multi-Source Data Blending: Data from multiple sources is combined in a single view, metric, or visualization.
  • Data transformation and enrichment: Data can be enriched for analysis. Examples include new metrics and calculated values that are frequently used, standardization of dates, aggregations, and manipulation of multi-part text (e.g., addresses).
  • Metadata: Data is made more accessible to end users for self-service analysis through friendly and recognizable names for tables and columns.
  • Create an efficient user experience where users can immediately take action on what they see in any visualization or report. Bi-Directional Data Flow: Data can flow back to source systems based on user input, such as for data updates and workflow processes.
    Transform your application into a vital hub of information by incorporating data from external sources into a single consolidated view. External Data: This could be in the form of third-party industry benchmarks, data feeds (such as weather and social media), and customer data from their specific data stores.
    Give your application ultimate flexibility to present information by consuming data services from the analytics solution. Data Services: The analytic solution becomes a provider of data in addition to analytic functionality. These data services produce output to be used by jQuery components, third-party charting, and other application functions.

    Embeddability, Customization, and Integration

    Embedded analytics implementations place a greater emphasis on customization and integration capabilities compared to standard business intelligence implementations. Application providers typically want to offer a seamless user experience within the context of their existing application and brand. Focus on enhancing the value of your application while minimizing the cost of development.

    Category Strategic Benefit Requirement
    Security It should be easy to adopt the security from your application to the analytics content. Scrutinize vendors on the flexibility of their security models, implementation of single sign-on, and whether data needs to synchronized or replicated between applications.
  • Authentication: Single sign-on should leverage the authentication of the parent application without the overhead of replicating and synchronizing user profiles in the analytics application.
  • Authorization: Roles and rights established in the parent application are passed to the analytics application to ensure end users are granted the appropriate levels of access.
  • Application Security: Fine-grained permissions can be applied to end-user visualizations and functionality, such as charts, reports, dashboards, input controls, and user functions.
  • Data Security: Security can be applied to data sources, tables, columns, and rows; this is crucial for multi-tenant applications.
  • Multi-Tenancy Accelerate development with a solution that has built-in multi-tenancy support, which allows you to create a report once and deploy for multiple customers. Multi-Tenancy: A single application has the ability to access the data for multiple customers, whether data is stored in the same database and/or in individual databases per customer. Look for parameterization and tokenization capabilities that do not require data replication or advanced data modeling to support multi-tenancy.
    User Experience Create a truly analytic application experience by embedding analytics as a natural part of your application.
  • White-Labeling: The look and feel of embedded analytics should match your brand and application, where you maintain complete control of the user experience. The logo of your analytics provider should not be visible.
  • Embedding API: Content is usually embedded via a JavaScript API; parameters can be passed from the parent application to ensure visualizations are rendered in the correct context.
  • Application Linking: Users can navigate between analytic content and the parent application, and vice versa. A common example is clicking on a part of a chart to go to the specific record in the application.
  • Workflow Create an efficient user experience where users can immediately take action on what they see in any visualization or report. Workflow Processes: Users can initiate API calls to your application from a report or dashboard in order to perform data operations or process transactions at the moment they see the data. For example, a user could select a region of a chart and perform an action on the selected records without having to leave the visualization.
    Extensibility Competing on analytics often means delivering unique functionality. Ensure you’ll be able to meet any future requirement with a solution that can be extended utilizing open standards approaches.
  • Custom Code: For specific presentation needs, see how custom HTML, CSS, and JavaScript can be incorporated. For specialized functionality requirements, understand how custom-compiled code can be integrated into the solution.
  • Third-Party Charts: For unique charting requirements, understand how third-party charting libraries and components can be utilized and embedded alongside “out-of-the-box” visualizations.
  • Development and Deployment

    Since time-to-value is so critical to the success of the project, having a development environment where you can create, style, embed, deploy, and iterate on embedded analytics will enable your team to deliver the functionality your business demands.

    Category Strategic Benefit Requirement
    Development Empower your development team with the tools to quickly create and iterate on embedded analytics capabilities.
  • Rapid Development: Evaluate the tools for how quickly you can create content, fine-tune how the content looks and behaves, and make changes to what you’ve done. Understand how to make both small changes to functionality as well as mass changes that affect the entire application.
  • Out-of-the-Box Functionality: A rich set of capabilities – visualizations, self-service analysis, input controls, and UI themes – will accelerate your product development.
  • Sample Applications: Access to working applications will accelerate the learning process and adoption of best practices.
  • Collaborative Development: Embedded analytics should integrate into your source control systems for version control and collaborative development.
  • Deployment Quickly deploy and scale an implementation that is aligned with your current technology stack. Be assured that you have the flexibility to shift as your technical environment evolves.
  • Web Architecture: The best solution fits into your web architecture, minimizing the need to deploy proprietary technology, and utilizes well-known techniques to scale the implementation.
  • Deployment Style: The greatest flexibility comes from solutions that can easily be deployed on-premise at customer sites, hosted in your data center, and made available in the cloud such as Amazon Web Services and Microsoft Azure.
  • Licensing, Services, and Company Expertise

    Choosing the right partner is not just about the technology; it’s also about finding the right level of expertise and commitment to shared success to get you to the finish line (and beyond).

    Category Strategic Benefit Requirement
    Licensing Software licensing terms should align the vendor with the value you provide to your customers. Licensing: Terms of the license can depend on a variety of factors, including number of users/customers, servers, usage, and whether you are embedding into a commercial product. Be sure the terms make business sense over the short and long run.
    Services Completing your project on time and in the right way can require resources outside your team – take comfort from a full range of services options even if you do not employ them.

  • Pre-Sales Technical Support: Leverage pre-sales resources to fully evaluate solutions; this experience will give you an indication of the vendor’s commitment to you as a customer.
  • Professional Services: Whether you simply need to augment your staff with a consultant or require a whole team to complete a large scope of work, evaluate the range of professional services offered and the extent of the partner network.
  • Training: Virtual and instructor-led training options will bring your development team up to speed quickly and help them gain a firm understanding of best practices.
  • Customer Success Vendors should supply a process that maps your path to success and provide a wide array of resources to address any issues along the way.
  • Onboarding: Look for a process that quickly ramps up your team on the solution, aligns resources so you have a clear path to success, and sets milestones for completing each phase in the project plan.
  • Account Management: Expect dedicated resources who proactively oversee your account, keep you updated on the latest product news and trends, and can be relied upon to handle your questions.
  • Documentation: The quality of product documentation is another sign of a vendor’s commitment to your success, so read carefully.
  • Support: A combination of live and self-service support options backed by experienced professionals should be available to help you work through any technical question. Service-level agreements should clearly set expectations for response times.
  • Community: An active user community can lend peer support and share valuable best practices so you can benefit from the experience of others.
  • Expertise Leverage the vendor’s experience and commitment to making you and others like you successful.
  • Company Expertise: Inquire about the vendor’s history with embedded analytics and the resources dedicated to partnering with software providers (OEMs).
  • Product Roadmap: Inquire about future product releases that will benefit you and your customers.
  • References: Ask to speak to existing customers similar to your own business.
  • The Evaluation Process

    Now that we’ve established the criteria for evaluating vendors for embedded analytics, let’s look at the overall process that will help you make the best decision for your business.

    1. Determine Your Goals

    To get where you want to go, write it down. Statistically speaking, you increase your likelihood for success simply by putting your goals on paper.

    Draw from the strategic benefits we discussed earlier in Chapter 2 (The Business Case for Embedded Analytics).

    • Quantifiable metrics may include increasing revenue, increasing user adoption, or improving customer retention.
    • Soft metrics may include an improving the user experience, creating a competitive differentiator, or increasing customer satisfaction.

    2. Establish the Timeline

    Identify the steps you’ll take to reach your goals. Ask yourself, “When do I want to…”

    • Begin the selection process?
    • Have detailed vendor presentations and demos?
    • Finish a proof of concept?
    • Make my final decision?
    • Start development?
    • Release product?

    3. Assemble the Team

    Determine the stakeholders who need to be involved. Who is going to care about embedded analytics internally (your executive team, product management, lead developers) and externally (your key customers, customer advisory board)? Build the business case collectively to secure buy-in to move forward.

    4. Identify Requirements

    Review your technical and non-technical requirements, using the previous pages as a guide to rank and weight the importance of these requirements. Research your competitors and talk to your customers to develop a firm understanding of the capabilities you want to add to your application.

    Take the functional scenarios that describe how end users will work with embedded analytics inside your application and what they will be able to accomplish, and turn those into technical requirements.

    Consider who will use the third-party products internally. Understand their skill sets and identify any potential resource gaps as you move into the evaluation phase.

    5. Research Potential Vendors

    Assign a point person to research potential vendors and evaluate whether their functionality matches your requirements. Utilize independent industry resources, such as the Gartner Magic Quadrant for Business Intelligence and Analytics Platforms report, to create your initial list. Pay special attention to vendors who specialize in the OEM market for software providers.

    Attend product demonstrations by each vendor to confirm a basic fit. Discuss your requirements and ask each one to demonstrate how they would deliver your specific processes and scenarios. Ask tough questions and make sure the vendors show you the functionality they promise. Confirm ballpark pricing to move forward.

    Evaluate each vendor’s ability to make you successful during the implementation process through access to best practices, community, consulting, support, and training.

    Avoid a feature bake-off. Instead, focus on the requirements you identified in step 4 above, and try not to be dazzled by features that don’t deliver on your criteria. Of course, during your search process you may update your goals as you learn what’s possible. Just remember to stick to the features that will provide value to your customers – and that you can really envision yourself embedding into your application.

    Embedded Analytics

    More Than Just Pretty Pictures
    During your evaluation process, it will be easy to get lost among a dizzying array of charts and graphs. Don’t forget everything we have discussed in this guide. Ultimately, you want to bring the most value to your application, your organization, and your users.

    • Embeddability is how tightly you’ll integrate analytics into the overall user experience, the existing application security, and the application workflow.
    • Customization is your ability to white-label and control the look and feel of the application to make it your own, and tailor the functionality so every user has access to the capabilities they need.
    • Extensibility gives you the ultimate flexibility to create a unique application experience so you will stand out from the crowd, as well as the ability to future-proof your solution so you can tackle any new requirement.

    6. Complete Technical Evaluations with a Select Few

    Narrow down your list to the top two or three vendors and begin a structured evaluation process with each one. This is where you’ll define a proof of concept and establish clear-cut guidelines for what you want to accomplish within a reasonable timeframe of, say, 30 days.

    The amount of interaction you have with each vendor is based on your preference. This can range from an assisted trial, where support is generally available if you run into issues, to a true structured evaluation where you and the vendor are building a proof of concept together.

    Always implement the proof of concept in a technical environment that is as close to the production environment as possible. That means it should be connected to your data sources, integrated with your security, and be embedded into your application. If you host a SaaS application in the cloud, do not simply evaluate desktop tools or run analysis off a cleansed spreadsheet – unless that is what you expect your customers to do.

    At the end of the evaluation, present the output back to your stakeholders to get feedback and validate your direction.

    7. Talk to References

    Now it’s time to find out if your vendor can actually make customers like you successful.

    Ask your vendors for references. Solicit feedback from others in your personal and social networks. Look for references that are similar to your organization (size, industry, use case, etc.).

    Find out whether your situation is similar to theirs. Don’t just ask whether they’re happy with the vendor; really drill into the functionality the vendor has delivered, the nature of vendor support and training, the duration of implementation, and any roadblocks they’ve encountered. Examine how the vendor handled any problems or issues.

    8. Select a Vendor and Get Started

    It’s go time! Choose the vendor you feel most confident in as a partner to reach your goals. Of course, you’ll have to compare and negotiate terms and conditions, but look beyond software for the vendor who gives you the highest chance of success.

    Make sure your vendor has the resources to help you, even if you don’t need the help today. Later on, you’ll appreciate being able to test ideas and leverage best practices as your needs evolve.

    Get training for those who will be using the platform to create analytics. Create your first set of reports. Work with your vendor’s enablement and consulting teams for best practices.

    9. Monitor, Adapt, and Optimize

    There’s a lot that can be said here, given the endless possibilities that come from using embedded analytics. But for the purpose of time and space, here are a few tips for this phase of your process:

    • Invest in the training you need to be successful.
    • After three to six months, do a check-up and consider reengaging with your vendor’s services. Evaluate additional services that could take you to the next level.
    • Engage with your vendor’s community to learn and share best practices. Suggest ideas for new features while you’re at it.

    Questions to Ask During a Reference Call

    Success Criteria & Selection

    • What were the key business processes and goals you set for the embedded analytics project? How well has the system delivered on these goals?
    • Were you the decision maker responsible for purchasing this solution?
    • What made you choose the solution you selected?

    Implementation & Ramp-up

    • Tell me about your implementation – what was better than expected and where did you run into challenges?
    • How long did it take you to learn basic functions, like creating a dashboard or report?
    • How complete is the itegration between the analytics and your core application (including securtiy, white labeling, etc.)? How hard was it to set up and maintain?
    • What has your experience with training and support been like?
    • How proactive has the vendor been to make sure you are successful?


    • Have you deployed your analytics solution yet? If so, what was the reaction like from your customers and prospects?
    • Have you seen any specific benefits – like time to market, competitive differentiator, better sales demos, increase in customers, and/or increase in revenue?
    • What do you love about this platform, and what do you hate?
    • Beyond the licensing, what other costs did you incur during implementation?
    • If you were to do things over again, would you make the same vendor decision? Would you do anything else differently?
    Section Nav