Evaluating an Embedded Analytics Solution
Great! You’ve decided to invest in embedded analytics. Now what?
Picking the right solution involves thoroughly evaluating the technology, understanding the expertise offered by the vendor, and implementing a process to ensure success.
First, let’s examine the evaluation criteria that are critical to embedded analytics implementations. These include both the technical and non-technical requirements that are common to most evaluations.
- Self-Service Capabilities: These are the core capabilities you will make available to your users. These may include dashboards and reports as well as the interactive and analytical functions they can perform.
- Data Environment: While the solution you choose will connect to your current data environment and meet your data security needs, it should also be flexible enough to meet future demands as your data tier evolves.
- Embeddability, Customization, and Integration: One of the major ways embedded analytics initiatives differ from standalone analytics projects is the need to integrate with the application environment. This means providing a look and feel aligned to your brand as well as the extensibility to meet evolving business requirements.
- Development and Deployment: Since time-to-value is so critical to the success of the project, having a development environment where you can create, style, embed, deploy, and iterate on embedded analytics will enable your team to deliver the functionality the business demands.
- Licensing, Services, and Company Expertise: Choosing the right partner is not simply about the technology; it’s also about finding the level of expertise you require for training, support, and services, as well as agreeing on the business terms that ensure shared success.
After detailing the common evaluation criteria in each of these categories, we will walk through the entire process for a successful evaluation.
|The Spectrum of Self-Service Personas||Increase the adoption of embedded analytics by providing a broad range of users with a tailored experience that matches their needs and skills. Users will typically fall into one or more self-service personas.||
|Presentation and Information Delivery||Empower everyone to quickly draw conclusions, monitor key performance indicators, and obtain a complete view of the business through visualization.||
|Provide an optimal user experience regardless of where and how users prefer to access information. Evaluate the compatibility of solutions across different devices and formats.||
|Interactivity and Automation||Create an engaging experience where users can explore the data and interact with the information presented.||
|Grow user adoption by embedding analytics into everyday work.||
|Analysis and Authoring||Empower users by giving them greater flexibility in their analysis and the ability to create and format the desired content on their own.||
|Extend the value of the data in your application by providing deeper insights into business trends.||
|Data Sources||Leverage native connectivity optimized for the data source. Ideally, your primary data source should belong in this group.||
|Benefit from data flexibility with generic connectors when a vendor-specific connector is not available.||
|Enjoy ultimate data source flexibility through APIs or plug-ins to connect to uncommon or proprietary data sources.||Data APIs and Plug-Ins coded in your language of choice provide customized data access.|
|Data Management||Balance your needs for real-time reporting and interactive self-service analysis with a solution that enables you to connect directly to underlying data sources and cache data from transactional systems.||
|Create a complete view of the data that users can easily work with by preparing the data for analysis.||
|Create an efficient user experience where users can immediately take action on what they see in any visualization or report.||Bi-Directional Data Flow: Data can flow back to source systems based on user input, such as for data updates and workflow processes.|
|Transform your application into a vital hub of information by incorporating data from external sources into a single consolidated view.||External Data: This could be in the form of third-party industry benchmarks, data feeds (such as weather and social media), and customer data from their specific data stores.|
|Give your application ultimate flexibility to present information by consuming data services from the analytics solution.||Data Services: The analytic solution becomes a provider of data in addition to analytic functionality. These data services produce output to be used by jQuery components, third-party charting, and other application functions.|
Embeddability, Customization, and Integration
|Security||It should be easy to adopt the security from your application to the analytics content. Scrutinize vendors on the flexibility of their security models, implementation of single sign-on, and whether data needs to synchronized or replicated between applications.||
|Multi-Tenancy||Accelerate development with a solution that has built-in multi-tenancy support, which allows you to create a report once and deploy for multiple customers.||Multi-Tenancy: A single application has the ability to access the data for multiple customers, whether data is stored in the same database and/or in individual databases per customer. Look for parameterization and tokenization capabilities that do not require data replication or advanced data modeling to support multi-tenancy.|
|User Experience||Create a truly analytic application experience by embedding analytics as a natural part of your application.||
|Workflow||Create an efficient user experience where users can immediately take action on what they see in any visualization or report.||Workflow Processes: Users can initiate API calls to your application from a report or dashboard in order to perform data operations or process transactions at the moment they see the data. For example, a user could select a region of a chart and perform an action on the selected records without having to leave the visualization.|
|Extensibility||Competing on analytics often means delivering unique functionality. Ensure you’ll be able to meet any future requirement with a solution that can be extended utilizing open standards approaches.||
Development and Deployment
|Development||Empower your development team with the tools to quickly create and iterate on embedded analytics capabilities.||
|Deployment||Quickly deploy and scale an implementation that is aligned with your current technology stack. Be assured that you have the flexibility to shift as your technical environment evolves.||
Licensing, Services, and Company Expertise
|Licensing||Software licensing terms should align the vendor with the value you provide to your customers.||Licensing: Terms of the license can depend on a variety of factors, including number of users/customers, servers, usage, and whether you are embedding into a commercial product. Be sure the terms make business sense over the short and long run.|
|Services||Completing your project on time and in the right way can require resources outside your team – take comfort from a full range of services options even if you do not employ them.||
|Customer Success||Vendors should supply a process that maps your path to success and provide a wide array of resources to address any issues along the way.||
|Expertise||Leverage the vendor’s experience and commitment to making you and others like you successful.||
The Evaluation Process
Now that we’ve established the criteria for evaluating vendors for embedded analytics, let’s look at the overall process that will help you make the best decision for your business.
1. Determine Your Goals
To get where you want to go, write it down. Statistically speaking, you increase your likelihood for success simply by putting your goals on paper.
Draw from the strategic benefits we discussed earlier in Chapter 2 (The Business Case for Embedded Analytics).
- Quantifiable metrics may include increasing revenue, increasing user adoption, or improving customer retention.
- Soft metrics may include an improving the user experience, creating a competitive differentiator, or increasing customer satisfaction.
2. Establish the Timeline
Identify the steps you’ll take to reach your goals. Ask yourself, “When do I want to…”
- Begin the selection process?
- Have detailed vendor presentations and demos?
- Finish a proof of concept?
- Make my final decision?
- Start development?
- Release product?
3. Assemble the Team
Determine the stakeholders who need to be involved. Who is going to care about embedded analytics internally (your executive team, product management, lead developers) and externally (your key customers, customer advisory board)? Build the business case collectively to secure buy-in to move forward.
4. Identify Requirements
Review your technical and non-technical requirements, using the previous pages as a guide to rank and weight the importance of these requirements. Research your competitors and talk to your customers to develop a firm understanding of the capabilities you want to add to your application.
Take the functional scenarios that describe how end users will work with embedded analytics inside your application and what they will be able to accomplish, and turn those into technical requirements.
Consider who will use the third-party products internally. Understand their skill sets and identify any potential resource gaps as you move into the evaluation phase.
5. Research Potential Vendors
Assign a point person to research potential vendors and evaluate whether their functionality matches your requirements. Utilize independent industry resources, such as the Gartner Magic Quadrant for Business Intelligence and Analytics Platforms report, to create your initial list. Pay special attention to vendors who specialize in the OEM market for software providers.
Attend product demonstrations by each vendor to confirm a basic fit. Discuss your requirements and ask each one to demonstrate how they would deliver your specific processes and scenarios. Ask tough questions and make sure the vendors show you the functionality they promise. Confirm ballpark pricing to move forward.
Evaluate each vendor’s ability to make you successful during the implementation process through access to best practices, community, consulting, support, and training.
Avoid a feature bake-off. Instead, focus on the requirements you identified in step 4 above, and try not to be dazzled by features that don’t deliver on your criteria. Of course, during your search process you may update your goals as you learn what’s possible. Just remember to stick to the features that will provide value to your customers – and that you can really envision yourself embedding into your application.
More Than Just Pretty Pictures
During your evaluation process, it will be easy to get lost among a dizzying array of charts and graphs. Don’t forget everything we have discussed in this guide. Ultimately, you want to bring the most value to your application, your organization, and your users.
- Embeddability is how tightly you’ll integrate analytics into the overall user experience, the existing application security, and the application workflow.
- Customization is your ability to white-label and control the look and feel of the application to make it your own, and tailor the functionality so every user has access to the capabilities they need.
- Extensibility gives you the ultimate flexibility to create a unique application experience so you will stand out from the crowd, as well as the ability to future-proof your solution so you can tackle any new requirement.
6. Complete Technical Evaluations with a Select Few
Narrow down your list to the top two or three vendors and begin a structured evaluation process with each one. This is where you’ll define a proof of concept and establish clear-cut guidelines for what you want to accomplish within a reasonable timeframe of, say, 30 days.
The amount of interaction you have with each vendor is based on your preference. This can range from an assisted trial, where support is generally available if you run into issues, to a true structured evaluation where you and the vendor are building a proof of concept together.
Always implement the proof of concept in a technical environment that is as close to the production environment as possible. That means it should be connected to your data sources, integrated with your security, and be embedded into your application. If you host a SaaS application in the cloud, do not simply evaluate desktop tools or run analysis off a cleansed spreadsheet – unless that is what you expect your customers to do.
At the end of the evaluation, present the output back to your stakeholders to get feedback and validate your direction.
7. Talk to References
Now it’s time to find out if your vendor can actually make customers like you successful.
Ask your vendors for references. Solicit feedback from others in your personal and social networks. Look for references that are similar to your organization (size, industry, use case, etc.).
Find out whether your situation is similar to theirs. Don’t just ask whether they’re happy with the vendor; really drill into the functionality the vendor has delivered, the nature of vendor support and training, the duration of implementation, and any roadblocks they’ve encountered. Examine how the vendor handled any problems or issues.
8. Select a Vendor and Get Started
It’s go time! Choose the vendor you feel most confident in as a partner to reach your goals. Of course, you’ll have to compare and negotiate terms and conditions, but look beyond software for the vendor who gives you the highest chance of success.
Make sure your vendor has the resources to help you, even if you don’t need the help today. Later on, you’ll appreciate being able to test ideas and leverage best practices as your needs evolve.
Get training for those who will be using the platform to create analytics. Create your first set of reports. Work with your vendor’s enablement and consulting teams for best practices.
9. Monitor, Adapt, and Optimize
There’s a lot that can be said here, given the endless possibilities that come from using embedded analytics. But for the purpose of time and space, here are a few tips for this phase of your process:
- Invest in the training you need to be successful.
- After three to six months, do a check-up and consider reengaging with your vendor’s services. Evaluate additional services that could take you to the next level.
- Engage with your vendor’s community to learn and share best practices. Suggest ideas for new features while you’re at it.
Questions to Ask During a Reference Call
Success Criteria & Selection
- What were the key business processes and goals you set for the embedded analytics project? How well has the system delivered on these goals?
- Were you the decision maker responsible for purchasing this solution?
- What made you choose the solution you selected?
Implementation & Ramp-up
- Tell me about your implementation – what was better than expected and where did you run into challenges?
- How long did it take you to learn basic functions, like creating a dashboard or report?
- How complete is the itegration between the analytics and your core application (including securtiy, white labeling, etc.)? How hard was it to set up and maintain?
- What has your experience with training and support been like?
- How proactive has the vendor been to make sure you are successful?
- Have you deployed your analytics solution yet? If so, what was the reaction like from your customers and prospects?
- Have you seen any specific benefits – like time to market, competitive differentiator, better sales demos, increase in customers, and/or increase in revenue?
- What do you love about this platform, and what do you hate?
- Beyond the licensing, what other costs did you incur during implementation?
- If you were to do things over again, would you make the same vendor decision? Would you do anything else differently?