Business Intelligence Tool Selection

Business intelligence tool selection

When I think back on my experience I have seen little to suggest that the choice of one vendor’s business intelligence software over another was the key driver of a successful project. More important are the cooperation between stakeholders and developers, and the expertise of the team in the software at hand. Another vital step is matching the business problem with the correct component within the generic business intelligence solution architecture. By example, budgeting and forecasting projects frequently use an analytical database because this technology makes it easy to spread revenue and costs dynamically, create complex time-based calculations, and support what-if scenarios.
When we are choosing between tools from different vendors we need to be sure that we are comparing like with like. For instance, we may love the interactivity and graphics of one vendor’s dashboard tool and be disappointed by another vendor’s report writing tool that lacks these features. This might seem like a blunt comparison but it is sometimes difficult to determine exactly what we are getting.
Vendors are keen to show their software as capable of every requirement but realistically most tools are very strong at a particular function and provide basic support for others.

Populating the business intelligence solution architecture

The technology strategy should aim to have one or more distinct tools available for each of the components in the generic business intelligence solution architecture. The optimum number of distinct tools depends on the component and the table below provides some high level guidance for each one.

Table – Software acquisition guidance for the components in the generic BI solution architecture

Tool/component	Relative cost	Distinct tools per organisation	Usage learning curve	Example application
ETL	Medium-High	1	Medium	Data integration and availability
DW RDBMS	High	1	Medium	Historical data repository and data availability
Analytical database	Low-Medium	1-2	Medium	Budgeting, profitability, business modelling, trend analysis
Data mining	Low-High	1	High	Discovering trends and patterns in data
Report Writer	Low-Med	1	Med	High quality scheduled reports with complex fixed layout
Dashboard/ Scorecards	Low-Medium	1-2	Low-Medium	High quality fixed layout with some interactive features
Interactive reporting and ad hoc analysis	Low	1-3+	Low	Business analysis
Spread sheet tool	Low	1	Low	Business process support and ad hoc analysis

Product evaluation

To make an informed decision about a product we must first understand what we are evaluating. Most BI vendor presentations will sound like they are selling an end-to-end BI platform. Dig deeper and you may find they are actually providing a front-end user tool and the consulting expertise necessary to extract your data and present it in the tool. There is nothing wrong with purchasing a product like this but at the end of the process, you will have an end user tool and some available data but you may not have an ETL, data warehouse, or analytical database solution that you can use independently of the user tool for other projects.
Many business intelligence products sold today are hybrids of one or more components in the generic business intelligence solution architecture. One example is the analytical database that comes with some rudimentary ETL capability. This is fine and can add to the ease of deployment and maintenance for a particular solution. However, in the medium term, my experience is that a mature BI capability will require a distinct product for each of the components in the generic BI solution architecture.
We can build our capability with products from a single vendor or mix and match, but each component should operate as a standalone tool capable of integration with other product sets and data sources. In short, if you are not ticking all the boxes, do not blow the entire software budget!

Business intelligence vendors

BI vendors come in two flavours: enterprise platform vendors that cover the entire spectrum of BI components, and niche product vendors that specialise in a particular area .

Enterprise BI platform vendors 

Enterprise BI vendors will provide a suite of integrated tools that cover most, if not all, of the components in the generic BI solution architecture. The potential advantages of the enterprise platform are:

• Software licence and support covers an array of products that would otherwise have to be negotiated separately
• Components should integrate together without additional custom development *
• BI specialists will have knowledge of all the components if they specialise in that vendor’s products

The potential disadvantages are:

• Cost, if you are paying for a large number of products that you do not need or do not have the expertise to use
• Quality of components within the product set may vary significantly. An enterprise platform vendor will want to tick all the boxes of the generic BI solution architecture. However, they may not be market leaders in many of the distinct categories in which they compete
• It can create a perception that there is no requirement for further tools. This makes it more difficult to justify niche tools if the product set is perceived to be the complete solution or has taken the whole budget

* Do not take seamless integration of products for granted on any enterprise platform. There has been a lot of M&A activity in the BI industry during recent years and not all tools offered as part of an integrated platform will have grown up together.

Niche product vendors

There are many niche vendors in the BI market, particularly in the area of end user tools. The focus of the vendor will be on usability and look and feel because this is what sells BI products. These products can be an excellent investment, especially those that are inexpensive.
Specialist vendors can survive in the BI market – now dominated by large technology firms – because their products often provide functionality or a particular style of interaction more effectively than the equivalent offering in enterprise solutions. Some advantages of these tools are:

• Usability of the tools is often very high. Unlike enterprise vendors, niche players generally trade power and flexibility for simplicity. They aim to fulfil the target user group’s common requirements without complicating the tool with integration or scalability features that enterprise tools must provide 
• Cost of licence, training and deployment should be significantly cheaper than the equivalent tools as part of an enterprise platform
• Innovation often comes from small players in the market. Enterprise vendors will eventually provide functionality that is popular in the market but there could be a lag of several years before they catch up

On the surface, some of the niche vendor products encapsulate the full functionality of the generic BI solution architecture. They can have their own data storage capability, data import tools, calculation and presentation layer. However, there are inevitable compromises and my experience is that no single product can satisfy all the requirements of a mature BI capability. Common weaknesses of these niche vendor tools are:

• ETL is limited, not user friendly, and with very basic functionality for troubleshooting errors
• Proprietary data model and data storage are only accessible through the user interface supplied by the vendor
• Data model does not impose constraints or allow for fine-grained updating. If the data is not clean before it is loaded, the database may not provide any feedback to suggest a problem. Loading data incrementally over time may be possible, but backdated corrections may be difficult to manage without performing a full reload. This may not be an option if the extraction process takes point-in-time snapshots of the source systems
• Documentation and online communities are limited and not very comprehensive. Detailed knowledge transfer can only occur through the vendor’s consultants

Tool selection criteria

Technology consolidation

A common concern of technology managers in medium to large organisations is the proliferation of BI and reporting tools, resulting in a maintenance and license-cost nightmare. For these good reasons, they would like to consolidate their technology infrastructure by shedding some of the apparent redundancy.
The extent to which this represents a real problem or an achievable solution depends on where the tool sits in the BI solution architecture. Put simply, the closer we get to the end user, the more likely and acceptable it is to have a diverse range of technology. Previous experience and personal preference are important contributors to whether a user group will accept a given end user tool. We cannot expect these factors to be common across all user groups in the company. 
On the other hand, ETL and RDBMS used for DW and historical data storage are prime candidates for consolidation because:

• Most specialist commercial RDBMS and ETL software is fully capable of satisfying the requirements for small to medium DW and data storage requirements. Some products do some things better than others but the choice of one technology over another is unlikely to make or break a project
• Management of ETL and RDBMS generally sits within the IT department so it should be easier to harmonise technology within a single department

Usability

Usability is most important when considering end user tools. ETL and database tools are predominately for technicians and power users who are more willing to sacrifice simplicity for power. Usability should be the number one priority for anyone designing an end user BI tool. When looking at a dashboard tool for instance, you need to consider both the usability of creating the dashboard, and the end user experience. We briefly consider some of the factors that influence usability in the following sections.

Interaction style

In common with other areas of technology, BI tools come in generations, with each new generation raising the bar on intuitive styles of interaction. If during a tool demonstration you are thinking it looks clunky, run for the hills!

Performance

The query response performance is difficult to judge without full data volumes and business rules. However, the smoothness and responsiveness of the application is possible to assess in a demonstration.

Online community and documentation

Some vendors are far better at providing current, thorough documentation than others. During one product evaluation, I opened the latest user manual to find it was referring to sample databases and operating systems that were at least 5 years out of date. The manual took me on a nostalgic – but ultimately frustrating – journey to a time long gone. Another vendor shortcut is to provide an online community. This is fine if it is open (does not require an array of passwords to enter), moderated, and supplemented by high quality vendor-produced technical articles. But without proper support, it seems to be the equivalent of telling the kids to go and play nicely amongst themselves!
Detailed technical articles and online help are not so crucial for small discrete end user products with high usability. If the product is for simple ad hoc slice and dice then quality documentation is not as important.

Cost

At the time of the dot-com bubble in the late 1990s, there were probably only a handful of vendors with enterprise reporting tools and BI capability. To a certain extent, this is still the case with database management systems and ETL, but the market for analytical databases, dashboards, and reporting tools now has dozens of good products. The competition has driven down prices so it is worth getting quotes from a number of vendors.

ETL and database tools

ETL and database management software can be expensive so the first thing a company should consider is whether they have existing licenses as part of their operational systems. The other obvious advantage to this is that you will have in-house expertise.
Even within the ETL and database sectors, there is a very significant variation to entry cost between vendors. Most tools are functionally similar in the core competencies. The main difference is that the high-end tools will provide better support for scalability, administration, auditing and automation of common tasks.
High-end enterprise tools are immediately appealing, especially if you have the budget, because they tick all the boxes. However, by using them in the early stages of a project you have immediately raised expectations of the BI program by requiring a far higher value add to balance the initial investment. This will necessarily mean you have to take on longer and riskier projects to justify the expense. These are dangerous waters for an immature BI program.
I would always recommend starting with a more commoditised toolset and taking on projects that are within the realms of their capability. Once you are confident with the process and outcome, and are testing the limits of these technologies – which may not be as soon as you think – then move to the high-end tools.

End user tools

There are large differences in licence costs between different user-facing databases and tools but less discrepancy in their functionality. The golden rule with these tools is not to spend the entire budget on a single product. However appealing a user-facing tool is, it will not suit everyone. Expect to deliver BI through a variety of tools and channels. With this in mind, you do not want your end user tools to be expensive and there is no reason why they should be.
Some vendors bundle user tools with more expensive database and ETL licences and so this is a good place to start. I have found that niche vendors often do a better job of end users tools than the enterprise vendors do, so it is worth keeping some budget in reserve even if you decide on an enterprise platform.

Product maturity 

Tools that have matured over time are generally a safer bet than new releases. This is especially the case if a major vendor has developed the tool because of a perceived gap in their product range. They will want to get the product to market as quickly as possible and will plan to build the full functionality into later releases. A new product may look enticing but the limitations may be undiscovered. The public knowledge base may also be limited with new products.
On the other hand, an established tool that has not had a new version in the last 2-3 years may be coming to the end of its development lifecycle. If no future releases are imminent, it may be worth looking at competitor products with a more modern architecture.

Experience

Once a user is familiar with a tool, they become very loyal to it, taking the product from one place to the next. This frustrates the efforts of a company trying to get uniformity but does lead to innovation. Expertise in a BI tool coupled with domain knowledge is the nirvana for productivity in BI so we should accommodate additional tools if new employees bring specific technical expertise to the company. Aim for a company standard DW and ETL, but analytical databases and end user tools should be in the realm of the business users.