Business Intelligence Project Management

Introduction to business intelligence project management

Business intelligence projects should be short and focused. Business intelligence tools exist to support a particular type of business problem or interaction. The challenge for a business intelligence project is to combine the business problem, BI tools, and data without the normal overhead associated with bespoke application development. With this in mind, the focus of this article is to provide guidelines for individual project goals, duration, and participants, rather than a discussion of formal project management methodologies. Each business intelligence project should contribute to all three pillars of the BI strategy by:

• Improving a business process and raising awareness of the BI program
• Increasing the availability of data in the organisation through ETL, DW, or simply raising visibility of the data
• Demonstrating to users the potential of business intelligence technology by supporting the business process with far less development effort than would have been required using regular software development

What do we cover in this article?

We start the article with a look at the recommended project duration and the breakdown of project time between process, data, and technology objectives. We look at similar considerations when we discuss business intelligence capability. Next, we look at the key participants in a business intelligence project and relate this back to the business intelligence cocktail. Finally, I provide guidance on setting project goals so that we capture the full value of each discrete project. As with previous articles, my intention is to discuss common issues, not provide a detailed implementation guide. If you execute short focused projects, you should be able to keep your formal project management overhead to a minimum.

Business intelligence project duration

In the first month of the business intelligence program, we should run a number of discovery projects to introduce the three pillars of the business intelligence strategy to the organisation and identify target business processes, data, and technology. Preferably, we should limit the scope of these discovery exercises to a single week’s effort and duration. We could spend an infinite time on any one of these tasks but we want to gain just enough information to start a concrete project.

Recommended project duration (weeks) and focus in the first year of the BI program

The Figure above shows an example of how you might structure BI projects in the first year of implementing your BI strategy. As discussed, we can use the first month to initiate each of the strategies. After the initial discovery projects, every full lifecycle BI project should provide some value to process, data, and technology strategies. At a minimum, each discrete BI project should:

• identify candidate business processes and whet the appetite of the business community for further process discovery
• improve availability of data by discovering, working with, and publishing data sources either as a virtual data dictionary or using physical data integration
• test the capabilities of the BI technology infrastructure and improve understanding of its use

Based on my experience I have shown a rough indication of the proportion of time that each of the strategies should consume within a project. The chart shows that data analysis and ETL – data strategy – typically consume much of the effort in the first few projects. However, as the data strategy matures we can tackle increasingly complex business problems and therefore the business process analysis takes an increased share of project time. 

Business intelligence project focus

It may be tempting to devote entire projects to integrating and publishing data. This common mistake often stems from not managing business expectations in the initial stages. The business community may give the feedback that they are not interested in looking at the data until sources x, y and z, are fully integrated. This is an early sign of a BI project in troubled waters. The focus is not on the business goal but the data, and the user community are not showing active engagement in all stages of the project.
The sooner we can publish data to the users; the sooner we can introduce BI tools and receive feedback about the data quality. Each of the early projects should devote at least 20% (~1 week) of the time to deployment of ad hoc tools, dashboards and focused delivery of information.
If the user community want to engage, they will help overcome the inevitable data integrity and availability challenges. If they do not, then better to raise the red flag on the project at this early stage and move to another business function until conditions change.     

Business intelligence project team

When we described the ingredients of the business intelligence cocktail, two key participants emerged; the subject matter expert, and the BI specialist. In the first few iterations of your BI strategy, you will definitely benefit from engaging BI specialists that understand the BI lifecycle and have experience in the selected toolset. They should work with the stakeholders, power users, and technology department to build an in-house capability. With each BI project, the organisation should take increasing ownership of the activities for eliciting requirements, development, and support.
BI projects should be short, especially in the first year, so project management should be lightweight. The most important project management steps are to determine the feasibility of the project and retrospectively analyse the outcome and lessons learned. A professional project manager may assume responsibility for the BI program, but if they are not available then the project sponsor, subject matter expert, or BI specialist can complete the management tasks for each individual project. 

Team interaction

My preference is for direct interaction between the subject matter expert and the BI specialist. I also recommended that the subject matter expert is the person responsible for the business process or decision. In reality, many large organisations have a structure similar to the one below.

Standard communication model in large corporations

In this model, the process owner and the BI specialist have no direct contact. The process owner communicates with a business analyst and the business analyst communicates with the BI specialist. The business analyst is normally an expert in software lifecycle and design methodologies and understands the business processes. For this reason, the structure above works well for traditional software development where requirements pass through stages of conceptual design, technical design, and finally implementation.
BI projects have a different emphasis to traditional software projects because we are looking for opportunities to apply new technology to existing problems. In short, requirements take shape with direct reference to the capabilities of the tools. If we assume that the process owner and the business analyst are not experts in the BI technology, then we must also accept that they cannot generate BI requirements as effectively as they could if guided by the BI specialist.

Recommended communication model for BI projects

In my experience, a face-to-face discussion with the business process owner is the quickest way to determine where BI can add value to their process. If a business analyst is available then this discussion will be even more effective because of their ability to unravel business jargon, clarify ambiguity and provide context for the technology.
As the business intelligence technology strategy matures, the business analyst will probably become expert in one or more BI tools. When this occurs then the business analyst can assume the role of the BI specialist, developing the BI solutions themselves in coordination with the subject matter expert. This is the ideal scenario for a BI strategy. The further we can extend the knowledge of the tools into the business, the more likely we will identify opportunities for innovation and process improvement.
When a new challenge arises that the business analyst cannot confidently resolve with the familiar toolset, this is a good time to re-engage a BI professional to evaluate whether other BI tools would provide a better fit.

Business intelligence project goals 

We opened the article by stating that each BI project should contribute to all three pillars of the BI strategy. If we judge the success of a project within very narrow parameters, we increase the chance of a negative outcome. Conversely, if we aim to achieve a broad range of objectives that provide value for the process, data, and technology strategy it is very likely that the project will meet at least some of these goals within the project budget and timelines. This might seem like semantics, but in the early stages of implementing the BI strategy, perception is important to gaining sponsorship for future projects. No one benefits if sponsors cancel the entire program just weeks before we expect to deliver measurable business value, due to a negative perception of current progress.
It is a shame to hear of DW or BI projects deemed failures because they did not meet all the expectations of the business. When you look at the project in more detail, you find a lot of good DW and ETL work ready for use in future projects. The obvious, but nonetheless difficult objective is to manage expectations for the project from the beginning. The business will be primarily concerned with the process improvement objectives, but it is worth emphasising the project’s contribution towards the data and technology pillars because this work will lay the foundations for tackling more complex business problems in future projects.
A straightforward way of setting project goals is to align each component in the business intelligence solution architecture with an objective of the three pillars of the business intelligence strategy. The following table provides some examples of this approach.

The focus of a project may be too narrow if it only includes a couple of activities in the table. We can normally add extra elements to the project with little additional effort. For instance, a project to create a daily extract from a source system may fail to deliver business value if there are unanticipated issues with the data. However, if we use the project to train new users in the ETL technology and create some exception reports with a new tool, then we have increased our capability to execute subsequent projects.

Data volumes and project goals

Response time is a key factor in user acceptance of a BI application. We have all cancelled a webpage request because we had to wait more than a few seconds for it to load. BI tools will suffer the same fate if they respond slowly to users requests. The need for speed guides the volume of data we should target. BI applications often use a large volume of historical data to provide trends and historical analysis. However, in early iterations of the BI program we should not be too ambitious.
On the one hand, we need to develop the BI application with the anticipated data volume to gauge the performance of the user tools. On the other, very large volumes of data add time and complexity to almost every aspect of the BI development process. We want to spend early BI projects adding business value and improving our process and capability with the tools, not wrestling with performance problems.
It can be difficult to judge what constitutes a challenging volume of data in advance of the project. It will depend on the tools, server infrastructure, and processing requirements. However, if more than 5% of the project time is spent performance tuning then this is an early warning sign to reassess the project objectives or design. In my experience, it is sometimes preferable to negotiate a compromise on data volumes part way through the project, rather than let performance tuning overtake the original objectives of supporting the business process.

Always remember that BI solution development should be fast and effective. When we apply BI technology to the right problem this is invariably the case. If it is not, we should redirect our effort to quicker wins and leave intractable problems until later in the program. At this later date, we will have increased our capability in the toolset, and may have redefined our approach to supporting the business objective based on project experience.