BI

International JournalAvenue, of Enterprise Systems, 4(3), 79-94, July-September 2008 79 701 E. Chocolate Suite Information 200, Hershey PA 17033-1240, USA Tel: 717/533-8845; Fax 717/533-8661; URL-http://www.igi-global.com
This paper appears in the publication, International Journal of Enterprise Information Systems, Volume 4, Issue 3 edited by Angappa Gunasekaran © 2008, IGI Global

IGI PUBLISHING

ITJ4344

Managing the Implementation of Business Intelligence Systems:
A Critical Success Factors Framework
William Yeoh, University of South Australia, Australia Andy Koronios, University of South Australia, Australia Jing Gao, University of South Australia, Australia

ABsTRAcT
The implementation of a BI system is a complex undertaking requiring considerable resources. Yet there is a limited authoritative set of CSFs for management reference. This article represents a first step of filling in the research gap. The authors utilized the Delphi method to conduct three rounds of studies with 15 BI system experts in the domain of engineering asset management organizations. The study develops a CSFs framework that consists of seven factors and associated contextual elements crucial for BI systems implementation. The CSFs are committed management support and sponsorship, business user-oriented change management, clear business vision and well-established case, business-driven methodology and project management, business-centric championship and balanced project team composition, strategic and extensible technical framework, and sustainable data quality and governance framework. This CSFs framework allows BI stakeholders to holistically understand the critical factors that influence implementation success of BI systems. Keywords: business intelligence (BI) System; critical success factors (CSFs); Delphi method; framework

BACKGROUND

Engineering asset management organizations (EAMOs), such as utilities and transportation enterprises, store vast amounts of asset-oriented data (Lin et al., 2007). However, the data and information environments in these organizations are typically fragmented and characterized by disparate operational, transactional and legacy systems spread across multiple platforms and

diverse structures (Haider & Koronios, 2003). An ever-increasing amount of such data is often collected for immediate use in assessing the operational health of an asset, and then it is either archived or deleted. This lack of vertical integration of information systems, together with the pools of data spread across the enterprise, make it extremely difficult for management to facilitate better learning and

Copyright © 2008, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

80 International Journal of Enterprise Information Systems, 4(3), 79-94, July-September 2008

make well-informed decisions thus resulting in suboptimal management performance. Yet large volumes of disperse transactional data lead to increased difficulties in analyzing, summarizing and extracting reliable information (Ponniah, 2001). Meanwhile, increased regulatory compliance and governance requirements have demanded greater accountability for decision making within such organizations (Logan & Buytendijk, 2003; Mathew, 2003). In response to these problems, many EAMOs are compelled to improve their business execution and management decision support through the implementation of a BI system. According to Negash (2004), “BI systems combine data gathering, data storage, and knowledge management with analytical tools to present complex and competitive information to planners and decision makers.” Implicit in this definition, the primary objective of BI systems is to improve the timeliness and quality of the input to the decision making process (Negash, 2004). Data is treated as a corporate resource, and transformed from quantity to quality (Gangadharan & Swami, 2004). Hence, actionable information could be delivered at the right time, at the right location, and in the right form (Negash, 2004) to assist individual decision makers, groups, departments, divisions or even larger units (Jagielska et al., 2003). Fisher et al. (2006) further posited that a BI system is primarily composed of a set of three complementary data management technologies, namely data warehousing, online analytical processing (OLAP), and data mining tools. A successful implementation1 of BI system provides these organizations with a new and unified insight across its entire engineering asset management functions. The resulting unified layer, in reporting, business analysis, and forecasting assures consistency and flexibility (Gangadharan & Swami, 2004). Critical information from many different sources of an asset management enterprise can be integrated into a coherent body for strategic planning and effective allocation of assets and resources. Hence, the various business functions and activities are analyzed collectively to generate

more comprehensive information in support of management’s decision-making process. BI systems come as standardized software packages from such vendors as Business Objects, Cognos, SAS Institute, Microstrategy, Oracle, Microsoft and Actuate, and they allow customers to adapt them to their specific requirements. In recent years, the BI market has experienced extremely high growth as vendors continue to report substantial profits (Gartner, 2006a; IDC, 2007). Forrester’s recent survey indicated that for most CIOs, BI was the most important application to be purchased (Brunelli, 2006). The results of the latest Merrill Lynch survey into CIO spending similarly found that the area with the top spending priority was BI (White, 2006). These findings are echoed by Gartner’s CIOs priorities surveys in 2006 which revealed that BI ranked highest in technology priority (Gartner, 2006b). In the most recent survey of 1400 CIOs, Gartner likewise found that BI leads the list of the top ten technology priorities (Gartner, 2007).

INTRODUCTION AND RESEARCH MOTIVATION

While BI market appears vibrant, nevertheless the implementation of a BI system is a financially large and complex undertaking (Watson et al., 2004). The implementation of an enterprisewide information system (such as a BI system) is a major event and is likely to cause organizational perturbations (Ang & Teo, 2000). This is even more so in the case of a BI system because the implementation of a BI system is significantly different from a traditional operational system. It is an infrastructure project, which is defined as a set of shared, tangible IT resources that provide a foundation to enable present and future business applications (Duncan, 1995). It entails a complex array of software and hardware components with highly specialized capabilities (Watson & Haley, 1998). BI project team need to address issues foreign to the operational systems implementation, including cross-functional needs, poor data quality derived from source systems that can often go unnoticed until cross-systems

Copyright © 2008, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

International Journal of Enterprise Information Systems, 4(3), 79-94, July-September 2008 81

analysis is conducted; technical complexities such as multidimensional data modeling; organizational politics, and broader enterprise integration and consistency challenges (Shin, 2003). Consequently, it requires considerable resources and involves various stakeholders over several months to initially develop and possibly years to become fully enterprise-wide (Watson & Haley, 1997). Typical expenditure on these systems, includes all BI infrastructure, packaged software, licenses, training and entire implementation costs, may demand a sevendigit expenditure (Watson & Haley, 1997). The complexity of BI systems is exemplified by Gartner’s recent study that predicted more than half of systems that had been implemented will be facing only limited acceptance (Friedman, 2005). Much IS literature suggests that various factors play pivotal roles in the implementation of an information system. However, despite the increasing interest in, and importance of, BI systems, there has been little empirical research about the critical success factors (CSFs) impacting the implementation of such systems. The gap in the literature is reflected in the low level of contributions to international conferences and journals. Although there has been a plethora of BI system studies from the IT industry, nonetheless, most rely on anecdotal reports or quotations based on hearsay (Jagielska et al., 2003). This is because the study of BI systems is a relatively new area that has primarily been driven by the IT industry and vendors, and thus there is limited rigorous and systematic research into identifying the CSFs of BI system implementation. Therefore, the increased rate of adoption of BI systems, the complexities of implementing a BI system, and their far-reaching business implications justify a more focused look at the distinctive CSFs required for implementing BI systems.

• •

explore and identify the CSFs, and their associated contextual elements that influence implementation of BI systems consolidate a CSFs framework for BI system implementation

Research Objective

Essentially, the authors argue that there is a set of factors influencing the implementation of BI systems and such antecedents (i.e., CSFs) are necessary. In alignment with Sum et al.’s (1997) argument, this research also recognizes that the associated contextual elements that make up each factor provide more specific, useful and meaningful guidelines for BI systems implementation. As asserted by Sum et al. (1997), Top management support has often been cited as a CSF, but what exactly constitutes top management support is not really known. Good performance of the CSFs requires that their elements (or constituents) be known so that management can formulate appropriate policies and strategies to ensure that the elements are constantly and carefully being managed and monitored. Lack of clear definitions of the CSFs may result in misdirected efforts and resources. Furthermore, the CSFs identified can be consolidated into a framework to provide a comprehensive picture for BI stakeholders, and hence allowing them to optimize their resources and efforts on those critical factors that are most likely to have an impact on the system implementation. Thereby ensuring that the initiatives result in optimal business benefits as well as maintaining effective uptake. The remainder of this article has been structured as follows. The following section describes the research methodology, before elaborating on the CSFs finding. The next section then presents the CSFs framework and detail of each CSF. In the last section the authors state the conclusion, research contribution and future study.

Given the background and motivation of this research, the authors used Delphi method to:

RESEARCH METHODOLOGY

In the absence of much useful literature on BI system, this study seeks to explore and identify a set of CSFs that are jointly agreed by a group

Copyright © 2008, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

82 International Journal of Enterprise Information Systems, 4(3), 79-94, July-September 2008

of BI system experts who possess substantial experience in EAMOs. The Delphi method was deemed to be the most appropriate method for this study because it allows the gathering of subjective judgments which are moderated through group consensus (Linstone & Turoff, 1975; 2002; Helmer, 1977). Moreover, this research assumes that expert opinion can be of significant value in situations where knowledge or theory is incomplete, as in the case of BI systems implementation in EAMOs (Linstone & Turoff, 2002). Unlike focus group method, this Delphi method is particularly suitable for this research situation where personal contact among participants and thus possible dominance of opinion-leaders is not desirable because of concerns about the difficulty of ensuring democratic participation. For this study, a Delphi panel composed of fifteen BI systems experts in EAMOs was established. Ziglio (1996) asserts that useful results can be obtained from small group of 10-15 experts. Beyond this number, further increases in understandings are small and not worth the cost or the time spent in additional interviewing (Carson et al., 2001). Thus, the size of such a Delphi panel is deemed suitably representative. As shown in Table 1, the Delphi participants have all been substantially involved in the implementation of BI systems within EAMOs in Australia and the United States. In addition, the range of engineering asset management organizations represented by these experts was diverse and included public utilities (such as electricity, gas, water, and waste management) and infrastructure-intensive enterprises such as telecommunications and rail companies. It should be noted that some of the large organizations in which the participants have been involved have implemented BI projects in a series of phases. Most of the EAMOs are very large companies with engineering assets worth hundreds millions of dollars and have committed immense expenditure to BI projects. So the expertise of the Delphi participants represents ‘state of the art’ knowledge of BI systems implementation in a broad range of engineering asset-intensive industries.

The Delphi study comprised three rounds. During the first round the authors conducted face-to-face interviews with each participant (and phone interviews in some cases due to geographical constraints), and these varied in duration from one to one and half hours. Rather than having an open-ended question, the authors adopted a different approach from traditional Delphi methods by beginning with a list of factors derived from data warehousing literature, which is the core component of a BI system. Having a prior theory has advantages such as allowing the opening and probe questions to be more direct and effective, and helping the researcher recognize when something important has been said (Carson et al., 2001). However, the existing literature is not comprehensive in regard to CSFs for an entire BI system, but mainly focuses on data warehousing. Therefore, those factors were mainly used to start each discussion. When the mention of particular factors elicited relevant responses then further probing questions would follow in order to gather more details on those factors. The panelists were indeed encouraged to suggest other factors that they deemed critical. At the commencement of the interviews, it was explained that the study focused on CSFs that facilitated the implementation success of BI systems in terms of infrastructure performance and process performance. The infrastructure performance consists of three major IS success dimensions proposed by Delone and McLean (1992; 2003), namely system quality, information quality, and system use, whereas process performance is composed of meeting time-schedule and budgetary constraints (Ariyachandra & Watson, 2006). After the interview, further clarifications (if any) were made by follow-up phone calls and e-mail communications. Subsequently, the data gathered from the first round of interviews were analyzed thoroughly by content analysis technique, a constant comparison (‘grounded’) technique, to identify major themes (Glaser & Strauss, 1967). This technique encourages the emergence of a finding from the data set by constantly comparing incidents of codes with each other and then abstracting related codes to a higher conceptual level (Glaser, 1992; 1998). In

Copyright © 2008, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

International Journal of Enterprise Information Systems, 4(3), 79-94, July-September 2008 83

Table 1. Delphi participants and������������������������������������� their BI systems experience in EAMOs
Current Position Principal consultant, Committee, Author, Speaker Principal consultant, Committee Principal consultant, Author, Speaker Principal consultant, Committee Principal consultant, Author, Speaker Principal consultant Organization Type BI Consultancy, TDWI Committee BI Consultancy, DWAA Committee BI Consultancy, TDWI Summit BI Consultancy, DWAA Committee BI Consultancy, TDWI Summit BI Consultancy BI System Business Objects, Information Builder, Cognos, Oracle Cognos, Business Objects, Actuate Cognos, Business Objects, Hyperion, Oracle, SAS Actuate, Microstrategy, Business Objects Hyperion, Informatica, Oracle, Actuate, Business Objects Business Objects, Cognos, Oracle SAS, Business Objects, Cognos, Microsoft, Oracle, Informatica Oracle, IBM, Hyperion, Informatica, Cognos, Microsoft Hyperion, Informatica, Oracle Oracle, Business objects Informatica, Oracle, Hyperion Cognos, SPF Plus Business Objects, SAS, Oracle Oracle, Business Objects, Hyperion Microstrategy Oracle, IBM EAMOs’ Industry Sector Electricity, gas, water & waste utilities, oil & gas production, defense, public transportation Telecommunications, airlines, municipal utility Energy utilities, transportation, mining industries Transportation & municipal utility, logistics Electricity, gas, water utilities, telecommunications Electricity, gas, water & waste utilities Rail infrastructure and fleets, public transportation, mining industries Telecommunications, electricity, gas, water utilities, Utilities, telecommunications, public transportation Energy utilities, logistic transportation company Rail infrastructure and fleets Energy utilities Utilities & logistics Utilities, telecommunications & manufacturing Municipal utilities

Principal consultant

BI Consultancy

Principal consultant Executive VP (global consulting), Speaker Principal consultant Principal consultant Principal consultant Principal consultant Academic, Consultant, Author, Speaker Principal consultant

BI Consultancy BI Consultancy, Conferences BI Consultancy BI Consultancy BI Consultancy BI Consultancy Academia, BI Consultancy BI Consultancy

other words, the qualitative data were examined thematically and emergent themes were ranked by their frequency and later categorized. The objective of the present research was to identify the CSFs that influence the implementation of BI systems. Hence, it is considered to be very important to determine what emerges from the

data regarding interpretations of the CSFs for implementing BI systems. In the subsequent round, the suggested factors of all the participants were consolidated into a single list. The list was then distributed among the participants to facilitate comparison of the expert’s perceptual differences. However, none

Copyright © 2008, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

84 International Journal of Enterprise Information Systems, 4(3), 79-94, July-September 2008

of them nominated any additional factors of their own. Also, based on feedback from participants, some further minor changes were incorporated. In addition, the participants confirmed that the classification of factors and their associated contextual elements is appropriate. For instance, several elements are grouped together because of the closed interrelationship. During the third round, the list of candidate CSFs was surveyed by the Delphi participants using a structured questionnaire survey approach. Specifically, a 5-point Likert scale was applied to rate the importance of the candidate CSFs in the process of seeking statistical consensus from the BI experts. The purpose of using a 5-point scale from 1 to 5 (where 1 meant ‘not important,’ 2 of ‘little importance,’ 3 ‘important,’ 4 ‘very important,’ to 5 ‘critically important’) was to distinguish important factors from critical success factors. From the survey feedback, only those factors with average rating of 3.5 and above were shortlisted as CSFs (as shown in Table 2). These CSFs ratings are considered legitimate because the participants were directly drawing on their hands-on experience in EAMOs’ BI system implementations. The details of the results are discussed below.

CSFS FINDING AND DISCUSSION

Table 2 depicts the average rating results for the respective CSFs in descending order of importance. It contains the consensus outcomes and

shows that the Delphi study captured the importance of the seven critical factors, namely committed management support and sponsorship, business user-oriented change management, clear business vision and well-established case, business-driven methodology and project management, business-centric championship and balanced project team composition, strategic and extensible technical framework, sustainable data quality and governance framework. Notably, data and technical-related factors did not appear to be the most critical in relation to other organizational factors. According to most interviewees, technological difficulties can be solved by technical solutions. However, it was found that achieving management and organizational commitment for a BI initiative poses the greatest challenge, because the BI teams considered them to be outside their direct control. The organizational support is reflected in the attitudes of the various business stakeholders; that is, their attitudes to change, time, cost, technology, and project scope. Based on a large-scale survey result, Watson and Haley (1997) pointed out that the most critical factors for successful implementations were organizational in nature. Committed management support and adequate resources were found to determine the implementation success, because these factors worked to overcome socio-political resistance, address change-management issues, and increase organizational buy-in. This finding was also converging with Gartner’s recent observation that “overcoming complex

Table 2. Ratings of critical success factors by Delphi participants
Critical Success Factors • Committed management support and sponsorship • Business user-oriented change management • Clear business vision and well-established case • Business-driven methodology and project management • Business-centric championship and balanced project team composition • Strategic and extensible technical framework • Sustainable data quality and governance framework Mean 4.16 4.10 4.09 4.08 3.94 3.90 3.82 Std. Dev 0.99 1.00 0.90 0.88 0.89 0.89 0.91

Copyright © 2008, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

International Journal of Enterprise Information Systems, 4(3), 79-94, July-September 2008 85

organizational dynamics will become the most significant challenge to the success of business intelligence initiatives and implementations” (Burton et al., 2006). In fact, the effort of implementing BI systems is highly regarded by the Delphi participants as a business-driven program as opposed to a technological one. The fulcrum of BI program success is thus dependent on the business personnel, whereas technical people are expected to support the analytical requirements via technologies and tools. The definition of strategic BI framework, project scoping and data quality initiatives were considered within the realm of business personnel. That is, this new understanding emphasizes the priority of business aspects, not the technological ones, in implementing BI systems. While the specific CSFs may seem to vary slightly between BI systems and general IS studies, the actual contextual elements of these CSFs are substantially different from the implementation effort required for conventional operational systems. Unlike those transactional systems, business stakeholders need to be involved interactively in order to meet their dynamic reporting and ever-changing analytical needs. Owing to the evolutionary information requirements, the BI team has to provide continual support not only on tools application, but also at broader data modeling and system scalability issues. This is in line with the adoption of an incremental delivery approach for implementing an adaptive decision support system, such as a BI system (Arnott & Pervan, 2005). Moreover, organizational and business commitment to a BI system implementation is critical to solve the complex organizational issues, especially in the democratization process of data ownership, selection of funding model, change of business process, definition of the scoping study, data stewardship and quality control, and the provision of domain expertise and championship. The following section presents the CSFs framework consolidated from these CSFs findings.

DEVELOPMENT OF A CRITICAL SUCCESS FACTORS FRAMEWORK

Based on the research finding, these seven critical factors were integrated with the implementation success measures to provide a comprehensive CSFs framework for implementing BI systems. As illustrated in Figure 1 below, this CSF framework outlines how a set of factors contribute to the success of a BI system implementation. It postulates that there is a set of CSFs influencing the implementation success that takes into account two key measures: infrastructure performance and process performance. The infrastructure performance has parallels with the three major IS success variables described by (Delone & McLean, 1992; 2003), namely system quality, information quality, and system use, whereas process performance can be assessed in terms of time-schedule and budgetary considerations. Specifically, system quality is concerned with the performance characteristics of the information processing system itself, which includes ease-of-use, functionality, reliability, flexibility, integration, and response time (Delone & McLean, 1992; Rai et al., 2002). Information quality refers to accuracy, timeliness, completeness, relevance, consistency, and usefulness of information generated by the system (Delone & McLean, 1992; Fisher et al., 2006). System use is defined as “recipient consumption of the output of an information system” (Delone & McLean, 1992). These success criteria serve as the operationalizations of this study’s dependent variables (i.e., the critical success factors). In brief, this framework treats the CSFs identified as necessary factors for implementation success, whereas the absence of the CSFs would lead to failure of the system (Rockart, 1979). Within the framework, each of the CSFs identified by the Delphi study is described as follows.

Copyright © 2008, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

86 International Journal of Enterprise Information Systems, 4(3), 79-94, July-September 2008

Figure 1. A critical success factors framework for the implementation of business intelligence systems

Committed Management Support and Sponsorship

Committed management support and sponsorship has been widely acknowledged as the most important factor for BI system implementation. All Delphi participants agreed that consistent support and sponsorship from business executives make it easier to secure the necessary operating resources such as funding, human skills, and other requirements throughout the implementation process (Watson et al., 2001). This observation is reasonable and expected because the whole BI system implementation effort is a costly, time-consuming, resourceintensive process (Watson et al., 2004). Moreover, the Delphi experts further argued that BI system implementation is a

continual information improvement program to leverage decision support. They believed that the typical application-based funding for implementation of transactional systems does not apply to BI systems that are adaptive in nature. That is, a BI system evolves through an iterative process of systems development in accordance to dynamic business requirements (Arnott & Pervan, 2005). Therefore the BI initiative, especially for the enterprise-wide scale, requires consistent resource allocation and top-management support to overcome organizational issues. These organizational challenges arise during the course of the crossfunctional implementation, as it often uncovers many issues in such areas as business process, data ownership, data quality and stewardship,

Copyright © 2008, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

International Journal of Enterprise Information Systems, 4(3), 79-94, July-September 2008 87

and organizational structure. Many functional units tend to focus on tactical gains, ignoring the rippling effects imposed on other business units, and one expert observed that, The whole BI effort cut across many areas in the organization that’s making it very difficult, it hits a lot of political barriers. For instance, for a systems owner, they are only interested in delivering day to day transaction, as long as all that done… that’s what they care about. Also, without dedicated support from top management, the BI project may not receive the proper recognition and hence the support it needs to be successful. This is simply because users tend to conform to the expectations of top management and so are more likely to accept a system backed by their superiors (Lambert, 1995).

Business User-Oriented Change Management

users can provide input to the process through review and testing to ensure that it meets the goals that they think it should. Furthermore, when users are actively involved in the effort, they have a better understanding of the potential benefits and this makes them more likely to accept the system on completion (Hwang et al., 2004). Thus through this ‘implicit’ education approach, it create a sense of ownership by the users. Most interviewees also agreed that consistent support for, and systematic training of, end users must not be ignored when aiming for successful BI system implementation (Ang & Teo, 2000). Many participants emphasized that training should focus on the technology itself as well as on the associated management and maintenance issues. This training is important to equip users to understand and experience the features and functions, and to learn about the configured environment and business rules of the BI applications.

Having an adequate user-oriented change management effort was deemed critical by the Delphi participants. The experts perceive that better user participation in the change effort can lead to better communication of their needs, which in turn can help ensure the system’s successful implementation. This is particularly important when the requirements for a system are initially unclear, as is the case with many of the decision-support applications that a BI system is designed to sustain (Wixom & Watson, 2001). Significant numbers of Delphi participants shared the same view that formal user participation can help meet the demands and expectations from various end users. No doubt, the user groups know what they need better than a secluded architect or developer that does not have day to day user experience. Hence, key users must be involved throughout the implementation cycle because they can provide valuable input that the BI team may overlook. The data dimensions, business rules, metadata, and data context that are needed by business users should be considered and incorporated into the system (Wixom & Watson, 2001). Furthermore,

Clear Business Vision and Well-Established Case

As a BI initiative is driven by business, so a strategic business vision is needed to direct the implementation effort. The Delphi participants indicated that a long-term vision, primarily in strategic and organizational terms, is needed to enable the establishment of BI business case. The business case must be aligned to the corporate vision because it would eventually impact the adoption and outcome of the BI system. Otherwise they will not receive the executive and organizational supports that are required to make them successful. Consequently, the investment return of a BI system implementation should be included in those of the business process as a whole (Liautaud & Hammond, 2000). Majority interviewees indicated that the mindset of ‘setting an excellent system there, then people will come to use it’ is totally inappropriate. In fact, one interviewee claimed that: A BI system that is not business-driven is a failed system! BI is a business centric concept. Sending IT off to solve a problem rarely results

Copyright © 2008, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

88 International Journal of Enterprise Information Systems, 4(3), 79-94, July-September 2008

in a positive outcome. There must be a business problem to solve. Most participants stressed that a solid business case that was derived from a detailed analysis of business needs would increase the chances of winning support from top management. Thus, a substantial business case should incorporate the proposed strategic benefits, resources, risks, costs and the timeline. Hence, a solid business case would provide justifiable motivations for adopting a BI system to change the existing reporting and analytical practices.

Business-Driven Methodology and Project Management

The next factor to be considered is businessdriven methodology and project management. According to the Delphi experts, adequate project scoping and planning allows the BI team to concentrate on the best opportunity for improvement. To be specific, scoping helps to set clear parameters and develops a common understanding as to what is in scope and what is excluded (Ang & Teo, 2000). For instance, a Delphi expert gave insight into his experience: The success of 90% of our project is determined prior to the first day. This success is based on having a very clear and well-communicated scope, having realistic expectations and timelines, and having the appropriate budget set aside. Hence, adequate scoping enables the project team to focus on crucial milestones and pertinent issues while shielding them from becoming trapped in unnecessary events. Many experts further indicate that it is advisable to start small and adopt an incremental delivery approach. Large-scale change efforts are always fraught with greater risks given the substantial variables to be managed simultaneously (Ang & Teo, 2000). Moreover, business changes very fast and is always looking to see immediate impact, and such an incremental delivery approach provides the tools for delivery of needed

requirements in a short time (Greer & Ruhe, 2004). Also, an incremental delivery approach allows for building a long-term solution as opposed to a short term one, as is the case for an evolutionary BI system development (Arnott & Pervan, 2005). Besides that, some interviewees commented that a BI program that starts off on a high-impact area is always valuable to provide tangible evidence for both executive sponsors and key users (Morris et al., 2002). According to them, adopting this so-called ‘low hanging fruits’ approach—projects with the greatest visibility and monetary impact— demonstrates to leadership that there is a payback (ROI) for their investment and it shows it in a short timeframe. This will increase leadership support and help the other associated initiatives to be supported readily. One interviewee elaborated that: You cannot role out the whole BI system at once but people want to see some key areas. You need to do data marts for a couple of key areas and then maybe a small number of other key reports in an attempt to keep all stakeholders happy. Then when the first release is done and you get some feedback, you can work on other data mart areas and enhance existing subject areas over time. Therefore, a ‘low hanging fruits’ approach allows an organization to concentrate on crucial issues, so enabling teams to prove that the system implementation is feasible and productive for the enterprise.

Business-Centric Championship and Balanced Project Team Composition

The majority of Delphi experts believed that having the right champion from the business side of the organization is critical for implementation success. According to them, a champion who has excellent business acumen is always important since he/she will be able to foresee the organizational challenges and change course accordingly. More importantly, this business-centric champion would view the BI

Copyright © 2008, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

International Journal of Enterprise Information Systems, 4(3), 79-94, July-September 2008 89

system primarily in strategic and organizational perspectives, as opposed to one who might over-focus on technical aspects. For example, as noted by an interviewee: The team needs a champion. By a champion, I do not mean someone who knows the tools. I mean someone who understands the business and the technology and is able to translate the business requirements into a (high-level) BI architecture for the system. All interviewees also agreed that the composition and skill sets of a BI team have a major influence on the implementation success. The project team should be cross-functional and composed of those personnel who possess technical expertise and those with a strong business background (Burton et al., 2006). As most interviewees stressed, a BI system is a business-driven project to provide enhanced managerial decision support, and so a suitable mix of IT expertise is needed to implement the technical aspects, whereas the reporting and analysis aspects must be under the realm of business personnel. Furthermore, most experts posited that the BI team must identify and include business domain experts, especially for such activities as data standardization, requirement engineering, data quality analysis, and testing. Many respondents also agreed with the critical role played by external consultants, especially at early phase. They believed that the lack of in-house experience and competencies can be complemented by external consultants who have spent the majority of their time working on similar projects. As well as being a subject matter expert, the interviewees indicated that an external consultant could provide an unbiased view of solution to a problem. This is because the organizational structure of an engineering asset management enterprise is traditionally functional-oriented and culturally fragmented with siloed information systems design (Haider & Koronios, 2003). There may even be situations where the client possesses the expertise to solve a particular problem, but are conflicted

on the organizational ground. An external consultant hence can evaluate and propose an unbiased course of action without having fear of political repercussions (Kaarst-Brown, 1999).

Strategic and Extensible Technical Framework

In terms of strategic and extensible technical framework, most experts asserted that stable source/back-end systems are crucial in implementing a BI system. A reliable back-end system is critical to ensure that the updating of data works well for the extraction, transformation and loading (ETL) processes in the staging area (Ponniah, 2001). Hence the data can be transformed to provide a consistent view into quality information for improved decision support. It is therefore crucial for BI team to assess the stability and consistency of source systems before embarking on a BI effort. Otherwise after the system implementation, the cost of changes in terms of time and money can be significant. A BI expert explained the importance of this factor in detail: It’s more important you got a reliable, consistent, stable back-end system, in my experience, I’m working with a mining company now, in their case, they don’t have consistent back-end systems, in some departments, they have just large number of spreadsheets, which call production data into their spreadsheets, it is scary. It’s a major impediment to BI system, and you got multiple bits over all the places. Another prime element concerned by the respondents was that the technical framework of a BI system must be able to accommodate scalability and extendibility requirements. Having a strategic view embedded in the system design, this scalable system framework could include additional data sources, attributes, and dimensional areas for fact-based analysis, and it could incorporate external data from suppliers, contractors, regulatory bodies, and industry benchmarks (Watson et al., 2004). It would then allow for building a long-term solution to meet incremental needs of business.

Copyright © 2008, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

90 International Journal of Enterprise Information Systems, 4(3), 79-94, July-September 2008

The majority of interviewees also agreed that a prototype is always valuable as proof of a concept. That is, constructing a fairly small BI application for a key area in order to provide tangible evidence for both executive sponsors and general users (Watson et al., 2001). They perceive that a prototype that offers clear forms of communication, and better understanding in an important business area, would convince organizational stakeholders on the usefulness of a BI system implementation. As a result of a successful prototype, senior management and key users would be more likely and more motivated to support larger-scale BI efforts.

Sustainable Data Quality and Governance Framework

The Delphi findings indicate that the quality of data, particularly at the source systems, is crucial if a BI system is to be implemented successfully. According to the interviewees, a primary purpose of the BI system is to integrate ‘silos’ of data sources within enterprise for advanced analysis so as to improve the decision-making process. Often, much data related issues within the back-end systems are not discovered until that data is populated and queried against in the BI system (Watson et al., 2004). Thus corporate data can only be fully integrated and exploited for greater business value once its quality and integrity are assured. The management are also urged to initiate data governance and stewardship efforts to improve the quality of the data in back-end systems because unreliable data sources will have a ripple effect on the BI applications and subsequently the decision outcomes (Chengalur-Smith et al., 1999). For instance, an expert expressed his concern: This is the most underrated and underestimated part of nearly every BI development effort. Much effort is put into getting the data right the first time, but not near enough time is spent putting in place the data governance processes to ensure the data quality is maintained.

Some interviewees further argued that a sound data governance initiative is more than ad-hoc data quality projects. Indeed, it should include a governing committee, a set of procedures, and an execution plan. More specifically, the roles of data owners or custodians and data stewards must be clearly defined (Watson et al., 2004). Frontline and field workers should be made responsible for their data source and hence data quality assurance. Meanwhile, a set of policies and audit procedures must be put into place that ensures ongoing compliance with regulatory requirements as most EAMOs like utilities are public-owned company. Apart from that, the Delphi participants believed that common measures and definitions address the data quality dimension of representational consistency. This allows all stakeholders to know that this term has such definition no matter where it is used across the source systems. Furthermore, it is typical for an EAMO to have hundreds of varying terms with slightly different meanings, because different business units tend to define terms in ways that best serve their purposes. Often accurate data may have been captured at the source level; however, the record cannot be used to link with other data sources due to inconsistent data identifier. This is simply because data values that should uniquely describe entities are varied in different business units. Once an organization collects a large number of reports it becomes harder to re-architect these areas. As a result, a cross-system analysis is important to help profiling a uniform ‘master data set’ which is in compliance with business rules. The development of a master data set on which to base the logical data warehouse construction for BI system will ease terminology problems (Watson et al., 2004). In order to have consistent measures and classification across subject areas, most interviewees asserted that business-led commitment is pivotal to establish consensus on data measurement and definition. Indeed, a BI system implementation is a business driven initiative to support the reporting and analytical requirements of business. As a result, the BI

Copyright © 2008, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

International Journal of Enterprise Information Systems, 4(3), 79-94, July-September 2008 91

team would use those common definitions to develop an enterprise-wide dimensional model that is business-orientated. Many participants asserted that a correct dimensional data model is the absolute cornerstone of every BI project. A faulty model will surely lead to failure of the project as it will fail to deliver the right information. As noted by an interviewee: Not understanding dimensional modeling will cause lots of grief later on and make it difficult to answer some questions. Once you have a large number of reports, it becomes harder to re-architect these areas. Better to get it right the first time with a star schema and well-designed dimensions and fact tables. Good use of aggregates can speed report results and make people happy. Also, a sustainable metadata model on which to base the logical and physical data warehouse construction for a BI system was deemed critical by many experts. Therefore, the metadata model should be flexible enough to enable the scalability of the BI system while consistently providing integrity on which OLAP and data mining depend (Watson & Haley, 1997).

CONCLUDING REMARKS AND FUTURE RESEARCH

This theory building research presents a CSFs framework derived from a Delphi study with 15 BI systems experts within engineering asset management domain. An analysis of the findings demonstrated that there are a number of CSFs peculiar to successful BI system implementation. More importantly, this study revealed a clear trend towards multi-dimensional factors in implementing BI systems. Organizational factors were perceived to be more important than the technological ones because the BI team considered them to be outside their direct control. Furthermore, the contextual elements of these CSFs appear to be substantially different from the implementation effort of conventional operational systems. The research is likely to make both theoretical and practical contributions to the field of BI systems implementation. First, this study fills

in the research gap by building theory of CSFs, addresses issues of concern to practitioners and supplements the current limited understanding on implementation issues of BI systems. Moreover, this research provides thought-provoking insights into multi-dimensional CSFs that influence the BI systems implementation. The contextual elements identified alongside for each of the critical factors and the consolidated CSFs framework provides a comprehensive and meaningful understanding of CSFs. Not only does this research contribute to the academic literature but it benefits organizations in several ways as well. Essentially, BI practitioners (both current and potential) will be better able to identify critical factors for successfully implementing BI systems. The findings will enable them to better manage their implementation of BI systems if they understand that such effort involves multiple dimensions of success factors occurring simultaneously and not merely the technical aspects of the system. With the CSFs framework, it could enable BI stakeholders to better identify the necessary factors, and to possess a comprehensive understanding of those CSFs. Such outcomes will help them to improve the effectiveness and efficiency of their implementation activities, by obtaining a better understanding of possible antecedents that lead to successful BI system implementation. For senior management, this research finding can certainly assist them by optimising their scarce resources on those critical factors that are most likely to have an impact on the BI systems implementation. Moreover, the management can concentrate their commitment to monitor, control and support only those key areas of implementation. In the next stage, it is planned to conduct case study with multiple engineering asset management organizations to further validate the CSFs findings. The multiple case studies will examine whether these critical factors and/or any other alternative factors influence the implementation success of BI systems.

Copyright © 2008, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

92 International Journal of Enterprise Information Systems, 4(3), 79-94, July-September 2008

ACKNOWLEDGMENT

Information Systems, 12(2), 37-57. Fisher, C., Lauria, E., Chengalur-Smith, I., & Wang, R. (2006). Introduction to information quality. Cambridge, MA: MITIQ Press. Friedman, T. (2005). Gartner says more than 50 percent of data warehouse projects will have limited acceptance or will be failures through 2007. Retrieved February 21, 2007, from http://www. gartner.com/it/page.jsp?id=492112. Gangadharan, G., & Swami, S. (2004). ��������� Business intelligence systems: Design and implementation strategies. Paper presented at the 26th International Conference Information Technology Interfaces ITI. Gartner Press Release. (2006a). Gartner says business intelligence software market to reach $3 billion in 2009. http://www.gartner.com/press_releases/asset_144782_11.html. Gartner Press Release. (2006b). Gartner survey of 1,400 CIOs shows transformation of IT organization is accelerating. http://www.gartner.com/press_releases/asset_143678_11.html. Gartner Press Release. (2007). Gartner EXP survey of more than 1,400 CIOs shows CIOs must create leverage to remain relevant to the business. Retrieved February 21, 2007, from http://www.gartner.com/it/ page.jsp?id=501189. Glaser, B., & Strauss, A. (1967). The discovery of grounded theory. Chicago, IL: Aldine. Glaser, B. (1992). Basics of grounded theory analysis. Glaser, B. (1998). Doing grounded theory: Issues and discussions. Sociology Press. Greer, D., & Ruhe, G. (2004). Software release planning: An evolutionary and iterative approach. Information and Software Technology, 46(4), 243-253. Haider, A., & Koronios, A. (2003). Managing engineering assets: A knowledge-based approach through information quality. Paper presented at the International Business Information Management Conference. Cairo. Helmer, O. (1977). Problems in futures research: Delphi and causal cross-impact analysis. Futures, 9(S 17), 31.

This research is conducted through the Cooperative Research Centre for Integrated Engineering Asset Management (CIEAM). The support of CIEAM partners is gratefully acknowledged. This article is based on a previous version presented at the IFIP TC8 CONFENIS 2007 Conference.

REFERENCES
Ang, J., & Teo, T. (2000). Management ��������������������� issues in data warehousing: Insights from the housing and development board. Decision Support Systems, 29(1), 11-20. Ariyachandra, T., & Watson, H. (2006). Which data warehouse architecture is most successful?. Business Intelligence, 11(1). Arnott, D., & Pervan, G. (2005). A critical analysis of decision support systems research. Journal of Information Technology, 20(2), 67-87. Brunelli, M. (2006). BI, ERP top 2007’s IT spending list. Retrieved July 8, 2007, from http://searchoracle. techtarget.com/originalContent/0,289142,sid41_ gci1233170,00.html. Burton, B., Geishecker, L., & Hostmann, B. (2006). Organizational structure: Business intelligence and information management. Carson, D., Gilmore, A., Gronhaug, K., & Perry, C. (2001). Qualitative research in marketing. London: Sage. Chengalur-Smith, I., Ballou, D., & Pazer, H. (1999). The impact of data quality information on decision making: An exploratory analysis. Knowledge and Data Engineering, IEEE Transactions on, 11(6), 853-864. Delone, W., & McLean, E. (1992). Information systems success: The quest for the dependent variable. Information System Research, 3(1), 60-95. Delone, W., & McLean, E. (2003). The DeLone and McLean model of information systems success: A ten-year update. Management Information Systems, 19(4), 9-30. Duncan, N. (1995). Capturing flexibility of information technology infrastructure: A study of resource characteristics and their measure. Management

Copyright © 2008, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

International Journal of Enterprise Information Systems, 4(3), 79-94, July-September 2008 93

Hwang, H., Ku, C., Yen, D., & Cheng, C. (2004). Critical factors influencing the adoption of data warehouse technology: A study of the banking industry in Taiwan. Decision Support Systems, 37(1), 1-21. IDC Press Release. (2007). Top-ranked business intelligence tools vendors maintain positions. Retrieved July 3, 2007, from http://www.idc.com/getdoc. jsp?containerId=prUS20767807. Jagielska, I., Darke, P., & Zagari, G. (2003). Business ��������� intelligence systems for decision support: Concepts, processes and practice. Paper presented at the 7th International Conference of the International Society for Decision Support Systems. Kaarst-Brown, M. (1999). Five symbolic roles of the external consultant: Integrating change, power and symbolism. Journal of Organizational Change Management, 12(6), 540-561. Lambert, S. (1995). An investigation of workers’ use and appreciation of supportive workplace policies. Best papers 1995: Proceedings of the Academy of Management. Liautaud, B., & Hammond, M. (2000). E-business intelligence: Turning information into knowledge and profit. New York, NY: McGraw-Hill. Lin, S., Gao, J., Koronios, A., & Chanana, V. (2007). Developing a data quality framework for asset management in engineering organizations. International Journal of Information Quality, 1(1), 100-126. Linstone, H., & Turoff, M. (1975). The Delphi method: Techniques and applications. Reading, MA: Addison-Wesley. Linstone, H., & Turoff, M. (2002). The Delphi method: Techniques and applications (Electronic version). Logan, D., & Buytendijk, F. (2003). The SarbanesOxley Act will impact your enterprise. Lucas, H. (1978). The evolution of an information system: From key-man to every person. Sloan Management Review, 39(52). Mathew, J. (2003). CIEAM business plan V1.0. Brisbane, Australia: Centre for Integrated Engineering Asset Management (CIEAM) Morris, H., Moser, K., Vesset, D., & Blumstein, R. (2002). The financial impact of business analytics. Framingham, MA: IDC.

Negash, S. (2004). Business intelligence. Communications of the Association for Information Systems, 13, 177-195. Ponniah, P. (2001). Data warehousing fundamentals. New York, NY: Wiley-Interscience. Ra���������������������������������������������������� i, A., Lang, S., & Welker, R. (2002). �������������� Assessing the validity of IS success models: An empirical test and theoretical analysis. Information Systems Research, 13(1), 50-69. Rockart, J. (1979). Chief executives define their own data needs. Harvard Business Review, 57(2), 81-93. Shin, B. (2003). An exploratory investigation of system success factors in data warehousing. Journal of the Association for Information Systems, 141(170), 170. Sum, C., Ang, J., & Yeo, L. (1997). Contextual elements of critical success factors in MRP implementation. Production and Inventory Management Journal, 38(3), 77-83. Watson, H., Abraham, D., Chen, D., Preston, D., & Thomas, D. (2004). Data warehousing ROI: Justifying and assessing a data warehouse. Business Intelligence Journal, 6-17. Watson, H., & Haley, B. (1998). Managerial considerations. Communications of the ACM, 41(9), 32-37. Watson, H., Annino, D., Wixom, B., Avery, K., & Rutherford, M. (2001). Current practices in data warehousing. Information Systems Management, 18(1), 1-9. Watson, H., Fuller, C., & Ariyachandra, T. (2004). Data warehouse governance: Best practices at Blue Cross and Blue Shield of North Carolina. Decision Support Systems, 38(3), 435-450. Watson, H., & Haley, B. (1997). Data warehousing: A framework and survey of current practices. Journal of Data Warehousing, 2(1), 10-17. White, C. (2006). New CIO spending survey. Retrieved July 11, 2007, from http://www.b-eyenetwork.co.uk/blogs/white/archives/2006/09/ new_cio_spending_survey.php. Wixom, B., & Watson, H. (2001). An empirical investigation of the factors affecting data-warehousing success. MIS Quarterly, 25(1), 17-41.

Copyright © 2008, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

94 International Journal of Enterprise Information Systems, 4(3), 79-94, July-September 2008 Ziglio, E. (1996). The Delphi method and its contribution to decision-making. Gazing into the oracle. The Delphi method and its application to social policy and public health (pp. 3-33). London: Jessica Kingsley Publishers.

EndnOTe
1



Implementation refers to an on-going process which includes the entire development of an information system from the original suggestions through the feasibility study, system analysis and design, programming, training, conversion, and installation of the system (Lucas, 1978).

Copyright © 2008, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.