library / libe81001e6bb4bd101
Business Intelligence and Big Data
Celina Olszak
In a sentence
A scholarly synthesis arguing that organizational success in the digital age depends on building Business Intelligence and Big Data (BI&BD) capabilities to convert data into knowledge, value, and competitive advantage.
Business Intelligence and Big Data: Drivers of Organizational Success makes the case that in a turbulent, information-saturated economy, intangible resources—information and knowledge—are an organization's most strategic assets, and that the ability to intelligently process and analyze them is the key competence separating winners from losers. Drawing on resource-based and dynamic-capabilities theory, knowledge management, and decades of decision-support research, Celina Olszak walks readers through the essence and architecture of BI systems, the techniques and technologies of Big Data, concrete application domains (marketing, CRM, insurance, banking, telecommunications, energy, logistics, health care, HR), and the maturity models and critical success factors that determine whether BI&BD investments actually create business value. It is at once a textbook for students, a research agenda for scholars, and a practical handbook for managers and ICT specialists who want to know not just what BI&BD can do but what determines its successful design, implementation, and value creation.
The four lenses
- Science
- Statistics
- Systems
- Strategy
The model
A path model in which design levers (ICT/BI&BD investment, governance, technology) and contextual conditions (environmental turbulence, analytical/learning culture, human skills) drive psychological and behavioral states (analytical capability, knowledge discovery, decision quality) that in turn produce outcomes such as business value, competitive advantage, and organizational success.
BI&BD Technology Investment and Capabilitydesign lever
The organization's investment in and deployment of Business Intelligence and Big Data infrastructure, tools, and technologies (data warehouses, ETL, OLAP, mining, NoSQL, Hadoop) used to collect, integrate, and analyze data.
Environmental Turbulence and Information Flow Complexitycontextual condition
The degree of change, competition, information excess, and uncertainty in the organization's external environment that creates pressure to anticipate and respond quickly to market signals and stakeholder demands.
Fact-based Analytical and Learning Culturecontextual condition
The organizational culture, leadership support, values, and norms that encourage gathering, analyzing, sharing, and acting on information and knowledge, including openness to change, creativity, and information-based competition.
Human Analytical and Knowledge Skillsdesign lever
The technical, statistical, business, communication, and entrepreneurial skills of employees and managers that enable effective use of BI&BD, including data science competences and willingness to share knowledge.
BI&BD Governance and Strategic Alignmentdesign lever
Mechanisms for managing BI&BD resources, aligning BI&BD initiatives with business strategy, management support and sponsorship, clear vision, and a deliberate data-exploration strategy.
Analytical Capability and Knowledge Discoverypsychological state
The organization's enacted capacity to explore and exploit data—sensing opportunities, mining and analyzing data, and discovering new, previously unknown knowledge and patterns to inform action.
Decision-Making Quality and Speedbehavioral pattern
The improvement in the efficiency, timeliness, accuracy, and breadth of decisions made at operational, tactical, and strategic levels as a result of BI&BD-supported information and knowledge.
Big Data-Based Business Value Creationoutcome metric
The economic and social value derived from BI&BD use, including innovative products and services, optimized processes, improved customer relations, new business models, and risk reduction.
Organizational Success and Competitive Advantageoutcome metric
The ultimate outcome of sustained competitive advantage, organizational efficiency, profitability, and market position achieved through effective use of information, knowledge, and BI&BD.
How they connect
- bibd investment → predicts analytical capability
- human analytical skills → predicts analytical capability
- governance strategy → predicts analytical capability
- analytical learning culture → moderates analytical capability
- analytical capability → predicts decision quality
- decision quality → predicts business value creation
- analytical capability → mediates business value creation
- business value creation → predicts organizational success
- environmental turbulence → moderates business value creation
- governance strategy → influences business value creation
The process
The book provides a comprehensive framework for organizations to leverage Business Intelligence (BI) and Big Data (BD) as drivers of success. The overall playbook begins with establishing a foundational understanding of how to design and develop data-centric systems. The core operational process involves the continuous cycle of data management, starting with the ETL (Extraction, Transformation, Load) process to populate data warehouses, and extending to the more complex Big Data analysis lifecycle for handling vast, unstructured datasets. These data management processes feed into various analytical applications. Once the data infrastructure is in place, the playbook guides organizations in applying these analytics to specific business functions, such as conducting Customer Lifetime Value (LTV) analysis to enhance marketing and customer relationships. The final, strategic layer of the playbook involves a meta-process for continuous improvement: assessing the organization's BI and BD maturity. By regularly evaluating their capabilities across dimensions like technology, people, and processes, organizations can create a roadmap to advance from a nascent, reactive state to a mature, visionary one, where data and analytics are deeply embedded in strategy and drive competitive advantage. This cyclical approach ensures that data systems and analytical capabilities evolve in alignment with business goals.
Designing a Decision Support System
To create an information system that improves organizational efficiency and effectiveness by integrating people, organizations, and technology.
When to use: When an organization needs to develop a new decision support system or significantly re-evaluate an existing one.
Step 1Identify the problem and motivate its solution.
Entry: A business need or opportunity has been recognized.
Exit: The problem is clearly defined and the motivation for solving it is documented and approved.
In: Business needs, Organizational strategies · Out: Problem statement, Project motivation document
Step 2Define system goals in the context of business needs.
Entry: The problem and motivation are defined.
Exit: A clear set of system goals aligned with business needs is documented.
In: Problem statement · Out: System goals document
Step 3Design and develop the system artifacts.
Entry: System goals are defined.
Exit: The system artifacts (e.g., models, software components) are created.
In: System goals, Technical requirements · Out: System artifacts (e.g., software, models, methods)
Step 4Demonstrate and use the artifacts to solve the problem.
Entry: System artifacts have been developed.
Exit: The system has been used to address the identified problem.
In: System artifacts, Business data · Out: Problem solution demonstration
Step 5Evaluate the solution.
Entry: The system has been demonstrated.
Exit: An evaluation report on the system's effectiveness is complete.
In: Demonstration results, System goals · Out: System evaluation report
Step 6Communicate the results.
Entry: The system has been evaluated.
Exit: Communication to stakeholders is complete.
In: System evaluation report, Problem statement · Out: Project documentation, Stakeholder communications
Developing a Business Intelligence System (BI Life Cycle)
To build, implement, and utilize a BI system that meets an organization's information needs and supports its strategy in an iterative lifecycle.
When to use: When an organization decides to implement or overhaul its Business Intelligence capabilities.
Step 1Define the BI venture, vision, and strategy.
Entry: An organizational need for improved business intelligence has been identified.
Exit: A clear BI vision and strategy document is approved.
In: Business strategy, User information needs · Out: BI vision and strategy document
Step 2Identify and prepare data sources.
Entry: The BI vision and strategy are defined.
Exit: Data sources are identified and a plan for the data warehouse is complete.
In: BI strategy, Existing data sources (internal and external) · Out: Data source inventory, Data warehouse design
Step 3Select BI tools and technologies.
Entry: Data sources and high-level design are understood.
Exit: A suite of BI tools has been selected and procured.
- Which BI platform to use (e.g., Business Objects, Microsoft, Oracle, SAS).
In: BI strategy, Technical requirements · Out: Selected BI toolset
Step 4Design, implement, and provide training for the BI system.
Entry: BI tools have been selected.
Exit: The BI system is implemented and users are trained.
In: Selected BI toolset, Data warehouse design · Out: Implemented BI system, Trained user base
Step 5Explore and discover new knowledge and needs.
Entry: The BI system is operational.
Exit: New information needs and potential system improvements are identified.
In: User interactions with the BI system · Out: Evaluation of current BI system, List of new information needs
Step 6Consume BI outputs for decision-making.
Entry: The BI system is providing analyses and reports.
Exit: BI outputs are regularly used to inform business decisions.
In: BI reports and analyses · Out: Informed business decisions
Step 7Optimize business activities and processes.
Entry: Decisions are being informed by BI.
Exit: Business processes are measurably improved.
In: Informed business decisions · Out: Optimized business processes
Step 8Drive fundamental organizational change and iterate.
Entry: Processes have been optimized.
Exit: Strategic changes are implemented and the BI development cycle is re-initiated.
In: Process optimization results · Out: New business models, New strategic initiatives
Execute the ETL (Extraction-Transformation-Load) Process
To obtain data from various sources, clean and standardize it, and load it into a data warehouse for analysis, ensuring data quality and consistency.
When to use: When data needs to be moved from operational systems and other sources into a central repository for analysis.
Step 1Extract data from source systems.
Entry: Source systems and required data have been identified.
Exit: All required data has been extracted and saved to a staging area.
In: Data from source systems · Out: Raw data in a staging database
Step 2Transform the data.
Entry: Data has been extracted to a staging area.
Exit: Data is cleaned, standardized, and in the target format for the data warehouse.
In: Raw data in a staging database · Out: Transformed and cleaned data
Step 3Load data into the data warehouse.
Entry: Data has been successfully transformed.
Exit: The data warehouse is updated with the new, transformed data.
In: Transformed and cleaned data · Out: Updated data warehouse
Managing the Big Data Analysis Lifecycle
To systematically acquire, process, analyze, and interpret large, complex datasets (Big Data) to extract business value.
When to use: When an organization needs to derive insights from Big Data sources like social media, IoT devices, or web logs.
Step 1Acquire and record data.
Entry: Relevant Big Data sources have been identified.
Exit: Data is captured and stored in a suitable repository.
In: Data streams from various sources (e.g., IoT, social media) · Out: Stored raw data
Step 2Extract, clean, and annotate data.
Entry: Raw data has been acquired.
Exit: Data is cleaned and in a usable format for analysis.
In: Stored raw data · Out: Cleaned and structured data
Step 3Integrate, aggregate, and represent data.
Entry: Data has been cleaned.
Exit: Data is integrated and properly represented for modeling.
In: Cleaned and structured data · Out: Integrated dataset
Step 4Analyze the data and build models.
Entry: Data is integrated and represented for analysis.
Exit: Analytical models are built and patterns are identified.
- Which analytical technique to apply (e.g., classification, clustering, regression).
In: Integrated dataset · Out: Analytical models, Identified patterns and trends
Step 5Interpret the results.
Entry: Analysis and modeling are complete.
Exit: Actionable business insights are generated and communicated.
In: Analytical models, Identified patterns and trends · Out: Interpretation report, Business recommendations
Conducting Customer Lifetime Value (LTV) Analysis
To determine the total value of a customer over time, taking into account their full history and expected future benefits, in order to increase their value to the organization.
When to use: When an organization wants to move beyond simple profitability analysis to a long-term view of customer value.
Step 1Collect data about customers and their purchasing behaviors.
Entry: A need to understand long-term customer value is established.
Exit: A comprehensive dataset of customer behavior is compiled.
In: Customer transaction history, Customer interaction logs · Out: Customer behavior dataset
Step 2Analyze data to segment customers and define profiles.
Entry: Customer behavior data has been collected.
Exit: Customer segments and profiles are defined and documented.
In: Customer behavior dataset · Out: Customer segments, Customer profiles
Step 3Prepare a personalized offer.
Entry: Customer profiles have been defined.
Exit: A personalized offer for each target segment is ready.
In: Customer profiles, Product/service catalog · Out: Personalized marketing offers
Step 4Present the offer to the customer.
Entry: Personalized offers have been prepared.
Exit: The customer has received the offer.
In: Personalized marketing offers · Out: Customer communication
Step 5Handle customer contact and transactions.
Entry: The customer has responded to the offer.
Exit: The customer interaction and any transaction are completed and documented.
In: Customer response · Out: Transaction records, Updated customer data
Assessing Business Intelligence and Big Data Maturity
To assess an organization's current capabilities in BI and Big Data, identify areas for improvement, and create a roadmap to guide development toward a more mature state.
When to use: Periodically, to benchmark BI/BD capabilities and plan strategic improvements.
Step 1Select an appropriate maturity model.
Entry: A decision has been made to formally assess BI/BD maturity.
Exit: A specific maturity model has been selected for use.
- Which maturity model to use (e.g., TDWI, Gartner, IBM, Dell).
In: Organizational goals for BI/BD · Out: Selected maturity model
Step 2Define the scope of the assessment.
Entry: A maturity model has been selected.
Exit: The scope of the assessment is clearly defined and agreed upon.
In: Selected maturity model · Out: Assessment scope document
Step 3Conduct the assessment to determine the current state.
Entry: The assessment scope is defined.
Exit: Data on current capabilities has been collected and analyzed.
In: Assessment scope document, Input from stakeholders · Out: Raw assessment data
Step 4Determine the current maturity level.
Entry: Assessment data has been collected.
Exit: The organization's current maturity level is identified for each assessed dimension.
In: Raw assessment data, Maturity model level definitions · Out: Current state maturity assessment report
Step 5Identify gaps and define a target maturity state.
Entry: The current maturity level is known.
Exit: Gaps are identified and a target maturity level is defined.
In: Current state maturity assessment report, Business strategy · Out: Gap analysis, Target state definition
Step 6Develop an improvement roadmap.
Entry: Gaps and a target state have been defined.
Exit: An actionable roadmap for BI/BD improvement is documented and approved.
In: Gap analysis, Target state definition · Out: BI/BD improvement roadmap
A candidate measure
Business Intelligence and Big Data — derived measurement candidates
BI&BD Technology Investment and Capability
Analytics IT spend; Count of deployed BI&BD components
self-report suitability: medium
Environmental Turbulence and Information Flow Complexity
Market change rate; Competitive intensity index
self-report suitability: medium
Fact-based Analytical and Learning Culture
Culture survey indices; Frequency of fact-based decisions
self-report suitability: high
Human Analytical and Knowledge Skills
Data scientist headcount; Skill/certification levels
self-report suitability: medium
BI&BD Governance and Strategic Alignment
Presence of competence center; Strategy-alignment ratings
self-report suitability: high
Analytical Capability and Knowledge Discovery
Maturity-level placement; Volume of advanced analyses
self-report suitability: medium
Decision-Making Quality and Speed
Decision cycle time; Forecast error reduction
self-report suitability: medium
Big Data-Based Business Value Creation
Revenue from new products; Cost savings; Retention gains
self-report suitability: medium
Organizational Success and Competitive Advantage
Profit margin; Market share; Growth rate
self-report suitability: low
The story
The reader A manager, researcher, or ICT specialist who wants their organization to succeed by turning information into competitive advantage.
External problem
Organizations are drowning in dispersed, inconsistent, fast-growing data yet under-utilize the potential of BI&BD to support decisions.
Internal problem
They feel uncertain whether success is luck or something that can be planned, and overwhelmed by complexity and rapid technological change.
Philosophical problem
It is wrong to let valuable information and knowledge—an organization's most strategic resources—go unused while competitors create new value from them.
The plan
- Understand the changing environment and treat information and knowledge as strategic resources.
- Learn the essence, architecture, and models of Business Intelligence systems.
- Master the techniques and technologies needed to analyze Big Data.
- Identify the application areas where BI&BD deliver value in your industry.
- Assess your BI&BD maturity, address critical success factors, and build a strategy for Big Data-based value creation.
Success
- Faster, better decisions at operational, tactical, and strategic levels.
- Improved customer relationships, business processes, and competitive advantage.
- A fact-based, learning, and creative culture that continuously creates value from data.
At stake
- Isolation and exclusion from economic and social activity for organizations that fall behind technologically.
- Lost value as information and knowledge go unused.
- Failed BI&BD projects due to weak culture, skills, and strategy.
Questions this book answers
- What determines the success of Business Intelligence and Big Data use in organizations?
- How should BI&BD systems be designed and implemented to support decision-making at all levels?
- What are the potential areas of BI&BD application across industries?
- How can organizational BI&BD maturity be measured and assessed?
- How do organizations create durable business value from Big Data resources?
Glossary
- BI&BD Technology Investment and Capability
- The extent to which an organization invests in and deploys Business Intelligence and Big Data technologies and infrastructure.
- Environmental Turbulence and Information Flow Complexity
- The degree of dynamism, competition, and information overload in the organization's environment.
- Fact-based Analytical and Learning Culture
- Shared values, leadership support, and norms favoring information gathering, analysis, sharing, learning, and creativity.
- Human Analytical and Knowledge Skills
- The technical, statistical, business, and interpersonal competences enabling effective BI&BD use.
- BI&BD Governance and Strategic Alignment
- The management mechanisms, sponsorship, vision, and strategy that align BI&BD with business goals.
- Analytical Capability and Knowledge Discovery
- The enacted capacity to explore and exploit data to discover knowledge and inform action.
- Decision-Making Quality and Speed
- The improvement in timeliness, accuracy, and scope of decisions enabled by BI&BD.
- Big Data-Based Business Value Creation
- The economic and social value generated through BI&BD, including innovation, process optimization, and improved relations.
Tools these methods power