peopleanalyst

library / libe81001e6bb4bd101

Business Intelligence and Big Data

Celina Olszak

In a sentence

A scholarly synthesis arguing that organizational success in the digital age depends on building Business Intelligence and Big Data (BI&BD) capabilities to convert data into knowledge, value, and competitive advantage.

Business Intelligence and Big Data: Drivers of Organizational Success makes the case that in a turbulent, information-saturated economy, intangible resources—information and knowledge—are an organization's most strategic assets, and that the ability to intelligently process and analyze them is the key competence separating winners from losers. Drawing on resource-based and dynamic-capabilities theory, knowledge management, and decades of decision-support research, Celina Olszak walks readers through the essence and architecture of BI systems, the techniques and technologies of Big Data, concrete application domains (marketing, CRM, insurance, banking, telecommunications, energy, logistics, health care, HR), and the maturity models and critical success factors that determine whether BI&BD investments actually create business value. It is at once a textbook for students, a research agenda for scholars, and a practical handbook for managers and ICT specialists who want to know not just what BI&BD can do but what determines its successful design, implementation, and value creation.

The four lenses

  • Science
  • Statistics
  • Systems
  • Strategy

The model

A path model in which design levers (ICT/BI&BD investment, governance, technology) and contextual conditions (environmental turbulence, analytical/learning culture, human skills) drive psychological and behavioral states (analytical capability, knowledge discovery, decision quality) that in turn produce outcomes such as business value, competitive advantage, and organizational success.

BI&BD Technology Investment and Capabilitydesign lever

The organization's investment in and deployment of Business Intelligence and Big Data infrastructure, tools, and technologies (data warehouses, ETL, OLAP, mining, NoSQL, Hadoop) used to collect, integrate, and analyze data.

Environmental Turbulence and Information Flow Complexitycontextual condition

The degree of change, competition, information excess, and uncertainty in the organization's external environment that creates pressure to anticipate and respond quickly to market signals and stakeholder demands.

Fact-based Analytical and Learning Culturecontextual condition

The organizational culture, leadership support, values, and norms that encourage gathering, analyzing, sharing, and acting on information and knowledge, including openness to change, creativity, and information-based competition.

Human Analytical and Knowledge Skillsdesign lever

The technical, statistical, business, communication, and entrepreneurial skills of employees and managers that enable effective use of BI&BD, including data science competences and willingness to share knowledge.

BI&BD Governance and Strategic Alignmentdesign lever

Mechanisms for managing BI&BD resources, aligning BI&BD initiatives with business strategy, management support and sponsorship, clear vision, and a deliberate data-exploration strategy.

Analytical Capability and Knowledge Discoverypsychological state

The organization's enacted capacity to explore and exploit data—sensing opportunities, mining and analyzing data, and discovering new, previously unknown knowledge and patterns to inform action.

Decision-Making Quality and Speedbehavioral pattern

The improvement in the efficiency, timeliness, accuracy, and breadth of decisions made at operational, tactical, and strategic levels as a result of BI&BD-supported information and knowledge.

Big Data-Based Business Value Creationoutcome metric

The economic and social value derived from BI&BD use, including innovative products and services, optimized processes, improved customer relations, new business models, and risk reduction.

Organizational Success and Competitive Advantageoutcome metric

The ultimate outcome of sustained competitive advantage, organizational efficiency, profitability, and market position achieved through effective use of information, knowledge, and BI&BD.

How they connect

  • bibd investment predicts analytical capability
  • human analytical skills predicts analytical capability
  • governance strategy predicts analytical capability
  • analytical learning culture moderates analytical capability
  • analytical capability predicts decision quality
  • decision quality predicts business value creation
  • analytical capability mediates business value creation
  • business value creation predicts organizational success
  • environmental turbulence moderates business value creation
  • governance strategy influences business value creation

The process

The book provides a comprehensive framework for organizations to leverage Business Intelligence (BI) and Big Data (BD) as drivers of success. The overall playbook begins with establishing a foundational understanding of how to design and develop data-centric systems. The core operational process involves the continuous cycle of data management, starting with the ETL (Extraction, Transformation, Load) process to populate data warehouses, and extending to the more complex Big Data analysis lifecycle for handling vast, unstructured datasets. These data management processes feed into various analytical applications. Once the data infrastructure is in place, the playbook guides organizations in applying these analytics to specific business functions, such as conducting Customer Lifetime Value (LTV) analysis to enhance marketing and customer relationships. The final, strategic layer of the playbook involves a meta-process for continuous improvement: assessing the organization's BI and BD maturity. By regularly evaluating their capabilities across dimensions like technology, people, and processes, organizations can create a roadmap to advance from a nascent, reactive state to a mature, visionary one, where data and analytics are deeply embedded in strategy and drive competitive advantage. This cyclical approach ensures that data systems and analytical capabilities evolve in alignment with business goals.

Designing a Decision Support System

To create an information system that improves organizational efficiency and effectiveness by integrating people, organizations, and technology.

When to use: When an organization needs to develop a new decision support system or significantly re-evaluate an existing one.

  1. Step 1Identify the problem and motivate its solution.

    Entry: A business need or opportunity has been recognized.

    Exit: The problem is clearly defined and the motivation for solving it is documented and approved.

    In: Business needs, Organizational strategies · Out: Problem statement, Project motivation document

  2. Step 2Define system goals in the context of business needs.

    Entry: The problem and motivation are defined.

    Exit: A clear set of system goals aligned with business needs is documented.

    In: Problem statement · Out: System goals document

  3. Step 3Design and develop the system artifacts.

    Entry: System goals are defined.

    Exit: The system artifacts (e.g., models, software components) are created.

    In: System goals, Technical requirements · Out: System artifacts (e.g., software, models, methods)

  4. Step 4Demonstrate and use the artifacts to solve the problem.

    Entry: System artifacts have been developed.

    Exit: The system has been used to address the identified problem.

    In: System artifacts, Business data · Out: Problem solution demonstration

  5. Step 5Evaluate the solution.

    Entry: The system has been demonstrated.

    Exit: An evaluation report on the system's effectiveness is complete.

    In: Demonstration results, System goals · Out: System evaluation report

  6. Step 6Communicate the results.

    Entry: The system has been evaluated.

    Exit: Communication to stakeholders is complete.

    In: System evaluation report, Problem statement · Out: Project documentation, Stakeholder communications

Developing a Business Intelligence System (BI Life Cycle)

To build, implement, and utilize a BI system that meets an organization's information needs and supports its strategy in an iterative lifecycle.

When to use: When an organization decides to implement or overhaul its Business Intelligence capabilities.

  1. Step 1Define the BI venture, vision, and strategy.

    Entry: An organizational need for improved business intelligence has been identified.

    Exit: A clear BI vision and strategy document is approved.

    In: Business strategy, User information needs · Out: BI vision and strategy document

  2. Step 2Identify and prepare data sources.

    Entry: The BI vision and strategy are defined.

    Exit: Data sources are identified and a plan for the data warehouse is complete.

    In: BI strategy, Existing data sources (internal and external) · Out: Data source inventory, Data warehouse design

  3. Step 3Select BI tools and technologies.

    Entry: Data sources and high-level design are understood.

    Exit: A suite of BI tools has been selected and procured.

    • Which BI platform to use (e.g., Business Objects, Microsoft, Oracle, SAS).

    In: BI strategy, Technical requirements · Out: Selected BI toolset

  4. Step 4Design, implement, and provide training for the BI system.

    Entry: BI tools have been selected.

    Exit: The BI system is implemented and users are trained.

    In: Selected BI toolset, Data warehouse design · Out: Implemented BI system, Trained user base

  5. Step 5Explore and discover new knowledge and needs.

    Entry: The BI system is operational.

    Exit: New information needs and potential system improvements are identified.

    In: User interactions with the BI system · Out: Evaluation of current BI system, List of new information needs

  6. Step 6Consume BI outputs for decision-making.

    Entry: The BI system is providing analyses and reports.

    Exit: BI outputs are regularly used to inform business decisions.

    In: BI reports and analyses · Out: Informed business decisions

  7. Step 7Optimize business activities and processes.

    Entry: Decisions are being informed by BI.

    Exit: Business processes are measurably improved.

    In: Informed business decisions · Out: Optimized business processes

  8. Step 8Drive fundamental organizational change and iterate.

    Entry: Processes have been optimized.

    Exit: Strategic changes are implemented and the BI development cycle is re-initiated.

    In: Process optimization results · Out: New business models, New strategic initiatives

Execute the ETL (Extraction-Transformation-Load) Process

To obtain data from various sources, clean and standardize it, and load it into a data warehouse for analysis, ensuring data quality and consistency.

When to use: When data needs to be moved from operational systems and other sources into a central repository for analysis.

  1. Step 1Extract data from source systems.

    Entry: Source systems and required data have been identified.

    Exit: All required data has been extracted and saved to a staging area.

    In: Data from source systems · Out: Raw data in a staging database

  2. Step 2Transform the data.

    Entry: Data has been extracted to a staging area.

    Exit: Data is cleaned, standardized, and in the target format for the data warehouse.

    In: Raw data in a staging database · Out: Transformed and cleaned data

  3. Step 3Load data into the data warehouse.

    Entry: Data has been successfully transformed.

    Exit: The data warehouse is updated with the new, transformed data.

    In: Transformed and cleaned data · Out: Updated data warehouse

Managing the Big Data Analysis Lifecycle

To systematically acquire, process, analyze, and interpret large, complex datasets (Big Data) to extract business value.

When to use: When an organization needs to derive insights from Big Data sources like social media, IoT devices, or web logs.

  1. Step 1Acquire and record data.

    Entry: Relevant Big Data sources have been identified.

    Exit: Data is captured and stored in a suitable repository.

    In: Data streams from various sources (e.g., IoT, social media) · Out: Stored raw data

  2. Step 2Extract, clean, and annotate data.

    Entry: Raw data has been acquired.

    Exit: Data is cleaned and in a usable format for analysis.

    In: Stored raw data · Out: Cleaned and structured data

  3. Step 3Integrate, aggregate, and represent data.

    Entry: Data has been cleaned.

    Exit: Data is integrated and properly represented for modeling.

    In: Cleaned and structured data · Out: Integrated dataset

  4. Step 4Analyze the data and build models.

    Entry: Data is integrated and represented for analysis.

    Exit: Analytical models are built and patterns are identified.

    • Which analytical technique to apply (e.g., classification, clustering, regression).

    In: Integrated dataset · Out: Analytical models, Identified patterns and trends

  5. Step 5Interpret the results.

    Entry: Analysis and modeling are complete.

    Exit: Actionable business insights are generated and communicated.

    In: Analytical models, Identified patterns and trends · Out: Interpretation report, Business recommendations

Conducting Customer Lifetime Value (LTV) Analysis

To determine the total value of a customer over time, taking into account their full history and expected future benefits, in order to increase their value to the organization.

When to use: When an organization wants to move beyond simple profitability analysis to a long-term view of customer value.

  1. Step 1Collect data about customers and their purchasing behaviors.

    Entry: A need to understand long-term customer value is established.

    Exit: A comprehensive dataset of customer behavior is compiled.

    In: Customer transaction history, Customer interaction logs · Out: Customer behavior dataset

  2. Step 2Analyze data to segment customers and define profiles.

    Entry: Customer behavior data has been collected.

    Exit: Customer segments and profiles are defined and documented.

    In: Customer behavior dataset · Out: Customer segments, Customer profiles

  3. Step 3Prepare a personalized offer.

    Entry: Customer profiles have been defined.

    Exit: A personalized offer for each target segment is ready.

    In: Customer profiles, Product/service catalog · Out: Personalized marketing offers

  4. Step 4Present the offer to the customer.

    Entry: Personalized offers have been prepared.

    Exit: The customer has received the offer.

    In: Personalized marketing offers · Out: Customer communication

  5. Step 5Handle customer contact and transactions.

    Entry: The customer has responded to the offer.

    Exit: The customer interaction and any transaction are completed and documented.

    In: Customer response · Out: Transaction records, Updated customer data

Assessing Business Intelligence and Big Data Maturity

To assess an organization's current capabilities in BI and Big Data, identify areas for improvement, and create a roadmap to guide development toward a more mature state.

When to use: Periodically, to benchmark BI/BD capabilities and plan strategic improvements.

  1. Step 1Select an appropriate maturity model.

    Entry: A decision has been made to formally assess BI/BD maturity.

    Exit: A specific maturity model has been selected for use.

    • Which maturity model to use (e.g., TDWI, Gartner, IBM, Dell).

    In: Organizational goals for BI/BD · Out: Selected maturity model

  2. Step 2Define the scope of the assessment.

    Entry: A maturity model has been selected.

    Exit: The scope of the assessment is clearly defined and agreed upon.

    In: Selected maturity model · Out: Assessment scope document

  3. Step 3Conduct the assessment to determine the current state.

    Entry: The assessment scope is defined.

    Exit: Data on current capabilities has been collected and analyzed.

    In: Assessment scope document, Input from stakeholders · Out: Raw assessment data

  4. Step 4Determine the current maturity level.

    Entry: Assessment data has been collected.

    Exit: The organization's current maturity level is identified for each assessed dimension.

    In: Raw assessment data, Maturity model level definitions · Out: Current state maturity assessment report

  5. Step 5Identify gaps and define a target maturity state.

    Entry: The current maturity level is known.

    Exit: Gaps are identified and a target maturity level is defined.

    In: Current state maturity assessment report, Business strategy · Out: Gap analysis, Target state definition

  6. Step 6Develop an improvement roadmap.

    Entry: Gaps and a target state have been defined.

    Exit: An actionable roadmap for BI/BD improvement is documented and approved.

    In: Gap analysis, Target state definition · Out: BI/BD improvement roadmap

A candidate measure

Business Intelligence and Big Data — derived measurement candidates

BI&BD Technology Investment and Capability

Analytics IT spend; Count of deployed BI&BD components

self-report suitability: medium

Environmental Turbulence and Information Flow Complexity

Market change rate; Competitive intensity index

self-report suitability: medium

Fact-based Analytical and Learning Culture

Culture survey indices; Frequency of fact-based decisions

self-report suitability: high

Human Analytical and Knowledge Skills

Data scientist headcount; Skill/certification levels

self-report suitability: medium

BI&BD Governance and Strategic Alignment

Presence of competence center; Strategy-alignment ratings

self-report suitability: high

Analytical Capability and Knowledge Discovery

Maturity-level placement; Volume of advanced analyses

self-report suitability: medium

Decision-Making Quality and Speed

Decision cycle time; Forecast error reduction

self-report suitability: medium

Big Data-Based Business Value Creation

Revenue from new products; Cost savings; Retention gains

self-report suitability: medium

Organizational Success and Competitive Advantage

Profit margin; Market share; Growth rate

self-report suitability: low

Run the assessment

The story

The reader A manager, researcher, or ICT specialist who wants their organization to succeed by turning information into competitive advantage.

External problem

Organizations are drowning in dispersed, inconsistent, fast-growing data yet under-utilize the potential of BI&BD to support decisions.

Internal problem

They feel uncertain whether success is luck or something that can be planned, and overwhelmed by complexity and rapid technological change.

Philosophical problem

It is wrong to let valuable information and knowledge—an organization's most strategic resources—go unused while competitors create new value from them.

The plan

  1. Understand the changing environment and treat information and knowledge as strategic resources.
  2. Learn the essence, architecture, and models of Business Intelligence systems.
  3. Master the techniques and technologies needed to analyze Big Data.
  4. Identify the application areas where BI&BD deliver value in your industry.
  5. Assess your BI&BD maturity, address critical success factors, and build a strategy for Big Data-based value creation.

Success

  • Faster, better decisions at operational, tactical, and strategic levels.
  • Improved customer relationships, business processes, and competitive advantage.
  • A fact-based, learning, and creative culture that continuously creates value from data.

At stake

  • Isolation and exclusion from economic and social activity for organizations that fall behind technologically.
  • Lost value as information and knowledge go unused.
  • Failed BI&BD projects due to weak culture, skills, and strategy.

Questions this book answers

What determines the success of Business Intelligence and Big Data use in organizations?
How should BI&BD systems be designed and implemented to support decision-making at all levels?
What are the potential areas of BI&BD application across industries?
How can organizational BI&BD maturity be measured and assessed?
How do organizations create durable business value from Big Data resources?

Glossary

BI&BD Technology Investment and Capability
The extent to which an organization invests in and deploys Business Intelligence and Big Data technologies and infrastructure.
Environmental Turbulence and Information Flow Complexity
The degree of dynamism, competition, and information overload in the organization's environment.
Fact-based Analytical and Learning Culture
Shared values, leadership support, and norms favoring information gathering, analysis, sharing, learning, and creativity.
Human Analytical and Knowledge Skills
The technical, statistical, business, and interpersonal competences enabling effective BI&BD use.
BI&BD Governance and Strategic Alignment
The management mechanisms, sponsorship, vision, and strategy that align BI&BD with business goals.
Analytical Capability and Knowledge Discovery
The enacted capacity to explore and exploit data to discover knowledge and inform action.
Decision-Making Quality and Speed
The improvement in timeliness, accuracy, and scope of decisions enabled by BI&BD.
Big Data-Based Business Value Creation
The economic and social value generated through BI&BD, including innovation, process optimization, and improved relations.

Tools these methods power