library / libf40423647d365cdb
People Analytics & Text Mining with R
Mong Shen Ng
In a sentence
A practical, beginner-friendly guide to using free R software to run People Analytics, predictive HR modeling, social media mining, and text/sentiment analysis to link HR levers to business outcomes.
This book demystifies People Analytics for HR professionals with no prior programming experience by teaching them R step-by-step alongside a structured five-step ARHAT analytics framework. It bridges statistical theory and hands-on application, showing readers how to run correlation, multiple regression, and logistic regression in R to predict outcomes like employee flight risk, customer satisfaction, performance, sales, and diversity's impact on revenue. Packed with real-world case studies (Deloitte, Best Buy, ISS, Nielsen, Rentokil, Xerox), data storytelling guidance, Facebook Graph API mining, and word/sentiment cloud generation, it equips analysts to uncover relationships between people factors and business results and to communicate those insights persuasively to stakeholders.
The four lenses
- Science
- Statistics
- Systems
- Strategy
Tags
The model
An inferred causal framework where HR design levers and conditions influence psychological and behavioral states that in turn drive business outcomes, validated through correlation and regression in R.
Employee Engagementpsychological state
The degree of emotional commitment, motivation, and involvement employees have toward their organization, frequently measured via surveys and eNPS and repeatedly linked to outcomes throughout the book.
Diversity and Inclusiondesign lever
The composition of the workforce across characteristics (ethnicity, gender, age) and the inclusive practices that give employees equal access, quantifiable via a Simpson's Diversity Index.
Learning and Developmentdesign lever
The provision and effectiveness of training programs intended to build employee skills, evaluated via Kirkpatrick/Phillips levels and linked to productivity, sales, and absenteeism.
Compensation and Paydesign lever
The level and structure of employee pay relative to market and performance, including incentives, market-ratio, and compa-ratio, linked to retention, productivity, and net income.
Personality Traitscontextual condition
Stable individual dispositions such as conscientiousness, extraversion, agreeableness, and grit measured via personality assessments and shown to predict service, performance, and retention.
Leadership Qualitydesign lever
The effectiveness of managers and leaders in setting expectations, communicating, and supporting teams, accounting for large variance in engagement and influencing turnover and productivity.
Internal Network and Communicationbehavioral pattern
The breadth and depth of an employee's relationships and communication patterns within the organization, including exposure to managers and senior leaders, predictive of sales and performance.
Commute and Demographicscontextual condition
Contextual employee attributes such as commute time, age, tenure, marital status, and gender used as conditions that moderate or predict turnover and accident risk.
Employee Turnover / Flight Riskoutcome metric
The likelihood and rate of employees leaving the organization, a key outcome predicted via correlation and logistic regression and costly to the business through lost productivity and knowledge.
Customer Satisfaction / Experienceoutcome metric
The level of customer happiness and loyalty (e.g., cNPS) driven by service employee engagement, training, personality, and organizational climate.
Employee Performanceoutcome metric
Individual job performance and productivity ratings influenced by engagement, training, communication, inclusion, and personality and used as a success outcome.
Sales and Profitabilityoutcome metric
Organizational financial outcomes including revenue, sales per employee, profit margin, and EBIT shown to be affected by engagement, diversity, training, and compensation.
Absenteeismoutcome metric
The frequency of unscheduled employee absence and sick days affected by inclusion, engagement, and learning opportunities and a cost-driving outcome metric.
Safety and Healthoutcome metric
Workplace safety incidents and employee health/wellbeing outcomes influenced by engagement, age, tenure, air quality, and incentives.
Data Storytelling and Stakeholder Communicationdesign lever
The structured combination of data, visuals, and narrative used to communicate insights and recommendations so that analytics drives stakeholder action and change.
How they connect
- employee engagement → predicts customer satisfaction
- employee engagement → predicts sales profitability
- employee engagement − predicts employee turnover
- employee engagement − predicts absenteeism
- employee engagement → predicts safety health
- diversity inclusion → predicts sales profitability
- diversity inclusion − predicts absenteeism
- diversity inclusion → predicts employee performance
- learning development → predicts sales profitability
- learning development → predicts employee performance
- learning development − predicts absenteeism
- compensation pay − predicts employee turnover
- compensation pay → correlates sales profitability
- personality traits → predicts customer satisfaction
- personality traits → predicts employee performance
- personality traits − predicts employee turnover
- leadership quality → predicts employee engagement
- leadership quality − predicts employee turnover
- internal network → predicts sales profitability
- internal network → predicts employee performance
- commute demographics → moderates employee turnover
- commute demographics → predicts safety health
- data storytelling → moderates sales profitability
The process
The book provides a comprehensive playbook for HR professionals to leverage data for business value using the R programming language. The core of this playbook is the five-step ARHAT framework, a structured methodology that guides practitioners from defining a business problem to delivering actionable, data-driven insights. This central process begins with asking the right business questions and formulating hypotheses, then moves to data gathering and analysis, and culminates in communicating the findings through effective data storytelling. The ARHAT framework is supported by a practical toolkit of specific analytical techniques taught in R. Before starting, the practitioner sets up their R environment. Then, within the analysis phase of a project, they can apply various statistical methods such as correlation, multiple regression for predicting continuous outcomes, and logistic regression for categorical outcomes like employee turnover. The book also details specialized procedures like text mining to analyze qualitative feedback, calculating a diversity index to quantify workforce composition, and even mining data from social media platforms like Facebook. Together, these processes form a complete, end-to-end workflow. A practitioner starts with a business challenge, uses the ARHAT framework as their road map, selects the appropriate R-based analytical tool to test their hypotheses, and finally, translates the numerical results into a compelling narrative with clear visuals to influence decision-making and drive organizational change.
Set Up R Environment for People Analytics
To install and configure the necessary open-source software (R, RStudio) and add-on packages required to perform statistical analysis and data visualization for people analytics.
When to use: This is the initial, one-time setup process performed before conducting any data analysis in R.
Step 1Download and install the R programming language from the Comprehensive R Archive Network (CRAN).
Entry: Access to a computer with an internet connection.
Exit: R is successfully installed on the computer.
Out: Installed R software
Step 2Download and install RStudio Desktop.
Entry: R has been successfully installed.
Exit: RStudio is successfully installed and can be opened.
Out: Installed RStudio software
Step 3Familiarize with the RStudio interface.
Entry: RStudio is open.
Exit: User understands the function of each pane in the RStudio interface.
Step 4Install necessary R packages for analytics.
Entry: RStudio is open with an internet connection.
Exit: Required packages are installed and can be loaded using the `library()` command.
In: List of required packages · Out: Installed R packages
Conduct a Predictive HR Analytics Project (ARHAT Framework)
To provide a structured, five-step methodology for executing an end-to-end HR analytics project that addresses a business problem and delivers actionable insights.
When to use: When initiating any new HR analytics project to solve a business problem.
Step 1Ask Questions.
Entry: A potential business issue or opportunity has been identified.
Exit: A well-defined business question and project scope are agreed upon with the project sponsor.
In: Business problems or goals · Out: Defined project scope, Stakeholder agreement
Step 2Review Literature.
Entry: The business question has been defined.
Exit: A summary of existing research and common approaches to the problem is compiled.
In: Project topic · Out: Literature review summary
Step 3Formulate Hypothesis.
Entry: Literature review is complete.
Exit: A testable hypothesis is formulated and agreed upon with stakeholders.
In: Business question, Literature review findings · Out: Formal hypothesis statement
Step 4Analyze Data.
Entry: A testable hypothesis has been formulated.
Exit: Data analysis is complete and insights have been generated.
- Which statistical method to use (e.g., correlation, linear regression, logistic regression).
In: Hypothesis, Raw data from HRIS, surveys, etc. · Out: Cleaned dataset, Statistical analysis results, Key insights
Step 5Tell the Story.
Entry: Data analysis is complete and insights are ready.
Exit: Stakeholders understand the findings and agree on the recommended actions.
In: Analysis results and insights · Out: Presentation deck, Actionable recommendations
Perform Correlation Analysis in R
To measure and visualize the strength and direction of the linear relationship between two or more numerical variables.
When to use: During the 'Analyze Data' step of an analytics project to explore relationships and test hypotheses about associations, such as the link between engagement scores and revenue.
Step 1Prepare the data in R.
Entry: A dataset is available in a readable format (e.g., .csv).
Exit: Data is loaded into an R data frame.
In: Dataset (e.g., .csv file) · Out: R data frame
Step 2Calculate the correlation matrix.
Entry: Data is in an R data frame.
Exit: A correlation matrix is generated.
In: R data frame · Out: Correlation matrix
Step 3Interpret the correlation coefficients.
Entry: Correlation matrix is generated.
Exit: Key relationships between variables are identified.
In: Correlation matrix · Out: Interpretation of variable relationships
Step 4Visualize the correlations.
Entry: Data is in an R data frame.
Exit: Scatter plots are generated to visualize the relationships.
In: R data frame · Out: Scatter plot matrix or individual scatter plots
Perform Multiple Regression Analysis in R
To build a statistical model that predicts the value of a continuous dependent variable based on the values of two or more independent (predictor) variables.
When to use: During the 'Analyze Data' step of an analytics project to quantify the impact of several factors on a specific outcome and to make predictions.
Step 1Prepare the data in an R data frame.
Entry: A dataset with one dependent and multiple independent variables is available.
Exit: Data is loaded into an R data frame.
In: Dataset (.csv file) · Out: R data frame
Step 2Build the regression model.
Entry: Data is prepared in a data frame.
Exit: A linear model object is created in R.
In: R data frame · Out: Regression model object
Step 3Analyze the model summary.
Entry: A regression model object has been created.
Exit: The statistical significance and predictive power of the model are understood.
In: Regression model object · Out: Model summary statistics
Step 4Interpret the model coefficients.
Entry: Model summary has been generated.
Exit: The impact of each predictor variable on the outcome is quantified.
In: Model summary statistics · Out: Interpretation of the regression equation
Step 5Make predictions with the model.
Entry: A validated regression model exists.
Exit: Predicted values for the outcome variable are generated.
In: Regression model object, New data with predictor values · Out: Predicted outcome values
Perform Logistic Regression Analysis in R
To build a model that predicts the probability of a binary or categorical outcome (e.g., employee will resign or stay).
When to use: During the 'Analyze Data' step of an analytics project to calculate flight risk, predict admission, or model other yes/no outcomes.
Step 1Prepare the data in an R data frame.
Entry: A dataset with a categorical outcome variable is available.
Exit: Data is loaded into an R data frame with appropriate variable types.
In: Dataset (.csv file) · Out: R data frame
Step 2Build the logistic regression model.
Entry: Data is prepared in a data frame.
Exit: A logistic regression model object is created in R.
In: R data frame · Out: Logistic regression model object
Step 3Analyze the model summary.
Entry: A model object has been created.
Exit: The impact and significance of each predictor on the outcome's log-odds are understood.
In: Logistic regression model object · Out: Model summary statistics
Step 4Make predictions with the model.
Entry: A validated logistic regression model exists.
Exit: Predicted probabilities for the outcome are generated.
In: Logistic regression model object, New data with predictor values · Out: Predicted probabilities
Perform Text Mining and Visualization in R
To extract key themes and sentiments from unstructured text data (e.g., survey comments) and create a visual representation as a word cloud or sentiment cloud.
When to use: When needing to quickly summarize and visualize large volumes of text data to identify prevalent topics or sentiments.
Step 1Install and load required packages.
Entry: R and RStudio are installed.
Exit: All necessary packages are loaded into the R session.
Step 2Import and create a text corpus.
Entry: Text data is saved in a file.
Exit: An R corpus object is created.
In: Text file · Out: R corpus object
Step 3Clean and preprocess the text.
Entry: A corpus object exists.
Exit: The text in the corpus is cleaned and standardized.
In: R corpus object · Out: Cleaned R corpus object
Step 4Build a term-document matrix.
Entry: The corpus has been cleaned.
Exit: A term-document matrix is created.
In: Cleaned R corpus object · Out: Term-document matrix
Step 5Generate the word cloud.
Entry: A term-document matrix has been created.
Exit: A word cloud image is generated in the Plots pane.
In: Term-document matrix · Out: Word cloud visualization
Mine Facebook Data Using Graph API and R
To programmatically extract data from Facebook's Graph API for social media analytics projects.
When to use: At the beginning of a social media analytics project that requires data from Facebook.
Step 1Set up a Facebook Developer App.
Entry: A Facebook account.
Exit: An App ID and App Secret are generated.
Out: Facebook App ID, Facebook App Secret
Step 2Generate a temporary Access Token.
Entry: A Facebook Developer App has been created.
Exit: An Access Token string is generated and copied.
Out: Access Token
Step 3Authenticate R session with Facebook.
Entry: App ID, App Secret, and R environment are ready.
Exit: R provides a localhost URL to be added to the Facebook App settings.
In: App ID, App Secret · Out: Localhost URL
Step 4Complete the authentication loop.
Entry: R is waiting for authentication to complete.
Exit: Authentication is confirmed in both the browser and the R console.
In: Localhost URL · Out: Authenticated R session
Step 5Extract data using Rfacebook functions.
Entry: The R session is authenticated.
Exit: Facebook data is successfully retrieved into an R object.
In: Access Token, User IDs or page names · Out: Data frame with Facebook data
Calculate Simpson's Diversity Index
To quantify the level of diversity within a group (e.g., ethnic diversity in a team) by converting categorical data into a single numerical index ranging from 0 (no diversity) to 1 (infinite diversity).
When to use: As a data preparation step before running a regression analysis where diversity is a predictor variable.
Step 1Count the number of individuals in each category (n) and the total number of individuals (N).
Entry: Raw categorical data for a group is available.
Exit: Counts for each category and the total count are known.
In: List of individuals with their category · Out: n for each category, N (total)
Step 2Calculate the sum of n(n-1) for all categories.
Entry: Counts for each category are known.
Exit: The sum of n(n-1) is calculated.
In: n for each category · Out: Sum of n(n-1)
Step 3Calculate N(N-1).
Entry: Total count (N) is known.
Exit: The value of N(N-1) is calculated.
In: N (total) · Out: Value of N(N-1)
Step 4Calculate the final Simpson's Diversity Index.
Entry: Values from Step 2 and Step 3 are calculated.
Exit: A single diversity index value between 0 and 1 is calculated.
In: Sum of n(n-1), Value of N(N-1) · Out: Simpson's Diversity Index
A candidate measure
People Analytics & Text Mining with R — derived measurement candidates
Employee Engagement
engagement survey score; eNPS; participation rate
self-report suitability: high
Diversity and Inclusion
Simpson's Diversity Index; inclusion survey scores
self-report suitability: medium
Learning and Development
training evaluation scores; training hours; ROI
self-report suitability: medium
Compensation and Pay
market-ratio; compa-ratio; merit increase spread
self-report suitability: low
Personality Traits
Big Five assessment scores; grit score
self-report suitability: high
Leadership Quality
leadership survey items; manager rating; manager tenure
self-report suitability: medium
Internal Network and Communication
network size; management exposure time; communication frequency
self-report suitability: low
Commute and Demographics
commute minutes; age; tenure; marital status
self-report suitability: medium
Employee Turnover / Flight Risk
attrition rate; flight risk probability
self-report suitability: low
Customer Satisfaction / Experience
cNPS; satisfaction scores
self-report suitability: medium
Employee Performance
performance rating; productivity metrics
self-report suitability: low
Sales and Profitability
revenue; profit margin; EBIT; sales per employee
self-report suitability: none
Absenteeism
days absent; absence rate
self-report suitability: low
Safety and Health
incident frequency; claims ratio; sick days
self-report suitability: low
Data Storytelling and Stakeholder Communication
adoption rate; audience recall; buy-in level
self-report suitability: medium
The story
The reader An HR or rewards professional (often non-technical) who wants to use data to predict workforce outcomes and influence business results.
External problem
They lack affordable tools and programming know-how to run predictive people analytics.
Internal problem
They feel intimidated by statistics and coding and unsure how to turn data into credible recommendations.
Philosophical problem
HR shouldn't be sidelined as a cost center when people factors demonstrably drive business value.
The plan
- Install free R and RStudio and learn the minimal needed syntax.
- Follow the ARHAT five-step framework to scope and run a project.
- Use correlation and regression in R to test hypotheses and predict outcomes.
- Mine text and social media for sentiment insights.
- Communicate findings through data storytelling and actionable recommendations.
Success
- The reader predicts flight risk, performance, and engagement impact and acts preemptively.
- HR earns credibility as a strategic, data-driven partner.
- Business heads seek out the analytics team to solve people-related problems.
At stake
- HR remains reactive, viewed as a cost center, and excluded from key decisions.
- Costly turnover, low engagement, and missed opportunities persist unaddressed.
- Projects fail due to poor framing, weak storytelling, or stakeholder resistance.
Chapter by chapter
ch01Chapter 1
This chapter introduces the essential role of R programming in People Analytics, highlighting its accessibility for beginners, while simultaneously arguing for its effectiveness in conducting complex statistical analyses relevant to human resources.
ch02Chapter 2
This chapter explores a range of analytics tools useful for HR professionals, detailing their advantages, limitations, and suitability based on varying data analysis needs.
ch03Chapter 3
Chapter 3 addresses the fundamentals of statistical analysis using R, focusing on linear regression techniques for examining relationships between variables, thereby enabling predictive analytics.
ch04Chapter 4
This chapter unpacks how HR analytics evolves from basic descriptive techniques to complex predictive and prescriptive approaches, emphasizing the importance of data-driven decision-making in modern organizations.
ch05Chapter 5
Effective presentations must engage audiences through storytelling, transforming complex data into relatable narratives that highlight the benefits of proposed solutions.
- Presentations should tell a story, framing data in an accessible and engaging way that highlights both the problems and the solutions.
- The three-act structure—setup, confrontation, and resolution—is a powerful tool for crafting compelling narratives that resonate with audiences.
- Visualize data to evoke emotions and connections, rather than relying solely on raw figures or extensive texts.
- Distillation of insights is crucial; sharing too much information can overwhelm audiences, causing them to disengage.
ch06Chapter 6
Effective HR Analytics requires a strong foundation of stakeholder relationships, business acumen, and strategic communication to navigate complex organizational landscapes and ensure successful outcomes.
- Building relationships with stakeholders is essential for successful HR analytics initiatives; the project sponsor can significantly influence project success.
- The clarity in defining project goals, timelines, and budgets upfront can alleviate potential misunderstandings later in the analytics process.
- Engaging business heads not only assists in identifying relevant analytics opportunities but helps in shaping the relevance and context of the analysis.
- Acknowledging data owners and domain experts fosters collaboration and enhances the accuracy and applicability of analytics findings.
ch07Chapter 7
This chapter explores the intricate factors influencing employee turnover, highlighting the importance of predictive analytics in understanding and mitigating attrition risks within organizations.
- Employee turnover is not merely a personal choice; it is a symptom of broader organizational issues that can be addressed through strategic interventions.
- Companies utilizing predictive analytics can identify 'at-risk' employees early, allowing proactive measures to retain them before they leave.
- A positive service climate combined with employee autonomy not only improves performance but can also foster a culture of retention.
- Engaged employees are less likely to leave; thus, understanding predictors of engagement is crucial for retention efforts.
ch08Chapter 8
This chapter methodically analyzes the predictive factors of employee engagement and its correlation to turnover, leveraging data analytics to empower organizations to retain talent effectively.
- Engagement is a critical predictor of employee retention; improving engagement metrics can lead to reduced turnover.
- Utilizing data analytics, HR managers can identify specific factors contributing to employee disengagement and implement targeted strategies to address them.
- The correlation does not imply causation; organizations must critically assess their findings to determine effective interventions.
- Companies that invest in understanding engagement through analytics are likely to see improvements in overall performance and job satisfaction.
ch09Chapter 9
This chapter argues that employee engagement and customer advocacy are crucial drivers of profitability, supported by a wealth of empirical research linking diversity in the workforce to financial performance.
- High employee engagement and customer advocacy directly correlate with higher profit margins, as evidenced by ISS's findings.
- Companies that reinvest in training and support for employees witness significant financial payoffs, proving this approach is not just beneficial but essential.
- Diversity within sales teams leads to improved customer understanding and market performance, supporting the claim that representation matters.
- Organizations with a high rate of racial diversity have been shown to achieve 15 times more sales revenue than those with lower diversity.
ch10Chapter 10
In this chapter, the author explores the critical intersection of diversity and inclusion within corporate environments, evidencing their substantial impact on business outcomes including financial performance and innovation.
- A diverse workplace catalyzes innovation and can lead to substantial financial gains, as demonstrated by numerous studies.
- Inclusion is more than hiring diverse candidates; it requires creating an environment where all employees feel empowered to contribute meaningfully.
- Empirical evidence places a strong correlation between employee engagement and organizational outcomes, including profitability and customer satisfaction.
- Using innovative metrics like the Simpson's Diversity Index allows for more nuanced understanding and management of workplace diversity.
ch11Chapter 11
This chapter explores the undeniable link between employee engagement and business performance, demonstrating how engagement metrics can serve as critical indicators of profitability and customer satisfaction.
ch12Chapter 12
Despite a strong desire for training impact and ROI data among CEOs, there is a significant disconnect between what executives want to know about employee training outcomes and the actual metrics many organizations provide.
- A significant gap exists between what CEOs seek regarding training impact and what organizations measure.
- Reaction scores, while easy to collect, provide minimal value in understanding a training program's efficacy on business performance.
- The most critical measures for training evaluation should focus on behavioral changes and business results.
- Implementing advanced metrics like ROI generates actionable insights that can help justify training investments.
ch13Chapter 13
This chapter argues that understanding and applying personality traits in recruitment can significantly enhance employee performance, while also demonstrating the nuanced importance of conscientiousness and agreeableness across various job roles.
ch14Chapter 14
This chapter navigates the critical metrics and methodologies for setting and adjusting sales quotas, emphasizing the importance of data-driven decisions to enhance sales performance while ensuring competitive pay levels for sales personnel.
Questions this book answers
- How can HR professionals use free R software to run predictive people analytics?
- Which people-related levers (engagement, diversity, training, personality, pay) drive business outcomes like sales, profit, turnover, and customer satisfaction?
- How do you structure and execute an HR analytics project from question to story?
- How can you predict employee flight risk, performance, and turnover using statistical models?
- How do you mine and analyze text and social media sentiment to surface insights?
Glossary
- Employee Engagement
- The emotional commitment, motivation, and involvement employees feel toward their organization and work.
- Diversity and Inclusion
- The demographic mix of the workforce and the inclusive practices ensuring equal access and belonging.
- Learning and Development
- The provision and effectiveness of training to build employee skills and capabilities.
- Compensation and Pay
- The level and structure of employee pay relative to market and performance.
- Personality Traits
- Stable individual dispositions such as the Big Five traits and grit.
- Leadership Quality
- The effectiveness of managers in setting expectations, communicating, and supporting teams.
- Internal Network and Communication
- The breadth and depth of an employee's organizational relationships and communication patterns.
- Commute and Demographics
- Contextual employee attributes including commute time, age, tenure, marital status, and gender.
Related in the library
- Goal Setting & Team Management with OKR - Objectives and Key Results_ Skills for Effective Office Leadership, Smart Business Focus, & Growth. How to Manage Projects, People & Employees. 2nd Editionshared: Strategy
- Predictive HR Analyticsshared: Statistics · Strategy · Systems
- 12_ The Elements of Great Managingshared: Statistics · Strategy
- First, Break All the Rules_ What the World_s Greatest Managers Do Differentlyshared: Statistics · Strategy
- Human Resource Management
- Predictive HR Analytics, Text Mining Organizational Network Analysis with Excel
Tools these methods power
Related in the literature
The measurement literature behind this signal — sourced, so you can defend it.
“Plot word frequencies- to plot the frequency of the first 10 frequent words, copy and paste the following codes in your RStudio left console pane, then click enter: barplot(d[1:10 , ]$freq , las = 2 , names.arg = d[1:10 , ]$word , col ="lightblue" , main ="Most frequent words" ,…”
— People Analytics Text Mining with Rmatch 66%
“As R is developed specially for statistical analysis, you can run complicated statistical number crunching (Correlation, Multiple & Logistic Regression, etc.) by simply entering a few commands. This book covers a wide People Analytics scope (Benefits, Compensation, Culture,…”
— People Analytics Text Mining with Rmatch 65%
“17.4) Case 4: Rentokil - Hiring Sales People With Certain Traits Can Enhance Sales 17.5) Case 5: Deloitte – Characteristics of High-Performing Salesperson In Financial Services 17.6) Case 6: HBR –What Makes Great Salespeople 18) Predict Total Shareholder Returns and Company…”
— People Analytics Text Mining with Rmatch 63%
Resources: People Analytics Text Mining with R