From Surveys to Signals: Interpretable Machine Learning for Predicting Employee Commitment and Hybrid-Work Effectiveness in Sri Lankan IT Firms

People analytics has started trusting machine-learning explanations it has never stress-tested. HR teams increasingly fit a model, rank the "top drivers" with SHAP, and act on the ranking — even though the underlying survey constructs (leadership, trust, communication, expectations) are correlated by design. This paper asks the question no one had tested on HR data: when features are this correlated, are the resulting driver rankings actually stable?

The study builds on the dataset collected for my earlier Gen Z Job Commitment Study, adding a second, independently collected dataset on hybrid-work effectiveness (n = 375). Three model families — linear regression, random forest, and gradient boosting — were evaluated through repeated cross-validation, and driver importance was estimated three separate ways: correlation, standardised linear coefficients, and SHAP. Rankings were then stress-tested across 75 resampled folds and a second ensemble type to check whether they actually held up.

They didn't, not fully. The three attribution methods disagreed on which construct mattered most, and the SHAP rankings shifted under resampling. But one result held: a model built on just two of the four constructs retained 96 to 100 percent of the full model's predictive accuracy — and the same two-construct pair was selected in 100 percent of resampling runs. That result anchors a proposed privacy-preserving scoring tool: a two-item pulse survey that can monitor commitment or hybrid-work effectiveness at a fraction of the length of a full instrument.