There is a burgeoning interest in the measurement of workforce engagement and performance – both from academic and practitioner perspectives.  The old ways of doing an engagement survey every year or two to extract data to produce an engagement score are progressively giving way to approaches that rely on more sophisticated use of automation and analytics.

HR and OD professionals and operational managers are ‘wising up’ to these developments and expecting more insightful analysis.  

One of the most exciting developments is natural language processing. Using complex algorithms these tools can automatically analyse free text responses to produce thematic, sentiment-weighted outputs. An example is shown below:

This graph shows the analysis of approx 20,000 individual free text responses, in answer to three questions:

1. What’s the best thing about working for your organisation?
2. How do you work with your line manager to get something done?
3. What’s the biggest tension you are faced with in the workplace?

The blue columns represent the frequency of detection of the different categories of comments, shown along the horizontal axis.

The vertical axis denotes the sentiment scale – the coloured dots identifying aggregate sentiment – positive (green), neutral (grey) and negative (red) – within each theme.

Examples of these different types of comments are set out below and we have shown the contrasting perceptions of highly engaged and highly disengaged groups:

Organisational Support:

Highly Engaged: “My line manager is very supportive and empowers me to make decisions and if I need help always supports me or pulls in the rest of the team.”

Highly Disengaged: “Lack of support when new procedures are implemented- managers seem to introduce new priorities without putting any support, training, and time in place for us. Managers constantly ‘reacting’ rather than planning.”

Creating Workable Solutions:

Highly Engaged: “Through conversational practice: talking through an issue and coming up with suggestions on both sides for potential solutions.

Highly Disengaged: “Discussions on how to solve problems are encouraged, but are not actually actioned or followed through by management, so services are rarely (if ever) improved.”

Lack of Resources

Highly Engaged: “We are trying to deliver more to our service users with less resources, both monetary and people.”

Highly Disengaged: “Number of staff has halved, more is expected of staff, no resources and more time spent on administration tasks rather than working with clients.”

Job Pressure:

Highly Engaged: “There is an energy and ambition across [the organisation], which is refreshing given the challenges we have had to overcome in the last decade, on top of the pressures of austerity.”

Highly Disengaged: “I am hugely frustrated by the pressures that are put on my team to provide a certain level of service, that result from being a hugely under-resourced department.”

Management:

Highly Engaged: “My managers take an active role in developing me personally and always takes suggestions on board.

Highly Disengaged: “Lack of immediate line management engagement in a balanced way – a way that takes seriously the competing demands of practitioner/ social work values/ principles on the one hand and managerialist “needs” on the other hand.”

Team:

Highly Engaged: “Being part of a successful team that meets profound current challenges, with a shared vision of future goals.”

Highly Disengaged: “Team is spread thinly over many centres so communication is patchy sometimes and it is hard to feel a sense of belonging.”

Until these new technologies came on the scene, it was usually a case of manually inspecting comments with a skilled eye to produce some broad themes. If large numbers of comments were involved, this was often a thankless task to be avoided. It is no coincidence that many so-called survey providers (whether internal or external) concentrated on statistical data collection and analysis, to avoid the potentially time-consuming and messy nature of free text or narrative analysis.

There is no excuse for these kinds of practices today. Furthermore, the free text responses add richness to other statistical information and create further insights.

So where is all this heading?

Well, as previously stated, the traditional survey is looking increasingly anachronistic – a ponderous ‘animal’ that is under constant and ever evolving threats to its survival.

The new approaches combine:

• High levels of rigour – by which we mean really good modelling underpinned by robust theory and not simply a long list of questions resembling a potpourrithat has grown over the years by well meaning but often ill-informed suggestions.
• Highly engaging interface – use of mobile technology, for instance, that makes the experience of data collection easier and more rewarding for participants.
• Highly impactful reporting – sophisticated analysis that combines different types of data (quantitative and qualitative) in ways that really inform good quality decisions. A primary reliance on an engagement score is simply not good enough.   

These features are only possible by using the best technology.

So our appeal to all decision makers looking to commission an investigation into workforce engagement and performance is to ask critical questions of any provider, whether internal or external, about their capabilities.

Do not view proof of ‘good practice’ simply in terms of a long list of clients or colourful reports with apparently vital scores in large boxes set out on the page.

Ask them about about the rigour of their model – what have they published in peer reviewed academic articles or reputable professional journals?

Ask about their technologies – just how do they analyse free text responses, for example, and how is this integrated with other data to create powerful insights you can act upon with confidence?

A technology-enabled future has the potential to revolutionise this field of enquiry – make sure you make the best of it.