Are External Norms in Employee Surveys Useful? (PART 1 of 2)

When discussing employee survey or engagement survey solutions with potential clients, I often get the question "do you have external norms?"  While we do have external norms, I'm convinced most clients don't really understand what they are asking for, why they believe they need them, and how they would use external norms to better understand their survey results. In this week's post I'll present both sides of the argument - for and against external norms - and then next week provide my opinion on where I stand on the issue and why.    

Let's deal with some of the basics first....

What Are External Norms?

Employee survey vendors collect data on employee perceptions from different companies over time, sometimes on similar topic areas. The term "norm" refers to the average of those perceptions or effective responses to particular questions. They are "external" in the sense that they come from outside the company conducting a current survey. The underlying premise is that normative comparisons are helpful in understanding the results from the current survey.

Making normative comparisons is intuitively appealing. In many aspects of personal life and in business situations, we rely on such comparisons. When we list our homes for sale, we refer to market data to know what the price should be. When we determine what the compensation for a job should be, we collect data on what other companies are paying for similar jobs. If the turnover rate in our call center is 30%, we look externally in our industry to see if this is high or low.

The Case For External Norms

These are the main arguments made in favor of using external norms in employee surveys:

• Norms are important in understanding what's really positive or negative. For example, if 40% of the employees are not satisfied with their pay, is that dangerously low and something that should be addressed? A company might feel differently if normative data indicated that 40% was on par with other companies, versus the average being in the 30% range.
• Norms help overcome a problem with ranking. Normative data can help companies prioritize their actions and resource allocation. Employees are typically less favorable on some areas (e.g., pay satisfaction, opportunity for advancement) than they are on other areas (e.g., the work itself, areas related to supervision). Normative data can provide a context for understanding such differences, so effort is not misdirected to areas that really aren't significant problems.
• Making normative comparisons is aligned with other ways the business is measured. In a highly competitive business environment, companies feel a compelling need to check themselves, not just against their past performance, but also how they're doing relative to the competition. And competition in the labor market isn't just with other companies providing similar products and services, but any company drawing from the same labor pool for talent.

Based on my experience in conducting many employee surveys across different industries, I suggest there is an additional argument that's often made (though not always admitted) for including external norms. The person responsible for the employee survey initiative knows the question that is going to come from senior management: "Yeah, we can see all that, but how do we compare with other companies?" Even if it's not a particularly good question, it is going to be asked, and most project managers don't want to be left without an answer.

The Case Against External Norms

These are the main arguments made against using external norms in employee surveys:

• Norms aren't accurate or comparable. Unless the items and response scales in the normative database and the current survey are identical, the comparisons won't be accurate. Furthermore, to be comparable, the norms should be from the same industry (or same labor market), the same size companies, the same time period (i.e., not too old), and so forth. The requirements for accuracy and comparability are very rarely met, and oftentimes the norms aren't even close.
• Norms aren't available for current items. The items in the survey should be targeted at the most important areas to measure, and be linked to the business strategies. It is a grave mistake to compromise the objectives of the survey initiative by including some items, and omitting others, just because normative data are available.
• "We have our own standards of excellence." Some companies simply don't care whether they're better or worse than other companies on most areas covered in the survey; their own standards of excellence are more important. They want to identify opportunities for improvement and strive for continuous improvement, regardless of what other companies are doing.
• There are better ways to tell what's positive or negative. The rationale here is that other methods for looking at the data are preferable (internal normative comparisons, relative scores, trend comparisons, subgroup comparisons, etc.) to external norms for focusing attention on the most important issues. This is especially true in interpreting the survey results for all organizational units below the total company.

People questioning the value of external norms also make the point that if a high percentage of employees are negative in an area, those employees are still negative regardless of the perceptions of employees in other companies. Going back to the turnover example, if the turnover rate could be decreased from 30% to 25%, the company would reduce costs by an enormous amount. That's true regardless of whether the norm in the industry is 20% or 40%.

That's it for now... stay tuned next week for more on the value of external norms in employee surveys.