Leadership Assessment with 360-Degree Feedback: Ensuring a High-Quality Rater Group

This time of year, many of Censeo's clients turn their attention to leadership assessment via 360 degree feedback.  It's a great time to take stock of the past year, establish a new benchmark of leadership performance, and use assessment results to identify and prioritize goals for the coming year.

One important factor in conducting an effective and accurate 360 degree feedback process involves the selection of high-quality raters - raters who are well positioned to provide candid and objective feedback.  So how can organizations accomplish this?  If 360 participants are given full control to assign their own raters does the quality of the process suffer?

In this short video, Censeo's Director of Consulting, Dr. Michael Zia Mian addresses this issue, and offers a specific best practice of ensuring a high-quality 360 rater population.

360 Degree Feedback in M.B.A. Programs – Helping to Deliver on Skills Employers Value Most

There has been a lot of press recently focusing on the curriculum in MBA programs.  To summarize, employers have been quite vocal in demanding a curriculum shift at the B-schools. Hard core quant skills are less in vogue as corporate recruiters are placing more emphasis on the skills employers value most - like the ability to write and speak clearly and persuasively, brokering compromises, and managing change. Business schools are reacting by bolstering their soft-skills training and adopting popular corporate employee development methods, like the use of psychological (personality profiling) and behaviorally (360 Degree Feedback) based assessments, and professional coaching.

At Rollins College Crummer Graduate School of Business, Dr. Susan Bach, the Executive Director of the Leadership Center, has been using 360 Degree Feedback for several years with her students. Dr. Bach believes "that 360 assessments are an essential leadership development tool. Many MBA programs across the country are beginning to focus on this approach as a way to call attention to the importance of soft skills."

In peer-oriented MBA programs, 360 Degree Feedback might be a new experience for students. The process at Rollins has students rating themselves and each other on their performance and effectiveness as team members. The competencies and behaviors assessed in the 360 align with what's needed to be a successful manager in any corporation today. "I have found that students see the process as an eye-opener and the feedback as the wake-up call," reports Dr. Bach. "The experience of seeing the ratings and reading the feedback makes students pause and take notice.  I think it's rare that a student is totally surprised by the information, however, seeing it on paper creates a heightened awareness that oral feedback can't always achieve."

For some schools, the assessment process also includes feedback from the student's employer. The resulting feedback report provides the framework for designing a development plan and for identifying subsequent coaching sessions - all common techniques in corporate development programs. Thoughts and reactions to the feedback are also captured in student journals, which are maintained throughout the program.

A recent graduate from the Rollins' Corporate MBA program, Tanya Juarez, is the External Affairs Manager for TECO Energy, a power company in Tampa, Florida. "Because my company incorporates a 360 assessment already, I was familiar with the process. The experience of the assessment during the MBA program only furthered my understanding of the strengths I needed to leverage more and the areas I needed to continue to work on.  This entire exercise, if you allow it, will provide clarity of how to become a more effective worker, manager, and leader."

Dr. Michael Zia Mian, Director of Consulting for Censeo Corporation, has worked with schools in developing post-360 processes for the classroom. "While students might have already been exposed to assessment instruments like the MBTI or the Birkman, the 360 feedback report is different - it is eagerly received, relatively easy to understand, and provides students with actionable development resources early on in their program. It is the one report they seem to revisit throughout their MBA experience as they monitor their progress." 

Are MBA programs meeting the needs of employers?  Don Davidson, founder of The Davidson Group (www.davidsongroup.com), a full service executive recruiting firm based in Novato, California, believes that business schools could do more.  "We deal with a very educated clientele.  There is very little tolerance for people with poor communication skills both written and oral.  Negotiation is a key skill that is highly valued among most of our clientele." A good 360 process can help identify weaknesses in these areas and provide the school and the individual with a road map for development.

The Graduate Management Admission Council's 2009 MBA Recruiters' Survey provides further support for the development of soft skills in MBA programs. Results indicate that employers hiring MBA graduates, are primarily looking for good oral and written communication skills (89%), a proven ability to perform (73%), strategic skills (69%), and sound core business knowledge (67%). In addition, initiative (82%), professionalism (81%), and integrity (81%) were among the most demanded intangible traits and abilities sought in MBA candidates."

While personally pursuing an MBA several years ago, the impact of the 360 assessment in my program became obvious. As the program went along, I became keenly aware of how I could more effectively engage people on my project team. I also noted the reaction of my peers to the process.  When the resulting reports were delivered, the common reaction was, "Wow...the insight into my performance gave me the opportunity to reevaluate and change my approach to more effectively use my leadership skills and to develop additional skills where I was lacking." Indeed, the 360 process proved to be one of the most rewarding parts of the MBA program. I'm hoping incoming students will have the same opportunity I had, to really jump start their soft skill development and growth!

Censeo Corporation strongly embraces the mission of business schools and their need to properly prepare graduates for their role as our future leaders. Contact us about our special educational pricing.

The Value of Employee Assessment (Part 5 of 5)

So... this will wrap up my five-part series on the Value of Assessment.  I thought I'd leave you with a few summary thoughts. 

As the business environment becomes more competitive, people, not processes and technology, increasingly become the differentiators. As a result, it is more important than ever to implement the best possible assessment systems in order to select, retain and develop talented people.

The value of assessment in this kind of environment is that it gets the "right people on the bus" and the "wrong people off," thereby decreasing the costs associated with bad hires and increasing revenue potential as a result of talented employees. In addition, assessment plays a vital role in employee development as it streamlines the development process by targeting KSAs in need of development.

At the individual level, assessment systems increase the probability of job fit, provide a road map for development, and open the door for rewards and advancement.

Finally, HR needs to become more strategic in influencing the organization's decisions. Assessments can help in this regard because they quantify the value of HR programs and services. In order to demonstrate true value, HR needs to measure the impact of programs on the key metrics driving organizational success.

As always, I welcome your thoughts... let us know if you have any opinions or have a new perspective you can add to this topic!

The Value of Employee Assessment (Part 4 of 5)

If you’ve been tracking this series of posts, you know we’ve covered some historical perspective on the use of assessments, and spent a couple weeks digging deeper into the world of employee selection.  This week, we’ll shift our focus to a brief discussion of how assessment is being used to support ongoing employee development.

While people typically associate the term "assessment" with selection, organizations are increasingly using assessments for developmental purposes. Here are a few of the ways assessments are adding value:

• Ensure employees have acquired the necessary knowledge and skills during training. Organizations spend about $60 billion on training each year, and a modest investment in assessments can significantly leverage those resources. The data are also very helpful in fine-tuning training programs.
• Better target training resources. Rather than assuming that all employees need the same training, assessments can help identify the specific areas where training would be most beneficial on an individual basis.
• Promote continuous learning and development. Good assessment programs periodically measure employees' capabilities and provide specific feedback to help them further develop. Some assessments measure the degree to which knowledge and skills are actually being applied on the job, which, of course, is the real point of development.
• Empower managers. Managers are just as responsible for developing the talent pools in their organizations as they are for accomplishing operational goals. Unfortunately, while they get a lot of information pertaining to how they're doing on operational goals, they get no information on the knowledge and skills of their people. Good assessment programs can provide this valuable data.
• Improve organizational effectiveness. For example, a large financial institution Censeo works with administers a product knowledge test every six months to all of its employees who interact with customers. Follow-up research showed very high relationships between product knowledge and a variety of financial measures. This propelled the organization to an even greater commitment to employee development.

Increasing the Value of Assessments in Development

This graphic illustrates the strategies involved in increasing the value of assessments.

The main point relevant to this discussion is how assessments can be used strategically to build a performance-oriented organizational culture. Here are a few suggestions on how to accomplish that:

1. Understand the business drivers and strategies of the organization. Examine what the organization is trying to achieve and what metrics will be used.
2. Identify the activities to be performed and the KSAs employees need to carry out the strategies. Set performance targets and then measure proficiency
   on the critical KSAs using appropriate assessments.
3. Assess the gaps between actual performance and set targets, and then implement performance feedback sessions in which employees and their managers agree on the development activities necessary for building capabilities linked to the organization's strategies.
4. Once employees have gone through training and other developmental activities targeted to improve knowledge and performance, measure their acquired knowledge and skills to ensure learning took place.
5. Finally, after an appropriate interval, measure the effect of the assessment process and all of the developmental activities on the organization. Did the metrics important to the organization significantly improve? Were customers more satisfied? Were employees more content in their jobs? The process is a continuous loop, as all the data should be used to refine the business strategies, required KSAs, targets, etc.

From the employee's perspective, this performance-oriented system ensures that their development is focused on the things that matter — things that will help them do their jobs better and faster, possibly opening the door to greater rewards and faster advancement. The value of assessment to the organization is in the cost savings and productivity gains from a targeted development process. Giving employees exactly what they need in development when they need it will make the organization that much more effective.

The Value of Employee Assessment (Part 2 of 5)

The Value of Assessments in Selection

Imagine the repercussions to an airline if its pilots were improperly certified, or if city administrations failed to screen the people who police our streets and protect our property. The high levels of on-the-job failure would lead to a loss of life, harm to the public, poor public image, and eventually, serious financial implications for those organizations.

While the consequences of error in such jobs could be fatal, the consequences of error in less hazardous and less complex jobs are also significant. For example, mistakes made in hiring customer service representatives can lead to low performance, high turnover, unhappy customers, lost business and higher labor costs.

The primary reason we assess people in a selection process is to increase the probability of hiring the best candidates - people with the skills to perform well, and who are more likely to be retained. Some assessments measure knowledge, skills and abilities (KSAs), while other assessments measure personal attributes such as personality, work preferences and motivation. Both KSAs and personal attributes are important, but the latter are particularly important in reducing turnover. Call centers, for example, typically struggle with very high turnover, but it's more of a fit issue than a skills issue.

What are the costs associated with making a bad hire? It depends on the job and how the costs are calculated. The American Management Association estimates that the cost is 1.4 times an employee's annual salary. The California Strategic HR Partnership (a consortium of Fortune 500 executives) calculated that, for certain positions, the cost can be almost 10 times an employee's annual salary (approximately 1 million dollars in the case of a software engineer).

What's behind these figures? They are comprised of some obvious costs such as advertising and recruiting, administrative costs for managing the selection process, travel and relocation, and the training and development new employees need to get up to speed. However, the less obvious costs are even higher - opportunity costs and lost business, lower performance until new hires are trained, negative impact on morale, and so forth. As the complexity and level of the job increases, the cost of a bad hire also increases. In some cases, making the wrong hiring decision at the CEO level can cost hundreds of millions of dollars. From the organization's perspective, there are huge financial benefits to including assessments in the selection process, insofar as the information will lead to better hiring and placement decisions. The individuals being assessed also benefit. Employees are more satisfied and advance further in their careers when their skills and personal attributes match the requirements of their jobs.

There are other benefits to selecting the right people. Jim Collins, author of "Good to Great," talks about "getting the right people on the bus." He notes that most people assume that great business leaders spark change by announcing to the people "on the bus" where they're going - by setting a new direction or by articulating a fresh corporate vision. Based on his research, the companies that moved from good to great were those that began their journey by asking "with whom" rather than "where." In other words, great companies start by getting the right people on the bus, the wrong people off, and the right people in the right seats.

Until next week...

The Value of Employee Assessment (Part 1 of 5)

Today I kick off a five part series on the value of leadership assessment.  Sure, folks tend to agree that assessing skills is generally a good thing, but why?  What role does assessment play and what value does it lend from a business perspective?  These are the questions I'd like to address over the next few weeks... hope you find it helpful, and I'd love to hear your thoughts on the topic.

With that, let's get started with a short post to provide some historical perspective.

For thousands of years, humans have measured work-oriented skills. As far back as the Chan dynasty in 1115 BC, civil servant candidates were subject to competitive examinations to test their proficiency in areas such as archery, horsemanship, military affairs, civil law, writing and arithmetic - the knowledge, skills and abilities (KSAs) deemed important for success in a civil servant role.

It wasn't until World Wars I and II, however, that the science of assessing people for the purposes of selection, placement and training was applied on a truly large scale. The US military used aptitude tests, among other tools, to quickly and efficiently place recruits into roles that matched their abilities. They also developed and used simulations (assessment centers) to screen and train secret agents. The value of assessment was quite evident at that time because of the large number of soldiers that needed to be quickly and effectively screened.

Today organizations spend millions of dollars on assessment tools to help attract, select, promote, train and develop their staff. The motives for assessing people are related to several criteria including productivity, retention, absenteeism, legal requirements and employee development.

Next week we'll get into the meat of the topic by discussing the use of assessment to select better leaders.

The US Supreme Court Ruling on Ricci v. DeStefano – An overview of the case and what the decision really means for test users (Part 1 of 2)

I've been reading quite a bit about the US Supreme Court (USSC) ruling over the last few weeks and have received questions from our clients on what the case was really about and what the ruling might mean for them.

This week I'll provide a brief overview of the case for our readers (for specific details, visit: http://www.lawmemo.com/supreme/case/Ricci/) and in part 2 of my post next week I'll discuss some of the issues with the test and the implications of the case. Here goes...

In 2003, 118 firefighters took examinations to qualify for promotions to the rank of lieutenant or captain in the New Haven Fire Department. The results of the exams showed that white candidates passed the exam at a much higher rate than African American candidates... by almost a 2 to 1 margin. A public debate ensued where some firefighters argued that the test results should be discarded as they appeared to be discriminatory and threatened to file a lawsuit if the City made promotions based on the tests. Another group of firefighters said the exams were fair and threatened a discrimination lawsuit if the City ignored the test results and denied promotions to the highest scoring candidates.

After looking at the disparity in results and some design flaws in the test, the City made the choice not to certify the test results. Here now are the various lawsuits and court rulings leading up to the USSC's decision: 

1. White (and 1 Hispanic) firefighters responded to the City's decision by filing a reverse discrimination suit claiming that discarding the results discriminated against them and was in violation of Title VII of the Civil Rights Act and the Equal Protection Clause of the Fourteenth Amendment.

2. The City defended their stance and argued that if they had used the results to make promotions, they could have been in violation of Title VII by using a selection method that demonstrated disparate (adverse) impact. The case was heard by the New Haven District Court and the plaintiffs' (white firefighters) claims rejected, citing prior cases where "race conscious" interventions were used to remedy adverse impact.

3. The 2^nd Circuit Court upheld the District Court's decision.

4. Enter the US Supreme Court (USSC), which then reversed the decisions of the lower courts (by 5 to 4 margin) stating that the City of New Haven did not have enough evidence/justification ("strong basis in evidence") to throw out the results for fear of being potentially sued for discrimination.

Essentially, the USSC's decision sets the potential precedent that test users cannot just discard results when there is a hint of adverse impact. The case further brings to light the looming conflict between remedying adverse impact (race based decision) and facilitating equal protection (creating a color blind society). It seems like the City of New Haven was in a "no win" situation... either avoid the adverse impact suit at the risk of facing disparate treatment claims or vice versa.

Next week I'll discuss some of the issues with the firefighter exams, why these issues were not fully considered in the USSC's arguments and also what test users should take away from the case.

Leadership Assessment Validity: What is it? Why Does it Matter?

Rarely does a week go by that one of our clients doesn't ask questions around the validity of assessments, be it leadership assessments or other tools... and the topics of validity and fairness have become even more front-and-center as a result of the recent Supreme Court decision in Ricci vs. DeStafano.

In my opinion, all this attention is a good thing... and clients should always be asking vendors these questions about validity.  Why?  Well, the driving factor in many cases is a company's desire to protect itself in the event of legal action, and that's understandable in today's litigious environment. 

However, there's a larger, extremely important, issue regarding the validity of assessments which companies seem to give less attention.  Simply put, valid assessments work better!  They are scientifically proven to predict on-the-job success or failure better than their non-valid (or unvalidated) counterparts.  They are also more accurate in assessing an employee's unique strengths and areas for improvement.

What's this mean for companies?  Better quality hires, more targeted employee development, and better ROI - just to name a few things.

Given it's such a timely topic; we thought you might benefit from a concise, practical definition of "validity."  In this video, I take a couple minutes to interview Industrial and Organizational Psychologist, Dr. Michael Zia Mian on the topic.

Hope you find it beneficial!

Also - if this topic is of interest, be sure to see our next two posts (August 28^th and September 4^th).  Dr. Zia Mian will be discussing the Ricci vs. DeStafano case in detail, and it's implications on HR practitioners.

Should 360 Degree Feedback be used in Performance Appraisals? (Part 2 of 2)

It's hard to refute the basic logic of arming both the manager and the employee being appraised with the most accurate data possible, and to use that information intelligently to help understand past performance and determine how performance can be improved in the future.

360 degree feedback can provide such data. Here are three suggestions on how to capitalize on that opportunity, while minimizing the risks:

1. Initially use 360s only for development - If 360 degree feedback is new to your organization, focus on using the process for development purposes only for a couple of years. Give people time to get used to the whole concept of 360 degree feedback, and to appreciate how valuable such feedback can be. Then consider the possibility of using it in some fashion as part of the PA system.

2. Don't tightly link 360s to the PA system - This means several things. First, 360 degree feedback should be only one of the data sources used by the manager in evaluating performance. Second, 360 degree feedback results should never be hard-wired to PA ratings in the sense that there's any kind of mathematical relationship. Third, don't link the timing of 360 degree feedback to the timing of PAs. Most PA systems use an annual cycle tied in with the company's fiscal year, but conducting 360 degree feedback every year is too often. Conduct 360 surveys when it makes sense to do so for development purposes. Then use the results as one of the inputs to the PA system, if they happen to be timely enough.

3. Clearly communicate how the 360 tool will be used - Clearly communicate to participants and raters how the 360 degree feedback will and won't be used with respect to the PA system. Don't underestimate the challenges in communicating these matters, particularly if you're announcing a policy change (e.g., a shift from "strictly for development purposes" to something else). It will take numerous and repeated communications to allay the fears and concerns on the part of all stakeholder groups - participants, managers and raters.

A reminder to those of you thinking that I've just provided great rationale for you to now quickly integrate your 360 and PA systems, or maybe you've been integrating them for some time and this just validates your stance - the success of the integration really will depend on following the guidelines mentioned and paying close attention to what happens during and after the performance review. A well planned structure is still necessary to ensure that employees actually engage in developing their competencies. While there is too much to cover in this post, some critical questions you should be asking include - Do employees understand your performance review process and how 360 ratings will be used? Are managers equipped to conduct effective feedback discussions in the review? Is there a development planning template and is regular follow-up part of your process?  

It's the age old question ... how effective is your system in helping people change behaviors? Essentially.... that's the bottom line!

Should 360 Degree Feedback be used in Performance Appraisals? (Part 1 of 2)

In the second half of every year, as annual performance reviews approach, I'm frequently faced with the client question "is it appropriate to integrate employee performance appraisals (PA) and 360 results?" There is considerable debate and controversy about the relationship between the two, but from where I stand, 360 degree feedback can play a beneficial role in PA systems, if several issues are carefully managed.

Before covering those issues, let me first describe the broader context of PA systems and one of the reasons they often don't work very well. Most PA systems include two components - evaluation of operational/business goals to be achieved and evaluation of performance factors (how the job is accomplished and the competencies required). Typically evaluations of both components are then integrated into some overall "score" or rating, and there are usually consequences linked to those ratings, such as merit salary increases, bonus allocations, and so forth.

The rationale for looking at performance factors is that companies want managers and their employees to place an emphasis on improving competencies and skills, in addition to operational goals, because improving them increases the potential for future business success.

Let's now talk about why this traditional PA process often doesn't work very well. The one most relevant to my post is that evaluations of performance factors (the behaviors and competencies demonstrated) are not as accurate when the assessment information comes only from the manager. (This is usually less of a problem for operational objectives because more quantitative, objective measures are available.) In an effort to improve this important part of their PA systems, some companies use 360 degree feedback to increase the accuracy of assessments and evaluation. Why? Because having several different perspectives on an individual is better than just one - this is the driving premise of the 360 tool. So, what's the problem and why is there controversy? Here are the four main issues:

1. Decreased rater honesty and accuracy - When they know that significant consequences are tied to their ratings, people providing 360 ratings may be less honest and accurate (i.e., positive or negative bias could be introduced).

2. Participants focus more on ratings than development - Participants might be less inclined, not more, to proactively engage in developmental activities when their PA ratings are influenced by the 360 process. In fact, their main motivation could be to do everything possible to select raters who will be predisposed to give favorable ratings, versus those who will give honest feedback.

3. 360 results lose their "future focus" - 360 degree feedback surveys are more focused on the future (using feedback to improve future performance) whereas PAs are more focused on the past (evaluating goals accomplished and behaviors demonstrated over the past year). Therefore, when put together, there's a risk of the 360 degree feedback process becoming "tainted," and losing its positive focus on the future.

4. 360 tool loses its positive appeal - Associating 360 degree feedback with the PA system (which typically has a very negative reputation in the organization) could jeopardize positive views people hold about the former.

My general advice to clients is to be very careful about incorporating 360 degree feedback into appraisal systems. The advantages of making modest improvements in the PA system could be outweighed by the potential damage done to the 360 degree feedback program. Carefully think through what you're trying to accomplish and why, and evaluate the advantages and disadvantages of moving in that direction.

Looking ahead to next week, I'll dive deeper into how 360 degree feedback can be leveraged while, at the same time, minimizing the risks.