By Dr. Sandra Mashihi, Senior consultant with Envisia Learning, Inc
Imagine you are shopping for a piece of antique furniture. Before purchasing it, you want to do some research about the product. You may want to find out about its quality and durability; how old it is; the criteria that the vendor used to identify and price the item; and the authenticity of the item.
Well, shopping for a reliable and valid 360-degree feedback assessment is similar. You want to know whether it is measuring what it says it’s measuring. You also want to know if what it is measuring is commonly used, and/or based on a solid theoretical or empirical foundation.
Most vendors and organizations creating 360 feedback assessments typically attempt to measure job relevant knowledge, skills or abilities (KSAs) that can be modified through coaching, training and education. Some might even measure personality factors that are slightly less malleable.
How the scales that underlie most 360s get derived is often a great mystery and many vendors sometimes have a hard time justifying just how the tool was developed (let alone being able to demonstrate it has adequate psychometric properties). The term “competency” as it is used today is often different from the original definition by Boyatzis (1982)1 who defined it as “A capacity that exists in a person that leads to behavior that meets the job demands within parameters of organizational environment, and that, in turn brings about desired results.” He largely used critical incident interviewing comparing high and low performers to discern important behaviors that were associated with results.
Even if we use the term “competencies” broadly to mean dimensions, success factors, key behaviors, signature strengths and KSAs, an important question is where they come from in most 360 feedback assessments. In general, the competencies measured for a 360-degree feedback assessment should be based on the following:
Theoretical model: Many 360 assessment competencies and items can be linked to particular leadership or organizational theory. For example, a leadership or management 360-degree feedback could be derived from the Transformational Leadership Model (Bass, 1985)2. Or, an assessment that is intended to measure interpersonal skills can be derived from models of emotional and social competence (Goleman, 1988)3. These 360 feedback competencies would be rationally derived based on specific theories and models available to the developer.
Job level: Is the 360-degree Feedback tool measuring appropriate competencies for the job level? For instance, the competencies for a sales or account representative would be different than those of supervisors or managers heading a sales force. Job level 360 feedback assessments would contain competencies and behaviors specific to a particular role or position in an organization and often be derived through job profiling and job analytic procedures.
Core/strategic competencies: There are common competencies within an industry or organization that are requirements for success in competitive markets. For instance, measuring sales people may require a tool with competencies that include customer service skills, persistence and consultative selling. Often 360-feedback assessments derived from core competencies would be shorter and support organizational values and strategic directions. As such, they might be developed based on strategic initiatives and behaviors in the culture that appear to contribute to market success.
Are Most Bottled Waters Sold by Vendors Really the Same?
A deep secret, not often shared with customers, is the tremendous overlap between vendor job level competency models (e.g., executive level) based on interviews, focus groups and survey approaches to identify job related skills, abilities, and success factors. In such cases, most might be interchangeable in many organizational cultures with the important being more on how these assessments are used and not what they are measuring. If you go about studying what high performing executives actually do, you are likely to come to the same group of competencies and behaviors to describe their success:
For example, the competency model behind our Executive View 360 (www.envisialearning.com/360_degree_feedback/executive_view) assessment was derived based on extensive job analysis interviews with senior level executives from diverse industries resulting in a total of 22 competencies grouped into four domains:
1) Performance leadership
2) Change leadership
3) Interpersonal leadership
4) Intrapersonal leadership
Coaches Critique:
I’ve participated in a number of organizational initiatives to define “competency models” that will serve for coaching and training leaders and other levels of talent as both an internal and external consultant. Too often, these are done by committees, tasks forces or large groups without much guidance about how to go about creating these “competencies” and much of the time and energy seems to be focused on editing competency labels and definitions and less time about creating meaningful behaviors and questions that make a difference in that organizational culture. It is important to buy or design 360 assessments that have been well constructed and utilize theoretical models, job specific skills and behaviors or support organizational plans and initiatives.
I’m all for psychometrically sound 360 feedback assessments. I’m even more in favor of having the process around them lead to enhanced self-awareness and actual behavior change. Competencies are important but it is a bit like shopping for antique furniture—some 360 feedback assessments are well constructed and others are imitations. What are your experiences with developing or using the competencies in most 360 feedback assessments?
- Boyatzis, R.E. (1982). The Competent Manager, NY: Wiley
- Bass, B. M. (1985). Leadership and performance beyond expectations. New York: Free Press.
- Goleman, D. (1998) Working with Emotional Intelligence. New York: Bantam Books.
Felix Global Corp. partners with Envisia Learning for their validated online assessments which are a powerful and easy to use suite of tools for consultants and in-house teams alike.
Dr. Sandra Mashihi is a senior consultant with Envisia Learning, Inc. She has extensive experience in sales training, behavioral assessments and executive coaching.
