General Approach for Analyzing Data to Build a Competency Model

STEP 4 – DATA ANALYSIS AND CODING

Previous blogs have described steps 1 through 3 in Workitect’s competency modeling process. Step 3, Data Collection, was covered in these blogs:

Collecting Data to Build Competency Models
Secondary Data Collection Methods
Conducting Behavioral Event Interviews
Using Resource/Expert Panels to Build Competency Models

The general approach for analyzing the data collected in step 3 has three steps:

  1. Reviewing/modifying a set of generic competencies that will be used as the conceptual framework guiding the analysis.
  2. Separately reviewing the evidence from each data collection method to identify a potential set of competencies, drawn from the generic competency framework.
  3. Reviewing and comparing the evidence across methods to identify a set of potential competencies to include in the competency model.

Each of these steps is described below. Note that the data analysis ends with identifying a potential set of competencies to include in the competency model. Moving from that step to building the actual competency model will be described in a separate blog.

Reviewing/Modifying a Set of Generic Competencies to Include in the Competency Model

If you are using a Multiple Jobs Approach to building the competency model, you will have already identified a set of generic competencies to serve as building blocks for the various competency models you are planning. If you are using the Single Job Approach or the One Size Fits All Approach, you may have used a set of generic competencies that were rated by a Resource Panel or by persons with whom you did Job Analysis Interviews. If, by chance, you have not yet begun to use a set of generic competencies, now is the time to start, because a framework of generic competencies is useful in guiding the analysis.

You will not necessarily end up with competencies as they appear in the initial generic competency framework. You may identify and add competencies that were not initially part of the framework. You may modify some of the competencies from the initial framework. But using the framework will facilitate the analysis of the data.

Based on the experience of collecting the data, you may already have some ideas about competencies that need to be added to the framework. For example, if you are preparing a competency model for Project Managers, you may have decided to add Project Management Knowledge to the framework as a technical skill/knowledge competency. It may be clear that the wording used to describe certain competencies can be improved by substituting language that better fits the job or organization. For example, if the organization in which you are working uses the term, “Driving Innovation” instead of “Managing Change,” you may want to change the competency name accordingly

If it is clear at the start of the data analysis process that changes like these will be needed, you might as well modify the generic competency framework now, so that you can look for and track evidence for these new or modified competencies during the analysis process. It will still be possible to modify generic competencies and add new competencies later on.

Separately Reviewing the Evidence Generated from Each Data Collection Method, to Identify a Set of Potential Competencies Drawn from the Generic Competency Framework

In this step you separately review the evidence from each of the data collection methods that you used – Job Analysis Interviews, for example – and try to answer the following question: Based on this data collection method alone, which competencies should be considered for inclusion in the competency model?

In carrying out this analysis, you may encounter direct evidence for certain competencies from the generic competency framework. For example, all or most of the participants at a Resource Panel may select a competency for inclusion in the competency model. Or, when analyzing Structured Interviews with superior performers, you may find that their descriptions of their behavior during key past events include many examples of some competency.

But your analysis may also reveal indirect evidence that a competency is important. For example, if you learn that an important task for sales representatives is to deliver formal and informal presentations to clients, this task implies the need for the competency, Persuasive Communication. Or if you learn that a key performance measure for a job is Customer Satisfaction, this implies the importance of the competency, Customer Orientation.

Review and Compare the Evidence Across Methods to Identify a Set of Potential Competencies to Include in the Competency Model

Although there will be some overlap in the competencies that are suggested by each data collection method, there will probably be some differences, at least in emphasis. Job Analysis Interviews with managers of persons in the job may lead to the identification of competencies that are most important and visible to the job holders’ managers. Interviews with customers of the job holders will highlight competencies that are most visible and important to customers. Structured Event Interviews may reveal thought processes that lead to the identification of competencies such as Conceptual Thinking that are not as likely to be surfaced through other data collection methods.

One way to compare the competencies revealed by different methods is to construct a matrix of generic competencies by methods, like the one below. You can omit generic competencies that are not identified as important in the analysis of any of the data gathering methods that you used. In each cell, use H, M, or L to indicate whether the competency seems high, medium or low in importance, based on data from this method. In the Overall column, make a judgment about the importance of including this competency in the competency model. You do not need to weight each data collection method equally.

AN OVERVIEW OF METHODS FOR ANALYZING DATA GATHERED DROM INDIVIDUALS

Details of this process, and instruction on analyzing and coding the data collected in step 3, are provided in the Workitect Building Competency Models workshop and practiced by Workitect consultants in the development of job competency models.

Analyzing Data from Job Analysis Interviews

Analyzing Data from Resource Panels
Analyzing Data from Structured Event Interviews
General Principle 1. Recognize Target Data
Target Data is defined as behaviorally specific information, volunteered by the interviewee in response to non-leading questions, about what the interviewee did, said, thought, or felt in one specific past situation.
General Principle 2. Focus on Effective, Impressive Behavior
General Principle 3. Code Against the Generic Competencies and Their Behavioral Indicators
This is one approach to implement coding:
A Simple Coding Process Using a Checklist
We have developed a simple coding process that is appropriate if you are doing a small number of Structured Event Interviews with superior performers and want to use the interviews to help identify behavioral indicators used by these persons.

This coding process involves using a checklist based on the framework of generic competencies and their behavioral indicators.

Reviewing and Analyzing Other Data
Survey Data, Customer Interviews, Observational Data

Details of this process, and instruction on analyzing and coding the data collected in step 3, are provided in the Workitect Building Competency Models workshop and practiced by Workitect consultants in the development of job competency models.

Instruction on developing competency models is also contained in the Quick-Start Competency Modeling program that is included with Competency Dictionary licenses.

©2018, WORKITECT, INC., ALL RIGHTS RESERVED
No part of this work may be copied or transferred to any other expression or form without written permission or a license Workitect, Inc. – 800.870-9490 – licensing@workitect.com

Share Button

Use an Expert Panel to Build a Basic Competency Model

Using Expert Panels, Focus Groups and Job Analysis Interviews. 

Many small and medium-size organizations want to develop competency models and integrate competencies into their talent management and HR systems. Unfortunately, many are constrained by limited budgets to use consultants or purchase competency dictionaries, software, interview guides, etc. In response to this problem, Workitect developed and began conducting a three-day workshop in 2004 to train internal HR professionals to build their own competency models. More than 1,200 people have attended these workshops and have built models using our methodology.

_____________________________________________________________________________

 The McClelland/McBer Model-Building Methodology

Our methodology for building job competency models is based on the Job Competence Assessment (JCA) methodology developed by Dr. David McClelland, a pioneer in motivation and competency research and testing at Harvard, and by consultants at McBer and Company in the 1970’s.

The modeling process starts with superior performers in a targeted job being identified, and then studied to identify the personal characteristics, skills, and knowledge that they possess that enables them to be superior performers. The methods used to collect data for building the model, such as behavioral event interviews and expert panels, are designed to get beneath mere opinions about superior performance and superior performers. _____________________________________________________________________________

Thirty years ago, we conducted research on job competence assessment and created a generic competency dictionary that has been tested and has evolved into a practical, comprehensive, and affordable dictionary consisting of 35 foundational competencies (leadership, management, and professional). Many organizations are now using this dictionary to build models and applications.

Still, many organizations are finding it difficult to launch a competency-modeling project, often due to a lack of time, staff, or budget. To help these organizations, we have taken material from our Building Competency Models workshop and developed a program to enable a competency dictionary licensee to build basic competency models using focus groups, supplemented with optional job task analysis interviews.

The program consists of these instructional materials:

  1. Overview of Competencies and Competency Models (16 page PDF)
  • What is a Competency?
  • What is a Competency Model?
    • Example of a Competency Model
  • Why Develop Competency Models?

            Integrating Key HR Processes (10 page PDF)
            Competencies 101 (Powerpoint)
            The Case for a Competency-Based HR System (Powerpoint)

  1. Planning a Competency Modeling Project (8 page PDF)
  • Analyzing and Identifying Stakeholders
    • Stakeholder Analysis Table
  • Structure of the Plan
  • Communicating with Stakeholders and Employees

        Worksheet for Planning a Competency Modeling Project (13 page PDF)

  • Scope of the Project
  • Organizational Context
  • Selecting the Approach to Model Building
  • Building Support for the Project
  • Deciding on Data Sources
  • Staffing the Model Building Project
  • Envisioning the Data Analysis and Model Building
  • Reviewing and Revising the Model
  1. Collecting Data & Developing a Basic* Competency Model (14 page PDF)            Using Focus Groups and Job Analysis Interviews
  • General Data Collection Tasks
  • Primary Data Collection Methods
    • Job Analysis Interviews
    • Resource Panels, aka Focus Groups or Expert Panels
      • Instructional manual on facilitating a Resource Panel
      • Alternative Methods
        • Virtual Resource Panel & Job Competency Profile
        • Competency Model Survey

     Resource Materials (separate documents and forms)

  • Competency Requirements Questionnaire
  • Competency Requirements Questionnaire Tabulation worksheet
  • Job Analysis Interview for Jobholders Template
  • Job Analysis Interview for Managers of Jobholders Template
  • Competency Dictionary

*A full model includes the conducting, analyzing, and coding of structured behavioral event interviews.

  • Licensees are expected to attend a future public or onsite workshop to learn how to collect and analyze additional data, including structured behavioral event interviews, and to develop competency-based applications.
  • Guidebook users will be given access to all materials in Dropbox folders.
  • Word versions of some customizable documents and forms are available.
  • Phone or live online coaching from a Workitect consultant is available.

THIS PROGRAM IS AVAILABLE FOR LICENSEES OF WORKITECT’S COMPETENCY DICTIONARY.

Contact Workitect for additional information about this program.

Join LinkedIn’s Competency-Based Talent Management group. This group is for HR, OD, training, and talent management professionals who want to network, share experiences, or seek answers about job competency modeling and competency-based HR, talent management, and leadership development.

Share Button

Develop a Job Competency “Profile”

Job Competency Model “Lite”.
Using a Virtual Resource Panel.
Apply the data to the development of a complete model.

This is an abridged competency model using a virtual resource panel and surveys to use when it is impractical to convene a standard resource panel of job incumbents, managers of job incumbents and other subject matter experts in one geographic location and at one time.

Data is collected and analyzed by a Workitect consultant who provides a summary and recommendations based on data about the job, obtained through a virtual resource/expert panel (VRP), i.e. an on-line, web-based survey containing open-ended questions and questions requiring ratings.

Purpose:

  • Improve an existing model
  • Build a model for a particular job or class of jobs that have similar tasks and competencies
  • Use when it is impractical to convene a standard resource panel of job incumbents, managers of job incumbents and other subject matter experts in one geographic location and at one time

The Output

  • A verbatim printout of all responses to all questions on the survey
  • A summary of the responses, including identification and tabulation of key themes in response to qualitative questions, tabulation of average ratings on all quantitative questions, and suggested competencies for a draft competency model.
  • Suggested non-technical generic competencies for the target job taken from Workitect’s Competency Dictionary. Any technical competencies that are identified are based on language used by panel members in their responses to questions about technical skill/knowledge requirements.

Advantages

  • Requires less time and cost to complete
  • Collect data from geographically dispersed participants
  • Apply the data to the development of a complete model

CONTENTS OF JOB COMPETENCY PROFILE REPORT

  • The Virtual Resource Panel Respondents
  • Main Responsibilities for the Job
  • Performance Criteria and Measures
  • Technical Skill Requirements
  • Trends in Technology, Industry, and the Company
  • Average Importance Ratings for Generic Competencies
  • Generic Competencies Ranked by Importance Rating
  • Relationship of Main Responsibilities to Generic Competencies
  • Competencies Recommended for Inclusion in the Competency Model
  • Most Challenging Aspects of the Job
  • Definitions and Behaviors for the Generic Competencies
  • Main Responsibilities

Sample Report

How it Works

As an alternative to a standard Resource Panel,  this virtual resource panel is an on-line survey allows panel members to provide the same information that they would in a standard session, but individually and at their own convenience and in their own work location. Participants are sent an email with a link to the survey set up in Survey Monkey.

Participants copy the survey link for their project into their internet browser and take about 30 minutes to complete the survey. A consultant downloads the survey responses, reviews and analyzes them, and prepares a synthesis based on themes mentioned by multiple panel members. It may be useful to set the stage for a Virtual Resource Panel by holding a conference call with the participants to explain and sell the project. Later, a second conference call can be held to review the synthesis of the panel members’ input.

Contact us for more information.

Learn more about competencies and competency models.

Share Button