Secondary Data Collection Methods

six steps horizontal 3STEP 3 – DATA COLLECTION

In addition to the primary data collection methods that were described in the previous sections, there are several other data collection methods that may be useful in selected circumstances. These secondary data collection methods include:

  • Interviews with Customers
  • Interviews with Industry Experts
  • Observation of Job Holders
  • Surveys of Job Holders

Interviews with Customers

If the job-holders have external or internal customers, the customers can provide useful information about effective and less effective behavior among persons holding this job. External customers often have experience with staff in similar jobs in competitors’ organizations.

Customer interviews can be relatively short (15 to 30 minutes). Possible questions include:

  • What are the skills and behaviors that you have observed in the most effective people that you have dealt with, in this job?
  • In what ways do superior performers in this job differ from average performers and less effective persons in this job?
  • Think of someone in this job that was very effective. What were some of the things that this person did that set him/her apart from less effective persons with the same job.

Interviews with Industry Experts

If a competency model is desired for a new job, especially in an industry that is undergoing rapid change, there will be few people in the organization with much knowledge about the job, beyond the expected job responsibilities. In this case it can be helpful to interview an industry expert from outside of the company. The industry expert should be able to describe:

  • Market and technology trends in the industry
  • Companies that are key players and their relative positions within the industry
  • The challenges likely to be encountered in the new job in the context of the industry

If you understand the challenges likely to be encountered in the new job, you can draw logical inferences about the competencies that will be needed for superior performance.

Learn how our consulting services can help you build competency models >>>


Share Button

Conducting Behavioral Event Interviews

six steps horizontal 3


A Behavioral Event Interview is a 1 to 1½ hour interview, in which the interviewee is asked to provide highly detailed accounts of how he/she approached 4-7 important accomplishments or other key events from the past year or two in the job. The interviewer uses a probing strategy to get the interviewee to walk through the sequence of what he/she did, said and thought at key points during each accomplishment or event.

The Behavioral Event Interview is usually conducted with superior performers. The assumption underlying the interview is that studying the interviewee’s actions, thoughts and words in these key situations will reveal underlying competencies responsible for superior performance.

For the analysis, each interview is tape recorded and transcribed. An analyst carefully reads each transcript and notes passages evidencing effective behaviors or thought patterns. These passages are noted on index cards or a spreadsheet template, along with the interviewee’s initials and the transcript page and line number. The analysts classify the themes by using a conceptual framework of generic competencies and behavioral indicators. The analysts then meet to review the evidence from their individual analysis and to identify competencies and behavioral indicators for the competency model. This process usually demonstrates that the superior performers used certain generic competencies and behavioral indicators from the conceptual framework used to classify the themes. But the process often reveals new behavioral indicators and competencies that were not part of the original conceptual framework.

Typical Structure of a Behavioral Event Interview

  • Introduction explaining the purpose of the project and of the format and purpose of the interview
  • Brief section on interviewee’s main responsibilities to provide orientation for the interviewer
  • “Event” questions asking the interviewee to provide detailed accounts how he/she approached key accomplishments and other work experiences
  • Follow-up probing of the interviewee’s response to each “event” question, to:
  • A closing question asking for the interviewee’s views about the personal characteristics needed for effectiveness in the job
  • Follow-up probing for examples from the interviewee’s experience

Advantages of Behavioral Event Interviews

  • Provide specific, high-quality behavioral data describing what superior performers do to achieve superior results
  • Surface non-obvious effective behaviors that job incumbents and their bosses may be unaware of or unable to articulate
  • Provide strong evidence for a competency model’s validity – evidence that is especially important if the model will be used for external selection
  • Provide excellent case material that can be adapted for use in developing training materials

Disadvantages of Behavioral Event Interviews

  • Are time consuming to conduct
  • Require extensive interview training and practice to ensure that high-quality data will be obtained
  • Are time consuming to analyze
  • Require training and practice to ensure the quality of the analysis

Structured Event Interviews

The Structured Event Interview is a simplified type of Behavioral Event Interview developed by Workitect to provide many of the benefits of Behavioral Event Interview, while significantly reducing the time and cost required to conduct and analyze the interview. This interview takes about one hour to conduct and focuses on three accomplishments, each of which is related to performance of a different main responsibility. The Structured Event Interview Protocol includes both “event” questions and specified follow-up questions that guide the interviewer through the process of probing each accomplishment. The protocol includes spaces to capture key information in response to each specified question.

This interview is taught in Workitect’s Building Competency Models workshop.

Share Button

Using Resource / Expert Panels to Build Competency Models

six steps horizontal 3Step 3 – DATA COLLECTION

A Resource Panel is a three to six-hour facilitated meeting with an agenda that is to similar to but broader than the one covered in a Job Analysis Interview. The participants usually include 3-4 capable job incumbents, 3-4 managers of job incumbents, and 1-2 HR staff who work closely with job incumbents. A Resource Panel has three main purposes: (1) to gather data needed to identify the competencies for the job, (2) to build consensus among a set of key stakeholders about what the job requires, and (3) to build support for the project.

Typical Agenda for Resource Panel

A typical agenda for a Resource Panel includes the following components:

  • Explain and sell the project.
  • Identify and reach consensus on four or five main responsibilities for the job.
  • For each responsibility, identify:
    • Key tasks
    • Performance measures or criteria
    • Skills and personal characteristics needed
    • Future scan:
    • – Identify ongoing or anticipated changes in the organization, industry, and relevant technology that may affect the job.
    • – Identify what each change implies, in terms of additional skills and personal characteristics that job incumbents will need.
    • Individually review a set of generic competencies and select a subset of these that are most important in the job.
    • Review individual rankings of competencies and reach consensus on a set that panel members consider to be most important for the job.

Advantages and Disadvantages of Resource Panels

Resource Panels have many advantages. They involve stakeholders early in the process and build support for the competency model and its planned applications. Conducting a resource panel is an inexpensive way to build a solid basis for a competency model. But Resource Panels also have some disadvantages. It can be difficult to obtain participation from job incumbents and their managers, especially if people are geographically dispersed. The standard agenda involves an analytical process that is too detailed and time-consuming for many senior managers. Resource Panels are a good method for identifying required competencies, but they usually provide little help in identifying behavioral indicators for the competencies judged to be important.

Considerations in Implementing Resource Panels

Since a Resource Panel may be seen as an important event within an organizational unit, there is a possibility that leaders and other key staff may feel snubbed, if they are not invited. Since the Resource Panel may make decisions that all staff will need to live with, the composition of the panel must have credibility, and key sub-groups within the organizational unit should be represented on the panel.

Facilitating Resource Panels

    • Among the challenges for facilitators are:
    • Capturing detailed responses quickly and legibly on flip chart pages and   posting the completed pages around the room
    • Ensuring a clear conceptualization of the main responsibilities
    • Keeping the process moving
    • Intervening appropriately if the group bogs down or goes off on a tangent
    • Maintaining control of the process

If possible, use two facilitators: one to facilitate and the other to capture responses on flip chart pages. The two can switch roles for different parts of the agenda.

Analyzing Resource Panel Data

Usually, only one Resource Panel is conducted to gather data about a job. After the panel session, the facilitator should transcribe the notes so that each question is listed, followed by the group’s responses. The result is a document that can easily be reviewed by the project team. Another useful addition to a Resource Panel is an administrative person to transcribe the flip chart pages onto a laptop computer, as the pages are being generated.

Variations on the Resource Panel Agenda 

  • Tailor agenda to the needs of the project and to constraints in the availability of panel members.
  • Devote a portion of the Resource Panel’s time to a planned application.
  • If the competency model needs to include technical competencies, members of the project team should spend some time with subject matter experts prior to the Resource Panel.
  • Use a virtual resource panel to allow panel members to provide the information individually and at their own convenience and in their own work location.
  • Focus on a set of jobs within one organizational unit and identify required levels for both technical and non-technical competencies.

 Learn more in Building Competency Models workshop.

Share Button