Performance Management - All Reports and Products
-
Thursday, January 1, 2009
Report: Best Practices for Event Review Committees
This document presents the results of a 12-month investigation of Event Review Committee (ERC) Best Practices. The identified best practices are listed and described. Strategies to achieve the best practice and factors that airlines should consider when implementing the best practice are also presented.
-
Tuesday, October 11, 2005
Report: Promising Practices in Motorcycle Rider Education and Licensing
Motorcycle rider education provides an opportunity for novice riders to learn the basic skills necessary to operate a motorcycle safely and for experienced riders to refresh and refine their techniques. The purpose of this report is to develop a research-based model of promising practices in rider education and licensing and to use the model to identify States that have implemented high-quality rider training and comprehensive licensing.
-
Wednesday, September 17, 2003
Report: Airline Pilots' Experiences in and Reactions to Their Check Rides: Results from a Nationwide, Representative Survey
The purpose of this study was to assess the relative effectiveness of different approaches to checking pilot performance at the end of training: the maneuver validation (MV) and the Line Operational Evaluation (LOE). Because the LOE provides greater contextual cues and integrates CRM skills with technical skills, it should simulate typical line operations more accurately than a traditional maneuver validation. Therefore, we hypothesized that pilots would rate the LOE as more useful than the MV. The results presented below are part of a much larger survey of airline pilots’ experiences in and reactions to their professional training (Baker et al., 2002).
-
Wednesday, June 4, 2003
Report: Improving the Construct Validity of Line Operational Simulation (LOS) Ratings: Lessons Learned from the Assessment Center
Because poor LOS construct validity can have real-world effects on pilot training and performance, assessing and improving the construct validity of Line Operational Simulations is more than just an academic or scientific issue. It is also a practical and political issue in that it involves multiple stakeholders who may have competing concerns. These include safety, justice/fairness, technical feasibility, and cost-effectiveness (Austin, Klimoski, & Hunt, 1996). Therefore, we recommend that all potential stakeholder groups be involved in identifying and improving the construct validity of Line Operational Simulations. These groups may include pilot unions, training staff, flight standards staff, and officials from the regional FAA offices. Moreover, all groups must be prepared to compromise some of their own goals/needs to achieve a balanced solution. In the end, only by working together can industry address the issue of LOS construct validity, and by extension, the quality of pilot crew training and evaluation.
-
Friday, September 27, 2002
Report: A Guide for Education Personnel: Evaluating a Program or Intervention
Often we think of evaluation research happening at the end of a program or intervention’s lifespan in order to determine whether the program worked – program impact. However, an equally important function served by evaluation research is monitoring program implementation. Evaluations of implementation are essential because they help identify problems with program implementation before the program ends, so that changes in programs or interventions can still have an impact. It doesn’t do us a lot of good to talk about results of an intervention if we find out the intervention was not really in place to begin with! You may find cases where the intervention changes a good bit as schools and teachers make it fit their particular circumstances or the needs of their students. Documenting and understanding these changes are important when you start to talk about how the intervention is affecting the problem or situation it was brought in to address. Without this information it may be difficult to replicate elsewhere.
-
Tuesday, September 17, 2002
Report: Airline Pilots' Perceptions of and Experiences in Crew Resource Management (CRM) Training
In the sections below, we describe the results of a large-scale survey of pilots' perceptions of and experiences in their training. In particular, we focus on their responses to a series of questions concerning Crew Resource Management (CRM) training. This project was a unique opportunity to conduct a scientifically rigorous, large-scale comparison of CRM training programs across multiple airlines. Nevertheless, we recognize that participants' reactions to training are only one measure of a training program's effectiveness.
-
Saturday, April 22, 2000
Report: Using Multifacet Rasch Analysis to Examine the Effectiveness of Rater Training
This paper sought to provide a comprehensive analysis of a rater training program through the use of multifacet Rasch measurement. The purpose was to display how such an analysis can provide specific information on raters that is useful for feedback, and also important information concerning the performance of the rating form and training materials.
-
Monday, April 17, 2000
Report: Utility of Event-Based Knowledge Elicitation
Using event-based knowledge elicitation, we obtained differences between respondents based on their experience level, providing some empirical evidence for the validity of the approach. The technique requires significant preparation time compared with other approaches, but because expectations can be developed, it may greatly simplify the data analysis. This is especially true if focused information is sough and the analyst plans to collect data from a number of experts.
-
Thursday, September 10, 1998
Report: Team Task Analysis: Lost but Hopefully Not Forgotten
Team task analysis refers not only to an analysis of a team's tasks, but also to a comprehensive assessment of a team's teamwork requirements (i.e., knowledge, skill, ability, and attitude requirements). Like job analysis, team task analysis is important because it forms the foundation for team design, team performance measurement, and team training. Essentially, it is the building block for all "team" resource management functions.
-
Wednesday, June 18, 1997
Report: Principles for Measuring Teamwork: A Summary and Look Toward the Future
According to Cannon-Bowers et al., shared mental models are organized bodies of knowledge that are shared across members of a team. They suggest that such models have the potential to affect teamwork at two levels. First, when communication channels are limited, shared mental models enable team members to anticipate other team member behaviors and information requirements. Second, shared mental models of a team task enable team members to perform team functions from a common frame of reference. Recent work, presented in this volume, is directly related to our first principle. Of these chapters, Dickinson and McIntyre focus on team knowledge and attitudes, and Kraiger and Wenzel focus on team mental models.
-
Saturday, August 17, 1996
Report: Analyzing Team Performance: In the Eye of the Beholder?
This study examined the effects of experience on the degree to which various team behavior characteristics were weighted in a team member's perception of team behavior importance. In general, the results supported the hypothesis that experience would moderate the extent to which team members weighted different team behavior characteristics when making judgments of overall team behavior importance.
-
Monday, September 18, 1995
Report: Situation Awareness and Team Performance: Implications for Measurement and Training
In this article, we examine situation awareness (SA) on two levels. First, we briefly summarize some elements that are common to several of the proposed explanations of individual SA in order to provide a base for examining team SA. Second, we identify critical variables that are associated with team SA and describe processes and behaviors that have been proposed as contributors to its establishment and maintenance. On the basis of the information reviewed, we identify issues related to the measurement and training of team SA.
-
Tuesday, September 21, 1993
Report: Aviation Computer Games for CRM Skills Training
Computer games have the capacity to engage the player, are inexpensive, and are readily available. These three qualities suggest possible value as a training medium, even though existing aviation game software has not been designed specifically for training or crew interactions. Reactions of pilots participating in this research indicated that the use of computer games with carefully designed scenarios can be an acceptable means of training CRM skills. Aircrews seemed to appreciate the training value of the system and became engaged in its scenarios. Acceptance was found by aviators of all experience levels.
-
Tuesday, August 18, 1992
Report: Principles for Measuring Teamwork Skills
The purpose of the present paper is to examine the measurement of teamwork skills. These skills are one of the most difficult components of team performance to capture because they are not readily quantifiable, as are team inputs and outputs. Therefore the present examination assessed the measurement of teamwork skills at several levels (i.e., theories, methodologies, and psychometric properties).
