Peer Assessment of Aviation Performance: Inconsistent for Good Reasons

Cognitive Science 39 (2):405-433 (2015)
  Copy   BIBTEX


Research into expertise is relatively common in cognitive science concerning expertise existing across many domains. However, much less research has examined how experts within the same domain assess the performance of their peer experts. We report the results of a modified think-aloud study conducted with 18 pilots . Pairs of same-ranked pilots were asked to rate the performance of a captain flying in a critical pre-recorded simulator scenario. Findings reveal considerable variance within performance categories, differences in the process used as evidence in support of a performance rating, different numbers and types of facts identified, and differences in how specific performance events affect choice of performance category and gravity of performance assessment. Such variance is consistent with low inter-rater reliability. Because raters exhibited good, albeit imprecise, reasons and facts, a fuzzy mathematical model of performance rating was developed. The model provides good agreement with observed variations



    Upload a copy of this work     Papers currently archived: 92,197

External links

Setup an account with your affiliations in order to access resources via your University's proxy server

Through your library

Similar books and articles

Lyotard’s performance.Robin Usher - 2006 - Studies in Philosophy and Education 25 (4):279-288.
Future of Ethically Effective Leadership.Chaudhary Imran Sarwar - 2013 - Journal of Business Ethics 113 (1):81-89.
Expanding Expertise: Investigating a Musician’s Experience of Music Performance.Andrew Geeves, Doris Mcllwain, John Sutton & Wayne Christensen - 2010 - ASCS09: Proceedings of the 9th Conference of the Australasian Society for Cognitive Science:106-113.
Performance, promotion and information.Zajac Jaroslav - 2004 - Journal of Business Ethics 50 (2):187-198.
Use and misuse of metrics in research evaluation.Ronald N. Kostoff - 1997 - Science and Engineering Ethics 3 (2):109-120.
Performing Society.Peter Snow - 2010 - Thesis Eleven 103 (1):78-87.


Added to PP

17 (#872,413)

6 months
1 (#1,477,342)

Historical graph of downloads
How can I increase my downloads?