Date of Award

Fall 2006

Document Type

Thesis

Degree Name

Master of Science (MS)

Department

Psychology

Program/Concentration

Psychology

Committee Director

Terry L. Dickinson

Committee Member

James M. Henson

Committee Member

Robert M. McIntyre

Call Number for Print

Special Collections LD4331.P65 S3482 2006

Abstract

Job performance ratings by supervisors and peers are very often the procedure for performance appraisal. The reliability of performance ratings is an important issue, because performance appraisal serves several purposes in the context of human resource management. Unfortunately, there are performance rating biases connected with raters, ratees, the interaction of raters and ratees, and situational and organizational characteristics. Interrater correlations are frequently interpreted as estimates of the reliability of performance ratings, and are also used to correct the correlations between ratings and other measures for attenuation. Some researchers (Murphy & DeShon, 2000a; 2000b) argue that interrater correlations are not reliability coefficients and should not be used to correct for attenuation, because job performance ratings reflect multiple sources of error and fail to meet appropriate assumptions. Others researchers do not agree (Schmidt, Viswesvaran, k. Ones, 2000), maintaining that interrater correlations are an appropriate measure of the reliability for job performance ratings. In addition to this controversy, these researchers also disagree on the relationship between reliability and validity of job performance ratings. The purpose of the current study was to investigate these arguments using simulated data and manipulating factors such as predictor interrater reliability, validity, and performance interrater reliability. The results indicated that the corrected validities provided by the classical test theory (or SVO) and G-theory procedures were virtually identical to those obtained with the SEM procedure. The corrected validities for the MD1 and MD2 conditions were relatively smaller in magnitude compared to those for SEM, SVO, and G-theory. Lastly, correction for attenuation was implemented with an SEM analysis for MD1 and MD2. These structural equation modeling results were identical to those obtained with the SVO conditions for SEM, SVO, and G-theory. Clearly, SEM is a powerful analytic tool that correctly estimates true validity even in the presence of rater idiosyncrasy. Limitations and suggestions for future research are discussed.

Rights

In Copyright. URI: http://rightsstatements.org/vocab/InC/1.0/ This Item is protected by copyright and/or related rights. You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s).

DOI

10.25777/yrf3-g477

Download

Share

COinS