Date of Award
Doctor of Philosophy (PhD)
Terry L. Dickinson
Robert M. McIntyre
Within the assessment center method, assessors' preliminary evaluations of candidates' performance are typically based on information obtained through direct observation of the candidate's performance in the situational exercise, or on another assessor's report of that performance. This variation is somewhat disconcerting, however, in that its impact on assessor ratings remains largely unstudied. The primary focus of the present study was to assess the differential effects of observation type (i.e., direct observation, dimension-specific report, narrative report) on various measures of rating accuracy. In addition, the present study investigated cognitive modeling as an assessor training strategy, and its impact on rating accuracy. Seventy-three undergraduates majoring in business administration were either trained or not trained, and either observed and rated nine videotapes depicting individuals conducting performance review sessions, or reviewed and evaluated corresponding dimension-specific or narrative reports describing the same performance. A two (cognitive modeling training, no training) by three (direct observation, dimension-specific report, narrative report) analysis of variance design was used to assess the effects of training and observation type on rating accuracy.
Moderate support was found for the hypotheses that direct observation would yield greater accuracy than report-based ratings. While ratings obtained in the direct observation condition were generally more accurate than narrative-report based ratings, dimension-specific report ratings produced superior accuracy. In addition, those participants receiving the cognitive modeling training, as predicted, were significantly more accurate in their ratings than the no-training participants.
These results suggest that in assessment situations where assessors must rely on reports to evaluate candidate performance, those reports should be constructed in a dimension-specific format. Furthermore, cognitive modeling appears to be a viable strategy for assessor training. Future research should examine reports which more closely typify actual assessment center products and their effects on rating accuracy. The cognitive modeling approach to assessor training should also be given greater attention in further study.
Johnson, Rudolph L..
"The Influence of Direct Versus Indirect Observation, Candidate Report Format, and Assessor Training on the Accuracy of Assessor Ratings"
(1987). Doctor of Philosophy (PhD), dissertation, Psychology, Old Dominion University, DOI: 10.25777/wj6z-be63