Date of Award

Summer 1986

Document Type

Thesis

Degree Name

Master of Science (MS)

Department

Psychology

Program/Concentration

Psychology

Committee Director

Terry L. Dickinson

Committee Member

Anthony T. Dalessio

Committee Member

David L. Pancoast

Committee Member

Glynn D. Coates

Call Number for Print

Special Collections LD4331.P65 C35

Abstract

The purpose of this study was to establish the construct validity of five dimensions, assessed by two role simulations, using behavioral checklist and BARS formats. Participants were 12 graduate students in psychology, most of whom were enrolled in an industrial/ organizational program. They served as assessors and evaluated the videotaped performance of ten ratees who had previously participated in assessment centers conducted at the university.

Substantial evidence for convergent and an adequate level of discriminant validity were hypothesized. Although it was anticipated that method bias would be low, it was hypothesized that the behavioral checklist would exhibit a higher level of discriminant validity than the BARS format, because the latter format required greater cognitive processing.

The analysis of variance approach was utilized in the analysis of the ratings. Significant F-ratios were obtained for convergent and discriminant validity, method bias, and differential discriminant validity with regard to format and exercise. As these ratios were based on mean squares with large degrees of freedom, variance components and intraclass correlation coefficients were also calculated. In contrast to the F-ratios, these latter statistics proved to be quite small for both convergent validity and method bias, accounting for three and one percent of the variance respectively. Thus, there was no support for the prediction made with regard to convergent validity, although the hypothesis concerning method bias was upheld. A moderate level of discriminant validity was attained, accounting for 20 percent of the variance, and thus confirming the hypothesis regarding this effect. The intraclass correlation coefficients associated with differential discriminant validity were .05 for the formats and .18 for the exercises. Post hoc tests revealed that the BARS format resulted in a larger number of significant differences between ratees on four of the five dimensions, while equal numbers of significant differences resulted on the fifth dimension. It was suggested that the large number of behavioral items on the checklist may have made rating with this format more difficult than with the BARS.

Even though both of the exercises were role simulations, a pronounced exercise effect was uncovered. The impact of the exercises may have accounted for the low levels of convergent validity, as suggested by Neidig and Neidig (1984). Furthermore, the variance explained by the Exercises x Dimensions x Ratees interaction may have depressed the results with regard to discriminant validity.

Additional work is needed to correct the deficiencies noted with regard to the behavioral checklist. Of greater importance, however, is the need to direct more attention to the impact of the exercises on the performance of participants in assessment centers and to examine more closely the manner in which the exercises interact with the dimensions. Further study in these areas may ultimately lead to more satisfactory findings with regard to convergent and discriminant validity.

Rights

In Copyright. URI: http://rightsstatements.org/vocab/InC/1.0/ This Item is protected by copyright and/or related rights. You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s).

DOI

10.25777/b3jf-vg86

Download

Included in

Psychology Commons

Share

COinS