Date of Award

Summer 8-2022

Document Type

Dissertation

Degree Name

Doctor of Philosophy (PhD)

Department

STEM Education & Professional Studies

Program/Concentration

Instructional Design and Technology

Committee Director

John Baaki

Committee Member

Noah Glaser

Committee Member

Joel Drake

Abstract

Confirmative evaluation is often noted as an important element of models of Human Performance Technology and Performance Improvement, but there exists confusion around what is and what is not confirmative evaluation. A significant issue limiting the use of confirmative evaluation is the Performance Improvement field’s inability to clearly classify confirmative evaluation in terms of its purpose and scope. Additionally, the performance improvement field lacks sufficient examples of confirmative evaluation in the literature necessary to adequately define confirmative evaluation and demonstrate its use. Without sufficient examples of confirmative evaluation, practitioners of performance improvement are left without a clear path of how to conduct confirmative evaluation in ways that are contextually relevant and cost effective. Additionally, those who train instructional designers and performance improvement practitioners lack the tools necessary to help students of the field learn about and use this vital aspect of evaluation.

Through the conceptual framework of systems theory and change management, this instrumental case study reviewed the literature around confirmative evaluation to identify the confusion around confirmative evaluation and then presented a case of confirmative evaluation applied in a context of change management and continuous improvement in online higher education. This case used job descriptions of remote leaders’ work to determine the degree to which an organizational restructuring had been implemented as it had been intended and had influenced remote leaders’ actual work. Time study methodology and matrix sampling were employed to observe remote leaders’ work. The data from the time study was compared against remote leaders’ job descriptions and institutional surveys gauging satisfaction among those served by these remote leaders. The difference between the expected work and the observed work demonstrated the degree to which remote leaders had embraced the new duties given to them in the organizational restructuring and identified key gaps in their collective performance. Together, this instrumental case study demonstrated that confirmative evaluation plays a linking or bridging role in continuous improvement processes by moving the process from summative evaluation to a new phase of formative evaluation.

The study then examined the results of the case to consider how it clarified the field’s understanding of confirmative evaluation. Key findings included the recognition that confirmative evaluation is distinct from other forms of evaluation because of its purpose and that purpose and timing are intertwined and inform one another in evaluation efforts. It also identified where confirmative evaluation fits within and can improve performance improvement and change management models. The study ended with a discussion of challenges faced when conducting confirmative evaluation and implications for future studies of confirmative evaluation.

Rights

In Copyright. URI: http://rightsstatements.org/vocab/InC/1.0/ This Item is protected by copyright and/or related rights. You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s).

DOI

10.25777/zcxn-gf46

ISBN

9798351481463

ORCID

0000-0003-4737-7078

Download

Share

COinS