Date of Award

Spring 2019

Document Type

Dissertation

Degree Name

Doctor of Philosophy (PhD)

Department

Engineering Management & Systems Engineering

Committee Director

Resit Unal

Committee Member

C. Ariel Pinto

Committee Member

Pilar Pazos

Committee Member

Kaitlynn Castelle

Abstract

The purpose of this research is to investigate the general criteria for assessing the technical implementation risk factors of proposed space science missions at the mission concept stage. According to the National Aeronautics and Space Administration (NASA) Space Flight Program and Project Management Handbook (NASA, 2012), the mission concept review objectives are “To evaluate the feasibility of the proposed mission concept(s) and its fulfillment of the program's needs and objectives. To determine whether the maturity of the concept and associated planning are sufficient to begin Phase A” (p.33). Experts previously defined two technical risk factors, to assess aspects of the space flight systems and mission design and operations of proposed mission concepts. Criteria were developed to address these two technical risk factors, which are comprised of 23 criteria. The space flight systems factor was assumed to be addressed by 16 criteria, while seven criteria were assumed to address the mission design and operations factor.

The criteria were developed by experts approximately 20 years ago, and no research has previously been conducted to determine whether all 23 of the evaluation criteria are necessary for assessing the implementation risk of proposed space flight systems and mission design and operations for proposed mission concepts. NASA uses these 23 criteria to conduct expert peer reviews to assess the implementation risk of over 500 unique space science mission concept proposals. An expert peer review process is used because the proposed concepts lack the detailed design information necessary for a quantifiable assessment of risk. The result of the expert peer review of each proposal is a set of ratings with a paragraph explaining the rationale for each rating, based on the 23 criteria.

This research used 356 records from past assessments of proposed mission concepts that have been assessed using a five-level qualitative rating scale. A research approach which utilizes exploratory factor analysis and past records to analyze the ratings of the 23 criteria was used. Factor analysis was used to determine if the current factor structure was valid, whether all criteria had substantial loadings on the current factor, and whether all current criteria were necessary. Factor analysis was also used to determine if any of the criteria measured the same construct. This research used a discriminatory power scale to code criteria scores for factor analysis and to identify the criteria of significance to decision makers. This research identified criteria that could be eliminated or could be combined with other criteria. A result of this research is a reduced set of criteria for assessing space flight systems and for mission design and operations that can be accomplished by an expert peer review panel for a diverse set of space mission concepts. A refined set of criteria could result in a less expensive and quicker evaluation process. This can enable decision makers on early assessments of space flight systems to make decisions more efficiently by allowing them to focus only on the most important criteria. This refined set of criteria contributes to the literature on the qualitative risk assessment of space flight systems and mission design and operations. This research is supported by the existing body of literature in using factor analysis to refine a measurement instrument. Using factor analysis to evaluate criteria for spaceflight systems contributes another application of the use of factor analysis, beyond its historical use in psychology, education, and healthcare (Williams, Onsman, & Brown, 2010). This research provides a method that engineering managers can use to analyze and to refine a qualitative measurement instrument for assessment by a group of experts. This method could be useful in assessments that require a broad scope of required expertise.

DOI

10.25777/ntny-g993

ISBN

9781392268254

ORCID

0000-0002-0662-9593

Download

Share

COinS