Verification and Validation of Complex Systems: Human Factors Issues

Despite its increasing importance, the verification and validation of the human-machine interface is perhaps the most overlooked aspect of system development. Although much has been written about the design and developmentprocess, very little organized information is available on how to verifyand validate highly complex and highly coupled dynamic systems. Inability toevaluate such systems adequately may become the limiting factor in our ability to employ systems that our technology and knowledge allow us to design. This volume, based on a NATO Advanced Science Institute held in 1992, is designed to provide guidance for the verification and validation of all highly complex and coupled systems. Air traffic control isused an an example to ensure that the theory is described in terms that will allow its implementation, but the results can be applied to all complex and coupled systems. The volume presents the knowledge and theory ina format that will allow readers from a wide variety of backgrounds to apply it to the systems for which they are responsible. The emphasis is on domains where significant advances have been made in the methods of identifying potential problems and in new testing methods and tools. Also emphasized are techniques to identify the assumptions on which a system is built and to spot their weaknesses.

Verification and Validation: Concepts, Issues, and Applications.- Resilience Theory and System Evaluation.- On The Future Of Hybrid Human-Machine Systems.- Basic Considerations in Verification and Validation.- Validation in Complex Systems: Behavioral Issues.- Defining Human-Centered System Issues for Verifying and Validating Air Traffic Control Systems.- Evaluating the Impact of New Technology on Human-Machine Cooperation.- Integrating Verification and Validation with the Design of Complex Man-Machine Systems.- Assessment of Complexity.- Limits to Analysis and Verification.- The Validation and Verification of Complex Knowledge-Based Systems.- The Reliability Of Interactive Systems: Simulation Based Assessment.- The Identification of Latent Organizational Failures in Complex Systems.- The Role of Incident Investigation in System Validation.- Problems of Systematic Safety Assessments: Lessons Learned from Aircraft Accidents.- Major Incidents, Safe and Reliable Verdicts and the Process of Verification and Validation.- The Human Component of System Validation.- When Task Demand is Variable: Verifying and Validating Mental Workload in Complex, “Real World” Systems.- Performance Evaluation of Human-Machine Systems.- Requirements Analysis for Human System Information Exchange.- Working Memory and Human-Machine Systems.- The Role of Verification and Validation in the Design Process of Knowledge Based Components of Air Traffic Control Systems.- Automation and Representation in Complex Man-Machine Systems.- How to Fit the Man-Machine Interface and Mental Models of the Operators.- Cultures with Requisite Imagination.- System Validation — A Step in a Continuous Improvement Process.- Cultural Behavior in the Airline Cockpit System: A Theoretical Framework Proposal.- The Inclusion of Future Users in the Design and Evaluation Process.- User Involvement in the Development of Highly Interactive Software Systems.- Psychological Aspects of Human Factors Testing and Evaluation of Military Human-Machine Systems.- Involving the User in the Design of Computer-Based Displays in Power Plant Control Rooms.- Systems Theory Versus Verification and Validation.- Controlling Factors: An Operator’s Perspective.- What They Want Is What They Get?.- Contemporary Issues in ATC System Development.- Validation Problems in Air Traffic Control Systems.- Simulating and Evaluating the Future — Pitfalls or Success?.- The National Plan for Aviation Human Factors.- Test and Evaluation Program for a Prototype of an Advanced Computerized Control Room for Nuclear Power Plants.- Validation Issues in Decision Support Systems for Maintenance Planning.- Artificial Habitat for Man in Extreme Environments as an Integrated Human-Machine System.- Concept of a FMS/ATC Air-Ground Data Link Testbed Employing an Airbus A340 Full Flight Simulator.- The Qualification of Military Aircraft Cockpits.- The Use of Video to Verify and Validate Human System Interactions: A Methodology.- Potential Application of Neural Networks to Verification and Validation of Complex Systems.- Verification and Validation of the Training Components of Highly Complex Systems.- An Expert Air Traffic Control Teaching Machine: Critical Learning Issues.- Interaction of Stages in Validating and Verifying ATC Training.- The Verification of Pilot Abilities as a Basis for Validating Flight Crew Competency.- Closing Remarks.- Complex and Integrated Human-Machine Systems: Retroflections.- Lecturers, Participants, and Staff.