A Sensitivity Analysis of Pathfinder
This addresses the bottleneck of knowledge elicitation for domain experts in expert system design, but it is incremental as it applies existing sensitivity analysis methods to a specific system.
The paper tackles the problem of parameter refinement in Bayes net-based expert systems by conducting a sensitivity analysis on Pathfinder, a system for diagnosing lymph system pathologies, finding that system performance is relatively insensitive to noise in parameters, indicating a point of diminishing returns for further refinements.
Knowledge elicitation is one of the major bottlenecks in expert system design. Systems based on Bayes nets require two types of information--network structure and parameters (or probabilities). Both must be elicited from the domain expert. In general, parameters have greater opacity than structure, and more time is spent in their refinement than in any other phase of elicitation. Thus, it is important to determine the point of diminishing returns, beyond which further refinements will promise little (if any) improvement. Sensitivity analyses address precisely this issue--the sensitivity of a model to the precision of its parameters. In this paper, we report the results of a sensitivity analysis of Pathfinder, a Bayes net based system for diagnosing pathologies of the lymph system. This analysis is intended to shed some light on the relative importance of structure and parameters to system performance, as well as the sensitivity of a system based on a Bayes net to noise in its assessed parameters.