Rigorous new approach to constructing a gold standard for validating new diagnostic criteria, as exemplified by the eosinophilia-myalgia syndrome.

BACKGROUND: Constructing diagnostic criteria, a common problem in clinical medicine, is particularly difficult for diseases that lack a pathognomonic "gold standard." To develop an improved strategy for constructing such criteria, we used the eosinophilia-myalgia syndrome as an example. The goal, for research classifications, was to construct validated clinically sensible criteria and to develop improved methods that can be used for other disorders.

METHODS: Using a "pattern-based" approach with data from several separate sources, a committee of investigators first prepared and informally tested criteria for the diagnosis of eosinophilia-myalgia syndrome. A gold standard challenge set of reports of cases and noncases was independently generated and separately validated by an external panel of clinical experts. The criteria were then tested using the gold standard set, and interobserver variability and diagnostic accuracy were determined.

RESULTS: Interobserver variability showed the following mean proportionate agreements: 98.7% for the presence of specific criteria elements, 99% to 100% for diagnosis, and 97% to 98% for diagnostic pattern. kappa Values were correspondingly high. Diagnostic accuracy showed sensitivity at 88%, specificity at 97%, and overall accuracy at 92%.

CONCLUSIONS: The proposed criteria are accurate and reproducible, and can be used in future clinical investigations of the eosinophilia-myalgia syndrome. The new strategy and methods developed for this challenge can be valuable for solving analogous problems in constructing criteria for other clinical disorders.