Data analysis occurs within the framework of a specific project or undertaking, so for purposes of this discussion we will consider data analysis within the context of an audacious proposal: a national, internet-based system for collecting standardized information about perioperative outcomes. The purpose of this endeavor would be to identify patterns of care associated with adverse and beneficial outcomes, and to translate these patterns into clinical lessons that could make a clinically and statistically significant contribution to future care. In simple terms, this project would act as rapid-feedback system for anesthesia patient safety and quality assurance in the United States. Ideally, new problems or opportunities could be identified, analyzed, and reported within the framework of one year or less. Based upon estimated participation by 10 to 50 medium-sized institutions, with each administering about 10,000 anesthetics per year, it might be feasible to collect data on approximately 100,000 to 500,000 anesthetics per year.

How would one create a foundation for data analysis in this type of project? Two basic tasks can be identified. The first task would be the creation of a standardized data collection tool and nomenclature. Superficially, it might appear that this entails two issues rather than one, but the process of developing a standardized data collection tool is so intimately associated with creating an unambiguous nomenclature that the two tasks are best considered and approached as a single project.

The second task would be the creation of a centralized system for data acquisition, analysis, and reporting. This task has three basic aspects. First, the project would need a stable source of funding. The entity providing funds would also be the expected source of oversight for the "politics of analysis," i.e., the general purposes for which the data would be used, and ways in which the data would be accessed and disseminated. Second, the project would require a stable core of individuals who would actually manage the project on a day-to-day basis. Based upon the experience of the Closed Claims Project, the minimum requirement for a core group would be several physicians, at least one seasoned health services analyst, and at least one expert in internet-based data management. Finally, the project would need to recruit a dedicated group of institutions from which the data on perioperative outcomes would be obtained. This last feature would be the most formidable because it would require the identification of individuals at each participating institution who would have a strong enough commitment to overcome the barriers to sharing outcome information and also have enough time to manage the ongoing process of local data collection and transmission. As an optimistic estimate, the creation of a centralized data system would take about three years.

Assuming the successful acquisition of data, how does one proceed with the task of analysis? A popular and effective approach is known as "exploratory data analysis." The underlying strategy is to look broadly for patterns of interest, and then use these patterns to test specific hypotheses. In specific terms, the first step of exploratory data analysis is usually an inventory or a descriptive summary of the available data. This activity often produces surprising results about the frequency or rarity of various events. The assessment of frequencies is typically accompanied by "Pareto analysis," in which the "objective" frequency or rarity of various events is coupled with a "subjective" weighting of importance. After a descriptive summary, another common exploratory technique is the search for changing trends over time. By this point, many data sets will exhibit patterns or recurrent features that are unexpected, unexplained, or difficult to understand or classify. These are extremely important observations, because they potentially point towards new or poorly understood areas of knowledge where there is an opportunity to make important discoveries.

The exploratory phase naturally leads to the formulation of hypotheses about cause and effect. A useful tactic at this stage is to formulate a hypothesis that is clinically meaningful or interesting whether it is affirmed or rejected (see Bailar JC: Statistics in practice; studies without internal controls; NEJM 1984;311:156-162.) Some of these hypotheses can be tested by comparing subsets within the data set as a whole. Alternatively, hypotheses generated by the data set can be used to stimulate the development of external models and studies with rigorous, prospective controls. This latter approach can lead to important advances in basic medical knowledge.

A project of this magnitude would obviously face obstacles. It is important to emphasize that these do not include basic analytic issues. In other words, the tools and techniques for analyzing this type of data are sufficiently defined and robust to easily accomplish the task at hand. The real problems lie upstream. Two very important and interrelated issues are confidentiality and liability, particularly as they relate to the fear that sharing outcome data will increase the risk of malpractice suits. Advances in data encryption probably make confidentiality easier to achieve on an incremental or case-by-case basis. However, the possibility that an electronic spy might eavesdrop on data transmission or copy files from a centralized database raises major concerns, because this type of activity could lead to the marketing of "tips" (perhaps called e-tips) that could be used by plaintiffs’ attorneys. Pilot or model studies may help us understand if these risks are significant. Cost is another problem. In the early 1990’s, before the maturation of internet-based data management, the cost of a centralized, nationwide, one-year anesthesia outcome study, designed and managed by the Center for Disease Control, was estimated at approximately $8 million. The experience of the Closed Claims Project suggests that considerable savings could be achieved with current technology and the use of volunteer resources.

The final and perhaps greatest hurdle is cultural or philosophical: Why bother? Haven’t we done well enough with patient safety? Is this the best investment for our limited dollars and time? What makes us think this activity will lead to important findings or improvements? My purpose here is simply to assert that our specialty has the ability to analyze a national repository of perioperative anesthesia outcome data in an effective and innovative manner. The other summaries and articles in this Newsletter may help the reader decide whether or not we should try.