4 Relating Domain Expertise to "Statistical Signatures": Modelling the Management of Critical Incidents

4.2 Results

The results of the simulations indicated that communication reduces the time required to resolve critical incidents but that events which are too complex to be resolved by the usual procedures are not affected by improving normal communications. In terms of outputs which correspond to observable statistics, the number of event cycles elapsing from the onset of an incident at an operating site until the absence of any events at the same site was recorded for every incident over the various simulation runs. When network controllers shared their successful mental models, the percentage of incidents resolved within two event cycles increased from some 50 percent to about 60 percent. The proportion that took more than two but not more than four event cycles increased from 5.9 to 7.65 per cent on average (with a confidence interval of 0.99). However, model sharing made no significant difference to the percentage of events that took more than eight event cycles to resolve.

Now the model was set up to yield very large numbers of incidents of greater and lesser complexity. The relative probabilities of the occurrence of each type of event were obtained from the company concerned. The absolute probabilities were much higher so that statistically meaningful results could be obtained relatively quickly with the available computational resources. By "much higher" here is meant a level which, if realized by the company concerned, would involve loss of licenses, jail terms for the directors and public enquiries. However, we did obtain a "statistical signature" derived from a well validated representation of agent cognition and an empirically accurate specification of company structure, systems and procedures.

Is this statistical signature in some sense accurate? We have no criteria by which to give an answer. The statistical signature, like the stylized fact, is clearly a useful notion but, for the sake of clarity and to give us confidence in the implications of our models, we should pin down what we mean by it in some formal or consistent way. One possibility is to show that models representing the time scales involved more realistically can be parameterized to yield low RMSEs and MAPEs or similar measures of simulation accuracy and that condensing the time scales and either aggregating or reducing the numbers of agents have predictable effects.

Simulation and Reality - 20 MAY 98

[Next] [Previous] [Up] [Top] [Contents]

Generated with CERN WebMaker