Affirming the Consequent

A third epistemological problem of knowing whether a given (simulation) model structure is appropriate, after Equifinality and Interactive Kinds, regards the comparison of model results with real-world empirical data. Comparison of models’ predictions with empirical events has frequently been used in an attempt to show that the model structure is an accurate representation of the system being modelled (i.e. demonstrate it is ‘true’). Such an idea arises from the hypothetico-deductive scientific method of isolating a system and then devising experiments to logical prove a hypothesis via deduction. As I've discussed, such an approach may be useful in closed laboratory-type situations and systems, but less so in open systems.

The issue here is that predictions about real-world environmental systems are temporal predictions about events occurring at explicit points in time or geographical space, not logical predictions that are independent of space and time and that allow the generation of science's 'universal laws'. These temporal predictions have often been treated with the same respect given to the logical prediction of the hypothetico-deductive method. However, as Naomi Oreskes points out, it is unclear whether the comparison of a temporal prediction produced by a simulation model with empirical events is a test of the input data, the model structure, or the established facts upon which the structure is based. Furthermore, if the model is refuted (i.e. temporal predictions are found to be incorrect) given the complexity of many environmental simulation models it would be hard to pin-point which part of the model was at fault.

In the case of spatial models, the achievement of partially spatially accurate prediction does little to establish where or why the model went wrong. If the model is able to predict observed events, this is still no guarantee that the model will be able predict into the future given it cannot be guaranteed that the stationarity assumption will be maintained. This assumption is that the processes being modelled are constant thought time and space within the scope of the model. Regardless, Oreskes et al. (1994) have argued that temporal prediction is not possible by numerical simulation models of open, middle-numbered systems because of theoretical, empirical, and parametric uncertainties within the model structure. As a consequence, Oreskes et al. (1994) warn that numerical simulation modellers must beware of making the fallacy of ‘affirming the consequent’ by deeming a model invalid (i.e. false) if it does not reproduce the observed real-world data, or valid (i.e. true) if it does.