Modelling is good, we all agree. The ultimate test of whether you can understand something is how well you can recreate it, whether in code, modelling clay or popsicle sticks. And once you have your little toy version of the world you can poke and prod it to your hearts content, without pesky considerations like “physics” or “damaging vital infrastructure”. We even make models inside our own heads, trying to puzzle out the consequences of our actions by mentally simulating cause and effect.
Yes it’s fair to say that modelling, and its more dynamic brother simulation, have delivered us the modern world. However, as anybody who has spent a long weekend chained up in a cave will tell you, the sign is not the signified. All models compromise on scale or detail, because otherwise you would simply observe the 1-to-1 scale, 100% detailed version of the world outside your window. The art of creating a good model is knowing what to leave out in order to capture what really matters about the system.
When you leave out too much, the failure mode is (appropriately) simple. Your model will render the world too crudely to derive any useful insights – you may be able render a skyscraper in Lego bricks, but you will struggle to design the wiring and HVAC systems that way. Frustrating as this is, it is often at least obvious, and even the simplest models can help one get “in the ballpark” of a solution. Over-complex models are harder to spot. There are only so many ways to simplify something, people are constantly discovering new kinds of pointless detail to cram into their pet projects. In the best case, this merely makes the model large, slow and unwieldy, but unfortunately the best case is rare. More commonly, our biases and individual priorities ensure the model is incredibly detailed in one area and bizarrely lacking in others. The result is deeply lopsided performance where accuracy plummets for no visible reason and recovers just as inexplicably.
Complex models are also largely rigid models. Because of their high level of detail they are very sensitive to the fine details of their input, and can react very poorly to incorrect or imprecise data. Where an oversimplified model is like a monster truck, indifferently flattening all it encounters, an overcomplicated one is like an F1 car – exceptional performance, but only in highly specific contexts. The ideal model avoids both pitfalls, working, if not perfectly, then well enough regardless of what you throw at it. This is the core of intelligence in practice, to constantly engage with the object of inquiry and build a living understanding of it via iteration. Unfortunately this is not the aesthetic of intelligence, though, which brings us to our next problem.
Increasingly we are told to put our faith in state-of-the-art machine learning models to solve our problems start to finish by applying gargantuan amounts of compute and training data. This appeals to the aesthetic of intelligence: the genius who reads hundreds of books, ponders deeply, and arrives at the solution fully-formed. This image is largely a myth. As humans, we habitually disguise the effort we put in because those stumbling first steps are embarrassing. We don’t want other people to see our messy first drafts, dead-end experiments and ill-thought-out prototypes – even we don’t want to see them! But a machine has no pride, and we are foolish to project ours on to them, for this is how we end up with digital confidence men that present nonsense with the authority of established fact. It is only when these models interact with something that is unavoidably, physically real, when the rubber hits the road, that the cracks show. You can’t impress a supply shortfall, or lie to a skills gap – these things exist in the real world, and cannot be solved without real critical thinking. A model can help, but unless people are empowered to disagree with it and use their own expertise, it is merely a comfort blanket to give the illusion of control.