I am a co-author of this review, published recently, and the product of some intensive workshopping, debate and drafting. Not surprisingly, my words are most in evidence in the chapter on economics, but we decided that the cut and thrust of debate had produced a successful joint effort and we are all listed as authors.
Our first decision was to define the subject. From modelling – which can include physical models such as those used by architects, to the models of the world that we all carry in our heads – we limited our topic to the kind of models which use data, equations and computers. This is an area which is not only itself large but which is also growing as we develop bigger and bigger data sets and artificial intelligence systems to use them
The members of the group included people from the business worlds of insurance, retail, planning, economics, manufacturing, data science, and from academic and policy worlds in sociology, cyber security, fluid mechanics, informatics, epidemiology, and others. The culture clash was fun, engaging and stimulating!
The aim of the report is to help policy and decision makers become intelligent clients for models, which means that they need to have some idea of how models work and the strengths and weaknesses of their results. While the majority of models may still sit on spreadsheets, these can become hard to validate and rest too on data and assumptions which might be wrong. When I joined a new business a few years back, I asked to look at the business plan. A complicated spreadsheet duly arrived. I looked at the results and the assumptions which passed the test of common sense. But how to check the calculations? Finally, I took a sample of cells and chased them through the spreadsheet to check they were performing as advertised. Fortunately, they passed the test, and so did the business plan. I’m now chair of this new challenger bank!
A third of our report is devoted to guidance on how to be an intelligent customer. Such a customer needs to be both sceptical and understanding. Models can be enormously powerful in unpacking possible outcomes of a decision, and understanding and testing relationships. But they are only as good as the data and assumptions that underpin them. And there is an enormous difference between models which explain the past and those which can successfully predict the future. An important question is to ask how you know that the future will be like the past. Ask how much variation the past has seen and how wide the possible futures are, even before you have asked how your decision might change that future. Ask what kind of human behaviour you are expecting.
Of course, not all models are of human behaviour but in my own subject of economics, the variability of human behaviour and people’s ability to change behaviour in the face of changed incentives can lead to unintended consequences. If the Bank of England raises interest rates, I might interpret this as a sign of economic strength and buy sterling, or I might think this means an incipient downturn and sell. Either can – and has – happened.
Big data and clever algorithms might do better, but can also be hard to interpret. Such algorithms are able to work with more subtle mixes of variables than traditional approaches. In cities, this might give us a better handle of what mixes of policy will support stronger performance. In the light of the failure of regional policy over the last fifty years, this has a big potential payoff. But we have a long way to go before we can be comfortable with the results and how they could be interpreted. It seems clear that without infrastructure there can be no growth, but it is much less obvious what else needs to be put in place alongside any such investment.
Another major challenge is that of resilience. We have become quite good in some areas at modelling the ideal. We are less good at managing perturbations, whether those of a train system which is so busy that a breakdown can affect services across the country and for twenty-four hours, or those of a food delivery system which would run out of stocks in a day. All of this is before any deliberate attack upon our systems. Cyber security, protection of individual data and the right to be forgotten will loom larger and larger over the coming years.
The final sentence in the summary of our report reads ‘Computational modelling matters’. It does indeed and I hope you too read our report. It’s available at:
https://www.gov.uk/government/publications/computational-modelling-blackett-review
Bridget Rosewell