It’s possible to have the slightest wince at the idea that it should be the experts running everything. For when we do call upon experts we often enough find out that they’re not all that so. This is a corollary to Hayek’s point about the centre never being able to gather enough information to be able to plan in any detail. However, sadly, we do actually need to make decisions at times. Minarchy is not the same as anarchism.
Yet we’re still stuck with that problem of not many experts being expert. That infection model from Neil Ferguson at Imperial being a case in point. There’s a well known review of some of the code here:
Conclusions. All papers based on this code should be retracted immediately. Imperial’s modelling efforts should be reset with a new team that isn’t under Professor Ferguson, and which has a commitment to replicable results with published code from day one.
Which is fairly damning. But that’s just one other expert, right? So, a couple more who I happen to know are very much experts in their own fields. One such field being the same as this one, the other at least being in modelling.
Biggest take away: due to bugs in the code, the model results are not reproducible. The code itself introduces random variation in the model. That means that runs with the same inputs generate different outputs.
Are you fucking kidding me?
Reproducibility is the essence of science. A model whose predictions can not be reproduced, let alone empirical results based on that model, is so much crap. It is the antithesis of science.
I think it’s fair to say he doesn’t like it. And from within the world of computing, Admiral Hopper:
Clearly, for unit testing to work, you want a unit tester who is at least as ingenious (and motivated) as the coder. In most cases, the coder is the unit tester, so “soft” unit tests are unfortunately common – still, at least they’re a basis to argue that the code meets some kind of spec. And if the client isn’t happy with the tests, they’re free to add their own.
Why am I so mad at Neil Ferguson? He’s free to make whatever epidemiological assumptions that he wants, but he usurped the “authority” of computer modelling to assert that his model should be trusted, without actually undertaking the necessary and fundamental computer science practices – not least, unit testing.
And she doesn’t like it either. We might even go so far as to conclude that it’s not good code nor good science. But on the basis of this we closed down the global economy. Well done.
As ever the argument in favour of the government and their experts doing less is watching what the government and their experts do.