Surveying the landscape of scientific and technological change, there are a number of small and relatively steady advances that have unobtrusively combined to yield something startling, almost without anyone noticing. Through a combination of computer hardware advances (Moore’s Law marching on and so forth), ever-more sophisticated algorithms for solving certain mathematical challenges, and larger amounts of data, we have gotten something new: really good weather prediction.
According to a recent paper on this topic, “Advances in numerical weather prediction represent a quiet revolution because they have resulted from a steady accumulation of scientific knowledge and technological advances over many years that, with only a few exceptions, have not been associated with the aura of fundamental physics breakthroughs.” Despite these advances seeming to be profoundly unsexy, these predictive systems have yielded enormous progress. Our skill at forecasting the weather has grown significantly over the past few decades—something that will change how we think about the enormously complex system of the weather. Forecasting ability has been increasing in accuracy by about an additional day per decade for those made several days into the future.
This is intriguing and important for a number of reasons. First, understanding weather is vital for a huge number of human activities, from transportation to improving agricultural output, to even managing disasters. Being able to foretell what the weather holds affects nearly every aspect of our lives.
But there’s a potentially bigger reason. While I am hesitant to extrapolate from the weather system to other complex systems, including many that are perhaps much more complex (such as living organisms or entire ecosystems), this development should give us some hope. The fact that weather prediction has improved through a combination of technological advancement and scientific and modeling innovations means that other problems we might think to be ever-unsolvable needn’t be this way. While we might never fully handle incredibly complicated systems, they are not completely beyond our grasp.
This news of a “quiet revolution” in weather prediction might then be a touchstone for how to think about predicting and understanding complex systems: never say never when it comes to complete intractability.