Max Planck famously described the progress of quantum physics as being "one funeral at a time" as the old-school physicists died off and their jobs were taken by young men who followed the new quantum religion.
This brutal style of scientific revolution has left some rather rigid scar tissue. For many years it has been almost taboo to suggest that the questions at the foundations of quantum mechanics might actually have an answer. Yet new results in different areas of physics, chemistry and engineering are beginning to suggest that there might possibly be an answer after all.
At the Solvay Conference in 1927, Niels Bohr and Werner Heisenberg out-debated Albert Einstein and Louis de Broglie; they persuaded the world that we should just take the tools of the new quantum mechanics on trust rather than trying to derive them from underlying classical principles. This Copenhagen school, the "shut up and calculate" school, of quantum mechanics rapidly became the orthodoxy. It was reinforced when calculations by John Bell were experimentally verified by Alain Aspect in 1982 and appeared to show that reality at the quantum level could not be both local and causal.
While some philosophers of physics toyed with exotic interpretations of quantum mechanics, most physicists shrugged; they accepted that quantum foundations were a "certified insoluble" problem, and told their graduate students not to even think about wasting their lives on that. Others just loved the idea that physics proves the world is too complex to understand, and that the proof is beyond the comprehension of outsiders. Physicists could be the new high priests as the quantum became the core magic. Recently we've got quantum with everything, from cryptography to biology; the word has become a magic spell for fundraising. So long as no-one dared challenge this for fear of being thought a crank or dismissed as an outsider, we were stuck.
Things are starting to change. In physics, Yves Couder and Emmanuel Fort found that bouncing droplets on a bath of vibrating oil mimic many phenomena previously thought unique to the quantum world, including single-slit and double-slit refraction, tunnelling and quantised energy levels. In chemistry, Masanao Ozawa and Werner Hofer have shown that the uncertainty principle is only approximately true: modern scanning probe microscopes can often measure the position and momentum of atoms slightly more accurately than Heisenberg predicted – which should worry people who claim that quantum cryptography is "provably" secure! In computing, the promised quantum computers are still stuck at factoring 15, despite hundreds of millions in research funding over almost twenty years. And the physicist Theo van Nieuwenhuizen has pointed out a contextuality loophole in Bell's theorem that looks rather hard to fix.
There's a striking parallel with another big problem in science—consciousness. For years, the few first-division academics who dared tackle such problems tended to be near retirement and famous enough to shrug off disapproval; just as Dan Dennett and Nick Humphrey wrote on consciousness, Tony Leggett and Gerard 't Hooft wrote on quantum foundations. So the flame was kept alight. But it's time to bring some tinder. Viennese physicists have now organised two symposia on emergent quantum mechanics, as people finally dare to wrestle with what might be going on down there.
So the idea I'd like to retire is the idea that some questions are just too big for normal working scientists to tackle. Old-timers should not try to erect taboos around the problems that eluded us. We must cheerfully challenge the young: "prove us wrong!" As for young scientists, they should dare to dream, and to aim high.