OK - Jean is sitting up now...
A few people have asked questions and commented about the notion of ignorance so I'll continue to explain ignorance as cultivated by scientists.
The popular form of ignorance suggests a state of being uninformed. But I'm talking about something that is deeply honest. A recognition that you really don't know something. If it is an area you are curious about you can begin to explore. Is it worthwhile? Do you have a sense it is something that can be explored, how connected is it to other lines of exploration?
The process of learning more and exploring is a sharpening of the ignorance. The process of this curiosity driven ignorance is very dynamic whereas the standard form of ignorance tends to be static and dogmatic. Using the old black cat in a pitch block room that scientists like to use, there are a lot of these rooms - some are empty, some have black cats and some have black non-cats that we might imagine as well as those we may not. There are some ideas that seem great, but after years of pursuing them carefully you may not find anything, but you have learned something and perhaps can find some other entrances to ponder.
This is very different from the public perception of science which favors the great discoveries. This makes science seem to be a deliberate and reliable method for learning new facts and even getting things done - Just to this research and you will get a new battery for an electric car. All you need is a pile of money and a group of scientists.
That can work for developing existing ideas. Frequently the problems are so constrained that you can be fairly certain solutions exist and a scientific approach just works. This is more of an applied science or perhaps engineering research. It is the sort of thing that allows people who know the field well to predict slices of the future. Most industrial labs live in this space. A few, like the old Bell Telephone Laboratories and parts of IBM's Watson Labs, actually did real science. They, at least pieces of them, focused on what wasn't known. They relied on the cultivated ignorance of some very talented people and changed the world.
As an example I was part of an applied group that had a very good idea of where digital music might be going. We weren't in a position to predict winners and losers, but it was easy to project that five years out X would be possible and 10 years out Y would. And we did something very similar for location based services. Not that any of this was useful to our employer. It also wasn't science. (scientists can be very good at engineering research)
Prediction can be an excellent mechanism for cataloging ignorance. If you sit a scientist down and ask them to predict what might be in their field ten or fifteen years out, you will get some educated conjectures. It is really a list of ignorance she has identified as ripe for cultivation and perhaps she is even beginning to sharpen it. Really good scientific projections are very rare and probably no more than statistic flukes. But cataloging ignorance is useful as it can illuminate potential pathways and asking your scientist friend what may happen is a good way to engage them in what is, for them, a very exciting process.
Cultivated ignorance is terribly exciting. Science is not a terribly easy sport, but this drives scientists to do it. Admitting ignorance and moving on it is the point at which the questions sharpen and become more real. It is the start of the journey and the end of the journey, like a great road trip, is not the point. At some point there may be a discovery, measurement or non-discovery (which, as we mentioned, can be extremely important!), but that is not the goal of science. What happens along the path is the creation of new questions. Questions that weren't accessible and perhaps not even possible to express before and you realize the game is still afoot.
All of this is self-sustaining and quite thrilling.
It is, I think, a manifestation of what my buddy Jean Russell would call thrivable.
Along the way you've probably wondered why I haven't mentioned hypotheses - after all, don't scientists observe or theorize, make a clever hypothesis and then test it?
I'm not terribly fond of that model. For one thing the real process is much more messy than that simple model. Another is a hypothesis can be far too limiting. People tend to come to believe them and not be as skeptical and curious as they should be - at least when you are working in new areas. Too much belief in a potential outcome can often lead you astray or not be sufficiently careful with your experimental procedure. But there is a place where hypotheses develop more formally and you have to deal with them.
Scientists rate the quality of their ignorance as well as their findings. You measure and report a statistically sound confidence level (scientists tend to understand statistics deeply). This can get in the way of conveying science to the public as nothing is really black and white - finding can be proven wrong by experiment if that is the way Nature really works. There is always a chance you haven't measured carefully enough - that something is wrong with your technique or that some other signal is present.
It doesn't matter how beautiful your theory is, it doesn't matter how smart you are. If it doesn't agree with experiment, it's wrong. - Richard Feynman
The public expects black and white pronouncements and this breakdown was exploited by the tobacco industry in the 80s and 90s and have been exploited by the fossil fuel industry for more than a decade. A few weeks ago someone asked me about the consequences of a scientifically literate vs illiterate public and these are beautiful manifestations of some of the issues.1
If you know a scientist and want to really learn more about what they might be doing, the wrong question to ask is "what do you do?" Some friends have asked me this and I'm sure most come up with the conclusion that I have no discipline and can't describe something that simple. The right question is "what are you trying to find out?", "what would you really like to know?", "where are you stuck?", "what is unknowable these days?", "is there any progress in how well you can measure something?", "what would be really exciting to work on, but the technology isn't quite there yet?", "tell me about some new questions you would like to ask of Nature that have come to you in the last six months..."
I've spent quite a bit of time trying to explain energy and power at a layman's level and haven't been terribly successful. On the other hand, when confronted with questions - the sort of questions I can't answer, people around me tell me I can be animated and perhaps even a bit interesting. The level of the material changes from Russian doll science 101 explanations where you keep having to define and explain one set of things to cleanly explain another, to something crisper and much more respectful of your attention and mind. I'm still hunting for some clear ways to do this and think they are out there, but I need to be more clever.
And that is an interesting point - information is being generated at a fantastic rate. There is no way to keep up with it and that makes any field you aren't a specialist in seem inaccessible. It reduces experts to high priests in lab coats who speak in a confusing tongue. It renders science intimidating.
Scientists don't really worry about the information explosion at a core level of their curiosity. Sure they may as necessary part of something they are working on - just ask an experimental particle physicist about his trigger logic or someone sequencing DNA about their data handling issues. The important task is not information driven, although that is part of the process at some level. What is important is being curious and able to recognize and cultivate your honest ignorance.
The scientist deals with the explosion of information by realizing ignorance is growing at a much greater rate and this is a good thing. You can pick a bite-sized piece off and jump in!
Dealing with ignorance is something scientists are great at. Science can thrive - the information explosion is something of a red herring here. Science makes the Universe more accessible and, hopefully, science itself can be more accessible to the public.
The black cat and pitch black room metaphor may not be the best. A great professor of mine explained cultivated ignorance in terms of surfing (this was in Southern California, so do as the natives). First click here to set the mood with a bit of background music - I loved these guys when I lived there and recommend the electronic version of Pipeline
Imagine surfing this big frothy wave. As you act on your cultivated ignorance (he used the phrase targeted ignorance) you keep moving along and you begin to realize there are other waves that you never noticed before that you can move along to that will take you even farther. Making it to the beach becomes just silly as you realize you can go forever.
1 As something is verified and re-verified with increasingly accuratue experiments its confidence level increases it takes on a more solid nature - perhaps this is moving towards an honest and generally accepted hypothesis. Proving it wrong means you have to be increasingly careful when you do your experiment. The speed of light, the core notion of evolution, and basic quantum mechanics have very high levels of confidence attached to them. It makes sense to keep testing them as techniques improve, but it is unlikely that you'll prove them wrong (still it would be thrilling!)
It is usually the case that science usually builds on itself rather than going through true revolutions. Newtonian physics breaks down in certain areas - the very small and the very fast for example. In these regimes quantum mechanics and special relativity are better descriptions of Nature, but they are no better than Newtonian physics at describing much of the world we live in. We can say with some confidence that the simple Newtonian physics that describes the mechanics of a car will not change. Relativity and Quantum Mechanics both reduce to Newtonian physics as speeds and sizes become closer to the physical world we feel and observe with our naked senses.