In 2008, celebrated editor and excellent prose-writer Chris Anderson wrote a provocative piece in Wired magazine with a more provocative title : The End of Theory: The Data Deluge Makes the Scientific Method Obsolete.
The article needs careful study because Mr. Anderson is a very knowledgeable man and he has enormous experience in the domain he is speaking. More important, he writes excellent prose with a rapid pace that entertains, informs, sways but unfortunately mis-guides where it matters most. The rebuttal is about those areas and this is true ‘correlationally and causationally’ as the rebuttal will show.
Before we start the argument, we need to make some terminological cleansing.
Theory : A model, a mental view of the reality to be tested.
Date Deluge : Volume, nature, formation speed, structure, mix of data which cannot be conceived through traditional methods and devices. Also known and marketed as BIG DATA
Scientific : Observation, Hypothesis, Testing
Here is the grand opening
Sixty years ago, digital computers made information readable. Twenty years ago, the Internet made it reachable. Ten years ago, the first search engine crawlers made it a single database. Now Google and like-minded companies are sifting through the most measured age in history, treating this massive corpus as a laboratory of the human condition. They are the children of the Petabyte Age.
The word ‘human condition’ is very suggestive. What is meant by human condition ? If this means humans of Pentabyte age only, then are there fundamental differences between humans of kilo-byte age ? Are human beings of successive ages getting different in a fundamental sense with each age ? What is the fundamental sense of being a human ? Mr. Anderson does not say anything about this most-important question and remains busy with the processes and data created by human beings. He is not asking the question that is the signature of human being and human being alone among all living beings : Who am I ? Am I a data radiator, to be captured and analysed ? Mr. Anderson does not halt to ponder this question and hence he is so free and happy to go next.
Speaking at the O’Reilly Emerging Technology Conference this past March, Peter Norvig, Google’s research director, offered an update to George Box’s maxim: “All models are wrong, and increasingly you can succeed without them.”
Agreed that all models are wrong. It implies that attempt to make any model will fail to interpret reality. But what is meant by ‘success’ then ? If I ask Mr. Norvig what is success for me as a human being (considering the fact that I have educated myself well not to make further models). What is the success of a human being then ? If we believe Mr. Anderson, we have progressed with the formidable achievement of ‘end of theory’ but left with two questions that are pricking us : a) What is a human being ? b) What is success for a human being ?
This is a world where massive amounts of data and applied mathematics replace every other tool that might be brought to bear. Out with every theory of human behavior, from linguistics to sociology. Forget taxonomy, ontology, and psychology. Who knows why people do what they do? The point is they do it, and we can track and measure it with unprecedented fidelity. With enough data, the numbers speak for themselves.
Here is a passage that is the key – the magnet that is holding everything together. Since none knows what is a human being (co-related with Who knows) and not interested to know the why (cause), the end of theory era is increasingly simple, insipid, and linear. We hear numbers and then do what….I ask a question now : Who created human beings ? This who and emboldened Who above bear a relationship and that relationship is critical. Without knowing the manufacturer of human being, studying the activities of human being, using however large data sets and engines will give us incorrect results. For example, let us consider an unfortunate boy who is not sure who his father his. He can approach BIG DATA, Google or other big data vendors but none can be trusted to be the ultimate authority. Who is the ultimate or no better authority : Mother.
Scientists are trained to recognize that correlation is not causation, that no conclusions should be drawn simply on the basis of correlation between X and Y (it could just be a coincidence). Instead, you must understand the underlying mechanisms that connect the two. Once you have a model, you can connect the data sets with confidence. Data without a model is just noise.
But faced with massive data, this approach to science — hypothesize, model, test — is becoming obsolete. Consider physics: Newtonian models were crude approximations of the truth (wrong at the atomic level, but still useful). A hundred years ago, statistically based quantum mechanics offered a better picture — but quantum mechanics is yet another model, and as such it, too, is flawed, no doubt a caricature of a more complex underlying reality. The reason physics has drifted into theoretical speculation about n-dimensional grand unified models over the past few decades (the “beautiful story” phase of a discipline starved of data) is that we don’t know how to run the experiments that would falsify the hypotheses — the energies are too high, the accelerators too expensive, and so on.
Very logical. We understand that the mental model of a scientist or a group is only limited version of what the reality is. Now bio-logy, the models are caricatures of the complex reality. But in that caricuature also, there was some attempt to penetrate into an ‘inconceivable’ aspect of creation and ‘human condition’. None of the models were fully true but they ‘mimicked’ something. Whom did they mimic ?
There’s no reason to cling to our old ways. It’s time to ask: What can science learn from Google?
Here the old ways means the old way of science and why so ?
Learning to use a “computer” of this scale may be challenging. But the opportunity is great: The new availability of huge amounts of data, along with the statistical tools to crunch these numbers, offers a whole new way of understanding the world. Correlation supersedes causation, and science can advance even without coherent models, unified theories, or really any mechanistic explanation at all.
This was expected… and all Sankhya Philosophy with an agonistic or atheistic philosophy will finally reach this voidistic or nihilistic conclusion. Pentabyte age and the philsophy thereof is nothing new – it has been quite ambient. Mr. Anderson is not aware of it and hence he delights so much. He mentions science can advance but what is advancement for science ?
Mr. Anderson and all people he quotes and refers to have no idea at all because of the fundamental take-off point for them : They are not asking the source of all this, they are checking the process somewhere in the middle and then trying to invent or predict the future.
The Theory Mr. Anderson has unconsciously felt and in a sense re-dicovered
a. Human beings suffer from four kinds of fundamental and irremovable defects : limited sense perception, propesnsity to commit mistake, to be illusioned and propensity to cheat.
b. Human beings are under stringent laws of nature. These laws operate in their own way irrespective ofatalogued th whether you have discovered the laws, named them, catalogued them or understood them. Your body with inconceivable amount of BIG DATA functions well without our understanding of them. No science or BIG DATA can create a human being but a human being can easily create another human being in a biological test-tube with the sexual pleasure as a great incentive. The four such laws under which we are completely controlled are : birth, death, old-age, disease.
c. The whole reality is so vast and complex that we shall not reach any conclusion on our own ( models, data, correlation) just like a boy will never reach the correct conclusion regarding his father without approaching the supreme authority in such matter : mother.
d. Mr. Anderson’s ‘end of theory’ is nothing but propunding another theory. His no-theory is a theory of testing teality. The ‘scientific advancement’ in this approach (or using this theory) is also bound to fail. If that is the case, why bother just like he admonishes us not to bother with the traditional scientific models.
e. There is a theory and that is the theory where we need to approach an authority who is beyond the four fundamental defects.
f. As for bio-logy, our very propensity to theorize and create a model shows that we share the same propensity with someone who has created us. Co-relation with the whole cosmic situation as the laboratory and eternal co-relation is no longer an error but a rule or law. An eternally co-incident happening is causation.
g. Mr. Anderson is very close to something very important – it is infinitely more valuable and important than the pentabyte childishness. His ‘end of theory’ points to the ‘Theory of God – the Supreme Person who has inconceivable potentcies.’
His end of theory is a great and purifying read but he could not take us there !