… one of the great myths of our time?
Manzi writing in City Journal “What Social Science Does – and Doesn’t – Know”.
… with applications to evidence based medicine … “By about a quarter-century ago, however, it had become obvious to sophisticated experimentalists that the idea that we could settle a given policy debate with a sufficiently robust experiment was naive. The reason had to do with generalization, which is the Achilles’ heel of any experiment, whether randomized or not. In medicine, for example, what we really know from a given clinical trial is that this particular list of patients who received this exact treatment delivered in these specific clinics on these dates by these doctors had these outcomes, as compared with a specific control group. But when we want to use the trial’s results to guide future action, we must generalize them into a reliable predictive rule for as-yet-unseen situations. Even if the experiment was correctly executed, how do we know that our generalization is correct?”
… one of the author’s conclusions … “It is tempting to argue that we are at the beginning of an experimental revolution in social science that will ultimately lead to unimaginable discoveries. But we should be skeptical of that argument. The experimental revolution is like a huge wave that has lost power as it has moved through topics of increasing complexity. Physics was entirely transformed. Therapeutic biology had higher causal density, but it could often rely on the assumption of uniform biological response to generalize findings reliably from randomized trials. The even higher causal densities in social sciences make generalization from even properly randomized experiments hazardous. It would likely require the reduction of social science to biology to accomplish a true revolution in our understanding of human society—and that remains, as yet, beyond the grasp of science.”
These are two very perceptive comments. Especially the second, “The experimental revolution is like a huge wave that has lost power as it has moved through topics of increasing complexity.” The notion of “higher causal density,” if I’m understanding it correctly, implies a greater number of factors/variables to account for in attaining a generalization based upon a sample set. Since what obtains in one area (biology), a sort of regularity due to the nature of the organic parts involved, does not obtain in another (social science), due to the irregularity of the “parts” involved (the acting human person), reductionism becomes harder. I think another way to say this is that biology deals with parts that are not fundamentally free. Social sciences (unknowingly?) treat of parts that have a fundamental freedom.
Charles de Koninck would liken this to two inverse orders of simplicity that derive from imperfection and perfection. The experimental sciences treat of things that have a scale of simplicity/complexity that is based on imperfection: the simplest parts of physics and chemistry submit to number measurement and repeatable experimentation with regularity because there is not much to their existence. Atoms and compounds are limited compared with higher substances (plants, animals, men). Thus, advancing “upwards” in the experimental sciences becomes a job with increasing complexity based upon the “simple” parts that one starts with.
By contrast, the philosophical sciences deal with objects that have a scale of simplicity/complexity based upon perfection: the essences of the various orders of substances have intrinsically more and more perfection, and this is an intensive characteristic. The pinnacle of the material order is man, whose immaterial soul is the simplest in comparison to all other essence, even though all the other material substances are ordered to the human (man is a microcosm). Thus, man has parts in common with all the other orders (material, vegetative, sensitive), but unites them in an essence that is simpler than all of them (the rational human essence–the lowest sort of intellectuality–which compares as “complex” to angelic intellects, all of which created intellects are infinitely more “complex” than the perfectly simple Divine Intellect).
So, experimental science treats of simple, imperfect parts, and composes to understand higher kinds of things–a ‘one’ that is imperfect composes various multitudes. The philosophical treats of simple, more perfect essences, and analyzes to understand more imperfect and lower kinds of things–a ‘one’ that is perfect contains a multitude (humanity) or is above lower forms that are imperfect and less unified (God to angels to men). The experimental method is a sort of ‘myth’ since it purposefully ignored, from its inception (Bacon, Descartes) the viability of this second way of understanding the world.
For what it’s worth. I hope I haven’t misrepresented my teacher.
Of course, the experimental method is a useful sort of ‘myth’. A ‘myth’ or story has to tell you about particulars (these men, these events), and after enough such stories, patterns begin to emerge. The methods complement each other, but in different ways–in different orders.
I have the same opinion with most of your points, but a few need to be discussed further, I will hold a small talk with my buddies and perhaps I will ask you some suggestion shortly.
– Rob