ANT-style critique of ABM

A short recent article in the Journal of Artificial Societies and Social Simulation by Venturini, Jensen, and Latour lays out a critique of the explanatory strategy associated with agent-based modeling of complex social phenomena (link). (Thanks to Mark Carrigan for the reference via Twitter; @mark_carrigan.) Tommaso Venturini is an expert on digital media networks at Sciences Po (link), Pablo Jensen is a physicist who works on social simulations, and Bruno Latour is — Bruno Latour. Readers who recall recent posts here on the strengths and weaknesses of ABM models as a basis for explaining social conflict will find the article interesting (link). VJ&L argue that agent-based models — really, all simulations that proceed from the micro to the macro — are both flawed and unnecessary. They are flawed because they unavoidable resort to assumptions about agents and their environments that reduce the complexity of social interaction to an unacceptable denominator; and they are unnecessary because it is now possible to trace directly the kinds of processes of social interaction that simulations are designed to model. The “big data” available concerning individual-to-individual interactions permits direct observation of most large social processes, they appear to hold.

Here are the key criticisms of ABM methodology that the authors advance:

  • Most of them, however, partake of the same conceptual approach in which individuals are taken as discrete and interchangeable ‘social atoms’ (Buchanan 2007) out of which social structures emerge as macroscopic characteristics (viscosity, solidity…) emerge from atomic interactions in statistical physics (Bandini et al. 2009). (1.2)
  • most simulations work only at the price of simplifying the properties of micro-agents, the rules of interaction and the nature of macro-structures so that they conveniently fit each other. (1.4)
  • micro-macro models assume by construction that agents at the local level are incapable to understand and control the phenomena at the global level. (1.5)

And here is their key claim:

  • Empirical studies show that, contrarily to what most social simulations assume, collective action does not originate at the micro level of individual atoms and does not end up in a macro level of stable structures. Instead, actions distribute in intricate and heterogeneous networks than fold and deploy creating differences but not discontinuities. (1.11) 

This final statement could serve as a high-level paraphrase of actor-network theory, as presented by Latour in Reassembling the Social: An Introduction to Actor-Network-Theory. (Here is a brief description of actor-network theory and its minimalist social ontology; link.)

These criticisms parallel some of my own misgivings about simulation models, though I am somewhat more sympathetic to their use than VJ&L. Here are some of the concerns raised in earlier posts about the validity of various ABM approaches to social conflict (linklink):

  • Simulations often produce results that appear to be artifacts rather than genuine social tendencies.
  • Simulations leave out important features of the social world that are prima facie important to outcomes: for example, quality of leadership, quality and intensity of organization, content of appeals, differential pathways of appeals, and variety of political psychologies across agents.
  • The factor of the influence of organizations is particularly important and non-local.
  • Simulations need to incorporate actors at a range of levels, from individual to club to organization.
And here is the conclusion I drew in that post:
  • But it is very important to recognize the limitations of these models as predictors of outcomes in specific periods and locations of unrest. These simulation models probably don’t shed much light on particular episodes of contention in Egypt or Tunisia during the Arab Spring. The “qualitative” theories of contention that have been developed probably shed more light on the dynamics of contention than the simulations do at this point in their development.

But the confidence expressed by VJ&L in the new observability of social processes through digital tracing seems excessive to me. They offer a few good examples that support their case — opinion change, for example (1.9). Here they argue that it is possible to map or track opinion change directly through digital footprints of interaction (Twitter, Facebook, blogging), and this is superior to abstract modeling of opinion change through social networks. No doubt we can learn something important about the dynamics of opinion change through this means.

But this is a very special case. Can we similarly “map” the spread of new political ideas and slogans during the Arab Spring? No, because the vast majority of those present in Tahrir Square were not tweeting and texting their experiences. Can we map the spread of anti-Muslim attitudes in Gujarat in 2002 leading to massive killings of Muslims in a short period of time? No, for the same reason: activists and nationalist gangs did not do us the historical courtesy of posting their thought processes in their Twitter feeds either. Can we study the institutional realities of the fiscal system of the Indonesian state through its digital traces? No. Can we study the prevalence and causes of official corruption in China through digital traces? Again, no.

 
In other words, there is a huge methodological problem with the idea of digital traceability, deriving from the fact that most social activity leaves no digital traces. There are problem areas where the traces are more accessible and more indicative of the underlying social processes; but this is a far cry from the utopia of total social legibility that appears to underlie the viewpoint expressed here.
 
So I’m not persuaded that the tools of digital tracing provide the full alternative to social simulation that these authors assert. And this implies that social simulation tools remain an important component of the social scientist’s toolbox.
 

John von Neumann and stochastic simulations

source: Monte Carlo method (Wikipedia)

John von Neumann was one of the genuine mathematical geniuses of the twentieth century. A particularly interesting window onto von Neumann’s scientific work is provided by George Dyson in his  book, Turing’s Cathedral: The Origins of the Digital Universe. The book is as much an intellectual history of the mathematics and physics expertise of the Princeton Institute for Advanced Study as it is a study of any one individual, but von Neumann plays a key role in the story. His contribution to the creation of the general-purpose digital computer helped to lay the foundations for the digital world in which we now all live.

There are many interesting threads in von Neumann’s intellectual life, but one aspect that is particularly interesting to me is the early application of the new digital computing technology to the problem of simulating large complex physical systems. Modeling weather and climate were topics for which researchers sought solutions using the computational power of first-generation digital computers, and the research needed to understand and design thermonuclear devices had an urgent priority during the war and post-war years. Here is a description of von Neumann’s role in the field of weather modeling in designing the early applications of ENIAC  (P. Lynch, “From Richardson to early numerical weather prediction”; link):

John von Neumann recognized weather forecasting, a problem of both great practical significance and intrinsic scientific interest, as ideal for an automatic computer. He was in close contact with Rossby, who was the person best placed to understand the challenges that would have to be addressed to achieve success in this venture. Von Neumann established a Meteorology Project at the Institute for Advanced Study in Princeton and recruited Jule Charney to lead it. Arrangements were made to compute a solution of a simple equation, the barotropic vorticity equation (BVE), on the only computer available, the ENIAC. Barotropic models treat the atmosphere as a single layer, averaging out variations in the vertical. The resulting numerical predictions were truly ground-breaking. Four 24-hour forecasts were made, and the results clearly indicated that the large-scale features of the mid-tropospheric flow could be forecast numerically with a reasonable resemblance to reality. (Lynch, 9)

image: (link, 10)

A key innovation in the 1950s in the field of advanced computing was the invention of Monte Carlo simulation techniques to assist in the invention and development of the hydrogen bomb. Thomas Haigh, Mark Priestley, and Crispin Rope describe the development of the software supporting Monte Carlo simulations in the ENIAC machine in a contribution to the IEEE Annals of the History of Computing (link). Peter Galison offers a detailed treatment of the research communities that grew up around these new computational techniques (link). Developed first as a way of modeling nuclear fission and nuclear explosives, these techniques proved to be remarkably powerful for allowing researchers to simulate and calculate highly complex causal processes. Here is how Galison summarizes the approach:

Christened “Monte Carlo” after the gambling mecca, the method amounted to the use of random, numbers (a la roulette) to simulate the stochastic processes too complex to calculate in full analytic glory. But physicists and engineers soon elevated the Monte Carlo above the lowly status of a mere numerical calculation scheme; it came to constitute an alternative reality–in some cases a preferred one–on which “experimentation” could be conducted. (119) 

At Los Alamos during the war, physicists soon recognized that the central problem was to understand the process by which neutrons fission, scatter, and join uranium nuclei deep in the fissile core of a nuclear weapon. Experiment could not probe the critical mass with sufficient detail; theory led rapidly to unsolvable integro-differential equations. With such problems, the artificial reality of the Monte Carlo was the only solution–the sampling method could “recreate” such processes by modeling a sequence of random scatterings on a computer. (120)

The approach that Ulam, Metropolis, and von Neumann proposed to take for the problem of nuclear fusion involved fundamental physical calculations and statistical estimates of interactions between neutrons and surrounding matter. They proposed to calculate the evolution of the states of a manageable number of neutrons as they traveled from a central plutonium source through spherical layers of other materials. The initial characteristics and subsequent interactions of the sampled neutrons were assigned using pseudo-random numbers. A manageable number of sampled spaces within the unit cube would be “observed” for the transit of a neutron (127) (10^4 observations). If the percentage of fission calculated in the sampled spaces exceeded a certain value, then the reaction would be self-sustaining and explosive. Here is how the simulation would proceed:

Von Neumann went on to specify the way the simulation would run. First, a hundred neutrons would proceed through a short time interval, and the energy and momentum they transferred to ambient matter would be calculated. With this “kick” from the neutrons, the matter would be displaced. Assuming that the matter was in the middle position between the displaced position and the original position, one would then recalculate the history of the hundred original neutrons. This iteration would then repeat until a “self-consistent system” of neutron histories and matter displacement was obtained. The computer would then use this endstate as the basis for the next interval of time, delta t. Photons could be treated in the same way, or if the simplification were not plausible because of photon-matter interactions, light could be handled through standard diffusion methods designed for isotropic, black-body radiation. (129)

Galison argues that there were two fairly different views in play of the significance of Monte Carlo methods in the 1950s and 1960s. According to the first view, they were simply a calculating device permitting the “computational physicist” to calculate values for outcomes that could not be observed or theoretically inferred. According to the second view, Monte Carlo methods were interpreted realistically. Their statistical underpinnings were thought to correspond exactly to the probabilistic characteristics of nature; they represented a stochastic view of physics.

King’s view–that the Monte Carlo method corresponded to nature (got “back of the physics of the problem”) as no deterministic differential equation ever could–I will call

stochasticism

. It appears in myriad early uses of the Monte Carlo, and clearly contributed to its creation. In 1949, the physicist Robert Wilson took cosmic-ray physics as a perfect instantiation of the method: “The present application has exhibited how easy it is to apply the Monte Carlo method to a stochastic problem and to achieve without excessive labor an accuracy of about ten percent.” (146)

This is a very bold interpretation of a simulation technique. Rather than looking at the model as an abstraction from reality, this interpretation looks at the model as a digital reproduction of that reality. “Thus for the stochasticist, the simulation was, in a sense, of apiece with the natural phenomenon” (147).

One thing that is striking in these descriptions of the software developed in the 1950s to implement Monte Carlo methods is the very limited size and computing power of the first-generation general-purpose computing devices. Punch cards represented “the state of a single neutron at a single moment in time” (Haigh et al link 45), and the algorithm used pseudo-random numbers and basic physics to compute the next state of this neutron. The basic computations used third-order polynomial approximations (Haigh et al link 46) to compute future states of the neutron. The simulation described here resulted in the production of one million punched cards. It would seem that today one could use a spreadsheet to reproduce the von Neumann Monte Carlo simulation of fission, with each line being the computed result from the previous line after application of the specified mathematical functions to the data represented in the prior line. So a natural question to ask is — what could von Neumann have accomplished if he had Excel in his toolkit? Experts — is this possible?

An evolutionary view of research frameworks

social science methods

gene comparison

It was noted in a prior post that there is a great diversity of research frameworks in the social sciences, and that there is much to be gained by attempting to understand the social processes through which these frameworks change and develop over time (link).

Is it possible to get more specific about how the various research frameworks are related to each other, how similar or different they are, and how far apart they are in the space of the total universe of scientific research frameworks?

Here is an intriguing idea. We might think of a research framework as the compound of a set of “genes” in the form of a collection of ideas and practices through which scientists approach their studies of the world. And we might define the similarity or difference between two research frameworks in the way that geneticists define the similarity of two mammalian species, in terms of the degree of shared genetic material they possess.

On this approach, the research framework (methodology) constitutes a “code” which the young scientist learns through advanced training in the discipline. The resulting body of theory and research findings is the phenotype that results from the expression of the genotype through the activities of individuals and groups of scientists within the social and political environment of the research community.

The genotype of a methodology might consist of items like these:

  • ontological assumptions
  • investigatory strategies
  • practices of scientific collaboration
  • experimental designs
  • ideas about good explanation
  • procedures for evaluating evidence

On this view, scientific innovation takes place when researchers modify one or more elements of the code — background ontological beliefs, investigative practices, modes of confirmation, regulative ideas about theory and explanation. A given innovation may confer greater efficacy on the research framework by leading to more discoveries, more publications, or more funding.

By invoking the ideas of genotypes, phenotypes, and ecologies into the domain of research methodologies it is tempting to consider whether other aspects of the evolutionary paradigm are applicable as well, including the idea of evolution through selection.

Key here is the question of the mechanisms of fitness and selection that might be at work in the field of science. Do methodologies compete for survival and reproduction? Are there selection pressures at work in the domain of epistemology? Do scientific research frameworks “evolve” through small adjustments at the “gene” level (components of methodology at the level of theory and practice)? If so, do those selection processes lead to the evolution of methodologies better suited to discovering truth, or do they lead instead to methods that better serve the proximate interests of researchers and micro research communities?

One might argue that the two possibilities — selection pressure and veridicality — converge. Researchers use and refine methodologies based on their effort to generate publications, influence, and funding. One methodology is more “fit” than another insofar as it contributes to comparative advantage in these related outcomes. But methodologies generate influence, publication, and funding in proportion to the collective judgment of the expert community that they are creating new insight and more truthful representations of the world. So selection processes lead to greater veridicality.

This would be an agreeable outcome — the concrete practice of science leads generally to greater levels of truthful representation of the natural and social world. But is it plausible? Or does the history and sociology of science suggest that the factors that select for research methodologies are less epistemic and more situational or political? Does the process of science favor innovations that align with received views? Do scientific careers depend more on non epistemic factors than epistemic qualities? Does the process of science favor politically acceptable findings (“fracking is harmless” rather than “fracking causes micro-seismic activity”)? Are there contrarian “predator” practices at foot that actively contrive to push scientific findings away from truth (climate deniers, smoking industry advocates; Naomi Oreskes, Merchants of Doubt: How a Handful of Scientists Obscured the Truth on Issues from Tobacco Smoke to Global Warming)?

There are visible differences between biological evolution and the evolution of complexes of ideas and methods, of course. Biological evolution takes place through random variation at the genetic level, whereas theories and methods are subject to design by the scientists who develop them. Functional adaptation is the result of a blind process of selection in the biological world, whereas it is an intended consequence in the realm of theories and methods. And it is possible to define ecological fitness more precisely and mathematically in the biological realm than it is in the realm of science and culture.

It will be noted that this approach has a lot in common with the evolutionary approach to economics and the evolution of firms. A key text in this approach is Richard Nelson and Sidney Winter, An Evolutionary Theory of Economic Change. Richard Dawkins explored a similar view in The Selfish Gene when he considered “memes” as cultural counterparts to genes in the biological world.

Mechanisms, experiments, and policies

The social mechanisms approach to the social sciences aligns well with two key intellectual practices, experiments and policies. In an experiment we are interesting in testing whether a given factor has the effect it is thought to have. In a policy design we are interested in affecting an outcome of interest by manipulating some of the background conditions and factors. In both instances having a theory of the mechanisms in play in a domain permits us to frame our thinking better when it comes to designing experiments and policies.

Let’s say that we are interested in reducing the high school dropout rate in a high-poverty school. We may have a hypothesis that one important causal factor that leads to a higher likelihood of dropping out is that high-poverty students have a much greater burden of family and social problems than students in low-poverty populations. We might describe the mechanism in question in these terms:

H1: (a) high burden of social/familial problems => (b) student has higher likelihood of becoming discouraged => (c) student has higher likelihood of stopping attending => (d) student has a higher likelihood of dropping out of high school
We can evaluate this hypothesis about one of the mechanisms of dropping out of high school in several ways. First, we note that each clause invokes a likelihood. This means that we need to look at sets of students rather than individual students. Single cases or individual pairs of cases will not suffice, since we cannot make any inference from data like these:
A. Individual X has high burden of social/familial problems; Individual X does not become discouraged; Individual X does not drop out of high school.
B. Individual Y has a low burden of social/familial problems; Individual Y does become discouraged; Individual Y does drop out of high school.
Observations A and B are both compatible with the possible truth of the mechanisms hypothesis. Instead, we need to examine groups of individuals with various configurations of the characteristics mentioned in the hypothesis. If H1 is true, it can only be evaluated using population observations.
In theory we might approach H1 experimentally: randomly select two groups G1 and G2 of individuals; expose G1 to a high burden of social/familial problems while G2 is exposed to a low burden of social/familial problems; and observe the incidence of dropping out of high school. This would be to treat the hypothesis through an experiment based on the logic of random controlled trials. The difficulty here is obvious: we are harming the individuals in G1 in order to assess the causal consequences of the harmful treatment. This raises an irresolvable ethical problem. (Here is a discussion of Nancy Cartwright’s critique of the logic of RCT methodology in Evidence Based Policy; link.)
A slightly different experimental design would pass the ethics test. Select two schools S1 and S2 with comparable levels of high-poverty students and high burdens of social/familial problems for the individuals at the schools and comparable historical dropout rates. Now expose the students at S1 to a “treatment” that reduces the burden of social/familial problems (provide extensive social work services in the school that students can call upon). This design too conforms to the logic of a random controlled trial. Continue the treatment for four academic years and observe the graduation rates of the two schools. If H1 is true, we should expect that S1 will have a higher graduation rate than S2.
A third approach takes the form of a “quasi-experiment”. Identify pairs of schools that are similar in many relevant respects, but differ with respect to the burden of social/familial problems. This is one way of “controlling” for the causal influence of other observable factors — family income, race, degree of segregation in the school, etc. Now we have N pairs of matched schools and we can compute the graduation rate for the two components of the matches; that is, graduation rates for “high burden school” and “low burden school”. If we find that the high burden schools have a lower graduation rate than the low burden schools, and if we are satisfied that the schools do not differ systematically in any other dimension, then we have a degree of confirmation for the causal hypothesis H1. But Stanley Lieberson in Making It Count poses some difficult challenges for the logic of this kind of experimental test; he believes that there are commonly unrecognized forms of selection bias in the makeup of the test cases that potentially invalidates any possible finding (link).
So far we have looked at ways of experimentally evaluating the link between (a) and (d). But H1 is more complex; it hypothesizes that social/familial problems exercise their influence through two behavioral stages that may themselves be the object of intervention. The link from (b) to (c) is an independent hypothetical causal relation, and likewise the link from (c) to (d). So we might attempt to tease out the workings of these links in the mechanism as well. Here we might design our experiments around populations of high burden students, but attempt to find ways of influencing either discouragement or the link from discouragement to non-attendance (or possibly the link from non-attendance to full dropping out).
Here our intervention might go along these lines: the burden of social/familial problems is usually exogenous and untreatable. But within-school programs like intensive peer mentoring and encouragement might serve to offset the discouragement that otherwise results from high burden of social/familial problems. This can be experimentally evaluated using one or another of the designs mentioned above. Or we might take discouragement as a given but find an intervention that prevents the discouraged student from becoming a truant — perhaps a strong motivational incentive dependent on achieving 90% attendance during a six-week period.
In other words, causal hypotheses about causal mechanisms invite experimental and quasi-experimental investigation.

What about the other side of the equation; how do hypotheses about mechanisms contribute to policy intervention? This seems even more straightforward than the first question. The mechanism hypothesis points to several specific locations where intervention could affect the negative outcome with which we are concerned — dropping out of high school in this case. If we have experimental evidence supporting the links specified in the hypothesis, then equally we have a set of policy options available to us. We can design a policy intervention that seeks to do one or more of the following things: reduce the burden of social/familial problems; increase the level of morale of students who are exposed to a high burden; find means of encouraging high-burden students to persevere; and design an intervention to encourage truants to return to school. This suite of interventions touches each of the causal connections specified in the hypothesis H1.

Now, finally, we are ready to close the circle by evaluating the success of interventions like these. Does the graduation rate of schools where the interventions have been implemented work out to be higher than those where the interventions were not implemented? Can we begin to assign efficacy assessments to various parts of the policy? Can we arrive at secondary hypotheses about why this policy intervention (“reduce the burden of social/familial issues”) doesn’t succeed, whereas another policy intervention (“bolster morale among high-risk students”) does appear to succeed?
The upshot is that experiments and policies are opposite sides of the same coin. Both proceed from the common assumption that social causes are real; that we can assess the causal significance of various factors through experimentation and controlled observation; and that we can intervene in real-world processes with policy tools designed to exert influence at key junctures in the causal process.

George and Bennett on case study methodology

 

Establishing causal relationships within the fabric of the social world is more challenging than in the biological or physical-chemical domains. The reasons for this difficulty are familiar — the high degree of contextuality and contingency that is characteristic of social change, the non-deterministic character of social causation, and the fact that most social outcomes are the result of unforeseen conjunctions of independent influences, to name several.

Alexander George and Andrew Bennett argue for the value of a case-study method of social research in Case Studies and Theory Development in the Social Sciences. The idea here is that social researchers can learn about the causation of particular events and sequences by examining them in detail and in comparison with carefully selected alternative examples.

Here is how they describe the case-study method:

The method and logic of structured, focused comparison is simple and straightforward. The method is “structured” in that the researcher writes general questions that reflect the research objective and that these questions are asked of each case under study to guide and standardize data collection, thereby making systematic comparison and cumulation of the findings of the cases possible. The method is “focused” in that it deals only with certain aspects of the historical cases examined. The requirements for structure and focus apply equally to individual cases since they may later be joined by additional cases. (67)

George and Bennett believe that the techniques and heuristics of the case study approach permit the researcher to arrive at rigorous and differentiated hypotheses about underlying social processes. In particular, they believe that the method of process-tracing has substantial power in social research, permitting the researcher to move from the details of a particular historical case to more general hypotheses about causal mechanisms and processes in other contexts as well (6). They discourage research strategies based on the covering-law model, in which researchers would seek out high-level generalizations about social events and outcomes: “highly general and abstract theories … are too general to make sharp theoretical predictions or to guide policy” (7). But they also note the limits of policy relevance of “independent, stable causal mechanisms” (7), because social mechanisms interact in context-dependent ways that are difficult or impossible to anticipate. It is therefore difficult to design policy interventions based on knowledge of a few relevant and operative mechanisms within the domain of behavior the policy is expected to govern, since the workings of the mechanisms in concrete circumstances are difficult to project.

Fundamentally they align with the causal mechanisms approach to social explanation. Here is how they define a causal mechanism:

We define causal mechanisms as ultimately unobservable physical, social, or psychological processes through which agents with causal capacities operate, but only in specific contexts or conditions, to transfer energy, information, or matter to other entities. In so doing, the causal agent changes the affected entity’s characteristics, capacities, or propensities in ways that press until subsequent causal mechanisms act upon it. (137)

And they believe that the case-study method is a suite of methodological approaches that permit identification and exploration of underlying causal mechanisms.

The case study approach – the detailed examination of an aspect of a historical episode to develop or test historical explanations that may be generalizable to other events – has come in and out of favor over the past five decades as researchers have explored the possibilities of statistical methods … and formal models. (5)

The case study method is designed to identify causal connections within a domain of social phenomena.

Scientific realists who have emphasized that explanation requires not merely correlational data, but also knowledge of intervening causal mechanisms, have not yet had much to say on methods for generating such knowledge. The method of process-tracing is relevant for generating and analyzing data on the causal mechanisms, or processes, events, actions, expectations, and other intervening variables, that link putative causes to observed effects. (214)

How is that to be accomplished? The most important tool that George and Bennett describe is the method of process tracing. “The process-tracing method attempts to identify the intervening causal process–the causal chain and causal mechanism–between an independent variable (or variables) and the outcome of the dependent variable” (206). Process tracing requires the researcher to examine linkages within the details of the case they are studying, and then to assess specific hypotheses about how these links might be causally mediated. 

Suppose we are interested in a period of violent mobilization VM in the countryside at time t, and we observe a marked upswing of religious participation RP in the villages where we have observations. We might hypothesize that the surge of religious participation contributed causally to the political mobilization that ensued. But a process-tracing methodology requires that we we consider as full a range of alternative possibilities as we can: that both religious and political activism were the joint effect of some other social process; that religious participation was caused by political mobilization rather than caused that mobilization; that the two processes were just contingent and unrelated simultaneous developments. What can we discover within the facts of the case that would allow us to disentangle these various causal possibilities? If RP was the cause of VM, there should be traces of the influence that VM exerted within the historical record — priests who show up in the interrogation cells, organizational linkages that are uncovered through archival documents, and the like. This is the work of process tracing in the particular case. And I agree with George and Bennett that there is often ample empirical evidence available in the historical record to permit this kind of discovery.

Finally, George and Bennett believe that process-tracing can occur at a variety of levels:

The simplest variety of process-tracing takes the form of a detailed narrative or story presented in the form of a chronicle that purports to throw light on how an event came about…. A substantially different variety of process-tracing converts a historical narrative into an analytical causal explanation couched in explicit theoretical forms…. In another variety of process-tracing, the investigator constructs a general explanation rather than a detailed tracing of a causal process. (210-211)

One of the strengths of the book is an appendix presenting a very good collection of research studies that illustrate the case study methodology that they explore. There are examples from American politics, comparative politics, and international relations. These examples are very helpful because they give substance to the methodological ideas presented in the main body of the book.

Geddes on methods

Earlier posts have examined some recent thinking about social science methods (link, link). Here I will examine another recent contributor to this field, Barbara Geddes.

Geddes is a specialist in comparative politics, and her 2003 Paradigms and Sand Castles: Theory Building and Research Design in Comparative Politics is a thoughtful contribution to the debate about how the social sciences should proceed. Her central concern is with the topic of research design in comparative politics. How should a comparative researcher go about attempting to explain the varying outcomes we observe within the experiences of otherwise similar countries? How can we gain empirical grounds for validating or rejecting causal hypotheses in this field? And how do general theories of politics fare as a basis for explaining these concrete trajectories — the rise of authoritarianism in one country, the collapse of communism in the USSR, an outbreak of democracy in that country, or a surprising populism in another? Geddes finds that the theories that guided comparative politics in the sixties, seventies, and eighties proved to be inadequate to the task of explaining the twists and turns the political systems of the world took during those decades and argues that the discipline needs to do better.

Geddes’s proposed solution to this cul de sac is to bring theory and research design closer together. She wants to find a way of pursuing research in comparative politics that permits for more accumulation of knowledge in the field, both on the side of substantial empirical findings and well grounded theoretical premises. Theoretical premises need to be more carefully articulated, and plans for data collection need to be more purposefully guided so the resulting empirical findings are well suited to evaluating and probing the theoretical premises. Here is a good summary paragraph of her view:

The central message of this book is that we could steer a course through that narrow channel between untested theory and atheoretical data more successfully, and thus accumulate theoretical knowledge more rapidly, if certain research norms were changed. Although research norms are changing, basic principles of research design continue to be ignored in many studies. Common problems include inappropriate selection of cases from which to draw evidence for testing theories and a casual attitude towards nonquantitative measurement, both of which undermine the credibility of evidence gathered to support arguments. The failure to organize and store evidence in ways that make it accessible to others raises the cost of replication and that also slows theoretical progress. Uncritical acceptance by readers of theories that have not undergone systematic empirical test exacerbates the problem. (5)

What does Geddes mean by “theory” in this context? Her examples suggest that she thinks of a theory as a collection of somewhat independent causal hypotheses about a certain kind of large social outcome — the emergence of democracy or the occurrence of sustained economic development, for example. So when she discusses the validity of modernization theory, she claims that some components were extensively tested and have held up (the correlation between democracy and economic development, for example; 9), whereas other components were not adequately tested and have not survived (the claim that the diffusion of values would rapidly transform traditional societies; 9).

Geddes does not explicitly associate her view of social science inquiry with the causal mechanisms approach. But in fact the intellectual process of inquiry that she describes has a great deal in common with that approach. On her view of theory, the theory comes down to a conjunction of causal hypotheses, each of which can in principle be tested in isolation. What she refers to as “models” could as easily be understood as schematic descriptions of common social mechanisms (33). The examples she gives of models are collective action problems and evolutionary selection of social characteristics; and each of these is a mechanism of social causation.

She emphasizes, moreover, that the social causal factors that are at work in the processes of political and economic development generally work in conjunction with each other, with often unpredictable consequences.

Large-scale phenomena such as democratic breakdown, economic development, democratization, economic liberalization, and revolution result from the convergence of a number of different processes, some of which occur independently from others. No simple theory is likely to explain such compound outcomes.  Instead of trying to “explain” such compound outcomes as wholes, I suggest a focus on the various processes that contribute to the final outcome, with the idea of theorizing these processes individually. (27)

What Geddes’s conception of “theory” seems to amount to is more easily formulated in the language of causal mechanisms. We want to explain social outcomes at a variety of levels of scale — micro, meso, macro. We understand that explanation requires discovery of the causal pathways and processes through which the outcome emerged. We recognize that social outcomes have a great deal of contingency and path dependency, so it is unlikely that a great outcome like democratization will be the result of a single pervasive causal factor. Instead, we look for mid-level causal mechanisms that are in place in the circumstances of interest — say the outbreak of the Bolshevik uprising; and we attempt to discern the multiple causal factors that converged in these historical circumstances to bring about the outcome of interest. The components of theories to which Geddes refers are accounts of reasonably independent causal mechanisms and processes, and they combine in contingent and historically specific ways.

And in fact she sometimes adopts this language of independent mid-level causal mechanisms:

To show exactly what I mean, in the pages that follow I develop a concrete research strategy that begins with the disaggregation of the big question — why democratization occurs — into a series of more researchable questions about mechanisms. The second step is a theorization of the specific process chosen for study — in this case, the internal authoritarian politics that sometimes lead to transition. The third step is the articulation of testable implications derived from the theorization. (43)

And later:

I argued that greater progress could be made toward actually understanding how such outcomes [as democratization and authoritarian rule] by examining the mechanisms and processes that contribute to them, rather than through inductive searches for the correlates of the undifferentiated whole. (87)

(This parallels exactly the view taken by McAdam, Tarrow, and Tilly in Dynamics of Contention, where they argue systematically for a form of analysis of episodes of contention that attempts to identify recurring underlying processes and mechanisms.)

It emerges that what Geddes has in mind for testing mid-level causal hypotheses is largely quantitative: isolate a set of cases in which the outcome is present and examine whether the hypothesized causal factor varies appropriately across the cases. Do military regimes in fact persist with shorter average duration than civilian authoritarian regimes (78)? Like King, Keohane, and Verba in Designing Social Inquiry: Scientific Inference in Qualitative Research, Geddes is skeptical about causal methods based on comparison of a small number of cases; and like KKV, she is critical of Skocpol’s use in States and Social Revolutions: A Comparative Analysis of France, Russia and China of Mill’s methods in examining the handful of cases of social revolution that she examines. This dismissal of small-N research represents an unwelcome commitment to methodological monism, in my view.

In short, I find Geddes’s book to be a useful contribution that aligns more closely than it appears with the causal mechanisms approach to social research. It is possible to paraphrase Geddes’s approach to theory and explanation in the language of causal mechanisms, emphasizing meso-level analysis, conjunctural causation, and macro-level contingency. (More on this view of historical causation can be found here.)

Geddes’s recommendations about how to probe and test the disaggregated causal hypotheses at which the researcher arrives represent one legitimate approach to the problem of giving greater empirical content to specific hypotheses about causal mechanisms. It is regrettable, however, that Geddes places her flag on the quantitative credo for the social sciences. One of the real advantages of the social mechanisms approach is precisely that we can gain empirical knowledge about concrete social mechanisms through detailed case studies, process tracing, and small-N comparisons of cases that is not visible at the level of higher-level statistical regularities. (A subsequent post will examine George and Bennett, Case Studies and Theory Development in the Social Sciences (Belfer Center Studies in International Security), for an alternative view of how to gain empirical knowledge of social processes and mechanisms.)

Heuristics for a mechanisms-based methodology

Let’s imagine that I’m a young sociologist or political scientist who has gotten interested in the social-mechanisms debates, and I’d like to frame my next research project around a set of heuristics that are suggested by the mechanisms approach. What might some of those heuristics look like? What is a “mechanisms-based methodology” for sociological research? And how would my research play out in concrete terms? Here are a few heuristics we might consider.

  1. Identify one or more clear cases of the phenomenon I’m interested in understanding
  2. Gain enough empirical detail about the cases to permit close examination of possible causal linkages
  3. Acquaint myself with a broad range of social mechanisms from a range of the social sciences (political science, economics, anthropology, public choice theory, critical race studies, women’s studies, …)
  4. Attempt to segment the phenomena into manageable components that may admit of separate study and analysis
  5. Use the method of process-tracing to attempt to establish what appear to be causal linkages among the phenomena
  6. Use my imagination and puzzle-solving ability to attempt to fit one or more of the available mechanisms into the phenomena I observe
  7. Engage in quasi-experimental reasoning to probe the resulting analysis: if mechanism M is involved, what other effects would we expect to be present as well? Do the empirical realities of the case fit these hypothetical expectations?

These heuristics represent in a rough-and-ready way the idea that there are some well understood social processes in the world that have been explored in a lot of empirical and theoretical detail. The social sciences collectively provide a very rich toolbox of mechanisms that researchers have investigated and validated. We know how these mechanisms work, and we can observe them in a range of settings. This is a realist observation: the social world is not infinitely variable, and there is a substrate of activity, action, and interaction whose workings give rise to a number of well understood mechanisms. Here I would include free rider problems, contagion, provocation, escalation, coercion, and log-rolling as a very miscellaneous set of exemplars. So if we choose to pursue a mechanisms-based methodology, we are basically following a very basic intuition of realism by asking the question, “how does this social phenomenon work in the settings in which we find it?”.

So how might a research project unfold if we adopt heuristics like these? Here is a striking example of a mechanisms approach within new-institutionalist research, Jean Ensminger’s account of bridewealth in the cattle-herding culture of Kenya (Making a Market: The Institutional Transformation of an African Society). First, some background. The cattle-herding economic regime of the Orma pastoralists of Kenya underwent substantial changes in the 1970s and 1980s. Commons grazing practices began to give way to restricted pasturage; wage labor among herders came to replace familial and patron-client relations; and a whole series of changes in the property system surrounding the cattle economy transpired as well. This is an excellent example for empirical study from a new-institutionalist perspective. What explained the particular configuration of norms and institutions of the earlier period? And what social pressures led to the transition towards a more impersonal relationship between owners and herders? These are questions about social causation at multiple levels.

Ensminger examines these questions from the perspective of the new institutionalism. Building on the theoretical frameworks of Douglass North and others, she undertakes to provide an analysis of the workings of traditional Orma cattle-management practices and an explanation of the process of change and dissolution that these practices underwent in the decades following 1960. The book puts forward a combination of close ethnographic detail and sophisticated use of theoretical ideas to explain complex local phenomena.

How does the new institutionalism approach help to explain the features of the traditional Orma cattle regime identified by Ensminger’s study? The key institutions in the earlier period are the terms of employment of cattle herders in mobile cattle camps. The traditional employment practice takes the pattern of an embroidered patron-client relation. The cattle owner provides a basic wage contract to the herder (food, clothing, and one head of cattle per year). The good herder is treated paternally, with additional “gifts” at the end of the season (additional clothing, an additional animal, and payment of the herder’s bridewealth after years of service). The relation between patron and client is multi-stranded, enduring, and paternal.

Ensminger understands this traditional practice as a solution to an obvious problem associated with mobile cattle camps, which is fundamentally a principal-agent problem. Supervision costs are very high, since the owner does not travel with the camp. The owner must depend on the herder to use his skill and diligence in a variety of difficult circumstances—rescuing stranded cattle, searching out lost animals, and maintaining control of the herd during harsh conditions. There are obvious short-term incentives and opportunities for the herder to cheat the employer—e.g. allowing stranded animals to perish, giving up on searches for lost animals, or even selling animals during times of distress. The patron-client relation is one possible solution to this principal-agent problem. An embedded patron-client relation gives the herder a long-term incentive to provide high-quality labor, for the quality of work can be assessed at the end of the season by assessment of the health and size of the herd. The patron has an incentive to cheat the client—e.g. by refusing to pay the herder’s bridewealth after years of service. But here the patron’s interest in reputation comes into play: a cattle owner with a reputation for cheating his clients will find it difficult to recruit high-quality herders.

This account serves to explain the evolution and persistence of the patron-client relation in cattle-camps on the basis of transaction costs (costs of supervision). Arrangements will be selected that serve to minimize transaction costs. In the circumstances of traditional cattle-rearing among the Orma the transaction costs of a straight wage-labor system are substantially greater than those associated with a patron-client system. Therefore the patron-client system is selected.

This analysis identifies mechanisms at two levels. First, the patron-client relation is the mechanism through which the endemic principal-agent problem facing cattle owners is solved. The normal workings of this relation give both patron and client a set of incentives that leads to a stable labor relation. The higher-level mechanism is somewhat less explicit, but is needed for the explanation to fully satisfy us. This is the mechanism through which the new social relationship (patron-client interdependency) is introduced and sustained. It may be the result of conscious institutional design or it may be a random variation in social space that is emulated when owners and herders notice the advantages it brings. Towards the end of the account we are led to inquire about another higher-level mechanism, the processes through which the traditional arrangement is eroded and replaced by short-term labor contracts.

This framework also illustrates the seventh heuristic above, the use of counterfactual reasoning. This account would suggest that if transaction costs change substantially (through improved transportation, for example, or through the creation of fixed grazing areas), that the terms of employment would change as well (in the direction of less costly pure wage-labor contracts). And in fact this is what Ensminger finds among the Orma. When villages begin to establish “restricted grazing areas” in the environs of the village, it is feasible for cattle owners to directly supervise the management of their herds; and in these circumstances Ensminger finds an increase in pure wage labor contracts.

What are the scientific achievements of this account? There are several. First, it takes a complicated and detailed case of collective behavior and it makes sense of the case. It illuminates the factors that influence choices by the various participants. Second, it provides insight into how these social transactions work (the mechanisms that are embodied in the story). Third, it begins to answer — or at least to pose in a compelling way — the question of the driving forces in institutional change. This too is a causal mechanism question; it is a question that focuses our attention on the concrete social processes that push one set of social behaviors and norms in the direction of another set of behaviors and norms. Finally, it is an empirically grounded account that gives us a basis for a degree of rational confidence in the findings. The case has the features that we should expect it to have if the mechanisms and processes in fact worked as they are described to do.

A final achievement of this account is very helpful in the context of our efforts to arrive at explanations of features of the social world. This is the fact that the account is logically independent of an effort to arrive at strong generalizations about behavior everywhere. The account that Ensminger provides is contextualized and specific, and it does not depend on the assumption that similar social problems will be solved in the same way in other contexts. There is no underlying assumption that this interesting set of institutional facts should be derivable from a general theory of behavior and institutions. Instead, the explanation is carefully crafted to identify the specific (and perhaps unique) features of the historical setting in which the phenomenon is observed.

(Here is a nice short article by David Collier on the logic of process-tracing; link. And here is an interesting piece by Aussems, Boomsma, and Snijders on the use of quasi-experimental methods in the social sciences; link.)

Realism and methodology

Methodology has to do with the strategies and heuristics through which we attempt to understand a complicated empirical reality (link). Our methodological assumptions guide us in the ways in which we attempt to collect data, the kinds of data we collect, the explanatory hypotheses we bring forward for that range of empirical findings, and the ways we seek to validate our findings. Methodology is to the philosophy of social science as historiography is to the philosophy of history.

Realism is also a set of assumptions that we bring to empirical investigation. But in this case the assumptions are about ontology — how the world works, in the most general ways. Realism asserts that there are real underlying causes, structures, processes, and entities that give rise to the observations we make of the world, natural and social. And it postulates that it is scientifically appropriate to form theories and hypotheses about these underlying causes in order to arrive at explanations of what we observe.

This description of realism is couched in terms of a distinction between what is observable and what is unobservable but nonetheless real — the “observation-theoretic” distinction. But of course the dividing line between the two categories shifts over time. What was once hypothetical becomes observable. Extra-solar planetary bodies, bosons, and viruses were once unobservable; they are now observable using various forms of scientific instrumentation and measurement. So the distinction is not fundamental; this was an essential part of the argument against positivist philosophy of science. And we might say the same about many social entities and structures as well. We understand “ideology” much better today than when Marx theorized about this idea in the mid-19th century, and using a variety of social research methods (public opinion surveys, World Values Survey, ethnographic observation, structured interviews) we can identify and track shifts in the ideology of a group over time. We can observe and track ideologies in a population. (We may now use a different vocabulary — mentality, belief framework, political values.)

There are several realist methodologies that are possible in the social sciences. The methodology of paired comparisons is a common part of research strategies in the historical social sciences. This is often referred to as “small-N research.” (Here is a description of the method as practiced by Sid Tarrow; linklink.) The method of paired comparisons is also based on realism and derives from causal ideas; but it is not specifically derived from the idea of causal mechanisms.  Rather, it derives from the simpler notion that causal factors function as something like necessary and/or sufficient conditions for outcomes. So if we can find cases that differ in outcome and embody only a small number of potential contributing causal factors, we can use Mill’s methods (or more general truth-table methods) to sort out the causal roles played by the factors. (Here is a discussion of some of these concepts; link.) These ideas contribute to methodology at two levels: they give the investigator a specific idea about how to lay out his/her research (“seek out relevantly similar cases with different outcomes”), and they embody a method of inference from findings to conclusions about causal relations (the truth-table method). These methods allow the researcher to arrive at statements about which factors play a role in the production of other factors. (This is a logically similar role to the use of multiple regression in quantitative studies.)

Another possible realist approach to methodology is causal mechanisms theory (CM). It rests on the idea that events and outcomes are caused by specific happenings and powers, and it proposes that a good approach to a scientific explanation of an outcome or pattern is to discover the real mechanisms that typically bring it about. It also brings forward an old idea about causation — no action at a distance. So if we want to maintain that class privilege causes ideological commitment, we need to be able to tell an empirically grounded story about how the first kind of thing conveys its influence to changes in the second kind of thing. (This is essentially the call for microfoundations; link.) Causal mechanisms theory is more basic than either paired comparisons or statistical causal modeling, in that it provides a further explanation for findings produced by either of these other methods. Once we have a conception of the mechanisms involved in a given social process, we are in a position to interpret a statistical finding as well as a finding about the necessary and/or sufficient conditions provided by a list of antecedent conditions for an outcome.

It is an interesting question to consider whether realism in ontology leads to important differences in methodology. In particular, does the idea that things happen as the result of an ensemble of real causal mechanisms that can be separately understood lead to important new ideas about methodology and inquiry?

Craver and Darden argue in In Search of Mechanisms: Discoveries across the Life Sciences that mechanisms theory does in fact contribute substantially to contemporary research in biology, at a full range of levels (link). They maintain that the key goal for much research in contemporary biology is to discover the mechanisms that produce an outcome, and that a central component of this methodology is the effort to explain a given phenomenon by trying to fit one or more known mechanisms to the observed process. So working with a toolbox of known mechanisms and “problem-solving” to account for the new phenomenon is an important heuristic in biology. This approach is both ontological and methodological; it presupposes that there are real underlying mechanisms, and it recommends to the researcher that he/she be well acquainted with the current inventory of known mechanisms that may be applied to new settings.

I think there is a strong counterpart to this idea in a lot of sociological research as well. There are well understood social mechanisms that sociologists, political scientists, and other researchers have documented — easy riders, prisoners dilemmas, conditional altruism — and the researcher often can systematically explore whether one or more of the known mechanisms is contributing to the complex social outcomes he or she is concerned with. A good example is found in Howard Kimeldorf’s Reds or Rackets?: The Making of Radical and Conservative Unions on the Waterfront. Kimeldorf compares two detailed case histories and strives to identify the concrete social mechanisms that led to different outcomes in the two cases. The mechanisms are familiar from other sociological research; Kimeldorf’s work serves to show how specific mechanisms were in play in the cases he considers.

This kind of work can be described as problem-solving heuristics based on application of a known inventory of mechanisms. It could also be described as a “normal science” process where small theories of known processes are redeployed to explain novel outcomes. As Kuhn maintains, normal science is incremental but creative and necessary in the progress of science.

A somewhat more open-ended kind of inquiry is aimed at discovery of novel mechanisms. McAdam, Tarrow and Tilly sometimes engage in this second kind of discovery in Dynamics of Contention — for example, the mechanism of social disintegration (kl 3050). Another good example of discovery of mechanisms is Akerlof’s exposition of the “market for lemons” (link), where he lays out the behavioral consequences of market behavior with asymmetric knowledge between buyer and seller.

So we might say that mechanisms theory gives rise to two different kinds of research methodology — application of the known inventory to novel cases and search for novel mechanisms (based on theory or empirical research).

Causal-mechanisms theory also suggests a different approach to data gathering and a different mode of reasoning from both quantitative and comparative methods. The approach is the case-studies method: identify a small set of cases and gain enough knowledge about how they played out to be in a position to form hypotheses about the specific causal linkages that occurred (mechanisms).

This approach is less interested in finding high-level generalizations and more concerned about the discovery of the real inner workings of various phenomena. Causal mechanisms methodology can be applied to single cases (the Russian Revolution, the occurrence of the Great Leap Forward famine), without the claim to offering a general causal account of famines or revolutions. So causal mechanisms method (and ontology) pushes downward the focus of research, from the macro level to the more granular level.

The inference and validation component associated with CM looks like a combination of piecemeal verification (link) and formal modeling (link). The case-studies approach permits the researcher to probe the available evidence to validate specific hypotheses about the mechanisms that were present in the historical case. The researcher is also able to try to create a simulation of the social situation under study, confirm as much of the causal internal connectedness as possible from study of the case, and examine whether the model conforms in important respects to the observed outcomes. Agent-based models represent one such set of modeling techniques; but there are others.

So the methodological ideas associated with CM theory differ from both small-N and large-N research. The search for causal mechanisms is largely agnostic about high-level regularities — either of things like revolutions or things like metals. It is an approach that encourages a more specific focus on this case or that small handful of cases, rather than a focus on finding general causal properties of high-level entities. And it is more open to and tolerant of the possibility of a degree of contingency and variation within a domain of phenomena. To postulate that civil disorders are affected by a group of well-understood social mechanisms does not imply that there are strong regularities across all civil disorders, or that these mechanisms work in exactly the same way in all circumstances. So the features of contingency and context dependence play an organic role within CM methodology and fit badly in paired-comparisons research and statistical modeling approaches.

So it seems that the ontology of causal-mechanisms theory does in fact provide a set of heuristics and procedures for undertaking social research. CM does have implications for social-science methodology.

What is methodology?

 

As social science researchers, we would all like to have an excellent methodology for carrying out the tasks we confront in our scientific work. But what precisely are we looking for when we aspire to this goal? What is a methodology, and what is it intended to allow us to do?

A methodology is a set of ideas or guidelines about how to proceed in gathering and validating knowledge of a subject matter. Different areas of science have developed very different bodies of methodology on the basis of which to conduct their research. We might say that a methodology provides a guide for carrying out some or all of the following activities:

  • probing the empirical details of a domain of phenomena
  • discovering explanations of surprising outcomes or patterns
  • identifying entities or forces
  • establishing patterns
  • providing predictions
  • separating noise from signal
  • using empirical reasoning to assess hypotheses and assertions
Here is what Andrew Abbott has to say about methods in Methods of Discovery: Heuristics for the Social Sciences:

Social scientists have a number of methods, stylized ways of conducting their research that comprise routine and accepted procedures for doing the rigorous side of science. Each method is loosely attached to a community of social scientists  for whom it is the right way to do things. But no method is the exclusive property of any one of the social sciences, nor is any social science, with the possible exception of anthropology, principally organized around the use of one particular method. (13)

So a method or a methodology is a set of recommendations for how to proceed in doing scientific research within a certain domain. Sometimes in the history of philosophy there has been a hope that science could proceed on the basis of a pure inductive logic: collect the data, analyze the data, sift through the findings, report the strongest regularities found in the data set. But scientific inquiry requires more than this; it requires discovery and imagination.

What form might a methodology take? The simplest idea is that a methodology is a recipe for arriving at justified scientific statements with respect to a domain of empirical phenomena. A recipe is a set of instructions for treating a number of ingredients in a sequential way and producing a specific kind of output — a soufflé or a bowl of pad thai. If you follow the recipe, you are almost certain to arrive at the soufflé. But it is clear that scientific methodology cannot be as prescriptive as a recipe. There is no set of rules that are certain or likely to lead to the discovery of compelling hypotheses and explanations.So if a scientific methodology isn’t a set of recipes, then what is it? Here is another possibility: a methodology consists of a set of heuristics that serve to guide the activities, data collection, and hypothesis formation of the scientist. A heuristic is also a set of rules; but it is weaker than a recipe in that there is no guarantee of success. Here is a heuristic for consumers: “If you are selecting a used car to purchase, pay attention to rust spots.” This is a good guide to action, not because rust spots are the most important part of a car’s quality, but because they may serve as a proxy for the attentiveness to maintenance of the previous owner — and therefore be an indication of hidden defects.

Andrew Abbott mentions several key topics for specification through methodology — “how to propose a question, how to design a study, how to draw inferences, how to acquire and analyze data” (13), and he shows that we can classify methods by placing them into the types of question they answer.

types of data gathering
           data analysis
        posing a question
  • history
  • direct interpretation
  • case-study analysis
  • ethnography
  • quantitative analysis
  • small-N comparison
  • surveys
  • formal modeling
  • large-N analysis
  • record-based analysis
Abbott suggests that these varieties can be combined into five basic approaches:
  • ethnography
  • historical narration
  • standard causal analysis
  • small-N comparison
  • formalization
And he arranges them in a three-dimensional space, with each dimension increasing from very particular knowledge at the origin to more abstract knowledge further out the axis. (Commonsense understanding of the facts lies at the origin of the mapping.) The three axes are formal modeling (syntactic program), pattern finding (semantic program), and cause finding (pragmatic program) (28).
Abbott is a sociologist whose empirical and theoretical work is genuinely original and important, and we can learn a lot from his practice as a working researcher. His meta-analysis of methodology, on the other hand, seems fairly distant from his own practice. And I’m not sure that the analysis of methodology represented here provides a lot of insight into the research strategies of other talented social scientists (e.g. Tilly, Steinmetz, Perrow, Fligstein). This perhaps illustrates a common occurrence in the history of science: researchers are not always the best interpreters of their own practice.

It is also interesting to observe that the discovery of causal mechanisms has no explicit mention in this scheme. Abbott never refers to causal mechanisms in the book, and none of the methods he highlights allow us to see what he might think about the mechanisms approach. It would appear that mechanisms theory would reflect the pragmatic program (searching for causal relationships) and the semantic program (discovering patterns in the observable data).

My own map of the varieties of the methods of the social sciences suggests a different scheme altogether. This is represented in the figure at the top of the post.

Skocpol on the 1979 revolution in Iran

 

An earlier post reviewed Theda Skocpol’s effort in States and Social Revolutions: A Comparative Analysis of France, Russia and China to provide a comparative, structural account of the occurrence of social revolutions. There I suggested that the account is too deterministic and too abstract. It gives the impression, perhaps undeserved, that there are only a small number of pathways through which social revolutions can take place, and only a small number of causal factors that serve to bring them about. The impression emerges that Skocpol has offered a set of templates into which we should expect other social revolutions to fit.

One of the benefits of re-reading a book that is now 35 years old, however, is that history presents new cases that are appropriately considered by the theory. One such case is the Iranian Revolution, which unfolded in 1979. And, as Skocpol indicates forthrightly, the Iranian Revolution does not fit the model that she puts forward in States and Social Revolutions very closely. Skocpol considered the complexities and challenges which the Iranian Revolution posed to her theory in an article which appeared in 1981, before the dust had fully settled in Tehran. The article is included in her collection, Social Revolutions in the Modern World. Here is the challenge that the Iranian Revolution created for Skocpol’s causal theory of social revolutions:

A few of us have also been inspired to probe the Iranian sociopolitical realities behind these events. For me, such probing was irresistible – above all because the Iranian revolution struck me in some ways is quite anomalous. This revolution surely qualifies as a sort of “social revolution.” Yet its unfolding – especially in the events leading to the Shah’s overthrow – challenged expectations about revolutionary causation that I developed through comparative-historical research on the French, Russian, and Chinese Revolutions. (240)

Skocpol finds that the large features of the Iranian Revolution did indeed fit the terms of her definition of a social revolution, but that the causal background and components of this historical event did not fit her expectations.

The initial stages of the Iranian revolution obviously challenged my previously worked-out notions about the causes of social revolutions. Three apparent difficulties come immediately to mind. First, the Iranian Revolution does seem as if it might have been simply a product of excessively rapid modernization…. Second, in a striking departure from the regularities of revolutionary history, the Shah’s army and police – modern coercive organizations over 300,000 men strong – were rendered ineffective in the revolutionary process between 1977 and early 1979 without the occurrence of a military defeat in foreign war and without pressures from abroad…. Third, if ever there has been a revolution deliberately “made” by a mass–based social movement aiming to overthrow the old order, the Iranian revolution against the Shah surely is it. (241-242)

So the Iranian Revolution does not fit the mold. Does this imply that the interpretation of social revolution offered in States and Social Revolutions is refuted? Or does it imply instead that there are more narrow limits on the strength of the generalizations offered in that book than appear on first reading? In fact, it seems that the latter is the case:

Fortunately, in States and Social Revolutions I explicitly denied the possibility of fruitfulness of a general causal theory of revolutions that would apply across all times and places…. The Iranian Revolution can be interpreted in terms analytically consistent with the explanatory principles I used in States and Social Revolutions – this is what I shall briefly try to show. However, this remarkable revolution also forces me to deepen my understanding of the possible role of idea systems and cultural understandings in the shaping of political action – in ways that I show indicate recurrently at appropriate points in this article. (243)

One important difference between the revolutions studied by Skocpol’s earlier work and the Iranian revolution is the urban base of the latter revolution. “Opposition to the Shah was centered in urban communal enclaves where autonomous and solitary collective resistance was possible” (245). “In the mass movements against the Shah during 1977 and 1978, the traditional urban communities of Iran were to play an indispensable role in mobilizing in sustaining the core of popular resistance” (246). This is a difference in the social composition of the social revolution; peasant unrest and uprisings were crucial in the cases of France, Russia, and China; but not in the case of Iran.

Another key difference in the circumstances of the Iranian Revolution was the role played by Shi’a Islam. This is what Skocpol was referring to when she indicated the important role of idea systems and cultural understandings.  “In sum, Shi’a Islam was both organizationally and culturally crucial to the making of the Iranian revolution against the Shah” (249). So ideas and values played a role in mobilizing and sustaining revolutionary actions by the population that does not have a valid counterpart in China, France, or Russia. This is a more serious divergence from the reasoning of SSR, because it introduces an entirely new causal factor — “idea systems”. In SSR the motivations that are ascribed to activists and followers are interest-based; whereas her treatment of Shi’a Islam and the Iranian Revolution forces a broadening of the theory of the actor to incorporate the workings of non-material values and commitments.

How does Skocpol think that ideas and culture function in the context of social unrest? “In and of themselves, the culture and networks of communication do not dictate mass revolutionary action. But if a historical conjuncture arises in which a vulnerable state faces oppositionally inclined social groups possessing solidarity, autonomy, and independent economic resources, then the sorts of moral symbols and forms of social communication offered by Shi’a Islam in Iran can sustain the self-conscious making of a revolution” (250). So the value system of Shi’a Islam, and the passions and commitments that it engendered, played a key causal role in the success of the revolutionary actors in Tehran, in the view that Skocpol offers in the current article.

So the social actors can be different and the causal factors involved can be different. What about the outcomes of the processes of social revolution? Can we at least keep the idea that a social revolution, once underway, has a certain logic of development that leads to certain kinds of outcomes? Here again, Skocpol is clear in saying that we cannot.

On the contrary, Skocpol brings the fact of contingency into her account here in a way that is not apparent in the earlier book. In her treatment of the Iranian Revolution she is brought to acknowledge and recognize the deep contingency that exists within a social revolution.

Of course, events in Iran may outrun that Shi’a revolutionary leadership. The clerics may lose their political unity and the army or a secular political party may step in. Or regional revolts and foreign subversion may lead to the dismemberment of the country. (254)

Or in other words: there is no necessary sequence of events in this social revolution, or any other.So what remains? How does comparative study of social revolutions contribute to explanation? Rather than hoping for a causal diagram that identifies factors, forces, and outcomes, it seems unavoidable that we need to look for more limited findings. And this pushes us in the direction of the disaggregated approach that McAdam, Tarrow, and Tilly take in their own subsequent treatments of social contention in Dynamics of Contention.

According to that approach, there are some common causal processes — we would now call them “mechanisms of contention” — that give some insight into the critical events that transpire within a given historical sequence. But these common mechanisms do not have primacy over the myriad other factors in play — the behavior of the military, the emergence of a secular political party, the sudden appearance of a charismatic movie actor turned political leader, the eruption of international conflict (like the war that Iran was forced to wage with Iraq), and countless other possible causal branches. And this means something very deep for the project of comparative theorizing about social revolution, or any other large-scale social change: we should regard these processes as importantly sui generis rather than general, and we should look for the sub-processes and mechanisms rather than high-level macro-causal relationships.

%d bloggers like this: