Downward causation

I’ve argued for the idea that social phenomena are generated by the actions, thoughts, and mental frameworks of myriad actors (link). This expresses the idea of ontological individualism. But I also believe that social arrangements — structures, ideologies, institutions — have genuine effects on the actions of individual actors and populations of actors and on intermediate-level social structures. There is real downward and lateral causation in the social world. Are these two views compatible?

I believe they are compatible.

The negative view holds that what appears to be downward causation is really just the workings of the lower-level components through their aggregation dynamics — the lower struts of Coleman’s boat (link). So when we say “the ideology of nationalism causes the rise of ultraconservative political leaders”, this is just a shorthand for “many voters share the values of nationalism and elect candidates who propose radical solutions to issues like immigration.” This seems to be the view of analytical-sociology purists.

But consider the alternative view — that higher level entities sometimes come to possess stable causal powers that influence the behavior and even the constitution of the entities of which they are composed. This seems like an implausible idea in the natural sciences — it is hard to imagine a world in which electrons have different physical properties as an effect of the lattice arrangement of atoms in a metal. But human actors are different from electrons and atoms, in that their behavior and constitution are in fact plastic to an important degree. In one social environment actors are disposed to be highly attentive to costs and benefits; in another social environment they are more amenable to conformance to locally expressed norms. And we can say quite a bit about the mechanisms of social psychology through which the cognitive and normative frameworks of actors are influenced by features of their social environments. This has an important implication: features of the higher-level social reality can change the dispositions and workings of the lower-level actors. And these changes may in turn lead to the emergence of new higher-level factors (new institutions, new normative systems, new social practices of solidarity, …). So enduring social arrangements can cause changes in the dynamic properties of the actors who live within them.

Could we even say, more radically and counter-intuitively, that a normative structure like extremist populism “generates” behavior at the individual level? So rather than holding that individual actions generate higher-level structures, might we hold that higher-level normative structures generate patterns of behavior? For example, we might say that the normative strictures of patriarchy generate patterns of domination and deference among men and women at the individual level; or the normative strictures of Jim-Crow race relations generate individual-level patterns of subordination and domination among white and black individuals. There is a sense in which this statement about the direction of generation is obviously true; broadly shared knowledge frameworks or normative commitments “generate” typical forms of behavior in stylized circumstances of choice.

Does this way of thinking about the process of “generation” suggest that we need to rethink the directionality implied by the micro-macro distinction? Might we say that normative systems and social structures are as fundamental as patterns of individual behavior?

Consider the social reality depicted in the photograph above. Here we see coordinated action of a number of soldiers climbing out of a trench in World War I to cross the killing field of no mans land. The dozen or so soldiers depicted here are part of a vast army at war (3.8 million by 1918), deployed over a front extending hundreds of miles. The majority of the soldiers depicted here are about to receive grievous or mortal wounds. And yet they go over the trench. What can we say about the cause of this collective action at a specific moment in time? First, an order was conveyed through a communications system extending from commander to sergeant to enlisted man: “attack at 7:00 am”. Second, the industrial wealth of Great Britain permitted the state the ability to equip and field a vast infantry army. Third, a system of international competition broke down into violent confrontation and war, leading numerous participant nations to organize and fund armies at war to defeat their enemies. Fourth, the morale of the troops was maintained at a sufficiently high level to avoid mass desertion and refusal to fight.  Fifth, an infantry training regime existed which gave ordinary farmhands, workers, accountants, and lords the habits and skills of infantry soldiers. All of these factors are part of the causal background of this simple episode in World War I; and most of these factors exist at a meso- or macro-level of social organization. Clearly this particular group of social actors was influenced by higher-level social factors. But equally clearly, the mechanisms through which these higher-level social factors work are straightforward to identify through reference to systems of individual actors.

Think for a minute about materials science. The hardness of titanium causes the nail to scratch the glass. It is true that material properties like hardness depend upon their microstructures. Nonetheless we are perfectly comfortable in attributing real causal powers to titanium at the level of a macro-material. And this attribution is not merely a way of summarizing a long story about the micro-structure of metallic titanium.

I’ve generally tried to think about these kinds of causal stories in terms of the idea of microfoundations. The hardness of titanium derives from its microfoundations at the level of atomic and subatomic causation. And the causal powers of patriarchy derive from the fact that the normative principles of partriarchy are embedded in the minds and behavior of many individuals, who become exemplars, enforcers, and encouragers of compliant behavior. The processes through which individuals acquire normative principles and the processes through which they behaviorally reflect these principles constitute the microfoundations of the meso- and macro-power of patriarchy.

So the question of whether there is downward causation seems almost too easy. Of course there is downward causation in the social world. Individuals are influenced in their choices and behavior by structural and normative factors beyond their control. And more fundamentally, individuals are changed in their fundamental dispositions to behavior through their immersion in social arrangements.

Shakespeare on tyranny

Stephen Greenblatt is a literary critic and historian whose insights into philosophy and the contemporary world are genuinely and consistently profound. His most recent book returns to his primary expertise, the corpus of Shakespeare’s plays. But it is — by intention or otherwise — an  important reflection on the presidency of Donald Trump as well. The book is Tyrant: Shakespeare on Politics, and it traces in fascinating detail the evolution and fates of tyrants through Shakespeare’s plays. Richard III gets a great deal of attention, as do Lear and Macbeth. Greenblatt makes it clear that Shakespeare was interested both in the institutions of governance within which tyrants seized power, and the psychology of the tyrant. The parallels with the behavior and psychology of the current US President are striking.
 
Here is how Greenblatt frames his book.

“A king rules over willing subjects,” wrote the influential sixteenth-century Scottish scholar George Buchanan, “a tyrant over unwilling.” The institutions of a free society are designed to ward off those who would govern, as Buchanan put it, “not for their country but for themselves, who take account not of the public interest but of their own pleasure.” Under what circumstances, Shakespeare asked himself, do such cherished institutions, seemingly deep-rooted and impregnable, suddenly prove fragile? Why do large numbers of people knowingly accept being lied to? How does a figure like Richard III or Macbeth ascend to the throne? (1)

So who is the tyrant? What is his typical psychology?

Shakespeare’s Richard III brilliantly develops the personality features of the aspiring tyrant already sketched in the Henry VI trilogy: the limitless self-regard, the lawbreaking, the pleasure in inflicting pain, the compulsive desire to dominate. He is pathologically narcissistic and supremely arrogant. He has a grotesque sense of entitlement, never doubting that he can do whatever he chooses. He loves to bark orders and to watch underlings scurry to carry them out. He expects absolute loyalty, but he is incapable of gratitude. The feelings of others mean nothing to him. He has no natural grace, no sense of shared humanity, no decency. He is not merely indifferent to the law; he hates it and takes pleasure in breaking it. He hates it because it gets in his way and because it stands for a notion of the public good that he holds in contempt. He divides the world into winners and losers. The winners arouse his regard insofar as he can use them for his own ends; the losers arouse only his scorn. The public good is something only losers like to talk about. What he likes to talk about is winning. (53)

One of Richard’s uncanny skills—and, in Shakespeare’s view, one of the tyrant’s most characteristic qualities—is the ability to force his way into the minds of those around him, whether they wish him there or not. (64)

Greenblatt has a lot to say about the enablers of the tyrant — those who facilitate and those who silently consent.

Another group is composed of those who do not quite forget that Richard is a miserable piece of work but who nonetheless trust that everything will continue in a normal way. They persuade themselves that there will always be enough adults in the room, as it were, to ensure that promises will be kept, alliances honored, and core institutions respected. Richard is so obviously and grotesquely unqualified for the supreme position of power that they dismiss him from their minds. Their focus is always on someone else, until it is too late. They fail to realize quickly enough that what seemed impossible is actually happening. They have relied on a structure that proves unexpectedly fragile. (67)

One of the topics that appears in Shakespeare’s corpus is a class-based populism from the under-classes. Consider Jack Cade, the lying and violent foil to The Duke of York.

Cade himself, for all we know, may think that what he is so obviously making up as he goes along will actually come to pass. Drawing on an indifference to the truth, shamelessness, and hyperinflated self-confidence, the loudmouthed demagogue is entering a fantasyland—“ When I am king, as king I will be”—and he invites his listeners to enter the same magical space with him. In that space, two and two do not have to equal four, and the most recent assertion need not remember the contradictory assertion that was made a few seconds earlier. (37)

And what about the fascination tyrants have with secret alliances with hostile foreign powers?

Third, the political party determined to seize power at any cost makes secret contact with the country’s traditional enemy. England’s enmity with the nation across the Channel—constantly fanned by all the overheated patriotic talk of recovering its territories there, and fueled by all the treasure and blood spilled in the attempt to do so—suddenly vanishes. The Yorkists—who, in the person of Cade, had pretended to consider it an act of treason even to speak French—enter into a set of secret negotiations with France. Nominally, the negotiations aim to end hostilities between the two countries by arranging a dynastic marriage, but they actually spring, as Queen Margaret cynically observes, “from deceit, bred by necessity” (3 Henry VI 3.3.68).

How does the tyrant rule? In a word, badly.

The tyrant’s triumph is based on lies and fraudulent promises braided around the violent elimination of rivals. The cunning strategy that brings him to the throne hardly constitutes a vision for the realm; nor has he assembled counselors who can help him formulate one. He can count—for the moment, at least—on the acquiescence of such suggestible officials as the London mayor and frightened clerks like the scribe. But the new ruler possesses neither administrative ability nor diplomatic skill, and no one in his entourage can supply what he manifestly lacks. His own mother despises him. His wife, Anne, fears and hates him. (84)

Several things seem apparent, both from Greenblatt’s reading of Shakespeare and from the recent American experience. One is that freedom and the rule of law are inextricably entangled. It is not an exaggeration to say that freedom simply is the situation of living in a society in which the rule of law is respected (and laws establish individual rights and impersonal procedures). When strongmen are able to use the organs of the state or their private henchmen to enact their personal will, the freedom and liberties of the whole of society are compromised.

Second, the rule of law is a normative commitment; but it is also an institutional reality. Institutions like the Constitution, the division of powers, the independence of the judiciary, and the codification of government ethics are preventive checks against arbitrary power by individuals with power. But as Greenblatt’s examples show, the critical positions within the institutions of law and government are occupied by ordinary men and women. And when they are venal, timid, and bent to the will of the sovereign, they present no barrier against tyranny. This is why fidelity to the rule of law and the independence of the justice system is the most fundamental and irreplaceable ethical commitment we must demand of officials. Conversely, when an elected official demonstrates lack of commitment to the principles, we must be very anxious for the fate of our democracy.

Greenblatt’s book is fascinating for the historical context it provides for Shakespeare’s plays. But it is even more interesting for the critical light it sheds on our current politics. And it makes clear that the moral choices posed by politicians determined to undermine the institutions of democracy are perennial, whether in Shakespeare’s time or our own.

Social generatively and complexity

The idea of generativity in the realm of the social world expresses the notion that social phenomena are generated by the actions and thoughts of the individuals who constitute them, and nothing else (linklink). More specifically, the principle of generativity postulates that the properties and dynamic characteristics of social entities like structures, ideologies, knowledge systems, institutions, and economic systems are produced by the actions, thoughts, and dispositions of the set of individuals who make them up. There is no other kind of influence that contributes to the causal and dynamic properties of social entities. Begin with a population of individuals with such-and-so mental and behavioral characteristics; allow them to interact with each other over time; and the structures we observe emerge as a determinate consequence of these interactions.

This view of the social world lends great ontological support to the methods associated with agent-based models (link). Here is how Joshua Epstein puts the idea in Generative Social Science: Studies in Agent-Based Computational Modeling):

Agent-based models provide computational demonstrations that a given microspecification is in fact sufficient to generate a macrostructure of interest…. Rather, the generativist wants an account of the configuration’s attainment by a decentralized system of heterogeneous autonomous agents. Thus, the motto of generative social science, if you will, is: If you didn’t grow it, you didn’t explain its emergence. (42)

Consider an analogy with cooking. The properties of the cake are generated by the properties of the ingredients, their chemical properties, and the sequence of steps that are applied to the assemblage of the mixture from the mixing bowl to the oven to the cooling board. The final characteristics of the cake are simply the consequence of the chemistry of the ingredients and the series of physical influences that were applied in a given sequence.

Now consider the concept of a complex system. A complex system is one in which there is a multiplicity of causal factors contributing to the dynamics of the system, in which there are causal interactions among the underlying causal factors, and in which causal interactions are often non-linear. Non-linearity is important here, because it implies that a small change in one or more factors may lead to very large changes in the outcome. We like to think of causal systems as consisting of causal factors whose effects are independent of each other and whose influence is linear and additive.

A gardener is justified in thinking of growing tomatoes in this way: a little more fertilizer, a little more water, and a little more sunlight each lead to a little more tomato growth. But imagine a garden in which the effect of fertilizer on tomato growth is dependent on the recent gradient of water provision, and the effects of both positive influencers depends substantially on the recent amount of sunlight available. Under these circumstances it is difficult to predict the aggregate size of the tomato given information about the quantities of the inputs.

One of the key insights of complexity science is that generativity is fully compatible with a wicked level of complexity. The tomato’s size is generated by its history of growth, determined by the sequence of inputs over time. But for the reason just mentioned, the complexity of interactions between water, sunlight, and fertilizer in their effects on growth mean that the overall dynamics of tomato growth are difficult to reconstruct.

Now consider the idea of strong emergence — the idea that some aggregates possess properties that cannot in principle be explained by reference to the causal properties of the constituents of the aggregate. This means that the properties of the aggregate are not generated by the workings of the constituents; otherwise we would be able in principle to explain the properties of the aggregate by demonstrating how they derive from the (complex) pathways leading from the constituents to the aggregate. This version of the absolute autonomy of some higher-level properties is inherently mysterious. It implies that the aggregate does not supervene upon the properties of the constituents; there could be different aggregate properties with identical constituent properties. And this seems ontological untenable.

The idea of ontological individualism captures this intuition in the setting of social phenomena: social entities are ultimately composed of and constituted by the properties of the individuals who make them up, and nothing else. This does not imply methodological individualism; for reasons of complexity or computational limitations it may be practically impossible to reconstruct the pathways through which the social entity is generated out of the properties of individuals. But ontological individualism places an ontological constraint on the way that we conceptualize the social world. And it gives a concrete meaning to the idea of the microfoundations for a social entity. The microfoundations of a social entity are the pathways and mechanisms, known or unknown, through which the social entity is generated by the actions and intentionality of the individuals who constitute it.

What the boss wants to hear …

According to David Halberstam in his outstanding history of the war in Vietnam, The Best and the Brightest, a prime cause of disastrous decision-making by Presidents Kennedy and Johnson was an institutional imperative in the Defense Department to come up with a set of facts that conformed to what the President wanted to hear. Robert McNamara and McGeorge Bundy were among the highest-level miscreants in Halberstam’s account; they were determined to craft an assessment of the situation on the ground in Vietnam that conformed best with their strategic advice to the President.

Ironically, a very similar dynamic led to one of modern China’s greatest disasters, the Great Leap Forward famine in 1959. The Great Helmsman was certain that collective agriculture would be vastly more productive than private agriculture; and following the collectivization of agriculture, party officials in many provinces obliged this assumption by reporting inflated grain statistics throughout 1958 and 1959. The result was a famine that led to at least twenty million excess deaths during a two-year period as the central state shifted resources away from agriculture (Frank DikötterMao’s Great Famine: The History of China’s Most Devastating Catastrophe, 1958-62).

More mundane examples are available as well. When information about possible sexual harassment in a given department is suppressed because “it won’t look good for the organization” and “the boss will be unhappy”, the organization is on a collision course with serious problems. When concerns about product safety or reliability are suppressed within the organization for similar reasons, the results can be equally damaging, to consumers and to the corporation itself. General Motors, Volkswagen, and Michigan State University all seem to have suffered from these deficiencies of organizational behavior. This is a serious cause of organizational mistakes and failures. It is impossible to make wise decisions — individual or collective — without accurate and truthful information from the field. And yet the knowledge of higher-level executives depends upon the truthful and full reporting of subordinates, who sometimes have career incentives that work against honesty.

So how can this unhappy situation be avoided? Part of the answer has to do with the behavior of the leaders themselves. It is important for leaders to explicitly and implicitly invite the truth — whether it is good news or bad news. Subordinates must be encouraged to be forthcoming and truthful; and bearers of bad news must not be subject to retaliation. Boards of directors, both private and public, need to make clear their own expectations on this score as well: that they expect leading executives to invite and welcome truthful reporting, and that they expect individuals throughout the organization to provide truthful reporting. A culture of honesty and transparency is a powerful antidote to the disease of fabrications to please the boss.

Anonymous hotlines and formal protection of whistle-blowers are other institutional arrangements that lead to greater honesty and transparency within an organization. These avenues have the advantage of being largely outside the control of the upper executives, and therefore can serve as a somewhat independent check on dishonest reporting.

A reliable practice of accountability is also a deterrent to dishonest or partial reporting within an organization. The truth eventually comes out — whether about sexual harassment, about hidden defects in a product, or about workplace safety failures. When boards of directors and organizational policies make it clear that there will be negative consequences for dishonest behavior, this gives an ongoing incentive of prudence for individuals to honor their duties of honesty within the organization.

This topic falls within the broader question of how individual behavior throughout an organization has the potential for giving rise to important failures that harm the public and harm the organization itself. 


Regulatory failure

When we think of the issues of health and safety that exist in a modern complex economy, it is impossible to imagine that these social goods will be produced in sufficient quantity and quality by market forces alone. Safety and health hazards are typically regarded as “externalities” by private companies — if they can be “dumped” on the public without cost, this is good for the profitability of the company. And state regulation is the appropriate remedy for this tendency of a market-based economy to chronically produce hazards and harms, whether in the form of environmental pollution, unsafe foods and drugs, or unsafe industrial processes. David Moss and John Cisternino’s New Perspectives on Regulation provides some genuinely important perspectives on the role and effectiveness of government regulation in an epoch which has been shaped by virulent efforts to reduce or eliminate regulations on private activity. This volume is a report from the Tobin Project.

It is poignant to read the optimism that the editors and contributors have — in 2009 — about the resurgence of support for government regulation. The financial crisis of 2008 had stimulated a vigorous round of regulation of financial institutions, and most of the contributors took this as a harbinger of a fresh public support for regulation more generally. Of course events have shown this confidence to be sadly mistaken; the dismantling of Federal regulatory regimes by the Trump administration threatens to take the country back to the period described by Upton Sinclair in the early part of the prior century. But what this demonstrates is the great importance of the Tobin Project. We need to build a public understanding and consensus around the unavoidable necessity of effective and pervasive regulatory regimes in environment, health, product safety, and industrial safety.

Here is how Mitchell Weiss, Executive Director of the Tobin Project, describes the project culminating in this volume:

To this end, in the fall of 2008 the Tobin Project approached leading scholars in the social sciences with an unusual request: we asked them to think about the topic of economic regulation and share key insights from their fields in a manner that would be accessible to both policymakers and the public. Because we were concerned that a conventional literature survey might obscure as much as it revealed, we asked instead that the writers provide a broad sketch of the most promising research in their fields pertaining to regulation; that they identify guiding principles for policymakers wherever possible; that they animate these principles with concrete policy proposals; and, in general, that they keep academic language and footnotes to a minimum. (5)

The lead essay is provided by Joseph Stiglitz, who looks more closely than previous decades of economists had done at the real consequences of market failure. Stiglitz puts the point about market failure very crisply:

Only under certain ideal circumstances may individuals, acting on their own, obtain “pareto efficient” outcomes, that is, situations in which no one can be made better off without making another worse off. These individuals involved must be rational and well informed, and must operate in competitive market- places that encompass a full range of insurance and credit markets. In the absence of these ideal circumstances, there exist government interventions that can potentially increase societal efficiency and/or equity. (11)

And regulation is unpopular — with the businesses, landowners, and other powerful agents whose actions are constrained.

By its nature, a regulation restricts an individual or firm from doing what it otherwise would have done. Those whose behavior is so restricted may complain about, say, their loss of profits and potential adverse effects on innovation. But the purpose of government intervention is to address potential consequences that go beyond the parties directly involved, in situations in which private profit is not a good measure of social impact. Appropriate regulation may even advance welfare-enhancing innovations. (13)

Stiglitz pays attention to the pervasive problem of “regulatory capture”:

The current system has made regulatory capture too easy. The voices of those who have benefited from lax regulation are strong; the perspectives of the investment community have been well represented. Among those whose perspectives need to be better represented are the laborers whose jobs would be lost by macro-mismanagement, and the pension holders whose pension funds would be eviscerated by excessive risk taking.

One of the arguments for a financial products safety commission, which would assess the efficacy and risks of new products and ascertain appropriate usage, is that it would have a clear mandate, and be staffed by people whose only concern would be protecting the safety and efficacy of the products being sold. It would be focused on the interests of the ordinary consumer and investors, not the interests of the financial institutions selling the products. (18)

It is very interesting to read Stiglitz’s essay with attention to the economic focus he offers. His examples all come from the financial industry — the risk at hand in 2008-2009. But the arguments apply equally profoundly to manufacturing, the pharmaceutical and food industries, energy industries, farming and ranching, and the for-profit education sector. At the same time the institutional details are different, and an essay on this subject with a focus on nuclear or chemical plants would probably identify a different set of institutional barriers to effective regulation.

Also particularly interesting is the contribution by Michael Barr, Eldar Shafir, and Sendhil Mullainathan on how behavioral perspectives on “rational action” can lead to more effective regulatory regimes. This essay pays close attention to the findings of experimental economics and behavioral economics, and the deviations from “pure economic rationality” that are pervasive in ordinary economic decision making. These features of decision-making are likely to be relevant to the effectiveness of a regulatory regime as well. Further, it suggests important areas of consumer behavior that are particularly subject to exploitative practices by financial companies — creating a new need for regulation of these kinds of practices. Here is how they summarize their approach:

We propose a different approach to regulation. Whereas the classical perspective assumes that people generally know what is important and knowable, plan with insight and patience, and carry out their plans with wisdom and self-control, the central gist of the behavioral perspective is that people often fail to know and understand things that matter; that they misperceive, misallocate, and fail to carry out their intended plans; and that the context in which people function has great impact on their behavior, and, consequently, merits careful attention and constructive work. In our framework, successful regulation requires integrating this richer view of human behavior with our understanding of markets. Firms will operate on the contour de ned by this psychology and will respond strategically to regulations. As we describe above, because firms have a great deal of latitude in issue framing, product design, and so on, they have the capacity to a affect behavior and circumvent or pervert regulatory constraints. Ironically, firms’ capacity to do so is enhanced by their interaction with “behavioral” consumers (as opposed to the hypothetically rational actors of neoclassical economic theory), since so many of the things a regulator would find very hard to control (for example, frames, design, complexity, etc.) can greatly influence consumers’ behavior. e challenge of behaviorally informed regulation, therefore, is to be well designed and insightful both about human behavior and about the behaviors that firms are likely to exhibit in response to both consumer behavior and regulation. (55)

The contributions to this volume are very suggestive with regard to the issues of product safety, manufacturing safety, food and drug safety, and the like which constitute the larger core of the need for regulatory regimes. And the challenges faced in the areas of financial regulation discussed here are likely to be found to be illuminating in other sectors as well.

 

Empowering the safety officer?

How can industries involving processes that create large risks of harm for individuals or populations be modified so they are more capable of detecting and eliminating the precursors of harmful accidents? How can nuclear accidents, aviation crashes, chemical plant explosions, and medical errors be reduced, given that each of these activities involves large bureaucratic organizations conducting complex operations and with substantial inter-system linkages? How can organizations be reformed to enhance safety and to minimize the likelihood of harmful accidents?

One of the lessons learned from the Challenger space shuttle disaster is the importance of a strongly empowered safety officer in organizations that deal in high-risk activities. This means the creation of a position dedicated to ensuring safe operations that falls outside the normal chain of command. The idea is that the normal decision-making hierarchy of a large organization has a built-in tendency to maintain production schedules and avoid costly delays. In other words, there is a built-in incentive to treat safety issues with lower priority than most people would expect.

If there had been an empowered safety officer in the launch hierarchy for the Challenger launch in 1986, there is a good chance this officer would have listened more carefully to the Morton-Thiokol engineering team’s concerns about low temperature damage to O-rings and would have ordered a halt to the launch sequence until temperatures in Florida raised to the critical value. The Rogers Commission faulted the decision-making process leading to the launch decision in its final report on the accident (The Report of the Presidential Commission on the Space Shuttle Challenger Accident – The Tragedy of Mission 51-L in 1986 – Volume OneVolume TwoVolume Three).

This approach is productive because empowering a safety officer creates a different set of interests in the management of a risky process. The safety officer’s interest is in safety, whereas other decision makers are concerned about revenues and costs, public relations, reputation, and other instrumental goods. So a dedicated safety officer is empowered to raise safety concerns that other officers might be hesitant to raise. Ordinary bureaucratic incentives may lead to underestimating risks or concealing faults; so lowering the accident rate requires giving some individuals the incentive and power to act effectively to reduce risks.

Similar findings have emerged in the study of medical and hospital errors. It has been recognized that high-risk activities are made less risky by empowering all members of the team to call a halt in an activity when they perceive a safety issue. When all members of the surgical team are empowered to halt a procedure when they note an apparent error, serious operating-room errors are reduced. (Here is a report from the American College of Obstetricians and Gynecologists on surgical patient safety; link. And here is a 1999 National Academy report on medical error; link.)

The effectiveness of a team-based approach to safety depends on one central fact. There is a high level of expertise embodied in the staff operating a surgical suite, an engineering laboratory, or a drug manufacturing facility. By empowering these individuals to stop a procedure when they judge there is an unrecognized error in play, this greatly extend the amount of embodied knowledge involved in a process. The surgeon, the commanding officer, or the lab director is no longer the sole expert whose judgments count.

But it also seems clear that these innovations don’t work equally well in all circumstances. Take nuclear power plant operations. In Atomic Accidents: A History of Nuclear Meltdowns and Disasters: From the Ozark Mountains to Fukushima James Mahaffey documents multiple examples of nuclear accidents that resulted from the efforts of mid-level workers to address an emerging problem in an improvised way. In the case of nuclear power plant safety, it appears that the best prescription for safety is to insist on rigid adherence to pre-established protocols. In this case the function of a safety officer is to monitor operations to ensure protocol conformance — not to exercise independent judgment about the best way to respond to an unfavorable reactor event.

It is in fact an interesting exercise to try to identify the kinds of operations in which these innovations are likely to be effective.

Here is a fascinating interview in Slate with Jim Bagian, a former astronaut, one-time director of the Veteran Administration’s National Center for Patient Safety, and distinguished safety expert; link. Bagian emphasizes the importance of taking a system-based approach to safety. Rather than focusing on finding blame for specific individuals whose actions led to an accident, Bagian emphasizes the importance of tracing back to the institutional, organizational, or logistic background of the accident. What can be changed in the process — of delivering medications to patients, of fueling a rocket, or of moving nuclear solutions around in a laboratory — that make the likelihood of an accident substantially lower?

The safety principles involved here seem fairly simple: cultivate a culture in which errors and near-misses are reported and investigated without blame; empower individuals within risky processes to halt the process if their expertise and experience indicates the possibility of a significant risky error; create individuals within organizations whose interests are defined in terms of the identification and resolution of unsafe practices or conditions; and share information about safety within the industry and with the public.

Machine learning

The Center for the Study of Complex Systems at the University of Michigan hosted an intensive day-long training on some of the basics of machine learning for graduate students and interested faculty and staff. Jake Hofman, a Microsoft researcher who also teaches this subject at Columbia University, was the instructor, and the session was both rigorous and accessible (link). Participants were asked to load a copy of R, a software package designed for the computations involved in machine learning and applied statistics, and numerous data sets were used as examples throughout the day. (Here is a brief description of R; link.) Thanks, Jake, for an exceptionally stimulating workshop.

So what is machine learning? Most crudely, it is a handful of methods through which researchers can sift through a large collection of events or objects, each of which has a very large number of properties, in order to arrive at a predictive sorting of the events or objects into a set of categories. The objects may be email texts or hand-printed numerals (the examples offered in the workshop), the properties may be the presence/absence of a long list of words or the presence of a mark in a bitmap grid, and the categories may be “spam/not spam” or the numerals between 0 and 9. But equally, the objects may be Facebook users, the properties “likes/dislikes” for a very large list of webpages, and the categories “Trump voter/Clinton voter”. There is certainly a lot more to machine learning — for example, these techniques don’t shed light on the ways that AI Go systems improve their play. But it’s good to start with the basics. (Here is a simple presentation of the basics of machine learning; link.)

Two intuitive techniques form the core of basic machine learning theory. The first makes use of the measurement of conditional probabilities in conjunction with Bayes’ theorem to assign probabilities of the object being a Phi given the presence of properties xi. The second uses massively multi-factor regressions to calculate a probability for the event being Phi given regression coefficients ci.

Another basic technique is to treat the classification problem spatially. Use the large number of variables to define an n-dimensional space; then classify the object according to the average or majority value of its m-closest neighbors. (The neighbor number m might range from 1 to some manageable number such as 10.)

There are many issues of methodology and computational technique raised by this approach to knowledge. But these are matters of technique, and smart data science researchers have made great progress on them. More interesting here are epistemological issues: how good and how reliable are the findings produced by these approaches to the algorithmic treatment of large data sets? How good is the spam filter or the Trump voter detector when applied to novel data sets? What kind of errors would we anticipate this approach to be vulnerable to?

One important observation is that these methods are explicitly anti-theoretical. There is no place for discovery of causal mechanisms or underlying explanatory processes in these calculations. The researcher is not expected to provide a theoretical hypothesis about how this system of phenomena works. Rather, the techniques are entirely devoted to the discovery of persistent statistical associations among variables and the categories of the desired sorting. This is as close to Baconian induction as we get in the sciences (link). The approach is concerned about classification and prediction, not explanation. (Here is an interesting essay where Jake Hofman addresses the issues of prediction versus explanation of social data; link.)

A more specific epistemic concern that arises is the possibility that the training set of data may have had characteristics that are importantly different from comparable future data sets. This is the familiar problem of induction: will the future resemble the past sufficiently to support predictions based on past data? Spam filters developed in one email community may work poorly in an email community in another region or profession. We can label this as the problem of robustness.

Another limitation of this approach has to do with problems where our primary concern is with a singular event or object rather than a population. If we want to know whether NSA employee John Doe is a Russian mole, it isn’t especially useful to know that his nearest neighbors in a multi-dimensional space of characteristics are moles; we need to know more specifically whether Doe himself has been corrupted by the Russians. If we want to know whether North Korea will explode a nuclear weapon against a neighbor in the next six months the techniques of machine learning seem to be irrelevant.

The statistical and computational tools of machine learning are indeed powerful, and seem to lead to results that are both useful and sometimes surprising. One should not imagine, however, that machine learning is a replacement for all other forms of research methodology in the social and behavioral sciences.

(Here is a brief introduction to a handful of the algorithms currently in use in machine-learning applications; link.)

Mechanisms, singular and general

Let’s think again about the semantics of causal ascriptions. Suppose that we want to know what  caused a building crane to collapse during a windstorm. We might arrive at an account something like this:

  • An unusually heavy gust of wind at 3:20 pm, in the presence of this crane’s specific material and structural properties, with the occurrence of the operator’s effort to adjust the crane’s extension at 3:21 pm, brought about cascading failures of structural elements of the crane, leading to collapse at 3:25 pm.

The process described here proceeds from the “gust of wind striking the crane” through an account of the material and structural properties of the device, incorporating the untimely effort by the operator to readjust the device’s extension, leading to a cascade from small failures to a large failure. And we can identify the features of causal necessity that were operative at the several links of the chain.

Notice that there are few causal regularities or necessary and constant conjunctions in this account. Wind does not usually bring about the collapse of cranes; if the operator’s intervention had occurred a few minutes earlier or later, perhaps the failure would not have occurred; and small failures do not always lead to large failures. Nonetheless, in the circumstances described here there is causal necessity extending from the antecedent situation at 3:15 pm to the full catastrophic collapse at 3:25 pm.

Does this narrative identify a causal mechanism? Are we better off describing this as a sequences of cause-effect sequences, none of which represents a causal mechanism per se? Or, on the contrary, can we look at the whole sequence as a single causal mechanism — though one that is never to be repeated? Does a causal mechanism need to be a recurring and robust chain of events, or can it be a highly unique and contingent chain?

Most mechanisms theorists insist on a degree of repeatability in the sequences that they describe as “mechanisms”. A causal mechanism is the triggering pathway through which one event leads to the production of another event in a range of circumstances in an environment. Fundamentally a causal mechanism is a “molecule” of causal process which can recur in a range of different social settings.

For example:

  • X typically brings about O.

Whenever this sequence of events occurs, in the appropriate timing, the outcome O is produced. This ensemble of events {X, O} is a single mechanism.

And here is the crucial point: to call this a mechanism requires that this sequence recurs in multiple instances across a range of background conditions.

This suggests an answer to the question about the collapsing crane: the sequence from gust to operator error to crane collapse is not a mechanism, but is rather a unique causal sequence. Each part of the sequence has a causal explanation available; each conveys a form of causal necessity in the circumstances. But the aggregation of these cause-effect connections falls short of constituting a causal mechanism because the circumstances in which it works are all but unique. A satisfactory causal explanation of the internal cause-effect pairs will refer to real repeatable mechanisms — for example, “twisting a steel frame leads to a loss of support strength”. But the concatenation does not add up to another, more complex, mechanism.

Contrast this with “stuck valve” accidents in nuclear power reactors. Valves control the flow of cooling fluids around the critical fuel. If the fuel is deprived of coolant it rapidly overheats and melts. A “stuck valve-loss of fluid-critical overheating” sequence is a recognized mechanism of nuclear meltdown, and has been observed in a range of nuclear-plant crises. It is therefore appropriate to describe this sequence as a genuine causal mechanism in the creation of a nuclear plant failure.

(Stuart Glennan takes up a similar question in “Singular and General Causal Relations: A Mechanist Perspective”; link.)

Technology lock-in accidents

image: diagram of molten salt reactor

Organizational and regulatory features are sometimes part of the causal background of important technology failures. This is particularly true in the history of nuclear power generation. The promise of peaceful uses of atomic energy was enormously attractive at the end of World War II. In abstract terms the possibility of generating useable power from atomic reactions was quite simple. What was needed was a controllable fission reaction in which the heat produced by fission could be captured to run a steam-powered electrical generator.

The technical challenges presented by harnessing nuclear fission in a power plant were large. Fissionable material needed to be produced as useable fuel sources. A control system needed to be designed to maintain the level of fission at a desired level. And, most critically, a system for removing heat from the fissioning fuel needed to be designed so that the reactor core would not overheat and melt down, releasing energy and radioactive materials into the environment.

Early reactor designs took different approaches to the heat-removal problem. Liquid metal reactors used a metal like sodium as the fluid that would run through the core removing heat to a heat sink for dispersal; and water reactors used pressurized water to serve that function. The sodium breeder reactor design appeared to be a viable approach, but incidents like the Fermi 1 disaster near Detroit cast doubt on the wisdom of using this approach. The reactor design that emerged as the dominant choice in civilian power production was the light water reactor. But light water reactors presented their own technological challenges, including most especially the risk of a massive steam explosion in the event of a power interruption to the cooling plant. In order to obviate this risk reactor designs involved multiple levels of redundancy to ensure that no such power interruption would occur. And much of the cost of construction of a modern light water power plant is dedicated to these systems — containment vessels, redundant power supplies, etc. In spite of these design efforts, however, light water reactors at Three Mile Island and Fukushima did in fact melt down under unusual circumstances — with particularly devastating results in Fukushima. The nuclear power industry in the United States essentially died as a result of public fears of the possibility of meltdown of nuclear reactors near populated areas — fears that were validated by several large nuclear disasters.

What is interesting about this story is that there was an alternative reactor design that was developed by US nuclear scientists and engineers in the 1950s that involved a significantly different solution to the problem of harnessing the heat of a nuclear reaction and that posed a dramatically lower level of risk of meltdown and radioactive release. This is the molten salt reactor, first developed at the Oak Ridge National Laboratory facility in the 1950s. This was developed as part of the loopy idea of creating an atomic-powered aircraft that could remain aloft for months. This reactor design operates at atmospheric pressure, and the technological challenges of maintaining a molten salt cooling system are readily solved. The fact that there is no water involved in the cooling system means that the greatest danger in a nuclear power plant, a violent steam explosion, is eliminated entirely. Molten salt will not turn to steam, and the risk of a steam-based explosion is removed completely. Chinese nuclear energy researchers are currently developing a next generation of molten salt reactors, and there is a likelihood that they will be successful in designing a reactor system that is both more efficient in terms of cost and dramatically safer in terms of low-probability, high-cost accidents (link). This technology also has the advantage of making much more efficient use of the nuclear fuel, leaving a dramatically smaller amount of radioactive waste to dispose of.

So why did the US nuclear industry abandon the molten-salt reactor design? This seems to be a situation of lock-in by an industry and a regulatory system. Once the industry settled on the light water reactor design, it was implemented by the Nuclear Regulatory Commission in terms of the regulations and licensing requirements for new nuclear reactors. It was subsequently extremely difficult for a utility company or a private energy corporation to invest in the research and development and construction costs that would be associated with a radical change of design. There is currently an effort by an American company to develop a new-generation molten salt reactor, and the process is inhibited by the knowledge that it will take a minimum of ten years to gain certification and licensing for a possible commercial plant to be based on the new design (link).

This story illustrates the possibility that a process of technology development may get locked into a particular approach that embodies substantial public risk, and it may be all but impossible to subsequently adopt a different approach. In another context Thomas Hughes refers to this as technological momentum, and it is clear that there are commercial, institutional, and regulatory reasons for this “stickiness” of a major technology once it is designed and adopted. In the case of nuclear power the inertia associated with light water reactors is particularly unfortunate, given that it blocked other solutions that were both safer and more economical.

(Here is a valuable review of safety issues in the nuclear power industry; link.)

Consensus and mutual understanding

Groups make decisions through processes of discussion aimed at framing a given problem, outlining the group’s objectives, and arriving at a plan for how to achieve the objectives in an intelligent way. This is true at multiple levels, from neighborhood block associations to corporate executive teams to the President’s cabinet meetings. However, collective decision-making through extended discussion faces more challenges than is generally recognized. Processes of collective deliberation are often haphazard, incomplete, and indeterminate.

What is collective deliberation about? It is often the case that a collaborative group or team has a generally agreed-upon set of goals — let’s say reducing the high school dropout rate in a city or improving morale on the plant floor or deterring North Korean nuclear expansion. The group comes together to develop a strategy and a plan for achieving the goal. Comments are offered about how to think about the problem, what factors may be relevant to bringing the problem about, what interventions might have a positive effect on the problem. After a reasonable range of conversation the group arrives at a strategy for how to proceed.

An idealized version of group problem-solving makes this process both simple and logical. The group canvases the primary facts available about the problem and its causes. The group recognized that there may be multiple goods involved in the situation, so the primary objective needs to be considered in the context of the other valuable goods that are part of the same bundle of activity. The group canvases these various goods as well. The group then canvases the range of interventions that are feasible in the existing situation, along with the costs and benefits of each strategy. Finally, the group arrives at a consensus about which strategy is best, given everything we know about the dynamics of the situation.

But anyone who has been part of a strategy-oriented discussion asking diverse parties to think carefully about a problem that all participants care about will realize that the process is rarely so amenable to simple logical development. Instead, almost every statement offered in the discussion is both ambiguous to some extent and factually contestable. Outcomes are sensitive to differences in the levels of assertiveness of various participants. Opinions are advanced as facts, and there is insufficient effort expended to validate the assumptions that are being made. Outcomes are also sensitive to the order and structure of the agenda for discussion. And finally, discussions need to be summarized; but there are always interpretive choices that need to be made in summarizing a complex discussion. Points need to be assigned priority and cogency; and different scribes will have different judgments about these matters.

Here is a problem of group decision-making that is rarely recognized but seems pervasive in the real world. This is the problem of recurring misunderstandings and ambiguities within the group of the various statements and observations that are made. The parties proceed on the basis of frameworks of assumptions that differ substantially from one person to the next but are never fully exposed. One person asserts that the school day should be lengthened, imagining a Japanese model of high school. Another thinks back to her own high school experience and agrees, thinking that five hours of instruction may well be more effective for learning than four hours. They agree about the statement but they are thinking of very different changes.

The bandwidth of a collective conversation about a complicated problem is simply too narrow to permit ambiguities and factually errors to be tracked down and sorted out. The conversation is invariably incomplete, and often takes shape because of entirely irrelevant factors like who speaks first or most forcefully. It is as if the space of the discussion is in two dimensions, whereas the complexity of the problem under review is in three dimensions.

The problem is exacerbated by the fact that participants sometimes have their own agendas and hobby horses that they continually re-inject into the discussion under varying pretexts. As the group fumbles towards possible consensus these fixed points coming from a few participants either need to be ruled out or incorporated — and neither is a fully satisfactory result. If the point is ruled out some participants will believe their inputs are not respected, but if it is incorporated then the consensus has been deformed from a more balanced view of the issue.

A common solution to the problems of group deliberation mentioned here is to assign an expert facilitator or “muse” for the group who is tasked to build up a synthesis of the discussion as it proceeds. But it is evident that the synthesis is underdetermined by the discussion. Some points will be given emphasis over others, and a very different story line could have been reached that leads to different outcomes. This is the Rashomon effect applied to group discussions.

A different solution is to think of group discussion as simply an aid to a single decision maker — a chief executive who listens to the various points of view and then arrives at her own formulation of the problem and a solution strategy. But of course this approach abandons the idea of reaching a group consensus in favor of the simpler problem of an individual reaching his or her own interpretation of the problem and possible solutions based on input from others.

This is a problem for organizations, both formal and informal, because every organization attempts to decide what to do through some kind of exploratory discussion. It is also a problem for the theory of deliberative democracy (link, link).

This suggests that there is an important problem of collective rationality that has not been addressed either by philosophy or management studies: the problem of aggregating beliefs, perceptions, and values held by diverse members of a group onto a coherent statement of the problem, causes, and solutions for the issue under deliberation. We would like to be able to establish processes that lead to rational and effective solutions to problems that incorporate available facts and judgments. Further we would like the outcomes to be non-arbitrary — that is, given an antecedent set of factual and normative beliefs by the participants, we would like to imagine that there is a relatively narrow band of policy solutions that will emerge as the consensus or decision. We have theories of social choice — aggregation of fixed preferences. And we have theories of rational decision-making and planning. But a deliberative group discussion of an important problem is substantially more complex. We need a philosophy of the meeting!

%d bloggers like this: