A new model of organization?

In Team of Teams: New Rules of Engagement for a Complex World General Stanley McChrystal (with Tantum Collins, David Silverman, and Chris Fussell) describes a new, 21st-century conception of organization for large, complex activities involving thousands of individuals and hundreds of major sub-tasks. His concept is grounded in his experience in counter-insurgency warfare in Iraq. Rather than being constructed as centrally organized, bureaucratic, hierarchical processes with commanders and scripted agents, McChrystal argues that modern counter-terrorism requires a more decentralized and flexible system of action, which he refers to as “teams of teams”. Information is shared freely, local commanders have ready access to resources and knowledge from other experts, and they make decisions in a more flexible way. The model hopes to capture the benefits of improvisation, flexibility, and a much higher level of trust and communication than is characteristic of typical military and corporate organizations.

 

One place where the “team of teams” structure is plausible is in the context of a focused technology startup company, where the whole group of participants need to be in regular and frequent collaboration with each other. Indeed, Paul Rabinow’s ethnography in 1996 of the Cetus Corporation in its pursuit of PCR (polymerase chain reaction) in Making PCR: A Story of Biotechnology reflects a very similar topology of information flows and collaboration links across and within working subgroups (link). But the vision does not fit very well the organizational and operational needs of a large hospital, a railroad company, or a research university. It seems plausible that the challenges the US military faced in fighting Al-Qaeda and ISIL are not really analogous to those faced by less dramatic organizations like hospitals, universities, and corporations. The decentralized and improvisational circumstances of urban warfare against loosely organized terrorists may be sui generis

McChrystal proposes an organizational structure that is more decentralized, more open to local decision-making, and more flexible and resilient. These are unmistakeable virtues in some circumstances; but not in all circumstances and all organizations. And arguably such a structure would have been impossible in the planning and execution of the French defense of Dien Bien Phu or the US decision to wage war against the Vietnamese insurgency ten years later. These were situations where central decisions needed to be made, and the decisions needed to be implemented through well organized bureaucracies. The problem in both instances is that the wrong decisions were made, based on the wrong information and assessments. What was needed, it would appear, was better executive leadership and decision-making — not a fundamentally decentralized pattern of response and counter-response.

One thing that deserves comment in the context of McChrystal’s book is the history of bad organization, bad intelligence, and bad decision-making the world has witnessed in the military experiences of the past century. The radical miscalculations and failures of planning involved in the first months of the Korean War, the painful and tragic misjudgments made by the French military in preparing for Dien Bien Phu, the equally bad thinking and planning done by Robert McNamara and the whiz kids leading to the Vietnam War — these examples stand out as sentinel illustrations of the failures of large organizations that have been tasked to carry out large, complex activities involving numerous operational units. The military and the national security establishments were good at some tasks, and disastrously bad at others. And the things they were bad at were both systemic and devastating. Bernard Fall illustrates these failures in Hell In A Very Small Place: The Siege Of Dien Bien Phu, and David Halberstam does so for the decision-making that led to the war in Vietnam in The Best and the Brightest.

So devising new ideas about command, planning, intelligence gathering and analysis, and priority-setting that are more effective would be a big contribution to humanity. But the deficiencies in Dien Bien Phu, Korea, or Vietnam seem different from those McChrystal identifies in Iraq. What was needed in these portentous moments of policy choice was clear-eyed establishment of appropriate priorities and goals, honest collection of intelligence and sources of information, and disinterested implementation of policies and plans that served the highest interests of the country. The “team of teams” approach doesn’t seem to be a general solution to the wide range of military and political challenges nations face.

What one would have wanted to see in the French military or the US national security apparatus is something different from the kind of teamwork described by McChrystal: greater honesty on all parts, a commitment to taking seriously the assessments of experts and participants in the field, an openness to questioning strongly held assumptions, and a greater capacity for institutional wisdom in arriving at decisions of this magnitude. We would have wanted to see a process that was not dominated by large egos, self-interest, and fixed ideas. We would have wanted French generals and their civilian masters to soberly assess the military function that a fortress camp at Dien Bien Phu could satisfy; the realistic military requirements that would need to be satisfied in order to defend the location; and an honest effort to solicit the very best information and judgment from experienced commanders and officials about what a Viet-Minh siege might look like. Instead, the French military was guided by complacent assumptions about French military superiority, which led to a genuine catastrophe for the soldiers assigned to the task and to French society more broadly.

There are valid insights contained in McChrystal’s book about the urgency of breaking down obstacles to communication and action within sprawling organizations as they confront a changing environment. But it doesn’t add up to a model that is well designed for most contexts in which large organizations actually function.

Advertisements

How organizations adapt

Organizations do things; they depend upon the coordinated efforts of numerous individuals; and they exist in environments that affect their ongoing success or failure. Moreover, organizations are to some extent plastic: the practices and rules that make them up can change over time. Sometimes these changes happen as the result of deliberate design choices by individuals inside or outside the organization; so a manager may alter the rules through which decisions are made about hiring new staff in order to improve the quality of work. And sometimes they happen through gradual processes over time that no one is specifically aware of. The question arises, then, whether organizations evolve toward higher functioning based on the signals from the environments in which they live; or on the contrary, whether organizational change is stochastic, without a gradient of change towards more effective functioning? Do changes within an organization add up over time to improved functioning? What kinds of social mechanisms might bring about such an outcome?

One way of addressing this topic is to consider organizations as mid-level social entities that are potentially capable of adaptation and learning. An organization has identifiable internal processes of functioning as well as a delineated boundary of activity. It has a degree of control over its functioning. And it is situated in an environment that signals differential success/failure through a variety of means (profitability, success in gaining adherents, improvement in market share, number of patents issued, …). So the environment responds favorably or unfavorably, and change occurs.

Is there anything in this specification of the structure, composition, and environmental location of an organization that suggests the possibility or likelihood of adaptation over time in the direction of improvement of some measure of organizational success? Do institutions and organizations get better as a result of their interactions with their environments and their internal structure and actors?

There are a few possible social mechanisms that would support the possibility of adaptation towards higher functioning. One is the fact that purposive agents are involved in maintaining and changing institutional practices. Those agents are capable of perceiving inefficiencies and potential gains from innovation, and are sometimes in a position to introduce appropriate innovations. This is true at various levels within an organization, from the supervisor of a custodial staff to a vice president for marketing to a CEO. If the incentives presented to these agents are aligned with the important needs of the organization, then we can expect that they will introduce innovations that enhance functioning. So one mechanism through which we might expect that organizations will get better over time is the fact that some agents within an organization have the knowledge and power necessary to enact changes that will improve performance, and they sometimes have an interest in doing so. In other words, there is a degree of intelligent intentionality within an organization that might work in favor of enhancement.

This line of thought should not be over-emphasized, however, because there are competing forces and interests within most organizations. Previous posts have focused on current organizational theory based on the idea of a “strategic action field” of insiders and outsiders who determine the activities of the organization (Fligstein and McAdam, Crozier; linklink). This framework suggests that the structure and functioning of an organization is not wholly determined by a single intelligent actor (“the founder”), but is rather the temporally extended result of interactions among actors in the pursuit of diverse aims. This heterogeneity of purposive actions by actors within an institution means that the direction of change is indeterminate; it is possible that the coalitions that form will bring about positive change, but the reverse is possible as well.

And in fact, many authors and participants have pointed out that it is often enough not the case that the agents’ interests are aligned with the priorities and needs of the organization. Jack Knight offers persuasive critique of the idea that organizations and institutions tend to increase in their ability to provide collective benefits in Institutions and Social Conflict. CEOs who have a financial interest in a rapid stock price increase may take steps that worsen functioning for shortterm market gain; supervisors may avoid work-flow innovations because they don’t want the headache of an extended change process; vice presidents may deny information to other divisions in order to enhance appreciation of the efforts of their own division. Here is a short description from Knight’s book of the way that institutional adjustment occurs as a result of conflict among players of unequal powers:

Individual bargaining is resolved by the commitments of those who enjoy a relative advantage in substantive resources. Through a series of interactions with various members of the group, actors with similar resources establish a pattern of successful action in a particular type of interaction. As others recognize that they are interacting with one of the actors who possess these resources, they adjust their strategies to achieve their best outcome given the anticipated commitments of others. Over time rational actors continue to adjust their strategies until an equilibrium is reached. As this becomes recognized as the socially expected combination of equilibrium strategies, a self-enforcing social institution is established. (Knight, 143)

A very different possible mechanism is unit selection, where more successful innovations or firms survive and less successful innovations and firms fail. This is the premise of the evolutionary theory of the firm (Nelson and Winter, An Evolutionary Theory of Economic Change). In a competitive market, firms with low internal efficiency will have a difficult time competing on price with more efficient firms; so these low-efficiency firms will go out of business occasionally. Here the question of “units of selection” arises: is it firms over which selection operates, or is it lower-level innovations that are the object of selection?

Geoffrey Hodgson provides a thoughtful review of this set of theories here, part of what he calls “competence-based theories of the firm”. Here is Hobson’s diagram of the relationships that exist among several different approaches to study of the firm.

The market mechanism does not work very well as a selection mechanism for some important categories of organizations — government agencies, legislative systems, or non-profit organizations. This is so, because the criterion of selection is “profitability / efficiency within a competitive market”; and government and non-profit organizations are not importantly subject to the workings of a market.

In short, the answer to the fundamental question here is mixed. There are factors that unquestionably work to enhance effectiveness in an organization. But these factors are weak and defeasible, and the countervailing factors (internal conflict, divided interests of actors, slackness of corporate marketplace) leave open the possibility that institutions change but they do not evolve in a consistent direction. And the glaring dysfunctions that have afflicted many organizations, both corporate and governmental, make this conclusion even more persuasive. Perhaps what demands explanation is the rare case where an organization achieves a high level of effectiveness and consistency in its actions, rather than the many cases that come to mind of dysfunctional organizational activity.

(The examples of organizational dysfunction that come to mind are many — the failures of nuclear regulation of the civilian nuclear industry (Perrow, The Next Catastrophe: Reducing Our Vulnerabilities to Natural, Industrial, and Terrorist Disasters); the failure of US anti-submarine warfare in World War II (Cohen, Military Misfortunes: The Anatomy of Failure in War); and the failure of chemical companies to ensure safe operations of their plants (Shrivastava, Bhopal: Anatomy of Crisis). Here is an earlier post that addresses some of these examples; link. And here are several earlier posts on the topic of institutional change and organizational behavior; linklink.)

Errors in organizations

Organizations do things — process tax returns, deploy armies, send spacecraft to Mars. And in order to do these various things, organizations have people with job descriptions; organization charts; internal rules and procedures; information flows and pathways; leaders, supervisors, and frontline staff; training and professional development programs; and other particular characteristics that make up the decision-making and action implementation of the organization. These individuals and sub-units take on tasks, communicate with each other, and give rise to action steps.

And often enough organizations make mistakes — sometimes small mistakes (a tax return is sent to the wrong person, a hospital patient is administered two aspirins rather than one) and sometimes large mistakes (the space shuttle Challenger is cleared for launch on January 28, 1986, a Union Carbide plant accidentally releases toxic gases over a large population in Bhopal, FEMA bungles its response to Hurricane Katrina). What can we say about the causes of organizational mistakes? And how can organizations and their processes be improved so mistakes are less common and less harmful?

Charles Perrow has devoted much of his career to studying these questions. Two books in particular have shed a great deal of light on the organizational causes of industrial and technological accidents, Normal Accidents: Living with High-Risk Technologies and The Next Catastrophe: Reducing Our Vulnerabilities to Natural, Industrial, and Terrorist Disasters. (Perrow’s work has been discussed in several earlier posts; linklinklink.) The first book emphasizes that errors and accidents are unavoidable; they are the random noise of the workings of a complex organization. So the key challenge is to have processes that detect errors and that are resilient to the ones that make it through. One of Perrow’s central findings in The Next Catastrophe is the importance of achieving a higher level of system resilience by decentralizing risk and potential damage. Don’t route tanker cars of chlorine through dense urban populations; don’t place nuclear power plants adjacent to cities; don’t create an Internet or a power grid with a very small number of critical nodes. Kathleen Tierney’s The Social Roots of Risk: Producing Disasters, Promoting Resilience (High Reliability and Crisis Management) emphasizes the need for system resilience as well (link).

Is it possible to arrive at a more granular understanding of organizational errors and their sources? A good place to begin is with the theory of organizations as “strategic action fields” in the sense advocated by Fligstein and McAdam in A Theory of Fields. This approach imposes an important discipline on us — it discourages the mental mistake of reification when we think about organizations. Organizations are not unitary decision and action bodies; instead, they are networks of people linked in a variety of forms of dependency and cooperation. Various sub-entities consider tasks, gather information, and arrive at decisions for action, and each of these steps is vulnerable to errors and shortfalls. The activities of individuals and sub-groups are stimulated and conveyed through these networks of association; and, like any network of control or communication, there is always the possibility of a broken link or a faulty action step within the extended set of relationships that exist.

Errors can derive from individual mistakes; they can derive from miscommunication across individuals and sub-units within the organization; they can derive from more intentional sources, including self-interested or corrupt behavior on the part of internal participants. And they can derive from conflicts of interest between units within an organization (the manufacturing unit has an interest in maximizing throughput, the quality control unit has an interest in minimizing faulty products).

Errors are likely in every part of an organization’s life. Errors occur in the data-gathering and analysis functions of an organization. A sloppy market study is incorporated into a planning process leading to a substantial over-estimate of demand for a product; a survey of suppliers makes use of ambiguous questions that lead to misinterpretation of the results; a vice president under-estimates the risk posed by a competitor’s advertising campaign. For an organization to pursue its mission effectively, it needs to have accurate information about the external circumstances that are most relevant to its goals. But “relevance” is a judgment issue; and it is possible for an organization to devote its intelligence-gathering resources to the collection of data that are only tangentially helpful for the task of designing actions to carry out the mission of the institution.

Errors occur in implementation as well. The action initiatives that emerge from an organization’s processes — from committees, from CEOs, from intermediate-level leaders, from informal groups of staff — are also vulnerable to errors of implementation. The facilities team formulates a plan for re-surfacing a group of parking lots; this plan depends upon closing these lots several days in advance; but the safety department delays in implementing the closure and the lots have hundreds of cars in them when the resurfacing equipment arrives. An error of implementation.

One way of describing these kinds of errors is to recognize that organizations are “loosely connected” when it comes to internal processes of information gathering, decision making, and action. The CFO stipulates that the internal audit function should be based on best practices nationally; the chief of internal audit interprets this as an expectation that processes should be designed based on the example of top-tier companies in the same industry; and the subordinate operationalizes this expectation by doing a survey of business-school case studies of internal audit functions at 10 companies. But the data collection that occurs now has only a loose relationship to the higher-level expectation formulated by the CFO. Similar disconnects — or loose connections — occur on the side of implementation of action steps as well. Presumably top FEMA officials did not intend that FEMA’s actions in response to Hurricane Katrina would be as ineffective and sporadic as they turned out to be.

Organizations also have a tendency towards acting on the basis of collective habits and traditions of behavior. It is easier for a university’s admissions department to continue the same programs of recruitment and enrollment year after year than it is to rethink the approach to recruitment in a fundamental way. And yet it may be that the circumstances of the external environment have changed so dramatically that the habitual practices will no longer achieve similar results. A good example is the emergence of social media marketing in admissions; in a very short period of time the 17- and 18-year-old young people whom admissions departments want to influence went from willing recipients of glossy admissions publications in the mail to “Facebook-only” readers. Yesterday’s correct solution to an organizational problem may become tomorrow’s serious error, because the environment has changed.

In a way the problem of organizational errors is analogous to the problem of software bugs in large, complex computer systems. It is recognized by software experts that bugs are inevitable; and some of these coding errors or design errors may have catastrophic consequences in unusual settings. (Nancy Leveson’s Safeware: System Safety and Computers provides an excellent review of these possibilities.) So the task for software engineers and organizational designers and leaders is similar: designing fallible systems that do a pretty good job almost all of the time, and are likely to fail gracefully when errors inevitably occur.

Positive organizational behavior

source: Rob Cross, Wayne Baker, Andrew Parker, “What creates energy in organizations?” (link)

Organizations need study for several important reasons. One is their ubiquity in modern life — almost nothing that we need in daily life is created by solo producers. Rather, activity among a number of individuals is coordinated and directed through organizations that produce the goods and services we need — grocery chains, trucking companies, police departments, universities, small groups of cooperating chimpanzees.

A second reason for studying organizations is that existing theories of human behavior don’t do a very good job of explaining organizational behavior. The theory of rational self interest — the premise of the market — doesn’t work very well as a sole theory of behavior within an organization. But neither does the normative theory of a Durkheim or a Weber. We need better theories of the forms of agency that are at work within organizations — the motives individuals have, the ways in which the rules and incentives of the organization affect behavior, the ways the culture of the workplace influences behavior, and the role that local level practices have in influencing individual behavior that makes a difference to the functioning of the organization.

Here are a few complications from current work in sociology and economics.

Economist Amartya Sen observes that the premises of market rationality make social cooperation all but impossible. This is Sen’s central conclusion in “Rational Fools” (link), and it is surely correct: “The purely rational economic man is indeed close to being a social moron”. Sen’s work demonstrates that social behavior — even conceding the point that it derives from the thought processes of individuals — is substantially more layered and multi-factored than neoclassical economics postulates. Sen’s own addition to the mix is his theory of commitments — the idea that individuals have priorities that don’t map conveniently onto utility schemes — and that lots of ordinary collective behavior depends on these behavioral characteristics.

Sociologist Michele Lamont argues that a major difference between upper-middle class French and American men is their attitudes towards their own work in the office or factory. In Money, Morals, and Manners: The Culture of the French and the American Upper-Middle Class she finds that professional-class French men express a certain amount of contempt for their hard-working American counterparts. Her findings suggest substantial differences in the “culture of work and profession” in different national and regional settings. (Here is an earlier post on Lamont’s work; link.)

Experimental economist Ernst Fehr finds that workplaces create substantial behavioral predispositions that are triggered by the frame of the workplace (link). In unpublished work he finds that individuals in the banking industry are slightly more honest than the general population when they think in the frame of their personal lives, but that they are substantially less honest when they think in the frame of the banking office. Fehr and his colleagues demonstrate the power of cultural cues in the workplace (and presumably other well-defined social environments) in influencing the way that individuals make decisions in that environment.

Fehr has also made a major contribution through his research in experimental economics on the subject of altruism. He finds — context-independently — that decision makers are generally not rationally self interested maximizers. And using some results from the neurosciences he argues that there is a biological basis for this “pro-social” element of behavior.  Here is an example of Fehr’s approach:

If we randomly pick two human strangers from a modern society and give them the chance to engage in repeated anonymous exchanges in a laboratory experiment, reciprocally altruistic behaviour emerges spontaneously with a high probability…. However, human altruism even extends far beyond reciprocal altruism and reputation-based cooperation taking the form of strong reciprocity. (Fehr and Fischbacher 2005:7;

link

)

(Here is an article by Jon Elster on Fehr’s experimental research on altruism; link.)

So what can we discover about common features of behavior that can be observed in different kinds of organizations? There is a degree of convergence between the theoretical and experimental results that have come out of this research in sociology and economics and the organizational theories of what is now referred to as positive organizational studies. Here is a brilliant collection of research in this area edited by Kim Cameron and Gretchen Spreitzer, The Oxford Handbook of Positive Organizational Scholarship. Cameron and Spreitzer define the field in their introduction in these terms:

Positive organizational scholarship is an umbrella concept used to unify a variety of approaches in organizational studies, each of which incorporates the notion of ‘the positive.’ … “organizational research occurring at the micro, meso, and macro levels which points to unanswered questions about what processes, states, and conditions are important in explaining individual and collective flourishing. Flourishing refers to being in an optimal range of human functioning” [quoting Jane Dutton] (2).

The POS research community places a great deal of importance on the impact that positive social behavior has on the effectiveness of an organization. And these scholars believe that specific institutional arrangements and actions by leaders can increase the levels of positive social behavior in a work environment.

Studies have shown that organizations in several industries (including financial services, health care, manufacturing, and government) that implemented and improved their positive practices over time also increased their performance in desired outcomes such as profitability, productivity, quality, customer satisfaction, and employee retention. That is, positive practices that were institutionalized in organizations, including providing compassionate support for employees, forgiving mistakes and avoiding blame, fostering the meaningfulness of work, expressing frequent gratitude, showing kindness, and caring for colleagues, led organizations to perform at significantly higher levels on desired outcomes. (6)

In a sense they point to the possibility of high level and low level equilibria within roughly the same set of rules. And organizations that succeed in promoting positive behavioral motivations will be more successful in achieving their goals. Adam Grant and Justin Berg analyze these positive motives in their contribution to the Handbook, “Prosocial Motivation at Work”.

What motivates employees to care about making a positive difference in the lives of others, and what actions and experiences does this motivation fuel? (29)

It is both a theoretical premise of the POS research community and an empirical subject of inquiry for these researchers that it is possible to elicit “prosocial” motivations through suitable institutional arrangements and leadership. Interestingly, this seems to be an implication of the work by Ernst Fehr mentioned above as well.

Positive organizational scholarship is a timely contribution to the social sciences because it stands on the cusp between the need for better theories of the actor and the imperative to improve the performance of organizations. Hospitals, manufacturing companies, universities, and non-profit organizations all want to improve their performance in a variety of ways: improve patient safety, reduce costs, improve product quality, improve student retention, improve the delivery of effective social services, and the like. And POS is an empirically grounded approach to arriving at a better understanding of the range of social behaviors that can potentially motivate participants and lead to better collective performance. And the category of “prosocial motivation” that underlies the POS approach is an important dimension of behavior for further research and investigation.

What drives organizational performance?

 

We have a pretty good idea of the characteristics that support very high individual performance in a variety of fields, from jazz to track to physics to business. An earlier post discussed some of the different combinations of features that characterize leaders in several different professions (link). And it isn’t difficult to sketch out qualities of personality, character, and style that make for a great teacher, researcher, entrepreneur, a great soccer player, or an exceptional police investigator. So we might imagine that a high-performing organization is one that has succeeded in assembling a group of high-performing individuals. But this is plainly untrue — witness the New York Yankees during much of the 2000s, the dot-com company WebVan during the late 1990s, and the XYZ Orchestra today. (Here is a thoughtful Mellon Foundation study of quality factors in symphony orchestras; link.) In each case the organization consisted of high-performing stars in their various disciplines, but somehow the ensemble performed poorly. The lesson from these examples is an obvious one: the performance of an organization is more than the sum of the abilities of its component members.

In fact, it seems apparent that organizational performance, like physical health, is a function of a number of separate parameters:

  • clarity about mission
  • appropriateness of internal functional specialization
  • quality of internal communication and collaboration across units and individuals
  • quality and intensity of individuals
  • quality of internal motivation
  • quality of leadership

We might say that an organization is like a physical mechanism in the sense that its overall performance depends on the quality of the design, the appropriate interconnections among the parts, and the quality of the individual components.

So what else goes into determining great organizational performance besides the quality of the individuals who make it up? A few things are obvious. Of course it is true that having individual participants who have the right kinds of talents is crucial. A technology company needs excellent engineers and designers. But it also needs highly talented marketing professionals, financial experts, and strategic planners. And it needs these talented specialists in a number of critical areas. Why did Xerox PARC fail in spite of the excellence of its scientists and engineers, and the innovativeness of the products that they created? Because the organization lacked the ability — and the individuals — to turn those ideas and innovations into products that the public wanted to buy. (Here is Malcolm Gladwell’s take on Xerox PARC in the New Yorker; link.)

A key aspect of the problem of designing and tuning an organization’s features to ensure high performance is being able to determine with precision what the mission of the organization is. What is the organization fundamentally established to bring about? If the Red Cross is an organization that is intended to deliver resources and assistance to communities that have suffered extensive disasters, that implies one set of functional needs to be satisfied by divisions and specialists within the organization. If it is primarily a fund-raising and marketing organization aimed at raising public awareness and generating large amounts of public donations to be used for disaster relief, that implies a different set of internal specialists. So being clear about the overall mission of the organization is crucial for the designers, so they can skillfully design a set of divisions, specialists, and work processes that can work together effectively to carry out the tasks necessary to succeed in achieving the mission.

This point highlights the fact that an organization needs to have a functional structure in which the activities of individuals or departments carry out specialized tasks. These sub-units depend upon the high-level work of other departments or individuals, and the functional structure of the organization can be more or less appropriate to the task. The organization succeeds to the extent that its component parts succeed in identifying the needs and opportunities facing the organization and in carrying out their roles in responding to those needs and opportunities. Poor performance in one department can have the effect of ruining the overall success of the organization to carry out its mission — even if other departments are highly successful in carrying out their tasks. Charles Perrow highlights this kind of organizational deficiency in Normal Accidents: Living with High-Risk Technologies.

Here is another important variable in bringing about organizational effectiveness: the procedures within the organization that are designed to encourage high-quality effort and results on the parts of the individuals who occupy roles throughout the organization. One line of response to this issue flows through a system of supervision and assessment. This approach emphasizes measurement of performance and positive and negative incentives to motivate satisfactory performance. Supervisors are tasked to ensure that employees are exerting themselves and that their work product is of satisfactory quality.

But a different response proceeds through a theory of internal motivation. Leaders and supervisors encourage high-quality effort and achievement by expressing the valuable goals that the organization is pursuing and by offering the reward of participation in effective work that one cares about to employees. This positive motivational feature is strengthened if the organization visibly maintains its commitment to treat its employees fairly and decently. If an employee is proud to work for Ben and Jerry’s, he or she is strongly motivated to make the best contribution possible to the work of the company. In a nutshell this is the theory that underlies the very interesting literature of positive organizational scholarship (Kim Cameron and Gretchen Spreitzer, The Oxford Handbook of Positive Organizational Scholarship).

A fifth facet of organizational performance plainly has to do with internal communication, coordination, and collaboration. The eventual success or failure of an organizational initiative will depend on the activities of individuals and units spread out throughout the organization. The work of various of those units can be made more effective or less effective by the ease and seriousness with which they are able to communicate with each other. Suppose a car company is designing a new model. Many units will be involved in bringing the design to fruition. If the body designers, the power train designers, and the manufacturing engineers haven’t talked to each other, there is a likelihood that solutions chosen by one set of specialists will create major problems for the other specialists. (The Saab 900 of the late 1970s was a beautiful and high-performing vehicle; but because the design process had not taken into account the need for convenient servicing, it was necessary to remove the engine to carry out some common kinds of repair.) Thomas Hughes provides an excellent analysis of the organizational deficiencies of the design process used in the United States military aerospace sector in the 1950s and 1960s in Rescuing Prometheus: Four Monumental Projects That Changed the Modern World. Here is his comparison of good and bad organizational forms:

The top diagram is entirely hierarchical, with decision-makers at the top deciding the flow of work below and essentially no communication across sub-units. The bottom diagram, by contrast, involves a great deal of internal communication, allowing for adjustment of design and timing decisions so that the eventual plan has the greatest likelihood for success. The latter permits the implementation of systems engineering rather than component engineering. Here is Hughes’s depiction of what happens when an organization lacks good internal communication and coordination:

What this implies is that improving organizational performance is a bit like tuning a piano: we need to continually adjust the factors (motivation, collaboration, mission, leadership, specialization) in such a way as to create a joint system of activity that succeeds at a high level in creating the desired results.

(I used images of musical ensembles to open this topic. But how good is the analogy? Actually, it is not a particularly good analogy. The issue of the quality of the players is obviously relevant, and quality of leadership has an exact parallel in the symphony orchestra. But the task of giving an excellent performance of Dvorak’s ninth symphony is much simpler than that of bringing about a successful intervention by FEMA in response to a hurricane. There is a score for the musicians; there is a central conductor who keeps them in step with each other; and most crucially, there is no uncertainty about what to do once the third movement is finished; the musicians turn the page and move on to the fourth movement. Perhaps the jazz ensemble pictured above is a slightly better metaphor for a complex organization in that it leaves room for improvisation by the players. But even here, the activity is orders of magnitude simpler and easier to coordinate than a large organization whose actions take place over months or years, dispersed over thousands of miles and multiple sites of activity. So organizational effectiveness is a more complex process than musical coordination and performance.)

(I emphasize here the importance of collaboration as a variable in organizational effectiveness. This suggests examples drawn from team activities like soccer or a research laboratory. But some experts doubt the idea that teams are always superior to more hierarchical structures. Here is J. Richard Hackman on the positives and negatives of teams (link).)

Gradual institutional change

Here are some very fundamental questions that we can ask about social institutions — the relatively durable sets of rules, practices, and norms through which a variety of human social activity is conducted.

  • How are institutions formed? 
  • How do they work — what are the enforcement mechanisms that exist within institutions that induce participants to conform to the rules? 
  • What factors support the stability of an institution over time? 
  • What kinds of processes lead to change within institutions over time? 
  • And why are institutions sometimes swept away and replaced wholesale? 

These are among the most fundamental questions that theories of institutions need to answer: how do they start, how are they sustained, and how do they change.

James Mahoney and Kathleen Thelen undertake to provide a basis for answering the fourth question here — the question of gradual change. Their volume, Explaining Institutional Change: Ambiguity, Agency, and Power, provides a simple but compelling answer: institutions change when position holders within them find circumstances in which they have both an opportunity and an incentive to change or reinterpret the rules in ways that serve their interests.

The issue of the internal processes of change within an institution is of interest here for several reasons. But central among them is the idea of plasticity that has been described in earlier posts (link, link, link). The basic idea of plasticity is that institutions and organizations are the product of various kinds of structured human action, and that they can change over time. So we shouldn’t think of institutions as having fixed characteristics, or as though they were equilibrium systems that tend to return to their original states after perturbances. Mahoney and Thelen’s volume demonstrates some of the ways in which this plasticity emerges; they prove an account of the mechanisms of gradual institutional change. And this approach makes plain the high degree of path-dependency that institutions display.

Here is how Mahoney and Thelen frame their research problem:

A growing body of work suggests that important changes often take place incrementally and through seemingly small adjustments that can, however, cumulate into significant institutional transformation. (preface) 

Once created, institutions often change in subtle and gradual ways over time. Although less dramatic than abrupt and wholesale transformations, these slow and piecemeal changes can be equally consequential for patterning human behavior and for shaping substantive political outcomes. (1) 

We have good theories of why various kinds of basic institutional configurations — constitutions, welfare systems, and property right arrangements — come into being in certain cases and at certain times. And we have theories to explain those crucial moments when these institutional configurations are upended and replaced with fundamentally new ones. But still lacking are equally useful tools for explaining the more gradual evolution of institutions once they have been established. (2)

The theory they offer of gradual institutional change is an actor-centered theory. Incremental change occurs as the result of the opportunistic and strategic choices made by a range of actors within the institution. In this respect it resembles the theories discussed earlier of Fligstein and McAdam (link) and Crozier and Friedberg (link). Consider the following series of comments:

We argue that institutional change often occurs precisely when problems of rule interpretation and enforcement open up space for actors to implement existing rules in new ways. (4) 

But institutional outcomes need not reflect the goals of any particular group; they may be the unintended outcome of conflict among groups or the result of “ambiguous compromises” among actors who can coordinate on institutional means even if they differ on substantive goals. (8) 

Given a view of institutional stability that rests not just on the accumulation but also on the ongoing mobilization of resources, one important source of change will be shifts in the balance of power. (8)

We propose that the basic properties of institutions contain within them possibilities for change. What animates change is the power-distributional implications of institutions. However, where we expect incremental change to emerge is precisely in the “gaps” or “soft spots” between the rule and its interpretation or the rule and its enforcement. (13)

These descriptions conform very well with the description of organizations offered by Fligstein and McAdam in their theory of strategic action fields (link). The theory of change offered by Mahoney and Thelen is an agent-centered view, and therefore has strong affinities with the strategic action field theory and the view of organizations offered by Crozier and Friedberg.

Institutions are generally thought of as “shaping” factors on human action and choice; individuals construct their actions and strategies within the context of the rules and norms of various institutions. But how do the rules of an organization actually constrain behavior?  Mahoney and Thelen do not take compliance within an institution as a given; instead, they look for the interests and opportunities of various agents within the organization or institution that interlock to secure compliance. This sounds a lot like “governance units” in the Fligstein and McAdam theory of strategic action fields.

If, instead, we break with a view of institutions as self-reinforcing (through whatever mechanism) and put distributional issues front and center, compliance emerges as a variable, and a variable that is crucially important to the analysis of both stability and change. The need to enforce institutions carries its own dynamic of potential change, emanating not just from the politically contested nature of institutional rules but also, importantly, from a degree of openness in the interoperation and implementation of these rules. (10)

Much of the content of the book takes the form of case studies of particular complexes of institutions in the midst of historical change.  Particularly interesting is Dan Slater’s analysis of the autocratic institutions of governance crafted by Suharto throughout his regime (1966-1998) (“Altering Authoritarianism: Institutional Complexity and Autocratic Agency in Indonesia”). This period is often treated as a homogeneous block of authoritarian military rule, but Slater argues that there was substantial institutional change throughout the period.

Ostensibly a model of long-term stability, the Suharto regime in fact experienced a gradual but significant transformation of its core institutional features. What started as a system of oligarchic military rule evolved into a highly personalized regime, backed in nearly equal measure by military and civilian organizations. (132)

The upshot was the gradual transformation of a collectivistic military-dominated regime (or “junta”) into a highly personalized regime, perched atop a mixed party-military infrastructure (or a “strongman” regime with pronounced elements of “bossism”). (139) 

Slater analyzes the strategic situations of the primary political players: Suharto, the mass political parties, and the military. And he understands the transition described here as one that followed from the strategic moves made by these players, with Suharto’s hand being the most influential. But this is a key point: Suharto was not able to simply command the changes he sought, and the outcomes reflected the agency of other important social actors as well.

Alan Jacobs’ treatment of the U.S. Social Security Program is also very interesting, as is Tulia Falleti’s account of health care reforms in Brazil between 1964 and 1988. Each case study provides just what Mahoney and Thelen call for in their introduction: careful, detailed studies of institutions in a process of change.

It is worth noting that all the examples treated in this collection are concerned with large institutions — the Brazilian policy arena, the Suharto dictatorship, the U.S. Social Security Program. It makes the reader wonder whether similar results would emerge when more meso-level institutions and organizations are studied: the Massachusetts Institute of Technology, the IBM Corporation, or the Michigan Department of Human Services. Do the observations about the strategic interactions of primary players within the institution play an equally important role in change within institutions at this level of scale? I drew a parallel between the Mahoney-Thelen approach to institutional change and the theories of Fligstein and McAdam, as well as Crozier and Friedberg. But this appears to be a salient difference — the scale of the institutions to which the analysis is applied. The latter two theories seem to be developed with primary emphasis on smaller-scale institutions and organizations.

Mayer Zald on organizations and bureaucracy

Mayer Zald helped to shape the field of organizational behavior in the United States, beginning with his time as a faculty member at Vanderbilt and continuing through his long career in sociology at the University of Michigan. In 1971 he published an early version of some of his thinking on this subject in a short book, Occupations and organizations in American society: The organization-dominated man?. Zald received his PhD from Michigan in 1961, so this book reflects his thinking and state of development during the first decade of his professional career.

As we examine the triple spectra of gigantic bureaucracy, of everyone a professional, and of guild-like professional associations, we must keep in mind that technology, client demands, economics, and politics are the underlying forces that shape the world of work. These basic factors not only shape professional autonomy and prestige, but also reshape organizations. Obviously, there is some truth in the image of man as organization-dominated, but American society is too diverse and complex to be summed up by this aphorism. (3)

One thing that this passage calls out to me is an emphasis on fluidity and heterogeneity in basic social institutions.  There are pulls and pushes that change institutions and practices over time, and they work through the activities of various of the actors involved in the system at a given time. This has a lot in common with the very recent ideas about “strategic fields” that McAdam and Fligstein have been developing in A Theory of Fields (link).

Much of the focus of Zald’s book is on the professions — the ways in which professions are shaped and propelled by a variety of forces within a given political economy. His work on this question pre-dates Andrew Abbott’s The System of Professions: An Essay on the Division of Expert Labor by seventeen years (link); and though it isn’t as detailed a study as Abbott’s, it certainly contains many interesting insights into the topic.

Here is how Zald distinguishes between “profession” and “occupation”:

Professions tend to be among the more prestigious occupations; they are commonly viewed as based on a relatively abstract body of knowledge. Whether a profession is based on a scientific discipline or not, usually, it is based on a fairly extensive abstract body of principles and practices that require a long training period to master. (17)

In Table 2 Zald provides a list of examples of “established” and “in process” professions, and he provides a set of important benchmarks in the development of a profession:

  • Established: accounting, architecture, civil engineering, dentistry, law, medicine
  • In process: nursing, optometry, pharmacy teaching, social work, veterinary medicine
  • Doubtful: advertising, funeral direction

Here are the benchmarks that he singles out as important markers in the emergence of a profession: date of becoming a full-time occupation, first training school, first university school first local professional association, first national professional association, first state license law, first code of ethics.  And he expects these to occur in roughly the chronological order in which they are listed.

It is worth reflecting on the conjunction that Zald brings together in this short book: occupation and organization. The connection between these two concepts isn’t entirely obvious; one has to do with specializations of work, and the other has to do with the social systems through which activities and work are conducted. So why are they conjoined here?

It seems that Zald has a somewhat complex set of ideas in mind: the specialization of skill and technique is a prerequisite to more complex forms of organization; but likewise complex organizations are necessary in order to support the training and indoctrination that is associated with the development of individuals’ skills and knowledge. So each of these social factors presupposes the other. This reading of the conjunction seems to be born out here:

A system’s centralization or decentralization affects the growth and development of organizations and occupations for managing and coordinating the components of the system… As long as American society permits major product innovation and development to reflect consumer sovereignty, occupational and organizational change and growth will be closely linked to the taste buds of mass society. (15)

The other side of the conjunction is organization and bureaucracy. Zald doesn’t exactly say what he means by an organization; rather, he talks about larger and smaller organizations. But it is possible to piece together what he has in mind. An organization is an extended group of individuals dedicated to bringing about a specified set of outcomes: make and sell automobiles, provide emergency room services, distribute and sell groceries, collect tribute from dependent tribes, enforce religious injunctions throughout an extended territory, … 

For example, a band of pirates is a small organization with direct control and supervision exerted by the pirate captain and with a small list of specialized functions: scan the horizon for prey, load the ammunition, attack the ship, distribute the booty, keep order among the pirates. But as scale increases, more complex social mechanisms are needed to maintain coordination of behavior by subordinates and to maintain effective exercise of the purposes of the organization. A manufacturing company with multiple factories requires a variety of kinds of specialists: engineers, designers, factory managers, auditors, tax specialists, marketing specialists, sales representatives. These are specialized “occupations” within the organization; and they require specialized forms of oversight and control if the specialists are to be expected to carry out their functions in the interest of the “management” (the guiding purposes of the organization).

The idea of a principal-agent problem is central here: how does the central management ensure that its agents are performing their duties in ways that conform to the organization’s goals, rather than using their positions to improve their own interests?

Zald puts this set of organizational challenges in terms of a set of fundamental questions:

Even though each industry has developed its own techniques, almost all large-scale organizations in America have come to grips with problems created by the transformation of organizations and occupations described in the last two sections. (1) Given the increasing size and complexity of organization, how is authority to be delegated? (2) Once delegated, how are the units to be controlled? (3) given the increasing size of the labor force and the distance between top managers or owners and lower-level workers, how is labor to be harnessed by incentive, by organization, and by ideology? (4) Given the increasing complexity of occupations, the emergence specialized techniques and professions, with their own concepts of professional teases and procedures, how are professions to be harnessed to organizational goals? (51-52)

What is a bureaucracy? A bureaucracy is a particular kind of organization; but what kind? Zald gives a simple definition: “the archetypical bureaucracy controls through rules and hierarchical supervision” (36). So codified rules and a well defined authority structure are key components of a bureaucracy.

It would appear that these features make innovation and creative responses to changing circumstances more difficult than they would be in a more opportunistic and fluid kind of organization. A more cellular form of organization, with an alignment of goals and values across cells but substantial local autonomy, seems to be one that has more potential for rapid adaptation to changing circumstances.

We can try to apply these ideas in unexpected places. Were the Black Panthers a bureaucracy, with central management in Oakland and a set of inspectors and enforcers who visited the “franchises” in Chicago or Detroit? Or were they more of a viral organization, with a loose set of shared goals but great diversity of activity, local organization, and effectiveness? What about Al Qaeda or the Taliban — are these organizations “bureaucracies”? And what about the Occupy Movement?

Here is a history of the dissemination of the Black Panther Party that provides a basis for answering the first of these questions; Judson Jeffries, On the Ground: The Black Panther Party in Communities across America. And much of the treatment Jeffries offers suggests the bureaucratic interpretation; for example —

In response to a critical letter sent in 1969 allegedly from a Detroit Panther and forwarded through Chicago, Landon B. Williams and Rory Hithe were dispatched to Detroit from the Oakland office to investigate a laundry list of allegations involving money and food coming up short, papers not being sold, money not being sent to the West Coast, and the disintegration of the NCCF into a dysfunctional and cliquish club of petty jealousies and dissension. Inspection tours from Oakland were not unheard of; in fact, Raymond “Masai” Hewitt and Donald Cox were reported to have visited numerous branches across the country as part of a regular inspection of each branch’s books and operations. Nonetheless, the arrival of Williams and Hithe was understood to be different, as the letter itself seems to have broken the protocol and chain of command whereby communication with Oakland was routinely done by phone and then only by two authorized officers — the communications secretary and defense captain. Moreover, Charlie Diggs Jr., for one, recognized the inspectors as what he referred to as “the goon squad from California” who were not there to examine the branch’s checkbook but to take the locals out into the alley and break their legs. “These guys you have to be scared of,” Berry remembered Diggs saying, “because that is why they sent them, if anything goes wrong in the chapter …. These guys come from out of town and wax you; they take care of you.” (156-57)

This passage captures many of the aspects of an organization that Zald highlighted — the need for control of the agents by the principals, the enforcement of rules of behavior within the organization, the need for supervision and oversight, and the need for internal processes of punishment for infractions. So it seems fair to say that the Black Panther Party of the 1970s was indeed an aspiring bureaucracy — paradoxical as that sounds. Here is another interesting collection on the history of the BPP; Yohuru Williams and Jama Lazerow, Liberated Territory: Untold Local Perspectives on the Black Panther Party.

What makes universities better?

Universities are large, complex organizations that have multiple goals — educating undergraduates, training graduate students, facilitating and expanding research activities, serving various communities. Each of these activities depends on complex contributions by very smart faculty and administrators, often in a highly decentralized way, and each can be more or less successful. The individuals involved are generally motivated to do the best work they can do. But the organization and its leaders have a responsibility to take steps to improve the quality and effectiveness of the results.

So the question here is this: what kinds of actions and strategies can university leaders take to help their universities to improve in performance with respect to the fundamental components of academic quality that they value? (Arthur Padilla’s ACE publication, Portraits in Leadership: Six Extraordinary University Presidents, raises some of these issues.)

We might ask, to begin, what the dimensions of quality are for a university. I would highlight at least three:

  • providing successful and effective education to undergraduate and graduate students (which means that on average, students who study at the university improve their intellectual and moral abilities over time); 
  • successful cultivation of high-quality research by faculty (which means increasing the flow of published and funded research results with measurable impact in both academic and non-academic spheres); 
  • contributing to the improvement of quality of life for the communities served by the university (region, state, city, nation). 

 We might add to these core characteristics several others: creating an inclusive environment for working and learning; creating a high sense of morale and shared aspirations for students, faculty, and staff; maintaining and increasing academic quality while reducing costs.

These high-level goals of a university cannot usually be measured directly. Therefore we need to have some set of factors that are closely related to the quality goals but that can be observed empirically. Quality measures in higher education are varied, but they include things like these: graduation rates, levels of Federal research funding, percent of Pell-eligible students, number of citations to faculty-authored research, number of patents issued, number of members of the national academies on the faculty, … There are steps that a university administrator or dean can take to influence any one of these variables; but all interventions require resources, which means that choosing to increase the graduation rate may mean not taking steps to increase the number of patents. More fundamentally, fine-tuning one part of the university’s processes may actually interfere with the workings of its other processes.

There are two large philosophies that might drive the idea of increasing the quality of a university over time. One is the philosophy of continuous improvement. Here the idea is that the university is already functioning at a certain level of effectiveness in a range of activities; the imperative is to increase the effectiveness, quality, and productivity of all of those areas, giving primary emphasis to those that have the greatest impact on core values. This is a philosophy of gradual improvement and refinement.

The second philosophy is one that pays attention to the need and opportunity that sometimes exist for radical change in product and process for the organization. This philosophy downplays the idea of continuous improvement and argues instead for “punctuated equilibrium” — the need to sometimes take actions that fundamentally change the nature of the enterprise in some way. The idea that instruction should be substantially more oriented towards online courses would reflect this philosophy; it corresponds to those critics of higher education who believe that universities are like film-based camera companies in the 1990s facing the sudden appearance of digital photography. Either they adapt quickly to a new disruptive technology, or they fail catastrophically in the market.

A philosophy of continuous improvement would not have served the buggy whip industry well in the 1910s; whereas a philosophy of punctuated equilibrium doesn’t seem to be appropriate for shipping industry at this point. (It was appropriate at the time when containerized shipping was a hypothetical possibility!) So there is no reason to believe that either “gradualism” or “punctualism” is always the best option for an organization.

So what kinds of changes are needed and feasible for universities in today’s environment? This will be a disappointing answer, I’m sure, but I think the answer is: some of both. There are many improvements that can be identified in the ways that universities handle the various components of their missions. Improving retention and graduation rates is a case in point; there are many new strategies that could be incorporated to help students be more successful in their academic progress. Peer-based tutoring is one example; online tools for checking progress towards graduation is another; and improving the quality and reach of the academic advising system is a third. But it is also true that there are opportunities for discontinuous improvement in the existing university. For example, could the resource of a well-developed online course with rich media materials and evaluation systems allow the business faculty member to effectively teach accounting to twice as many students? Could a high-level learning goal for students (e.g. “ability to sort out contextual factors in a complex problem”) be better achieved by closer coordination among clusters of faculty and courses rather than by a general education curriculum that assigns one goal to one course? Is the traditional Ph.D. dissertation an antiquated ritual rather than a crucial learning opportunity for advanced students?

(Vincent Tinto has a short but interesting article on “Universities as Learning Organizations” here. Here is a piece by Judith White and Rita Weathersby on a similar topic.)

Organizations and the Chicago School

Andrew Abbott is a fascinating sociologist, and he is an expert on the Chicago School. His book Department and Discipline: Chicago Sociology at One Hundred provides an excellent analytical discussion of the theories and people of the school. So his contribution to The Oxford Handbook of Sociology and Organization Studies: Classical Foundations, “Organizations and the Chicago School,” on the place of organizations in the research tradition of the Chicago School is doubly interesting. The piece is posted here on Abbott’s website.

The piece is interesting for a number of reasons. First, it gives Abbott an opportunity to articulate once again what he thinks is most distinctive about the Chicago School approach to the study of society. Second, it allows the reader to infer how Abbott himself thinks “organizations” ought to be studied. And third, it presents a very interesting example of how to do the sociology of a knowledge field.

In several ways Abbott thinks that the perspectives offered on “organization” by the Chicago School sociologists are more insightful than those provided by the other more mainstream approaches in US sociology — “the Harvard based human relations school, the Weberian tradition descending from Parsons through Merton at Columbia, and the more formal and economic approach associated with March and Simon at Carnegie Tech” (399).

This additional insight starts with the grammar of organizations. For the Chicago School, organization is more verb-like, a process, ‘the organizing of social life’, whereas for the other main traditions organization is a noun, a concept that refers to a static thing to be studied.  For Abbott this is an important differentiating factor in the Chicago School approach generally: a tendency to look at social affairs as an active and dynamic set of processes rather than a settled and stable set of durable structures.  “‘The world’, in the ringing phrase of the Chicagoans’ philosopher colleagues George Herbert Mead, ‘is a world of events'” (401).

Abbott argues that there are complicated reasons why the Chicago School is not regarded as having developed a significant sociological theory of organizations.  One part of the story is the fact of inter-paradigm politics. Influential figures in the human relations school at Harvard made deliberate efforts to minimize the Chicagoans’ involvement in organizational studies (402).  Another part of the story is the fact that the Chicago School contribution to this field of study took place largely through dissertations rather than through large theoretical treatises.  And finally, there is the historical fact that American organizations underwent profound change from 1920 to 1960, both nationally and in Chicago.  The super-sized organizations of giant corporations, federal bureaucracy, and the army emerged in the years of World War II; whereas much of the research agenda of the Chicago School was set in the earlier part of the twentieth century. 

Nonetheless, Abbott argues here that there is indeed a broad body of work within the Chicago School on the subject of organizations, and it is of high quality.  It presents an alternative way of engaging in a sociology of organizations.  He picks out three themes around which to organize the Chicago research that he studies here:

(1) static aspects of the interiors of organizations, (2) diachronic aspects of single organizations, and (3) aspects of relations among organizations. (402)

Referring to Charles Richmond Henderson, he writes:

Like most of the Chicago sociologists, Henderson located what we now call organizations as one among many types of ‘institutions’. The concept of institutions was general at the time, denoting any body of social behavior or structure dedicated to carrying out what we would now call a function of society. (404)

Abbott finds that Henderson and Small supervised thirty-one dissertations, and nine of these were relevant to the topic of organizations: stockyards, churches, the steel industry, and employment agencies, for example.  Abbott finds that “the focus is much more on dynamics: institutionalization, ossification, change, and evolution” (407). Another feature of the Chicago “sociological imagination” was an abiding interest in peripheral social roles — delinquents, hobos, criminals, and taxi dancers (412). This conforms to the Chicago penchant for studying the social particular and the Chicago tendency towards progressive causes. But he also finds that the division of academic labor shifted during the 1930s, with the study of organizations going to the political economists rather than the sociologists (409).

One of the sociologists whom he cites is Paul Goalby Cressey, whose sociology dissertation centered on the organization of the “taxi dance hall.” The dissertation was published as The Taxi-Dance Hall: A Sociological Study in Commercialized Recreation and City Life.  Abbott praises this work:

The other Chicago works on entertainment institutions — Paul Cressey’s The Taxi-Dance Hall (1932) and Walter Reckless’s Vice in Chicago (1933) — are first-rate organizational studies.  Cressey’s analysis of types of dance halls touches on the resource dependencies of the dance halls, the evolution of different types of halls, competition and specialization among them, and, of course, locational patterns. (413)

Another work that he admires is a dissertation by Edwin Thrasher:

A final work of this type is Edwin Thrasher’s The Gang (1927), probably the ultimate Chicago statement of organizations as things that are always in transition. For Thrasher, gangs are rooted in communities, and thus are evidence of ongoing ‘social organization’. But for us, Thrasher’s study provides clear evidence that the organizations Chicagoans found most interesting were those that were the most transient. (414).

And here is another study that Abbott admires:

Even more extraordinary, however, is Ernest Shideler’s ‘The Chain Store’ (1927). This is a comprehensive organizational analysis of the emergence of a new organizational form…. This is organizational sociology of a comprehensiveness and theoretical sophistication that would not be seen again for another fifty years. (415)

World War II brought about a set of changes in organizational America that made the Chicago School approach less appealing to the mainstream. “The conflictual, processual, local theories of the Chicago School made little sense in a world now conceived as grand, unified, and even static, a huge mechanism for steady expansion in a non-ideological, managed world” (416).  But this stability was short-lived, and Abbott believes that the Chicago approach is once again worth studying. Here is how Abbott concludes:

What then are the lessons contemporary organization theory can learn from the Chicago organizations tradition and its vicissitudes? The most important lesson is that there is no necessary reason for seeing the social world as a world of organizations. The Chicago School’s sublimation of organizations into an epiphenomenon of social processes reminds us that to see the social world in terms of organizational entities — as the human relations school did — is to take a quite historically specific view, one anchored firmly in the worldwide importance of large, stable bureaucratic structures in the years from about 1925 to 1975.

But that epoch of stable, fixed social structures is over, Abbott seems to say, and the fluid, shifting, and event-ful social world that we now confront is better studied using the approaches of the Chicago School. We now face “a world of rapid turnover and change in organizations, a world of continuous organizational restructurings and financial prestidigitation, of networks and arm’s length relationships, a world in which the employment and production structures that were laboriously built by scientific management and human relations have been deconstructed through outsourcing and offshoring” (419).

And for that more fluid social world, the Chicago School approach is very well prepared.

Do organizations have causal powers?

An organization is a meso-level social structure. It is a structured group of individuals, often hierarchically organized, pursuing a relatively clearly defined set of tasks.  In the abstract, it is a set of rules and procedures that regulate and motive the behavior of the individuals who function within the organization.  There are also a set of informal practices within an organization that are not codified that have significant effects on the functioning of the organization (for example, the coffee room as a medium of informal communication).  Some of those individuals have responsibilities of oversight, which is a primary way in which the abstract rules of the organization are transformed into concrete patterns of activity by other individuals. Another behavioral characteristic of an organization is the set of incentives and rewards that it creates for participants in the organization. Often the incentives that exist were planned and designed to have specific effects on behavior of participants; by offering rewards for behaviors X, Y, Z, the organization is expected to produce a lot of X, Y, and Z. Sometimes, though, the incentives are unintended, created perhaps by the intersection of two rules of operation that lead to a perverse incentive leading to W. For example: a farm supervisor may ask peach pickers to discard the bruised peaches rather than placing them in the basket to be weighed. But if the laborers’ salaries are determined solely by the weight of the baskets they present for weighing, they will have an incentive to include the bruised peaches (at the bottom!).

Examples of organizations include things like these:

  • the Atlanta police department
  • a collective farm in Sichuan in 1965
  • the maintenance and operations staff of a nuclear power plant
  • a large investment bank on Wall Street
  • Certus Corporation (discoverer of the PCR process)
  • the land value assessment process in late Imperial China

The organization consists of a number of things:

  • a set of procedures for how to handle specific kinds of tasks
  • a set of people with skills and specific roles
  • a set of incentives and rewards to induce participants to carry out their roles effectively and diligently
  • a set of accountability processes permitting supervision and assessment of performance by individuals within the organization
  • an “executive” function with the power to refine / revise / improve the rules so as to bring about overall better performance

Let’s take the nuclear power plant staff as an example. The tasks of the organization are to control the complex technology and its instruments over an extended time; to conduct inspections of the physical infrastructure of the plant to discover failures before they occur; to conduct routine maintenance of machines and other physical systems; to respond quickly to failures, both large and small; and to sometimes conduct major upgrades on the hardware of the system. We may imagine that there are detailed, written procedures for each of these activities, as well as procedures for action during times of malfunction or breakdown. The people of the plant represent a range of specialized skills and specialized tasks. Wainwrights maintain and repair machinery; computer technicians maintain computer systems; nuclear technicians oversee the measured functioning of the system (pressures, temperatures, power production); safety workers inspect various system; and supervisors assign tasks and monitor performance.

Failures of the system arise for several different kinds of reasons: technical failure (a device fails for unexpected technical reasons, such as a faulty weld); operator failure (an operator disregards or misinterprets a pressure warning, and a pipe explodes before corrective action is taken); training failure (staff are technically or operationally unprepared for performing their tasks routinely or in exceptional circumstances); system failure (two or more sub-systems function as designed, but in an unusual circumstance may interact in such a way as to bring about an explosion, a computer crash, or a release of energy or heat); supervisory failure (procedures were good but supervisors permitted deviation from the procedures); venality failure (individuals in a position to control purchasing decisions authorize bad contracts for faulty materials for their personal profit).

The idea of a principal-agent problem is highly relevant within organizations, at every level. The executive expects the supervisor to faithfully perform his/her tasks of supervision. But since the executive does not directly monitor the performance of the supervisor, it is possible for the supervisor to shirk his/her duties and permit faulty performance by those he supervises. Likewise, the supervisor expects that the operator will continue to monitor and control the machine throughout the day; but it is possible for the operator to keep a solitaire window open on the screen. Each level of accountability, then, requires both formal expectations and a basis for trust in the good faith of the participants in the organization.

Now we are in a position to address the central question here: do organizations have causal powers? It seems to me that the answer is yes, in fairly specific ways. First, the rules and procedures of the organization may themselves have behavioral consequences that lead consistently to a certain kind of outcome.

Second, different organizational forms may be more or less efficient at performing their tasks, leading to consequences for the people and higher-level organizations that are depending on them. For example, two tax-collection systems may be designed for the same goal — to collect 10% of the grain produced everywhere in the kingdom. If one system is 75% successful in this task and the other is 50% successful, the state depending on the second system will be starved for resources.

Third, the discrepancy between what the rules require of participants and what the participants actually do may have consequences for the outputs of the organization. Police department regulations may require that each piece of physical evidence is separately bagged and catalogued with appropriate information about its collection. If police operatives are careless in the cataloguing of evidence it may be more difficult to convict the accused; this may lead to a rising disregard for the likelihood of conviction and a rise in the crime rate. Corruption (venal failure to perform one’s tasks faithfully) may lead to large consequences: the company is less profitable, the city is discredited to its citizens, the Church is delegitimated by the self-interested behavior of its clergy.

Fourth, the specific ways in which incentives, sanctions, and supervision are implemented differentiate across organizations. We may find that organizations with supervision system X are on average more productive or more effective than those with system Y.

Fifth, the organization has causal powers with respect to the behavior of the individuals involved in the organization. By presenting its rules, sanctions, and rewards to its participants, it changes their behavior in specific ways. Google and Apple have organized their internal procedures and rewards in such a way as to encourage creativeness, teamwork, and confidentiality. These organizations look quite different in their functioning and their products from a steel company or a shoe company.

This means two things. First, we can say with some confidence that the way an organization is structured makes a difference to its performance; this is a causal power all by itself. And second, we may be able to discover that there are broad characteristics that differentiate organizational types, and it may turn out that these distinct types also have different performance characteristics. We might discover, for example, that one system of oversight and employee motivation is significantly more likely to permit theft and corrupt behavior by its agents than another. In that case, we might say that these two systems differ in their propensities for generating corrupt behavior. (This is an argument that Robert Klitgaard makes in Controlling Corruption.)

So far we haven’t mentioned the familiar subject of “microfoundations” at all; we have considered an organization as a complex social entity. It is easy to specify the microfoundations of the causal powers we have identified. The organization’s performance is determined by the behaviors of the individuals who fall within it, and the aggregate individual behaviors are explained by the rules and procedures embodied in the organization. So the causal powers having to do with efficiency, effectiveness, and corruptibility can be disaggregated into the incentives and behaviors of typical individuals. But here is the key point: we don’t need to carry out this disaggregation when we want to invoke statements about the causal characteristics of organizations in explanations of more complex social processes. This is a case illustrating the point of relative explanatory autonomy developed in a prior post, and it also illustrates the point that David Elder-Vass makes in The Causal Power of Social Structures: Emergence, Structure and Agency.

These observations lay a basis for concluding that meso-level social entities have causal powers that can legitimately be invoked in social explanations.  Significantly, there are clear and convincing examples of sociological explanations that take the causal powers of organizations as fundamental to their explanations of important social outcomes — for example, technology failure (Charles Perrow,Normal Accidents: Living with High-Risk Technologieslink), corruption (Robert Klitgaard, Controlling Corruption), and the use of common property resources (Elinor Ostrom, Governing the Commons: The Evolution of Institutions for Collective Action). 

%d bloggers like this: