Social Physics: How Good Ideas Spread – The Lessons From A New Science by Alex Pentland (The Penguin Press, 2014)
Sensor rich mobile devices meet Big Data. The result: awesome new social science. Also, opportunity for massive hubris.
Sandy Pentland has been doing fascinating research for decades now, from early wearable computing through his previous book, Honest Signals (2008). He updates this work with even more massive datasets. Really, really massive. And really, really personal.
Working with these massive collections of personal data as he does, Pentland is deeply concerned with data privacy issues, and has long been active in promulgation of what he calls a New Deal for Data. He wants people to shared data in public “commons” that benefit everyone. The basic idea is to get things organized so individuals own their own data, and can opt in to share, and have safe ways to share.
Pentland has a clear vision of how this technology should be used to make the world better. His benign goals are admirable, but neither the technology nor its possible uses are beyond criticism.
Let me dedicate this commentary to the memory of my father, who would have loved to get his hands and mind on this stuff. A millions times better social science data! A thousand times better theory! Wow! JEM was born 50 years too early, he would have loved this stuff.
Social Physics “brings together big data about human behavior and social science theory” and breaks new ground in social learning and many social behaviors. “Big data” means really big: the book is accompanied by 5 large longitudinal datasets, including one that tracks dozens of variables 5 months for the entire nation of Ivory Coast.
In many ways, his studies are mainstream social science, both the theory and designs are similar to work done for a century now. The difference is, of course, now they are able to collect gazooga-bytes of fine-grained behavioral data using phone traces and other digital records. Who talks to who, who phones and texts, where people go and meet each other: these are very familiar data to social scientists, but they have been difficult or impossible to obtain from the field until recently. Low cost sensors and wireless networks have created a bold new world for behavioral research.
The resulting data analyses provide a “god’s-eye view” (p. 30) of whole social networks, from which the “god” may perceive social isolation, herd behavior, and other phenomena not visible to us mere mortals. The mighty “gods” may also choose to intervene, to “tune” and otherwise fix the behavior of the masses. If this stuff didn’t work for real, these ambitions would be either comedy or tragedy. But it is real (see the appendices, web site, and cited research papers), so we are talking about technology here.
From this lofty vantage, it is apparent how much human behavior is socially learned and peer influenced, and how little is driven by individual reasoning or drives. Pentland concurs with the view that people are “collectively rational, not individually rational”. Well, I wouldn’t say people are “rational” at all, but it is clear that humans are largely “collectively driven”.
Several of the studies illustrate the importance of what they call “engagement”, repeated, personal contacts, which promotes social learning, cooperative behavior, and lot’s of other good things. This isn’t news to social scientists, but the ability to observe and understand these behaviors in vivo is something new. Pentland’s group also experiments with manipulations that use these personal networks to influence behavior. Notably, they find that incentives for a few of your friends to help you change your behavior may well have far more impact on your behavior than incentives for you personally.
Organizations and Groups
These concepts are applied in a variety of social settings including small groups, larger organizations, and whole cities. (“I think of organizations as a group of people sailing in a stream of ideas.” (p. 44)) The essential image of ideas flowing through social networks scales amazingly well. Their studies link this flow to “collective intelligence”, productivity, and other measures of overall success. In other words, attending to interpersonal communication patterns, can yield both understanding and control over much wider phenomena. Cool!
It is important to note that these effects are reported even though there is no obvious direct mechanism, or specific ideas flowing. For example, it is not obvious exactly what kinds of problem solving or decision making is done by workers in a large call center, but improvement in the “social physics” yielded improved productivity (faster call resolution). In other words, they would say we don’t have to prove that there is idea flow (or that it is important), we should assume there is, and try to optimize it. Their studies also show that it doesn’t matter what people are saying, it matters how and to who they talk.
It is interesting to see their critique of the “mechanical turk” model, which they find theoretically backward (no social interaction at all) and demonstrate their social network based approach can beat a mechanical turk like a drum (See Chapter 7 of Social Physics).
One general finding is the importance of face-to-face interactions, both within and across “teams”. Their data shows, over and over, that this is critical for productive, creative, and, I would say, happy, team work. This finding is especially salient given the trend toward remote work, epitomized in a pure form by Automattic (makers of WordPress), which has no offices at all. (See my earlier review of The Year Without Pants by Scott Berkun). However good WordPress is, Pentland’s studies would suggest that it would be even better if the teams had more face to face contact.
At the same time, it is interesting to consider another extreme, “all one team”, that Menlo exemplifies (see my earlier review of Joy, Inc. by Richard Sheridan}. The radical practices of this company force face to face interaction all the time, and also remix teams routinely forcing the kind of idea flow Pentland talks about. Their (really, really) open office certainly maximizes the “overheard” interactions, so say the least. The effect is basically everyone is a peer, and there is lots of communication among your peers. Pentland’s studies would say this should be highly creative and productive, unless there is a lack of diversity (“herding”) Menlo pushes to hire from diverse backgrounds and doesn’t punish new ideas, both of which should mitigate an “echo chamber” effect.
Both these organizations would make very interesting cases for Pentland’s sociometers.
Cities and Societies
Turning to larger scales, Pentland looks at better “management” of cities. “The proliferation of mobile phones makes it possible to leap beyond demographics to directly measure human behavior.” (p. 141) Fine grained behavior data enables, they show, reliable forecasts of transport, electricity use, and so on. (The “so on” includes spending and political leanings—which is way into “creepy” territory for me.)
“Cities are idea machines in the same way companies are idea machines.” (p. 156) OK, stop right there.
This may be true, but there are important ways in which cities are not “idea machines” and not like companies or families. If nothing else, cities are amorphous, ill-defined entities, with “membership” poorly defined. This matters a great deal, because his measures of success rely on operational definition of what and who is part of the city, and what activities matter most. (For example, at one point he counts population growth as a positive outcome for any city—which is certainly debatable.)
Apparently not concerned with the chaotic nature of cities, Pentland imagines that, for instance, transportation can be optimized to eliminate congestion. While I’m certain you can improve city traffic using fine-grained data, I have to think that, absent unimaginably dictatorial controls, the city is going to remain chaotic. I don’t care how good your data is, if you can’t intervene at a fine grain then you are only going to peck at the margins of the behavior.
Pentland also envisions “interventions”, deliberate manipulations of behavior at the level of a city. He starts from the misunderstanding that the privileging of elites is a side effect rather than the entire goal of many policies, assuming—despite the evidence—that cities are designed for all the inhabitants, rather than for the powerful.
So he envisions “Social Mobilization” (witch hunts?), “Tuning Social Networks”, e.g., to assure “optimal” diversity of ideas (Fox News certainly will and can resist this) , “Leveraging Social Engagement” (AKA neighborhood watches). As he notes, “The main barriers to achieving these goals are privacy concerns and the fact that we don’t yet have any consensus around the trade-offs between personal and social values.” (p.151) And how!
The Realm of the Philosopher Kings
There some seriously Orwellian language in this book. “Chapter 11. Design for Harmony”. The goal is to design society for people to “make correct decisions and develop useful behavioral norms.” (p. 203) A section title is “Summary: Promethian Fire” (p. 215) The use of the phrase “gods eye view” is telling phrase, too.
Uh oh. “Philosopher King” alert. Warning, Warning, Will Robinson!
My father firmly warned me about the pitfalls of academics naively applying their learned doctrines in the real world, which he referred all the way back to Plato’s Republic.
Of course, Pentland’s notion of a “Data Driven Society” needs a lot more work. For example, he describes a mapping app that “shows maps of poverty, infant mortality, crime rate, change in GDP, and other social indicators, updated daily and on a neighborhood-by-neighborhood basis.” This, he says, “can quickly [see] where government initiatives are working and where they are not.” (p. 184) Huh? What initiatives would be expected to change these things day-to-day, block-by-block? This makes little sense.
This tendency toward short term thinking is extremely problematic. If we focus on (measurable) instant impact, we will limit policy toolkits. It is notable that Pentland implies that social “crashes” are due to inability to respond quickly enough to events. Many of us believe that the opposite is true: short term, reactive behavior, rather than long term, adaptive strategies are the source of panics and crashes. On this point, the “data driven society” may be going the exact wrong direction.
Minute-by-minute data is not especially relevant for many important problems. For example, the growth of children takes place on the scale of years, the “result” comes out decades later. “Real time data” are not necessarily indicative of the ultimate outcomes for schools or parenting. Similarly, infrastructure work may create horrible headaches today, for potential benefits years from now. Zooming in on microscopic details gives us no insight into these policies, and may actually obscure the benefits.
It doesn’t take a genius to see how, as with any technology, these techniques could be used to create more economic instability, more conflict, more corruption, and more concentration of power. Who has the resources to exploit these techniques? The usual suspects; corporations, powerful elites, and wealthy individuals. Google. Amazon. NSA. FSB. PLA. ‘Nuff said.
As an engineer, I have to wonder what will happen when (not if) multiple, competing, manipulations are applied simultaneously. Pentland calls for a diversity approaches, but it is not obvious what happens when we are subject to dozens of competing “incentives”. If everyone is pulling the same “Social Physics” levers, what happens? If Coca Cola and Pepsi are each tweaking the same social networks to boost their own sales, how does that work? And what about the simultaneous competition from advocates for drinking less sugary beverages? I suspect that there will be “winners” based on fashion, luck, and power—not rational self determination and communitarian collaboration.
The most ironic feature of this book is how much it sounds like the product of the MIT echo chamber. The problems and solutions are all from the world of—wait for it—MIT researchers. Pentland’s great success and good fortune has given him access to a large number of extremely bright people. But how diverse is his social network? Perhaps he might benefit from some time out here in flyover country.