Book Reviews: Sociotechnical Shelf

A couple of recent books, in the growing space of “sociotechnical” studies.  The first is excellent, the second has a lot of problems.

Writing On the Wall: Social Media-The First 2,000 Years by Tom Standage (Bloomsbury, 2013)

Standage writes a nice review of the history of “social media”, based on the observation of the essential behaviors associated with shared, collective, mediated conversations. The theme of the book is that today’s social media—broadly defined to include most digital media—have historical precedents. Obviously, part of the point is that digital technologies did not create the motivation or characteristics of social networks, distributed conversations, or collective intelligence. These exist as part of human cultures, and have been implemented with different technologies including hand written papyrus, printed papers and books, telegraph, radio, TV—and the Internet.

He reviews historic cases, including the distribution and sharing of written texts in the Roman Empire, the Protestant Reformation, seventeenth century coffeehouses, and during the American and French Revolutions. In the nineteenth century, steam powered printing presses + railroads + telegraph created large scale, one way, mass media—no longer social media.

Part of the point is that many of the “trends” and anxieties discussed today were seen in earlier eras—complaints about time wasting and/or stupid-making of social media, the unpredictable explosive effects of “viral” stories, the messy ramifications of media freeforalls. Inventions like moveable type, the telegraph, and radio all spread explosively, upending economies and political orders. As he says in the epilog, “History retweets itself”.

Standage discusses the use of the Internet and social media in China, but surprisingly omitted its role in the fall of the Soviet Union and the opening of China to the world. Even before Mosaic and smart phones, the Internet, along with phones, fax, and personal computer, obliterated distance and boarders, and, as per the Internet philosophy, “routed around censorship”. Arguably, the Internet was “the war winning weapon” of the Cold War, and may have helped prevent (or at least push back) the pointless nuclear war that seemed inevitable fifty years ago. (You’re welcome.)

Standage provides a pretty solid set of historical cases that help understand the impact of social media. Neither necessary nor sufficient for a revolution, it can pour fuel on a fire, synchronizing and coordinating action. Of course, this is exactly the same dynamic as a lynch mob, not to mention government crackdowns on minorities.

This review is appearing in a blog, which is one of the current permutations of social media, so obviously I don’t consider the medium to be pointless (even if the blog is often “write only”). As I said to a new blogger, after her first “hello, world” post in her own blog: “you are now exactly the same height at the New York Times or anyone else”. Next to learning to read, what could be more liberating?

Uncharted: Big Data as a Lens on Human Culture by Erez Aiden and Jean-Baptiste Michel (Riverhead Books, 2013)

This book is a non-technical summary of Aiden and Michel’s work developing and using the Ngram Viewer from Google Books (https://books.google.com/ngrams).

They call Google books “long data”—not only “big” but a long time covered (centuries). This collection is a large sample of written words, and the Ngram viewer is  therefore a dataset of word  use over time.

A&M have mined this collection to discover trends in word use over time.  They interpret these trends as representing important underlying trends in language and culture.  They call it “culturnomics”.

But what is the significance of this work?  What does it really mean?  Much less than they want to claim, that’s for sure.

Their canonical artifact use a comic book chart, plotting two or more ngrams over time, with a caption like “chocolate vs. vanilla”. The resulting graphs show the frequency of the terms for a century or more, which they interpret to discover cultural change. This comes from Randal Munroe’s xkcd Web comic, and is available to play with from Google.

These graphs are often suggestive, and in some cases counterintuitive. But mostly, they are pretty obvious. “Einstein” was unmentioned before his birth, and is now very often mentioned. Mentions or “Einstein” now outnumber mentions of “Newton”. So on. Fine so far.

This entire enterprise hinges on how we interpret these graphs, which depends on what the data is. The authors make clear how the data was assembled, though we’ll have to look at some technical papers to understand the methods they used to “clean” the data. (I have done such work myself, so I both respect the significance and non-trivial nature of this effort.) Of course, this dataset is so large that it is beyond human comprehension: it is impossible to check it by hand.

The authors are careful to state the limits of the sample: they believe that it is built from something like 4% of all the books published over the last few centuries. This is a large but hardly complete sample, even of books. And we have no way to know what biases may have sneaked in simply through the survival process. How does a book published 200 years ago come to be in Google books? Obviously, it must be published, copies must survive, generally it must have been collected and cataloged by a library or other collector, and so on. It is clear that this sample does not represent any entire “culture”, though it may represent the politically dominant intellectual culture of a period.

This sampling bias is well known, the authors talk about it in Chapter 7. Books do not represent the whole culture, or the whole language. This “4%” visible in their “lens” is more like .0004% of actual language use. In short, it is not clear that “culturomics” is about “culture” at all.

The authors generally skate over a second aspect of interpreting these graphs: what do the terms mean? Specifically, what did the author mean to say at the time? This data set averages over hundreds of authors and hundreds of years. The entire point of ngram analysis is to remove much of the context, which is how we discover the meaning of the term. In the worst case, the graph could be plotting a term than has multiple, inconsistent meanings, or drastically different meanings at different times.

There is even an example in the appendix on page 221: they plot the use of the term “men” versus “women” from 1800 to now. The plot appears to show far more mentions of “men” than “women”, up to the 1980s, when the curves cross over. Aside from the fact that it is difficult to interpret “mentions” of these terms at all (What is significant about writing something about “men”?), and the known issues about who was writing (far more men were published than women until about the 1980s….), there is an even more basic problem: the term had a different meaning before about 1970.

In American English, “Men” was considered the same term as “people” or “humans” (i.e., including women). In 1850, some of the authors were referring to males, and some to all humans.   But we now don’t accept that usage, and often write “men and women” or just “people”, where we might have written “men” fifty years ago. The graph on page 221 may capture that, although it misses the other words “people”, the use of “men and women”, etc..

Bottom line: this graph is meaningless. Or at least, we have no idea what it might mean, if it means anything at all.

Similar pitfalls exist in many of their terms, and it can be very difficult to know. What did the terms “war” and “peace” mean to people in the nineteenth century? What do “correlation” and “causation” mean to anyone, at any time? I don’t know, but they mean different things to different people and the meaning wasn’t the same in 1850 as it is today, that is for sure.

The authors are clearly aware of this issue. Most of the cases they present are selected to minimize this general problem. In particular, the use personal names in many cases. While “Newton” and “Einstein” have gained a variety of meanings besides reference to the individuals, those meanings are generally indirect references to the person (“Hey, Einstein—don’t forget to turn off the lights when you leave.”). Similarly, terms with a specific reference, such as “Tiananmen Square”, or “DNA”, feel safer than, say, “Man” vs “Nature” (p. 224).

On the other hand, they obliviously use this methodology to study “collective learning”, looking at use of words for famous inventions. They interpret the graphs (which plot word frequency in printed books) as implying something about the spread of technology. And worse, they call this “collective learning”. Really?

You know this whole thing is bogus when they discover that the generic terms like “revolver” take longer to peak than product names like “Walkman”. Well, duh! A word with an advertising campaign spreads quicker than one without. Maybe society isn’t learning faster and faster (p. 173), maybe we invented advertising.

To be fair, the actual search tool supports much more complex selections than the simple terms presented throughout the book; including position in phrases, parts of speech, and other shallow context. It is possible, therefore, to ask more and more detailed questions about the same shaky dataset. These queries will surely increase understanding of the dataset, but not necessarily anything else. Considering that the dataset is too large for human comprehension, we are talking about finer and finer searches of … we’re not sure what.

I have blogged about aspects digital tools for cultural heritage preservation, so don’t take me as against the enterprise. But I must say that, on a broader level, many of their studies are uninteresting to me. The use of a comic book graphic to deliver results should have been a give away that this isn’t particularly deep stuff anyway.

A&M “discover” censorship, which is seldom difficult to know about. They “discover” the impact of events such as wars, or the effect of important achievements. The chapter on “collective learning” is absurd. They have a whole chapter on fame, for goodness sake (which, by the way, certainly doesn’t mean the same thing everywhere or at different times).

Why? Who cares?

In every case, this analysis yields shallow results that require much deeper examination to interpret. As in the case of Google’s Flu Trends , this may give a quick look that suggests interesting directions to look. But, as in the case of flu surveillance, it is hard to say whether this is overall helpful or not. When it is wrong—as in the case of the GFT – it can be harmful.

Certainly, anyone who believes that “culturomics” is a particularly valid view of human culture or behavior is being deceived. It may be interesting (though not to me), but it isn’t necessarily a view of “culture”, and may not be a view of anything real.

I also note that the call for massive funding for digitization (in the name of “humanities” and “culture”) makes me wonder what planet they live on. All over the globe, research is being defunded, except through the patronage of oligarchs. (I’m sure you could “discover” this trend somewhere in Google’s big data.)  It’s kind of pointless to call for gigantic projects of this kind, when we are struggling to keep our schools, museums, and libraries open at all.

Sigh.

One thought on “Book Reviews: Sociotechnical Shelf”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

A personal blog.

%d bloggers like this: