Three-Toed Sloth

Statistics 36-350: Data Mining (Fall 2008)

Since class begins Monday, this is a good time for the public website to make its appearance. As before, lecture notes will also be posted here; you can use the RSS feed for this entry to keep track of them.

1. Introduction to the course (25 August)
2. Information retrieval and similarity searching (25 August)

Corrupting the Young; Enigmas of Chance

Fire, Metal, Form

For almost as long as I can remember — at least since seeing the ancient Chinese bronzes at the Sackler as a teenager — I've wanted to learn metal-casting. To my great good fortune, Carly Jean Parrish and Ed Parrish (of hot metal happening) offered a class on iron-casting for beginners this summer at the Pittsburgh Center for the Arts near my house, which I've been taking. This Saturday the class cast our molds. (Here's mine, before and after.) It was one of the most awesome things I've ever seen or been part of; I took a few pictures, but really you needed to be there.

Fortunately, our teachers will be performing on Saturday the 23rd at the PCA, as part of the closing of the biennial exhibit; you will not find a better entertainment value in the city that evening.

(Of course, lots of the things which went towards making it an incredible experience as an occasional spectacle — the heat of working under layers of protective clothing, the noise of the furnace, the hammers, the yelling, the flames, the muscle-tension of carrying and controlling a big bucket of molten metal, the adrenaline shock of noticing that your glove doesn't quite meet your sleeve around your wrist and there are sparks going everywhere — are also the things which go towards making iron-working a very unpleasant job. As for doing it twelve hours a day, six days a week, with periodic 24-hour shifts and no insurance, the way Mr. Carnegie and co. used to run the plants — well, there was a reason a visitor called old Pittsburgh "hell with the lid off", and they had to keep the unions down by shooting people.)

Self-Centered

Books to Read While the Algae Grow in Your Fur, July 2008

Nicolò Cesa-Bianchi and Gábor Lugosi, Prediction, Learning, and Games

A wonderful synthesis of the literature on competitive, individual-sequence forecasting with expert advice. That is, the problems considered are all variants on a situation where you need to make a prediction about the future (or more generally take an action whose consequences will only be revealed in the future), have access to a range of "experts" or forecasting algorithms, and want to ensure that, no matter what actually happens, your performance will be close to that of the best expert. This is thus a study of sequential decision-making under uncertainty without probability. Often, but not always, the solution lies in taking weighted averages of the experts, giving more weight to those which have done well in the past. This works not because past performance provides any kind of inductive evidence of future success, but merely because it keeps your predictions from drifting too far from what is, in fact, working. (Perversely, many of the proofs rely on probabilistic arguments, but they don't make probabilistic assumptions.) Of course, it may be that even the best expert is very bad, but the possibility of improving on the experts is not really considered, though it's certainly possible (at least with convex loss functions).

Anyone at all interested in machine learning, forecasting, information, game theory, or decision-making under uncertainty needs to read this. It may also be useful to epistemologists (cf.).

Max Brooks, World War Z: An Oral History of the Zombie War

An oral history (a la Studs Terkel) of the early 21st-century global struggle against the zombie apocalypse (a la George Romero). This is a happy choice of form, because it lets him tell the story of a global disaster from many viewpoints, without taking the space which would be required in a conventional cast-of-thousands novel. Also, he gets to tell lots of variously creepy, horrifying, thrilling, and/or moving stories this way.

Query: does this qualify as a "modern epic", sensu Moretti?

Marc Van De Mieroop, The Ancient Mesopotamian City

As every school-child knows, "history begins at Sumer", with the first cities and the first writing. This book is the only accessible synoptic view of the cities of ancient Mesopotamia as such.

After opening by quoting some jaw-droppingly ignorant (and recent) remarks by classicists on how there were no real cities before the Greeks, Van De Mieroop describes the geographic scene, and lays out some of the limitations on our evidence — peculiarities in what scribes thought worth recording, and other peculiarities in what archaeologists have thought worth excavating. Next he considers theories of the origins of cities in Mesopotamia, a peculiarly difficult problem since there were no other cities to learn from or be influenced by. He favors the idea that they originated around the temples, which acted as institutions for redistributing the products of multiple ecological regions, but he is fair to other ideas. (He is even fair to Jane Jacobs's wacky idea that cities preceded, and caused, agriculture, which is to say he does some simple calculations to show it makes no sense whatsoever.) He then goes on to consider social organization, leading institutions like the palace and the temple, the hints of self-government among city-dwellers and their growth over time, the relations between cities and their agricultural hinterlands, how food moved into the cities, long-distance trade, credit and finance, and cities as centers of religion and learning, including divination and astronomy. (He says scribes were taught "calculus", presumably meaning "calculation".) He quotes frequently from Mesopotamian documents, without any philological apparatus, and despite a ritual rejection of strict "positivism", he is very cautious in advancing hypotheses, and very good about marking conjectures as such, and emphasizing that we simply have little or no evidence about many matters.

Mesopotamian history is usually considered to last from the first writing around -3100 to the Macedonian conquests around -300. As Van De Mieroop says, this period of 2800 years is longer than the interval separating us from Homer. It is an astonishing act of hubris, or at least of abstraction, to try to summarize the features of all cities over such a period, even in a restricted region — one can only presume that there must have been extensive variation. Nonetheless, Van De Mieroop does a really remarkable job.

Lucy Snyder, Installing Linux on a Dead Badger

You remember "Installing Linux on a Dead Badger", don't you? Well, imagine a full hundred page book of such stories. C'mon, you know you want it.

Jack Campbell, Valiant

Continuing science-fictional anabasis; see here for previous installments.

Charles E. Lindblom and David K. Cohen Usable Knowledge: Social Science and Social Problem Solving

This is a short (100 pp.) book from 1979, largely given over to sketches of arguments and directions for further inquiry (mostly not undertaken in the ensuing years) about why the social sciences, and "professional social inquiry" more generally, have not been very directly useful for social problem solving. They suggest that this rests on a number of basic widespread mistakes about how social problems are solved. In particular, they allege, social scientists vastly under-rate the importance and competence of ordinary-life social knowledge, and, yet more consequentially, fail to see that social problems can be solved either by analyzing them in some discursive/analytic form, or by setting up patterns of social interaction where the participants' acts collectively solve the problem, though none of them need to grasp the solution or even realize that is what they are doing. Markets are of course one example of such "interactive problem solving", but they also, and quite correctly, emphasize others: democratic politics, bargaining processes, and the "republic of science". They emphasize that interactive problem-solving should not be seen as a poor substitute for formal problem solving, to be displaced in due time by scientifically-informed social engineering, but rather as inevitable, and indeed often superior.

The alternative, of analytically finding solutions to social problems, is basically impossible, because the problems are too complex, and even systematic investigation into them is not just prohibitively expensive, but so slow that the world has moved on before research findings can become very accurate or precise. (Obviously these obstacles can all be bigger or smaller in various cases, and I don't think they'd quibble if someone wanted to assert that very small, stable social problems could be successfully analyzed if enough resources were thrown at them.) Worse, the very definitions of "social problems" are themselves contested, and properly so. The authors' view is that while the natural sciences can (often) legitimately claim independent authority, for social scientists to aim at such authority is to set themselves a target they cannot possibly hit. Since they do aim at that target, however, social scientists and other "practitioners of professional social inquiry" systematically waste their efforts.

Given all this, fruitful roles for social analysts become things like advising individual participants in the interactions, or looking at the over-all performance of an interactive mechanism and searching for ways in which it might be improved. (They suggest that economists are better about this than other social scientists. Given the recent vogue among economists for replacing all kinds of institutions with arbitrary intellectual constructions, planned by analogy with the idealized markets of their Micro 1 textbooks, I suspect the authors might wish to revise and extend these remarks.) A further, if more diffusive, constructive role would be in hoping to shape the general framework within which participants in interactive problem-solving think about things; and of course the kind of detailed reportage which statistical bureaus engage in.

The name "Hayek" does not appear anywhere in this book.

Douglas W. Tallamy, Bringing Nature Home: How Native Plants Sustain Wildlife in Our Gardens

This is a rather straight-forward argument for North American suburbanites (and urbanites, too) to plant more native plants, roughly as follows. (1) We like having wildlife such as birds and (some) mammals around. (2) These animals sit high in the food-web; below them are many insects, especially insect larvae. (3) Most insects are specialized to eat only a small range of plants with which they have co-evolved, in no small part because of the chemical defenses evolved by plants. (4) Thus, native plants support a much larger and much more diverse population of insects than do introduced ones. (5) The process of evolutionary adaptation is very slow, and even plants introduced almost 500 years ago are still substantially less good as insect hosts than natives. (6) Suburbia occupies such a huge part of the American landscape that if native plants are to thrive anywhere, it has to be there. Therefore, (7) suburbanites should shift what they plant towards natives.

Obviously, the key empirical parts of this argument are (4) and (5); here the evidence that Tallamy presents is good, but — he is admirably up-front about this — not conclusive, and he is happy to admit that there are some species of introduced plants which are so closely related to natives that bugs like them just fine. Unlike many books on native plants, this is empirical, consequentialist, modest, and un-mystical.

David Pollard, Empirical Processes: Theory and Applications (full text free online)

The simplest sort of empirical process arises when trying to estimate a probability distribution from sample data. The difference between the empirical distribution function F_n(x) and the true distribution function F(x) converges to zero everywhere (by the law of large numbers), and — this is non-trivial — the maximum difference between the empirical and true distribution functions converges to zero, too (by the Glivenko-Cantelli theorem, a uniform law of large numbers). The "empirical process" E_n(x) is the re-scaled difference, n^1/2[F_n(x) - F(x)], and it converges to a Gaussian stochastic process that only depends on the true distribution (by the functional central limit theorem). Empirical process theory is concerned with generalizing this sort of material to other stochastic processes determined by random samples, and indexed by infinite classes (like the real line, or the class of all Borel sets on the line, or some space parameterizing a regression model). The typical objects of concern are proving uniform limit theorems, and with establishing distributional limits. (For instance, one might one want to prove that the errors of all possible regression models in some class will come close to their expected errors, so that maximum-likelihood or least-squares estimation is consistent. [For more on that line of thought, see Sara van de Geer's book on Empirical Processes in M-Estimation.]) This endeavor is closely linked to Vapnik-Chervonenkis-style learning theory, and in fact one can see VC theory as an application of empirical process theory. (I'd guess Vapnik himself would disagree with that, however.)

This short book by Pollard is an introduction to empirical process theory by a statistician for statisticians. As such it succeeds admirably; as always, Pollard does a really good job of explaining what the technical apparatus is doing and why it takes the form it does. People coming to it from other backgrounds (I am particularly thinking of computer scientists) will probably find it harder going, not least because the implied reader has an extremely sure grasp on measure theory. (Such as one might acquire from, oh, Pollard's User's Guide to Measure-Theoretic Probability.) If you can handle Pollard's 1989 survey paper, then you will probably enjoy this book; and if not, not. The applications he describes are all interesting and challenging, though I was a bit disappointd that none of them involve dependent data.

Clubbing

Fall of Cthulhu: The Fugue

Scalped: Indian Country

Proof: Goatsucker

Various flavors of comic-book mind-candy.

Jenny Davidson, The Explosionist

It is impossible to describe this better than the author, so I'll steal her words:

the story of a 15-year-old girl growing up in an alternate version of 1930s Edinburgh, one where the legacy of Napoleon's victory a century earlier at Waterloo is a standoff between a totalitarian Federation of European States and a group of independent northern countries called the New Hanseatic League. This world is preoccupied with technology (everything from electric cookers to high explosives) but also with spiritualism, a movement our world largely abandoned in the early twentieth century; Sigmund Freud is a radio talk-show crank, cars run on hydrogen and the most prominent scientists experiment with new ways of contacting the dead.

My biggest complaint with this book is that it ends in the middle of the story, and nothing warns the reader about this. Grrrr.

Charles Tilly, Democracy

An attempt to explain the mechanisms by which states come to engage in "broad, equal, protected, mutually-binding consultation" with their citizens, singling out three especially important processes: (1) integrating "trust networks" into public politics, (2) screening off public politics from categorical forms of inequality, and (3) suppressing non-state centers of coercive power. As usual with Tilly, he draws on a huge range of historical sources, in an impressive display of erudition and clear thinking. Also as usual with Tilly, one does not get a comprehensive theory, but perhaps this is the sort of material where such a theory isn't really possible, and the best one can hope for is a catalog of recurring mechanisms.

Charles Tilly, Big Structures, Large Processes, Huge Comparisons

A brisk debunking of pernicious ideas about how societies work and change that we have inherited from the 19th century, together with a smart and enthusiastic brief for comparative, historical social science. (The description at Powell's is definitely for another book!)

Thanks to Doug White for lending me his copy.

Books to Read While the Algae Grow in Your Fur; Enigmas of Chance; Biology; Scientifiction and Fantastica; The Dismal Science; Cthulhiana; Writing for Antiquity

In Memoriam Abdussattar Shalizi

My grandfather passed away at his home in Kabul on July 4th. He was, most likely, 92.

Nothing I could say right now would be adequate.

Books to Read While the Algae Grow in Your Fur, June 2008

James R. Flynn, What Is Intelligence? Beyond the Flynn Effect

A full review will be forthcoming in a magazine, so I don't want to spoil that, but will just hit a few key notes.

(1) This is mostly about explaining the large, long-term, world-wide rise in IQ, a.k.a. the Flynn Effect (not so named by Flynn!), and especially the fact that parts of IQ tests show different rates of gain, or none at all, with no particular reference to their correlation with the (IMSAO mythical) "general factor of intelligence" g.

(2) Flynn's preferred explanation is much closer to mine than I would have guessed before reading this: it has to do with the cultural diffusion of new habits of thinking, what he calls "putting on scientific spectacles". I do not think he follows this argument as far as he could, or should. (I will elaborate on that in the full review.)

(3) A reasonable chunk of this already short book consists of Flynn's animadversions on ethical relativism, postmodernism, etc. I like seeing these bashed as much as anyone else who thought "Transgressing the Boundaries" was the funniest thing ever, but here it's a tangent on a tangent, and Flynn doesn't even bash them especially well; his editor shouldn't have let him indulge himself to this extent.

(4) Flynn is clearly trying to write for a general audience, but I am not sure that someone who reads, e.g., his account of the model he and Dickens devised of "social amplification" would understand it, unless they had read and grasped the relevant papers first.

— On re-reading this seems more negative than my actual opinion. It's definitely worth reading if you care at all about the IQ controversy; it's probably not so helful as a first exposure to that subject.

Roger Th. A. J. Leenders, Structure and Influence: Statistical Models for the Dynamics of Actor Attributes, Network Structure and Their Interdependence

(In lieu of a full review): Two extremely important phenomena in social networks are that (1) people don't make social ties randomly, but tend to link up with others who are either similar to them in some salient way, or to whom they are complemntary; and (2) people learn from and imitate each other. This creates a very serious inferential problem: when we observe that neighbors in a social network are more similar than random members of the population, is that because being linked made them similar, or did they link because of pre-existing similarities? (In the jargon, there is confounding between homophily and contagion.) Leenders's book, a revision of his dissertation, is a first attempt at prising this appart, by using models which represent both how people might influence each other and how they might decide on who to interact with. It is straightforward but worthy stuff, and I can think of a number of high-profile recent papers whose authors — or, better yet, referees — should be whacked over the head with this. (It's only a 250 pp. paperback so that's not that bad.)

The writing is exactly as bad as you'd expect from a doctoral dissertation in mathematical sociology. Nonetheless, I strongly recommend it for the collection of anyone seriously interested in social networks or dynamic network analysis.

C. J. Sansom, Dark Fire

Historical mystery, sequel to Dissolution. This time our hero has the misfortune to get mixed up in a complicated plot involving alchemists, royal marital dissatisfaction, and politico-theological disputes. Manages to mater-of-factly convey the awfulness and alienness of Tudor England, without slipping into the trap of making the narrator a modern man on the inside.

Jane Haddam, Cheating at Solitaire

Haddam takes on Martha's Vineyard Margaret's Harbor and the culture of celebrity, with special reference to pop tarts. It would have been easy, in the interest of entertainment, to make many of the characters completely unsympathetic; she doesn't. And the mystery was baffling, at any rate to me.

Books to Read While the Algae Grow in Your Fur; Enigmas of Chance; IQ; Networks

Short Story Sunday Reading

Three for your delectation:

• K. J. Bishop, "The Art of Dying" [Featuring the return of a character from The Etched City, which you should go read right now if you haven't. When did The Boston Review start publishing fantastica?]
• David Moles, "Long Past Midnight"
• Catherynne M. Valente, "A Buyer's Guide to Maps of Antarctica"

Scientifiction and Fantastica

An Undertaking of Great Advantage, But Nobody to Know What It Is

Shorter FBI to Congress:

Our "National Security Analysis Center" data mining project is too important and super-secret to explain to mere legitimate authority, but it's expanding exponentially, so the money you give us for it had better grow too.

As Noah Shachtman notes, the remarkable thing is that this did, in fact, lead to the House appropriations committee voting to at least not expand the program. (They did not, apparently, vote to kill it altogether, though I can't see why not.)

(Speaking of wasteful, fradulent, and/or abusive data-mining...)

Manual Trackback: Earning My Turns

The Continuing Crises

Chris Anderson: Aware of All Statistical Traditions (with bonus fall course announcement)

Attention conservation notice: Someone is wrong in Wired magazine.

I recently made the mistake of trying to kill some waiting-room time with Wired. (Yes, I should know better.) The cover story was a piece by editor Chris Anderson, about how having lots of data means we can just look for correlations by data mining, and drop the scientific method in favor of statistical learning algorithms. Now, I work on model discovery, but this struck me as so thoroughly, and characteristically, foolish — "saucy, ignorant contrarianism", indeed — that I thought I was going to have to write a post picking it apart. Fortunately, Fernando Pereira (who actually knows something about machine learning) has said, crisply, what needs to be said about this. I hope he won't mind (or charge me) if I quote him at length:

I like big data as much as the next guy, but this is deeply confused. Where does Anderson think those statistical algorithms come from? Without constraints in the underlying statistical models, those "patterns" would be mere coincidences. Those computational biology methods Anderson gushes over all depend on statistical models of the genome and of evolutionary relationships.

Those large-scale statistical models are different from more familiar deterministic causal models (or from parametric statistical models) because they do not specify the exact form of observable relationships as functions of a small number of parameters, but instead they set constraints on the set of hypotheses that might account for the observed data. But without well-chosen constraints — from scientific theories — all that number crunching will just memorize the experimental data.

I might add that anyone who thinks the power of data mining will let them write a spam filter without understanding linguistic structure deserves the in-box they'll get; and that anyone who thinks they can overcome these obstacles by chanting "Bayes, Bayes, Bayes", without also employing exactly the kind of constraints Pereira mentions, is simply ignorant of the relevant probability theory.

By coincidence, I am going to teach our data mining course (36-350) again in the fall. The theme for the semester, which I decided on back in the spring, will be "waste, fraud and abuse" — not so much detecting suspicious activity, though some examples of that might be fun, as warnings against wasteful, fraudulent and/or abusive data mining.

Update, 29 June: see next post.

Update, 2 July: A correspondent writes to let me know that Anderson's essay and the linked pieces from Wired are up at Edge.org, along with responses from some of the other clients of John Brockman's literary agency leading public intellectuals associated with that site. So far, the only one whose reaction is both substantial and not completely clueless is Danny Hillis, who politely says that Anderson's idea does not have "even a little bit of truth in it".

There's no reason we couldn't have an interesting public discussion about what big data, and data-mining, could contribute to science. We already have a very large and successful scientific discipline which routinely generates and deals with petabytes of data, namely experimental high-energy physics. Its example suggests that theory becomes more rather than less important with huge volumes of data. That may not hold for the biological and social sciences, but I'd like some argument as to why. Of course, if one looks at actually-existing quantitative models in those sciences, it seems clear that part of what they are doing is representing scientists' substantive knowledge and/or guesses, but another part is just put in for tractability, especially statistical tractability — linear or logistic dependence, Gaussian noise, etc., etc. One of the things modern statistics and big data could do is to drastically weaken those tractability constraints. (To repeat a slogan from my class, "More science, fewer t-tests.")

We could have a conversation about these matters. But its participants would have to know something about scientific practice, about statistics and about data-mining. Some of these participants might even argue quite strongly that discovery can be automated, if one goes about it the right way. If someone — say, a literary agent and impresario whose client list includes just about every well-known popular science writer in America — wanted to organize such a discussion, it would certainly be possible and a contribution to public enlightenment. That would, however, require such impresarios to have somewhat more critical acumen than a puppy, which evidently is not the case. So the actually-existing conversation is a source not of light but of noise.

Why oh why can't we have a better consciousness industry?

Manual trackback: Entertaining Research; Tongue but no door; O Hermenauta; Whimsley; Quantum of Wantum; The Statistical Mechanic; Lies and Stats

Enigmas of Chance; Corrupting the Young

Books to Read While the Algae Grow in Your Fur, May 2008

Stan Washburn, A Moral Alphabet of Vice and Folly: Embellished with Nudes and Other Exemplary Materials

A series of etchings by "California's foremost sixteenth century artist", accompanying short, mordant little fables. For example, under "P": "A Philosopher concluded that man's pretentions are absurd, and that worldly endeavor is without purpose. So assiduous were his ruminations on this insight that he neglected to publish, and in due course he perished. Moral: publish."

(Thanks to Carl Worth for introducing me to this book.)

John Sutton, Marshall's Tendencies: What Can Economists Know?

A wonderful little book about how economists do, and (what is not quite the same) should confront models with empirical data. Along the way he discusses the history of econometrics, the theory of the tides, option pricing, the origins of thermodynamics, the price of taxi-cabs in San Diego, how to bid for oil rights, etc., etc. It's a wonderful performance in only about a hundred pages, and requires no technical knowledge of econometrics or game theory, though some would probably help. Much of what he says would apply, mutatis mutandis, to any social or natural science, though many of them will not have such strong convictions about the form models should take.

Timur Kuran, Islam and Mammon: The Economic Predicaments of Islamism

This is a collection of essays on self-proclaimed "Islamic economics" and "Islamic banking", along with a final one on the causes of the comparative economic backwardness of the Islamic world (by which Kuran means the Middle East and not e.g. India or southeast Asia), as compared to western Europe.

The key point of the essays on Islamic economics and Islamic banking is sound: these are recent ideological creations, entirely a product of the last half century, and their goal is expressive, rather than actually being concerned with understanding or improving economies. Islamic banking, in particular, while it purports to avoid interest-based loans in favor of risk-sharing, in fact does so only through transparent dodges, and that for very good reasons. (It is, in fact, not at all clear that the Qu'ran prohibits interest as such, as opposed to certain usurious practices.) Kuran also speculates, plausibly but entirely without evidnece, that the creation of an "Islamic sub-economy" of banks, manufacturing and service firms, grocers, etc., allows up-and-coming emigrants to the great cities of the Islamic world to find networks of reasonably trust-worthy peers in similar situations, since they are shut out of the existing elite networks.

These points are made in most of the essays here, many times over, and highly repetitiously. They are also marred by what I can only call a very strange ethical scheme. Kuran claims, fairly enough, to be a Hayekian, and so what worries him about Islamism is whether it might be redistributive, or mess with the rights of property owners. To quote from p. 68,

Modern Islamist movements possess, then, the ideological capacity and flexibility to sustain a liberal economic agenda. Even if they promote illiberal policies while in opposition, they may be able to assume a liberal orientation once in power. In any case, to pursue effectively liberal policies they need not make deliberate or explicit ideological adaptations. By giving low priority to economic issues, they may end up promoting private investment, self-management, private ownership, and free trade by default. Such unintended liberalism is all the more likely where illiberal economic goals are overshadowed by objectives concerning family, sexuality, manners, and education. Though a prominent theme in Khomeini's pre-revolutionary rhetoric was the elimination of poverty and exploitation, once he rose to Iran's helm he subordinated his stated economic objectives to the general goal of restoring the centrality of Islam in public life &emdash; even to such particular objectives as eliminating the consumption of alcohol, veiling women, banning Western music, and severing Iran from its pre-Islamic heritage. After the revolution, he dismissed demands for concrete economic reforms on the ground that economic well-being is worthy of the donkey.

To put this in "shorter" form: "Sure, the new regime is using the coercive power of the state to impose a single scheme of values on all citizens, censoring all forms of expression, and forcibly subjecting half the people to lives of grossly restricted choices, but they're against rent control, the minimum wage, industrial policy and tarriffs — liberal values are safe!"

The last chapter tries to construct an explanation for the economic backwardness of the Islamic Middle East, drawing on Kuran's truly impressive book on preference falsification. This is extremely unpersuasive. The fact that Kuran sets out to explain is that, while this part of the world used to be on an economic level with western Europe, or even more advanced, it is no longer and has not been for some centuries. However, this is not just true of the Middle East but also of India, China, Japan, etc. By constructing an explanation which could, at most, apply to one of those cases, he is asking us to believe in a remarkable coincidence... The alternative, which he mentions but doesn't really address, is that something very peculiar started happening in one place, namely western Europe, and that this explains the relative decline of the rest of the world. (Don't get me started on his cross-country regression of growth rates.)

To sum up, Kuran has some important points to make, but also some truly remarkable lapses and blind-spots. His book is short (147 pages + notes) yet highly repetitive; it should have been even shorter. It's worth reading only if you are extremely interested in the subject; worth buying only for specialists.

Fritz Stern, The Politics of Cultural Despair: A Study in the Rise of the Germanic Ideology

Studies in the lives and works of three intellectuals who helped, each in their own way, to lay the road to ruin: Paul de Lagarde, Julius Langbehn and Arthur Moeller van den Bruck. Good at not just seeing them in that light, and at seeing that many of their trends they participated in — for example, a driving hatred of liberalism — were not just German. (A nice study could be done comparing these ideologues' idea of "the West", home of "civilization", as opposed to German "culture", to the idea of decadent, socialist Europe among contemporary American conservatives.) The contrast cases Stern seems to have in mind, but who mostly show up in his footnotes and asides rather than his main text, are Nietzsche and Thomas Mann, who shared (at least for a time, in Mann's case) these writers' pretense of being "unpolitical", but with vastly more sense and vastly less moral depravity.

Robert Pinsky, Gulf Music: Poems

The poet wrestling with things, disasters, and his own "insomniac monkey-mind". I'll quote the last part of "First Things to Hand", titled "Door", because it is nice, but it's not representative of the range. (No one poem here is.)

The cat cries for me from the other side.
It is beyond her to work this device
That I open and cross and close
With such ease when I mean to work.

Its four panels form a cross—the rood,
Impaling gatepost of redempton.
The rod, a dividing pike or pale
Mounted and hinged to swing between

One way or place and another, meow.
Between the January vulva of birth
And the January of death's door
There are so many to negotiate,

Closed or flung open or ajar, valves
Of attention. O kitty If the doors
Of perception were cleansed
All things would appear as they are,

Infinite. Come in, darling, drowse
Comfortably near my feet, I will click
The barrier closed again behind you, O
Sister will, fellow mortal, here we are.

Andrea Camilleri, The Patience of the Spider and The Paper Moon

Wonderful as always; Montalbano continues to be a superb detective, and the tone of outrage at injustice and astonishment at human depravity and folly is nicely balanced with self-mockery (the scene with the alarm clock at the beginning of The Paper Moon, for example) and good food. (This I think distinguishes itself from American hard-boiled crime stories, which seem to take themselves and their disillusionment so seriously.) Previous installments discussed here, here, here, here and, most recently, here. — Many thanks to "Uncle Jan" for copies!

George R. Milner, The Moundbuilders: Ancient Peoples of Eastern North America

Over-view of the archaeology of the pre-historic inhabitants of what is now the eastern US, with a little bit of Ontario thrown in, emphasizing the mound-building cultures of the mid-west and south-east. Milner seems somewhat more confident in some of his statements (e.g., about artifacts circulating by gift exchange rather than trade, or about social organization) than the evidence he presents would seem to warrant, but then I often have this problem when reading archaeologists.

Jessica Hagy, Indexed

Fun with Venn diagrams and little graphs on two axes. The effect is a little hard to describe, but fortunately you can just see examples.

David Rees, Get Your War On II

Reading this in 2008 brings back, in a truly vivid way, just how much of a feverish nightmare 2002--2004 really was.

Books to Read While the Algae Grow in Your Fur; Islam; The Dismal Science; Enigmas of Chance; The Running Dogs of Reaction; Writing for Antiquity; The Commonwealth of Letters; The Continuing Crisis

A Note to My Alma Mater: John Yoo Is Not a Campus Treasure

Dear UC Berkeley: I understand you have reasons, more or less good, for not firing John Yoo immediately. But there is no call to put puff-pieces about him in the magazine you send to donors. This is not making me any more likely to contribute. I'm not asking for displays of public repentance in publications like this, though that would be nice; just not rubbing my nose in the fact that I got my degree from a school which has as one of its faculty a man who helped make us a nation of torturers. Sincerely yours, Cosma Shalizi ('93).

Update: see also Marty Lederman at Balkinization.

The Running Dogs of Reaction; Learned Folly; The Continuing Crises

End-of-Semester Inventory

New classes taught alone: 1
New classes co-taught: 1

Thesis committees I was on at beginning of semester: 5
Number of those students who successfully defended their dissertations this semester: 1 (congratulations, Dr. Damouras!)
Number of thesis committees I joined: 2
Co-supervised Ph.D. students who successfully proposed and are now ABD: 2 (yay, Linqiao and Justin!)
Undergrad RAs going on to graduate school: 1 (yay, Shawn!)
Undergrad RAs not heard from in months: 1

Grant applications rejected: 3
Grant applications still pending: 3

New papers mostly finished at beginning of semester: 3
Papers mostly finished at end of semester: 5
Papers under review or revision at beginning of semester: 5
Papers under review or revision at beginning of semester: 5
New papers half finished at beginning of semester: 4
Papers half finished at beginning of semester: 5
"We should really do a paper together on X" ideas at start of semester: 5
"We should really do a paper together on X" ideas at end of semester: 8

Blog posts written: 31
Unfinished posts in my drafts folder: 50

Papers submitted: 0
Papers accepted: 0
Papers published: 0

Self-Centered

Memos to Self, re: Pedagogy

Attention conservation notice: An exercise in public self-embarrassment as an aid to behavior modification.

1. [REDACTED]
2. The next time a research student gets you a book from your wish-list as a gift, do not let the first words out of your mouth be "Wow, I just bought that the other week!".
3. Keep the wish-list up to date.

Self-Centered; Corrupting the Young

Today in Et in Arcadia Ego Blogging

Both brought to you via Bill Tozier.

First, from LOL Manuscripts:

The post is worth at least a sardonic glance.

Second, a discussion of the whole Rennes-le-Château/Priory of Sion mythology as an alternate reality game devised by Pierre Plantard. In other words, Holy Blood, Holy Grail and The Da Vinci Code are the "Tlön, Uqbar, Orbis Tertius" we deserve.

— The fact that death and delusion are on my mind has nothing whatsoever to do with the fact that my students in 462 are turning in their final papers today.

Psychoceramica; Writing for Antiquity; Linkage

Assorted Link Roundup, May 2008

Without style or grace.

Wolfgang Beirl explains why financial engineers (like the ones I've been teaching this semester) are also known as "rocket scientists". There are connections here to Wolfgang's thoughts on telephones and the foundations of statistics.

Man's role in changing the face of the Earth dep't.: Ben Fry's map of the 48 contiguous states, showing only streets and roads. Everything else, astonishingly, emerges from that. (Via Unfogged.)

The radical right revives the theory of magical kingship propounded by Sir J. G. Frazer in The Golden Bough, in which the health of the land is sympathetically tied to the character of the ruler, as an account of the American presidency, and correspondingly prophecies doom, doom, DOOM! should Hillary be elected. Illustrated with kittens. Note: WorldNetDaily, unlike the Landover Baptist Church, is not a parody. (I've mentioned them before.)

Speaking of signs of the apocalypse, Thomas "The Baffler" Frank is now writing op-eds for the Wall Street Journal. (Via Aaron Swartz.)

Mind Hacks offers two neurologically-themed tattoos, observing of the second that it produces "a markedly different effect, despite the fact it resides in the same location".

Steve Laniel and Tom Slee review Clay Shirky's Here Comes Everybody. You have probably already seen or read Shirky's talk "Gin, Television and Social Surplus". His social history is over-simplified, and I get a bit leery of my own response to things which push my buttons so thoroughly, but nonetheless — preach it, brother Clay, preach it!

Brooks Simpson, in an interview with the Southern Poverty Law Center, briskly shreds various lies about the US Civil War and the Confederacy propagated by modern apologists for "treason in defense of slavery". Via Abiola Lapite, who has a good post on the genetics of height.

Sierpinski cookies (via Dave Feldman).

Because I am a mean and vicious person, I take great pleasure at reading Kathy G. toy with someone who pretends to know something about economics (1, 2, 3, 4). G. is a public-spirited person, so when she says "I write about economic theory because I believe it is Really. Fucking. Important. Bad economic models make for bad economic policies.", I believe her. But I enjoy reading her for the sheer pleasure in the evisceration. Similarly, I think that in a juster world, Camille Paglia would now be remembered only as the occasion for this 1991 Molly Ivins essay.

Kit Whitfield explains the concept of a "Macho Sue":

A disagreeable variant of Mary Sue, often found in action films, cop shows and the more battly kind of science fiction. While Mary Sue is a fictional character who bends the universe around herself with her amazing specialness, Macho Sue bends the universe around his manhood. He has a particular ability to get away with behaviour that would be considered bad in a woman — to the point of behaviour that would be considered typically female by a misogynist if displayed by a woman.

These traits usually involve poor self-control, such as outbursts, tantrums, sulks, and a refusal to take responsibility for his own behaviour towards others when he's upset. It's not uncommon for Macho Sue to be prejudiced, or at least suspicious of the unfamiliar, and he's almost always unusually disrespectful to others; he has a particular propensity for taking an unreasonable dislike to somebody on sight (only to have it validated later). When thwarted, he tends to be affronted as well as frustrated, in a way that suggests neither he nor the narrative think it right that anyone but him should ever get their way. The story tends to throw straw men at him by way of obstacles, but they're never shown as equally masculine, and thus are without any heroism of their own. Macho Sue is emotional, but with such an assumption of gendered authority that nobody questions the manliness — in the rightful sense of 'adulthood' — of his behaviour.

She instances (the characters played by) John Wayne, but, oddly enough, neglects to mention Achilles.

Further on the literary-critical vein, a remarkably funny, yet thoroughly horrifying, review of a set of novels I will not be reading. It ends thus

The PALADIN OF SHADOWS series is arguably the most horrifying series of books I have ever read. It has a hero I can't stand, politics so strong they're comical, and sex scenes that are downright horrifying. And I cannot stop reading it. I am going to buy every single one, and if Ringo ever comes out with a spin-off featuring Katya as Cottontail the Bionic Whore, I will buy that too. Because dammit, there's bad, and then there's so bad you have to memorialize it for future generations.

but you really need to read what comes before it to get the full effect. The reaction by the author of the books in question is — startling. (Via Kate Nepveu.)

Thematically not-unrelated, an experiment with a famous comic book author. (For the record, I liked Ronin well enough when I read it as a teenager, but generally haven't seen what there was to get excited about in Miller's work; at most a "lower and distorted form" of a general theme.)

Second in our series of great moments in Afghan Buddhism: the earliest known oil paintings may be from Bamiyan (via Matthew Berryman).

You should read Existence Is Wonderful. She changes my mind about things.

I become more and more convinced that one of the keys to understanding our intellectual life is the Skolnick Effect. It is hard to understand the success of neuromarketing otherwise, for example. It's not that functional brain imaging can't be scientifically useful (I'm involved in some projects myself), but the level of the usual study which gets popular attention is to tell us, on the basis of tiny samples, that some part of the brain is differentially activated by thoughts of attaining money, chocolate, justice and sex and/or dirty pictures. (That last link in particular offers a glimpse into a remarkable clusterfuck of bad science journalism amplifying sloppy thinking.) At this point what you are really learning is that there isn't a straightforward mapping from our psychological concepts to paticular brain regions, which is something the neuropsychologists have been trying to tell you for quite a while now. You can even say it with math, but that doesn't seem to make people any more inclined to listen.

Worse, the journalists — and even many of the scientists — seem incapable of separating "implemented in the brain" from "innate". (A recent offender, via Abiola. [It wouldn't surprise me in the least if some sense of social hierarchy is innate in human beings — with all the disclaimers about what such statements mean hereby incorporated by reference — but the point is that the results reported are completely irrelevant to the question of innateness.]) I realize we have thousands of years of ingrained ideas about mind-body dualism and human nature to work through here, but honestly, people, could we at least get into the eighteenth century? All our thoughts and actions involve our brains somehow; detecting them in the brain with current technology says nothing about their being innate, unless you want to seriously say that the rules of chess are hard-wired into our genomes. But if I pursue this further I will get into the bog of free will, and the idiotic conclusions about it people draw from weird experiments...

(Meanwhile, the fact that people can get papers in Science out of the astonishing prediction that territorial ethnic conflict requires the geographic proximity of (self-perceived) ethnic groups, and is rare in locales where one group is an overwhelming majority, suggests that there is a version of the Skolnick Effect involving toy-model simulations.)

The newly-risen Fafblog shows that prophetic parody is the only way to keep up with the real news.

Carlos Yu has, sadly, stopped blogging. I feel a bit bad because I always liked his stuff and rarely told him so. I will miss very much the only blogger capable of writing about ancient Sanskrit plays featuring "creepy horny drunk carnivorous beggars covered in human ash, accompanied by hott chick acolytes, carrying around someone's skull, asking you for money" (parenthetically adding "I think they used to squat in Tompkins Square Park"); the grand unified theory of wingnuts; Garry Wills; paleobiochemistry; football and other pure products of America; and God knows what else. I hope that, like Fafblog, he will one day return.

A while back, Brad DeLong linked to a parody of Thomas Aquinas's proofs of the existence of God, Five Ways of Proving the Existence of Santa Claus. This is ridiculous, of course, but really not much more so than such serious topics treated by the Angelic Doctor as the relation of the saints to the damned (the saints in Heaven will see the suffering of the damned perfectly; will have no pity towards them; and will in fact rejoice in their suffering); whether the weeping of the damned will be corporeal (yes, but there will be no tears); and whether the damned will be in material darkness:

The disposition of hell will be such as to be adapted to the utmost unhappiness of the damned. Wherefore accordingly both light and darkness are there, in so far as they are most conducive to the unhappiness of the damned. Now seeing is in itself pleasant for, as stated in Metaph. i, "the sense of sight is most esteemed, because thereby many things are known."

Yet it happens accidentally that seeing is painful, when we see things that are hurtful to us, or displeasing to our will. Consequently in hell the place must be so disposed for seeing as regards light and darkness, that nothing be seen clearly, and that only such things be dimly seen as are able to bring anguish to the heart. Wherefore, simply speaking, the place is dark. Yet by Divine disposition, there is a certain amount of light, as much as suffices for seeing those things which are capable of tormenting the soul. The natural situation of the place is enough for this, since in the centre of the earth, where hell is said to be, fire cannot be otherwise than thick and cloudy, and reeky as it were.

Some hold that this darkness is caused by the massing together of the bodies of the damned, which will so fill the place of hell with their numbers, that no air will remain, so that there will be no translucid body that can be the subject of light and darkness, except the eyes of the damned, which will be darkened utterly.

On which note, I have a final exam to give.

Linkage; The Commonwealth of Letters; Minds, Brains, and Neurons; The Natural Science of the Human Species; The Beloved Republic; Afghanistan and Central Asia; The Dismal Science; Math; Learned Folly; The Running-Dogs of Reaction; The Continuing Crises; Philosophy; Psychoceramics

Books to Read While the Algae Grow in Your Fur, April 2008

Richard Bookstaber, A Demon of Our Own Design: Markets, Hedge Funds, and the Perils of Financial Innovation

One part "financial crises I have known" to one part general thoughts about market dynamics, and in particular the difficulties that arise due to complexity, "tight coupling" of markets, and leverage. The stories are going to be familiar to most people interested in the subject. The latter are interesting but under-argued. This is true even when I agree with him, about, e.g., the limitations of statistical modeling in financial markets. (The pages on Gödel's Theorem, the Heisenberg Uncertainty Principle and chaos were painful, but also completely logically independent of the stuff about finance.)

This may get a full review later. For now I'd just say that his main recommendations — avoid complex and novel financial instruments, avoid leverage, and avoid trying to optimize to current conditions, in favor of responding adequately to a wide range of situations, including ones you can't currently anticipate — are not bad as words of wisdom, but he has no hint as to how they could be implemented under current conditions, i.e., in the actually-existing capitalist financial system he describes.

This interview with Andrew Leonard in Salon serves as a decent summary.

John McGowan, American Liberalism

Unapologetic advocacy of modern liberalism as an attempt to provide equal and, crucially, effective freedom to all. Liberalism tries to achieve this by creating institutions which make arbitrary, unaccountable, unchecked power ineffective, because powers are checked and balanced by other sources of power and made to answer for theirs actions to those over whom power is exercised. (This distinguishes it from anarchism, whose ideal is simply to eliminate power.) The means by which these things are achieved are secondary, and evaluated pragmatically, by their effectiveness and side-effects in given conditions as compared to available alternatives. (Liberalism, though he doesn't put it this way, becomes in his hands a general ideology of the second best.) Seen thus, there is a clear line of descent between the 18th century liberalism of (most of) the American founders and the modern ideology, with the main development being taking seriously the bit about all men being created equal.

McGowan tries very hard here to reach the general educated public, rather than fellow academics, and almost succeeds. (There are turns of phrase which make it obvious that he's read his post-structuralists, but they're not unreadable ones.) The ideal book along these lines would be something at the level of, say, Milton Friedman's Free to Choose, and McGowan isn't there, is still a little too committed to academic forms, but this is clearly a labor of love, and I hope it will succeed in being influential.

(I confess, though, that I don't get why he thinks cell phones are worse for involvement in the public sphere than land-lines. The reverse, if anything.)

John McCleary, A First Course in Topology: Continuity and Dimension

Well-written textbook of topology, with a historical flavor (but modern methods), and an emphasis on (as the subtitle suggests) the problem of showing that dimension is invariant under continuous and invertible mappings (homeomorphisms). The reader needs a solid grasp of basic real analysis, linear algebra and abstract algebra.

William R. Cross, The Burned-Over District: The Social and Intellectual History of Enthusiastic Religion in Western New York, 1800--1850

A very solid historical work, though it presumes a fair degree of familiarity with the Protestant sects of early 19th century America, and even with the political history of New York. (I lose any right to review this by the fact that I had to look up the Holland Company, and was boggld by what I found.) Though he does not put it this way, a big part of his thesis as to why much but not all of western New York was so susceptible to religious and semi-religious fads then was that the pure products of Yankeedom go crazy. He makes this very plausible, in a way which nonetheless manages to be sympathetic to the enthusiasts.

Some remarks about feminine weaknesses, and the places where he seems to blame the Civil war on, of all people, the Abolitionists, are distasteful, but also a sign of the moral progress separating us from 1950...

David Ruelle, The Mathematician's Brain: A Personal Tour Through the Essentials of Mathematics and Some of the Great Minds Behind Them

An eminent mathematical physicist's take on mathematics and mathematicians. It manages to be sane, pragmatic, thoroughly unromantic, and yet highly enthusiastic for the subject. I actually think anyone who remembers high school math could follow everything; his trick, here, is to start with that sort of stuff and explain how mathematicians generalize it, why they generalize it, and especially why they generalize it in certain ways and not others. — Despite the title, this is strictly psychological, with negligible neuroscience. Given the utter lack of useful neuroscientific data about mathematical thinking, this is sound.

Draws on his "Conversations on mathematics with a visitor from outer space" (PDF), but with all traces of Gallic whimsy removed. (They would probably have become unbearable at book length.)

Matthew Yglesias, Heads in the Sand: How the Republicans Screw Up Foreign Policy and Foreign Policy Screws Up the Democrats

Young master Yglesias finally delivers on that early promise with a book, which, mercifully, is not about blogging and not just a collection of his blogging. Rather it is a sustained, sober, well-written argument in favor of robustly and forthrightly re-embracing the tradition of liberal internationalism, which tries to create institutions that will channel international affairs in peaceful directions and restrain raw power, in order to create a better world for all, including the powerful. As against this we have various strains of nationalist and/or imperialist viciousness and idiocy. Yglesias argues for liberal internationalism and against other ideologies on grounds of morals, practical benefits (the life of a hegemonic power being nasty, brutish and short), and sheer political expediency for the Democratic party, since the alternative hasn't been working out all that well. (He also offers up some brisk but sincere mea culpas.) I would have preferred more argument about morals, e.g. reminding people that the point of our country is not supposed to be a thousand years of crushing global military dominance, but suspect my own impulses in that direction.

Can be read in a day, if you're stuck on planes. Highly recommended if you're in to this sort of thing.

Warren Ellis and Salvador Larroca, Newuniversal: Everything Went White

Comic-book candy. — OK, it deserves a little more than that. From time to time Timothy Burke complains about how astonishing things happen in comic books, which ought to transform the world, but somehow life goes on exactly as before. This series starts from a world slightly askew from our own, where the appearance of superhumans does, in fact, change things.

Charles Stross, The Jennifer Morgue

Sequel to The Atrocity Archive. More lightheartedly chilling Lovecraftian spy fiction, from the perspective of the geeks in IT. Only, this time, haunted by the ghost of James Bond.

John Dewey, Liberalism and Social Action

"And now abideth liberty, individuality, and the critical use of intelligence, these three; but the greatest of these is intelligence." (Not an actual quotation.)

Brian K. Vaughan et al., Ex Machina: Tag; Fact vs. Fiction; March to War; Smoke Smoke; Power Down

Comic books. Actually, I read these back in February, not too long after the first in the series, but forgot to mention them here. I suspect I can guess where this is going, but even if I'm right I want to see how they get there.

Margaret Maron, Up Jumps the Devil; Killer Market; and Home Fires

More unreasonably charming mystery novels about murder in increasingly-exurban North Carolina. Series fatigue will doubtless set in eventually.

Books to Read While the Algae Grow in Your Fur; The Progressive Forces; Scientifiction and Fantastica; Cthulhiana; The Continuing Crises; Mathematics; Psychoceramica; The Beloved Republic; Minds, Brains, and Neurons; The Dismal Science