Three-Toed Sloth

Posted by crshalizi at December 10, 2006 14:12 | | permanent link

December 03, 2006

Books to Read While the Algae Grow in Your Fur, November 2006

Theodore Judson, Fitzpatrick's War: A futuristic re-telling of the story of Alexander the Great, making him out to be the psychopathic catastrophe-in-human-form he really was.
Lois McMaster Bujold, The Sharing Knife: Beguilement: A fun fantasy, but it's the first volume in a new series, and the protagonists are getting off entirely too easy for a Bujold book. This makes me very suspicious about what she's setting up for volume two...
Liz Williams, Snake Agent: It's sometime in the middle of the twenty-first century. Singapore has franchised itself, and Singapore Three is somewhere on the coast of southern China. Detective Inspector Chen is a member of the police force. His job is to interface with the Celestial and Infernal bureaucracies, which are exactly as described in traditional Chinese beliefs about the afterlife. And, oh yes, his wife is a demon whom he helped get out of the arranged marriage from (forgive the expression) Hell. And then things get complicated.
Laura Lippman, No Good Deeds: Latest in the series (previous installment noticed in August 2005). Absorbing as always; for once I figured out who did it long before the ending (this doesn't seem that hard here!), and that had very little to do with the interest of the novel.
Christopher Priest, The Prestige: I see, from the inside front cover, that I bought my copy in October of 1999. It therefore took me seven years and one month to get around to reading this. It is really, really good, so you should read it much sooner. I will not say anything more, lest I spoil Priest's illusions.
Julia Spencer-Fleming, Out of the Deep I Cry and To Darkness and to Death: Continues the "Claire Fergusson" series begun with In the Bleak Midwinter and A Fountain Filled with Blood. A bit reminiscent, actually, of Phil Rickman's Merrily Watkins mysteries (noticed here and here): a female Anglican/Episcopalian minister investigating murders in an isolated rural town which has lost its economic reason to exist --- and where potential new ones are themselves insidious. The major differences are (1) Spencer-Fleming writes about upstate New York, while Rickman writes about the Welsh marches; (2) Spencer-Fleming is entirely in the mundane world, with none of Rickman's occult intrusions; (3) Spencer-Fleming devotes a lot of space to her heroine's complicated relationship with her town's married chief of police, whereas Watkins's love life is mercifully straightforward.
Toby Musgrave, Courtyard Gardens: Nice as a collection of ideas, with drool-worthy accompanying photographs. Alas, many of them seem ill-suited to the climate of the Allegheny Plateau...
Lindsay Allen, The Persian Empire: Readable, extremely well-illustrated brief summary of the history of the Achaemenid Empire, combining ancient sources with archaeological findings. Good on trying to see things from a less parochial perspective than that of the Greeks (Vidal's Creation [review by Danny Yee; review by Anoop Sarkar] gets a [humorous] plug in an end-note), and on integrating what we know with how we learned it. The last chapter gives an interesting description of later Persian and Iranian ideas about the empire, Alexander, the ruins at Persepolis, etc.
Claudia Koonz, The Nazi Conscience: Horrifies, because there was one.
R. R. Bahadur, Some Limit Theorems in Statistics: Concise yet remarkably lucid summary of pioneering (< 1971) work on statistical applications of large deviations theorems, much of it Bahadur's. Reads somewhat awkwardly now, because it came before the work by Donsker, Varadhan, etc. which led to the modern generalized, streamlined theory of large deviations, so Bahadur has to spend time proving special cases of general theorems, can't just talk about "rate functions", etc. But, strangely, I can't seem to find anyone re-presenting this work using modern large deviations theory...
Warren Ellis and Stuart Immonen, Nextwave: Agents of H.A.T.E.: This Is What They Want: "Healing America by beating people up."
Elaine Cunningham, Shadows in the Darkness: More mind candy.
Laurence Gough, Heartbreaker: Nicely-done police procedural set in Vancouver. The 8th novel in a long series, but I read it with enjoyment without having read any of the others, which I will now look for.
Patricia Briggs, Steal the Dragon and When Demons Walk: Mind-candy fantasy, but tasty. Briggs gets points for the fact that the heroics in the first book did not, in fact, prevent the successful Invasion by the Eastern Hordes, which is simply a settled matter of fact in the second book.

Networks; Enigmas of Chance

Posted by crshalizi at December 03, 2006 23:17 | | permanent link

December 01, 2006

Notes on "A Likelihood approach to Analysis of Network Data"

Attention Conservation Notice: Over 3000 words of technical commentary on a paper on the statistical analysis of networks. Does a poor job of explaining things to those without background knowledge of networks, statistical inference and Markov chains. Includes some geeky jokes and many equations.

Yesterday in the Statistical Learning for Networks seminar, we discussed the following paper; what follows are a mix of my notes from before and after the discussion.

Carsten Wiuf, Markus Brameier, Oskar Hagberg and Michael P. H. Stumpf, "A likelihood approach to analysis of network data", Proceedings of the National Academy of Sciences (USA) 103 (2006): 7566--7570 [Freely available from PNAS as over six months old]: Abstract: Biological, sociological, and technological network data are often analyzed by using simple summary statistics, such as the observed degree distribution, and nonparametric bootstrap procedures to provide an adequate null distribution for testing hypotheses about the network. In this article we present a full-likelihood approach that allows us to estimate parameters for general models of network growth that can be expressed in terms of recursion relations. To handle larger networks we have developed an importance sampling scheme that allows us to approximate the likelihood and draw inference about the network and how it has been generated, estimate the parameters in the model, and perform parametric bootstrap analysis of network data. We illustrate the power of this approach by estimating growth parameters for the Caenorhabditis elegans protein interaction network.

It's perhaps not completely clear from the abstract that their method works for a particular class of network growth models, which they call "duplication-attachment" (DA) models. These are pure growth models, where the network only expands, and never loses nodes or edges. (The network is assumed to be undirected, without self-loops or multiple edges between a given pair of nodes.) At each time step, we add one node. This is randomly chosen, with fixed probability, to be either a duplication or an attachment. If it's an attachment, the new node attaches to an old one, chosen uniformly over the graph (possibly with some fixed probability < 1). If it's a duplication event, we pick an existing node to duplicate, and the new one gets a certain probability of copying each of its model's links (independently), and a different probability of being linked to its model. It is entirely possible that a new node is added with no links to any existing node. Notice that nodes (and edges) have no intrinsic properties in this model; their names are arbitrary. Any two isomorphic graphs should therefore be assigned the same probability.

(The motivation for such models is that gene duplication is apparently fairly common, at least over evolutionary time, which would duplicate the interactions between genes, or their proteins. Attachment, here, is supposed to summarize all the other processes of network growth. There are several models of the evolution of protein interaction networks, e.g. those of Sole et al., 2002, and Vazquez et al., 2003, which are basically of the duplication-attachment type, and yield networks which at least qualitatively match some features of real-world examples, like degree distributions. These papers are not cited here.)

From any starting graph, it is easy to run the model forward and generate new, larger random graphs; the probabilities involved are all pretty simple uniform and binomial things. In fact, the current state of the graph completely determines the distribution of future graphs, so this is a Markov chain. The transition probabilities are fixed by the duplication and attachment parameters, collectively $\theta$ , and these, together with a distribution over starting graphs, give us a fully-specified stochastic model.

Normally, statistical inference for Markov chains is fairly straightforward, because most of the classical conveniences which make inference for IID data tractable are still available. (This is, after all, what led Markov to his chains!) So why then does the paper not end on the second page, with a citation "See Billingsley (1961)"? Because normally, when we observe a Markov chain, we observe a sequence of values from the chain, and that lets us estimate the transition parameters. Here, however, we have only the end-state, the final graph, as our data. The Markov chain will let us assign likelihoods to paths, but we don't know where we started from, and we don't know where we went from there, just how we ended up here.

Suppose we did know where we started from, some graph of size $t_0$ , $G_{t_0}$ . (Remember here that each step of the chain adds one node, so t really counts nodes, not clock-time. This is why it's natural to start at $t_0$ , as opposed to 0 or 1. The paper does not seem to explain this point.) If we knew our initial state, then in principle we could figure out the probability of reaching our final state from it, as a sum over all possible paths:

$L(G_t,\theta) \equiv \mathrm{Pr}(G_t|G_0;\theta) = \sum_{G_{t_0 +1}, \ldots G_{t-1}, G_{t} \in \mathcal{G}(t_0,t)}{\prod_{s={t_0+1}}^{t}{\mathrm{Pr}(G_s|G_{s-1};\theta)}}$
where $\mathcal{G}(t_0,t)$ is set of all growing sequences of graphs which start at $G_{t_0}$ and end at $G_t$

. This mathematical expression is a mouthful, admittedly, but it's probably clearer in a picture.

There are only so many paths from the initial graph $G_{t_0}$ to the final, observed graph $G_t$

. The chain tells us the probability of each such path. Since we had to take one, and only one, of these paths, the total probability of making the journey is the sum of the probabilities of all the individual paths.

At this point, any physicists in the audience should be nodding their heads; what I've just said is that the likelihood, from a given starting configuration, is a sum over histories, or a path integral. Along with the authors, I'll return to how to evaluate this path integral presently, but first we need to figure out how to get that starting configuration.

If we had a known distribution over starting graphs, we could (in principle) just evaluate the likelihood conditional on each starting graph, and then take a weighted sum over graphs. This, however, is not what the authors do. (I'm really not sure where one would find such a distribution, other than another model for graph development. Bayesian practice would suggest picking something which led to easy computations, but this makes a mockery of any pretense to either modeling nature, or to representing incomplete prior knowledge.) Instead, they try to use the known dynamics of the DA model to fix on an unambiguous starting point, and do everything conditional on that.

They observe that you can take any graph, and, for each node, identify the other nodes it could have been copied from. (If A could have been copied directly from B, then A's neighbors must be a subset of B's [ignoring the link between A and B, if any].) So, from any starting graph, you can recursively remove nodes that could have arise through simple duplications. In general, at each stage in this recursion there will be multiple nodes which could be removed, and their choice is arbitrary. Remarkably enough, no matter which choices one makes, the recursion always terminates at the same graph. (More exactly, any two end-points are isomorphic to each other, and so identical for statistical purposes.) The proof is basically a fixed point theorem about a partial order defined on graphs through deletion-of-duplicates, but they confine it to the supplementary materials, so you can take it on trust (and they don't use such lattice-theoretic language even there). This graph --- the data, minus everything that could be pure duplication --- is what they take as their starting point. This is the $G_{t_0}$ to the data's $G_t$ . Everything is then done conditional on $G_{t_0}$ .

OK, we have our initial condition and our final condition, and we have our Markov chain, so all we've got to do so is evaluate the integral over paths linking the two.

Problem: There are too many paths. In the worst case, the number of paths is going to grow factorially with the number of nodes in the observed graph. Even though along each path we've just got to do some straight-forward multiplication, simply enumerating all the paths and summing over them will take us forever. (The authors discuss some algorithmic tricks for speeding up the exact calculation, but still get something super-exponential!) Thus, evaluating the path integral for the likelihood is intractable, even for a single parameter value.

Solution: Don't look at all the paths. Rather, sample some paths, say N of them, evaluate the likelihood along each, and average. Hope that this converges quickly (in N) to the exact value of the integral. This is, after all, how physicists approach many path integrals.

Problem: Even if N is fairly small, we need to examine many settings of the parameter $\theta$ . It could still kill us to have to sample N distinct paths for each parameter value.

Solution: Use importance sampling. Draw a single path, valid for all parameter values, and evaluate the likelihood in terms of the value of an "importance weight" along this path. The weight has to be a function of $\theta$ , but it should be the only thing which is. We do this here by writing the likelihood, $L(G_t,\theta)$ , as an expectation with respect to a reference measure, which the authors write $\theta_0$ . This reference measure is given by another Markov chain, called the "driving chain"; despite its name, it is not a member of the DA family of chains. The trick here is that one sample of possible paths, generated according to this chain, can be used to (approximately) evaluate the likelihood at all parameter settings of the DA model.

The crucial equation is [3] on p. 7568

$L(\theta,G_t) = \mathbf{E}_{\theta_0}\left[ \prod_{s=t_0}^{t}S(\theta_0,\theta,G_s,\nu)\right]$
where
$S(\theta_0,\theta,G_t,\nu) = \frac{1}{t} \omega(\theta,G_t,\nu) \frac{\omega(\theta_0,G_t)}{\omega(\theta_0,G_t,\nu)}$
(N.B., the paper writes the second factor as $\omega(\nu,G_t,\nu)$ , but this is wrong.) Let's unpack this a bit.

$\omega(\theta,G_t,\nu)$ is the probability of producing the graph $G_t$ through the addition of the node $\nu$ , with parameter setting $\theta$ . (N.B., $\nu$ must be a "removable" node, one which could have been added by duplication.) $\omega(\theta,G_t)$ is this transition probability, summed over all possible $\nu$ . The first two factors in S are what we want, the probability we'd get moving forward along the path according to the parameter $\theta$ . The third term is the reciprocal of the transition probabilities according to the driving chain. Its only job is to cancel those probabilities out.

The algorithm for generating the ith sample path is then as follows. Start with the observed graph $G_t$ . Count backwards, $s = t, t-1, \ldots t_0+1$ Pick a node $\nu^{(i)}_s$ to delete, with probability proportional to $\omega(\theta_0, G^{(i)}_s, \nu^{(i)}_s)$ . (Once again, this limits us to the "removable" nodes, the ones which could have been produced by duplication.) Set $G^{(i)}_{s-1}$ to be the result of deleting that node. Keep going back until we hit the irreducible core, $G_{t_0}$ . (We will always hit this core, by the fixed point theorem proved in the supplementary results.) Then

$\begin{eqnarray*} l^{(i)}(\theta) & = & \prod_{s=t_0}^{t}{S(\theta_0,\theta,G^{(i)}_{s},\nu^{(i)}_s)}\\ \hat{L}(\theta,G_t) & = & \frac{1}{N}\sum_{i=1}^{N}{l^{(i)}(\theta)} \end{eqnarray*}$
To ease the calculation of this for multiple parameter settings, it may be worth noting (though the authors do not) that
$l^{(i)}(\theta) = \left(\frac{(t_0-1)!}{t!}\right)\left(\prod_{s=t_0}^{t}{\omega(\theta,G^{(i)}_s,\nu^{(i)}_s)} \right)\left(\prod_{s=t_0}^{t}{\frac{\omega(\theta_0,G^{(i)}_s)}{\omega(\theta_0,G^{(i)}_s,\nu^{(i)}_s)}}\right)$
and the middle factor is the only one which depends on $\theta$ .

So, to summarize: We can generate a sample of paths connecting the observed final graph to the unobserved initial graph, according to the driving chain, and then approximate the likelihood for any parameter value by multiplying the importance weights along those paths and summing over paths. (The importance weights themselves even factor nicely.) We have thus solved the problem of evaluating a path integral, when we've got only one end of too many possible paths.

The trick used here to pull this off depended on having a uniquely-defined starting point for all parameter settings, namely the $G_{t_0}$ defined through undoing duplications. (According to the authors, they took this from papers on the coalescent process in population genetics, but I have not been able to track down their references.) Strictly speaking, everything is conditional on that starting point. Of this, more below.

Left unaddressed by the above is the question of how many paths we need to sample. Remember, the whole point is to not have to look at every possible path! If it turns out that accurate approximations to the likelihood require us to sample some substantial fraction, then this is all for nothing. However, the authors' figures reveal something quite remarkable. Whether N is 10 or 1000, the approximate likelihood changes very little (at least on a log scale), even with real data. This suggests that we don't, actually, need a lot of paths, but why?

For each i, $l^{(i)}(\theta)$ is an independent realization of a random variable, whose distribution depends only $\theta$ (holding fixed the driving chain, and the initial and final graphs). Since they are IID, we can apply the central limit theorem, which tells us that their mean should converge at rate $1/\sqrt{N}$ . Since that's noticeably smaller for N=1000 than for N=10, it must be the case that the variance of the likelihood along the individual sample paths is already pretty small. Why?

The lame physics answer is, "the principle of least action". There will be optimal, most-probable paths, and they will dominate the sum, the others tending to make negligible contributions. With high probability, a random sample will pick out the most probable paths. Q.E.D. This argument could, perhaps, be made less lame through an application of large deviations results, specifically conditional-limit-theorem- (or "Gibbs's conditioning principle"-) type results for Markov chains, which roughly say that if something improbable (a passage from $G_{t_0}$ to $G_t$ ) happens, it does so in the least-improbable possible way, and deviations from that trajectory are exponentially suppressed. <televangelist>In the name of Cramér, in the name of Varadhan, in the name of Freidlin and Wentzell, I call on you, Brother Argument, to be healed! Arise and prove! And, cf. Eyink (2000).</televangelist>

An information-theoretic answer is to invoke the asymptotic equipartition principle, a.k.a. the Shannon-McMillan-Breiman theorem. This says that if $X_1, X_2, \ldots X_t$ are generated according to a (well-behaved) stochastic process, whose distribution is $\mu$ , and $\theta$ is a sufficiently well-behaved model, then

$-\frac{1}{t}\log{L(\theta,X_1^t)} \rightarrow h(\mu) + d(\mu,\theta), ~\mu-a.s.$
where $h(\mu)$ is the entropy rate of the data-generating process $\mu$ , and $d(\mu,\theta)$ is the relative entropy rate between the data source and the model, i.e., the asymptotic growth rate of the Kullback-Leibler divergence. (For details, see Algoet and Cover, 1988, or Gray, 1990). So
$\log{L(\theta)} = - (h+d(\theta))t + o(t), ~\mu-a.s.$
In words, there are only two kinds of sample long sample paths: those which aren't generated at all, and those where the normalized log-likelihood are equal, at least to first order. It's not clear to me here whether t is big enough in the examples for this effect to kick in.

The biggest unclarity of all, probably, is the role of $G_{t_0}$ . Recall that we reached this by removing nodes which could have been added by pure duplication. There is, however, no particular reason to think that the actual growth of the graph ever passed through this state. It has the advantage of giving us a unique starting point for the chain, but there are, potentially, others. One, of course, is the trivial network consisting of a single node! Another possibility (which came up in the discussion, I think mostly due to Anna Goldenberg) is to first remove potential duplicates, as the authors do, and then remove nodes which have only one link to them, as clear attachments. This process of unwinding the attachments could potentially be iterated, until no "danglers" are left. This, too, is a uniquely-defined point. We can then go back to removing nodes which are, now, potential duplicates, and so on. Someone (I forget who) suggested that this might always terminate at the one-node network; it would be nice to either show this or give a counter-example. But if there is some principled reason, other than tractability, to use their $G_{t_0}$ , I can't figure out what it is from this paper.

Only using a growing network, and in particular only focusing on growth through duplication, certainly simplifies the computational problem, by reducing the number of possible paths which could terminate in the observed graph. Deletion of nodes and edges is however going to be very important in more biologically-plausible models, to say nothing of models of social or technological networks. Presumably the trick of using a backward-looking chain which stops when it hits a unique starting configuration could still be used with deletions --- I think the authors are hinting as much in their conclusion --- but it's not clear to me that a unique starting point is appropriate. With biological interaction networks, for example, one might argue that, e.g., metazoans have been around for a long time, so the distribution of networks ought to be close to stationary, and so starting configurations should be drawn from an invariant distribution of the appropriate chain...

This raises two further points, which are not un-related: the asymptotics of the DA model, and the biological utility of such a model. Run for a long time, the DA model will produce graphs of unbounded size, but it's not immediately obvious what these graphs will look like. In particular, what will be their degree distribution? The Barabasi-Albert model (Albert and Barabasi, 2002) produces scale-free distributions, <boosterism>because it uses the same mechanism as Herbert Simon's classic paper</boosterism> (Bornholdt and Ebel, 2001). This relies on a rich-get-richer dynamic, where nodes with high degree are more likely to attract new edges. My initial thought was that this wasn't present in the DA model, because targets for attachment and targets for duplication are both chosen uniformly. However, someone in the discussion --- I think, though I may be mis-remembering, that it was Tanzy Love --- pointed out that while high-degree nodes are no more likely to be copied than low-degree nodes, edges into high-degree nodes are more likely to be copied than edges into low-degree nodes. This is because if a node has degree k, there are k other nodes whose duplicated could end up linking to it. It may even be the case that this is falls under the theorems in Simon... Presumably the asymptotics would only become harder to handle if we added events deleting nodes or edges.

As for the biological utility, I'll repeat that none of the nodes have any identity of their own; only their role in the network of relations represented by the edges has any bearing on the model. "If this be structuralism, make the most of it": by turning it into a neutral model for the evolution of biological networks. After all, there is no reason here to duplicate certain nodes or edges, it's all just uniform chance. One key use of neutral models is to provide a background against which to detect adaptation; how could we do that here?

References:
Albert, Réka and Albert-László Barabási (2002), "Statistical Mechanics of Networks", Reviews of Modern Physics 74 (2002): 47--97 = cond-mat/0106096
Algoet, Paul H., and Thomas M. Cover (1988), "A Sandwich Proof of the Shannon-McMillan-Breiman Theorem", The Annals of Probability 16: 899--909
Billingsley, Patrick (1961). Statistical Inference for Markov Processes. Statistical Research Monographs, vol. 2. Chicago: University of Chicago Press.
Bornholdt, Stefan and Holger Ebel (2001), "World-Wide Web scaling exponent from Simon's 1955 model", Physical Review E 64: 035104 = cond-mat/0008465
Eyink, Gregory L. (2000), "A Variational Formulation of Optimal Nonlinear Estimation", Methodology and Computing in Applied Probability (submitted) = physics/0011049
Gray, Robert M. (1990), Entropy and Information Theory. Berlin: Springer-Verlag. Full text free online.
Simon, Herbert A. (1955), "On a Class of Skew Distribution Functions", Biometrika 42: 425--440
Solé Ricard V., Romualdo Pastor-Satorras, Eric Smith and Thomas B. Kepler (2002), "A model of large-scale proteome evolution", Advances in Complex Systems 5 (2002): 43--54 = cond-mat/0207311
Vázquez, A. and A. Flammini and A. Maritan and A. Vespignani (2003), "Modeling of protein interaction networks", Complexus 1: 38--44 = cond-mat/0108043

Posted by crshalizi at December 01, 2006 08:15 | | permanent link

November 29, 2006

Again with the Statistics 754, Stochastic Processes

Speaking, as I was, of this course, I've just spruced up the course homepage in anticipation of the coming semester. Comments, particularly on last year's lecture notes, are most welcome. (Reader M.H. has already won a free copy of the book, if it ever materializes, for his very helpful suggestions.)

Also, I have made a promise to try to post at least once a week, so that certain people know I'm alive (or at least that one of my scripts is still running).

Posted by crshalizi at November 29, 2006 02:40 | | permanent link

November 27, 2006

I Taught Him Everything He Knows

"Concentration of measure" is a phenomenon in probability theory where, roughly speaking, any set which contains a substantial fraction of the probability can be expanded just a little to yield a set containing most of the probability. Another way to say this is that, given any reasonably continuous function, the probability that it deviates from its mean is exponentially small, and the exponential rate does not depend on the precise function. This makes concentration of measure results extremely useful for questions involving the estimation of complicated and ugly functions. The classical work in this area proves concentration-of-measure for various kinds of sequences of independent variables, but for real applications in statistics, machine learning or physics you'd want to be able to handle dependence. The natural way to do this would be to look at mixing processes, which are at least asymptotically independent.

Leo Kontorovich, who was one of the students in my stochastic processes class this past spring, now has a paper summarizing his work on, precisely, concentration of measure for mixing sequences:

Leonid Kontorovich, "Metric and Mixing Sufficient Conditions for Concentration of Measure", math.PR/0610427: Abstract: We derive sufficient conditions for a family $(X^n,\rho_n,P_n)$ of metric probability spaces to have the measure concentration property. Specifically, if the sequence $\{P_n\}$ of probability measures satisfies a strong mixing condition (which we call $\eta$-mixing) and the sequence of metrics $\{\rho_n\}$ is what we call $\Psi$-dominated, we show that $(X^n,\rho_n,P_n)$ is a normal Levy family. We establish these properties for some metric probability spaces, including the possibly novel $X=[0,1]$, $\rho_n=\ell_1$ case.

This paper serves as a good entry-point to Leo's earlier papers on measure concentration for hidden Markov processes (math.PR/0608064) and Markov tree processes (math.PR/0608511), as well as his paper with Kavita Ramanan on the general martingale-difference method of which these other results are special cases (math.PR/0609835).

Since I'll be teaching stochastic processes again in the spring, I would very much like to claim that Leo wrote these papers as a direct result of having taken my class. But the truth is that Leo knew so much about this already that, so far teaching him everything he knows, I learned almost all I know about concentration from him. But this is one of the real pleasures of teaching...

Posted by crshalizi at November 27, 2006 22:57 | | permanent link

Lecture Notes on Stochastic Processes (Advanced Probability II)

I've started putting the notes for my lectures on stochastic processes (36-754) online at the course homepage.

Contents: Table of contents, which gives a running list of definitions, lemmas, theorems, etc. This will be updated with each new lecture.
Lecture 1 (16 January): Definition of stochastic processes, examples, random functions
Lecture 2 (18 January): Finite-dimensional distributions (FDDs) of a process, consistency of a family of FDDs, theorems of Daniell and Kolmogorov on extending consistent families to processes
Lecture 3 (20 January): Probability kernels and regular conditional probabilities, extendings finite-dimensional distributions defined recursively through kernels to processes (the Ionescu Tulcea theorem).
Homework Assignment 1 (due 27 January): Exercise 1.1; Exercise 3.1. Solutions.
Lecture 4 (23 January): One-paramater processes and their representation by shift-operator semi-groups.
Lecture 5 (25 January): Three kinds of stationarity, the relationship between strong stationarity and measure-preserving transformations (especially shifts).
Lecture 6 (27 January): Reminders about filtrations and optional times, definitions of various sorts of waiting times, and Kac's Recurrence Theorem.
Homework Assigment 2 (due 6 February): Exercise 5.3; Exercise 6.1; Exercise 6.2. Solutions
Lecture 7 (30 January): Kinds of continuity, versions of stochastic processes, difficulties of continuity, the notion of a separable random function.
Lecture 8 (1 February): Existence of separable modifications of stochastic processes, conditions for the existence of measurable, cadlag and continuous modifications.
Lecture 9 (3 February): Markov processes and their transition-probability semi-groups.
Lecture 10 (6 February): Markov processes as transformed IID noise; Markov processes as operator semi-groups on function spaces.
Lecture 11 (8 February): Examples of Markov processes (Wiener process and the logistic map). Overlaps with solutions to the second homework assignment.
10 February: Material from section 2 of lecture 10, plus an excursion into sofic processes.
Lecture 12 (13 February): Generators of homogeneous Markov processes, analogy with exponential functions.
Lecture 13 (15 February): The strong Markov property and the martingale problem.
Homework Assignment 3 (due 20 February): Exercises 10.1 and 10.2
Lecture 14 (17, 20 February): Feller processes, and an example of a Markov process which isn't strongly Markovian.
Lecture 15 (24 February, 1 March): Convergence in distribution of cadlag processes, convergence of Feller processes, approximation of differential equations by Markov processes.
Lecture 16 (3 March): Convergence of random walks to Wiener processes.
Homework Assignment 4 (due 13 March): Exercise 16.1, 16.2 and 16.4.
Lecture 17 (6 March): Diffusions, Wiener measure, non-differentiability of almost all continuous curves.
Lecture 18 (8 March): Stochastic integrals: heuristic approach via Euler's method, rigorous approach.
Lecture 19 (20, 21, 22 and 24 March): Examples of stochastic integrals. Ito's formula for change of variables. Stochastic differential equations, existence and uniqueness of solutions. Physical Brownian motion: the Langevin equation, Ornstein-Uhlenbeck processes.
Lecture 20 (27 March): More on SDEs: diffusions, forward (Fokker-Planck) and backward equations. White noise.
Lecture 21 (29, 31 March): Spectral analysis; how the white noise lost its color. Mean-square ergodicity.
Lecture 22 (3 April): Small-noise limits for SDEs: convergence in probability to ODEs, and our first large-deviations calculations.
Lecture 23 (5 April): Introduction to ergodic properties and invariance.
Lecture 24 (7 April): The almost-sure (Birkhoff) ergodic theorem.
Lecture 25 (10 April): Metric transitivity. Examples of ergodic processes. Preliminaries on ergodic decompositions.
Lecture 26 (12 April): Ergodic decompositions. Ergodic components as minimal sufficient statistics.
Lecture 27 (14 April): Mixing. Weak convergence of distribution and decay of correlations. Central limit theorem for strongly mixing sequences.
Lecture 28 (17 April): Introduction to information theory. Relations between Shannon entropy, relative entropy/Kullback-Leibler divergence, expected likelihood and Fisher information.
Lecture 29 (24 April): Entropy rate. The asymptotic equipartition property, a.k.a. the Shannon-MacMillan-Breiman theorem, a.k.a. the entropy ergodic theorem. Asymptotic likelihoods.
Lecture 30 (26 April): General theory of large deviations. Large deviations principles and rate functions; Varadhan's Lemma. Breeding LDPs: contraction principle, "exponential tilting", Bryc's Theorem, projective limits.
Lecture 31 (28 April): IID large deviations: cumulant generating functions, Legendre's transform, the return of relative entropy. Cramer's theorem on large deviations of empirical means. Sanov's theorem on large deviations of empirical measures. Process-level large deviations.
Lecture 32 (1 May): Large deviations for Markov sequences through exponential-family densities.
Lecture 33 (2 May): Large deviations in hypothesis testing and parameter estimation.
Lecture 34 (3 May): Large deviations for weakly-dependent sequences (Gartner-Ellis theorem).
Lecture 35 (5 May): Large deviations of stochastic differential equations in the small-noise limit (Freidlin-Wentzell theory).
References: The bibliography, currently confined to works explicitly cited.

In the staggeringly-unlikely event that anyone wants to keep track of the course by RSS, this should do the trick.

Posted by crshalizi at November 27, 2006 22:28 | | permanent link

Books to Read While the Algae Grow in Your Fur, October 2006

Ray Bradbury, Something Wicked This Way Comes: Why is it that every time I re-read this, the boys seem a bit more remote, and the old man a bit closer?
Stephen King, Cycle of the Werewolf
Karen Rose Cercone, Steel Ashes: First in the series of which I reviewed one book (Blood Tracks) at length. There was a third, Coal Bones, but apparently the series never went any further. This is a shame: they're good, fun, books, and after a bit more than a year of living in Pittsburgh I enjoy them even more. (For instance, Amberson Avenue, where one of the villains in Steel Ashes lives, is two blocks from my house.)
Karin Slaughter, Triptych: Not part of her regular series. To say anything beyond what you can read on the cover blurb would spoil an intricately-constructed series of surprises, so I won't. Like all Slaughter's books, good but disturbing.
Stephen Biddle, Military Power: Explaining Victory and Defeat in Modern Battle: Or: the "revolution in military affairs" happened in 1918. More exactly, even early 20th century weaponry, used at capacity, sufficed to create an utterly leathal "storm of steel". What Biddle calls the "modern system" consists of the tactics and operational skills which let armies nonetheless survive and manuveur on the battlefield --- extensive exploitation of all available cover while moving, detached units, combined arms, defense in depth, etc., etc. Biddle's argument is that success in mastering the modern system goes a lot further towards explaining who wins battles than does superiority in resources and materiel, or even superior technology per se. He makes his case through nicely-selected case studies, statistical studies on what systematic data exists, and some not-crazy simulations. (The statistical studies are definitely good by the standards of applied social science, but they use regression, rather than some method more appropriate to causal inference.) Biddle sounds extremely plausible to me, i.e., someone who admittedly knows nothing about military science. One of the big limitations, though, is that he's explicitly confining himself to land warfare among regular armies --- nothing about urban warfare, guerrillas, etc...
Terry Pratchett, The Amazing Maurice and His Educated Rodents; The Wee Free Men; A Hat Full of Sky: Strongly recommended for weekends when you are feeling glum and mildly ill. Many thanks to "Uncle Jan" for copies!

Posted by crshalizi at November 27, 2006 11:42 | | permanent link

October 22, 2006

Continuing the Flow of Brainy Material

Two more publications are now available (but were done some time ago).

One is a fairly straightforward paper, which you can get from arxiv.org.

Matthew J. Berryman, Scott W. Coussens, Cosma Shalizi, Yvonne Pamula, David Parsons, Kurt Lushington, David Saint, Andrew Allison, A. James Martin, Declan Kennedy and Derek Abbott, "Nonlinear Aspects of EEG Signals from Sleep Patients", pp. 40--48 in Fluctuations and Noise in Biological, Biophysical, and Biomedical Systems III (Bellingham, Washington: SPIE, 2005); q-bio.NC/0506015: Abstract: Electroencephalograph (EEG) analysis enables the neuronal behavior of a section of the brain to be examined. If the behavior is nonlinear then nonlinear tools can be used to glean information on brain behavior, and aid in the diagnosis of sleep abnormalities such as obstructive sleep apnea syndrome (OSAS). In this paper the sleep EEGs of a set of normal and mild OSAS children are evaluated for nonlinear behaviour. We consider how the behaviour of the brain changes with sleep stage and between normal and OSAS children.

This paper grows out of work Matthew did in Ann Arbor, when he visited last summer. (Our summer, not his.)

The other is a little more convoluted.

Cosma Rohilla Shalizi, "Functionalism, emergence, and collective coordinates: A statistical physics perspective on 'What to say to a skeptical metaphysician'", Behavioral and Brain Sciences 27 (2004): 635--636

This is part of the peer commentary on Don Ross and David Spurrett's target paper, "What to Say to a Skeptical Metaphysician: A Defense Manual for Cognitive and Behavioral Scientists" (pp. 603--627, same issue). The commentary (pp. 627--637) is followed by Ross and Spurrett's intelligent reply to comments (pp. 637--647). My contribution isn't particularly readable outside of that context, which of course the journal doesn't make publicly available.

Now, back to work.

Minds, Brains, and Neurons; Complexity; Physics; Philosophy; Self-Centered

Posted by crshalizi at October 22, 2006 16:42 | | permanent link

Agathidium cheneyi

When considering slime molds, about which I hope to have more to say soon, we should not forget their natural predators [cache]. (Via Matthew Berryman, in e-mail.)

Posted by crshalizi at October 22, 2006 16:42 | | permanent link

Return of Friday Cat-Blogging (Science Geek Edition)

John M. Abbott and Michael C. Abbott, "Determination of cat tongue length using the 'White Wine' method" [link]: Abstract: The "White Wine" method is a novel technique for feline tongue length determination, making use of the latest advances in digital technology. It supersedes the earlier more labour-intensive "Beefy Bovril" method [1] and the "Creamy Yoghurt" method [2].

The results are striking:

(Via Matthew Berryman in e-mail)

Friday Cat Blogging

Posted by crshalizi at October 22, 2006 16:42 | | permanent link

October 15, 2006

Books to Read While the Algae Grow in Your Fur, September 2006

E. R. Chamberlin, The Bad Popes: Not, sadly, all of the bad popes, but rather a selection of seven of (to Chamberlin's mind) the worst.
Max Beerbohm, Seven Men: Many thanks to John Burke for recommending this wonderfully funny, yet surprisingly creepy, little book. (The story of one of the six men, "'Savonarola' Brown", is online.)
Sheri Berman, The Primacy of Politics: Social Democracy and the Making of Europe's Twentieth Century: A highly interesting book which nonetheless does not deliver what it promises. I will defer detailed comments to the forthcoming Crooked Timber symposium.
Steven Runciman, Byzantine Civilization: Short, introductory history of an era of which I am largely ignorant; so, I'm not actually sure I have any grounds to recommend it, except that I feel less ignorant now, and it's well-written. Definitely shows its age, however; nobody, surely, would now worry over "Hellenistic" versus "Aramaean" influences in art?
Anthony Shadid, Night Draws Near: Iraq's People in the Shadow of America's War: For America, the war is merely a politico-military disaster and thousands of causalties; for the Iraqis, it is infinitely worse. An intelligent presentation of how the war looked to a wide spread of Arab Iraqis, by a very good Arabic-speaking American journalist.

Corrupting the Young; Enigmas of Chance

Posted by crshalizi at October 15, 2006 20:12 | | permanent link

Data Mining (36-350) Lecture Notes, Weeks 4--7

These handouts are ~~shamelessly ripped off~~ derivative work, amplifying and expanding those created by Tom Minka when he invented this course. (See his originals here.) Posted here in response to a number (> 1) of requests. See here for the first three weeks' handouts.

Note to students in 36-350: This page will not keep up to date with the handouts, or with other course documents; use Blackboard!

September 20 and 25 (Lecture 6): Partitioning Data into Clusters. Supervised and unsupervised learning. Social and organizational aspects of categorization. Finding categories in data via clustering. Characteristics of good clusters. The k-means algorithm for clustering. Search algorithms, search landscapes, hill climbing, local minima. Algorithms for hierarchical clustering. Avoiding spherical clusters. See also: slides to accompany the second half, showing clustering of images.
September 27 (Lecture 7): Making Better Features. Transforming features to enhance invariance. Transforming features to improve their distribution. Projecting high-dimensional data into lower dimensions. Principal component analysis: informal description and example.
October 2 (Lecture 8): More on Principal Component Analysis. Mathematical basis: maximizing the variance of the projected points. Mathematical basis: minimizing reconstruction error. Interpretation of PCA results.
October 4: Review of course to date. (No handout.)
October 9 (Lecture 9): Evaluating Predictive Models. Classification and linear regression as examples of predictive modeling. Error measures a.k.a. loss functions; examples. In-sample error. Out-of-sample or generalization error; why it matters, relation to in-sample error. Model selection. An example of over-fitting. Approaches to limiting over-fitting and its ill effects.
October 11 (Lecture 10): Regression Trees. Difficulties of fitting global models in complex systems. Recursive partitioning and simple local models as a solution. Prediction trees in general. Regression trees in particular. An example. Tree growing. Tree pruning via cross-validation.

Posted by crshalizi at October 15, 2006 19:57 | | permanent link

October 12, 2006

"The Invisible Academy: Non-Linear Effects of Linear Learning"

I am sure that I speak for all of us in the Statistics Department at CMU (especially my erstwhile fellow bloggers) when I say we're very pleased and excited to have Prof. Mark Liberman as our seminar speaker on 16 October, a week from tomorrow. Mark is coming to us not as the impressario of LanguageLog (though, in that role, his scourging of sloppy data analysis, whether socio-political or neurosexual, is a joy to behold), nor as the director of the Linguistic Data Consortium, nor even to speak about bibliomics. Rather, he'll be talking about some work which combines the stochastic linear learning models (of the sort pioneered by, among others, Frederick Mosteller) with agent-based modeling of cultural evolution. Which is to say, he's talking about aggregate behaviors of interacting stochastic processes which are more interesting than just the central limit theorem.

The invisible academy: non-linear effects of linear learning: Abstract: When linguists, psychologists or engineers try to understand, explain or imitate human speech and language, they usually do so by modeling individual speakers, hearers or learners. Nevertheless, language is an emergent property of groups (of humans), and elementary arguments suggest that non-trivial characteristics of speech and language emerge from interactions within groups of individuals over time. This talk will argue that some (old and pathetically) simple ideas about learning and perception have obvious but non-trivial consequences for the emergence of cultural norms, including linguistic ones.

Monday, October 16, at 4:30 pm in Baker Hall A51; free and open to the public.

The Collective Use and Evolution of Concepts; Enigmas of Chance

Posted by crshalizi at October 12, 2006 11:42 | | permanent link

October 08, 2006

It's Not Who You Know, It's What You Do

Blog near-silence will continue until bootstrap testing results improve. In the meanwhile, it occurs to me that I haven't mentioned that I have a new paper.

CRS, Marcelo F. Camperi and Kristina Lisa Klinkner, "Discovering Functional Communities in Dynamical Networks", q-bio.NC/0609008, forthcoming in the proceedings of the ICML 2006 workshop on "Statistical Network Analysis: Models, Issues and New Directions" (Springer Lecture Notes in Computer Science): Abstract: Many networks are important because they are substrates for dynamical systems, and their pattern of functional connectivity can itself be dynamic — they can functionally reorganize, even if their underlying anatomical structure remains fixed. However, the recent rapid progress in discovering the community structure of networks has overwhelmingly focused on that constant anatomical connectivity. In this paper, we lay out the problem of discovering functional communities, and describe an approach to doing so. This method combines recent work on measuring information sharing across stochastic networks with an existing and successful community-discovery algorithm for weighted networks. We illustrate it with an application to a large biophysical model of the transition from beta to gamma rhythms in the hippocampus.

Thanks — over and above the acknowledgments in the paper &mdash are due to Jörg Reichardt for help implementing his algorithm, and to Anna Goldenberg for very thorough (not to mention patient!) editing, resulting in a much better paper.

There are about a zillion possible extensions and applications, which is a good note on which to get back to work.

Networks; Self-Centered; Minds, Brains, and Neurons

Posted by crshalizi at October 08, 2006 13:47 | | permanent link

October 02, 2006

We Can Have a Better Academic Publishing System

Paul "arXiv" Ginsparg, evangelizing in the pages of the latest issue of The Journal of Neuroscience, under the rubric "As We May Read":

The e-print arXiv (http://arXiv.org/), initiated in August 1991, has effectively transformed the research communication infrastructure of multiple fields of physics and could play a prominent role in a unified set of global resources for physics, mathematics, and computer science. It has grown to contain >375,000 articles (as of July 2006), with >50,000 new submissions expected in calendar year 2006 and >40,000,000 full-text downloads per year. It is an international project, with dedicated mirror sites in 17 countries and collaborations with United States and foreign professional societies and other international organizations, and it has also provided a crucial lifeline for isolated researchers in developing countries...
The arXiv is entirely scientist driven: articles are deposited by researchers when they choose (either before, simultaneous with, or after peer review), and the articles are immediately available to researchers throughout the world. As a pure dissemination system, it operates at a factor of 100-1000 times lower in cost than a conventionally peer-reviewed system.... This is the real lesson of the move to electronic formats and distribution: not that everything should somehow be free, but that with many of the production tasks automatable or off-loadable to the authors, the editorial costs will then dominate the costs of an unreviewed distribution system by many orders of magnitude. ...
The site has never been a random Usenet newsgroup- or blogspace-like free-for-all. From the outset, arXiv.org relied on a variety of heuristic screening mechanisms ... to ensure insofar as possible that submissions are at least "of refereeable quality." This means that they satisfy the minimal criterion, that they would not be peremptorily rejected by any competent journal editor as nutty, offensive, or otherwise manifestly inappropriate, and they would instead at least in principle be suitable for review. These mechanisms are an important, if not essential, component of why readers find the arXiv site so useful. ...
The arXiv repository functions are flexible enough either to coexist with the preexisting publication system or to help it evolve into something better optimized for researcher needs. Although there are no comprehensive editorial operations administered by the site, the vast majority of the 50,000 new articles per year are nonetheless subject to some form of review, whether by journals, conference organizers, or thesis committees. Physics and astronomy journals have learned to take active advantage of the availability of the materials before journal publication ...
On the one-decade time scale, it is likely that more research communities will join some form of global unified archive system without the current partitioning and access restrictions familiar from the paper medium, for the simple reason that it is the best way to communicate knowledge and hence to create new knowledge. Ironically, it is also possible that the technology of the 21st century will allow the traditional players from a century ago, namely the professional societies and institutional libraries, to return to their dominant role in support of the research enterprise.

Ginsparg's title is a riff on Vannevar Bush's "As We May Think"; hopefully it will not take 48 years for these suggestions to be widely implemented.

Enigmas of Chance; Complexity

Posted by crshalizi at October 02, 2006 17:53 | | permanent link

September 29, 2006

Gauss Is Not Mocked

By now, everyone and her brother has read, or at least read about, the papers by Albert-László Barabási and co., purporting to show that response times in e-mail, and in Darwin and Einstein's correspondence, follow a power law distribution, and that this is due to queuing processes.

Unfortunately, this is not true; the apparent power law is merely an artifact of a bad analysis of the data, which which is immensely better described by a log-normal distribution. (Via Aaron Clauset.)

Daniel B. Stouffer, R. Dean Malmgren and Luís A. N. Amaral, "Comment on Barabási, Nature 435, 207 (2005)", physics/0510216: Abstract: In a recent letter, Barabási claims that the dynamics of a number of human activities are scale-free [1]. He specifically reports that the probability distribution of time intervals tau between consecutive e-mails sent by a single user and time delays for e-mail replies follow a power-law with an exponent -1, and proposes a priority-queuing process as an explanation of the bursty nature of human activity. Here, we quantitatively demonstrate that the reported power-law distributions are solely an artifact of the analysis of the empirical data and that the proposed model is not representative of e-mail communication patterns.; Authors' comment: This manuscript re-analyzes data from Barabási's paper in Nature, "The origins of bursts and heavy tails in human dynamics", but it should be clear that the same problems are to be found in physics/0510117 and the upcoming Nature advertised in Barabási's web site concerning the correspondence of Einstein and Darwin

As every school-child knows (at least, these school-children do!), adding together many independent random variables, each of which makes a small contribution to the over-all result, generally gives you a Gaussian or normal distribution (unless the contributing variables are, themselves, kind of pathological). This fact is the central limit theorem.

What happens if the inputs are multiplied together, rather than added? Well, take the logarithm: log(XY) = log(X) + log(Y). The logarithm of the product will be the sum of the logarithms of the inputs. The latter will still be independent, so the logarithm of the output will be normally distributed. Undoing the log gives what's imaginative called the log-normal distribution. Log-normals are very common, for the same reasons that normals are. Unlike normals, they are very easy to mistake for power law distributions, especially if your knowledge of statistics is as limited as most theoretical physicists'. (The distribution of links to weblogs, for instance, is much better fit by a log-normal than a power law, as we've seen.) In their comment, Stoffer et al. show that a log-normal actually gives a textbook-quality fit to Barabási's data. (The only change I'd make to their procedure is that I'd report the likelihood ratio directly, and let people work out their own Bayesian posteriors if so inclined.) Looking at the data reported in the new Nature paper on Darwin's and Einstein's correspondence, if it's not log-normal too — well, I'd say I'd eat my hat, but I don't own one; I'll buy a Notre Dame hat and eat it.

Let me turn the microphone over to Francis Galton (as quoted in Ian Hacking's The Taming of Chance):

I know of scarcely anything so apt to impress the imagination as the wonderful form of cosmic order expressed by `the law of error.' A savage, if could understand it, would worship it as a god. It reigns with severity in complete self-effacement amidst the wildest confusion. The huger the mob and the greater the anarchy the more perfect its sway. Let a large sample of chaotic elements be taken and marshalled in order of their magnitudes, and then, however wildly irregular they appeared, an unexpected and most beautiful form of regularity proves to have been present all along.

As Hacking notes, on further consideration Galton was even more impressed by the central limit theorem, and accordingly replaced the sentence about savages with "The law would have been personified by the Greeks and deified, if they had known of it." Whether deified by Hellenes or savages, however, the CLT has a message for those doing data analysis, and the message is:

Thou shalt have no other distribution before me, for I am a jealous limit theorem.

I restrain myself from making any observations on the editorial process at Nature, or on the competence of the referees of Barabási's papers. I do wish it to be noted, however, that this post is not an entry in the "Why Oh Why Can't Physicists Learn Better Probability and Statistics?" series, as Amaral and Barabási are both associated with Gene Stanley's school of statistical physics.

Update, Halloween: Suresh Venkatasubramanian, at Geomblog, turns his microphone over to Michael Mitzenmacher, who has some very good comments. (This led me to read Mitzenmacher's nice paper on generating mechanisms for power-laws.) I am more convinced by Mitzenmacher by the difference in the goodness of fits, simply because it is so overwhelmingly large. It hardly seems to make sense, in this case, to say that the data are even approximately power-law distributed...

Update, 23 November: Barabási's group has posted a reply (physics/0511186). To my eyes, the crucial observation by Stouffer et al. was that the fit of the data to a power law is in fact really, really bad, so it's pointless to talk about what mechanism might produce a power law in such situations. The reply's take on this point is that this is "merely" a statistical issue! In short, I don't find the reply at all convincing on the major points, but if you care, by all means read it. (The reply claims that Stouffer et al.'s comment was rejected by "three referees" at Nature; one wonders if they were the referees who approved Barabási's original paper.)

Update, 25 November: To hammer the point home, let's look at Figure 1b from Stouffer et al.'s comment. (Click for a larger version.)

The solid black line is the empirical distribution of the data. The red dashed line is the lognormal distribution. This is, as I said, a textbook-quality fit. Correcting for right censoring — the measured response intervals are all less than 83 days, because that's the length of time over which the data were collected &mdash would only improve the fit. (Thanks to Prof. Amaral for permission to reproduce the figure.)

Update, 29 November: Yet more commentary, from Aaron Clauset.

Update, 29 September 2006: In the event you still care about this, see G. Grinstein and R. Linsker, "Biased Diffusion and Universality in Model Queues", Physical Review Letters (2006): 130201. Grinstein and Linsker analytically solve for the asymptotic distribution of Barabási's queueing model, finding either a power law or a power-law with an exponential cut-off; they also show that the result is very sensitive to introducing a cost for switching between different kinds of tasks.

Manual trackback: In Search of 42; Pharyngula; hakank.blogg; Juan de Mairena [v.2.718]; Three Quarks Daily; Metamerist; Zoltán Sylvester; Language Log; Statistical Modeling, Causal Inference, and Social Science

Posted by crshalizi at September 29, 2006 19:06 | | permanent link

September 21, 2006

Seeking Advice on Introducing Ideological Bias

Well, it looks like I have some VIGRE-funded undergraduate research assistants for the semester; specifically, I have takers both for my idea about aggregated network data, and neutral cultural diffusion on assortative networks. The latter, however, leaves me with a puzzle.

The project makes sense (to the extent that it does) against a background assumption that culture is not neutral, but an adaptation to the physical or social environment. This is a pretty pervasive assumption on the part of social scientists, historians, ethnographers, cultural critics, etc., but I'm not sure that my students will have been exposed to this idea as such. So, I need background reading which will rapidly persuade a wholesome, technically-inclined Carnegie Mellon undergrad that a person's beliefs and values ought to be correlated with their status in society. "Rapidly" is important, because there is some technical research to get to, and recapitulating the whole history of the sociology of belief is not an option. Similarly, assigning The German Ideology and/or The Protestant Ethic and the Spirit of Capitalism seems like asking for trouble. (And, with Weber, I'd feel like I'd have to spend a lot of time unteaching the errors.) So, I invoke the collective wisdom of the Web for reading suggestions: please write me at cshalizi [at] cmu [dot] edu [dot] oryx, deleting the name of a genus of antelope.

Corrupting the Young

Posted by crshalizi at September 21, 2006 08:43 | | permanent link

September 20, 2006

Arrrrgh! Owwwwwww! Noooooooooo! (A Remark on Power Laws)

Via Language Log, there comes this fun post on the distribution of the number of repetitions of "R" in strings of the form "AR+", as in, "Arrr, mateys!" These findings, like Mark Liberman's on "AW+", are in line with the results of the seminal paper in this area, Dennis Chao and Patrik D'haeseleer's "The Distribution of Variable-length Phatic Interjectives on the World Wide Web" (University of New Mexico Computer Science Department Tech Report TR-CS-2001-23). I eagerly await further results in this exciting pico-field.

Being what I am, however, I can't resist pointing out that looking for a straight line on a log-log plot, and even finding one with high r-squared, is simply not a reliable way of checking whether a distribution is a power-law. Please do not do this. (And yes, I should be finishing that paper on the right approach, rather than blogging.)

Learned Folly; Corrupting the Young; The Progressive Forces; Enigmas of Chance; Afghanistan and Central Asia

Posted by crshalizi at September 20, 2006 16:21 | | permanent link

A Triumph of Socialist Realism

My copy of Michael Bérubé's What's Liberal About the Liberal Arts? arrived in the mail yesterday (thanks to the unionized workers at Powell's), only to coincide with the release of the graphic novel (4Mb PDF). As a left-liberal professor, I can proudly testify to the complete and utter accuracy of this document's depiction of our revolutionary praxis. (At least, when we are not teaching fluffy courses about enchiladas.) My students should take notes, because this will be on the exam, particularly the bits about camels.

However, What's Liberal About the Liberal Arts? The Graphic Novel shows me that I must also engage in self-criticism. I have been complacent about enforcing political correctness, behaving as though statistics and machine learning could be taught in a neutral manner, when in reality every classroom is a site of conflict. In particular, I have been insufficiently vigorous in struggling against my students' tendencies to embrace borgeois individualism, in the form of the theory of merely personal, subjective and idealist probability associated with a clerical ideologue of British imperialism. I see that I must double and re-double my efforts to indoctrinate them in the principles of frequentist probability, based as it is on collectives and the work of social democrats. Might I be permitted to hope that comrade Gonick (also cited by comrade-professor Burke) will produe a proletarian, graphic version of the great red book epitomizing Peircism-Popperism-Neyman Pearson Thought?

(Via Pharyngula, Crooked Timber and John Burke [in e-mail] more or less at once)

Posted by crshalizi at September 20, 2006 13:28 | | permanent link

September 19, 2006

Shorter Rajiv Chandrasekaran

"Better red than expert".

(Some people managed to report this three years ago.)

The Continuing Crisis; The Running-Dogs of Reaction

Posted by crshalizi at September 19, 2006 13:49 | | permanent link

Piracy and Peer Production

In honor of International Talk Like a Pirate Day, I bring you:

Odd Ends gives us The Dead Men's Song: being the Story of a Poem and a Reminiscent Sketch of its Author Young Ewing Allison, Together with a Browse Through Other Gems of His and Recollections of Older Days, by Champion Ingraham Hitchcock, and the combination of lyrics and music for a certain Piratical Ballad. But when I say "Odd Ends", I really mean "Barbara Tozier and the whole cooperative peer-production crew at Distributed Proofreaders". (Which site is itself posting in a most pirate-like fashion today.) — I like a good anarchist utopia as much as the next guy raised on The Dispossessed and Mutual Aid, but it seems to me that Distributed Proofreaders is a much better prefiguration of that hope than the actual pirates, and those who pretend otherwise are silly.
George MacDonald Fraser's The Pyrates, which I actually think is funnier than the Flashman books. (However, the cover of the American edition does it no justice, for which you need the UK cover.)
Gary Farber blogging about Jewish pirates, including the notorious Jean Lafitte.
The Dread Pirate bin Laden, an argument for treating, under international law, as "common enemies of mankind", a category established long ago for pirates.
The Church of the Flying Spaghetti Monster's detailed correlation of global temperature and the number of pirates, a piece of climatology at least as sound as anything funded by Exxon.

Coincidentally, the campus movie club just screened the latest Disney pirate movie; at least I think that was what they were showing, since their poster showed a strong influence, in its iconography, of a similarly-titled but not quite so work-safe production.

Corrupting the Young; Enigmas of Chance

Posted by crshalizi at September 19, 2006 12:47 | | permanent link

Data Mining (36-350) Lecture Notes, Weeks 1--3

Lecture 5 is also a ~~shameless rip-off~~ explication of Aleks Jakulin's "Quantifying and Visualizing Attribute Interactions" (cs.AI/0308002).

Note to students in 36-350: This page will not keep up to date with the handouts, or with other course documents; use Blackboard!

Searching Documents by Similarity (28 August 2006). Why similarity search? Defining similarity and distance. The bag-of-words representation. Normalizations. Some results.
More on Similarity Search (30 August 2006). Stemming, linguistic issues. Picking out good features, or at least ignoring non-discriminative ones. Inverse document frequency. Using feedback from the searcher.
Searching Images by Similarity (6 September 2006). Representation and abstraction. How to search images without looking at images; a failure-mode. The bag-of-colors representation. More examples. Invariance and representation. See also: slides illustrating this lecture.
Finding Informative Features (11--13 September 2006). More on finding good features. Entropy and uncertainty. Information and entropy. Ranking features by informativeness. Examples.
Interactions Among Features (18 September 2006). Redundancy and enhancement of information. Information-sharing graphs. Examples.

Posted by crshalizi at September 19, 2006 12:28 | | permanent link

September 16, 2006

Glory and $500 (VIGRE-funded Undergraduate Research in Statistics at Carnegie Mellon)

Hey, kid! Interested in winning eternal intellectual glory and entering the glamorous world of scientific research? Interested in $500 for the semester? Are you an undergrad at Carnegie Mellon University? If so, the statistics department has no less than nine possible projects for you. (Some of them are mine; one began as a blog-post, another as a notebook entry.) Apply now!

Engimas of Chance; Corrupting the Young

Posted by crshalizi at September 16, 2006 12:58 | | permanent link

September 11, 2006

Monday Exam Blogging

In honor of the end of the semester and the arrival of spring, here is my first, and so far only, attempt at writing an exam. This was given as a take-home midterm by a friend teaching a statistics-for-people-who-don't-like-math course at a school which, to protect the innocent, I'll call the University of Winnemac. (It's a long story.) I'm fond of it, but the students at Winnemac hated it, and apparently none of them got any of the jokes. Solutions are available upon request. (Note that Problem 5 is a simplified version of the "Carnival Booth" algorithm due to Samidh Chakrabarti and Aaron Strauss.)

Take-Home Midterm, Generic Statistics Service Course, Generic Department, University of Winnemac at J. Random College Town

Problems are longer and harder than exercises; they also count for twice as much. Some questions are in multiple choice format, but you should always show your work.

Exercise 1: Average Models

A designer measures the height of a hundred models, randomly chosen from the runways in Milan. The sample mean height is 5.85 feet, with a sample standard deviation of 0.15 feet.
(a) What is the 95% confidence interval for the mean height, in feet, of Milanese models?
(b) What is the standard deviation of their height in inches?
(c) What is the confidence interval for their height in inches?
(d) What is the confidence interval (in feet or inches) for their height while wearing three-inch platform shoes?
(e) The standard deviation of their height in platforms?

Exercise 2: Statistics for Con Artists

Every week the market price for frozen concentrated orange juice (FCOJ) futures either goes up or down, with equal probability. A market analyst obtains a list of 1024 FCOJ traders. At the beginning of the year he sends them a letter announcing a free trial period of his new market prediction service; half the letters say the market will rise that week, and half that it will fall. The next week he discards the names of the traders to whom he made the wrong prediction, and repeats the process. Thus after k weeks, the remaining names on the list have received k correct predictions in a row for free.
(a) What is the probability that any given trader is still on the list after seven weeks?
(b) How many names are still on the list after seven weeks?

Exercise 3: Coins

What are the probabilities of getting the following sequences of heads and tails from 30 consecutive tosses of a fair coin?

HHHHHHHHHHHHHHHHHHHHHHHHHHHHHH
HTHTHTHTHTHTHTHTHTHTHTHTHTHTHT
HHTTTTHHTHHHHHTHHHTTHTHTTHTTTH

Exercise 4: Beating the Market

Mutual fund annual rates of return are normally distributed. What fraction of funds will have returns between one and three standard deviations above the mean in a given year?

Exercise 5: Cats and Their Humans

A survey of cats in J. Random College Town finds their weight is normally distributed, with a mean of 9 pounds and a standard deviation of 1.5 pounds. The same survey finds that the weight of cat owners is also normally distributed, with a mean of 150 pounds and a standard deviation of 15 pounds. Describe the distribution of the weight of cat-owners holding their cats, assuming feline and human weights are independent random variables.

Fun with Venn Diagrams

In Exercises 6--8, A, B and C are three events. P(A) = 0.75, P(B) = 0.65 and P(C) = 0.40. Note: Drawing Venn diagrams is not required to solve these problems, but it may help.

Exercise 6

(a) What is the smallest possible value of P(A and B)?
(b) The largest possible value?

Exercise 7

Which of the following statements could be true?

C is the complement of A
B completely contains A
C is the complement of (A and B)
C is the complement of (A or B)
A and B are mutually exclusive

Exercise 8

(a) If C is (A and B), what is P(B or A)?
(b) If C = (A and B), what is P(B|A)?

Exercise 9: Truly Great Poetry

In his book I Am Sickened by Your Ignorance, the critic Orpheus Bruno declared that "no more than one poem in ten thousand is truly great; the rest might as well be shopping lists". Bruno was subsequently abducted by renegade experimental psychologists and made to rate a large number of randomly-selected poems from 0 to 100, and also to say which ones were "truly great". He, of course, ignored the 0--100 scale entirely, preferring a boundless scale to mirror his boundless magnificence. His ratings, in fact, were normally distributed (or, as he said, "followed the law of the immortal Gauss"): the mean rating was 45, with a standard deviation of 16, and poems which scored 93 or above were "truly great". What is the proportion of truly great poetry?

Problem 1: Educational Testing

Professor Sheila Nagig of the Miskatonic University Department of Statistics refuses to give tests to her students, saying that most students who get high scores are just lucky, not knowledgeable, so the test isn't informative. Pressed by the Dean to explain herself, she argues as follows. Consider a test with 100 yes-or-no questions. A student's degree of knowledge of the subject (say, finite-temperature canonical quantum gravity) can be measured by the probability p of their answering a given question correctly. Assuming the questions are independent (which is the case on Prof. Nagig's tests), a student's score is therefore a binomial random variable, B(p, 100). Normally a passing score is 70% correct, or 70 questions out of 100. Note that someone who knows nothing and guesses completely at random has p = 0.5. (In the following, you may use the normal approximation if you wish.)
(a) What is the probability of a student passing if their p = 0.5?
(b) What is the probability of a student passing if their p = 0.7?
(c) Assume that one American in a million has a finite-temperature canonical quantum gravity p of 0.7, and the rest have p = 0.5, i.e., they know nothing about it. What is the probability that a random American who passes a test in the subject knows nothing about it?
(d) Explain what is wrong with Prof. Nagig's argument.

Problem 2: Glenn and Glenda Martingale and Their Angora Goats

Glenn and Glenda Martingale have a successful angora sweater business, and, in a fit of vertical integration, buy an angora goat farm. As you know, the most important trait of an angora goat is its fuzziness, measured in hairs per square millimeter. The Martingales, being statistically sophisticated, determine the fuzziness of their goats as follows. For each goat in the herd, fuzziness is measured at a random spot on its body, and then averaged across all goats in the herd. These are their results.

Sample mean fuzziness	10.30
Sample standard deviation	1.21
Low end of 95% confidence interval	9.99
High end of 95% confidence interval	10.61

Assume, like the Martingales, that fuzziness is normally distributed. Calculate the number of goats in the herd, i.e., the number of samples. Round to the nearest goat. You may assume that there are a lot of goats.

Extra credit. Can you re-do the calculation, without assuming the number of samples is large?

Problem 3: Glenn and Glenda Martingale, cont'd.

The International Group of Angora Fanciers (IGAF) stipulates that only wool from goats whose fuzziness is at least 9.09 can be used to make angora sweaters. Assume that the fuzziness is normally distributed with mean 10.30 and standard deviation 1.21 (as in the previous problem).
(a) What is the probability that a random goat on the farm is fuzzy enough for IGAF?
(b) What is the probability that at least 25 out of a random group of 30 goats meets the IGAF standard? Calculate this exactly, using the binomial distribution.
(c) Repeat the previous calculation using the normal approximation.

Problem 4: Cagliostro Consulting PLC

In his book Alchemical Management: Getting the Lead Out and the Gold In, Alex Cagliostro, the famous business consultant, profiles ten companies that achieved excellence after adopting his system of alchemical management (seminars available through appointment with Cagliostro Consulting PLC). The rival consultancy of Hooke, Waterhouse, Comstock and Root points out that there were seventy-two other companies which tried alchemical management without achieving excellence.
(a) Assuming these 82 firms form a representative sample (aside: is that reasonable?), calculate a 95% confidence interval for the proportion of excellent firms among alchemically-managed companies.
(b) It is known that 12% of all firms achieve excellence. Test HWCR's claim that excellence is no more common among alchemically-managed firms than among non-alchemical companies. State the null and alternative hypotheses. Is this a one-sided or two-sided test? Calculate the p-value.

Problem 5: Airport Screening

Airport security cannot give a detailed screening to everybody trying to fly by plane, so they select a fraction for detailed screening. Suppose there are four kinds of passengers: innocent-looking law-abiding citizens, suspicious-looking law-abiding citizens, suspicious-looking terrorists and innocent-looking terrorists. Security officials decide to screen all suspicious-looking passengers, and a random 2% of all innocent-looking people just to be safe. 10% of the total population is suspicious-looking, but 80% of all terrorists are.
(a) What is the probability that no one in a group of four random terrorists will be screened the next time they fly?
(b) Say a terrorist has evaded scrutiny if they have taken five flights without being screened once. What is the probability that a terrorist is innocent-looking, given that he has evaded scrutiny?
(c) Supposing one has a group of four terrorists who have all evaded scrutiny, what is the probability that none of them will be screened the next time they fly?
(d) Repeat the calculation in (c), supposing that airport security ignores who looks suspicious or innocent, and screens 12% of all passengers completely at random.

Self-Centered; Corrupting the Young; Enigmas of Chance

Posted by crshalizi at September 11, 2006 12:00 | | permanent link

A View from Incense-Burner Mountain

The atmosphere just outside Beijing is hot, humid, and almost unbelievably hazy. The atmosphere of the complex systems summer school where I'm teaching this week is, thankfully, rather different...

Corrupting the Young; Self-Centered; Complexity

Posted by crshalizi at September 11, 2006 12:00 | | permanent link

Will There Be a Text in My Class?

Attention conservation notice: An appeal to the reader's knowledge of textbooks on stochastic processes; also a plea not to be thrown into the briar-patch.

In the spring, I'm going to be teaching the department's advanced course on stochastic processes (36-754, for those keeping track at home). The catalogue description of the course reads, in full, as follows:

This course introduces advanced topics in Probability Theory such as Brownian motion, Markov processes, stationary processes, stochastic integration, etc.

It's intended for students from math or statistics who've had a first course in measure-theoretic probability, such as our 36-752, which goes up through the laws of large numbers for independent variables, a little martingale theory, and the central limit theorem. Most if not all of them will have already had a course on stochastic processes at the level of Grimmett and Stirzaker. My plan is to take advantage of the "etc." in the description, and teach a course on ~~my favorite topics~~ some interesting and highly useful material which is perhaps otherwise under-emphasized. To be more concrete, I'd like to start with the Wiener process, stochastic calculus and stochastic differential equations (which is pretty standard), but then do a lot on ergodic theory and mixing, Markov operators and their asymptotics, large deviations, information theory (as it connects to hypothesis testing and the like, not so much to coding), and spatial processes, including the Hammersley-Clifford (-Griffeath-Grimmett-Preston-...) theorem. Ideally, we'd end with some interesting spatio-temporal models, e.g. cellular automata or interacting particle systems. Markovian representations of non-Markovian processes (per Knight, rediscovered by various smart computer scientists and some confused physicists) would also be nice, but maybe too much. (Since any one of those topics could be stretched to cover a semester, the whole class is too much!)

I am looking for a textbook which covers all of this, or at least most of it; I'd be willing to change the material to match a good text. The students currently in 752 are using Ash and Doleans-Dade, which is good, and the last two chapters (which they won't get to) introduce a little ergodic theory and a little stochastic calculus, respectively, but not in enough depth. No one book I know seems to fit, and making them buy more than one expensive book doesn't seem right. If you have any suggestions, please mail them to me at cshalizi [at] oryx [dot] cmu [dot] edu (removing the name of a genus of antelope, which is there only to confuse spammers). I am going to have to spend a lot of time on my lecture notes; I really don't want that to have to grow into, in effect, writing my own book.

Update, next day: Thanks to Bill Tozier, Anand Sarwate and Wolfgang Beirl for writing with suggestions. Wolfgang, in particular, pointed me to Alexandre Stefanov's useful collection of online probability texts and notes (part of a bigger collection of mathematics resources). One of these, Robert Gray's Probability, Random Processes and Ergodic Properties, is something I was already planning to mine, along with his Entropy and Information Theory.

Update, Halloween: We will be using Olav Kallenberg's Foundations of Modern Probability as a reference, with the primary text being my lecture notes.

Manual trackback: Nothing Funny About Feldspar

Posted by crshalizi at September 11, 2006 12:00 | | permanent link

Statistics 754, Stochastic Processes (Advanced Probability II)

General course information (includes gratuitous eye-candy), syllabus, detailed course outline. To ward off the evil eye, I should say that I do not expect to get through all of what's in the outline. (Suggestions about the outline are still most welcome.) Lecture notes will be posted on the course website as I write them. Thanks, again, to everyone who helped me pick a text.

Update, January 2006: See downstream for links to the lecture notes.

Corrupting the Young; Enigmas of Chance

Posted by crshalizi at September 11, 2006 12:00 | | permanent link

The Absorbing Boundary

Two hundred and fifty pages later, it's the last day of classes, and my last lecture in advanced probability is done. (I still need to tex up the one lecture I gave from hand-written notes, but there's no rush.) Looking over the tome, I didn't get to a lot of stuff I wanted to (easily enough for another semester!), it's far from self-contained, and it almost certainly contains mistakes, but I think it's not bad for a first draft. It also took an order of magnitude more of my time than was wise. I'll be teaching the class again in the spring, and hope I'll have the strength of will to leave it alone, rather than succumbing to the temptation to do something.

Posting about science and outrages against common decency will recommence shortly.

Posted by crshalizi at September 11, 2006 12:00 | | permanent link

September 09, 2006

Friday Cat Blogging (Latest Research in Chaos Theory and Complex Systems Issue of Science Geek Edition)

In his writings on the theory of automata, John von Neumann defines a complex system as one where the best model is, in fact, the system itself, and specifically mentions cats as an example of such complexity (thereby demonstrating not merely sagacity but taste). Since then, I am sorry to say, there has been a sad lack of interest in feline behavior within complex systems theory; but no longer. Going to the pet store the other day, I blundered into an aisle of motorized cat toys. Mindful of Abbas Raza's post on 3QD, I actually looked at them; and of course when I came to one that boasted an Amazing Chaos Wand, I had to look at it; and of course I had to buy it when I read the following ad copy on the side of the box (my links):

Professor Nozawa
CHAOTIC TOYS FACTORY, LTD.
The Cat Attack uses the latest research in chaos theory and complex systems to emulate the movements and personality of a cat's favorite prey. This "virtual mouse" technology utilizes algorithms based on a six-dimensional coupled nzmap system modeled on the neural network of a real mouse. What that all means is that the Cat Attack's "virtual mouse" will become your cat's new best friend!
Professor Shimada
NIHON UNIVERSITY
Dr. Nozawa has developed an amazing and practical use for "deterministic chaos". It was Norbert Wiener who pointed out the importance of random noise in the brain and in automatic control systems. However, Dr. Nozawa showed that is was possible to solve the difficult "Traveling Salesman Problem" of optimization faster by using nonlinear dynamics, rather than random noise. The "Cat Attack" toy developed by Dr. Nozawa's company operates using powerful nonlinear dynamics algorithms, and this leading edge of human knowledge is appreciated by, among others, the Nozawa family cats, Tal, Fu and Phi, who seem to think the "Cat Attack" is truly alive.

I have turned this apparatus over to a collaborator who specializes in these issues, and she reports good results while the batteries last.

Friday Cat Blogging; Complexity

Posted by crshalizi at September 09, 2006 22:59 | | permanent link

September 08, 2006

Completing the Circuit

Since everyone is going on about Leszek Kolakowski (starting with Tony Judt, and thence Jonathan Goodwin, 3QD, DeLong) or, separately, Althusser (Bérubé, McLemee), I bring you: Kolakowski demolishing Althusser. Read it and weep, or laugh, as hailed. And do read Main Currents of Marxism, now that it's out in one volume ("so handy for pocket or purse", in the words of a certain rootless cosmopolite).

This completes your moment of Marxist harmonic convergence.

The Progressive Forces; Philosophy

Posted by crshalizi at September 08, 2006 11:24 | | permanent link

September 07, 2006

Two Links on Scientific Programming

This looks useful (via Fresh Tracks).

This is only too accurate.

Posted by crshalizi at September 07, 2006 09:13 | | permanent link

Books to Read While the Algae Grow in Your Fur, August 2006

Thomas G. Pavel, The Spell of Language: Poststructuralism and Speculation: Translated by Linda S. Jordan, with the author, from Le Mirage linguistique; also published in translation as The Feud of Language: A History of Structuralist Thought. Basically, an attempt to answer the question of why mid-century French thought became so taken not just with the idea of language, but with certain thematics aspects of one particular approach to language which was already being surpassed in linguistics itself. Basically: it was an attempted "modernization of the human sciences", which were felt, for reasons Pavel goes into, to be comparatively backwards in France. (This would be stronger, I think, if he could point to other efforts in this direction, contemporaneous with the beginnings of structuralism, even if he couldn't say why those failed to thrive.) Erudite about a wide range of scholarly fields, often very shrewd (see especially chapter 6, "On Discretionary Intellectual Behavior"), but in places over-written in much the same style as the authors critiqued, and sometimes rhetorically over-stated. Recommended if you care about structuralism and its spawn.
Thomas E. Ricks, Fiasco: The American Adventure in Iraq: How we got into this goddamn mess, and just how hard it will be to even start to act in ways which won't make things worse. (He only hints, though, at why the Army was so resistant to learning about counter-insurgency.) Depressing as hell, but strongly recommended.; Some bloggers have been upset that Ricks wrote much more "on message" stories for the Washington Post (e.g., his 2003 profile of Wolfowitz, not at all like the portrait in Fiasco, has been singled out for criticism) at the time all this was going on. (If you want to follow this polemic/flame-war, start from here and work backward through the links.) I'm prepared to offer at least a partial and cynical defense. It's clear, from his excellent earlier book on the Marines, that he wants to like the US military, but isn't naive about it. He wrote a remarkable joint story with Anthony Shadid from 2 June 2003, in which Ricks went on a patrol through Baghdad with a US Army unit, and Shadid followed behind talking to the Baghdadis; this doesn't leave you with the illusion that it's all going to go swimmingly and the boys will be home by Christmas. So my guess is that he felt there was just no way he could continue to work on the story, without filing the kind of pieces he did; they were the price of access. Even the Wolfowitz profile ran in parallel with a profile of General Zinni, in fine anti-this-war-now form, making it reasonably clear to any reader which man knew what they were talking about, without actually saying that one of them had his head in the clouds, if not up his ass. It's depressing, and disturbing, that an excellent journalist at a major American newspaper felt he had to employ such Aesopian (not to say Straussian) devices, yes, but am I in a position to say he was wrong?
Frank J. Sorauf, Inside Campaign Finance: Myths and Realities: A well-written look at how the campaign finance system actually worked between the post-Watergate reforms and 1992, when the book was printed. A lot of this was quite eye-opening: it's surprisingly hard to find evidence that PAC donations affected roll-call votes, for instance. Incumbent members of Congress, according to Sorauf, were already much better electoral bets than challengers, and had been for decades, which is why the former found it much easier to raise money than the latter, though money had a much higher marginal impact on challengers' ability to get votes than it did on that of incumbents. So (to gloss over a lot of Sorauf's nuances) campaign donations weren't so much bribes, to get politicians to do things they wouldn't've otherwise as tribute, to keep the contributors from being shut out of influence. (Who was it who said that "If you can't take their money, drink their whiskey, sleep with their women, and vote against them anyway, you don't belong in Congress"?) — I used the past perfect tense because the book ends with the 1992 election, and that as an epilogue; it was frustrating to repeatedly find Sorauf making a good point, and then wonder what had happened in the last decade and a half.; Update: John Burke, in e-mail, remembers the "If you can't take their money" line as originating with Jesse Unruh, and Wikipedia agrees, so that must be right. (Wikipedia quotes the saying as "If you can't take their money, drink their booze, eat their food, screw their women, and still look them in the eye and vote against them, you don't belong here", i.e. in the legislature.)
Rashid Khalidi, Resurrecting Empire: Western Footprints and America's Perilous Path in the Middle East: Capsule introduction to the unhappy experiences of the Middle East with interventions by western powers over the last two centuries, and some reasons why the latest self-proclaimed liberators of Baghdad could expect to be received with some skepticism, whatever their actual intentions. Written in mid-2003, and pretty much borne out by events.

Posted by crshalizi at September 07, 2006 09:13 | | permanent link

August 25, 2006

The Evolution of Complexity (Encore Performance)

Constant readers may recall the advert for a workshop on the evolution of complexity that ran here in January. Because of the way that workshop went, the call for papers ~~will be contined until morale improves~~ has been renewed, with all papers to appear in a special issue of Artificial Life. I reprint the CFP below. Notice that this time, the length limits are the ordinary ones of the journal (2000 words for letters, 12000 words for articles), not the harder limits of the workshop. This should be good.

Special Issue on the Evolution of Complexity

Artificial Life journal

Call for Papers

Guest Editors:

Carlos Gershenson
Centrum Leo Apostel, Vrije Universiteit Brussel
Krijgskundestraat 33. B-1160, Brussels, Belgium

Tom Lenaerts
SWITCH, Flanders Interuniversity Institute for Biotechnology
Vrije Universiteit Brussel, Pleinlaan 2, 1050 Brussels, Belgium

Motivation

As a result of the quality of the Evolution of Complexity workshop at ALife X last June in Bloomington and the interest of the attendants; we announce a call for papers for a special issue on this theme for the Artificial Life journal.

The evolution of complexity is a central theme in Biology. Yet it is not without ambiguity. Complexity has been used to refer to different things. For instance, complexification has been interpreted as a process of diversification between evolving units or as a scaling process that is related to the idea of transitions between different levels of complexity. Other meanings of complexity have been introduced, both inside and outside of Biology. In most cases, though, the central concern is to understand what produces complexity.

The focus of this special issue will be on biological interpretations of complexity and on evolutionary and related dynamics as driving mechanisms for producing complexity. Questions to be addressed in the special issue include:

How could complexity growth be measured or operationalised in natural and artificial living systems?

How can existing data from nature be brought to bear on the study of this issue?

What are the main hypotheses about complexity growth that can actually be tested today?

Are the principles of natural selection as they are currently understood sufficient to explain the evolution of complexity in living systems?

What are the environmental and other constraints of the evolution of complexity in living systems?

What is the role of developmental mechanisms in the evolution of complexity in living systems?

What are conditions could reduce evolved complexity in living systems?

How factors allow the evolution of complexity in living systems to be manipulated and controlled?

What models are most appropriate for understanding the evolution of complexity in living systems?

Paper Submission:
Submitted articles and letters should follow the submission guidelines of the Artificial Life Journal, available at http://mitpress.mit.edu/ALIFE. Authors should also include a cover letter describing briefly the relevance of their article to the specific topic of this call.
These articles and letters should NOT be submitted to the journal editor, but should be uploaded through the special issue website (single PDF files only, include cover letter as the first page of the paper).
Papers will be judged by members of the Program Committee on their relevance to the call for papers, originality, clarity of the presentation, and overall quality.
Important Dates:
Submission deadline: December 15th, 2006
Notification of acceptance: February 1st, 2007
Camera-ready papers due: March 1st, 2007

Programme Committee:
Chris Adami
Lee Altenberg
Mark Bedau
Hugues Bersini
John Bonner
Dominique Chu
Jim Crutchfield
Bruce Edmonds
Carlos Gershenson
Mario Giacobini
Franics Heylighen
Tom Lenaerts
Juan Julián Merelo
Barry McMullin
Chrystopher Nehaniv
Charles Ofria
Jorge Pacheco
Tom Ray
Jon Rowe
Stanley Salthe
Cosma Shalizi
Richard Watson
Larry Yeager

Biology; Complexity

Posted by crshalizi at August 25, 2006 07:40 | | permanent link

August 23, 2006

Go Outside and Play, Why Don't You?

Posting will be light as I gear up for teaching (36-350, data mining for undergraduates, based on this excellent course template by Tom Minka, and Principles of Data Mining by Hand, Mannila and Smyth), and try to finish some papers. I'd advise you to go outside and enjoy what's left of the summer, but since you're reading this, you're not the type of person to do that. Go read Three Quarks Daily instead: they've pre-empted my idle notions of writing posts on deep states and Jack Chick, and they have a better sense of humor and much better taste in art than I do.

The regularly-scheduled cursing of R, LaTeX, time, and an inadequate supply of ibuprofen will now recommence.

The Commonwealth of Letters; The Running-Dogs of Reaction

Posted by crshalizi at August 23, 2006 21:37 | | permanent link

August 22, 2006

Bush as Intellectual

This is a joke that never grows old — just more bitter. (The latter via everyone in the known universe; provoked into posting by John Burke in e-mail.)

Posted by crshalizi at August 22, 2006 15:17 | | permanent link

On the Superiority of Sociology to String Theory

One of my friends in graduate school had an adviser with a gift for memorable expressions, at least for a theoretical physicist, especially when dismayed by some stupidity. Two of his put-downs which stuck with me were "I could go crazy tomorrow and find an appointment in the sociology department", and "I don't want to criticize you, but this is the way superstring people think". I was never sure which was supposed to be worse, but now I know. Sociologists have many faults, but they do know better than to try explaining a variable with a constant, while string theorists evidently do not. (Via CapitalistImperialistPig, who has better things to write about.)

The fact that Prof. Motl reasons so badly here that he'd fail my freshman stats class is, of course, infinitely less offensive than fact that he's a bigot (of the "we must squarely face the harsh light of my pseudo-scientific prejudices" variety). But I can't help feeling — hoping, even — that the two sorts of idiocy are linked.

Update, next day: Greetings, readers from Reference Frame and Brad DeLong! Just to correct some mis-apprehensions: my Ph.D. is in statistical physics, not sociology; I'm an assistant professor of statistics (not sociology) at Carnegie Mellon (not Michigan, where I was a post-doc); the closest I have ever come to "committing a social science" was drawing a map with some other physicists. If you're actually interested in my qualifications, you can look at my CV, or my research and teaching.

Second Update, 20 August: I'm afraid I was too elliptical above. The occasion for Motl's outpourings was this story in the New York Times on how the fraction of black and Hispanic students at New York's specialized high schools, like the famous Bronx High School of Science, has fallen over the last ten years. Fallen, as in, decreased, as in, changed over time, as in, been variable. To account for this, Motl pointed to the black-white IQ gap, which he proclaims one of the great invariant facts of human life, as in, nothing changes it, as in, constant. Even if one grants him his premise (which I would not), the IQ gap might, with a lot of other assumptions, explain why the number of blacks in these schools is low, but cannot explain why it has fallen. This is why I said he would fail my freshman stats class. A Marxist who tried to use the conflicting interests of capital and labor to explain the wage stagnation of recent decades would be guilty of exactly the same fallacy, and I'd fail them for the same reason.

I hope I have made myself clear.

Manual trackback: Nanopolitan

The Running-Dogs of Reaction; Learned Folly

Posted by crshalizi at August 22, 2006 09:56 | | permanent link

August 18, 2006

Analogous Partitions or Alphabetical Provocation?

A cheerful comparison problem, in honor of the day:

India:Pakistan::Israel:Palestine
or
India:Pakistan::Palestine:Israel

Which analogy is more valid? Which one is more inflammatory?

The Continuing Crisis

Posted by crshalizi at August 18, 2006 23:45 | | permanent link

August 17, 2006

Rootless Cosmpolites

Reading Billmon and Adele Stan reminds me of a question which has been bugging me for a while, but not so much as to actually investigate it. Namely: how much did traditional anti-Semitic stereotypes contribute to the stereotypes associated nowadays with conservative attacks on the "liberal cultural elite", "transational professional class", etc.? (I'm thinking of ideas like: Jews run the media etc. behind the scenes; Jews are clever but shallow; the bankers are all Jews; the Communists are all Jews; the Jews want to do away with our wholesome institutions and religion.) It would seem like a natural translation for someone to have made, but I don't know of any evidence that it did happen that way. Maybe, after all, there are only so many ways of disliking other groups that any pair of negative stereotypes is going to have a lot in common, if you look for it. (Of course, people who harbor or play to such stereotypes about liberals are not necessarily anti-Semites, even if those stereotypes historically developed out of anti-Semitic ones.)

Has anyone with some actual knowledge looked into this?

Update 17 August 2006: Edward Burns writes to point out that before the current fabricated outrage over the "war on Christmas", it was being pushed as a UN plot by the John Birch Society in the 1950s, and before that Henry Ford was warning that the Jews were trying to get rid of Christmas and Easter. (See this good story by Michelle Goldberg in Salon.) It's not clear, however, that there was any actual transmission of ideas from one episode to the next; if I were a historian I'd think it would be worth looking into, though.

The Running-Dogs of Reaction

Posted by crshalizi at August 17, 2006 15:52 | | permanent link

August 15, 2006

Parasites and Reintermediaries of Death

Starting about a year ago, I have refused to referee papers for journals owned by Elsevier, since it sticks in my craw to provide free labor for people who turn around and gouge the academic community mercilessly. This reasoning applies, to some degree, to all commercial journal publishers, though Elsevier is unusually exploitative in its pricing. There is however a more substantial reason to dislike them: their — forgive the phrase — mercenary involvement in the international arms trade. Tom Stafford, who blogs at Idiolect, is organizing a petition of academics to try to get Elsevier to stop organizing arms fairs; it's worth signing.

The Eternal Silence of These Infinite Spaces

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

The Starry Heavens Above

Cross-posted to Crooked Timber.

Now this is what I call "filling the mind with ever new and increasing admiration and awe, the more often and steadily we reflect upon them".

(Via David R. in e-mail.)

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Zadig, or, The Book of Fate

Almost two years ago, I complained that there was no English e-text of Zadig, Voltaire's oriental Enlightenment detective fable. At the time, I thought this was my usual pointless whining into the void; but no. Through the good efforts of Barbara Tozier, and the rest of the people associated with Project Gutenberg and Distributed Proofreading, one of the most charming of M. Voltaire's literary productions is now available for free to the English-reading web, in both plain-text and handsome HTML versions.

You realize this means I'm only going to whine here more.

The Commonwealth of Letters; Linkage

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Maladministration of Organs

Cross-posted to Crooked Timber.

Kieran Healy's post about his book on organ donation gives me a hook to write something about the other end of the system, about organ recipients and the institutions which are supposed to match them up with donated organs. More specifically, how one such institution, the Kaiser HMO of Northern California, quite spectacularly failed several thousand people who were depending on them, by not matching them up. The story has been around since early May, when it was broken by Charles Ornstein and Tracy Weber in the Los Angeles Times (cached here), since confirmed by an investigation by Medicare/Medicaid. It doesn't seem to have gotten all that much attention among the blogs, but it's outrageous, and deserves, for that reason alone, to be better known.

Kaiser is a very large and old HMO, with a huge presence in the Bay Area and northern California. (In fact it was one of the very first HMOs, started by Kaiser Steel during WWII as a way of attracting workers to its foundries and shipyards when wages were frozen.) It actually consists of non-profit health plans and hospitals, and for-profit physicians' partnerships, the Permanente Medical Groups; normally, patients in the health plans have to go to physicians in the groups to have their medical expenses covered. One notable exception, up until 2004, was the 1500--2000 Kaiser patients in northern California who needed kidney transplants because of renal failure: those procedures were sub-contracted to the hospitals at UCSF and UC Davis, which also managed the patients' places on the waiting list.

That last bit requires some explanation. Once organs are donated, they need to be matched up to recipients. This is done by an organization called the United Network for Organ Sharing, which tries to trade off urgency, seniority (i.e., time spent waiting for an organ), proximity and compatibility (since the closer the match between the donor's immune system and the recipient's, the less problems from rejection). There is a heavy weight put on seniority, though especially close immunological matches can over-ride it. Each transplant center is responsible for keeping the network up-to-date about their patients who need organs, their immunological profiles, and their time spent on the waiting list.

What seems to have happened is that in 2002, a transplant surgeon named Arturo Martinez proposed to Kaiser that it could save money, and increase the utilization of its hospitals' surgical capacity, by bringing the kidney transplant program in-house, and Kaiser agreed, with Martinez becoming head transplant surgeon. (It would be unfair, at this point, to say that Kaiser did this because it meant more business for the for-profit Permanente Medical Group, but it's hard to imagine that counted against the proposal.) As of mid-2004, Kaiser patients on the waiting list were informed that they would no longer be covered for transplants at UCSF or UC Davis, though they were free to go ahead and have them if they could come up with the money (roughly $100,000).

So far, all this is maybe a little self-serving on Kaiser's part, but not, in itself, appalling. (It's certainly more than legitimate for health-care organizations to try to save money.) What happened, though, was that Kaiser completely screwed up the program. Remember that organs are allocated (basically) through the UNOS system. The patients were being removed from the listings under the university hospitals, and being added to the listings under the new Kaiser transplant program. Unless this was done correctly, this would mean that they'd look like new names on the list, and so all of their accumulated waiting time, one of the main determinants of priority, would vanish. This happened to a huge number of people on the list, basically reducing the chance that they'd get a kidney to next to nothing. This becomes less surprising when one learns that Kaiser never consulted UNOS about the massive transfer of patients it was planning, and "placed responsibility for submitting patient data ... in the hands of a single clerk who had one hour of telephone training on UNOS's database", though not any more excusable. Needless to say, patients were not told that by staying with Kaiser, they were losing their place on the lists, and thereby reducing their odds of survival. Some of them, at least, seem to have been assured that they were keeping their places, when that wasn't true, though this is less clear to me.

Losing seniority on the transplant lists wasn't the only problem. Kaiser did very few transplants, compared to the number of organs which were available. This happened in part because they just didn't have the capacity to keep up with their many patients (at one point they were down to a single nephrologist for the whole program, who was also supposed to be its medical overseer), and in part because of what seems to have been mis-placed perfectionism or caution. These combined to the point of repeatedly turning down "zero mismatch" kidneys, ones where the likely compatibility over-rode considerations of seniority. This happened several dozen times at least — twice for one patient alone. Again, needless to say, patients weren't told about this. In a "it's not a bug, it's a feature" moment, Kaiser initially attempted to defend its program by pointing out how few patients had died after transplants — since they'd done so few.

What strikes me as especially outrageous about all this is that the people being screwed over were people who needed new kidneys. To state the obvious, anyone who needs an organ transplant is very ill. It's maybe less obvious that being that ill is a full-time job. One of the vital parts of the body is no longer working; to substitute for it requires extraordinarily complicated, time-consuming and generally unpleasant procedures. People who need new kidneys are people who are kept alive by dialysis, which is, indeed, complicated, time-consuming, often painful, almost always exhausting, and carries a non-trivial risk of infections, possibly fatal. People who need new kidneys are also often people who are very ill in other ways, since it's not that common for both your kidneys to just stop working if nothing else is going on. (Kidney problems are, for instance, a not-uncommon complication of diabetes, and of high blood pressure. Dialysis, naturally, messes with blood pressure, adding yet another variable to monitor and regulate.) Simply staying alive, when you are multiply-sick person with organ failure, can pretty much demand all the time and attention you have to give, and a fair chunk of your loved ones' as well. (There are good reasons why the families of people in situations like this tend to fall ill themselves.) You are certainly not in a position to check up on whether your medical organization has, through incompetence, messed up your position on the transplant lists. (Some Kaiser patients actually tried to keep up with their place on the lists, but were given the run-around.) And as for switching to another medical organization, do please show me the company which will extend coverage to someone who needs a new kidney, at a price which can be afforded by someone who needs a new kidney.

Since the Times broke the story, there's been some improvement. The doctor who was medical head of the program, and apparently at least partly responsible for snafu of not transferring patients' time, Sharon Inokuchi, has been relieved of her administrative duties. (In fact, if memory serves, she left Kaiser, but now I can't find where I think i read that.) The program has been investigated by the Center for Medicare and Medicaid Services, which basically confirmed the newspaper reports, and forced it to promise major changes; it could still lose its eligibility for funding under those programs. The California state agency which regulates managed care is still, I believe, investigating. There's talk of large fines, and there will certainly be lawsuits. All of this is to the good; it's certainly better than nothing. But still, thinking about this makes me angry: Kaiser had a duty towards many very sick people, who were in a very poor position to look after themselves. It failed in that duty quite dramatically. In any organ transplant program, patients will die while waiting for a match. In most kidney transplant programs, though, about twice as many patients receive transplants as die while waiting; Kaiser managed to reverse that ratio. While it's hard, in the nature of things, to identify any one patient who's died and say "They would have lived, if only Kaiser hadn't done this", it's almost certain that more of these people have died than would have otherwise. I don't have a better remedy to propose than fines or lawsuits or institutional tinkering, but they all seem horribly inadequate.

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Frederick Mosteller Is Dead

Cross-posted to Crooked Timber

Via everyone in the profession: the statistician Frederick Mosteller has died. Mosteller was one of the great leaders of the generation of statisticians in which our field went from being an annex of mathematics (as it was when he attended Carnegie Tech) to an autonomous, institutionalized discipline. He had an astonishing range as a researcher, but is perhaps best known for his work on stochastic theories of learning theory and the authorship of the Federalist Papers. He was also a notable teacher, as his essay "Classroom and Platform Performance" suggests, and in the later part of his career tried to bring elementary inferential hygenie to educational research. More anecdotes are available from Tales of the Statisticians, or this brief sketch by his student Stephen Fienberg.

The Dismal Science; The Great Transformation

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Inducing Disorientation in Larval Economists

Cross-posted to Crooked Timber

As a good neoclassical, neoliberal economist, Brad DeLong is acutely aware that the market system is not natural at all, but a delicate historical anomaly. He is worried that it is so familiar to his students that they will find alternate modes of social organization almost incredible; accordingly he wants to mess with their heads:

Would making Berkeley's first-year economics Ph.D. graduate students this fall read short biographies of William Gates and William Marshall as a way of getting at the idea that there are non-market societies that work very differently from our own today--would that be a teaching idea of extraordinary brilliance or of total insane lunacy?

The rest of the post is an extended excerpt from the New York Review of Books review of a biography of William Marshal (which goes on to my to-read list). The question I have is, what should DeLong make his students read, to give them a vivid sense of just how differently production and distribution could be and have been organized? Argonauts of the Western Pacific, perhaps? Gilgamesh?

And: those of us who teach things other than economics, what books do or should we hand out as ice-axes for our students' frozen seas? (This one is mine.)

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Brunch in the Ruins

Cross-posted to Crooked Timber.

It's a hot, lazy Sunday, which seems like a good time for browsing through livejournal communities dedicated to photos of peacefully rusting machines, quietly crumbling buildings, and similar modern ruins:

Abandoned Places [via David Chess]
Decayed Machinery [via I forget who, years ago]

The photographers are all amateurs, so the quality (to the slight extent I can judge) is quite variable, but many manage to capture the suggestion of sunset and sadness, of unhappy stories brought to a close, which fascinates me about such scenes. Some of these photos, in fact, seem as good as, say, those in Terry Evans's book on the former Joliet Arsenal, Disarming the Prairie, bringing to mind the words of the poet:

These are the halls of the dead, where the spiders spin and the great circuits fall quiet, one by one.

— But I see I'm getting melodramatic, and it's just too hot and sticky and still to sustain that.

Update: John Burke (who needs to revive his blog) writes to point to Jef Poskanzer's great industrial archaeology page, with many fine pictures and links.

Modest Proposals; The Dismal Science

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

One Big Mutual Fund, or, The Ownership Society (Modest Proposals for the Reform of Corporate Governance, Part 2)

Attention Conservation Notice: Over 1500 words on a wacky quasi-socialist economic scheme, from someone utterly lacking in credentials in economics. The scheme does not respect the sanctity of private enterprise, but at the same time would not reduce the alienation of labor one iota. Includes a lengthy quotation of a game-theoretic impossibility result. Also, you already saw it when it was cross-posted at Crooked Timber.

In the previous installment in this series of modest proposals, I looked at ways of making the incentives of the managers of large, publicly-held corporations align more closely with those of their long-term shareholders. This left alone the question of the beneficiaries of corporate value; assuming that the managers are busily working to maximizing their revenue streams, who benefits from their industry and diligence? Having just read Mark Greif's great essay on redistribution in n+1 (issue 4; sadly, not online), I would like to make a suggestion.

The text for today is Gary Miller's Managerial Dilemmas: The Political Economy of Hierarchy, an excellent book which I learned about from Henry Farrell. Ambitiously, Miller tries to explain why hierarchical corporations exist at all, why they take some of the forms they do, and how, in part, their form relates to their performance. Much of the book, especially the first part, is a partially-successful attempt to find good economic reasons for their features, i.e., efficiency-enhancing ones. (He does not seriously consider the option that enterprises are hierarchical for non-economic reasons, say that some people like bossing others around, which hierarchies let them do, and those people are able to select hierarchies over other, more efficient, forms. After all, it's hardly historically unprecedented for powerful people to prefer institutions which lower aggregate output but give them a bigger share of the product. See, e.g., here.) He also tries to explain why theories of corporate organization that rely solely on economic "mechanism design", i.e., structuring information and material incentives, will actually lead to sub-optimal results, for pretty basic game-theoretic reasons; getting beyond these impasses is fundamentally a political problem. This is potentially quite subversive in its own way, but it's really the first part of the work, about the economic justification of the hierarchical enterprise, that I'm going to twist and abuse.

One of the features of the modern corporation that Miller attempts to rationalize is the existence of shareholders who are passive and, in the overwhelming majority, utterly disconnected from the day-to-day or even year-to-year operations of the company. He does so by means of the following impossibility theorem, attributed to Bengt Holmstrom. Having tried to summarize Holmstrom's theorem better than Miller, and failed, I'll just quote Miller.

Holmstrom assumes that there are n agents whose actions determine a level of revenue x. The actions taken are unobservable and are costly to each of the agents. In particular, we assume the production function is a team production in which the productivity of each individual's action is determined by other individuals' levels of effort.
Holmstrom points out the desirability of three characteristics of an incentive system — and then shows that they are logically inconsistent. First, Holmstrom examines the Nash equilibrium outcome of an incentive system. At such an equilibrium, each individual will find that he or she could not do better by choosing a different effort level, as long as all others do not change their effort levels. Simple marginal analysis tells us that, in such an equilibrium, each person will find that his or her marginal cost of effort is exactly equal to the marginal gain; otherwise, the individual could be better off by working harder or not as hard. Second, Holmstrom stipulates that the outcome be budget balancing — that is, the incentive system should exactly distribute the revenues generated by the actors among the actors. Third, Holmstrom examines Pareto efficiency. This means that the outcome should be such that the individuals in the organization could not find a different outcome that would make them all better off.
Holmstrom shows that no budget-balancing system can create a Nash equilibrium that is also Pareto efficient. In other words, every budget-balancing incentive system will induce a social dilemma among its participants. The reason is that individuals will bring their own marginal costs of effort into equality with their own marginal gain. This means that each individual will not undertake an additional unit of effort that will produce less individual gain than individual cost — even if that extra unit of effort produces more gain for other individuals on the team.
As an example, suppose there is some individual who has a marginal revenue productivity of $12: Each unit of her own effort generates an extra $12 for the team. According to Pareto optimality, she should exert additional effort as long as the cost to her of that effort is less than or equal to $12; each such unit of effort generates more revenue for the team that it costs her as an individual. The only way to motivate her is to make sure that she gets all of the marginal revenue of her last unit of effort. In a team, it is impossible for this to be the case for every individual, as long as the incentive system is budget balancing. If everyone gets all of the last dollar produced, the team will have to pay out more in incentives than it generates. But if the individual gets only one-third of the marginal revenue from her actions, she will work only as long as her effort costs her less than $4 per unit. [pp. 129--130]

This suggests a rather unusual role for shareholders: they provide a money-sink, someplace money can go other than those actually involved in production. This means that the economic mechanism no longer has to be budget-balancing, which actually makes efficiency possible. Miller suggests that this is one reason why the modern public corporation, with its separation between legal ownership (by stockholders) and day-to-day control (by managers) can work, to the extent that it does. It is precisely because the shareholders are passive, with very limited influence over the actual running of the corporation!

Today's modest proposal — and I should make it very clear that Miller suggests nothing of the kind — is to take this separation of functions even further. Shareholders can use their legal ownership to intervene in the running of the company, though it is hard (and managers try to make it harder). By doing so, however, they become players in the team-production game, and lose their useful role as a money-sink. To limit this danger, while retaining the advantages of competitive markets for capital allocation and corporate control, I suggest the following. A substantial fraction — say three-quarters — of all profits of publicly-held corporations are to be paid to a new institution, which we might call the National Mutual Fund. (Close corporations and partnerships are exempt.) Once a year, the Fund would pay out its accumulated profits as dividend checks, giving an equal amount to every adult citizen. And that's it.

Substantially reducing the flow of dividends associated with stock ownership should cause a large one-time shock to the level of the stock market. (Roughly speaking, shares should drop by about 3/4.) However, because the Fund collects uniformly, it should not distort relative prices, which are what matter for purposes of capital allocation. The net worth of stock-holders, likewise, will suffer a one-time drop, but this will be partially compensated for by their receiving payments from the Fund in the future. Anyway, lots of things affect the value of stock holdings; it's not like someone purchased their labor with a promise of future benefits, and then tried to back out of a freely-entered contract when it came time to pay up.

A further wrinkle would be to curb the practice of retained earnings. These account for a huge fraction of corporate capital formation, but they are also one of the ways in which managements escape market discipline. (For some figures on this, see Henwood's Wall Street, pp. 72--76.) I suppose one could make a Hayekian argument in favor of the practice, but, really, if management can make a good case that a pet project will earn at least a normal rate of return, it shouldn't be hard for them to raise funds on the open capital market, and if they can't make such a case, it's hard to see how they'd be discharging their fiduciary duties to shareholders by pursuing it anyway. This reform, I should add, is logically separate from that of instituting the National Mutual Fund. However, since corporations would pay more out in dividends, it would tend to increase the value of shares, reducing the shock to the level of the stock market.

It is hard to see why the actions of the National Mutual Fund could not be at least as rule-bound and de-politicized as those of a central bank run by skilled technocrats. Indeed, it would seem easier to reduce the discretion of the Fund's officials to the vanishing point, and to strictly keep it from meddling with the affairs of any corporation, which would be deeply counter-productive. For their part, the citizens receiving the dividends would get the benefits of "portfolio diversification in their income", but their incentives to meddle politically with individual firms, even quite large firms, would be quite muted. Moreover, they would have a direct and tangible incentive in the health of the corporate sector as a whole, making them less likely to support market-distorting measures to benefit particular firms, geographical regions or industrial sectors. We would move, in short, towards a true ownership society.

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

How to Make Our Ideas Clear — to Others

Cross-posted to Crooked Timber

In the comments to my post on Onsager, Maynard Handley explains why he finds himself somewhat unsympathetic, as Onsager apparently did not expend the effort necessary to make himself understood by others.

You, the author of the paper, have a responsibility to make your ideas comprehensible. If the first method you choose to explain them fails, then you listen to what people say about where they lost all track of understanding and write a new paper with NEW explanations, not the same explanations that failed last time only renumbered. ... [This is] not something that is drilled into young scientists; that it is YOUR responsibility to make your ideas clear to others, not their responsibility to try to figure out what you are talking about. As science grows ever larger and ever more complex, I think it is time for all scientists to be much more explicit and much more ruthless on this point.

Whether this is really a fair criticism of Onsager, I couldn't say, but the general point is true, important, and a perfect hook for the next thing I wanted to post about.

Science is a social, collaborative process, so part of being a good scientist is effective communication. Scientific communication is overwhelmingly written communication (scientific disciplines are, in a sense, literary communities), so part of being a good scientist is being a good writer. Unfortunately, scientists get little training in writing, and much of that consists of being advised to follow the rules found in horrid little compendia. Fortunately, there is some actual research on effective written communication, that is, on how to arrange your words so that their readers tend to acquire clear notions of your ideas. The best practical guide here, I've found, is Joseph William's Style: Towards Clarity and Grace. However, I have just discovered (via Paradise Blogged) a fine essay by George Gopen and Judith Swan, "The Science of Scientific Writing", which gives a clear yet concise presentation of the work. (Gopen and Williams are collaborators.) Here is their own summary of how to be clear:

Follow a grammatical subject as soon as possible with its verb.
Place in the stress position the "new information" you want the reader to emphasize.
Place the person or thing whose "story" a sentence is telling at the beginning of the sentence, in the topic position.
Place appropriate "old information" (material already stated in the discourse) in the topic position for linkage backward and contextualization forward.
Articulate the action of every clause or sentence in its verb.
In general, provide context for your reader before asking that reader to consider anything new.
In general, try to ensure that the relative emphases of the substance coincide with the relative expectations for emphasis raised by the structure.

If these rules, and the notions behind them, are valid, they should apply to more than just scientific writing; in particular, they should hold for other kinds of academic prose. Looking at their examples of revising scientific writing, expressing the same complicated and precise ideas in more easily grasped ways, I couldn't help but be reminded of the humanistic "bad writing" controversy, where the case for the defense often seems to rest on complexity, and the example of scientific jargon. The examples of Gopen and Swan, Williams, etc., show that those defenses do not hold. Scholars of the humanities may have reasons for being unclear which don't apply to scientists, but I can't think of any good ones.

The Commonwealth of Letters

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Two Menances to the Keystone State

Cross-posted to Crooked Timber.

Two of my more public-spirited fellow citizens have recently identified looming threats to our own Commonwealth of Pennsylvania.

Our beloved junior senator, Rick Santorum (via Pharyngula):
Most scientists unfortunately, those that certainly are advocating for this [embryonic stem cell research], and many others feel very little moral compulsion. It's a utilitarian, materialistic view of doing whatever they can do to pursue their desired goals.

I, for one, will be happier voting on Mr. Santorum's re-election in November, knowing that my ballot will play a part in the age-old struggle between utilitarian materialism and deontological idealism, as well as the sagas of human-canine relations and Old Corruption.
Our beloved linguistics professor, Mark Liberman:
More than a third of all Pennsylvanians are native speakers of a language other than English — and many of them have not even tried to learn English since immigrating, or at least prefer to carry out their daily lives in another language, living together in neighborhoods where their native language dominates. Some people worry that the majority status of English is critically endangered. 25 years ago, a major political figure warned that these "aliens ... will never adopt our language or customs, any more than they can acquire our complexion", and so far, his prediction seems to be right on the money.

And let's not forget what they've done to our cooking!

The Running-Dogs of Reaction

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

The Nobel Prize Winner as Neglected Genius

Cross-posted to Crooked Timber.

A staple of bad movies and trashy novels about scientists (i.e., the kind I read) is the neglected genius whose ideas are rejected with incomprehension by the scientific establishment during his life, because they are simply Too Far Ahead Of His Time to be grasped by lesser mortals, only for the scientific community to rediscover these insights decades later. This sort of thing can make for entertaining fiction (if dreary self-mythologization), but it simply doesn't happen all that often in real life, especially not when the hero is a part of the establishment, and indeed a much-honored one. It certainly doesn't show up, with documentary evidence, in the staid, reliable pages of Reviews of Modern Physics. Nonetheless:

Gregory L. Eyink and Katepalli R. Sreenivasan, "Onsager and the theory of hydrodynamic turbulence", Reviews of Modern Physics 78 (2006): 87--135; no free copy: Abstract: Lars Onsager, a giant of twentieth-century science and the 1968 Nobel Laureate in Chemistry, made deep contributions to several areas of physics and chemistry. Perhaps less well known is his ground-breaking work and lifelong interest in the subject of hydrodynamic turbulence. He wrote two papers on the subject in the 1940s, one of them just a short abstract. Unbeknownst to Onsager, one of his major results was derived a few years earlier by A. N. Kolmogorov, but Onsager's work contains many gems and shows characteristic originality and deep understanding. His only full-length article on the subject in 1949 introduced two novel ideas — negative-temperature equilibria for two-dimensional ideal fluids and an energy-dissipation anomaly for singular Euler solutions — that stimulated much later work. However, a study of Onsager's letters to his peers around that time, as well as his private papers of that period and the early 1970s, shows that he had much more to say about the problem than he published. Remarkably, his private notes of the 1940s contain the essential elements of at least four major results that appeared decades later in the literature: (1) a mean-field Poisson-Boltzmann equation and other thermodynamic relations for point vortices; (2) a relation similar to Kolmogorov's 4/5 law connecting singularities and dissipation; (3) the modern physical picture of spatial intermittency of velocity increments, explaining anomalous scaling of the spectrum; and (4) a spectral turbulence closure quite similar to the modern eddy-damped quasinormal Markovian equations. This paper is a summary of Onsager's published and unpublished contributions to hydrodynamic turbulence and an account of their place in the field as the subject has evolved through the years. A discussion is also given of the historical context of the work, especially of Onsager's interactions with his contemporaries who were acknowledged experts in the subject at the time. Finally, a brief speculation is offered as to why Onsager may have chosen not to publish several of his significant results. [My links.]

Nobody outside of statistical physics (and maybe physical chemistry) has heard of Onsager, but he was indeed a great physicist, albeit in a very technical, non-flashy way. By the time he did this work on turbulence, he was already well-known in statistical mechanics for the analytical solution of the Ising model, his theory of phase transitions in liquid crystals, and, perhaps most importantly, a pair of classic papers from 1931 which basically founded modern irreversible thermodynamics, for which he would eventually win the Nobel Prize. (Eyink and Sreenivasan give a fuller discussion of his accomplishments, including the Onsager-Machlup theory of non-equilibrium processes, on which Eyink himself has done important work.) We're definitely not talking about some marginal figure cut off from the scientific community.

Nonetheless, his attempts to get people to pay attention to these ideas on turbulence were singularly unsuccessful. The reaction of Theodore von Kármán, a deservedly great name in fluid mechanics, was to describe it (in a letter to his student C. C. Lin) as "somewhat 'screwy' "; Onsager also corresponded with Lin, who replied in the classic manner of someone wanting to put an end to a conversation (quoted on p. 117): "I am sorry to say that I have not made much progress, except that I desire still more to see something done in this line to bring your ideas down to my level of understanding." As for the statistical physicists, Eyink and Sreenivasan describe their reaction as one of "polite incomprehension" (except on the part of von Neumann — in an unpublished report). The fact that one of Onsager's letters describing his ideas (reproduced as Appendix A in this paper) is headed "The little vortices who wanted to play", and begins "Once upon a time there were n vortices of strengths K1, ... , Kn in a two-dimensional frictionless incompressible fluid" probably didn't help, either. Most of all, a combination of discouragement over this reception, a tendency to be a slow and perfectionist author, and having scads of major research projects going simultaneously kept Onsager from even trying to publish any of this material.

The moral, I hope, is clear: statistical physicists who wander into other areas of science, and find their ideas dismissed by the best experts on those subjects, should nonetheless publish in Physical Review, in a "Fools! I'll show them all!" spirit, provided they are Lars Onsager.

(It's interesting that this paper was written by two physicists active in this area, rather than by a historian of science. It seems doubtful to me that a historian, reading the relevant materials in the Onsager archives, would have realized that there was a story here, unless they were familiar with modern work on turbulence at a deeply technical level — unless they had "contributory" as well as "interactional" expertise. And if anyone had gone over those archives around 1990, before these ideas were re-discovered, what would they have made of it?)

Physics

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Chronicle on Cole

Cross-posted to Crooked Timber, hence the parting shot about comments.

Under the rubric "Can Blogging Derail Your Career?", the Chronicle of Higher Education has seven bloggers discussing Yale's decision to not hire Juan Cole as a professor of history, and the role, if any, played by his blog in that decision: Siva Vaidhyanathan, Dan Drenzer, Brad DeLong, Michael Bérubé (all: yay!), Glenn Reynolds and Ann Althouse (both: hiss), and Erin O'Connor (null result), with a "response" by Cole, which doesn't actually address the others' posts specifically, and reads like a separate essay on the same subject as the others. (Via De Long.)

(Some of the things which were written about Cole as part of the controversy (e.g.,) give the impression of a professor who attains incomprehensibility not through obscurity but through foaming at the mouth. As it happens, though, I sat in on his seminar on millenarian movements when I was a post-doc at Michigan, and nothing could be further from the truth. I suppose I could have missed all the sessions which degenerated into hours-long rants about Zionist Entities... Of course, I don't know why Yale didn't give him the job, but if it was because they thought he was too spittle-flecked to be presentable to parents and alumni, they were misinformed.)

The fact that this post is not filed under "Middle East Politics" isn't going to stop anyone in the comments, is it?

The Eternal Silence of These Infinite Spaces

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

"Where Alph, the sacred river, ran"

The Xanadu region of Titan has rivers:

No images, yet, of "caverns measureless to man", or a "sunless sea" of liquid methane.

(Via Beyond the Beyond.)

Update, 20 July: I should have read the press release first (via Uncertain Principles). More importantly: Venus! (via The Daily Llama).

Update, 21 July: The Quantum Pontiff wonders whether Einstein's work on the physics of rivers will hold on Titan.

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

George Hersey's The Monumental Impulse: A Declaration of Defeat

Attention conservation notice: 1600 words of pedantic whining about how a book on architecture didn't meet my parochial expectations. Also, it was mostly written and then abandoned to the gnawing criticism of the mice over a year ago.

I give up; it's got me beat.

I asked the good people at the MIT Press for a review copy of this, this thing, years and years ago, back when I was still regularly writing book reviews, because it sounded cool.

We humans owe an immense architectural debt to many other species. Indeed, the first hexagons humans saw may have been in honeycombs, the first skyscrapers termitaries (termite high-rises), and the first tents those of African weaver ants. In The Monumental Impulse, art historian George Hersey investigates many ties between the biological sciences and the building arts. Natural building materials such as wood and limestone, for example, originate in biological processes. Much architectural ornament borrows from botany and zoology. Hersey draws striking analogies between building types and animal species. He examines the relationship between physical structures and living organisms, from bridges to mosques, from molecules to mammals.
Insects, mollusks, and birds are given separate chapters, and three final chapters focus on architectural form and biological reproduction. Hersey also discusses architecture in connection with the body's interior processes and shows how buildings may be said to reproduce, adapt, and evolve, like other inanimate or "nonbiotic" entities such as computer programs and robots. The book is both learned and entertaining, and is abundantly illustrated with fascinating visual comparisons.

It would have been cool, too, if Hersey weren't an ignorant idiot. Oh, I'm pretty sure he's OK when it comes to purely architectural matters — though how would I know if he wasn't? What drives me up the wall every time I try to write about this book is that he gets everything else wrong.

Let's take history and languages first. Hersey is a professor of art history, and so I don't think it's unfair to expect him to get these straight. But we are talking about a man who can instance a spiral design from a Byzantine church as an example of an "Islamic spiral", whatever that may be (p. 47). He doesn't seem to realize (pp. 28–29) that the early Greek architects who first defined the classical orders wouldn't have used a Latin word (capitulum) to refer to the top of a column. I'm not even sure he realizes (pp. 7–8) that Hermes Trismegistus was a Hellenistic myth. For that matter, after presenting a fanciful analogy (pp. 17–18) between the ground-plan of Lemba, a Chalcolithic village in Cyprus, c. 3900 B.C., and cross-sectional diagrams of biological cells, he writes:

Certainly the builders of these dwellings would have known similar forms in their immediate surroundings — things that, unlike true body cells, were visible to the naked eye. One prototype would be the egg, which begins as a contained for a single-cell embryo embedded in the nourishing matter it will need in order to reproduce and grow. ... And then there are cell-like beehives, birds' nests, and plants. To the Greeks, moreover, and therefore maybe even to the Chalcolithic residents of Lemba, the word for cell (kutos) also meant uterus, and even the whole human body. So we must not relinquish the thought that the Lemba cells are the extended phenotype of builders whose own bodies, though they did not consciously know this, were put together similarly.

Let me try to extract everything that's broken in these sentences, and see what's left.

Whatever they were speaking in Cyprus in 3900 B.C., it wasn't Greek, or even in the Indo-European family. Since kutos derives from the proto-Indo-European root *(s)keu- meaning "to cover" or "to conceal", it's irrelevant to the Lemban vocabulary and mentality. (See e.g. Calvert Watkin's Dictionary of Indo-European Roots, pp. 78–79.)
He's right that kutos also meant uterus in Greek, but so what? Nobody thought mammals had eggs until the Renaissance, much less eggs somehow related to things called "cells". It's just as relevant that kutos also referred to "the fourth stomach of the ox".
Suppose that the Lemban word for their houses was nothing at all like their word for "cell", and furthermore that their designs for their houses owed nothing at all to observations of eggs, birds' nests, beehives or anything like. Would that mean their buildings were not part of the builders' extended phenotypes? Obviously, the answer is "no", or at least that should be obvious to anyone who understands what "extended phenotype" is supposed to mean.
He moves from "maybe" to "must not abandon" in the space of a sentence.

However, it does seem plausible that the Lembans would have noticed that eggs, birds' nests, and their houses were all roundish.

Bad as he is on history and languages, Hersey is worse at science and mathematics, and the history of science. He has no idea what "a topology" is (p. 51; at best this is a garbling of a curved manifold). He is capable of writing about distances "increasing at a fixed angle" (p. 45; he seems to mean rate), and of "steel molecules" (p. 13). He thinks sharks are not vertebrates (p. xvii). He literally does not know what a virus is (p. 15). (He also doesn't know when they were first discovered, nor when their shapes were first determined [pp. 15–16], and so his suggestion that illustrations of the shapes of viruses influenced some of Gaudí's designs on the Sagrada Família [p. 16] is just wrong.) He doesn't realize that DNA molecules don't actually look like the simple diagrams people draw of double helices, but are bent, folded and twisted, and so resemble spiral staircases not at all (pp. 6–8). And so on, and so forth.

Let me give a last example of the kind of thing which irritates the hell out of me; it comes from very early in the book (pp. xviii-xix), but it's central to whatever attempt at an argument Hersey makes.

Homo sapiens shares something that I don't yet dare call a gene sequence for building — shares it, perhaps homologously, perhaps convergently, with other constructing creatures such as birds, crustaceans, ants, termites, and bees. I will also be claiming, as a corollary, that the shapes of our monumental shelters, whether bicycle sheds or cathedrals, reflect and often derive from the shapes first created by these other species — species that, like us, are subject to the monumental impulse. ...
But now comes a paradox: certain ants, termites, honeybees, and birds build elaborate structures. So do humans. But, as humans, we are anomalous in doing this. Only a few other mammals build — most obviously beavers and badgers. Worse still, our own closest cousins, the other primates, hardly build at all. An African termitary might remind us of Wright, of a Gaudí spire, or of a skyscraper by Hermann Obrist. But no such thoughts come to mind when we look at the rudimentary retreats of chimps and gorillas. Thus any genetic homology that brackets us with the other builder-species will have to be very ancient and, also, will have to have bypassed our immediate ancestors and cousins.

This isn't a paradox for his idea — it's a refutation. The last common ancestor of humans and termites lived before the Cambrian explosion, presumably in the oceans; whatever bizarre wormy thing it may have been, it assuredly didn't build. Even if the genes "for" building in humans and termites are both descended from the same set of genes in that remote common ancestor, they are no more homologous than flight is homologous in birds and pterodactyls, because they both independently modified vertebrate forelimbs into wings. A little reading on comparative methods, and how homologies are actually established, would have kept Hersey from wasting his and his reader's time. (Similarly for the chapter on the reproduction and evolution of architectural designs, which is completely innocent of all actual work on, say, the evolution of technology, or even on the cognitive processes of architectural representation.)

The whole book is like that — a series of conceits which a little thought or research would've shown don't work, presented as real scholarship. To be fair, Hersey sometimes allows himself a certain levity of presentation: in chapter eight he claims (basically) that people like domes because they remind them of breasts, which he illustrates by juxtaposing a picture of the Taj Mahal with one he attributes to "D-Cup Superstars, February 1992". But even then, he concludes, on no basis whatsoever, that the Taj is "an architectural thernody to the queen's breasts" (p. 155), and means us to take that seriously.

I won't say that Hersey's book is bullshit, because Hersey buys it. But I will say it's crap. It's crap in the same way that much too much of what I read by scholars in the humanities is crap: Hersey doesn't think carefully and critically, he tries to use ideas he doesn't understand, he's sloppy about facts, and he thinks he's establishing reliable conclusions when he can't argue his way out of a wet paper bag. Lest by saying this I call up the wrathful wraith of Chun the Unavoidable, I hasten to add that (1) many scholars in the humanities are, indeed, excellent and careful scholars, who do not suffer from these debilities, and (2) I think this cannot in the least be blamed on any value of post-*ism, or or "theory" or anything of that sort. Certainly Hersey is not a post-*ist, and I have no reason to believe this sort of crappiness has become more common among humanists in recent years. (If anything, I'd guess that the causal arrows point from crappiness to post-*ism, rather than the other way. For this, too, however, I have no evidence.)

— But, you see, this is what always happens when I try to write a proper review of The Monumental Impulse: I end up wandering hopelessly off topic, in order to avoid having to think of all the ways the book vexed me.

Do not read this book.

Learned Folly; The Natural Science of the Human Species

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Critical Sensation

Attention conservation notice: 1500 words on psychophysics and the statistical mechanics of disordered excitable media. Also, it was cross-posted to Crooked Timber, where I am guest-blogging this week, so you've seen it already.

First off, I should thank Henry and the rest of the Timberites for the kind invitation to guest-post, and that very warm introduction. In exchange, I'm going to blog more or less as I usually would, only here. This means some big bricks of posts about "complex systems", so called, which is or was my scientific field, more or less; and also any miscellaneous outrages which catch my eye this week. Mounting my usual hobby-horses on this stage is a poor exchange for their generosity, but mounting hobby-horses is why I started blogging in the first place, and anyway I'm big on conscienceless exploitation of cooperators.

Today I want to talk (below the line) about some recent work in the statistical mechanics of disordered systems, which might help explain how our sense organs work, and actually involves some good uses of the self-organized criticality and power laws; tomorrow or the day after I'll get to the smoldering question of "Why Oh Why Can't We Have Better Econophysics?"

Folklore says that the dark-adapted human eye can detect a single photon; this isn't quite true, but we can consciously detect a few tens of photons, and some species are that sensitive. Of course, we can see not only in the dark but also during broad daylight, but then the number of photons falling on every part of the retina is huge; the eye isn't overwhelmed and saturated, though now one or ten photons more or less makes no discernible difference. In the jargon, the eye, and the other sensory organs, have both a large "dynamic range" (we can see in the dark and in the daylight), and "nonlinear response" (changes which are noticeable in the dark aren't against a high-intensity background). Some version of these facts, including the basic (power-law) form of the relationship between physical stimulus intensity and perceived sensory magnitude, have been known since the nineteenth century. This makes it all the more puzzling that sensory neurons show a linear response over a narrow dynamic range, beyond which they saturate.

You could evade this difficulty by having lots of neurons with different operating ranges, so that raising stimulus intensity saturated some but activated others. The problem is that there don't seem be that wide a spectrum of operating ranges for individual neurons. In a recent paper, Osame Kinouchi and Mauro Copelli (who blog together at Semciência) offer another way, which has to do with the way sensory neurons interact with each other in a network.

Osame Kinouchi and Mauro Copelli, "Optimal dynamical range of excitable networks at criticality", Nature Physics 2 (2006): 348--351; free preprint version at q-bio.NC/0601037 *: Abstract: A recurrent idea in the study of complex systems is that optimal information processing is to be found near phase transitions. However, this heuristic hypothesis has few (if any) concrete realizations where a standard and biologically relevant quantity is optimized at criticality. Here we give a clear example of such a phenomenon: a network of excitable elements has its sensitivity and dynamic range maximized at the critical point of a non-equilibrium phase transition. Our results are compatible with the essential role of gap junctions in olfactory glomeruli and retinal ganglionar cell output. Synchronization and global oscillations also emerge from the network dynamics. We propose that the main functional role of electrical coupling is to provide an enhancement of dynamic range, therefore allowing the coding of information spanning several orders of magnitude. The mechanism could provide a microscopic neural basis for psychophysical laws.

Neurons, like muscle cells, are "excitable", in that the right stimulus will get them to suddenly expend a lot of energy in a characteristic way — muscle cells twitch, and neurons produce an electrical current called an action potential or spike. Kinouchi and Copelli use a standard sort of model of an excitable medium of such cells, which distinguish between the excited state, a sequence of "refractory" states where the neuron can't spike again after it's been excited, and a resting or quiescent state when the right input could get it to fire. (These models have a long history in neurodynamics, the study of heart failure, cellular slime molds, etc.) Normally, in these models the cells are arrayed in some regular grid, and the probability that a resting cell becomes excited goes up as it has more excited neighbors. This is still true in Kinouchi and Copelli's model, only the arrangement of cells is now a simple random graph. Resting cells also get excited at a steady random rate, representing the physical stimulus.

Kinouchi and Copelli argue that the key quantity in their model is how many cells are stimulated into firing, on average, by a single excited cell. If this "branching ratio" is less than one, an external stimulus will tend to produce a small, short-lived burst of excitation, and there will be no spontaneous activity; the system is sub-critical. If the branching ratio is greater than one, outside stimuli produce very large, saturating waves of excitation, and there's a lot of self-sustained activity, making it hard to use a super-critical network as a detector. At the critical point, however, where each excited cell produces, on average, exactly one more excited cell, waves of excitation eventually die out, but they tend to be very long-lived, and in fact their distribution follows a power law.

(People who teach courses on random processes are very fond of branching processes, because the basic model can be solved exactly with hundred-year-old math, but there are endless ramifications, and some of the applications are very technically sweet. Like most mathematical scientists, Kinouchi has certain tools he tends to return to, and critical branching processes are one of them.)

As Kinouchi and Copelli say in their abstract, the idea that the critical point, where things are just about to go unstable, is a useful place for processing or transmitting information is a persistent theme of complex systems. (You could, arguably, even trace a version of the idea back to William James's Principles of Psychology.) It has also, before this, been one of the weakest of our ideas. The original work from the 1980s on "evolving to the edge of chaos" has proved impossible to replicate, I would even say experimentally refuted. (Why the phrase and idea continue to propagate is another question for another time.) Stu Kauffman's studies of models of gene regulatory networks certainly suggests that information moved through these most easily near their critical point, but I don't think anyone has done a careful information-theoretic analysis of that. In any case, E. coli doesn't care about the bandwidth of its regulatory network: it cares about reliably making lactase when it only has lactose to eat, i.e., specific adaptive functions. Prior to this, I can only think of one situation where the idea has been made precise and has strong evidence to back it up (namely, this paper), but that's a purely mathematical exercise of no biological relevance.

What Kinouchi and Copelli have done is very different: they've actually identified something biologically important which is maximized at the critical branching ratio, namely the dynamic range. The network as a whole responds to the stimulus, and its dynamic range can be many orders of magnitude wider than that of its component cells. It is this enhancement which is maximized at the critical branching ratio, and falls off sharply for networks which are even a little sub- or super- critical. As a bonus, the shape of the response function is of the correct power-law form, though, in their model, the exact exponent isn't right. Modifying the network structure, or some model details, changes the exponent, but the dynamic range is still sharply peaked at the critical branching ratio.

There are a lot of other nice things about this paper, which I won't get in to least I repeat it all, but I will point out one thing: while their central qualitative results are pretty robust to small tweaks, there are some details of their model which make it a fair caricature of some excitable media, but not all. These are quite deliberately matched to properties of the olfactory system and the retina, but wouldn't work in, say, the cortex, where the dynamics of excitation are different. So this isn't an "over-universal" model, but one of particular phenomena produced by particular mechanisms. In fact, looking at olfaction, they are able to make a prediction about the effects of knocking out specific genes which are involved in the fast, symmetrical electric couplings they assume. Nobody seems to have done those experiments yet, but, at least to this non-biologist, it seems feasible, and, now, very interesting.

*: Here's an anecdote illustrating how broken academic publishing is. Kinouchi and Copelli work at the University of Saõ Paulo, which doesn't, for reasons of economy, subscribe to Nature Physics. To get an electronic copy of their own published paper, they were forced to write correspondents at other universities. I couldn't help them, because my school doesn't feel like it can afford to subscribe to Nature Physics either.

Complexity Minds, Brains, and Neurons;

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Friday Cat Blogging (Leaving on a Jet Plane Issue of Non-Science-Geek Edition)

Speaking of air travel, as we were, Kara dreams of traveling to the Beijing Complex Systems Summer School.

Alas! She will not be gracing the Fragrant Villa with her presence. Twenty hours in planes and airports would not agree with her temperament, nor would jet-lag be pleasant for one so distressed by any interruption of her accustomed nap cycle.

As for me, I have cleverly worked on my three lectures in reverse order, so the last two are finished...

Friday Cat Blogging

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

The Awful Turkish Language

I wrote this about a year and a half ago; I'm cleaning out my drafts folder.

There are reasons why one might think that Turkey should not be admitted to the European Union, but surely the silliest must be that Turkish is not an Indo-European language. Following Phersu, I can just imagine the consequences of taking this seriously. First, the Basque-speaking provinces of France and Spain leave the EU, along with Hungary, Finland, Estonia and Malta. But then, of course, India and Pakistan will submit rival applications to join, closely followed, no doubt, by the Iraqi Kurds. The whole idea is so stupid that I can't believe it was meant seriously, or even guess what Giscard d'Estaing thought "Indo-European" meant.

That said, Turkish does have features which are absent or attenuated in (most) Indo-European languages. (Disclaimer: I do not speak Turkish.) For instance, it's highly agglutinative, forming new words by adding suffixes to roots, and doing so recursively. (German does this too, but to nowhere near the same degree.) This leads to words like yapabilecekdiyseniz, "if you were going to be able to do". (Readers may amuse themselves by analyzing this example using the Turkish Suffix Dictionary.) Moreover, these words are not oddities, like "antidisestablishmentarianistic", but in everyday use. I once heard a talk by a computational linguist specializing in Turkish — Gerjan van Schaaik, who oddly seems to have no web presence — where he mentioned that if one studied the corpus of Turkish daily newspapers, one could easily build a lexicon of 500,000 entries, and still cover only 95% of the words in the corpus. (I can't tell, from my notes, whether van Schaaik was talking about something that had actually been done, or just making a rough estimate.) This property of Turkish becomes very important for a number of technologies, including one without which the modern world would simply grind to a halt: spam filtering.

Levent Özgür, Tunga Güngör and Fikret Gürgen, "Adaptive anti-spam filtering for agglutinative languages: a special case for Turkish", Pattern Recognition Letters 25 (2004): 1819--1831 [PDF reprint via Prof. Güngör]: Abstract: We propose anti-spam filtering methods for agglutinative languages in general and for Turkish in particular. The methods are dynamic and are based on Artificial Neural Networks (ANN) and Bayesian Networks. The developed algorithms are user-specific and adapt themselves with the characteristics of the incoming e-mails. The algorithms have two main components. The first one deals with the morphology of the words and the second one classifies the e-mails by using the roots of the words extracted by the morphological analysis. Two ANN structures, single layer perceptron and multi-layer perceptron, are considered and the inputs to the networks are determined using binary model and probabilistic model. Similarly, for Bayesian classification, three different approaches are employed: binary model, probabilistic model, and advanced probabilistic model. In the experiments, a total of 750 e-mails (410 spam and 340 normal) were used and a success rate of about 90% was achieved.

Özgür et al. do not report on the ability of their classifiers to discriminate between spam, and weirdly pseudo-learned pronouncements from former presidents of France.

Enigmas of Chance; The Dismal Science

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Sow the Wind, Reap the Drag Coefficient (Dept. of "We are as gods, and might as well get good at it")

I wrote this about a year ago; I'm cleaning out my drafts folder.

As a paid-up member of the Viridian movement, I it would be a Very Good Thing for industrial civilization to find a power source which will let it run and grow without choking on its own waste. (I'm no good at ranting about this, so if you're curious I refer you to the master.) But it would be foolish to pretend that non-fossil-fuel power sources will not carry their own costs, including the possibility to alter the climate, and something worse than foolish not to consider those costs. After all, one of the Viridian design principles advises us to "Look at the Underside First":

Legions of people are paid large sums to promote the positive aspects of commercially available products. Very few people earn their daily bread by pointing out malfunctions, bugs, screw-ups, design failures, side-effects and the whole sad galaxy of trade-offs and failings that are inherent in any technological artifact. To counteract this gross social imbalance, a wise designer and a wise critic will make it a matter of principle to look at the underside first.

In which spirit, I bring you the following.

David W. Keith, Joseph F. DeCarolis, David C. Denkenberger, Donald H. Lenschow, Sergey L. Malyshev, Stephen Pacala and Philip J. Rasch, "The influence of large-scale wind power on global climate", Proceedings of the National Academy of Sciences (USA) 101 (2004): 16115--16120 [free via open access]: Abstract: Large-scale use of wind power can alter local and global climate by extracting kinetic energy and altering turbulent transport in the atmospheric boundary layer. We report climate-model simulations that address the possible climatic impacts of wind power at regional to global scales by using two general circulation models and several parameterizations of the interaction of wind turbines with the boundary layer. We find that very large amounts of wind power can produce nonnegligible climatic change at continental scales. Although large-scale effects are observed, wind power has a negligible effect on global-mean surface temperature, and it would deliver enormous global benefits by reducing emissions of CO2 and air pollutants. Our results may enable a comparison between the climate impacts due to wind power and the reduction in climatic impacts achieved by the substitution of wind for fossil fuels.

There are basically three channels through which wind power would influence climate. The first is direct increase of friction in the vicinity of the turbine, which dissipates kinetic energy in the wind into heat. The second is that the increased drag produced by the turbines will change the speed and direction of prevailing winds, and since these winds carry heat and moisture, can potentially alter the climate over large areas. The third, and most indirect, channel is that increasing use of wind power may reduce the amount of fossil fuel consumption, and interact with climate changes produced by the existing load of greenhouse gases.

The present paper is a first cut at considering all three channels, using two standard global climate models and a variety of different assumptions about where wind-turbines will be situated, how efficient they will be, and the manner and magnitude of the resulting increase in surface-layer drag. There are plenty of crude approximations made here (for instance, that the increase in drag is uniform over the affected area), but they're recognized as such. The results thus need to be treated with caution, but they are interesting. The second channel (changing atmospheric transport) has a much bigger impact than the first (direct friction); together, under plausible assumptions, they give a near-zero impact on the global mean, root-mean-square change in seasonal means of about one twentieth of a degree centigrade at one point, and peak changes of half a degree. Under further plausible, but of course more arguable, assumptions, the third channel (reduced carbon emissions) swamps these effects, too, by about a factor of five. Interestingly, the configurations for wind power they consider tend to cool the poles and warm the lower lattitudes, while carbon dioxide warming has the opposite effect. This suggests (though the authors don't go there) that we might consider massive wind farms as a pure climate-change moderator, even if they weren't a practical power source from an engineering point of view.

It should probably be noted that the first two sponsors listed in the acknowledgments are British Petroleum and the Electric Power Research Institute.

The Great Transformation

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Statistical Arbitrage in the Sky

Reading this piece in the New York Times about Farecast, Oren Etzioni & co.'s attempt to forecast airline ticket price movements (through the power of machine learning!) leaves me with three reactions.

First, it's a relief to read something about data-mining and airlines which isn't Yet Another Unconstitutional Step Towards a National Surveillance State. In fact, this would be cool if it works.

Second, I'm professionally curious about how well Farecast's predictions, with over 100 independent variables, would compare to simple alternatives, like low-memory hidden Markov models, or throwing out all the variables except the time remaining until departure. (Since I'm teaching data mining in the fall, I'm also professorially curious.) I suspect that, even if there is some real improvement, it is small. Come to that, the trivial predictor which always forecasts a price increase is going to set a pretty high baseline accuracy...

Third, I'll be surprised if does work, not on technical grounds, but because I don't see how it makes sense for airlines to cooperate. The ideal airline pricing scheme is one which gouges you just enough that you're indifferent between taking the flight and not going at all (or taking some alternate mode of transport, etc.). This is why flights at ungodly hours are cheaper than those on the same route at decent times: by showing up at five in the morning to have your luggage prodded, etc., you signal to the airline that they really can't get any more money out of you. [1] Now suppose that you want to take a certain flight, and there's a maximum amount you'd pay for it. As things currently work, you look up the price and see what the airline is currently charging. If that price is less than its value to you, you buy the ticket, and the difference is your "consumer surplus". Now Farecast comes along and says, in effect "sure, you could do that, but the price is going to drop --- hold on and you'll do even better." So you buy when the price hits its trough, and are better thereby. (Yes, some people act like this now, I'd guess not many, and not very successfully.) From the airlines' point of view, however, every dollar by which your consumer surplus grows is a dollar they could have had. ("And I would have gotten away with it, too, if it hadn't been for you meddling KDDs!") Consumers and airlines are engaged in a zero-sum competition over the potential surplus, and this doesn't help the airlines.

Which, again, makes me very puzzled about why they would cooperate with it. The smart move on their side, I think, would be to systematically undermine the reliability of Farecast. This could be done very simply, without even attempting to reverse-engineer the predictor: monitor its forecasts of your own flights, and, all else being equal, do the opposite. It's true that a reliable forecast of a price increase isn't so bad, for the airlines, as a forecast of a price decrease, but systematically jamming and confusing Farecast should be easier than selectively doing so. But I defer to real economists about the importance of this wrinkle.

The larger moral ought to be a familiar one: in strategic interactions, you have to assume that the other side will adapt to you. This doesn't mean that statistical methods have no place in studying strategic interaction (see, e.g., the second paper here), but it does mean we should be very dubious about the ability of simple data mining to give us an advantage over an opponent as smart and determined as a commercial airline.

(Thanks to K. for sending me the article, and discussing it.)

1: If you want to understand the logic of airline pricing, among much else, a great read is Carl Shapiro and Hal Varian's Information Rules: A Strategic Guide to the Network Economy. [I have been sitting on a draft review for seven years now, and am not about to stop.] Despite the very late-1990s title, this is really about the general economic principles involved in any industry where high first-unit costs and low marginal costs give you positive economies of scale, or where there are strong positive network externalities. Airlines are in the first category, because the cost of getting a jet from New York to LA with 100 passengers is almost the same as getting it there with 101 passengers, and is mostly the cost of getting it there empty. Alas, appreciating the rational essence of the process does not help make the lived experience any more endurable.

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

The Sharks of DARPA

I wrote this about a year ago; I'm cleaning out my drafts folder.

As every schoolchild knows, what became the Internet was at first ARPANET, one of the many projects by which the Pentagon's Advanced Research Projects Agency nurtured the development of computer science and technology in the US. ARPA eventually became DARPA, but for a long time retained its character of a patron of basic and curiosity-driven research. Lately, however, it seems to have lost its touch, what with the whole terrorism futures markets debacle, "total information awareness", and so on. (I myself was for several years sustained, as a graduate student and post-doc, by my adviser's DARPA grant, familiarly known as cooperative agreement F30602-00-2-0583, part of the TASK program. Of course, it's not for me to say where our work fell in the spectrum from foundational to flaky.) The sad but predictable response has been to demand that DARPA de-emphasize basic academic research in favor of working with private-sector contractors for short-term military payoff.

All of which is by way of lead-in for this post to the connectionists mailing list:

Postdoctoral position in neurobiology / engineering in Woods Hole
A 4-year DARPA research project, funded annually, to steer the behavior of sharks in the natural environment through stimulation of selected sensory brain areas. Expertise in brain stimulation, multi-electrode recording and neural data analysis most desirable. Interfacing with wireless data transmission and stereotactic electrode positioning.

A year earlier, a Boston Globe story had more information, but lacked the admirably matter-of-fact tone of the job-ad.

I don't see much by way of follow-up, but I haven't looked very hard. For all I know, this work is already well on its way to giving us a beloved childrens' classic.

Learned Folly; Minds, Brains, and Neurons

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Topping from Below

Nature, in its wisdom, has compiled a list of the fifty most popular weblogs by scientists. I am more than a bit astonished to see this blog on the list, but my faith in the reading public is bolstered by seeing that it's only in the fiftieth place, and everything ahead of it that I recognize definitely deserves to be. (Via Aetiology [#7] and Pharyngula [#1].)

Linkage; Self-Centered

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Signs I Will Not Recommend Your Manuscript Be Published As Is (No. 734)

You make me want to quote the Medium Lobster's cost-benefit analysis of pre-emptively blowing up the moon in my report.

I hope you appreciate what it costs me to resist this temptation.

Friday Cat Blogging; Minds, Brains, and Neurons

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

This, This Is My People

From John Hall and Charles Lindholm's Is America Breaking Apart? (pp. 83--90, omitting footnotes):

One thing that has often been held to characterize Americans is the ambiguity, confusion, and "contagious vagueness" of their understanding of political theory. Americans may know, for example, that citizens have rights, but are extremely unclear about what those rights might be; they know Americans are supposed to be free, but not how freedom is limited, or what freedoms are permitted; they know that "all men are created equal," but cannot reconcile that precept with the protection of property. In other words, most Americans are very proud indeed of the principles that their country is built upon, but for them those principles consist primarily of abstract notions, such as liberty, justice, and equality, rather than a systematic set of specific precepts or practices.
The major sources of "contagious vagueness" are to be discovered in the priority Americans have always given to pragmatic experience over philosophical system-building. Under the fluid circumstances of American life, ambivalence and woolly abstraction have certain advantages. ... While remaining faithful to nebulous abstractions of "freedom," "individualism," and "equality," Americans can still be pliable in action without betraying their integrity or finding themselves immobilized by contradiction. Ideological vagueness thus allows Americans to feel a sense of unity without the trouble of actually considering exactly what that unity is based upon.
The abstraction and ambiguity of taken-for-granted foundational principles also allows Americans easily to "hold contradictory ideas simultaneously without bothering to resolve the potential conflict between them." This, too, is not necessarily a bad thing in a pluralistic society, where central authority is relatively weak. The blurry quality of American assumptions about their shared creed allows them to accept innovations easily, so long as the innovations are metaphorically bathed in the aura of tradition, and to react according to circumstances without too much concern about agreement with prior positions. Most important, an ability to ignore contradictions permits Americans to overlook disputes that might tear a more ideologically consistent society apart.
Corresponding to American abstraction and vagueness in the realm of political philosophy is a positive can-do approach to ordinary problems. ... It is no surprise that the predominant American philosophy is pragmatism.... However, what pragmatism takes for granted as "common sense" is actually a culturally constructed perspective, based in large measure on what has recently been called modular thinking. This is a strategy for instrumental action which assumes that complex wholes can be broken into elementary parts; these parts can then be efficiently recombined according to need. Modular thinking is American to the core: it is an atomistic, flexible, anti-organic, and anti-authoritarian view of the world — one which dispenses with tradition in favor of efficiency, and places all alternatives on an equal footing, subject to personal evaluation by the active innovator, who decides which combination is best.
Modular thinking has had a successful history in America. It is responsible for the development of the assembly line and Taylorist innovations in scientific management, and it provides the foundational principles behind everything from the construction of shopping malls to the planning of school curricula. ... [T]he pervasive pragmatic modular approach to life permits Americans to avoid divisive ideological issues by visualizing the world around them as a machine that can be retooled, or taken apart and rebuilt, in order to achieve maximum efficiency. ... Disagreements are not over principles, but over design. Though this mechanistic instrumental worldview may remove much of the magic from the cosmos, and though it certainly does not grasp complex social realities, it is not likely to arouse great passions either — and so is conducive to social peace.
It is especially striking that for Americans even the self is considered to be a kind of modular entity, capable of being reconfigured to fit into preferred life styles. This malleability is often decried as indicating American shallowness, or else praised as the postmodern triumph of the signifier. But the American emphasis on perpetual self-transformation also serves the cause of unity, though perhaps not in the way Protestant moralists would prefer. This is because the search for identity is a notoriously solipsistic pursuit: such quests do not lead to revolution, but to harmless participation in the therapeutic, self-help, and twelve-step groups that have so mushroomed in America. At the very worst, the search for an authentic self draws the most perplexed seekers towards immersion in the multitude of sects and cults that have always sprouted on American soil. Occasionally, it is true, these groups spiral into psychosis ... but generally these new religions are akin to the "healthy-minded," "once-born" faiths that William James found so characteristic of America. They typically affirm the goodness of all creation and preach accommodation with the world as it is, stressing mental discipline, while applying the optimistic American "can-do" attitude to spiritual uplift and practical self-betterment. Membership in them is no more harmful than membership in any local PTA.
American faith in the power of individuals to change themselves is quite understandable as a product of the immigrant experience in combination with the Protestant ethos. Protestant sects believe that individuals can be spiritually transformed through disciplined, virtuous action in this world. For most of the original settlers immigration to America was just such a transformative action, a voluntary pilgrimage in search of the City on a Hill. In secular garb, this model continues to hold: becoming an American is a kind of conversion experience. The newcomer "is not required to learn a philosophy," Daniel Boorstin notes, "so much as to rid his lungs of the air of Europe." This point is not invalidated by the fact that more recent immigrants, male and female, rid their lungs of the air of China, India, and Africa. For all these newcomers, past and present, America has been the "Mother of Exiles. From her beacon-hand glows world-wide welcome." The content of the glowing welcome offered by the Statue of Liberty is not a dogma, but an opportunity. America presents itself as a place where newcomers can achieve their dreams, free at least from the chains of tradition, class, and history...
For the zealous believer of colonial times, the end sought through migration to America was a passage into heaven.... For the modern entrepreneur, the goal is likely to be far more mundane: owning one's own business and acquiring a house in an exclusive suburb. Whether what they sought was spiritual or material, immigrants to America have worried little about conceptual consistency or a systematic organization of principles. Central instead is a belief that individuals have the capacity, through personal effort, dogged discipline, and creative innovation, to leave the past behind, to pursue happiness, and to become whatever their potential allows. Only in America would the Army call on its recruits to "be all you can be."
We can feel the heady appeal of this transformative aspect of American life in a letter sent by a French migrant to California during the Gold Rush:
In the mist of this world of adventurers, who change their occupations as often as they do their shirt, egad, I did as the others. As mining did not turn out remunerative enough, I left it for the town, where in succession I became a typographer, a slater, plumber, etc. In consequence of thus finding out that I am fit for any sort of work, I feel less of a mollusk and more of a man.
For such adventurous souls, America indeed offered — and continues to offer — an opportunity for taking on a new and better identity, for making a mollusk into a human being.

They go on to talk about why we are so friendly and nice, and why believe that love is the only legitimate bond between people, but I'm skirting the bounds of fair use as it is, even in the interest of patriotic display.

The Beloved Republic

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Neuropharmacological Foundations of the Public Sphere

The importance of coffee-houses in the Enlightenment, and the rise of the public sphere, is a historical common-place. But it's also puzzling: historians can say a lot of sensible things about how, as a social setting, the cafe was conducive to the give and take of (more or less) rational argument, and (relative) indifference to social standing in favor of persuasion. But I've never heard a good story for why coffee houses had to be run that way, nor that (say) taverns weren't, or couldn't have been, run that way. So, while not denigrating the social factor, it doesn't seem to explain why this connection took hold. Now, at last, scientific proof that Enlightenment had a sound material basis (via Mind Hacks):

Pearl Y. Martin, Jenny Laing, Robin Martin, and Melanie Mitchell, "Caffeine, Cognition, and Persuasion: Evidence for Caffeine Increasing the Systematic Processing of Persuasive Messages", Journal of Applied Social Psychology 35 (2005): 160--182 [full-text PDF free via that link]: Abstract: Caffeine is known to increase arousal, attention, and information processing—all factors implicated in facilitating persuasion. In a standard attitude-change paradigm, participants consumed an orange-juice drink that either contained caffeine (3.5 mg/kg body weight) or did not (placebo) prior to reading a counterattitudinal communication (anti-voluntary euthanasia). Participants then completed a thought-listing task and a number of attitude scales. The first experiment showed that those who consumed caffeine showed greater agreement with the communication (direct attitude: voluntary euthanasia) and on an issue related to, but not contained in, the communication (indirect attitude: abortion). The order in which direct and indirect attitudes were measured did not affect the results. A second experiment manipulated the quality of the arguments in the message (strong vs. weak) to determine whether systematic processing had occurred. There was evidence that systematic processing occurred in both drink conditions, but was greater for those who had consumed caffeine. In both experiments, the amount of message-congruent thinking mediated persuasion. These results show that caffeine can increase the extent to which people systematically process and are influenced by a persuasive communication.

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Friday Cat Blogging (The Parallelogram Paw Issue of Science Geek Edition)

Kara has a charming habit of secreting herself behind some corner and landing a devastating swipe of her paw on the calves of passers-by. This would be very effective at crippling us, if only she had not been cruelly declawed at some point before we adopted her from the shelter. Today's post, however, will not be about the irreducible intensionality revealed by her desire to bring down prey much larger than herself, nor about the qualia of her phantom claws (which I am convinced she experiences). Rather, I want to talk about the mechanism which brings her paw — and with it her fore-arm and shoulder and the rest of her body — to move just so as to effectively ham-string us. This mechanism is accessible to experimental investigation, and, as it turns out, pretty nearly linear.

Christian Ethier, Laurent Brizzi, Warren G. Darling and Charles Capaday, "Linear Summation of Cat Motor Cortex Outputs", The Journal of Neuroscience 26 (2006): 5574--5581 [Thanks to Prof. Capaday for a reprint]: Abstract: Recruitment of movement-related muscle synergies involves the functional linking of motor cortical points. We asked how the outputs of two simultaneously stimulated motor cortical points would interact. To this end, experiments were done in ketamine-anesthetized cats. When prolonged (e.g., 500 ms) trains of intracortical microstimulation were applied in the primary motor cortex, stimulus currents as low as 10-20 \mu A evoked coordinated movements of the contralateral forelimb. Paw kinematics in three dimensions and the electromyographic (EMG) activity of eight muscles were simultaneously recorded. We show that the EMG outputs of two cortical points simultaneously stimulated are additive. The movements were represented as displacement vectors pointing from initial to final paw position. The displacement vectors resulting from simultaneous stimulation of two cortical points pointed in nearly the same direction as the algebraic resultant vector. Linear summation of outputs was also found when inhibition at one of the cortical points was reduced by GABA_A receptor antagonists. A simple principle emerges from these results. Notwithstanding the underlying complex neuronal circuitry, motor cortex outputs combine nearly linearly in terms of movement direction and muscle activation patterns. Importantly, simultaneous activation does not change the nature of the output at each point. An additional implication is that not all possible movements need be explicitly represented in the motor cortex; a large number of different movements may be synthesized from a smaller repertoire.

To elaborate a little on the experimental procedure: the cats were anesthetized with ketamine, which allowed for surgery to attach sensors measuring electrical activity to eight muscles, as well as brain surgery opening up the motor cortex. They also attached a sensor to the paw, tracking its position in space. Then, maintained on a steady flow of ketamine, the "body of the cat was laid on a cushion with its forelimbs hanging perpendicular to the ground and free to move in all directions against gravity". At this point, they began small-scale, low-current stimulation of the motor cortex, until they identified a number of points each of which, when stimulated, produced a repeatable and distinct motion of the paw and fore-limb. They recorded both the displacement of the paw, and the total activity, over the course of the movement, of each of the eight muscles. Thus, each point in the cortex corresponded to both a three-dimensional vector, in ordinary space, and an eight-dimensional vector, in muscle space.

In a linear system, if you add two inputs, you also add two outputs; this, and nothing more, is what "linear" means. Vectors add according to the parallelogram rule, so if the cat motor system is linear, if we stimulate two points simultaneously, the movement of the paw should be the superposition of the movements produced by either point on its own. I wish I could build some suspense at this point, by harping about how neurons are notoriously nonlinear devices, so it's madness to expect any sort of linearity here, but Ethier et al.'s title rather gives the show away. When they simultaneously stimulated pairs of points, they got paw motions which were almost perfect linear sums of the individual movements. This was true of both the paw-displacement vector, and the muscle-activation vectors. (See especially their figures 2 and 5, and accompanying text.) It didn't matter whether they electrically stimulated both points, or stimulated one point while chemically reducing the inhibition at the other. This is very nice, but what I like even more is the experiment summarized in their figure 7, where they took two cortical points, which produced nearly perpendicular movements on their own, and by varying the magnitude of the stimuli at each, got a sequence of movements which smoothly interpolated between them, exactly as one would hope for a linear control system.

Now, I should say that the linearity of response wasn't perfect. The largest systematic deviations from linearity occurred when summing the individual motions would have produced the largest displacements — in a word, the muscle response saturated. To avoid this effect, Ethier et al. first established an input-output relationship for each cortical point on its own, and kept the stimulus magnitude low enough to avoid saturation there. They suspect the place where the response became sub-linear was in the spinal cord, but they don't, that I can see, really establish it.

There are two different larger morals to be drawn from this story. One has to do with the functional anatomy of the motor cortex (a larger, long-running story nicely presented in an older paper of Capaday's). There is clearly a great deal of localization of function there — this point produces a swiping motion of the paw, that that one pulls it back towards the chest, neither makes the tail twitch — but of a peculiar sort. One might well imagine that each point in the motor cortex would correspond to a particular muscle or group of muscles; instead, at least in the part Ethier et al. worked with, they seem to correspond to motions involving many muscles to varying degrees, overlapping from one cortical point to another. Linearity means that a reasonably small set of motions could serve as a basis for a vast range of coordinated actions, without all of those having to be separately stored in the motor cortex.

The other moral has to do with the general principles of neural representation and computation. Neurons are, indeed, horribly nonlinear little things, so it would be entirely reasonable to suppose that neural codes are too; but that would be too quick. One of the few efforts in this area that is general, abstract and predictive enough that it seems to me to be worth calling a theory, the "neural engineering" advanced by Chris Eliasmith and Charles Anderson in a book of that title, takes as its first principle "nonlinear encoding and linear decoding". That is, while the mapping from input to output is hairy and nonlinear, for typical outputs you can recover the input, to high accuracy, using a linear rule. This is especially easy to arrange in neural systems where excitation and inhibition are neural balanced, so Ethier et al.'s findings on dis-inhibition fit in nicely.

Nonlinear encoding and linear decoding is not just an assumption of Eliasmith and Anderson, but, e.g., features quite prominently in Spikes, and is implicit in the now-standard "reverse correlation" method. While I am not, usually, one to argue with scientific success, I have reservation about this. William James used to decry, as "the psychologist's fallacy", the "confusion of his own standpoint with that of the mental fact about which he is making his report" (Principles of Psychology, ch. 7). Something similar (the "computational neuroscientist's fallacy", perhaps?) seems to me involved here. Neural representations do not exist to be decoded by scientists, but to be used by other parts of the organism, and ultimately to produce adaptive actions. What is lacking, in most of these studies, is evidence that linear decodings of neural activity are in any way biologically relevant. (One of the nice things about Eliasmith and Anderson is they see at least part of this, since their second principle is that other parts of the brain use a neural representation by applying alternately-weighted linear transformations to it, i.e., biased linear decodings. But they present less evidence for this than for their first principle.)

In this case, however, I don't find much room for doubt: the points in the motor cortex represent actions, and those representations are, when the paw meets the calf, linearly decoded. It's still not clear to me that linear decodings and transformations are any easier for the brain to implement, but at least in this case that's what's going on, and it's an empirical fact we will have to incorporate in our models, or ideally explain in our theories.

And now, if you will excuse me, I'm being attacked.

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Corners Bumped

A year and a half ago, I commenced a major purge of my book collection. A year ago, when I left Ann Arbor for Pittsburgh, I consigned thirty-odd boxes of used books to Corners Bumped, a.k.a. my friends Bill and Barbara Tozier. (This tells you something about the number of books I did end up shipping here.) Recently, they have been refurbishing their online book-selling presence, listing new stuff from my boxes and their own stock, and musing (1, 2, 3, 4) on used books and the trade therein. Go buy something, will you? It will make all of us — you, me, Bill and Barbara, and not least the books — better off.

This concludes today's crass commercial plug.

Linkage; Self-centered

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Problems in the Doctrine of Chances

Paradoxes and fallacies are fun teaching tools, and elementary courses in probability theory are well supplied with them. I have long thought that more advanced probability classes would be more palatable if we presented our students with more opportunities to go "Wait, that can't be right", and sink their teeth into something really just-plain-wrong. Happily, we now have about ten more such morsels to offer them:

Guang-Liang Li and Victor O. K. Li, "Inconsistencies in Current Theories of Real Numbers, Measure, Probability, and Stochastic Processes", math.GM/0606635: Abstract: Current theories of real numbers, measure, probability, and stochastic processes appear to be self-inconsistent. We exemplify the inconsistencies with Poisson processes, Markov chains, continuous random variables, the Lebesgue measure, and the Cantor-Dedekind theory of real numbers.

Anyone taking 36-754 from me next spring should expect to see sections 2 and 3 as assignments.

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Snowclones in May!

I'm afraid I was a bit premature in announcing my return to posting, since I find my time being taken up by actually doing some science, and being, for the first time, on the sending rather than the receiving end of a qualifying exam. In the meanwhile, let me enthuse about Far from the Madding Gerund, a selection (and annotation) of the best of Mark Liberman's and Geoffrey Pullum's posts from Language Log. Doing a proper review, or even justice to their weblog, is really beyond what I have time for, but this book seems to capture quite perfectly their mixture of whimsy, skepticism, accessible scholarship, and pure good-natured zeal for their subject. If you have been reading them, then you will find these little essays as good as you remembered, if not better, and you'll wonder how you forgot about some; if you have not been reading Language Log, then their book should convince you to start. Either way, reading it will make you better and happier. (Disclaimer: I got an unsolicited review copy of this book this week, which didn't help my productivity any. Also, when I gave a talk at Penn on April Fool's Day last year, Mark was kind enough to let me crash in the hospitality suite at One Language Log Plaza.)

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Friday Cat Blogging (Staging an Intervention Issue of Non-Science-Geek Edition)

Kara has Views about humans working on Friday nights:

Friday Cat Blogging

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Reasons to Be Cheerful: Higher Primates Issue

Marc Hauser and Bailey Spaulding, "Wild rhesus monkeys generate causal inferences about possible and impossible physical transformations in the absence of experience", Proceedings of the National Academy of Sciences (USA) 103 (2006): 7181--7185 [PDF reprint. More exactly, they are less surprised if they see an apple being cut in half by a knife than by a glass of water, even if they have not seen knives cutting apples over and over again, so as to build up an expectation of constant conjunction on that basis. The deeper point Hume was making (with al-Ghazzali before him) was that, however we acquire our causal expectations, all they mean empirically is constant conjunction, a "habit of the world". Higher primates can be very good at making guesses about those habits.]
The Second Mrs. Kong, an opera with libretto by Russell Hoban and music by Sir Harrison Birtwistle. [Via John Burke, in e-mail]
Victory Brewing Company's Golden Monkey tripel ale.

Philosophy; The Commonwealth of Letters

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Partway Down the Danube

Hopefully, it's just jet-lag, but seeing highway signs marked "Wien", "Praha" and "Budapest" makes me feel much the same way one marked "Minas Tirith" would, that something from known only from books has become unsettlingly real.

Self-Centered

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

A Scene from the Café Central

Or rather, from Ernest Gellner's Language and Solitude: Wittgenstein, Malinowski and the Habsburg Dilemma (pp. 108--109):

Imagine the conversation of a few Viennese characters, in the Cafe Central, on the assumption that the Tractactus is a correct account of the human condition.
CHARACTER A: High up on the left hand of my visual field, I note a fact in which a 9-value predicate links the appropriate number of things... I don't think I have had this one before, I'd like a snapshot for my collection.
B: I have a much more interesting one right in the middle of my field, a variable with 127 things attached — it has a lot of tentacles, holding those things. I've counted them.
C (to A): Don't you believe him. I have known him for years, he is invariably given to exaggerating the complexity of his facts, just to make himself interesting.
D (to C): I don't think he is deliberately lying, he just drinks too much and then imagines things.
B (furiously, to both C and D): What you have both said is extremely offensive and I have no option but to call you out! My seconds will call on you. That is, assuming you have honour: my fraternity has decreed that Jews have none and we may not duel with them. Is either of you at least half Aryan? That is all one can hope for in Vienna these days.
A: Gentlemen, gentlemen, please calm down. May I remind you first of all that dueling is forbidden by law and, secondly, death not being an event in life, is totally pointless anyway.
The above conversation is not copied out of Die letzten Tage der Menscheit or Der Weg ins Freie or Der Mann ohne Eigenschaften or even Radetzky Marsch. It is all my own work and it is copyright.

[Thanks to Wolfgang for checking my transcription of German titles!]

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Statistical Network Analysis: Call for Papers

As a member of the program committee for the workshop Statistical Network Analysis: Models, Issues and New Directions, part of the 2006 International Conference on Machine Learning, I urge you to submit your best work by 28 April; you can send us the bad stuff after that. (That, in response to hearing endless variants on "First prize, a trip to Pittsburgh; second prize, two trips", which was old in my grandfather's day.)

CALL FOR PAPERS

Statistical Network Analysis:
Models, Issues and New Directions
a workshop at the
23rd International Conference on Machine Learning
(ICML 2006)
Thursday, June 29, 2006, Pittsburgh PA, USA
Overview:

This workshop focuses on probabilistic methods for network analysis, paying special attention to model design and computational issues of learning and inference.
Many modern data analysis problems involve large data sets of artificial, social, and biological networks. In these settings, traditional IID assumptions are blatantly inappropriate; the analyses must take into account the structure of relationships between the data. As a result, there has been increasing research developing techniques for incorporating network structures into machine learning and statistics.
Network modeling is an active area of research in several domains. Statisticians have mostly concentrated on models of static networks. These models are concerned with the existence of edges between individual nodes, but do not attempt to model aggregate properties. In contrast, physicists have addressed global properties of large complex networks. Their models describe average statistics of the network, or properties of typical networks in large ensembles; the links between particular nodes are less meaningful.
This workshop aims to bring together statistical network modeling researchers from different communities, thereby fostering collaborations and intellectual exchange. Our hope is that this will result in novel modeling approaches, diverse applications, and new research directions.
We wish to clarify that in this workshop, the word "relational" carries a different meaning from the usual sense of the word in Probabilistic Relational Models (PRMs). For example, in real life, any two random people maybe connected through a complex web of friendships; estimation of interpersonal connections thus cannot be done independently of the rest of the network. We focus on modeling statistical properties of the network, as opposed to different types of probabilistic relations. This differentiates us from the co-located ICML workshop on Statistical Relational Models.
Online Submissions:

We welcome the following types of papers:

research papers that introduce new models or apply established models to novel domains,
research papers that explore theoretical and computational issues,
position papers that discuss shortcomings and desiderata of current approaches, or propose new directions for future research.
We encourage authors to emphasize the role of learning and its relevance to the application domains at hand. In addition, we hope to identify current successes in the area, and will therefore consider papers that apply previously proposed models to novel domains and data sets.
Submissions should be limited to a maximum of 8 pages, and adhere to ICML format. Please email your submissions to: edo [at] cmu.edu.
Deadline for Submissions: Friday, April 28, 2006
Notification of Decision: Friday, May 5, 2006
Format:
This is a one-day workshop. It will consist of several themed sessions targeting methodological and application issues (e.g., estimation in static models, network evolution modeling, and statistical modeling of large scale networks) with talks (invited and contributed) and moderated discussion. Discussions at the workshop will facilitate exchanging of research ideas and help identify other challenging problems in the area. At the end of the workshop, a panel of statisticians, physicists, and computer scientists will discuss the points arising throughout the day and identify the most promising and challenging directions.
Publication:
Accepted papers will be distributed on a CD and made available for download.
Organizers:
Edo Airoldi, Carnegie Mellon University
David Blei, Princeton University
Stephen Fienberg, Carnegie Mellon University
Anna Goldenberg, Carnegie Mellon University
Eric Xing, Carnegie Mellon University
Alice Zheng, Carnegie Mellon University
Program Committee:
David Banks, Duke University
Peter Dodds, Columbia University
Lise Getoor, University of Maryland
Mark Handcock, University of Washington, Seattle
Peter Hoff, University of Washington, Seattle
David Jensen, University of Massachusetts, Amherst
Alan Karr, National Institute of Statistical Sciences
Jon Kleinberg, Cornell University
Andrew McCallum, University of Massachusetts, Amherst
Foster Provost, New York University
Cosma Shalizi, Carnegie Mellon University
Padhraic Smyth, University of California, Irvine
Josh Tenenbaum, Massachusetts Institute of Technology
Stanley Wasserman, Indiana University

Networks; Enigmas of Chance

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

European Conference on Complex Systems 2006

will be held at Oxford, 25--29 September. Since I'm on the program committee, I ought to point interested parties to the call for papers (deadline 7 April) and the call for workshop proposals (deadline 5 May).

Complexity

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

SUMO versus the Dendritic Claws (Sunday Sibling Blogging)

I occasionally dabble in mathematical aspects of neuroscience. My brother does the real thing:

Aryaman Shalizi, Brice Gaudillière, Zengqiang Yuan, Judith Stegmüller, Takahiro Shirogane, Qingyuan Ge, Yi Tan, Brenda Schulman, J. Wade Harper,and Azad Bonni, "A Calcium-Regulated MEF2 Sumoylation Switch Controls Postsynaptic Differentiation", Science 311 (2006): 1012--1017: Abstract: Postsynaptic differentiation of dendrites is an essential step in synapse formation. We report here a requirement for the transcription factor myocyte enhancer factor 2A (MEF2A) in the morphogenesis of postsynaptic granule neuron dendritic claws in the cerebellar cortex. A transcriptional repressor form of MEF2A that is sumoylated at lysine-403 promoted dendritic claw differentiation. Activity-dependent calcium signaling induced a calcineurin-mediated dephosphorylation of MEF2A at serine-408 and, thereby, promoted a switch from sumoylation to acetylation at lysine-403, which led to inhibition of dendritic claw differentiation. Our findings define a mechanism underlying postsynaptic differentiation that may modulate activity-dependent synapse development and plasticity in the brain.

Also check out the accompanying "perspectives" piece.

Minds, Brains, and Neurons

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Friday Cat Blogging (Measure-Zero Exception to the Hiatus Issue of Non-Science-Geek Edition)

Kara consults her authorities. (What, you didn't think I was writing this without help, did you?)

Friday Cat Blogging; Enigmas of Chance

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Graphs, Trees, Materialism, Fishing

Attention conservation notice: Over 6000 words of dubious value. Written as a contribution to the Valve event on Franco Moretti's Graphs, Maps, Trees. Contains advice to literary scholars from someone utterly unqualified to give it, hypothesis testing, rastergrams, many long quotations, and ruminations on materialism and rational history. If this is the sort of thing you're interested in, you'd really be better off reading Moretti yourself.

Cross-posted to the Valve, where there is a comments section. I can't promise to reply...

A few years ago, I wrote a review of Moretti's Atlas of the European Novel, in which I presumed to tell him how to go about his business. When he ran across it, his reaction was not (as mine would've been, had our situation been reversed) to tell me where to get off, but to invite me to a workshop he was organizing at Stanford on new interdisciplinary work on the novel — its motto, the quotation from Brecht about "questions that appear to us completely unsolved", is recycled for this book — where I had a great time. Reading these essays as they came out in New Left Review, I enjoyed them greatly, and recall thinking that Moretti could hardly have done a better job of appealing to my prejudices if he'd tried. (Said prejudices are those of someone almost equally fond of The Extended Phenotype and Main Currents of Marxism.)

With this kind of background, it comes as no surprise, I trust, that I really like this book, and finding objecting to what he actually proposes here highly wrong-headed. In what follows, I want to say a bit about "graphs" and a bit about "trees", and explain why this sounds so promising to me. I am not going to say anything about "maps", because I don't think I have anything to add to that discussion, but I will, for the sake of getting an M in there, end with some remarks on "materialism". At no point can I pretend to be competent to evaluate the originality of Moretti's work within literary scholarship, to say how much of a departure, say, the trees really are. In a feeble attempt to pretend that my price is higher than a weekend in California and a review copy, I will make some criticisms, most about tedious extra stuff I wish Moretti had also done. I'd like to think that what I say will also have some value for those who don't share my rather haphazard intellectual trajectory, but my experience with trying to communicate across disciplines means I'll get a warm glow if I'm even comprehensible, never mind persuasive. I am accordingly very grateful to the Valve, and especially to Jonathan Goodwin, for letting someone with my credentials (viz., none) participate in this event.

Graphs

Do Genres Come in Bunches?

Moretti makes a very striking claim in his first chapter: that genres of novels appear together, in clusters, separated by about 25 years, and disappear together too. Looking at his graph, my eye agrees, but my eye also tells me that there are faces in clouds (the East African Plains Ape is an incorrigible pattern-finder), and probability theory tells me that purely random processes can produce a lot of apparent clustering and regularity. What reason is there to think that what looks like genres coming in clusters isn't just coincidence?

Let's be a little more precise about what we'd mean by "chance" and "coincidence" here. One natural possibility is that new genres appear at a constant rate over time, utterly independently of one another. Every year, then, there would be a constant probability of a new genre forming, but whether it did or not would have no bearing on whether the next year saw a new genre. This is our null model — the one which says what things should look like if we're just fooling ourselves, and there are no clusters. To get slightly technical, the distribution of intervals between genre-arrivals should have what's called a geometric distribution. Assuming, for the sake of argument, that that's true, we can use the average time between genre-appearances (3.44 years) to estimate the most likely value for the probability of a new genre appearing in any given year (about 29%).

Once we assume that the inter-arrival distribution is geoemtric and find the parameter, we can simulate from it, and get examples of what Moretti's graph would look like, if only chance were at play.

The top line shows the appearance dates of Moretti's 44 genres; the next two lines give the results of simulating from a model of uniform random appearance, with the same mean time between genres as the actual history.

Is there more clustering in reality than in the results of the null model? I couldn't say, by eye, but I don't have to. I can calculate the probability of generating Moretti's history from the null model: it's somewhat less than 1 in 10^45. This in itself isn't decisive, since any particular history becomes less and less probable as one considers longer and longer intervals of time (cue Stoppard), so we need to know what fraction of all histories of that length are at least that unlikely. I could work this out exactly, if I were willing to do some actual math, but I'm lazy, so I just had the computer simulate a million histories and evaluated all of their likelihoods. If the null model were actually true, we'd see histories like Moretti's only about 0.4% percent of the time. [1] So this is actually pretty good evidence that the null model is not true, and Moretti's history does show the kind of clustering he thinks it does.

Of course, this only underlines the question of why Moretti's data is clustered. I can think of a couple of deflating explanations (maybe the clusters match the periods more intensely scrutinized by historians; maybe they tend to adjust when they report genres appearing towards certain focal dates). Or it could be due to some sort of exogeneous influence, from war, politics, economic shifts, etc. (I did not try removing the obviously-topical genres, like Chartist novels, and repeating the analysis.) Or it could be due to some sort of endogenous mechanism within the system of literary production and consumption — generational turn-over of authors, of readers, of editors and publishers (suggested by my friend Bill Tozier). Or: maybe there's some space of things-people-like-in-novels, which the popular genres at any one time partition up in various ways; if one genre dies out or another appears, this might destabilize all the others as well. I don't think Moretti's time series, by itself, is enough to begin to let us decide among these mechanisms (some of which are compatible), but I do think it lets us see that some mechanism is called for.

Here is my first reproach: Moretti should have been the one to do this analysis, not me. If testing hypotheses is too banausic and mechanical for the pages of New Left Review, then it should either be in another article, or in the book. Moretti is a shrewd man, and in this case his intuitive analysis of the data was right, but there is no reason to rely on intuition alone for something like this. And, if one is going to go to the trouble to collect quantitative data, one ought to use it quantitatively. Mathematical abstraction (quantitative or otherwise) is not valuable for its own sake, but for the inferences it lets us make, when the proper tools are applied. In this case, those tools are pretty easy to bring to bear. They should be.

Dissolving Genre History

Here is Moretti at the end of "Graphs":

For most literary historians ... there is a categorical difference between 'the novel' and the various 'novelistic (sub)genres': the novel is, so to speak, the substance of the form, and deserves a full general theory; subgenres are more like accidents, and their study, however interesting, remains local in character, without real theoretical consequences. The forty-four genres of figure 9, however, suggest a different historical picture, where the novel does not develop as a single entity—where is 'the' novel, there?—but by periodically generating a whole set of genres, and then another, and another... Both synchronically and diachronically, in other words, the novel is the system of its genres: the whole diagram, not one privileged part of it. Some genres are morphologically more significant, of course, or more popular, or both—and we must account for this: but not by pretending that they are the only ones that exist. And instead, all great theories of the novel have precisely reduced the novel to one basic form only (realism, the dialogic, romance, meta-novels...); and if the reduction has given them their elegance and power, it has also erased nine tenths of literary history. Too much.

On the one hand, this seems to me to be obviously correct. On the other hand, I wonder very much why Moretti stops here. If we look within any one of those forty-four genres, I think we have every reason to suppose that we'd find it composed, in its turn, of sub-genres, and so on, and ultimately of a shift succession of individual texts. "The" Bildungsroman (to pick one of the forty-four, not entirely at random) is a short-hand way of referring to the most common and enduring features of a historically-changing and always-various population of books, just as "the" bottle-nosed dolphin is an abbreviation for the leading tendencies of a certain population of organisms. What Moretti hints at, in the paragraph I quoted, is that "the" novel is itself a population, either of genres, or of texts structured into genres. But he doesn't say outright what seems very plain to me, and so I'd like to know why, and specifically whether he thinks it's actually wrong, or unhelpful.

The assumptions of population thinking are diametrically opposed to those of the typologist. The populationist stresses the uniqueness of everything in the organic world. What is true for the human species—that no two individuals are alike—is equally true for all other species of animals and plants. Indeed, even the same individual changes continuously throughout its lifetime and when placed into different environments. All organisms and organic phenomena are composed of unique features and can be described collectively only in statistical terms. Individuals, or any kind of organic entities, form populations of which we can determine the arithmetic mean and the statistics of variation. Averages are merely statistical abstractions, only the individuals of which the populations are composed have reality. The ultimate conclusions of the population thinker and of the typologist are precisely the opposite. For the typologist, the type (eidos) is real and the variation an illusion, while for the populationist the type (average) is an abstraction and only the variation is real. [2]

This makes salient the question of how we mark off different populations as distinct. The usual biological criterion is through common descent, and the possibility of inter-breeding —- Mayr's "biological species concept". (There is a vast controversial literature on the details.) Ruth Garrett Millikan has a closely related notion of "reproductively-established families", which doesn't lean so heavily on the details of biology, and which would seem to fit the case of genres of novels. One could also define classes of texts purely morphologically, which might include many unrelated lineages (just as one might consider all streamlined marine predators which live in the water all the time, a class including dolphins, killer whales, sharks, tuna, ichthyosaurs, etc.). Just as such organic forms have appeared in several lineages, morphologically-defined categories could appear in multiple places and periods, the way novels arose, apparently quite independently, in both the Hellenistic world and in China (and elsewhere, for all I know). Historical populations, however, are unique.

Trees

One could ... take evolutionary bibliography as the prototypical evolutionary science and think of biology in terms of bibliographic analogies... [3]

The Cabinet of Horrors

When trying to explain cultural change and cultural variation, people have generally sought to do so by supposing culture is causally driven by something else (the climate, the social structure), or, even more strongly, that it is adapted to something else, or, more strongly yet, that it functions adaptively for the benefit of something else (here social structure, or ruling classes, are favored as suspects over the climate). This has led to an awful lot of (if I may use the phrase) adaptationist just-so stories, and uncritical analogy-mongering on a level with the sort of thinking which leads rhinoceros horn to be prescribed for impotence. Jon Elster is worth quoting at some length:

In his comments on the links among capitalism, Protestantism, and Catholicism Marx set a disastrous precedent for many later writers who have attempted to find "structural homologies" or "isomorphisms" (two fancy terms for "similarities") between economic structures and mental products. Because virtually any two entities can be said to resemble each other in some respect, this practice has no constraints other than the inventiveness of and ingenuity of the writer: There are no reality constraints and no reality control.
Marx suggests two inconsistent lines of argument. One is that there is a strong connection between mercantilism and Protestantism, the other that there is an elective affinity between mercantilism and Catholicism. He was confused, apparently, by the fact that money has two distinct features that point to different religious modes. On the one hand, money (gold and silver), unlike credit, can be hoarded. Hoarding easily turns into an obsession, which is related to the fanatical self-denying practices of extreme Protestantism. On the other hand, money can be seen as the "incarnation" or "transubstantiation" of real wealth. In that sense the money fetishism associated with mercantilism is related to the specifically Catholic practice of investing relics and the like with supernatural significance. Both arguments are asserted several times by Marx, each serving to show up the essential arbitrariness of the other. Later attempts to explain the theology of Port Royal, the philosophy of Descartes, or the physics of Newton in terms of similarities with the underlying economic structure are equally arbitrary. Like the analogies between societies and organisms that flourished around the turn of the century, they belong to the cabinet of horrors of scientific thought. Their common ancestor is the theory of "signs" that flourished in the century prior to the scientific revolution inaugurated by Galileo — the idea that there are natural, noncausal correspondences between different parts of the universe. What Keith Thomas refers to as the "short-lived union of science and magic" maintained a subterranean existence of which the doctrine of ideology, in one of its versions, has been one manifestation. [4]

Even if we shutter and lock the Cabinet of Horrors, and go to look for explanations of trends in such cultural products as novels (which is, after all, what Moretti wants), I'm afraid we will find most of them in the capacious Closet of Mildly Appalling Objects. There is no shortage of attempts to give such changes meaning as signs of something else, some aspect of the social or economic structure, of the way we live now (or the way they lived then), but very, very few of them are convincing. In his great book on changing fashions, A Matter of Taste, the sociologist Stanley Lieberson looks at some of the reasons why these attempts at ad hoc explanation are so often bad. (He puts things more politely; I paraphrase.) First, the facts are often just screwy, both about the developments to be explained: non-existent trends, non-existent causes, weirdly mis-characterized trends, trends being explained by events which happened long after the former began, etc. (In fairness, such "scholarly misconstruction of reality" is a lot more common than we academics like to think.) Second, the mechanism connecting the explanatia to the explananda is left totally obscure. Third, no attempt is made to test the explanation, by checking that it can account for the magnitude of the observed change, by ruling out alternative explanations, or by much of anything else. The result is a steady stream of claims about how culture works which are advanced with what is, under the circumstances, an astonishing degree of assurance. Lieberson's book provides many fine examples of such cavalier just-so story-telling for names, the decline of hats, etc. [5]

Checking hypotheses about causation, and still more about adaptation, is really hard with just one case, arguably hopeless. What you need is the ability to reliably detect departures from the hypothesis, if they are actually present — "power", in the statisticians' jargon. It is hard to get much power when n=1. If you want to claim that certain aspects of 19th century British novels were the way they were because those features fitted with ideologies of British imperialism — a fairly strong hypothesis about adaptation — I don't see how you can do it just by interpreting Mansfield Park, no matter how subtle and sophisticated your reading. On the other hand, if you look at lots of contemporary novels, and the ones which (say) depict Great Britain's relations with its colonies in the same way as Mansfield Park does are systematically more successful, on average, than those which depict it differently, well then I don't see how that couldn't be good news for your idea, though even that would really only be the beginning of backing it up.

Biologists have given a lot of thought to checking hypotheses about adaptation, and developed many means of doing so. Mutatis mutandis, many of these means could also be applied to literature, or other aspects of culture. Eric Rabkin, Carl Simon and their collaborators have started doing just this with their Genre Evolution Project, looking at short stories from 20th century American science fiction, and no doubt there are others doing this kind of thing too.

One way of checking adaptive hypotheses, especially relevant here, is the "comparative method", or rather methods, which work much, much better when combined with good phylogenies. I think a literary historian who wants to study the evolution of genres and devices would be very well advised to look at the comparative methods biologists employ to study the evolution of qualitative characteristics of organisms. (The major issue would be that literary phylogenies will not be trees but more complicated lattices. But this is analogous to the effects of lateral gene transfer, common among bacteria, and so I'd suspect not only solvable but solved, someplace in the literature. Whether inheritance is by means of discrete-valued, particulate factors, i.e., genes, is not a crucial issue for such methods.) What I really want to see from Moretti (or someone) is a study along these lines of clues in the detective story; I'd be even more interested in one of free indirect discourse.

A crucial aspect of testing hypothesis about adaptation is a contrast with the outcome of a well-crafted neutral model — a way of saying what to expect if no adaptation were present, or not that adaptation anyway. These often have surprising consequences; for instance, neutral genetic drift will tend to fix some version of a gene in a given population, even if it confers no fitness advantage. (This is described in any book on population genetics.) So I wonder about things like whether we should expect, under a reasonable neutral model, that some formal device should become universal within a genre? If so, did clues take over detective stories any faster than neutrality would predict? (It's hard to imagine a successful genre where every story relies on confessions found by accident, but whether that's intrinsically weirder than actually existing detective stories, I can't say.)

The foregoing shouldn't be taken to mean that comparative literature should slavishly imitate comparative biology. There are people who have thought about the application of evolutionary ideas to social and cultural change in ways which are much more sophisticated about psychology, social organization and human interaction than (most) advocates of memetics; I am thinking particularly of David Hull, W. G. Runciman, Dan Sperber, Stephen Toulmin's great The Collective Use and Evolution of Concepts, and even the fragmentary MS. of Adam Westoby. As the economist Richard Nelson writes, we should expect our ideas of general evolution to change as we learn more about cultural evolution. We should also expect to have to develop different methods of data analysis. But, as always, we start with what we already know how to do.

Materialism

I share Moretti's hope for a "materialist sociology of literary form"; Hell, I'd like a materialist sociology of culture generally. But I suspect it won't be able to do everything he wants it to.

When Moretti quotes D'Arcy Thompson on how the form of an object is a diagram of the forces which produced it, I'm happy to go along, and even happy to agree that this gives us some ability to work backwards, from form to force. But this sort of inverse problem generally doesn't have a unique solution, especially if some of the forces were transient and highly contingent... Less metaphorically, something Lieberson argues very convincingly is that we often have to distinguish between the social forces causing there to be a change in some taste, and those which shape the content of the new taste. Often the latter mechanisms are more or less internal to the bit of culture in question, like ratcheting. Or: culture doesn't have to express or reflect the social order. I suspect Moretti would be disappointed if this were the case for, say, genres of novels. Well, so would I. But this needs to be checked. One way would be to try to develop good neutral models, and see whether, and where, they break down

Dan Sperber has a great essay, in his Explaining Culture, on "how to be a genuine materialist in anthropology", where he complains about treating Capital, the World-System, cultural symbol-systems, mentalities, etc. as reified causal forces, if not self-interested foresightful agents, forgetting that human history, society and culture are actually "real individuals, their activity and the conditions under which they live". It seems, at least to this interested outsider, that the study of literature in society suffers from this, too. And I think what Sperber advocates there should go here, too: give actual causal accounts of how macroscopic patterns emerge from the interaction of many material bodies (notably, people and books), of the sort we know to exist, endowed with the kinds of abilities we know them to have.

This commitment may sound harmless, because contentless, but it does actually have implications. It means that you have to do a lot of work to justify functionalist explanations (though it's not impossible). It should make you very dubious about ideal types. It should make you more interested in exploring variation, and not dismissing it. It should make you very dubious about "practices" and other shared mental objects, at least as ordinarily conceived. And it suggests a lot of productive directions, investigating communication, cognition, and the collective patterns they produce.

In Graphs, Maps, Trees, as in his Atlas, Moretti is basically looking at the communication end of things. He doesn't say much about cognition, or individual thought more generally. Elsewhere (see e.g. Signs Taken for Wonders) he has dabbled in psychoanalysis, but I hope that's past. A materialist theory of literary form will ultimately have to concern itself with the organic processes of reading and composition, but the way to do this is through empirical study of readers and writers, not more interpretation of texts, or armchair ruminations (whether those are on the primal scene, the environment of evolutionary adaptation, or conceptual blending). Of course literary scholars have been making stabs in this direction at least since Richards's Practical Criticism, but with the advent of cognitive psychology this can be done in a much more systematic way, combining modeling of cognition with experimental tests of the models. [6] Again, many people (e.g., Jerry Hobbs, Herbert Simon) have been proposing this for some little while, but it's only recently, with works like Bortolussi and Dixon's Psychonarratology, that people have begun to actually do it, taking the predictions of various theories of narrative, which say that changing stories in certain ways should affect readers' responses, and seeing whether that's actually right. This, and not desk-bound speculation about analogies, seems to me the proper way to start on a cognitive psychology of literature. It is obviously complementary to what Moretti wants to do, and (this is the sweet part) the two enquiries can be pursued in parallel; neither has to wait for the other.

One thing Moretti does not do, anywhere, is construct models linking interacting individual behavior to aggregate patterns. Economists and sociologists already make such models, and anthropologists are starting to do so. It may be premature here, but ultimately it will be vital. If different social groups have different beliefs, is that because those beliefs express their relations to the mode of production, or is it because they tend to talk more with in the group than across group boundaries? Adaptationist theories of culture tend to go for the first choice, but we don't really know whether the latter could account for the specific patterns of cultural difference and change that we see.

How Not to Learn from the Natural Sciences

What I said above about not mindlessly imitating biology deserves some amplification.

Evolution ought to have a bad name in the study of literary history. Reading Rene Wellek's "The Concept of Evolution in Literary History" (or his article for the Dictionary of the History of Ideas) is actually quite depressing. (It brings to mind Kurt Vonnegut's line "they deserved to fail, because they were all so stupid".) The many post-Darwinian ventures in this direction went, essentially, nowhere, at least as far as understanding literature better goes. It surely didn't help that their understandings of biological evolution were often very bad, generally some kind of Spencerian or even Lamarckian belief in tendencies of progressive development — perhaps inspiring, but hopelessly un-explanatory. (This has vitiated far too much evolutionary theorizing about social processes; cf. Toulmin's chapter 5.) As for the more recent wave, since the 1980s, the people who seem to think that literature exists because humanity craves dramatizations of Daly and Wilson's Sex, Evolution and Behavior drive me up the wall. (Their idea makes no sense even if you are very sympathetic to evolutionary psychology, which I am.)

Which said, this is not at all what Moretti is proposing, and I don't see the harm in trying to make this all fit together as another instance of a general pattern, alongside biological evolution, because they have similar causally-relevant features, and so similar mechanisms are at work. Many people have pointed out, in some detail, that explaining biological processes through the joint action of variation and selective transmission in populations is one instance of a general pattern of historical explanation; Toulmin is particularly clear on this [7]. There is a demography of businesses, of interest groups, even of medieval manuscripts of classical works, and so why not one of literary texts? Inheriting discrete, particulate hereditary factors from a small, fixed number of immediate ancestors is not the sine qua non of this form of historical explanation, though the details of the process of inheritance will very strongly affect the character of the resulting dynamics. It might be that theories of literary change cast in this form are too complicated to be useful, or that we just don't know enough yet to find the useful ways to formulate them. But it wouldn't hurt to seriously try, and we'd learn a lot, no matter the eventual outcome.

Varieties of Rational History

One way to take the bit from Braudel about "a more rational history" that Moretti adopts as a motto is simply to hope that literary history will be a rational enterprise. There are various aspects to this — the accumulation of knowledge, a desire to give explanations, a realization that more than one explanation might be possible and a desire to check which one is right, and so on. To do all this, it's important to develop, use and refine reliable methods of inquiry — ones which are unlikely to lead you into error, and where errors are apt to be self-correcting. You want to be able to persuade others, and you want to know that you're not just persuading yourself. As a statistician, my job is to help with that bit, so it looms large for me. I think this is more or less what Moretti has in mind when he talks (elsewhere) about wanting "falsifiable" literary history — for ideas which have enough content that they can not only be communicated from one person to another (without tripping Liberman's detector), but checked. Which said, I wish that here, as in his Atlas, Moretti had done a more systematic job of checking his conclusions. Would it be unfair to suggest that, while he sees the need for data analysis, it will be left to a successor generation to put it into routine practice?

If you want to say that asking literary history to be communicable, testable and reliable is asking it to be scientific and that's icky, well, it's a free country (at least for now). The more I think about what makes something a science, the less that seems like an important question. But whether something is a rational enterprise of inquiry matters. I'm sure it's possible to object to wanting history to be more rational in this sense, but I find that thought so alien and pointless I won't even try to engage it.

Another take on "rational history" is that the vast mass of details in small-scale history are essentially random, or, more exactly, the connections among them are as convoluted and involved as the details themselves. (This is one way to define randomness, mathematically.) But looking at larger scales, the randomness averages out, leaving regularities which are simpler and more nearly comprehensible by finite minds, and more reliable. As a statistical physicist and a statistican, I am the last to disagree: "In fact, all epistemological value of the theory of probability is based on this: that large-scale random phenomena in their collective action create strict, nonrandom regularity." [8] The small-scale details of literature and of human life have an intrinsic interest and value that is missing from the small-scale detail of molecular chaos, so there is certainly all the room in the world for what Moretti would like to do and close reading, and even essayistic appreciation. (But there is not, I am afraid, room enough in the world for Harold Bloom.) Whether there is room in an academy organized around the production of peer-reviewed research findings for all of them, is fortunately not a question I need to have an opinion on.

Finally, you might be tempted to go from the last sense of "rational" to supposing that large-scale history must be the working-out of some scheme which is "rational" in that it's really deterministic, or even teleological. This would be a mistake. It is not at all hard to give examples of stochastic processes which combine random evolution and feedback, which converge on very nice large-scale regularities, but which regularity they converge on is completely random and indeterminate. [9] Brian Arthur, among others, argues that processes like this are important in the evolution of technology. Is literature like that? I have no idea. But I don't see any reason it can't be, and this needs to be borne in mind.

Go Fish

Let me close by quoting the same paragraph twice, once from the version in NLR, and then again from the closing pages of the book. In both cases, he is enumerating themes which stretch across his chapters.

First, a total indifference to the philosophizing that goes by the name of 'Theory' in literature departments. It is precisely in the name of theoretical knowledge that 'Theory' should be forgotten, and replaced with the extraordinary array of conceptual constructions, —theories, plural, and with a lower case 't'—developed by the natural and by the social scences. 'Theories are nets', wrote Novalis, 'and only he who casts will catch'. Theories are nets, and we should learn to evaluate them for the empirical data they allow us to process and understand: for how they concretely change the way we work, rather than as ends in themselves. Theories are nets; and there are so many interesting creatures that await to be caught, if only we try.

First of all, a somewhat pragmatic view of theoretical knowledge. 'Theories are nets', wrote Novalis, 'and only he who casts will catch'. Yes, theories are nets, and we should evaluate them, not as ends in themselves, but for how they concretely change the way we work: for how they allow us to enlarge the literary field, and re-design it in a better way, replacing the old, useless distinctions (high and low; canon and archive; this or that national literature...) with new temporal, spatial and morphological distinctions.

Whether this pragmatic message is what Novalis meant, I have no idea; I only know the line because Popper used it as the epigraph for The Logic of Scientific Discovery. But that's what Popper meant by it, and I think it's right, and I look forward to seeing the coelacanths and tube-worms and giant squid which will be brought up from the deeps in years to come.

[1]: More on testing the null model of genre appearance, for those into that kind of thing: Really, of course, the most suitable null model for random appearance would be a continuous-time Poisson process. Since the data are discretized by years, however, I'm faking it by using a geometric distribution of inter-arrival intervals. (I also tried simulating from a Poisson process and then discretizing the result; the results weren't much different.) The only parameter of such a process is the mean inter-arrival time, or equivalently the "intensity", the probability per year of producing a new genre. Simple maximum likelihood estimation gives this as 0.2905405, which implies a log-likelihood for the original data of -103.9498. To evaluate the significance, I generated 1,000,000 sample paths, of the same length as Moretti's, and then for each one re-estimated the intensity and used that to evaluate the log-likelihood. (This sort of "bootstrapping" should account for the fact that I fit that parameter to the data in the first place. It wouldn't be appropriate if, say, Moretti had advanced the conjecture that the mean inter-arrival time should be 10 years on independent grounds.) Of the 1,000,000 sample paths, only 3,802 had log-likelihoods as small or smaller than the original data. That is to say, if the null model were correct, we'd see results like this only about 0.38 percent of the time. So we can certainly reject the null model at the conventional 5 percent significance level, or even the 1 percent level, and in fact this is a considerably more severe test than that.

[2]: Ernst Mayr, What Evolution Is, p. 84, quoting a 1959 paper of his own.

[3]: This is from Sidney Winter's article on "Natural Selection and Evolution" in the New Palgrave Dictionary of Economics (1987), where he works out the analogy in some detail.

[4]: An Introduction to Karl Marx, pp. 183--184.

[5]: "Adventures of a Man of Science", Elif Batuman's wonderfully-titled review of Graphs, Maps, Trees in n+1 magazine, is a quite nice essay, but it also provides what looks like a typical example of the kind of mere plausibility I have in mind:

Perhaps the Holmes stories are not half-baked versions of the "correct" mystery story, but a different kind of mystery story, wherein the nondecodability of clues is not a bug, but a feature. Conan Doyle was writing during the conquest of England by industry and rationalism; perhaps his readers wanted stories about the kinds of magic that are possible within the constraints of science. Holmes categorically rejects the supernatural, not in order to show that the new, rational rules preclude magic, but in order to show that you can still have magic even if you play by the rules. Decodable clues came a "generation" later, with Agatha Christie and the first World War, and became more rigorous after the second—by which time readers wanted to be reminded that the world was still rational. [pp. 146--147]

First of all, it seems bizarre to say that Britain was being conquered by "industry and rationalism" in the 1890s, long after the scientific revolution, the Enlightenment, the Industrial Revolution and all its social consequences, utilitarianism, etc. (Indeed, Mr. Lecky might want to have a few words...) Second, Batuman gives us no reason to think that contemporary readers saw what Holmes did as (pardon the phrase) magic within the bounds of reason alone. Third, even if she were right about the social situation and the cultural product, the hypothesized causal connection is really just another arbitrary analogy, of the sort Elster complained about. Suppose Conan Doyle had been better about using decodable clues than Christie. Would it not then sound just as plausible to say this expresses the triumph of rationalism, followed by a post-war weakening? As it is, Batuman's account seems to appeal, implicitly, to a desire to hang on to older ways of thinking. Either the whole reading public of Britain in the 1890s is being treated, in a grossly anthropomorphic fashion, as a single person, with such a desire, or she is making a quite specific prediction about which readers Conan Doyle appealed to, one which does not seem especially plausible, though it might be tested. (It is utterly unclear whose purposes or needs are invokes by the in-order-to's — Conan Doyle's? his original readers'? society's? — but I fear the worst.) Finally, no attempt is made to check that this is the source of the appeal, nor that the later strict decodability of clues really was caused by the World Wars, for the reasons given. I don't know enough to say that this suggestion is false, or that checking it would be impossible. I don't even want to suggest that a book review in a little magazine would be a good place to do such tests. But it doesn't seem to worry Batuman that there is no support for this idea (yet). — Let me repeat that I like the essay.

[6]: Incidentally, thinking that cognition is computational, and even that its computational architecture is strongly constrained by organically-evolved developmental processes, in no way commits one to denying that thought is also profoundly cultural and historical. Sperber is very good on this, but also see Frawley's Vygotsky and Cognitive Science, or the papers collected in The Elements of Reason.

[7]: Of course it isn't the only pattern of successful historical explanation. Even within the natural sciences, geology and astronomy provide very different ones.

[8]: Gnedenko and Kolmogorov, Limit Distributions for Sums of Independent Random Variables, p. 1.

[9]: More exactly, there are stochastic processes ("urn schemes") where the relative frequencies of different outcomes are guaranteed to converge, with 100% probability, but the ratio at which they converge is itself a random variable, not determined by the initial set-up in any way. The models of lock-in developed by Brian Arthur and his collaborators in the 1980s are urn models, but actually less indeterministic than the classical ones.

Manual trackback: Reprieved; Crooked Timber; Pedantry; Three Quarks Daily; Idiocentrism; An Unenviable Situation ("deeply offensive").

Update, 7 February: Seth Edenbaum has more on why he dislikes this post so much &emdash; and why he dislikes me (or at least my online persona; I don't believe we've ever met). I think he's wrong, both about this and about me, but it's only right to point to criticisms.

The Commonwealth of Letters; Writing for Antiquity; Biology; Enigmas of Chance

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Self-Commitment

I seem to have locked myself into writing five pages of math textbook three times a week between now and the end of the semester. If I am to pull this off while still doing research and have something like a life, I can't spend any time posting here, or even really reading weblogs. So I'm putting this on hold until after May 5th, and deleting my RSS reader for the duration. (I will keep updating the book log in the side-bar, though.) Regular service will resume once school's out, and in the meanwhile you can get along without me for three months.

(There, that should be sufficiently emphatic to discourage backsliding.)

Self-centered

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

7x7

The last time I did one of these things, my weblog broke, resulting in a several-month hiatus. But, since you asked, and since this lets me procrastinate working on my lecture notes, my research, my house and my contribution to the Moretti event at the Valve all at once...

1. Seven things to do before I die

omitting the really important ones, which are all private: (i) Learn Persian. (ii) Learn Chinese. (iii) Travel the Silk Road from Chang'an to Constantinople. (iv) Cast bronze. (v) Write that textbook on statistical analysis of complex systems I keep talking about. (vi) Write that novel I keep not talking about. (vii) See New York.

2. Seven things I cannot do

to my regret: (i) Roll my R's. (ii) Speak anything other than English. (iii) Ignore a live television in my field of vision. (iv) Make small talk. (v) Finish things ahead of time. (vi) Understand musicians talking about music. (vii) Let things go.

3. Seven things that attract me to [Pittsburgh]

(i) The abundance of romanesque arches. (ii) Being told so often that it's a lot cleaner than it used to be, or so they've heard. (iii) Living and working in the shadow of a tower which was manifestly built to summon the Dark Gods. (iv) Wholey's, the Pennsylvania Macaroni Company, and the rest of the Strip. (v) The ability to afford a leafy yuppie-scum neighborhood. (vi) The feeling that, at last, I am among my people. (vii) The best job at the best university I could possibly hope for (until I publish more).

4. Seven things I say most often

(i) "Which, clearly, is absurd." (ii) "Take your favorite [mathematical object]". (iii) "There's a neat paper where". (iv) "Have you read [book/paper/story/post]?" (v) "This is a good idea why, exactly?" (vi) "Let me try to say that again, only this time coherently." (vii) "I'm sorry."

5. Seven books (or series) that I love

Seven? Just seven? Couldn't I pick forty-nine instead? Or three hundred and forty-three? Well, by free association then. (i) Science and Civilisation in China (Needham). (ii) The Cartoon History of the Universe (Gonick). (iii) The Pursuit of the Millennium (Cohn). (iv) The City and the Stars (Clarke). (v) Vehicles (Braitenberg). (vi) His Master's Voice (Lem). (vii) Patricia McKillip's fantasy novels. You're sure I can only pick seven?

6. Seven movies that I watch over and over again (or would if I had the time)

or would if I watched movies over and over again. (i) The Seven Samurai. (ii) Yojimbo. (iii) Sanjuro. (iv) The Hidden Fortress. (v) Apocalypse Now. (vi) Repo Man. (vii) The Usual Suspects.

7. Seven people I want to join in, too.

(i) Dave Bacon. (ii) Wolfgang Beirl. (iii) John Burke. (iv) Tim Burke. (v) Aaron Clauset. (vi) The crew at Halfway Down the Danube. (vii) Ken MacLeod.

Linkage; Self-Centered

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

"Not actually surprising to anyone who had bothered to think"

From Joan Didion's Democracy. Applications to current events are entirely your affair.

At the time I thought that she had decided to talk to me only because Jack Lovett's name was just beginning to leak out of the various investigations into arms and currency and technology dealings on the part of certain former or perhaps even current overt and covert agents of the United States government. There had even been hints about narcotics dealings, which, although they made good copy and were played large in the early coverage (I recall the phrase "Golden Triangle" in many headlines, and a photograph of two blurred figures leaving a house on Victoria Peak, one identified as a "sometimes Lovett business associate" and the other as a "known Hong Kong Triad opium lord"), remained just that, hints, rumors, that would never be substantiated, but the other allegations were solid enough, and not actually surprising to anyone who had bothered to think about what Jack Lovett was doing in that part of the world.
There had been the affiliations with interlocking transport and air courier companies devoid of real assets. There had been the directorship of the bank in Vila that put the peculiarities of condominium government to such creative use. There had been all the special assignments and the special consultancies and the special relationships in a fluid world where the collection of information was indistinguishable from the use of information and where national and private interests (the interests of state and non-state actors, Jack Lovett would have said) did not collide but merged into a single pool of exchanged favors.
In order to understand what Jack Lovett did it was necessary only to understand how natural it was for him to do it, how at once entirely absorbing and supremely easy. There had always been that talent for putting the right people together, the right man at the Department of Defense, say, with the right man at Livermore or Los Alamos or Brookhaven, or, a more specific example with a more immediately calculable payout, the Director of Base Development for CINCPAC/MACV with Dwight Christian.
There had always been something else as well.
There had been that emotional solitude, a detachment that extended to questions of national or political loyalty.
It would be inaccurate to call Jack Lovett disloyal, although I suppose some people did at the time.
It would be accurate only to say that he regarded the country on whose passport he traveled as an abstraction, a state actor, one of several to be factored into any given play.
In other words.
What Jack Lovett did was never black or white, and in the long run may even have been (since the principal gain to him was another abstraction, the pyramiding of further information) devoid of ethical content altogether, but since shades of grey tended not to reproduce in the newspapers the story was not looking good on a breaking basis. That Jack Lovett had reportedly made some elusive deals with the failed third force (or fourth force, or fifth force, this was a story on which the bottom kept dropping out) in Phnom Penh in those days after the embassy closed there did not look good. That the London dealer who was selling American arms abandoned in South Vietnam had received delivery from one of Jack Lovett's cargo services did not look good....

This is from the end of chapter 2 in part 4 --- pp. 207--209 of the Pocket Books edition (New York, 1985).

The Commonwealth of Letters; The Running-Dogs of Reaction

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

In Memoriam James Allan Klinkner

My father-in-law passed away last month. Here is his obituary.

I wasn't lucky enough to know Jim before his illness; even so, he was one of the most vital people I've ever known, eager to embrace all life's possibilities. He was a man of considerable talents in many directions, of curiosity and accomplishment. A holder of strong opinions, he respected them in others, even when he disagreed. His love for those who mattered to him was sincere and profound. If I had to bear the same burden, I hope I would have it in me to do so with as much fortitude.

That burden has, at last, been lifted from him. I miss him. I'm going to keep missing him.

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

The Evolution of Complexity

... is a fascinating area of inquiry, containing a small quantity of outstanding work, surrounded by a vast expanse of rubbish. This is your chance to help improve the ratio, if you can fit your contribution into six pages by February 28th.

Workshop on the Evolution of Complexity
June 3rd, 2006, Bloomington, IN, USA
as a part of the
Tenth International Conference on the Simulation and Synthesis of Living Systems: ALife X
The evolution of complexity is a central theme in Biology. Yet it is not without any ambiguity. Complexity has been used to refer to different things. For instance, complexification has been interpreted as a process of diversification between evolving units or as a scaling process that is related to the idea of transitions between different levels of complexity. Other meanings of complexity have been introduced, both inside and outside the realm of Biology. What concerns most researchers is to get insight into the mechanisms that produce their notion of complexity.
The focus of this workshop will be on biological interpretations of complexity and the driving mechanisms: primarily we want the focus to be on evolutionary and related dynamics as mechanisms for producing complexity. Furthermore, we want to bring together historical and novel research in this context.
Questions to be addressed at the workshop include:

What are the environmental constraints of complexity growth in living systems?
What is the origin and role of developmental mechanisms in evolution?
Are the principles of natural selection, as they are currently understood, sufficient to explain the evolution of complexity?
What are the limits at different levels to the evolution of complexity, and which conditions could reduce evolved complexity?
<Which models are | What language is> more appropriate to <understand | speak about> the evolution of complexity in living systems?
How could complexity growth be measured or operationalised in natural and artificial systems?
How can data from nature be brought to bear on the study of this issue?
What are the main hypotheses about complexity growth that can actually be tested?
Is it possible to <direct|manipulate> the evolution of complexity, or which benefits would bring its understanding?

See the full call for papers for more information about submission, publication, peer-review, etc. (I am on the program committee, which means I will be one of the reviewers.)

Biology; Complexity

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

All of Peer Review

Larry "All of Statistics/All of Nonparametric Statistics" Wasserman has just started a weblog, The Academic Curmudgeon. By my count, this brings the number of faculty blogs in the department to three (and the proportion to about 0.09); none of our students seem to have any. Larry's second post is about peer review.

Here is a summary of common reasons for rejecting papers:
Good Reasons For Rejecting a Paper:
The results are incorrect (unfixable, critical errors).
The results are not new.

Bad Reasons For Rejecting a Paper:
The referee doesn't like the paper.
The referee doesn't like the author's approach.

Tricky:
The contribution of the paper is too small.

Numbers 3 and 4 are bad because they are based on the taste of the referee which is far too subjective. Number 5 is problematic. True, we don't necessarily want top journals publishing every small idea that occurs to someone. The problem is this: almost all research, including good research, is incremental. The idea that most papers in top journals are breakthough papers is fantasy. What is too incremental to publish is highly subjective.
I think the basic problem is that most referees have the wrong view of the purpose of publishing. Ideally, publishing is about disseminating knowledge. It should not be regarded as admittance to a high and mighty priesthood.

I am going to use Larry's post as an excuse to ruminate about peer review, partly because I spent yesterday whittling down the stack of manuscripts I'd agreed, in weak moments, to review, and partly because I think Larry might be unhappy with my refereeing. Maybe he gets to see a better class of manuscript than I do (I wouldn't be surprised), but it seems to me that his list of good reasons to reject a paper is seriously incomplete.

The paper is so ill-written that it is not comprehensible with reasonable effort. (This is distinct from bad English, where I generally respond with a long list of corrections.)
The paper is comprehensible, but so vague that one cannot tell what was actually done.
One can tell what was done, but none of it amounts to a result — the experiments are inconclusive at best, the simulations wander aimlessly, the math is mere algebraic noodling.
There are results alright, but they're all old ones with a new set of labels attached to the variables. (Algorithmic information theory and the logistic map are favorite targets for this treatment.)
There are new results, but they belong in a different journal. (Physical Review keeps sending me papers that are, as far as I can tell, perfectly decent and straightforward ventures into population dynamics, and I keep telling the authors that there are journals like Theoretical Population Biology and the Journal of Theoretical Biology...)

I am happy to reject papers on such grounds, because they seem to follow from my understanding of the point of peer review. This is, as Larry says, about the dissemination of knowledge, not initiation into a priesthood. But the people peer-reviewers serve are not the authors but the potential readers. Passing peer review ought to endorse a manuscript, not as correct, but as possibly worthy of attention: not obviously wrong, not disconnected from the field, not out to lunch, not a waste of the reader's time. (This is more modest, and more achievable, than actively picking out the good stuff; it's a type I/type II error issue.) One important wrinkle here is that, if you're already an expert in a given field, the extra value of having somebody else filter the stream of manuscripts in that field is small — part of your expertise is being able to make such judgments reliably and cheaply yourself. But if you need to use results or ideas from another field, then you either need to become an expert there, too, or you need experts there to tell you what's worth attention. Very few scientists never need ideas from other fields, which is to say that most of us will benefit from peer review. (Similarly for hiring and tenure, but let's not dwell on such unpleasant subjects.) For complete non-experts, i.e., the lay public who ultimately support us all, peer review is about the only way they have of telling possibly-legitimate scientists from the cranks and the frauds. (More exactly, because peer review only says "not obviously wrong": anyone who can't get over the peer review barrier is so weak as to not be worth bothering with.)

I rather doubt, however, that the current journal/peer-review system is the ideal way of doing this filtering. Journals can be too conservative. Journals can not be conservative enough, when the topic is fashionable. (Not that I have anything in mind.) Journals can get locked into a vicious cycle in which they become so bad that publication there constitutes an anti-endorsement, so that only really bad scientists publish there, and they in turn become recruited as referees. (Still, an anti-endorsement is not without its own value.) There is something perverse about refereeing for commercial publishers, since publishers charge scientists larcenous rates to bring them the results of free labor on the part of authors and referees.

Ultimately, I hope that we move away from the current system towards something more like Paul Ginsparg's ideas. He envisions a system of "tiers" of publication; the lowest tiers, like the current arxiv.org, would have nearly-open submission and dissemination, and be most valuable to experts. Above them, operating more slowly, would be more selective tiers of peer review, commentary, review papers, etc., which will be more valuable to less expert readers, and won't try to filter the whole manuscript stream, which is what peer review now does. Getting there from here will probably involve ~~lining the war-mongering parasites at Elsevier up against a wall~~ a good deal of time and effort, but would lead to something much more efficient and intellectually valuable.

And now, back to revising our manuscript to please its referees.

Update, 29 November 2005: Dave Feldman, propelled by the burning need to procrastinate which drives so many academics to blogging, suggests that the peer review system would be much improved by free socks. He's right.

Update, 7 December 2005: For once, I wish I had comments here. (I don't feel like committing myself to the endless struggle against spam.) "Cog", of The Abstract Factory, writes in (quoted with permission):

I believe your 3 and 4 are subspecies of "the contribution is too small" (5 on Larry's list). Your 1 and 2 are both subspecies of "the paper does not communicate effectively enough for the contribution to be evaluated", which is a new reason. Number 5 is also a new reason. So you've really come up with two distinct extra reasons.
I would actually subsume Larry's reason 1, and your 1+2 under:
1'. The results are not clearly shown to be (probably) correct.
Because, of course, the burden of proof is on the authors to convince the reader that their result is not obviously wrong, not on the reviewer to show that the result is obviously wrong. To discharge this obligation requires both technical soundness and effective, precise communication.

Manual Trackback: An Ergodic Walk

The Collective Use and Evolution of Concepts; Learned Folly

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Strike-Breaking at NYU

NYU's graduate student teaching assistants are unionized, and have been for several years, but the National Labor Relations Board recently reversed itself and ruled that the university did not have to recognize the union or negotiate with it. The teaching assistants are now on strike, apparently with wide support from the faculty (so that the administration has been snooping through course websites to see who the faculty sympathizers are). To break the strike, NYU's president, John Sexton, is threatening to withhold the whole semester's stipend from any TAs who are not back to work by today, and that any TAs who strike next semester will lose their stipend for the whole year. I understand withholding wages during a strike, but this is simply vicious, and so far as I can work out would be illegal in any normal labor dispute. (Of course Sexton's position is that the TAs are not really employees, which is hogwash.) You can sign a petition against this travesty via Faculty Democracy at New York University. Leaving aside the claims of justice and elementary fairness, how many other chances will you have to agree with Andrew Ross and Alan Sokal?

(Surveying the treatment of our graduate student employees from the lofty perch of half a year on the faculty, it seems to me that CMU, at least in the statistics department, treats them pretty well, and much better than we had it at Madison when I was a TA there, and a member of AFT local #3220. But still, if they wanted to unionize, I'd be completely behind them, and I think it's idiotic and reprehensible for universities to refuse to even recognize and negotiate with graduate student unions. Unions can ask for stupid and/or selfish things, of course — which distinguishes them from any other organization how, exactly? — but the merits of particular proposals isn't the issue here; punishing people who attempt to organize to exert their rights is.)

Via John Burke (in e-mail) and Michael Bérubé; reporting and photos of the strike from Majikthise.

Update, 7 December 2005: I now see that Asad Raza, one of the strikers, has been filing dispatches (1, 2) at Three Quarks Daily.

Manual trackback: Crooked Timber

The Progressive Forces; Learned Folly

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Friday Cat Blogging ("Hear, Kitty!" Issue of Science Geek Edition)

It's been a while since I did one of these, hasn't it? But sometimes they just hop onto the keyboard and start meowing in my face.

D. K. Ryugo, E. A. Kretzmer and J. K. Niparko, "Restoration of Auditory Nerves Synapses in Cats by Cochlear Implants", Science 310 (2005): 1490--1492 [no freely available version?]: Abstract: Congenital deafness results in abnormal synaptic structure in endings of the auditory nerve. If these abnormalities persist after restoration of auditory nerve activity by a cochlear implant, the processing of time-varying signals such as speech would likely be impaired. We stimulated congenitally deaf cats for 3 months with a six-channel cochlear implant. The device used human speech-processing programs, and cats responded to environmental sounds. Auditory nerve fibers exhibited a recovery of normal synaptic structure in these cats. This rescue of synapses is attributed to a return of spike activity in the auditory nerve and may help explain cochlear implant benefits in childhood deafness.

In this paper, Ryugo et al. are mostly concerned with structures at the synapses in certain nerve fibers, rejoicing in the name of "the endbulbs of Held", which they describe in fairly flowery language: "Endbulbs have a calyxlike appearance that is formed from the main axon as several gnarled branches that arborize repeatedly to enclose the postsynaptic cell in a nest of en passant swellings and terminal boutons." These are abnormal in congenitally deaf animals: they don't branch so much, they're enlarged, they've got a flat rather than an undulating profile, and they've got fewer of the vesicles containing neurotransmitters that make synapses work. Not surprisingly, these endbulbs don't seem to transmit signals very well. This is a problem, especially since the nerve pathways where they tend to be found are the ones which encode precise timing information about sounds, important alike for for predators fond of twilight and leaping in ambush, and for chattering East African Plains Apes ("The critical nature of temporal resolution in facilitating speech recognition is underscored by studies that show speech recognition based on temporal cues while spectral content is systematically degraded").

What they did was to take congenitally deaf cats and, as kittens, give them cochlear implants which restored their capacity to hear. The physical capacity was verified by recording the propagation of neural signals; also by the fact that "we could routinely 'call' implanted cats for a food reward." After several months, they examined the development of the end-bulbs of Held in these cats, compared to matched normal animals, and to congenitally deaf cats which received no implant. (Don't ask how.) The results, photographically, are pretty convincing: the endbulbs look a lot more like those of normal cats than deaf, non-implanted cats, and quantitative comparisons of e.g. size are also fairly persuasive.

It would be interesting to know how old cats can be before simply providing the cochlear implant isn't enough for these synapses to develop properly. They speculate (but don't really show) that the same effect takes place in people, and that this is why congenitally deaf children benefit more from implants the earlier they get them. If that's so, it would just reinforce the importance of making sure all the children who need them get such implants swiftly. It would also make it nice to know what (if anything) could be done in conjunction with such implants, to help gnarl-up children's endbulbs.

Notice, by the way, that, as I've had occasion to remark before, that the whole nature-nurture division is not actually useful to understanding what's going on in processes like the development of hearing in these cats. (Go on, calculate a heritability here and tell me it means something, I dare you.) But this is generally the case with cognition.

Friday Cat Blogging; Minds, Brains, and Neurons

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

"Experimental Reasoning, Reliability, Objectivity, Rationality": The Names Men Give to Their Mistakes

Deborah Mayo's great book on Error and the Growth of Experimental Knowledge was published in 1996. In it, she lays out a way of thinking about statistics, and about learning from experience more generally, which is at once principled, powerful and useful. It helps make sense of what we already do, and suggests new ways in which we can improve our practices. I read it in 1998, and it roused me from my dogmatic slumbers about statistical inference. It's hard to imagine another path to where I am now if I hadn't read it. All of which is to say that I was very pleased to find the following in my inbox this morning:

First Symposium on
Philosophy, History, and Methodology of
E.R.R.O.R*
*Experimental Reasoning, Reliability, Objectivity & Rationality:
Induction, Statistics, Modelling
Virginia Tech, Blacksburg, VA
June 1-5, 2006

Check out the invited speakers and the call for papers.

Enigmas of Chance; Philosophy

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

In the Mountains of Madness, East of Las Vegas, New Mexico

What mysterious civilization carved these designs in the high desert — designs which can only be seen from the air?

Did these so-called primitives unravel the mysteries of electricity by themselves?

For how many aeons have these forbidding mountains born witness to eldritch rites?

What disturbing pre-human legends are engraved on these tablets — products of an advanced metallurgy far beyond any native to the region?

Which unspeakable cult conceals its world-shattering secrets and shambling, amorphous blasphemies behind the walls of its armed compound nearby?

Would you like red, green, or Christmas with your clams?

Psychoceramica; Cthulhiana

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

"Ceterum censeo, Carthago delenda est" (Saturday Classical Precedent Blogging)

Marcus Porcius Cato was famous for, among other things, ending all of his speeches in the Senate by saying "that is my opinion. It is further my opinion, that Carthage must be destroyed" (ceterum censeo, Carthago delenda est). Eventually, so the story goes, his fellow senators got so worn down that they launched the Third Punic War, which ended with Carthage destroyed, and the ground sowed with salt.

Senator McCain seems to be adopting a similar tactic, in a much higher cause than establishing hegemony over the western Mediterranean basin:

McCain Vows to Add Detainee-Abuse Provision to All Senate Bills: The U.S. Senate added language barring inhumane treatment of enemy combatants to legislation that sets military policy, the second major defense measure the chamber has amended with this provision.
The amendment sponsored by Arizona Republican Senator John McCain passed by a voice vote. It was attached to the Senate's fiscal 2006 defense spending bill Oct. 6 by a vote of 90-9. That bill is being negotiated with members of the U.S. House, including Republicans whose support is in question.
McCain said his intent is to prevent abuses such as those at Abu Ghraib prison in Iraq. He vowed today that his measure would be "on every vehicle that goes through this body" until it's enacted into law. "It's not going away," he said on the Senate floor. "This issue is incredibly harmful to the United States of America and our image throughout the world."

Via Michael Froomkin; I agree with his comment that "I still think he'd be an awful President, but this is good stuff." Unfortunately, the link Froomkin gives to the Bloomberg news story he quotes is broken, so I can't see if anyone else is drawing the classical parallel.

The Continuing Crisis

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

A Last Night in the October Country

I used to re-read Something Wicked This Way Comes every Halloween. This year I'm without my copy of both that and A Night in the Lonesome October, so I might as well blog.

The Little Professor offers some Victorian terrors. Actual Victorian-era terrors: the resurrectionists of Ann Arbor.

Barcelona, city of jack-o-lanterns and bull-rings in the air.

Fafblog: the world's only source for haunted Fafblog. (Plus, the Medium Lobster explains the Plame affair in one sentence.)

Mad science: The Annals of Improbable Research re-runs its Halloween Research Review from 2000 (1, 2)

Zombies: Kids! Did you know you can use Rocky Mountain Spotted Fever to make zombies? By Wil McCarthy, who wrote two novels I liked and many others I've not gotten around to. (Via /dev/null, who reproduces a tribute to the Little Prince.) "And I say to any flesh-eating zombies who might be listening to the Factor this evening: Bill O'Reilly is looking out for you." (For more zombies, see Destroy All Bookmarks!.)

Vampires: Carmilla is free online, along with many other good stories by J. Sheridan LeFanu. (The Oxford collection of LeFanu's stories is nice, too.) I've always thought this was a much better-written book than Dracula. It's worth noting, though, that the traditional eastern European vampire was not a pale, skinny, strangely seductive aristocrat, but a fat, red-faced peasant whose carnal designs on the living are limited to their blood, such as drips from his (fangless) mouth, and in many ways corresponded pretty well to what peasants would find if they opened up Uncle Ivan's grave after a few weeks. The whole sexual aspect of vampirism — now, apparently, its main selling point — appears to have been invented by 19th century writers in western Europe (paging Dr. Praz, paging Dr. Mario Praz, to the locked stacks please). — It would be a shame to pass up this opportunity to plug, again, Suzy McKee Charnas's The Vampire Tapestry, unquestionably the most intelligent interpretation of the modern vampire.

Less definable horrors: Probably nobody now producing horror fiction is a better writer than Peter Straub. Here's the beginning of his latest, In the Night Room.

"I can't get that monster out of my mind": If you want to know why we're fascinated by stories of being preyed upon by monsters, Barbara Ehrenreich's observations on the effects of several million years of predation on the hominid psyche is a good place to start. (Here's chapter 1.)

All too definable horrors: Ultimately, as Bradbury's readers know, Halloween is about death; and we turn to magic because grief and loss are intolerable. For a wrenching reminder of just how intolerable, read Joan Didion's Year of Magical Thinking — here is a long, moving excerpt, and here is a review by John Leonard, in the New York Review of Each Others' Books. (How good a writer is Didion? Well, while she was going crazy with grief, she was able to write like this.)

Manual Trackback: The Mystery of the Haunted Vampire.

Psychoceramica; Complexity

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Some Iron Is Best Struck Cold

Back when Wolfram's monstrous tome came out, in the summer of 2002, I wrote a long review, which I "left to the gnawing criticism of the mice", thinking I'd get back to it when the cultural moment was safely past. Since ~~I'm not getting anywhere tonight showing that the error in my filtering scheme is a non-negative supermartingale, and need to procrastinate~~ some people who saw the manuscript liked it, and Wolfram keeps on keeping on, I felt I might as well brush off the dust, make a few adjustments, and put it online. Enjoy, if you're into that kind of thing.

Manual trackback: Bill Tozier; Quantum Pontiff; Danny Yee; Bruce Sterling ("maybe reviewers shouldn't pick on isolated, wealthy math geniuses who have intensely private, highly bonkers-sounding, self-published cosmological schemes. I mean -- what if he comes out of his ivory basement and deliberately DISTURBS THE UNIVERSE? We could be looking at the pixelated rags and tatters of reality by Friday!"); Geekable; Omniorthogonal; Three Quarks Daily; Jonathan Goodwin/The Valve; Crooked Timber; Chrononautic Log; Nonplatonic; East of the Sun, West of the Moon; Paracelsus Rambles; Blog Khoa Hoc Máy Tính; Dubbings and Diversions

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Cronyism, Corruption and Incompetence: A Network Analysis

Attention Conservation Notice: a 2100-word, semi-serious proposal for politically inflammatory social-scientific research, worked out by non-social scientists. Does not adequately explain all the technical concepts involved. Contains media criticism and blogging in-jokes.
Note: This idea was developed with a close collaborator who prefers, however, not to be named. Using the first person singular just means that I'm the only one doing the actual writing here, not that I deserve the credit for whatever merits this may have. (The blame for its faults, however, is all mine.)

I haven't written anything about Hurricane Katrina and its aftermath, because nothing I could think to say seemed at all adequate in the face of the horrible human disaster and national shame. Others have said what I'd want to say better than I could have. But about the political fall-out, and the cronyism that preceded the catastrophe, and which continues to be relevant day in and day out — there I think I can suggest something.

It's by now abundantly established, at least to all sentient beings, that Michael Brown had no business running much of anything, never mind FEMA, and had the job not because people thought he was the most qualified possible person, or even a reasonably qualified person with the right political ties, but solely and exclusively because of his political ties. This naturally leads to two logically independent questions: (1) How many other people hold important government positions just because of who they know in the GOP apparatus and/or Bush family retinue? (2) Is this any worse than it was under any fairly recent previous president? It could be, after all, that the level and proportion of incompetence is no higher than it ever was, or even lower, but the Bush crew has had the bad luck of having disasters happen which exposed that incompetence; or perhaps the quality and morale of the civil service has been steadily eroded by many decades of serving under incompetent political hacks, and so can no longer adequately compensate for the folly of their masters. (Think of an unreleased episode of Yes, Minister where Sir Humphrey is merely counting the days until he can begin collecting his pension, and Bernard has split for a private-sector consulting gig.)

I've now read a couple of news stories which attempt to address these questions, and I am dissatisfied. Take, for instance, this one in the New York Times. It purports to be about this subject, but, while it reports some striking instances of cronyism and patronage, contains nothing like facts on which a reader could judge whether this problem is any worse than it's ever been, or even worse than it's been recently. It also contains the following sentence about half-way through: "People who have studied the workings of the federal government for years say this administration is no worse than President Bill Clinton's or any other recent ones in the qualifications of political appointees." This is followed by a number of quotes from such experts, none of which, carefully examined, actually say anything of the kind.

For another example, here's a story from Time, again going through the anecdotes, and quoting one of the same experts (Paul C. Light) to the opposite effect, saying that things are now much more "centralized" and politicized than before. So, from reading the reports filed by — one presumes — well-regarded journalists, not only can the concerned citizens learn absolutely nothing about whether the Bush administration is unusually incompetent and cronyist, they can't even learn what one presumptive expert (Prof. Light) thinks about the topic. Citizens might even wonder what "centralized" means here, and the news isn't going to help. The New Republic's list of the administration's fifteen worst hacks, while at once amusing, depressing and frightening, shouldn't actually convince anyone that "no administration has etched the principles of hackocracy into its governing philosophy as deeply as this one". It may be — it is! — unacceptable to treat the government of a free people this way, but it doesn't mean that this is anything new.

Let us not gnash our teeth in despair over the mainstream media, however: social network analysis can come to our rescue! What's wanted — but what the journalists don't provide — is a study where one builds the network of Presidential cronies, cronies' cronies, cronies' cronies' cronies, etc., and then asks questions such as:

How likely are close cronies to be named to government positions?
How much influence does position in the network — centrality, say, or distance from the President — have on the likelihood of getting a government job?
How likely are cronies to get jobs for which they are not qualified?
Is position more important for incompetent cronies?
How far are people with positions from the President anyway? (Presumably, when someone talks about thinks like political appointments being more centralized, what they mean is something like the average social distance of appointees from the President being smaller in this administration than previously.)
How far are they from whoever's actually at the center of the network?

Many people have asserted that networks of influence and social connection are important to how the modern GOP works — Henry Farrell reports that this is an important part of Jacob Hacker and Paul Pierson's Off Center, which I'm eager to read, and it's more or less explicit in Michael Lind's Up from Conservatism and John Judis's The Paradox of American Democracy — but nobody seems to have really studied this thoroughly. To do it right, you need to carefully define what you mean by "crony". Since, ultimately, the whole species forms a single human web, you want to only consider ties which are actually meaningful indicators of political alliance and, still more, of nepotism and cronyism. Also, you want to set out your criteria carefully and rigidly before collecting data, otherwise there'll be a lot of temptation to manipulate things as you go along, and the result will be closer to Lyndon LaRouche than to Randall Collins (or even Malcolm Gladwell). At the very least, I'd think you want to include the following kinds of ties:

"relative of";
"business partner of";
"made political donation to" (for donations large enough to be matters of public record);
"worked for campaign of";
"worked as a lobbyist for";
"hired as consultant by";
"hired as attorney by";
"served on same board as" (for advocacy groups or companies)

There are probably others, which an experienced political journalist (or political operator) could suggest. The kind of intelligence analysis described by Marc Sageman in his (excellent) Understanding Terror Networks has (I hope!) well-thought-out and operationalized criteria for deciding which kinds of relations to include. It would be good to know what they are, and use them.

Having fixed our criteria for which kind of relationships will count as links in the network, it'd then be necessary to build the network. A natural starting point would be the strategy sometimes called "snowball sampling": pick an initial target, say G. W. Bush, and identify everyone who counts as one of his cronies (by our criteria). Then go over each of his cronies, and see who their cronies are — Bush will be one of them, but presumably there will be others. Repeat this until either all the cronies are exhausted, or you're exhausted. Note that, if there really is just one network, then it doesn't matter whether you start with Bush, or Karl Rove, or Tom DeLay, or Jack Abramoff, or any of their other unindicted co-conspirators, except for the people who are so marginal to the network that you might reach them from one starting point but not another.

Once you have people in the network, we need to see whether they've been named to government positions (not necessarily confirmed, just named), and whether they met the legally-defined norms of competence for those positions. A simple scheme would be to code them 0 if they just met the qualifications, +1 if they were clearly more than qualified, and -1 if they had no discernible qualifications. This could be hard to do — some key positions don't seem to actually have any minimum qualifications at the moment — but something like this is necessary to answer questions like, "How much more likely are you to be named to a post if you're qualified than not, controlling for social position?" and "How much more likely are you to be named to a post as a function of social position, controlling for qualifications?" (For aficionados: I'm contemplating logistic regression coefficients here.) It's probably completely unrealistic to imagine having a matrix of qualifications scores for all people for all 3000-odd appointed posts, which would let us see whether favorably-situated cronies get named to posts they might be able to do, or just to any old thing, or what.

Now, to really do this right, we'd need to do it all over again, not just for the current administration, but for another one as a control — the Clinton administration, say, or Bush's father; Reagan or earlier is probably too far back. This seems to be the only way to answer questions like whether this administration is more centralized than its predecessors, or more likely to nominate incompetents. The crucial question, for us, is whether your odds of being nominated are more or less dependent on your distance from the center of the network under this administration than under previous ones.

Even without doubling our workload by doing a comparative study, however, simply seeing the network of cronies would let us answer some interesting questions. Who really are the most central members of the network? Are they people with formal positions of authority? Are they people you've ever even heard of? Or are they comparatively little-known fixers with huge address books, but no officially constituted authority? (Bruce Sterling, back during the Clinton troubles, compared our current mode of government to being ruled by some sort of literary movement, where often the most well-connected and subterraneanly influential people are not the most public figures.) Could we discern factions or communities, in the form of cohesive sub-networks? Is the president — the object of such veneration, verging on idolatry (no, that's not a joke) — actually at the head of his machine?

We could also compare the structure of the crony network to that of other well-studied networks of interest. Sageman looked at al-Qaeda, and while the comparison would be provocative, it's probably not really fair: al-Qaeda is very small, comparatively, and also very hard to study, so issues of missing data are much more serious. Perhaps more interesting would be a comparison with the network of people who sit on boards of directors of corporations, where two people are linked if they serve on the same board. This is a fairly sizeable network — some data sets contain over 7000 people — but one with very little formal structure. (Once you take into account the distribution of the number of boards people serve on, it looks almost perfectly random.) Economic sociologists have established that this network is a very important coordinating mechanism for big business, and, less adaptively for the corporations concerned, a mechanism for cronyism, patronage, and giving responsibility to incompetents. (Despite its coordinating role, the board network is not group which tells, e.g., the gas-station owners of America how much to charge for a gallon of regular unleaded, as apparently imagined by certain rabble-rousers who fear the market system because they don't understand it.) It would be interesting to see, then, whether the presidential crony network can be distinguished, in its broad, structural features, from the board interlock network, or whether they are both, in practice, acephalous.

It will not have escaped the reader's notice that I do not present anything like the kind of network I say we should find. In the immortal words of Stephen Pinker, "Good science is pedantic, expensive, and subversive", and this is certainly all three. We're talking, after all, about collecting and manually processing an immense amount of information on at least 3,000 people, and then doing it all over again on another administration. This would be a lot of work, of a kind to which I am totally unsuited; to get results in less than a year would need a team. Moreover, it is completely unfundable, unless the Ancient and Hermetic Order of the Shrill is now giving out grants to further the study of the "mendacity, malevolence, incompetence, corruption, uselessness, simple idiocy, or sheer disconnection from realty of the George W. Bush administration". Nonetheless, I would very much like somebody to do it, because it seems to me that it could actually answer some important questions about how our country now works.

Manual Trackback: In Search of 42; Green Gabbro; Crooked Timber

Networks; The Running Dogs of Reaction; Modest Proposals; The Beloved Republic

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Destroy All Bookmarks!

Recent advances in high-energy physics. (Apropos of the second link, I bought my copy of Streater and Wightman's PCT, Spin & Statistics, and All That at university library booksale, where it was shelved under "politics/current events"; I guess they missed the "T" in "PCT".)

"Pittsburgh Unprepared For Full-Scale Zombie Attack", warns The Onion (via Johnny Logic). Here in Shadyside, we can expect to be attacked by fashionably-dressed yuppie zombies (yombies?), who will serve our freshly-sliced brains in a balsamic reduction, dusted with fennel pollen and accompanied by organic heirloom tomatoes. (I rented Night of the Living Dead shortly after moving here, but found it unwatchably bad.)

The Head Heeb looks at 18th century forensics.

The Abstract Factory turns out "the only debate on Intelligent Design that is worthy of its subject".

Bill Tozier finds tongues in trees. (I had no idea that maples actually fluoresce.) Also, an astute observation on cell-phone hazards.

Why does Leon Kass say modern women are "car-owning, pill-popping, body-piercing, career-oriented, degree-granted, sexually confident, frequent-flyer, atheistic sluts" like it's a bad thing?

Tim Burke on UNESCO, Department of Bad Ideas.

Larry Bartels on "What's the Matter with What's the Matter with Kansas?" (via Phil Klinkner at PolySigh, who somehow forgot to actually give the link).

Mark Liberman elucidates an important linguistic question: when does "fuzzy" mean "smoothed piecewise linear"?

Michael Bérubé looks forward to the Miers court (as does Brad DeLong), and dares anyone to mess with his reading of Thomas Kuhn. (I like Bérubé's writing, generally, and I hope that I'll get around to posting something about The Employment of English, and how complex systems is like cultural studies. But it does bug me that he seems to care so much more about whether he got Kuhn right than about whether Kuhn got it right, since subsequent work has revealed a lot of problems with Kuhn's scheme as an accurate description of scientific change; see e.g. the papers in Scrutinizing Science.)

I have mixed feelings about stuff like this, since things like this, this and this seem pretty well institutionalized. The fact that the career military (like career academia) is socially quite isolated from the rest of the country, and tends to look down on the people it's pledged to serve, is a long-standing problem. (It's a bit more worrisome in the case of the officer corps than the professoriat.) Under the circumstances, one should be encouraging the decent, sane, capable people who are left in the service to do what they can to redeem its honor, rather than shame them into leaving. Speaking of which: Phil Carter of Intel Dump makes his first post after being deployed to Iraq.

Matthew Yglesias on the intellectuals' war, or rather case for war, and its basic folly. (Includes self-criticism.) Also from MY: why blaming declining benefits for American workers on globalization is bullshit (ObKrugman: Pop Internationalism), and why some form of American social democracy is nearly inevitable. (Brad DeLong points out that however much sense that might make, barbarism is always an alternative.) Relatedly, Nathan Newman points out that the only reason GM's workers are getting reduced health benefits rather than none is that they have a strong union; also that Harriet Miers would benefit from restoring the estate tax.

Biology; Enigmas of Chance

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Schooled by Selection

Attention conservation notice: Promotes a technical preprint by some friends. I make no attempt to explain the science, owing to lack of time.

As you know, Bob, in the 1950s J. L. Kelly established that there were intimate connections between optimal strategies for repeated gambling and Shannon's information theory. (For instance, the best achievable growth rate for the gambler's wealth is set by the entropy rate of the random sequence of gambling outcomes.) As you know also know, Bob, the mathematical theory of natural selection is closely connected to that of repeated gambling (so that, e.g., John Holland's Adaptation in Natural and Artificial Systems is in some ways an extended treatise on multi-armed bandits.) This suggests that information theory could be useful in analyzing natural selection, and it would be natural to suppose that information about the environment should manifest itself as increased fitness somehow. There's been sporadic interest in the topic (e.g., J. B. S. Haldane, with his usual prescience had an early paper in this area), but really, in my humble opinion, not enough. By way of rectification, I submit the following for your favorable consideration:

Carl T. Bergstrom and Michael Lachmann, "The fitness value of information", q-bio.PE/0510007, submitted to PNAS: Abstract: Biologists measure information in different ways. Neurobiologists and researchers in bioinformatics often measure information using information-theoretic measures such as Shannon's entropy or mutual information. Behavioral biologists and evolutionary ecologists more commonly use decision-theoretic measures, such the value of information, which assess the worth of information to a decision maker. Here we show that these two kinds of measures are intimately related in the context of biological evolution. We present a simple model of evolution in an uncertain environment, and calculate the increase in Darwinian fitness that is made possible by information about the environmental state. This fitness increase -- the fitness value of information -- is a composite of both Shannon's mutual information and the decision-theoretic value of information. Furthermore, we show that in certain cases the fitness value of responding to a cue is exactly equal to the mutual information between the cue and the environment. In general the Shannon entropy of the environment, which seemingly fails to take anything about organismal fitness into account, nonetheless imposes an upper bound on the fitness value of information.
Edo Kussell and Stanislas Leibler, "Phenotypic Diversity, Population Growth, and Information in Fluctuating Environments", Science 309 (2005): 2075--2078: Abstract: Organisms in fluctuating environments must constantly adapt their behavior to survive. In clonal populations, this may be achieved through sensing followed by response or through the generation of diversity by stochastic phenotype switching. Here we show that stochastic switching can be favored over sensing when the environment changes infrequently. The optimal switching rates then mimic the statistics of environmental changes. We derive a relation between the long-term growth rate of the organism and the information available about its fluctuating environment.

Kussell and Leibler consider Markovian environments (technically, the environmental state is a semi-Markov process), and show that the fitness penalty paid for getting the statistics of environmental changes wrong is proportional to the relative entropy (Kullback divergence) rate between the organism's switch rates and the environments. Bergstrom and Lachmann consider only independent, identically-distributed environments, but go much further in relating the fitness value of signals about the environment to traditional information-theoretic quantities, essentially considering those signals as transmission channels. (They like thinking about the value of signals.) In both cases, my feeling is that, since the Kelly gambling results carry over to general ergodic environments (see the papers of Thomas Cover, especially the ones with Paul Algoet), the evolutionary results should too. I am not, however, volunteering to perform the extensions.

I happen to know that Bergstrom and Lachmann's work is part of a more general program investigating the role of information in evolution, because I've been bugging Michael to publish his results since I heard him talk about them at the first "Science et Gastronomie" workshop two years ago. I won't say any more, for fear of spoiling their surprises, except to say that further exciting revelations are close at hand.

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

A Thought I Have No Time to Pursue

Start with your favorite large Erdos-Renyi random graph. Color all of the nodes, in such a manner that the number of nodes of a given color follows a strongly skewed distribution, perhaps a power law. (Exponential growth easily gives power-law size distributions.) Now form the aggregated graph, with one node for each color, and an edge between colors if any two disaggregated nodes of those colors are linked. Query: What is the degree distribution of the aggregated graph? (Inspired by thinking, while walking home, about attempts to model the structure of the Internet at the autonomous system level. Why I was doing that, I have no idea.)

Update, later that night: Aaron Clauset writes to point me to this paper:

M. Fayed, P. Krapivsky, J.W. Byers, M. Crovella, D. Finkel and S. Redner, "On the emergence of highly variable distributions in the autonomous system topology", ACM SIGCOMM Computer Communication Review 33 (2003): 41--49 [PDF reprint via Prof. Redner]: Abstract (omitting references): Recent studies observe that vertex degree in the autonomous systems (AS) graph exhibits a highly variable distribution. The most prominent explanatory model for this phenomenon is the Barabasi-Albert (B-A) model. A central feature of the B-A model is preferential connectivity --- meaning that the likelihood a new node in a growing graph will connect to an existing node is proportional to the existing node's degree. In this paper we ask whether a more general explanation than the B-A model, and absent the assumption of preferential connectivity, is consistent with empirical data. We are motivated by two observations: first, AS degree and AS size are highly correlated; and second, highly variable AS size can arise simply through exponential growth. We construct a model incorporating exponential growth in the size of the Internet and in the number of ASes, and show that it yields a size distribution exhibiting a power-law tail. In such a model, if an AS's link formation is roughly proportional to its size, then AS out-degree will also show high variability. Moreover, our approach is more flexible than previous work, since the choice of which AS to connect to does not impact high variability, thus can be freely specified. We instantiate such a model with empirically derived estimates of historical growth rates and show that the resulting degree distribution is in good agreement with that of real AS graphs.

This isn't exactly the model I had in mind; it's more realistic, for the Internet, than aggregating a static random graph. (I'm pleased to see that people who know what they're doing also thought to employ the idea that exponential growth leads to a power-law size distribution; presumably a re-invention, since they don't cite Reed and Hughes.) I remain a bit curious about the effects of aggregating a random network, but now will definitely not pursue it.

Update, 7 October: Aaron was too well-bred to point out his own papers on why many (in fact, almost all) networks seem to have power-law link distributions, when you probe them the wrong way. Fortunately, someone reminded me.

Update, 21 October: This looks relevant, if anyone's interested.

Enigmas of Chance; Networks

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

A New Kind of Ringtone

Wolfram Research has now released what is, without question, the most convicing demonstration yet of the power and utility of Stephen Wolfram's New Kind of Science: a cellphone ringtone generator. I will be terribly, terribly disappointed if these don't contain subliminal commands furthering a plan for world domination.

Update, 21 October: By coincidence, I've just run across this paper reviewing the history of using cellular automata to generate music, by Dave Burraston and Ernest Edmonds. (Pulbic-access copies of related papers here, under "cellular automata and music papers".)

Manual trackback: Three Quarks Daily

Psychoceramica

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Almond Madelines

Or, the role of the amygdala in the remembrance of things past:

Joel S. Winston, Jay A. Gottfried, James M. Kilner, and Raymond J. Dolan, "Integrated Neural Representations of Odor Intensity and Affective Valence in Human Amygdala", The Journal of Neuroscience 25 (2005): 8903--8907

Abstract: Arousal and valence are proposed to represent fundamental dimensions of emotion. The neural substrates for processing these aspects of stimuli are studied widely, with recent studies of chemosensory processing suggesting the amygdala processes intensity (a surrogate for arousal) rather than valence. However, these investigations have assumed that a valence effect in the amygdala is linear such that testing valence extremes is sufficient to infer responses across valence space. In this study, we tested an alternative hypothesis, namely that valence responses in the amygdala are nonlinear. Using event-related functional magnetic resonance imaging, we measured amygdala responses to high- and low-concentration variants of pleasant, neutral, and unpleasant odors. Our results demonstrate that the amygdala exhibits an intensity-by-valence interaction in olfactory processing. In other words, the effect of intensity on amygdala activity is not the same at all levels of valence. Specifically, the amygdala responds differentially to high (vs low)-intensity odor for pleasant and unpleasant smells but not for neutral smells. This implies that the amygdala codes neither intensity nor valence per se, but a combination that we suggest reflects the overall emotional value of a stimulus.

That is all.

Minds, Brains, and Neurons

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Outside of a Dog

Since it is finally sinking in that I teach my first class on Monday, posting will be light for a while. So, read a book! I've updated the book recommondations in the sidebar, and put up the archived recommendations from May, June and July.

Self-Centered

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

"The Paradox of Expertise"

When you want a smart take on how the more robust findings of cognitive psychology relate to the social organization of interdisciplinary knowledge-production, where do you turn but to the Central Intelligence Agency?

Rob Johnston, "Integrating Methodologists into Teams of Substantive Experts", Studies in Intelligence 47:1 (2003): 6 [Better-formated alternate page]: No abstract.

I'm feeling lazy, so I'll quote extensively. (All ellipses are mine.)

Intelligence analysis, like other complex tasks, demands considerable expertise. It requires individuals who can recognize patterns in large data sets, solve complex problems, and make predictions about future behavior or events. To perform these tasks successfully, analysts must dedicate a considerable number of years to researching specific topics, processes, and geographic regions. ...
The very method by which one becomes an expert explains why experts are much better at describing, explaining, performing tasks, and problem-solving within their domains than are novices, but, with a few exceptions, are worse at forecasting than actuarial tables based on historical, statistical models.
A given domain has specific heuristics for performing tasks and solving problems. These rules are a large part of what makes up expertise. In addition, experts need to acquire and store tens of thousands of cases within their domains in order to recognize patterns, generate and test hypotheses, and contribute to the collective knowledge within their fields. In other words, becoming an expert requires a significant number of years of viewing the world through the lens of one specific domain. It is the specificity that gives the expert the power to recognize patterns, perform tasks, and solve problems.
Paradoxically, it is this same specificity that is restrictive, narrowly focusing the expert's attention on one domain to the exclusion of others. It should come as little surprise, then, that an expert would have difficulty identifying and weighing variables in an interdisciplinary task such as forecasting an adversary's intentions. ...
One obvious solution to the paradox of expertise is to assemble an interdisciplinary team. Why not simply make all problem areas or country-specific data available to a team of experts from a variety of domains? ...
Ignoring potential security issues, there are practical problems with this approach. First, each expert would have to sift through large data sets to find data specific to her expertise....
Second, during the act of scanning large data sets, the expert inevitably would be looking for data that fit within her area of expertise. Imagine a chemist who comes across data that show that a country is investing in technological infrastructure, chemical supplies, and research and development.... The chemist recognizes that these are the ingredients necessary for a nation to produce a specific chemical agent, which could have a military application or could be benign. The chemist then meshes the data with an existing pattern, stores the data as a new pattern, or ignores the data as an anomaly.
The chemist, however, has no frame of reference regarding spending trends in the country of interest. The chemist does not know if this is an increase, a decrease, or a static spending pattern—answers that the economist could supply immediately. There is no reason for the chemist to know if a country's ability to produce this chemical agent is a new phenomenon. Perhaps the country in question has been producing the chemical agent for years and these data are part of some normal pattern of behavior.
One hope is that neither expert treats the data set as an anomaly, that both report it as significant. Another hope is that each expert's analysis of the data... will come together at some point. The problem is at what point? Presumably, someone will get both of these reports somewhere along the intelligence chain. Of course, the individual who gets these reports may not be able to synthesize the information. That person is subject to the same three confounding variables described earlier: processing time, pattern bias, and heuristic bias. Rather than solving the paradox of expertise, the problem has merely been shifted to someone else in the organization.
In order to avoid shifting the problem from one expert to another, an actual collaborative team could be built. Why not explicitly put the economist and the chemist together to work on analyzing data? The utilitarian problems with this strategy are obvious. Not all economic problems are chemical and not all chemical problems are economic. Each expert would waste an inordinate amount of time. Perhaps one case in one hundred would be applicable to both experts; during the rest of the day, the experts would drift back to their individual domains, in part because that is what they are best at and in part just to stay busy.
Closer to the real world, the same example may also have social, political, historical, and cultural aspects.... In order for collaboration to work, each team would have to have experts from many domains working together on the same data set.
Successful teams have very specific organizational and structural requirements.... Effective teams require cohesion, formal and informal communication, cooperation, and shared mental models, or similar knowledge structures. While cohesion, communication, and cooperation might be facilitated by specific work practices, creating shared mental models, or similar knowledge structures, is not a trivial task. Creating shared mental models may be possible with an air crew or a tank crew, where an individual's role is clearly identifiable as part of a larger team effort—like landing a plane or acquiring and firing on a target. Creating shared mental models in an intelligence team is less likely, given the vague nature of the goals, the enormity of the task, and the diversity of individual expertise. Moreover, the larger the number of team members, the more difficult it is to generate cohesion, communication, and cooperation. Heterogeneity can also be a challenge: It has a positive effect on generating diverse viewpoints within a team, but requires more organizational structure than does a homogeneous team.
Without specific processes, organizing principles, and operational structures, interdisciplinary teams will quickly revert to being just a room full of experts who ultimately drift back to their previous work patterns. That is, the experts will not be a team at all; they will be a group of experts individually working in some general problem space. ...
Intelligence analysis uses a wide variety of expertise to address a multivariate and complex world. Each expert uses his or her own heuristics to address a small portion of that world. Intelligence professionals have the perception that somehow all of that disparate analysis will come together at some point, either at the analytic team level, through the reporting hierarchy, or through some computational aggregation.
The intelligence analyst is affected by the same confounding variables that affect every other expert: processing time, pattern bias, and heuristic bias. This is the crux of the paradox of expertise. Domain experts are needed for describing, explaining, and problem solving; yet, they are not especially good at forecasting because the patterns they recognize are limited to their specific fields of study. They inevitably look at the world through the lens of their own domain's heuristics.
What is needed to overcome the paradox of expertise is a combined approach that includes formal thematic teams with structured organizational principles; technological systems designed with significant input from domain experts; and a cadre of analytic methodologists. Intelligence agencies continue to experiment with the right composition, structure, and organization of analytic teams; they budget significant resources for technological solutions; but comparatively little is being done to advance methodological science.
Advances in methodology are primarily left to the individual domains. But relying on the separate domains risks falling into the same paradoxical trap that currently exists. What is needed is an intelligence-centric approach to methodology, an approach that will include the methods and procedures of many domains and the development of heuristics and techniques unique to intelligence. In short, intelligence analysis needs its own analytic heuristics designed, developed, and tested by professional analytic methodologists. This will require using methodologists from a variety of other domains and professional associations at first, but, in time, the discipline of analytic methodology will mature into its own sub-discipline with its own measures of validity and reliability.

I have to say it's a bit obscure to me how Johnston thinks the development of intelligence-specific methods will rectify the central problem he diagnoses. (He might just mean that it can't possibly be fixed without such methodology.) That said, the whole thing's well worth reading, especially if you're interested in the earlier discussions of heuristic diversity, or interdisciplinary science. According to this, Johnston, a post-doc at the CIA's Center for the Study of Intelligence, is by training an anthropologist, and has a forthcoming book (based on his dissertation?) titled The Culture of Analytic Tradecraft: An Ethnography of the Intelligence Community, which I'd now like to read...

The archive of declassified Studies in Intelligence articles, 1955--1976, has a lot of interesting stuff in it too, though the transcription into HTML is occasionally shaky, and it's not convenient to link directly to articles.

Update, 25 August: Henry Farrell writes to point to a forthcoming paper in Studies in Intelligence, D. Calvin Andrus's "The Wiking and the Blog: Toward a Complex Adaptive Intelligence Community". I haven't had a chance to read it yet, but it might be worthwhile. And, yes, this post was missing for a few days. I could tell you what happened, but then I'd have to...

(Profuse thanks to K. for pointing out Johnston's paper and discussing it with me.)

The Collective Use and Evolution of Concepts

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Friday Cat Blogging (Roof the the World Issue of Science Geek Edition)

Snow leopards (Uncia uncia) are big cats native to the mountains of Central Asia. They are not in fact particularly closely related to leopards, but they are solitary, beautiful animals (the young are intensely cute), and, unsurprisingly, endangered. Hearteningly, their numbers are actually increasing. Somewhat dishearteningly, part of this is due to an "involuntary park" effect: a lot of their habitat lies along the borders of unfriendly states, where armies exclude people who might otherwise want to use that land for grazing. (Obviously this excludes actual areas of continuing hostilities, like Siachen Glacier, site the "war above the clouds", of one of the most extraordinary, and pointless, conflicts of modern times.) You have to be very poor to find land like this desirable, but there's no shortage of really poor people in Central Asia. However, good work is being done by the Snow Leopard Trust in community-based conservation, trying to devise ways of actually making the presence of the animals beneficial to their human neighbors. (Iowa could use some of this.) The latest effort, in conjunction with the International Finance Corporation (a part of the World Bank group) is to bring this approach to the Sary-Chat Ertash nature reserve in Kyrgyzstan, next door to a substantial gold-mining area. Since the park rangers appear to have exactly one jeep, this seems like a good thing. In the meanwhile, if you find yourself yearning for some Central Asian handicrafts (and who doesn't, from time to time?), the Snow Leopard Trust's online store seems like a beneficent way to get them. (Via the Private Sector Development Blog at the World Bank.)

Friday Cat Blogging; Afghanistan and Central Asia

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Structure and Dynamics (On the Economic Geography of Gilgamesh)

Due to circumstances which I am at a loss to explain, I find myself on the editorial board of a new, open access, peer-reviewed journal, Structure and Dynamics: eJournal of Anthropological and Related Sciences. I am really happy to report that Volume 1, Issue 1 is now live, though I believe there are still some papers which are being processed for this issue, and the next two issues of volume 1 should follow shortly. While they are all good papers (seriously!), I will, somewhat arbitrarily and unfairly, single out one of them for special mention, on the grounds of popular appeal:

Guillermo Algaze, "The Sumerian Takeoff": Economic geographers correctly note that regional variations in economic activity and population agglomeration are always the result of self-reinforcing processes of resource production, accumulation, exchange, and innovation. This article proposes that essentially similar forces account for the emergence of the world's earliest cities in the alluvial lowlands of the Tigris and Euphrates rivers (Southern Mesopotamia), sometime during the second half of the fourth millennium BC.; That emergence of early cities in the southern Mesopotamian alluvium must be understood in terms of the unique ecological conditions that existed across the region during the fourth millennium, and the enduring geographical framework of the area, which allowed for the efficient movement of commodities via water transport and facilitated interaction between diverse social units alongside natural and artificial river channels. These conditions promoted evolving long-term trade patterns that, inadvertently, differentially favored the development of polities in the southern Mesopotamian alluvium over contemporary societies in neighboring regions.; More specifically, my contention is that by the final quarter of the fourth millennium the social and economic multiplier effects of trade patterns that had been in place for centuries — if not millennia — had brought about substantial increases in population agglomeration throughout the southern alluvial lowlands. Concurrent with these increases, and partly as a result of them, important socio-economic innovations started to appear in the increasingly urbanized polities of southern Mesopotamia that were unachievable in other areas of the Ancient Near East where urban grids of comparable scale and complexity did not exist at the time. Most salient among these innovations were (1) new forms of labor organization delivering economies of scale in the production of subsistence and industrial commodities to southern societies, and (2) the creation of new forms of record keeping in southern cities that were much more capable of conveying information across time and space than the simpler reckoning systems used by contemporary polities elsewhere. These innovations furnished southern Mesopotamian polities of the fourth millennium with what turned out to be their most important competitive advantage over neighboring societies. More than any other factor, they help explain why complex regionally organized city-states emerged earlier in southern Iraq than elsewhere in the Near East, or the world.

There is much more good stuff in this issue, though, and more to come soon. If you're work falls within the aims and scope of SDEAS, I strongly encourage you to submit.

Linkage; Writing for Antiquity

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Thursday Sloth Blogging

Adorable pictures of orphaned baby sloths (and their teddy bears) in Costa Rica (via just about everybody).

(My class continues to eat up all my time. I continue to find certain of Bill's posts uncomfortable reading. What to do, what to do...)

Afghanistan and Central Asia; Modest Proposals; The Continuing Crisis

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

"The Case for Building an Afghan Auxiliary Military Force"

It was almost two years ago that I (modestly) proposed to solve our geo-political difficulties by turning the Afghan militias into US auxiliaries, and legalizing drugs. Now I see (via Intel Dump) that the first part of this idea has also occurred to Roberto Bran, who recently gave a presentation sketching out the details on the model of the Gurkhas. When I said this, it was just a sarcastic expression of despair, but Bran is trying to make a constructive policy proposal, has thought carefully about issues like language training, and, again unlike me, has some idea of what he's talking about:

I spent six months as an embedded advisor under Task Force Phoenix with a Quick Reaction force kandak (battalion) and tolei asleyah (weapons company) of the Afghan National Army (ANA). Following that assignment, the 10th Mountain Division returned to Fort Drum and I was sold to Combined Forces Command - Afghanistan (CFC-A) in Kabul, where I served as the Interagency Strategic Plans Officer and worked under some of the smartest men I have ever met. I don't pretend this qualifies me for "expertise," but at least I do have some practical experience.

I'm led to believe that this could be a good thing, if well-implemented. But I really doubt it will happen, because there is simply no domestic political constituency for it. (Thanks to Captain Bran for graciously answering some questions in e-mail.)

Posted by crshalizi at August 15, 2006 13:22 | | permanent link

Slack from the Past

Behold: The High Weirdness Project, a wiki descendant of the Rev. Mr. Ivan Stang's classic High Weirdness by Mail. As one of the people responsible, along with Mitch Porter, for producing High Weirdness by World Wide Web (back when the Late Chalcolithic of Internet time was just giving way to the present Iron Age), what can I say except "Praise 'Bob'!"? (Thanks to Modemac for the pointer.)