Posted by crshalizi at March 22, 2007 17:46 |
| permanent link
I have been too lazy busy to update the book
recommendations sidebar.
The last three
months should preceed
this post, and April's
recommendations follow it. Some TV
shows on DVD are now included, since a lot of my time for reading fiction has
gone into watching them; I suspect my taste in video is even less trustworthy
than my taste in books, but why not?
Posted by crshalizi at May 01, 2007 15:40 |
| permanent link
Archived book recommendations from the sidebar, with brief descriptions, and purchasing links to Powell's where applicable. Full-length reviews live elsewhere.
Posted by crshalizi at May 01, 2007 15:40 |
| permanent link
Posted by crshalizi at May 01, 2007 15:40 |
| permanent link
From this week's Carnegie Mellon 8 1/2 x 11 News (emphasis added):
Carnegie Mellon researchers have developed a series of robots that are simple enough for almost anyone to build with off-the-shelf parts, but are sophisticated machines that wirelessly connect to the Internet. The robots can take many forms, from a three-wheeled model with a mounted camera that people could use to monitor their home while they're away to a robotic, six-pedaled flower that can open and close based on moods. The robots can be customized and their ability to wirelessly link to the Internet allows users to control and monitor their robots' actions from any Internet-connected computer in the world.
I realize this is a typo for "six-petaled", but the mere fact that I couldn't be immediately sure has brightened my day. The vision of mobile fields of flowers, pedaling in unison to follow the sun over the hills and valleys of western Pennsylvania, may yet come to pass...
Oh yeah --- here's the link to Telepresence Robot Kit website.
Update: Thanks to Alex Mallet for pointing out my typo; I believe at least one is required, thermodynamically, when making fun of someone else's.
Posted by crshalizi at April 30, 2007 15:20 |
| permanent link
I did actually manage to read more than two books this month; but a lot of them were mediocre even by my standards, and one of them was Ethier and Kurtz's Markov Processes, plugged this time last year.
Posted by crshalizi at April 30, 2007 14:54 |
| permanent link
Posted by crshalizi at April 30, 2007 14:54 |
| permanent link
Posted by crshalizi at April 30, 2007 14:54 |
| permanent link
Posted by crshalizi at April 30, 2007 11:10 |
| permanent link
Terence Tao is posting the text of his lectures on "Structure and Randomness" (part I on number theory and Fourier analysis; part II on combinatorial number theory, graph theory, ergodic theory and "ergodic graphs"; and part III on partial differential equations). It's a fascinating glimpse into the mind of a truly accomplished pure mathematician. Most striking to me is how Tao completely avoids rounding up any of the usual suspects --- Paul Erdös and Mark Kac's work on statistical independence in number theory*, statistical mechanics, Kolmogorov complexity, etc. --- while still finding fascinating things to say about, e.g., Wick rotation. All three posts, I guess I should say, are for mathematically mature audiences only.
Tao's earlier post on Why global regularity for Navier-Stokes is hard is also very worth reading.
*: Kac's beautiful little book on Statistical Independence in Probability, Analysis and Number Theory is still in print (ISBN 978-0-88385-025-1, $21.95), but the publisher, the Mathematical Association of America, makes it impossible to actually link to its catalogue page. The best I can do is refer to the page for the series in which it is published.
Posted by crshalizi at April 30, 2007 11:10 |
| permanent link
Since the first lecture of my class coincided with the first non-trivial snow-fall of the winter, talk of the "spring" semester seems like a cruel joke, but there you go. One of my New Year's resolutions was to leave the notes as nearly alone as possible, so they will largely follow last year's, but with typo corrections, a few occasional improvements, more examples, and some pictures (not, I dare say, enough).
Update, 26 April 2007: The link at the end of this list to complete set of notes now goes to the complete notes, including chapters yet to be covered by the lectures.
This page will be updated with new lecture notes as the semester goes on. If you want an RSS feed, this should do it.
Posted by crshalizi at April 26, 2007 17:43 |
| permanent link
Today, of all days in the year, it is important to keep constantly in our minds the message and the meaning of the story of Our Lord; a message which has the most tremendous, one may even say awful, significance for the fate of every member of the human race; a message which, if only they would grasp it, would utterly transform the way they saw the world and their place in it. It is a message so powerful, so overwhelming, that most of us hide from it; we seek the false comfort of the distractions and petty views of the mundane — false comfort, for though we may not be interested in Him, He is interested in us. We refuse to put together all that we know of Him and face up to its implications; to acknowledge the utter inadequacy of all human action and merit in the face of His might and knowledge and plans. It has pleased Him to make what the world calls "wisdom" into folly, and to ensure that true wisdom shall be found in what the world calls folly and madness. What is this mind-shattering message that we would rather flee into ignorance and darkness than admit into our consciousness with all its power? Just this:
That is not dead which can eternal lie,
And with strange aeons even death may die.
Posted by crshalizi at April 08, 2007 19:31 |
| permanent link
Posted by crshalizi at April 08, 2007 19:29 |
| permanent link
Probability theory, random processes, statistical inference and machine learning
Posted by crshalizi at April 08, 2007 19:29 |
| permanent link
Posted by crshalizi at April 08, 2007 18:38 |
| permanent link
Attention conservation notice: I wrote the following in late August, trying to clear out my stuff-to-blog bookmarks folder. For reasons I don't remember, I left it aside then, and only ran across it again now. I haven't updated anything, just checked that none of the links have rotted. You've probably already seen any which you would have found interesting.
Jack Balkin has put the full text of his 1998 book Cultural Software: A Theory of Ideology on-line for free. This is a really good book, where Balkin makes a serious attempt to tackle two huge problems, namely how we manage to have shared cultural meanings, and how culture can help produce injustice. The tools he uses are the idea of memes (in a broad sense, compatible with say Sperber's critiques), along with some more experimentally-grounded psychology. I think he succeeds, but what he ends up with is 190-proof liberal evolutionary naturalism, which is mostly what I believe anyway. (He doesn't make much of the way he's recapitulating both the origins of American pragmatism in evolutionary and psychological science, e.g. this, and its outcome in a liberal social philosophy, but I can't imagine it's escaped his notice.) What's really curious, though, is that Balkin does all of this while suffering from a mild strain of the French Disease (he does teach at Yale), so that he appeals to Lyotard and Foucault in the course of defending motherhood, apple pie, and even the flag. Straightforward appeals to not let over-simplified stereotypes of group differences blind us to the reality of individual diversity therefore get prefaced by elaborate Derrida-for-beginners deconstructions of all binary oppositions. I suspect that Balkin has thus managed to write a book which will irritate almost all of its prospective readers — some with "naive scientism", and others with "postmodern bullshit" — but is nonetheless actually very good and worth reading. And, now, free. [This is the short version of the review which has been sitting, in draft form, on my hard-disk since the fall of 2000.]
Meera Nanda gives a progressive Indian perspective on American affirmative action, in the context of the debate on "reservation for backward castes". (Via Nanopolitan.) — Has anyone done a systematic comparison of the Indian caste system and the American racial system? It seems to obvious to have been left alone...
Charlie Stross contemplates the future, and sees a world whose constitution was written by Gary Gygax. It's not pretty to imagine how this will intersect with the economy of phishing.
Michael Bérubé has his head split open by Yeats. (I predicted Jonathan Goodwin's response, but not publicly, so that doesn't count.)
Elif Shafak writes about having a novel which is charged with the crime of insulting Turkishness:
The fictional Armenian characters in my latest novel, The Bastard of Istanbul, are blamed for defaming and belittling Turkishness. Thus for instance, a character named Auntie Varsenig is in trouble now for saying the following on page 57:Commenting on this idiocy, Walter Jon Williams (one of my favorite authors) takes a break from blogging a fascinating account of a trip to Turkey (now at part eleven and counting) to write "I would say something like, 'In solidarity with our literary siblings, let us all insult Turkishness together,' except that I happen to like Turkishness. It's just shithead Turkish politicians I despise.""Tell me how many Turks ever learned Armenian. None! Why did our mothers learn their language and not vice versa? Isn't it clear who has dominated whom? Only a handful of Turks come from Central Asia, right, and then the next thing you know they are everywhere! What happened to the millions of Armenians who were already there? Assimilated! Massacred! Orphaned! Deported! And then forgotten! How can you give your flesh and blood daughter to those who are responsible for our being so few and in so much pain today? Mesrop Mashtots would turn in his grave!"Similarly, another character, Dikran Stamboulian, is in dire straits now for saying the following:"What will that innocent lamb tell her friends when she grows up? My father is Barsam Tchakhmakhchian, my great-uncle is Dikran Stamboulian, his father is Varvant Istanboluian, my name is Armanoush Tchakhmakhchian, all my family tree has been Something Somethingian, and I am the grandchild of genocide survivors who lost all their relatives in the hands of Turkish butchers in 1915, but I myself have been brainwashed to deny the genocide because I was raised by some Turk named Mustapha! What kind of a joke is that ... Ah, marnim khalasim!"As much as I believe in their vivacity, my Armenian fictional characters cannot go to court to be tried under Article 301. Instead of them, my Turkish publisher, Semi Sökmen, and I, will be there when the time comes. It will be a long legal battle from then on, and certainly a hassle and cause of stress. But, we Turkish writers are not pitiful or forlorn victims unable to go out into the street for fear of nationalist assault. After all, we do know, perhaps not intellectually but intuitively, that a similar clash of opinions between the progressive-minded and the close-minded xenophobes is under way almost everywhere and the world is not a safe planet anymore.
Not-unrelated, the Editors call for the rectification of names.
The Sarong Theorem archive is "an electronic archive of images of people proving theorems while wearing sarongs."
Ilya Nemenman has put his bibliography file online, with rather uninhibited remarks on the papers concerned. Since Nemenman is very smart, this is a valuable resource for anyone interested in scientific applications of information theory, and should be emulated.
Nothing is eternal dep't: old sand in the Taklimakan Desert; old rocks in the the Sierra Nevada.
Giant ground sloths in Iowa.
Gary Farber reads about crackpot Nazi science so you don't have to! (Unless you find that sort of thing amusing, of course.) — Amygdala, by the way, is one of the most consistently interesting, and broad-ranging, weblogs I've found; Gary really does blog about almost everything three to six months before everybody else does. Since contributions really do help keep him on the air, it's a good idea to follow the links at the top of each page and contribute a little, if you can.
I have far, far too many links to arresting images and off-beat ideas from Geoff Manaugh's consistently-delightful BLDGBLOG, which is poised somplace near the triple point of photography, urbanism and architecture. Without pretending that these are the best, here the ones in my folder: Mount St. Helens of Glass; When Landscapes Sing; or, London Instrument; Where Cathedrals Go to Die; The Knot Driver; the Mine, the Rivers, the Caves and Drainscaping Nevada's Gold; The Scrap Lung; Famous Hulls of the Alaskan Sea; Silt; Optometric Metropolis; Urban Diptychs; The Hedge-Bridge; Landscapes Undone; Euclidean Agriculture; A Mars Supreme; glowing Oceans; Cities of Amorphous Carbonia; Earth Surface Machine; The Architecture of Spam; Seal Silo.
Posted by crshalizi at April 08, 2007 18:37 |
| permanent link
Today's find, via Mind Hacks, is an online archive at UCSD dedicated to the memory of the great Soviet neuropsychologist Alexander R. Luria. (Lots of the links are broken, though.)
Today Luria's probably best known for the "neurographies" he wrote, like The Mind of a Mnemonist and The Man with a Shattered World, which inspired Oliver Sacks's famous ventures in this line. But he actually made really important scientific contributions, which deserve to be remembered.
Luria began his career as a disciple of Lev Vygotsky, who had a fascinating pre-cognitive theory of how individuals acquire higher mental functions through a scaffolding provided by cultural traditions (especially language) and social interaction. Vygotskyism was an explicitly Marxist theory: it was supposed to be a scientific account of how thought arises from practice. While it is very hard to accept some of Vygotsky's more extreme statements, there is I think a core of very real insight here, about both individual development and collective cognition, and one which moreover is fundamentally compatible with sound computational views of the mind.
To support the theory, Luria led an expedition to Uzbekistan which sought to document how the Soviet introduction of modern education and collective agriculture (!) was transforming the mentality of the natives. The resulting report — translated as Cognitive Development: Its Cultural and Social Foundations — is an astonishing mixture of fascinating experiments and conjectures, and equally fascinating displays of colonialist blindness. Most of Luria's subjects were Uzbekistani peasants who'd been forced onto collective farms a few years earlier; a decade previously the whole province was the scene of the basmachi revolt, which was suppressed by the Red Army with the usual measures. It never crossed Luria's mind, so far as I can tell, that a bunch of Russian academics, asking questions which clearly indicated that the Russians thought the Uzbeks were idiots, would meet with anything less than full and sincere cooperation. Consider the following dialogue (p. 112) with an illiterate peasant named Nazir-Said:
The following syllogism is presented: There are no camels in Germany. The city of B. is in Germany. Are there camels there or not?Luria's interpretation was that Nazir-Said had difficulty with hypothetical syllogistic reasoning, as opposed to more concrete inferences in practical situations, difficulties typical of those "whose cognitive activity was formed by experience and not by systematic instruction or more complex forms of communication" (p. 115). But it's also easy to interpret this as Nazir-Said parrying the question with a perfectly valid, if enthymemic, syllogism ("Every large city has camels; B. is a large city; therefore B. has camels"), and then supporting his major premise with another valid syllogism ("Every large city has Kazakhs or Kirghiz; Kazakhs and Kirghiz always have camels; therefore every large city has camels"). The greater success of members of collective farms in "solving" the syllogisms might just reflect their greater willingness to cooperate with the Russians. In other words, there is a whole layer of issues here, involving the social relations between the scientists and their subjects, to which Luria turned a blind eye...
Subject repeats syllogism exactly.
So, are there camels in Germany?
"I don't know, I've never seen German villages."
Refusal to infer.
The syllogism is repeated.
"Probably there are camels there."
Repeat what I said.
"There are no camels in Germany, are there camels in B. or not? So probably there are. If it's a large city, there should be camels there."
Syllogism breaks down, inference drawn apart from its conditions.
But what do my words suggest?
"Probably there are. Since there are large cities, there should be camels."
Again a conclusion apart from the syllogism.
But if there aren't any in all of Germany?
"If it's a large city, there will be Kazakhs or Kirghiz there."
But I'm saying that there are no camels in Germany, and this city is in Germany.
"If this village is in a large city, there is probably no room for camels."
Even in Russian, this book wasn't published until 1974. One reason, to which Luria and his translators allude, was the political sensitivity of saying that Central Asians had a child-like mentality, even if that was being transformed by socialist labor. The other, though, on which they are conspicuously silent, was the well-known fact that a crude Pavlovian behaviorism became the Official Soviet Line in psychology. Vygotsky was in a sense lucky to die of tuberculosis then, rather than be purged, and Luria had to lie low in an institute for retarded children. (An old New York Review piece on Luria goes into some of the history.) Luria's memoirs, written in the 1970s, are, let us say, extremely tactful about this turn of events. His American discipline Michael Cole, in an epilogue to those memoirs, is rather more open these matters, and confesses to finding some of what Luria wrote when, as it were, he was compelled to speak Pavlovian "unnerving". What I find unnerving is that none of this seems to have turned him against the Soviet system.
In any case, this forced switch in research ultimately led Luria, during and after the war, to rehabilitation work with soldiers with brain injuries, and so to neuropsychology, where he made his greatest contributions. His academic works from this period, like The Working Brain, present a picture of how cognition can work through what we would now call parallel, distributed processing, in which small brain regions perform specialized processing tasks, but none of the "higher cortical functions" maps directly, as it were phrenologically, onto a particular cortical area, but rather recruits these areas in shifting configurations. In particular, this would explain how lesions in single areas can lead to deficits in multiple functions, and conversely how there are many lesions which can cause a given functional deficit.
One could draw an analogy between this view of how the brain works and Marx's idea of how communism will overcome the division of labor. (This connection was never, so far as I know, even hinted at by Luria.) In a famous passage in The German Ideology, Marx and Engels write as follows:
[A]s soon as the distribution of labour comes into being, each man has a particular, exclusive sphere of activity, which is forced upon him and from which he cannot escape. He is a hunter, a fisherman, a herdsman, or a critical critic, and must remain so if he does not want to lose his means of livelihood; while in communist society, where nobody has one exclusive sphere of activity but each can become accomplished in any branch he wishes, society regulates the general production and thus makes it possible for me to do one thing today and another tomorrow, to hunt in the morning, fish in the afternoon, rear cattle in the evening, criticise after dinner, just as I have a mind, without ever becoming hunter, fisherman, herdsman or critic.Analogously, according to Luria, a region in the frontal cortex (say) might be involved in grammatical parsing in the morning, the planning of rapid motion in the afternoon, and mental arithmetic in the evening, without ever being a parser, a planner or a calculator exclusively. I am tempted to turn this conceit into a just-so story about why Hayek and Hebb, in their accounts of distributed neural information processing, put so much less emphasis on functional flexibility, but I am afraid that someone might take me seriously. (For the record: Marx and Engels's ideas on overcoming the division of labor were profoundly utopian, and that is not a compliment.)
For what it's worth, I think Luria was really on to something here, and the fundamental point against a purely "phrenological" view of the brain is valid. There are times when I wish that no one would write a press release about a neuroimaging study without reading The Working Brain first. (The rest of the time, I wish no one would write press releases about neuroimaging at all.) But I also think it's really a matter of how much and in what manner. Part of the subtext of my own work on information in networks is to develop tools to make these questions quantitative ones, about estimation, rather than qualitative ones, about interpretation.
Which is a good note on which to do some calculations...
Minds, Brains, and Neurons; The Progressive Forces; Afghanistan and Central Asia
Posted by crshalizi at March 30, 2007 16:06 |
| permanent link
Posted by crshalizi at March 30, 2007 15:38 |
| permanent link
Posted by crshalizi at March 30, 2007 15:38 |
| permanent link
Posted by crshalizi at March 30, 2007 15:38 |
| permanent link
Well, do you? If so, it's probably the casino magnetically stimulating your right dorsolateral prefrontal cortex:
How long, I wonder, before the tinfoil hat becomes the hallmark of the professional gambler?
Posted by crshalizi at March 30, 2007 15:38 |
| permanent link
Becaue this worked pretty well last time:
Hey, kid! Got anything lined up for the summer? No? Interested in winning eternal intellectual glory and entering the glamorous world of scientific research? Interested in $1500 a month for two summer months? Are you an undergrad at Carnegie Mellon University? If so, the statistics department has no less than eleven possible projects for you. (One of them is mine, building on this paper.) Apply now!
Posted by crshalizi at March 30, 2007 12:38 |
| permanent link
Posted by crshalizi at March 29, 2007 11:51 |
| permanent link
Via John Burke in e-mail, a fantastic video of the Helsinki Complaints Choir --- i.e., a choral work, reciting complaints collected around Helsinki.
A Pittsburgh complaints choir is being organized by Jen and Ray Strobel, and turns out to be rehearsing just down the street from where I live. Details here (under "January 23, 2007"). I am a little disappointed that we will no longer be able to grumble about having less amusing conceptual public art than the Finns.
Manual trackback: Nanopolitan
Posted by crshalizi at March 29, 2007 11:51 |
| permanent link
There is an old Soviet-era joke about a nail factory which was assigned a target, under the five year plan, of 1600 tons of nails, and spent the whole five years producing a single gargantuan nail weighing (of course) 1600 tons. The joke illustrates not only the follies of "actually existing socialism", but a broader problem with using quantitative performance targets, namely that people will tend to to meet the quantitative criteria, which can be only very poorly related to the real job they are supposed to be doing. This is not to say that objective performance criteria are always bad, because often the alternative is subjective evaluations by superiors, i.e., prejudice and caprice; but it does point to the need to carefully design those criteria, so that, as far as possible, they track what you actually want to have happen, and not just what's easy to measure or to calculate.
One place where easy calculation threatens to overwhelm substantive validity is in "bibliometrics", or the use of numerical methods to study patterns of scientific publication. For many years now, scientific journals have been advertising their "impact factor", as determined by ISI/Thompson Scientific, which is roughly the number of citations (as tracked by ISI/Thompson) to that journal, divided by the number of papers published in the journal. The idea is that journals with high impact factors are ones which publish articles people take note of, and go on to cite. Now, leaving to one side the big gap between "is cited a lot" and "is good science", there are huge, glaring holes with this as a way of measuring the quality or influence of a journal. An obvious one is that a citation from the World Journal of Cartesian Snooker and Even More Obscure Problems means much less than one from Nature. But another problem, perhaps even larger, is that different fields have different patterns of citation.
A stereotypical math paper, for example, will use a huge number of previously existing results, but contain very few citations, on the presumption that most of those results are assimilated background which its readers have already absorbed from any number of standard sources. If I write a paper on stochastic processes, I might well use the ergodic theorem for Markov chains, which says (roughly) that there is a way of assigning probabilities to states which is invariant under the chain's dynamics, and moreover the amount of time any sufficiently long trajectory spends in any one state is equal to that state's probability. This is a result with a very intricate history, going back to Markov himself in his struggles with his arch-enemy, but I'd look ridiculous if I cited any of this history, or even a textbook like Grimmett and Stirzaker. On the other hand, sociologists have a reputation for providing as many citations as possible for absolutely everything, and a pious habit of referring back to the 19th and early 20th century Masters. A leading sociology journal, then (say, American Journal of Sociology) might have an impact factor of around 5, while a leading mathematics journal (say, Annals of Probability) would have one significantly lower, even though both are near the top of their respective prestige hierarchies.
Now, you could say this is just another reason why we shouldn't try to rank journals. But there are times when doing things like this is going to be very helpful, e.g. when trying to decide which journals to spend a limited subscription budget on. So it would be nice if there was a way of doing something like this, which corrected for problems like the differences in citation customs across academic tribes.
One way to imagine doing this is as follows. Pick a completely random journal, and a random article from that journal. Now pick one of its references, again completely at random, and follow it up. Repeat this process by following a random reference in that paper, until you come to a dead end, namely a citation to something outside of your data set. Pick another random starting point and repeat, many times. Looking back over your random walks through the scientific literature, how much time did you spend in any given journal? It's not hard to convince yourself that you will spend more time in journals whose papers are highly cited by papers in other journals which are themselves highly cited. If you come to a paper with many references, you are that much less likely to follow any one of them, and so you will spend less time, all else being equal, on those papers than you will in the references of papers which are more sparing of citation. Saying "influential journals are ones which are often cited by influential journals" makes the definition sound hopelessly circular, but the random walk procedure makes it clear that it's not, or at least not hopelessly so.
It turns out that the random walk scheme is computationally very demanding — you need a lot of random walkers, taking a lot of very long walks, to get good results — but there is a short cut. The random process I've described is a well-behaved Markov chain. The ergodic theorem now tells us that a time average (how often does the walk hit a given journal?) can be replaced with a "space" average (what is the probability of being at a given journal?), where the probability weights are left unchanged by the action of the Markov chain. Finding these invariant distributions is an exercise in linear algebra; specifically it's going to be the leading eigenvector of the chain's transition matrix. (One of the beauties of the theory of Markov processes is how it lets us replace nasty nonlinear problems about individual trajectories with clean linear problems about probabilities.) And there are very nice, very fast algorithms for finding eigenvectors, even of very large matrices.
Thus the reasoning behind eigenfactor.org, the latest brainstorm from Carl Bergstrom's lab — most of the actual code and elbow-grease being provided by Jevin West and Ben Althouse. It covers all the journals that impact factor would, but also gives an estimate of the impact of citations to non-journals (which lets us see that some software is more influential than some journals). Plus you get to see all kinds of useful things about how much the journals cost (something Carl's been interested in for some time), and how that breaks down by paper or by citation. All in all, it's a very fun and potentially very useful tool for anyone interested in the academic publishing system, and/or applications of Markov chains.
Disclaimer/Incestuous Amplification: Rumors that Carl arranged for me to publicize everything his lab does in this weblog in exchange for beers from his private collection whenever I'm in Seattle are — sadly exaggerated.
Manual trackback: Geomblog; Muck and Mystery; Outsider; Structure+Strangeness; Flags and Lollipops
(Thanks to Own "Vlorbik" Thomas for typo correction.)
Posted by crshalizi at March 23, 2007 10:14 |
| permanent link
Posted by crshalizi at March 22, 2007 17:46 |
| permanent link
Academic life, especially in its sillier and more exasperating aspects.
Posted by crshalizi at March 22, 2007 14:19 |
| permanent link
Posted by crshalizi at March 20, 2007 21:09 |
| permanent link
...will be held July 9--13, 2007, at the Centre de Recherches Mathématiques, Université de Montréal, organized by David Campell, Giles Hooker and Jim Ramsay. They could hardly have come up with something I'd be more into if they'd been trying (which they weren't):
The term "dynamic system" typically implies a mathematical model expressed by a system of nonlinear differential or difference equations. Models of this nature have had a very long history in the physical sciences. More recently, these models have been employed for new areas such as clinical medicine, ecology, neurophysiology and the social sciences. There is, in addition, more and more attention given to assessing how well these models fit measured data in addition to displaying characteristics of the system being modeled at a qualitative level.Statisticians have played a relatively limited role in these developments, in part because methods for fitting data with models of this nature that could spin off approaches to testing hypotheses and supplying confidence intervals for estimated quantities have not been easy to develop. Consequently, we have proposed this workshop as a means of bringing those working with dynamic models together with statisticians so as to stimulate further development, collaboration and application of statistical methodology in this important area.
Official website here (or, in French, here). Financial support is available for graduate students.
This is also a good occasion to plug some of the work of the organizers, which I've been meaning to do since Hooker came here to give a talk about a year ago:
There has been a lot of work in the physical and nonlinear dynamics communities on reconstructing the state space of smooth dynamical systems (a.k.a. "geometry from a time series"), which more or less assumes that the time series you're interested in is the solution to a set of nonlinear differential equations. (Much of my own work has been based on these ideas, as extended to certain kinds of discrete stochastic processes.) What it concentrates on are the "qualitative" properties of the system, like the geometric type of the attractor, or the Lyapunov exponents. (More exactly, "qualitative" here means "left alone by a smooth change of coordinates", or in the jargon "invariant under a diffeomorphism". This is how the numerical values of the Lyapunov exponents, i.e., quantities, get to be "qualitative".) The strength of these methods --- not needing to know the actual variables comprising the physical state, or the precise form of the dynamics --- is also their weakness; they can tell us that we're dealing with a limit cycle, but not (say) how strongly the calcium and potassium concentrations are coupled.
To answer questions of the latter sort, we need information about the form of the equations of motion and their parameters. Perhaps oddly, the nonlinear dynamics community has done less work on these questions. (Less, but not exactly none.) But this is precisely what Ramsay et al. are doing, by ingeniously using existing spline-smoothing techniques to learn the parameters of the equations of motion in a statistically reliable manner. This opens the door to testing hypotheses about those parameters (are calcium and potassium concentrations coupled? are they as strongly coupled as other experiments would suggest?, etc.), estimating errors, and all the other conveniences of statistical inference. Those of us who care about modeling dynamics should all be very interested in what these two approaches can be made to say to each other.
Posted by crshalizi at March 20, 2007 21:09 |
| permanent link
The taxonomy of the yeti, the sasquatch, and related fauna has long been a vexed question. The most common position, exemplified by the magisterial work of Sanderson (still in print after forty years!) is that they are primates and probably either apes from some otherwise-extinct genus (e.g., Gigantopithecus), or hominids. The Church of the SubGenius, of course, lays down that SubGenii are, in fact, "Tibetan yetis from Atlantis" (among other things). A more extravagant suggestion was Lovecraft's, that they were mobile, intelligent extraterrestrial fungi, based in this solar system on Yuggoth (traditionally identified by commentators with Pluto, though whether that will survive the recent demotion of that body is unclear). I am happy to report, however, that the taxonomic question has recently been solved, through the power of Science.
To quote from the Milinkovitch et al. paper:
In 1992, Peter Matthiessen and photographer Thomas Laird were the first Westerners in over three decades to visit a remote region in the northernmost Himalaya. Located close to the boarder of Tibet, Sao Kohla is a mysterious valley outside of the main city of Lo Monthang. Here Matthiessen, Laird, and their Nepalese colleagues came upon some unusual foot prints in the snow, and were informed by locals that they were the prints of the Mehti (the local name for Yeti). Near a river at the bottom of the gorge, samples of twisted hair were recovered which were clearly identified as Mehti hair by their local guides (Matthiessen, 1995, p. 75--80). We were asked to analyze these samples, but first had to agree that any identification of a "new species" would have to be reported to the government of Nepal before publication.
They sequenced mitochondrial ribosomal RNA from the samples, and constructed a phylogenetic tree, which I reproduce below:
Similarly for Coltman and Davis:
In July 2005, nine residents of Teslin, Yukon, witnessed through a kitchen window a large bipedal animal moving through the brush. The next morning, they collected a tuft of coarse, dark hair and also observed a footprint measuring 43 cm in length and 11.5 cm in width. The tuft of hair was sent to Philip Merchant, a wildlife technician of the Government of Yukon Department of Environment...and so eventually to the authors, who sequenced the DNA. This produced the following tree:
A simple application of the comparative method leads us to conclude that both in western North America and in the Himalayas curious selective pressures have resulted in the convergent evolution of two different groups of ungulates with primates.
In all seriousness, it's not completely implausible that large mammals, even primates, remain undiscovered. As Coltman and Davis note, a new of bovid was described in Vietnam in 1992, and a new species of monkey in Tanzania in 2003, so it's by no means impossible that there is an undescribed primate at large in the Himalayas, or that something rare is shambling around in the Yukon. If you want fodder for speculation, note that Gigantopithecus is known to have survived to about 100,000 years ago, and the ground sloths even more recently than that in the Americas. (Of course, given the degree of armed conflict in and around the Himalayas in recent years, I for one find it only too easy to further imagine the last yetis getting caught in the cross-fire between India and Pakistan, or the Nepalese government and the Maoists.) But, really, every culture I've ever heard of has legends about the roughly human-sized and roughly human-shaped, but not human, creatures who live nearby, and for pretty obvious reasons. I honestly don't see any why cryptozoologists should take these stories more seriously when they come from Nepal or the Yukon than when the come from the British Isles.
(Thanks to Danny Yee for alerting me to these papers.)
Manual trackback: Pathologically Polymathic; Chrononautic Log; Untyping; Greg Laden; Gene Expression; MetaFilter; Southeast Sasquatch Association.
Posted by crshalizi at March 20, 2007 20:16 |
| permanent link
Posted by crshalizi at March 10, 2007 18:29 |
| permanent link
Posted by crshalizi at March 09, 2007 16:53 |
| permanent link
Via Nanopolitan, Yoram Bauman's hilarious and accurate translation of Greg Mankiw's "ten principles of economics", from the Annals of Improbable Research. (Bauman's free downloadable principles textbook, Quantum Microeconomics, looks interesting and, despite the title, sound; and reminds me that I still need to finish my post on econophysics.)
Posted by crshalizi at March 09, 2007 16:50 |
| permanent link
Posted by crshalizi at March 09, 2007 16:50 |
| permanent link
Posted by crshalizi at March 09, 2007 16:50 |
| permanent link
Posted by crshalizi at March 09, 2007 16:50 |
| permanent link
Posted by crshalizi at March 09, 2007 16:50 |
| permanent link
Posted by crshalizi at March 09, 2007 16:50 |
| permanent link
Posted by crshalizi at March 09, 2007 16:50 |
| permanent link
Posted by crshalizi at March 09, 2007 16:50 |
| permanent link
Posted by crshalizi at March 09, 2007 16:50 |
| permanent link
Posted by crshalizi at March 09, 2007 16:50 |
| permanent link
Posted by crshalizi at March 09, 2007 16:50 |
| permanent link
Posted by crshalizi at March 09, 2007 16:50 |
| permanent link
Posted by crshalizi at March 09, 2007 16:50 |
| permanent link
Posted by crshalizi at March 09, 2007 16:50 |
| permanent link
Posted by crshalizi at March 09, 2007 16:50 |
| permanent link
Posted by crshalizi at March 09, 2007 16:50 |
| permanent link
Posted by crshalizi at March 09, 2007 16:50 |
| permanent link
Posted by crshalizi at March 09, 2007 16:50 |
| permanent link
Posted by crshalizi at March 09, 2007 16:50 |
| permanent link
Posted by crshalizi at March 09, 2007 16:50 |
| permanent link
Posted by crshalizi at March 09, 2007 16:50 |
| permanent link
Much to my loss (and, less importantly, embarrassment), I had never read this before this week. It really is as brilliant as everyone says, one of Lem's best, and bleakest, meditations on intelligence and alienness, cosmic strangeness and human pain. Most science fiction, like most fiction of any kind, is crap. Of the rest, most is mere brain-candy (which I devour eagerly, see side-bar at left). Of the rest, most is the literature of the great transformation, of humanity's passage out of pre-industrial darkness (perhaps into a different kind of darkness). This is science fiction as a literature that goes beyond the confines of our species.
I will not attempt a proper review, but I do want to draw out just one thread — I'm sure it's an old story to those who actually study Lem. The novel seems to owe something to two classic American stories of alien contact in the Antarctic, Lovecraft's At the Mountains of Madness and John W. Campbell's "Who Goes There?", though I have no idea if that's even historically possible, and Solaris is unquestionably at a far higher intellectual level. (There are a few places where the passage from Polish to English via French has reduced technical terms to gobbledygook, though I think I can guess what Lem meant.) In fact, I can't help but wonder if Solaris wasn't, in part, Lem's response to the challenge Campbell, as editor, set to his authors: "Write me a creature that thinks as well as a man, or better than a man, but not like a man". All those writers failed. (I think Lovecraft wanted to do this, but his best efforts ran a-ground in sentiments like this: "Radiates, vegetables, monstrosities, star spawn — whatever they had been, they were men!"). Lem actually succeeded here in making his readers imagine something which is so orthogonal to any sort of terrestrial mentality that even terms like "mind" or "intelligence" seem dubious, but inescapable. That he achieves this effect through, in part, an even more extreme version of the literal anthropomorphism indulged in by Campbell, that is artistry.
There is artistry, too, in the way Lem's protagonist realizes he has had a profound encounter with the utterly alien, but what matters to him is the all-too-human hope its side-effects offer of a tormented emotional redemption. "I knew nothing, and I persisted in the faith that the time of cruel miracles was not past."
Merry Christmas.
Posted by crshalizi at March 05, 2007 23:55 |
| permanent link
If you are in love, then this is obviously the most appropriate and touching image for the holiday. If you are out of love, then this is, again, the most appropriate image for the holiday:
(Via Sean Carroll at Cosmic Variance, who evidently belongs to the former set.)
Posted by crshalizi at March 05, 2007 15:34 |
| permanent link
On the one hand, this rings true.
On the other hand, I'm still going to Scott Aaronson's talk this afternoon.
Posted by crshalizi at March 05, 2007 15:34 |
| permanent link
Anand Sarwate's ergodic walk takes him to the the 2007 Information Theory and Applications workshop, from which he brings back reports on network coding, coding theory, spectrum allocation, networks, source coding and publication. (I do not understand the objections to using the arxiv.)
On a not-unrelated note, Aaron Clauset reports from two conferences I wish I'd been at, the DIMACS workshop on complex networks and their applications (days 1, 2 and 3), and the BK21 Workshop on Complex Systems.
Looking to the future, the workshop on Extending Computational Cognitive Modelling to Multi-Agent Interaction which Sule Yildirim and Bill Rand are organizing looks like it will be very interesting to anyone who cares about collective cognition, but there's no way I'll be able to make it, so I hope someone will go and post about it.
Complexity; Networks; Enigmas of Chance; The Collective Use and Evolution of Concepts
Posted by crshalizi at February 14, 2007 09:44 |
| permanent link
IFiCSaS will be an irregular series of comments on papers I've read recently, and which feel like they have some kind of connection to my aggressively ill-defined field. I borrowed the name from John C. Baez's This Week's Finds in Mathematical Physics, from which I, like many other physicists of my generation, derived a great deal of my scientific education. This isn't going to be as regular (hence "intermittent", not "this week's"), or as focused (hence "and stuff"), or even as cohesive within each installment (because that's too much work). Whether it will be worth anyone's time, I dunno.
Oh, one more thing before we begin. Speculative, anti-reductionist and essayistic writing making broad claims is an important part of the tradition of complex systems, and I'd like to ask anyone who's looking for that to leave, right now.
Dominik Janzing and Daniel J. L. Herrmann, "Reliable and Efficient Inference of Bayesian Networks from Sparse Data by Statistical Learning Theory", cs.LG/0309015; also Pawel Wocjan, Dominik Janzing and Thomas Beth, "Required sample size for learning sparse Bayesian networks with many variables", cs.LG/0204052.
Graphical probability models, a.k.a. Bayesian networks, are a way of representing the statistical dependencies among multiple variables which facilitaties all kinds of calculations, and lets one prove probabilistic results by manipulating the graphs. (See my notebook, and the books recommended therein.) Some computer scientists call them "Bayesian networks", because they've been misinformed that any application of conditional probability is "Bayesian". (If that last sentence means nothing to you, consult Prof. Mayo.) It would be very nice to be able to learn graphical models from data, in a statistically reliable fashion. The very nice work Janzing et al. have done here is to provide bounds on the VC dimension of different classes of graphs, in terms of the number of nodes and their in-degree, i.e., the number of variables and the number of connections per variable. The lead paper uses this to give a structural risk minimization algorithm, efficient in both data-size and computational time, for learning sparse graphical models.
Dean P. Foster and H. Peyton Young, "Learning, hypothesis testing, and Nash equilibrium," Games and Economic Behavior 45 (2003): 73--96; pdf
Peyton Young will be familiar to constant readers for his work on evolutionary game theory and the self-organization of social institutions. Dean Foster does very nice work on information theory, learning and statistics. This extremely cool paper shows how agents can learn themselves into Nash equilibria, or very nearly so:
Consider a finite stage game G that is repeated infinitely often. At each time, the players have hypotheses about their opponents' repeated game strategies. They frequently test their hypotheses against the opponents' recent actions. When a hypothesis fails a test, a new one is adopted. Play is almost rational in the sense that, at each point in time, the players' strategies are e-best replies to their beliefs. We show that, at least 1-e of the time t these hypothesis testing strategies constitute an e-equilibrium of the repeated game from t on; in fact the strategies are close to being subgame perfect for long stretches of time. This approach solves the problem of learning to play equilibrium with no prior knowledge (even probabilistic knowledge) of the opponents' strategies or their payoffs.
The hypotheses have the form of assuming that the other players in the game are basing their moves on some function of the last m moves, for some fixed and finite m. This isn't right, because they're doing hypothesis-test learning, too, but in equilibrium it looks right, and near-equilibrium configurations are persistent and tend to drift into equilibrium. These are proper frequentist hypothesis tests, too, which is nice, since there is some evidence from experimental psychology (not discussed in this paper) that people in sequential prediction tasks act very much as though they were doing a kind of learning-through-testing.
Beong Soo So, "Maximized log-likelihood updating and model selection", Statistics and Probability Letters 64 (2003): 293--303
This is an interesting attempt to make better statistical sense of Jorma
Rissanen's minimum
description length principle, especially in its "predictive MDL"
avatar. Rissanen defines the predictive description length, within a given
model class, as
, where
is the negative log-likelihood of the data seen up to time t, under model
, and 
is the likelihood-maximizing value of the parameter within the class, given the data up to time t-1. Think of this as starting with some guess about a model, predicting one step ahead, then revising your model in light of what actually happened, and so on. The
predictive description length is the sum of the log likelihoods you assigned at
each step to what actually happened, so it measures the cumulative length of
the encoding you'd assign at each step as you went along, with the subtlety
that the code you're using gets revised (within a fixed class) at each
step.
What So does is to decompose the predictive description length into two
terms. One is simply the summed log-likelihoods, i.e. the accumulated coding
lengths, using the parameter estimated from the total data (rather
than re-estimating at each step). The other term is a penalty, equal to the
sum of
. At each step,
this is the improvement in coding made possible by knowing
, over
and above knowing the previous values of 
. Thus we are penalizing
model classes which are hard to estimate, in the sense that they continue to
show big changes in the optimal parameter value as more data arrives. So never
spells out that intuition, but does show how to re-write his penalty term in
terms of the derivative of the log-likelihood with respect to the parameters,
and ultimately a Fisher-information-like matrix of the derivatives of the
coding lengths with respect to the parameters.
Note that So gives the wrong citation for the paper of Rissanen's on which he draws, both in the abstract and in the bibliography; the correct citation is Journal of the Royal Statistical Society B 49 (1987): 223--239.
Peter Mandik and Andy Clark, "Selective Representing and World-Making", Minds and Machines 12 (2002): 383--395 [CiteSeer has a copy of the PDF]
Ernest Gellner used to say that a large part of modern philosophy consists of the "care and feeding of Cartesian demons", the creatures which threaten us with the prospect that everything we know is wrong, or at best a dream. This paper sets out to starve one species of evolutionary demon, namely the one which says that organisms evolve to represent only those aspects of the world which are relevant to their ecological niches, therefore no organism truly represents the world. This is a variant of the obviously stupid argument which David Stove identified at the heart of (all) idealism and (most) relativism, and dubbed "the Gem". In Mandik and Clark's formulation, "the only world that we represent is a world that is represented by us", therefore "it depends on being represented by us". Mandik and Clark say that can't possibly be what their opponents mean, but I think they're wrong in saying so (though it speaks highly of their courtesy). They argue (correctly) that not only is there no incompatibility between the fact of selective representation and "the realist conception of a mind-independent world", but that "the latter provides the most powerful perspective from which to motivate and understand the differing perceptual and cognitive profiles" of different organisms. They go on to note, sounding one of Clark's recurring themes, that the thesis seems particularly inapplicable to people, given out apptitude for expanding our perceptual and cognitive systems through new technologies and social interaction.
Incidentally, Clark should really update his publication list, and Mandik should revive his blog. Moreover, I am extremely skeptical of the claim, which they take as given, that a tick's representation of the world consists of "three receptor cues" (buytric acid, pressure and temperature) and " three effector cues" (dropping, running about and borrowing). The beasts could hardly reproduce, if that was the case, and in any event some of them have eyes. (A quick google turns up this handy guide to the anatomy of ixodid ticks, for instance.) The arguments don't turn on this point, however.
Jochen Bröcker and Ulrich Parlitz, "Analyzing communication schemes using methods from nonlinear filtering",Chaos 13 (2003): 195--208 [PDF]
Here's the abstract: "We investigate a certain class of communication schemes including chaotic systems. Nonlinear filtering theory is employed to obtain a representation of the optimal receiver. Using known results on the filtering process we investigate the bit error probability. It is well known that in general there is no closed form expression of the nonlinear filter. Therefore, in practice approximations are necessary for the nonlinear filter in general and the optimal receiver in particular. We obtain bounds on the approximation error using stability properties of the filter. These bounds also apply to approximations of the optimal receiver."
The basic idea of filtering, a.k.a. state estimation, goes like this. There is some state X(t), of which we observe a noisy function Y(t). Assuming the dynamics of the hidden state is known, and we have a sequence of observations of Y, how should we estimate X? That is, what filter should we apply to the Y signal to recover the state? In the 1940s, Norbert Wiener and Andrei Kolmogorov (independently) found the solution which gives the optimal linear, time-invariant filter. In the early 1960s, Kalman and Bucy (together) found the optimal filter for systems with linear dynamics and Gaussian noise, and later that decade Stratonovich and Kushner (independently) solved the problem of the optimal nonlinear filter, which gives not just a point-estimate (like the Wiener or Kalman filters), but the whole probability distribution of the state conditional on all previous observations. Their nonlinear filter has the very nice property that it's recursive, meaning that our estimate at time t+1 is a function of our estimate at time t and the new observation we take at t+1 --- we don't need to keep around the entire previous measurement history. There's been a lot of work on this theory in probabiliy and stochastics (see e.g. this site by R. W. R Darling, who has some nice papers on the uses of differential geometry in filtering), but people in nonlinear dynamics don't attend to it much.
Bröcker and Parlitz's paper is a nice illustration of why we should: it lets us solve some tricky problems! In particular they give a very nice analysis of some of the recently-popular schemes for encoding bits into dynamical systems, which has applications in communications and especially cryptography. Despite a few lapses in English grammar, this is a very well-written and well-laid-out paper, moving from a broad overview of the problems of communication theory, and the differences between the Shannon and the Wiener approaches thereto, to nonlinear filtering and applications, and even including, as an appendix, a review of the basics of the ergodic theory of Markov processes.
Gregory L. Eyink, "A Variational Formulation of Optimal Nonlinear Estimation", physics/0011049
While on the subject of nonlinear filtering, this somewhat older paper includes a very nice introduction to the problem and the Stratonovich-Kushner solution, as well as proposing a tractable numerical approximation scheme. In addition to straight-forward applications to our problems, complex-systems wallahs should be interested in the (sound) analogies Eyink draws to non-equilibrium statistical mechanics and turbulence.
Vladimir K. Vanag and Irving R. Epstein, "Segmented spiral waves in a reaction-diffusion system", Proceedings of the National Academy of Sciences (USA) 100 (2003): 14635--14638 [link]
Abstract: "Pattern formation in reaction-diffusion systems is often invoked as a mechanism for biological morphogenesis. Patterns in chemical systems typically occur either as propagating waves or as stationary, spatially periodic, Turing structures. The spiral and concentric (target) waves found to date in spatially extended chemical or physical systems are smooth and continuous; only living systems, such as seashells, lichens, pine cones, or flowers, have been shown to demonstrate segmentation of these patterns. Here, we report observations of segmented spiral and target waves in the Belousov-Zhabotinsky reaction dispersed in water nanodroplets of a water-in-oil microemulsion. These highly ordered chemical patterns, consisting of short wave segments regularly separated by gaps, form a link between Turing and trigger wave patterns and narrow the disparity between chemistry and biology. They exhibit aspects of such fundamental biological behavior as self-replication of structural elements and preservation of morphology during evolutionary development from a simpler precursor to a more complex structure."
This is an extremely cool experimental result, and the movies published as supporting information are not to be missed.
Next time, I'll try to talk about distributed information in networks, but no promises.
Trackback: Pharyngula; Crooked Timber
Biology; Minds, Brains, and Neurons; Complexity; Philosophy; Enigmas of Chance; The Dismal Science
Posted by crshalizi at January 29, 2007 14:24 |
| permanent link
Making fun of the Modern Language Association's annual convention, and making fun of those who make fun, has all been good clean fun for many years (and at least one good mystery novel). But it's hard to see how anyone could top Margerye Kempe's account, as relayed by Geoffrey Chaucer.
Posted by crshalizi at January 16, 2007 11:27 |
| permanent link
Posted by crshalizi at January 05, 2007 10:30 |
| permanent link
Leo Kontorovich was last seen in these parts when I plugged his recent paper on concentration of measure in mixing processes. He has just started a weblog, Absolutely Regular. As he explains in his first post, his aim is to discuss topics related to his research, which means ideas in math, computer science and learning theory. The fare ranges from accessible musing on subjects like what makes math "deep", through notes for "mathematically mature" audiences (like this, contrasting measure concentration and large deviations), to technical yet fascinating questions about the learnability of formal languages. (As for the rumors that he picked up the habit of assigning unsolved questions from his own research as student problems from this class, well, "You might think that; you might very well think that; but I couldn't possibly comment.") There is far too little of any of this online, never mind all of it, in one place, with a keen mind behind it. Leo and I regard each other's politics as unsound, to put it mildly, but I am very happy to have him posting from the next building, and hope he will long continue.
Posted by crshalizi at January 02, 2007 13:26 |
| permanent link
Hosted, but not endorsed, by the Center for
the Study of Complex Systems