Steve Krause : Blog

Dan Ariely's Predictably Irrational

Dan Ariely is a Duke University professor specializing in behavioral economics. In Predictably Irrational, he explains his field like so:

According to the assumptions of standard economics, all human decisions are rational and informed, motivated by an accurate concept of the worth of all goods and services and the amount of happiness (utility) of all decisions are likely to produce....Behavioral economists, on the other hand, believe that people are susceptible to irrelevant influences from their immediate environment (which we call context effects), irrelevant emotions, shortsightedness, and other forms of irrationality.

Ariely provides many real-world examples of economic irrationality. For example, here's an anecdote about a restaurant consultant who learned that...

...high-priced entrees on the menu boost revenue for the restaurant—even if no one buys them. Why? Because even though people generally won't buy the most expensive dish on the menu, they will order the second most expensive dish. Thus, by creating an expensive dish, a restaurateur can lure customers into ordering the second most expensive choice (which can be cleverly engineered to deliver a higher profit margin).

While a book of such anecdotes would make for good reading, Ariely's academic work is about going beyond the anecdote to the experiment. Thus, much of the book covers controlled experiments (often by Ariely) designed to test people's rationality in decision-making. And to Ariely's credit, he still makes it a good read.

Among the experiments Ariely describes:

• Seeing if people who knew they were starting an auction from an arbitrary price (the last two digits of their social security number) would nevertheless bid relative to that price. Without knowing others' bids, each participant made a single "best offer" bid, which could be either up or down from his or her social-security-number starting point. "In the end, we could see that students with social security numbers ending in the upper 20 percent placed bids that were 216 to 346 percent higher than those of the students with social security numbers ending in the lowest 20 percent."
• Exploring the distorting effect of free pricing. Given a choice between a single ultrafancy chocolate for 15 cents (well below normal price) and a single Hershey's Kiss for 1 cent, 73% chose the ultrafancy chocolate. But when the price of each was reduced 1 cent—to 14 cents and "FREE!", respectively—69% chose the Kiss. Note that it was one chocolate per person, so everyone faced an either/or choice, and the relative price difference was unchanged at 14 cents. "According to standard economic theory (simple cost-benefit analysis), then, the price reduction should not lead to any change in the behavior of our customers....And yet here we were, with people pressing up to the table to grab our Hershey's Kisses, not because they made a reasoned cost-benefit analysis before elbowing their way in, but simply because the Kisses were FREE!"
• Testing whether people value items more highly after buying them. Because of high demand and limited supply, Duke ran a lottery to determine who could buy tickets to Duke basketball games. After a lottery, Ariely and another researcher called more than 100 students who either won or lost in the lottery. "In general, the students who did not own a ticket were willing to pay around $170 for one....Those who owned a ticket, on the other hand, demanded about $2,400 for it." In other words, there was initially a single group of students, "all hungry for a basketball ticket before the lottery drawing; and then, bang—in an instance after the drawing, they were divided into two groups—ticket owners and non-ticket owners. It was an emotional chasm that was formed, between those who now imagined the glory of the game, and those who imagined what else they could buy with the price of the ticket."

Having found that people can indeed be predictably irrational, Ariely makes the move from descriptive to prescriptive:

The good news is that these [irrationalities] also provide opportunities for improvement. If we all make systematic mistakes in our decisions, then why not develop new strategies, tools, and methods to help us make better decisions and improve our overall well-being?

Among the new strategies he mentions is Save More Tomorrow, a program designed by behavioral-economics proponents Richard Thaler (University of Chicago) and Shlomo Benartzi (UCLA):

When new employees join a company, in addition to the regular decisions they are asked to make about what percentage of their paycheck to invest in their company's retirement plan, they are also asked what percentage of their future salary raises they would be willing to invest in their retirement plan. It is difficult to sacrifice consumption today for saving in the distant future, but it is psychologically easier to sacrifice consumption in the future, and even easier to give up a percentage of a salary increase that one does not yet have.

Save More Tomorrow is a poster child for behavioral economics because it famously succeeded in a real-world test at an actual company, nearly quadrupling savings rates. Aiming for a similar real-world win, Ariely tried to sell the credit-card industry on the concept of a "self-control credit card," where the consumer could self-restrict spending by category. There were no takers, but he is still holding out hope.

I've gone on at length about Predictably Irrational because I thought it was well worth my, and probably your, time. It's a great combination of data-driven insight, real-world application, and well-told stories—by one of the principles in the field, no less.

Here's the link to the book at Amazon, which has an excerpt and some author videos. See also Ariely's Predictably Irrational site.

Joshua Ferris' "Then We Came to the End"

The novel Then We Came to the End by Joshua Ferris starts like this:

We were fractious and overpaid. Our mornings lacked promise. At least those of us who smoked had something to look forward to at ten-fifteen. Most of us liked most everyone, a few of us hated specific individuals, one or two people loved everyone and everything. Those who loved everyone were unanimously reviled.

It's snappy, snarky, and smart throughout.

The book's narrator is a collective "we" who hope and mope and cope together amid waves of layoffs. Ferris' adroit use of this narrative device is half the attraction of the book.

Word spread fast that Tom had been laid off and naturally Benny was the guy to go down there. He said Tom was pacing in his office like a man recently jailed. He said he could picture what Tom had looked like the night he went to the Naperville house with the aluminum bat and the authorities had to restrain him. We had never heard that story before. Right there and then we had to stop Benny from telling us the story of Tom's final hour so he could first tell us the story of the aluminum bat. Benny was shocked we had never heard that story; he was sure we had. No, we never had. "Get out of here," he said. "You've heard that story." No, we hadn't. This is always how conversations went.

Ferris obligingly provides the standard repertoire of office hijinks, like the furtive scramble for a departed co-worker's chair. In this respect, you get Dilbert by Tom Wolfe.

However, instead of fun and games, most of the time it's futility and games—amusement while waiting for the ax to fall. This dark undercurrent often cuts to the surface as characters face their shortcomings, their mortality, and various other challenges beyond losing their jobs. If you know the author Don DeLillo, whom Ferris references in the acknowledgments, think of DeLillo interpreting the TV show The Office.

Chapter 1 is online. It's indicative of what you're in for, so try before you buy.

Lane and Oreskes' "The Genius of America"

In The Genius of America, Eric Lane and Michael Oreskes write:

The message we are hearing is that our government does not work. The message we should be hearing is that our government is a reflection of our own divisions. What we need is not a new system of government. We need a renewed willingness to work out our differences and find compromises, consensus and that other now-popular phrase, common ground.

While that last sentence might sound like a naive call to higher principle, the authors argue that finding compromise amid conflicting interests is what the United States has done superbly well for more than 200 years—but not so well lately. They believe that political developments of the past few decades are starting to undermine the U.S. Constitutional system, which manages conflicts by a process of checks, balances, and compromise.

For example, Lane and Oreskes critique the rise of voter initiatives, such as California's Proposition 13:

It won in a landslide among those who voted. But even so, fewer than 50 percent of California's registered voters cast a yes vote for the proposition. Thus a minority of registered Californians decided to reduce tax burdens and and limit the capacity of the government to increase its future revenues.

It was appealingly easy to adopt this initiative. There was none of the scrutiny of normal legislative process nor the need for coalition building and compromise. No committee of one legislative house and then of the second could block its path to a vote. No requirement that a bill pass two separate legislative houses stood in the way. No executive stood waiving a veto pen. No colleagues were hovering around demanding compromise for support. No lobbyists were demanding changes. No time-consuming hearings from which concerns of the public or experts had to be addressed and weighed. No long public debates in which legislators had to explain why they favored or disfavored the issue. No arcane procedural rules blocked a vote either in a committee or legislative house. No worries that a wrong step might anger constituents. No competition among this idea and the thousands of other ideas wanting legislative attention. In short, the initiative did not have to pass through any of the screens that in the legislature protect against the tyranny of the majority or even the minority and require deliberation and consensus.

Although voter initiatives are a state-level phenomenon, the authors note the existence of a proposed National Initiative, associated with the long-shot presidential campaign of former Senator Mike Gravel. The authors are not forecasting doom at the hands of Mike Gravel, but they hold him as a symbol of what's going wrong: a former Senator trying to make government better by short-circuiting it.

Another interesting example from relatively recent history:

In the mid-1980s, politicians were worried about the federal deficit and about the danger that voters would punish them for it. But there was no consensus on the steps to take to curb it: raise taxes, cut defense spending, cut social spending. So they adopted Gramm-Rudman, what [political author John] Ehrman calls "one of the most disgraceful and irresponsible laws ever passed." The law gave to a non-elected official the power to simply cut the budget if the deficits didn't shrink. This [cutting] was enacted without any legislative process. No hearings. No committee debate or vote. "Congress and the White House abandoned their political responsibilities for making fiscal decisions, and rushed instead to hand power to automatic, technical mechanisms," wrote Ehrman. "This is hardly how republican institutions are intended to function."

But during virtually the same period, Congress also enacted a tax reform measure that made substantial strides toward a simpler and fairer tax code. The measure demonstrated that "when politicians acted seriously, the political system was able to deal with complex issues quite well." But this accomplishment was not the product of public virtue. "None of those in the process rose above party or personal interests." The parties advocated their positions as hard as they could and then, recognizing that a winner-take-all attitude would fail, compromised. Tax reform was the system working just as the framers had designed it to. "In contrast, Gramm-Rudman was the product of panic and a desire to circumvent politics."

The mention of "public virtue" touches a key theme in the book, that the genius of the Constitution is how it channels individuals' and groups' private self-interests into public virtue. The book's first part recounts how the framers got there, as they learned from the flaws of the Articles of Confederation, the Constitution's predecessor.

The Articles assumed that each state would itself act with public virtue, doing what was best for the national interest. But there was no enforcement mechanism, so individual states often pursued their own interests, conflicting with each other and the national interest. This system rarely and barely worked, to the point that during the Revolutionary War, the bickering states almost starved George Washington's Continental Army.

As a result, the framers designed the Constitution to create public virtue without requiring it as an input. It would result from competition among self-interested groups and ideas. But instead of "winner take all," the system would force deliberation and compromise among competing factions—thereby producing a best-possible consensus given the conflicting aims.

The authors demonstrate that the Constitution itself was the result of such a competitive, give-and-take process. This process did not produce the exact result that any side wanted at the time. But more than 200 years later, we can say that it obviously worked well.

Which brings us back to where we started. The authors are concerned that the American people and their representatives are losing touch with why our system has worked so well. The people see government gridlock and assume the system is broken. But what's broken is the way the participants are playing—or not playing—their institutional roles.

In their most recent examples, the authors blame the second Bush administration for its ambitious moves to expand executive power without congressional—and in some cases judicial—oversight. In addition, from 2001 to 2006, the Republican congressional leadership "operated as the president's floor leaders in the Congress, rather than his separate and coequal partners in government."

These moves led to a "winner take all" environment, which is counterproductive in two ways. If the opposition has the votes, there's gridlock; if the majority can push legislation through without compromise (as the authors believe was the case with the second Bush administration's Iraq policy), then checks and balances have been defeated.

To be clear, the authors do not expect politicians to ignore their party in favor of their institutional roles. But the authors make the point that the reverse has not—and should not—be the case either:

Senator Harry S. Truman investigated President Roosevelt's administration, and Senator Lyndon B. Johnson investigated President Truman's administration. It was a Republican Senator, Howard Baker, whose incessant questions crystallized the belief that a Republican president knew more about Watergate than he had told. And it was a Democratic Senator, Daniel Patrick Moynihan, who blocked the Democratic president and his wife from their plan to overhaul American health care.

So we find the framers did not entirely succeed in removing the need for public virtue as an input to the political process. A willingness to play one's institutional role—especially at critical times, when that role and one's other interests may conflict—is necessary for the Constitution's system to work. The authors call this our Constitutional Conscience, and they want American politics to get more Constitutionally Conscientious.

For those with further interest, here is a link to the book. The first part, which I glossed over, is worth the price of admission alone. It recounts how the Constitution emerged from a historic combination of big-thinking, practical problem-solving, and shrewd politicking. The book then examines how the Constitution adapted to the massive changes and challenges that came with America's growth. Finally, it considers the recent challenges highlighted above, which the authors feel are novel and serious.

Mortenson and Relin's "Three Cups of Tea"

Don't take my word for it. On Amazon.com, Three Cups of Tea has 482 user reviews, 91% of which are five out of five stars. It's one of the few books I'd recommend to anyone, no caveats.

Three Cups of Tea is the true story of an unlikely saint, an American who against all odds builds schools in Pakistan.

Here is a good summary from Publishers Weekly:

Some failures lead to phenomenal successes, and this American nurse's unsuccessful attempt to climb K2, the world's second tallest mountain, is one of them. Dangerously ill when he finished his climb in 1993, [Greg] Mortenson was sheltered for seven weeks by the small Pakistani village of Korphe; in return, he promised to build the impoverished town's first school, a project that grew into the Central Asia Institute, which has since constructed more than 50 schools across rural Pakistan and Afghanistan. Coauthor [David Oliver] Relin recounts Mortenson's efforts in fascinating detail, presenting compelling portraits of the village elders, con artists, philanthropists, mujahideen, Taliban officials, ambitious school girls and upright Muslims Mortenson met along the way. As the book moves into the post-9/11 world, Mortenson and Relin argue that the United States must fight Islamic extremism in the region through collaborative efforts to alleviate poverty and improve access to education, especially for girls. Captivating and suspenseful, with engrossing accounts of both hostilities and unlikely friendships, this book will win many readers' hearts.

Just to clarify the extremes of the human spirit, and human condition, that are involved here, a few notes:

• At the time of his K2 near-death experience, Mortenson was the mountain-climbing equivalent of a ski bum, intermittently working as an emergency room nurse to fund his next climb. When he returned to Berkeley, California, to fulfill his promise to build the Korphe school, he had no money, no contacts, and no idea what to do. To save money while raising funds, he lived in the back of a Buick.
  
• Korphe is a place where central heating is a yak-dung fire in the middle of a room made from rock and mud. When Mortenson got there, it not only had no school, the nearest doctor was a week's walk away. One out of three Korphe children died before reaching their first birthday. The basic medicines in Mortenson's first-aid kit and his training as a nurse were like godsends—one of many examples where only a little technology and know-how could alleviate much suffering. Later, Mortenson was involved in a simple clean-water project that halved the infant-mortality rate of a 2,000-person community.
  
• Even for the healthy, life in Korphe was a constant struggle amid few resources. For example, during a rare celebration in which a ram was slaughtered, "forty people tore every scrape of roasted meat from the skinny animal's bones, then cracked open the bones themselves with rocks, stripping the marrow with their teeth."
• "Traveling with a party of [Korphe] men hunting to eat, rather than Westerners aiming for summits with more complicated motives, Mortenson saw this wilderness of ice with new eyes. It was no wonder the great peaks of the Himalaya had remained unconquered until the mid-twentieth century. For millenia, the people who lived closest to the mountains never considered attempting such a thing. Scratching out enough food and warmth to survive on the roof of the world took all of one's energy." (An interesting counterpoint to the famous phrase, "Why climb the mountain? Because it is there.")
• About his focus on girls' education: "'Once you educate the boys, they tend to leave the villages and go search for work in the cities,' Mortenson explains. 'But the girls stay home, become leaders in the community, and pass on what they've learned. If you really want to change the culture, to empower women, to improve basic hygiene and health care, and fight high rates of infant mortality, the answer is to educate girls.'"
• Mortenson on the War on Terror: "If we try to resolve terrorism with military might and nothing else, then we will be no safer than we were before 9/11. If we truly want peace for our children, we need to understand that this is a war that will ultimately be won by books not bombs."

If you wish to buy Three Cups of Tea online, take an extra click and go to the Three Cups of Tea site, where you'll see a link to buy the book at Amazon.com. Clicking the link and then buying will get Mortenson's organization, the Central Asia Institute, up to 7% of the sale.

Review: Shaffer's "How Computer Games Help Children Learn"

Part critique, part proposal, David Williamson Shaffer's "How Computer Games Help Children Learn" is about what kids need to learn and how they need to learn it.

Shaffer asserts that the common practice of school is already a kind of game, with rules rooted in the industrial revolution:

School is a game about thinking like a factory worker. It is a game with an epistemology of right and wrong answers in which Students are supposed to follow instructions, whether they make sense in the moment or not. Truth is whatever the teacher says is the right answer, and actions are justified based on appeal to authority. School is a game in which what it means to know something is to be able to answer specific kinds of questions on specific kinds of tests.

Back when a high-school degree led to a good manufacturing job, this version of school may have made sense. However, U.S. manufacturing jobs are somewhere between going and gone. That leaves the low-end service sector as the primary employer of the high-school-educated workforce. Thus, yesterday's middle-class auto worker is today's barely-getting-by burger flipper.

In Shaffer's view, the only good jobs will increasingly be those beyond the traditional high-school education: jobs that address problems with many possible answers, that require creativity, and that reward innovation. Everything else will be automated, offshored, or marginalized to the low-wage economy.

So if society wants to leave no child behind, the education system needs to change its game. The key shift is from an emphasis on teaching facts to teaching skills. Put another way, Shaffer wants more learning by doing. He wants students to learn by making them participants in simulations of real-world challenges that engineers, urban planners, journalists, and other professionals face. And this is where computers come in.

Think about SimCity, the computer game where you manage a simulated city as it grows. It's a simulation. It's a computer game. It's a learning experience:

[Players] see what happens when they make changes in urban ecosystems. For example, if you put more parks in a city, the cost of public utilities goes up because you have to keep the parks clean. If you put an industrial site next to residential housing, the residential land values fall and the crime rates rise. As a result, players must decide whether to raise taxes, decrease the green space, move the industry, or risk urban flight—or, more realistically, decide which combination of these choices and in what measure will lead to the best long-term outcomes for the city.

Immersing kids in such a world lets them learn key concepts and ways of thinking as they progress through the game. Succeeding in the game requires learning and understanding. Contrast that with reading a bunch of articles about urban planning and being tested on the facts. Will those facts ever mean anything to the students? How much more would a student want to read about urban planning after getting hooked on SimCity?

But Shaffer's learning games (developed by him and his colleagues at the University of Wisconsin-Madison) are different than SimCity. Whereas SimCity is an imaginary place, Shaffer's games take care to simulate real places, things, processes, and constraints from the real world. The computer simulation provides an extension of reality rather than a replacement.

For example, an urban-planning game he highlights...

...is modeled on the real world of the city players live in and the real work of planners who shape that city. Players are redesigning a city, but it is their city. They can see and touch the places they are redesigning and can see how those changes might make their lives and the lives of those around them richer and more satisfying. However, their choices are constrained by the economic, social, and physical realities of life in a city and by the norms and practices of the profession of urban planning.

Given this world, students learn key skills necessary to do urban planning in the real world. These include ways of thinking and talking about problems that apply well beyond urban planning—which is the larger point: The underlying creativity, critical thinking, and innovation are what these students will need to compete in the global marketplace, whether they become urban planners or anything else that pays more than subsistence wages.

Of course, this all sounds reasonable, but the games need to be good. SimCity was successful because of fantastic game design and execution, not because millions of consumers inherently craved a city simulator. So can a bunch of academics make something that both satisfies Shaffer's educational vision and is compelling enough to keep kids at the screen?

Shaffer and colleagues' games are at early stages and are themselves academic projects. Initial pilot results are promising, but the results are from tests with close participation from mentors and facilitators. While the human element is part of Shaffer's design, how much mentoring and facilitating is necessary for success? Is that amount practical for large-scale use?

Finally, how do Shaffer's games get integrated into schools? Here Shaffer has a surprising answer: Maybe they don't, at least initially. Several of the pilot tests have been in after-school programs. He speculates that they could also be embedded in larger virtual worlds such as Second Life. Although some might see this as a cop-out, avoiding the main institution that Shaffer critiques, I see it as a practical path that brings change where it's easiest to make change. If the change is good, it will spread.

As is probably apparent, I like Shaffer's ideas. But if you want a good read along with good ideas, this book might not meet your standards. Although the prose is relatively direct, it carries two burdens. First, Shaffer often tries to use and/or explain his field's jargon, which I've spared you because I found it a distraction as a general reader. Second, Shaffer spends many pages describing computer games using words and the occasional picture, yet the thing that makes the games compelling is their visual and interactive appeal. If the book's content could be presented as a short documentary film or even a screencast, it would be far more likely to hold a general audience's attention.

(This suggests an accompanying book: "How Videos of Children Learning from Computer Games Help Adults Learn the Value of Children Learning from Computer Games." Or should it be a video?)

Bottom line: "How Computer Games Help Children Learn" gave me a fresh angle on today's educational challenges while detailing the first steps of a promising way forward. For people already interested in the issues, the book will be well worthwhile. For anyone else that wants to put a toe in the water, check Shaffer's Epistemic Games Web site, and if that gets you going, then get the book.

Walter Murch and the Long View of Film

Walter Murch was the film editor and/or sound mixer for American Graffiti, Apocalypse Now, Ghost, the Godfather, and Cold Mountain, to name a few movies you might know. I know him from a book, The Conversations: Walter Murch and the Art of Editing Film, by Michael Ondaatje (author of The English Patient, the film version of which Murch edited).

The book is a series of conversations between Ondaatje and Murch about filmmaking: the techniques, stories, and people behind the scenes. Along the way, we get two Renaissance Men's worth of eclectic digressions and connections, weaved together by Ondaajte in his role as editor of the text.

To give you a taste, I'll excerpt from two passages that interested me because of Murch's "long view" perspective on film's development as an art form.

We look at ancient Egyptian painting today and may find it slightly comic, but what the Egyptians were trying to do with the figure was reveal the various aspects of the person's body in the most characteristic aspect. The face is in profile because that reveals the most about the person's face, but the shoulders are not in profile, they're facing the viewer, because that's the most revealing angle for the shoulders. The hips are not in profile, but the feet are. It gives a strange, twisted effect, but it was natural for the Egyptians. They were painting essences, and in order to paint an essence you have to paint it from its most characteristic angle. So they would simply combine the various characteristic essences of the human body....

That's exactly what we do in film, except that instead of the body of the person, it's the work itself. The director chooses the most characteristic, revealing, interesting angle for every situation and every line of dialogue and every scene....It may be, five hundred years from now, when people see films from our era, they'll seem "Egyptian" in a strange way. Here we are, cutting between different angles to achieve the most interesting, characteristic, revealing lens and camera angle for every situation. That may appear perfectly normal to us, but people 500 years from now may find it strange or comic.

If that sounds unlikely, think an eventual future where "film" = holodeck.

On to the second passage:

I think cinema is perhaps now where music was before musical notation—writing music as a sequence of marks on paper—was invented. Music had been a crucial part of human culture for thousands of years, but there had been no way to write it down. Its perpetuation depended on an oral culture, the way literature's did in Homeric days. But when modern musical notation was invented, in the eleventh century, it opened up the underlying mathematics of music, and made that mathematics emotionally accessible. You could easily manipulate the musical structure on parchment and it would produce startlingly sophisticated emotional effects when it was played. And this in turn opened up the concept of polyphony—multiple musical lines playing at the same time. Then, with the general acceptance of the mathematically determined even-tempered scale in the mid-eighteenth century, music really took off. Complex and emotional changes of key became possible across the tonal spectrum. And that unleashed all the music of the late eighteenth and the nineteenth centuries: Mozart, Beethoven, Mendelssohn, Berlioz, Brahms, Mahler!

I like to think cinema is stumbling around in the "pre-notation" phase of its history. We're still doing it all by the seat of our pants. Not that we haven't made wonderful things. But if you compare music in the twelfth century with music in the eighteenth century, you can clearly sense a difference of several orders of magnitude in technical and emotional development, and this was all made possible by the ability to write music on paper. Whether we will ever be able to write anything like cinematic notation, I don't know. But it's interesting to think about.

While these excerpts typify Murch's erudition, they are more abstract than most of the book, which often is about how specific scenes in movies achieved their affect: how a distant, quiet sound ended up being more powerful than a layered mass of loud sounds in George Lucas' first feature film, THX 1138; how the framing of a scene in the Godfather tells the audience the character is lying; how a specific technique for recording and mixing crickets led to a "hyperreal" soundscape in Apocalypse Now; and so on.

If you read the book, you will not only know more about what makes movies tick, you'll also feel like you know Walter Murch. And that's a good thing.

Issues, Character, and Tuchman's The March of Folly

With presidential elections, it is often said that character trumps issues. For example, from the 3/11/2007 USA Today:

A new Associated Press-Ipsos poll says 55% of those surveyed consider honesty, integrity and other values of character the most important qualities they look for in a presidential candidate.

Just one-third look first to candidates' stances on issues; even fewer focus foremost on leadership traits, experience or intelligence.

[Side note: The article is far better than the usual media treatment of polls. It includes a comment about the implications of how the poll was structured. It also notes consistencies with past polling results. A companion article provides the detailed breakdown of answers and explains the margin of error in layman's terms. Good job, USA Today.]

Whenever I see a "character trumps issues" story, I feel like We the People are taking the easy way out. Compared to understanding the issues, deciding which candidate seems more honest is easy—not necessarily easy to get right, but easy in terms of effort required.

I've always assumed this easy way out was a bad thing. But having read The March of Folly by historian Barbara Tuchman, I may reconsider. Written in 1984, The March of Folly examines why various rulers and governments throughout history have engaged in policies that were obviously counterproductive in their time yet were pursued anyway.

The short answer is: corruption, misguided ambition on behalf of leaders' egos, and obliviousness to reality when the facts challenged an existing course of action. These factors manifest themselves differently in modern democracies than ancient kingdoms, but Tuchman makes the case that they've persisted in one form or another for a long time.

Near the end of the book, Tuchman says, "Aware of the controlling power of ambition, corruption and emotion, it may be that in the search for wiser government we should look for the test of character first."

I don't take this to mean, "ignore the issues." However, in a modern democracy, most mainstream positions on issues average-out to something workable over time, as one side wins for a while and then the other. Most important to Tuchman is avoiding the occasional but disastrous policy or institution that goes uncorrected despite widely understood flaws and viable alternatives. In her view, the true disasters have historically been, and will continue to be, driven by leadership failures involving character. So when character differences between candidates are significant, she might literally want the best man or woman to win, as opposed to who she agrees with most.

That said, it's worth noting that Tuchman's key character flaws involve power's corrupting and delusional influences. She is less concerned about personal vices or, for that matter, virtues such as heroism demonstrated in war. On this point she probably differs with some percentage of the poll respondents.

But even if she gets there for different reasons, Tuchman's version of "character first" is an interesting way to think about what the majority of voters apparently do. Tuchman no doubt would say voters should also understand the issues, but to the extent that many do not, voting on character may still achieve a lot.

Organic, Inc. by Samuel Fromartz

The U.S. organic food movement started as counterculture but is now accelerating toward the mainstream. Samuel Fromartz's Organic, Inc. tells the story of how and why.

A business reporter with a soft spot for healthy food, Fromartz pays due respect to both the organic purists, who decry their movement is being sold out to big business, and the organic popularizers like Whole Foods and Earthbound Farms, which have made megabucks spreading the organic gospel far and wide. Along the way, government agencies, agribusiness, and various others players make appearances.

Of the book's themes, the one I found most interesting was the divergence between healthy food and organic food. In the early years of the organic movement these concepts were nearly synonymous. The goal was food that's healthy for you and healthy for the planet; organic farming was a key means to the end. However, at the time, whether that healthy food tasted good was a secondary consideration, leading to the societal stereotype of "health food" as bland, killjoy food. But today, people increasingly believe they can have their organic cake and eat it too:

[O]rganic food persisted and grew precisely because the movement defined organic as a production method rather than a prescriptive diet such as Atkins, South Beach, the Zone, or Weight Watchers. The benefit came from eating the food, not from avoiding foods or counting calories. In this way, organic food became associated with a "healthy lifestyle," which meant you ultimately decided what made you feel good. Whole Foods's organic chocolate truffles epitomize this for me; they taste good because they contain chocolate, sugar, and saturated fat—not the healthiest mix. Yet by making them organically, Whole Foods tempered the "bad" quotient and transformed them into something "good."

For the purists, organic chocolate truffles are on the slippery slope that leads to the organic Twinkie, a totemic symbol of the final organic betrayal. Yet for the popularizers, an organic Twinkie is still better, for you and the planet, than a traditional Twinkie.

Adding a twist to this debate, Fromartz notes:

[Organic popularizers] argued that making an organic Twinkie would "Grow the market! Convert more land!" The purists said, "No! Organic food should be kept pure and the Twinkie banned!" What neither side imagined was that consumers might buy conventional Twinkies and wash them down with organic milk, or that such mixed consumption might be preferable.

Per that last quote, Fromartz covers various consumer research that says organic currently is nowhere near an all-or-nothing choice even for price-insensitive people who could buy organic alternatives for most of their food products. Today, people are paying the premium for organic foods selectively, in areas where the benefit is perceived to be most important. For example, organic is particularly strong in baby food, even for lower-income purchasers.

Now, with Wal-Mart looking to drive down organic prices, the further mainstreaming of organic food is inevitable. You may not know it, but healthy-brand icons Odwalla, Boca Burgers, and Kashi are already owned by Coca-Cola, Kraft, and Kellog's, respectively. And, by the way, not all of these healthy brands' products are organic—a further reminder that the relationship between "organic" and "healthy" is not simple.

It's a story with many chapters to play out. Organic, Inc. is a good guide to the action so far.

Unexpected Numbers from the Economist's 2006 Pocket World in Figures

Next time you come up short for cocktail-party chatter, just remember: "Equatorial Guinea."

I take that lesson from the Economist magazine's 2006 Pocket World in Figures, a book that compiles a wide range of numbers about various countries and regions of the world. Following are a few unexpected results that caught my eye.

Which country had the highest economic growth from 1993 to 2003, measured by the average annual percentage increase in real Gross Domestic Product (GDP)?
Everybody knows about China's big growth, but at 8.9% it's only enough for 4th place. The winner is Equatorial Guinea at 25.9%, due to its relatively recent exploitation of oil reserves. The other two ahead of China (Bosnia and Liberia) both experienced bounce-back growth after wars.

Which country is the largest donor of bilateral and multilateral aid, as a percentage of GDP?
If you're expecting a Scandinavian country to be the winner here, you are close. Norway (0.92% of GDP) and Denmark (0.84% of GDP) are numbers two and three. But number one is Saudi Arabia at 1.11% of GDP. What about the United States? Although it is by far the largest donor nation in absolute dollars, it ranks 26th when measured as a percentage of GDP (0.15%).

Which country is most energy efficient, in terms of GDP per unit of energy use?
This one is measured in "purchasing power parity dollars per kilogram of oil equivalent." I take that to mean economic output per energy input. The winner is Peru, and the rest of the top-10 countries are strange bedfellows: Hong Kong, Bangladesh, Namibia, Morocco, Uruguay, Colombia, Costa Rica, Ireland, and Italy.

Which country has the greatest number of cars per 1,000 people?
Unless you somehow already know the answer, don't bother guessing. The winner is Lebanon at 732 cars per 1,000 people. The United States is 14th at 481 per 1,000.

That result about car-happy Lebanon begs to have its source checked. However, specific sourcing is absent from 2006 Pocket World in Figures, an unfortunate omission even if the book's purpose is more toward entertainment than serious reference material.

So there you have it. If, after dishing these facts and figures, you aren't the life of the party, then you're not partying with the Council on Foreign Relations.

Ormerod's "Why Most Things Fail" and Schelling's Segregation Models

I recently read Paul Ormerod's Why Most Things Fail. Focusing on the frequent failure of companies and government policies, Ormerod argues that the environment in which these entities exist is so complex and unpredictable that even the best laid plans cannot reflect what will really happen. Between this complexity and the competition of many players laying plans, the ones that succeed often get there by luck. And once successful, the only viable strategy for long-term survival is constant adaptation via trial and error.

For a review of the book, I'll just agree with this Financial Times review's mixed bag of praise and criticism. However, I will highlight one of the book's better examples.

Economist Thomas Schelling created a model of how racial segregation happens. First, he posited a large grid, like a chessboard but much larger. Each square either has a red person's house, a green person's house, or nothing. A person will move if a certain number of his immediate surrounding neighbors are a different color—the number is fixed across all people, representing a societal level of tolerance for different-race neighbors.

It's a complex system because a single person's move can have cascading effects on the former and new neighbors, which in turn can have cascading effects. Thus, the results are not obvious from the initial conditions.

Northwestern University has a Schelling-inspired segregation simulator, where I generated this before-and-after combination.

Those familiar with cellular automata (CA) may have already expected that the few rules would give rise to order from randomness, but the nature of the order is surprising. For my model, I assumed that each person wanted at least 40% of neighbors to be the same race, yet the system ended up 85% segregated. Due to the system's interconnections, the relatively weak individual preferences, upon interaction, led to a strongly segregated society, a result typical of Schelling segregation models.

This type of result—where a societal outcome emerges "nonlinearly" from complex interactions—is what Ormerod sees everywhere, albiet in yet more complex form than this model.

And finally, if you're thinking that the Schelling model has its own kind of predictability, it does at an aggregate level. Ormerod does not give this point enough weight with regard to Schelling's work, but elsewhere in the book he describes how company failures are predictable in the aggregate by a power law distribution similar to that of biological species extinctions. However, these high-level patterns won't tell us when particular companies will fail or, in a real-world city, which people will move exactly where.

When is a Nutshell not a Nutshell?

I was in a bookstore yesterday and noticed that Java in a Nutshell, a popular reference book about the programming language Java, has become one of the fatter books on the Java shelf. The current, fifth edition is 1,252 pages and weighs 3.3 pounds—in paperback. In comparison, the book's first version was 460 pages.

I then came across a blog post by Kathy Sierra that notes (a) the first public release of Java's API had roughly 200 classes, and (b) Java's current standard edition has something like 3,500 classes.

So Java in a Nutshell deserves praise for growing less than 3x between its first and current edition, given that the Java class library has grown 17x. And one could argue that the nutshell-ness of the book is in the concise coverage of topics, not the number of topics. After all, it's not meant to be read straight through.

But that said, I can't help thinking if a book exceeds 1,000 pages, or requires multiple volumes, it's time to trade-in the nutshell metaphor—something more spacious would be less specious.

[Photo is from Wikipedia's "Coconut" page.]

All the Shah's Men

A book from 2003 that I read recently, Steven Kinzer's All the Shah's Men is a history of the CIA's 1953 coup in Iran. It was the CIA's first successful regime change, toppling Mohammad Mossadegh, the elected prime minister. However, Kinzer argues that the near-term win was a long-term loss, planting the seeds of the 1979 Iranian revolution and the virulent anti-U.S. sentiments that came with it.

The highlights:

It was about oil. Mossadegh incurred the wrath of the British by nationalizing the British-run company that had the exclusive franchise on Iran's oil. The British, in turn, packaged their frustrations with Mossadegh into scary scenarios about Iranian instability, baiting the Cold Warriors in the Eisenhower administration. The British wanted their oil franchise back, and the Americans wanted to ensure that Iranian oil didn't somehow end up under Soviet influence.

The British could not let go of the colonial mindset. As Iran's overlord in the early 20th century, the British had cut oil deals that left little for Iran. Before and during Mossadegh's reign, Britain had opportunities to correct this imbalance and defuse tensions. Yet even when American companies began doing 50/50 partnerships with oil-producing countries, Britain refused to go beyond modest concessions from its original deal. The British reasoning was that its involvement in Iran was a noble act of modernization, duly compensated by a long-term contract, and that the ungrateful Iranians needed to be kept in line. These rationalizations of the colonial order, even as its foundations were crumbling, precluded a true British partnership with Iran.

Mossadegh also had trouble letting go of the colonial mindset. Having gained his fame—and, at some level, his identity—as the opposition, he had trouble envisioning solutions that involved the British. While this single-mindedness was instrumental in bringing him to power, it was an obstacle later, as it confirmed for the British that they were dealing with (in their view) ungrateful and irrational natives.

A key enabler for the coup was the relative openness of Iranian society at the time. Mossadegh's reign had, by Middle East standards, relatively broad freedom of the press and assembly, factors the CIA exploited. The Agency bought-off newspaper editors to plant articles aimed at destabilizing Mossadegh's rule. And in the critical moments of the coup itself, the CIA hired thugs to mount violent demonstrations in favor of Mossadegh, with the purpose of provoking violent counter-demonstrations and clashes to deepen the chaos. The clerics of the Iranian revolution would later point to this exploitation of freedoms as reasons against granting them.

The coup's result was to make Iran's monarch, the Shah, its absolute ruler for the next quarter century. In that time, the Shah was suitably pro-Western albeit increasingly repressive at home. His crushing of political dissent radicalized the opposition, which eventually broke through in the 1979 revolution, rallying a significant vanguard of the population against the Shah and his main backer, the United States.

Meanwhile, within the CIA of the Cold War era, the Iran operation's perceived success was taken as a model for getting things done. Or as another review of All the Shah's Men put it, the Iran operation...

...got the CIA into the regime-change business for good—similar efforts would soon follow in Guatemala, Indonesia, and Cuba—but that the Agency has had little success at that enterprise, while bringing itself and the United States more political ill will, and breeding more untoward results, than any other of its activities.

This point of view fairly represents the book's bigger-picture perspective about unintended consequences. And if you didn't follow the link, you might be surprised to find that the quote comes from a historian on staff at the CIA, writing in the CIA's official journal.

Book Browsing in San Francisco

San Francisco book stores are plentiful and varied. The downtown area has a good sampling, with seven stores in close proximity. Starting at the upper left of this map and going counter-clockwise...

Borders (400 Post St.)
An above-average Borders location.

Cody's Books (2 Stockton St.)
Almost as big as Borders but tilted toward the literary and intellectual end of the spectrum. Good representation of academic/university-press titles.

Chronicle Books (101 Fourth St., inside the Metreon)
Small store featuring books by San Francisco publisher Chronicle Books. Meant for browsing and discovery.

SFMOMA Museum Store (151 Third St.)
Part of the San Francisco Museum of Modern Art. Lots of art and design books, from serious to silly.

Foto Grafix (655 Mission St.)
Specializes in books by and for artistic photographers.

Alexander Book Company (50 Second St.)
A relatively large independent book store that maintains a neighborly feel. Known for its African-American literature section.

Stacey's Bookstore (581 Market St.)
Big store, strong in computer books.

And if you are venturing beyond the downtown area, you won't see these on the map above, but they are San Francisco classics:

City Lights Books (261 Columbus Ave.)
Home to the Beat Generation poets and now a literary landmark. Co-founded by poet Lawrence Ferlinghetti.

Green Apple Books (506 Clement St.)
A sprawling mass of new and used books. If you can't find something to surprise you at Green Apple, you're not looking hard enough.

John Battelle's The Search

I recently read The Search by John Battelle. It's about how search has become central to the Internet economy and why today's search businesses, epitomized by Google, are the beginning of something even bigger.

John talked to me for the book, so I'm not trying to be Mr. Objective here. I'll just offer a perspective on its main theme.

To start, let me say that The Search dishes generous helpings of insider history about Google and other search players, past and present. John had access to most of the key people involved, so a lot of the quotes represent first-hand, new stuff. Given today's rampant Googlephilia, the book would have been plenty successful if it stopped there. 

The Search's distinction, however, is that it threads Google's story into a larger theme about what John calls the Database of Intentions: "the aggregate results of every search ever entered, every result list ever tendered, and every path taken as a result." (For the technical crowd, he does not mean "aggregate" in the analytical sense of summarizing/abstracting data; rather, he is referring to all the disparate bits of detailed behavioral data, accumulated together.)

No one company owns the Database of Intentions. It is spread among millions of Web sites, each of which collects its own data, as well as other network-based media (mobile applications, Tivo-style services, and so on). It turns out that the big search companies have among the largest concentrations of such data. Moreover, John argues that Goto.com/Overture was first, and Google has so far been best, at commercializing it on a large scale.

They have done so in a simple but extremely effective way, selling advertising associated with search keywords. When you search for "Sony VAIO," you are expressing intent, massively qualifying yourself to a certain set of companies: How much is it worth to Sony to appear next to search results for "Sony VAIO"? What about Sony's competitors? How about all the possible retailers of Sony VAIO products? Bidding is open for this and a practically unlimited number of other keyword combinations. And the kicker is, companies only pay when you click their ads, so the incentive to participate is high. Even a one-person small business can sign-up with a credit card. Hundreds of thousands have.

Now, let's reinforce two important points:

(1) This new ad marketplace allows targeting down to super-specific niches (or Long Tails). An example: In an attempt to get highly obscure, I searched "Charles Fourier," the utopian-socialist philosopher of the 1800s. I got an ad targeted to people researching him. The company behind this ad might have paid a nickel for my click on its ad, but as John summarizes the business model, it's "a billion dollars, one nickel at a time." A substantial number of those nickels represent new spending in the ad economy, from small businesses that have never had a venue beyond the yellow pages, or from bigger business (like Amazon.com) that can efficiently sell products with niche appeal along with mainstream products.

(2) Companies pay for ad responses, not ad impressions. This is a major change. Because a large percentage of advertisers can calculate what an ad response is worth to their business, they are ready to spend more on advertising than ever, for as long as it continues to pay back.

That last phrase—"as long as it continues to pay back"—is crucial. It brings us to The Search's big idea: that the Database of Intentions, in primitive form, is the key factor behind the current system's rise. Whereas a traditional advertising venue would be bragging if it could offer 31 flavors of content or demographics, this new system allowed search engines to sell millions of flavors of intent. These were sufficiently targeted to make the pay-per-click business model a winner. And the whole thing was automatable enough to allow a marketplace where anyone could participate.

It's a powerful combination of factors, yet John's bigger point is that these factors are only starting to play out. They will apply to other media, like television, soon enough. And as the Database of Intentions evolves and is mined more intelligently, it will make "as long as it continues to pay back" go longer and longer for the average advertiser. With billions of extra dollars awaiting ever more efficient forms of advertising and micro-targeting, further growth is at hand.

This extra efficiency can come from many angles. The Googles of the world will get smarter about mining and leveraging full clickstreams, not just your keywords-of-the-moment. Sites, or networks of sites, specialized in certain areas will make search and advertising more powerful by injecting domain-specific knowledge into their systems. John cites GlobalSpec, a site specialized in engineering parts, as an "intelligent island" that has manually created associations of concepts from its world. These associations—a Semantic Web (that is, web of meaning)—in turn can make searching and advertising smarter. Meanwhile, an informal version of the Semantic Web is emerging with user-created tags for various Web pages, which provides yet another kind of grist for the intelligence mill.

Finally, I would not underestimate the potential of people taking their data into their own hands, something that John does not delve into. In a world driven by data about intent, your intent becomes a kind of currency. As I've written elsewhere, the next Google may be a company that does not use data about people as a proprietary asset but rather becomes an asset manager for people's data. Given the Database of Intentions' privacy and societal implications, which John raises in the book, this type of approach has much to offer. It may well also be economically optimal.

Whatever happens, The Search makes clear that we're far closer to the beginning than the end of this important story. Kudos to John for going deep to tell it, conceptualize it, and popularize it.

My Freakonomics Encounter

This weekend I had the opportunity to chat with Steven Levitt, professor of economics at the University of Chicago and co-author/subject of the best-selling book Freakonomics. Thanks go to my wife Jacqueline, an alum of Chicago's Graduate School of Business, who had the "in" on a small get-together with Levitt at a private home in Tiburon.

For those that don't know, Levitt is famous for finding unexpected answers to real-world questions via quantitative analyses. For example, which is more dangerous to a child: a household with a gun or a household with a swimming pool? (Answer: When Levitt looked at cause-of-death data for children in the United States, he found that swimming-pool-related deaths were roughly 100 times more prevalent than gunplay-related deaths.)

If such nuggets interest you, you'll love Freakonomics. His co-author Stephen Dubner does for Levitt what Michael Lewis, author of Moneyball, did for baseball's sabermetricians and quants, bringing the numbers to life with well-told stories.

That said, the in-person version of Levitt was remarkably similar to the voice of the book. He's not an ivory-tower type that Dubner had to decode for the world. If fact he comes across as instinctively interesting. By that I mean his research seems motivated entirely by what intrigues him, yet a large number of people find the results fascinating. His brilliance is that you don't know you are interested in the guns-versus-pools question—or whether sumo wrestling is rigged, or whether the legalization of abortion led to reduced crime in the 1990s, or whether real estate agents act in your best interest, and so on—until he leads the way.

Levitt tends to focus on societal questions, but those of us in business analytics should thank him. Through Freakonomics, he is getting ordinary people interested in the value of using data and analytics to understand problems. Given that the alternatives—conventional wisdom, intuition, and "common sense"—are much easier for people to relate to, this is progress.

A few other Levitt resources for those interested:

Freakonomics Blog — Levitt and Dubner keep the freak-out going, including pointers to their pieces that appear in the New York Times.

Levitt's Papers — For those accustomed to academic papers and college-level math, you'll find Levitt an unusually clear writer. Also, the economics part of his work is more prominent here than in the Freakonomics book. I particularly liked the empirical analysis of gambling in the National Football League.

"Treating HIV Doesn't Pay" — Levitt mentioned this research about AIDS in Africa by Emily Oster. She used a Freakonomics-style analysis that generated surprising conclusions about the most effective way to minimize loss of life, given the fixed amount of money available. It may make disquieting reading, but there's no question this stuff matters.