Megan McArdle

Guest post by Laura McKenna, former political science writer, blogger, and freelance writer. 

In yesterday's New York Times, David Kirp, a public policy professor from Berkeley, explains that school integration made a large, long term impact on African-American students.

The experience of an integrated education made all the difference in the lives of black children -- and in the lives of their children as well. These economists' studies consistently conclude that African-American students who attended integrated schools fared better academically than those left behind in segregated schools. They were more likely to graduate from high school and attend and graduate from college; and, the longer they spent attending integrated schools, the better they did. What's more, the fear that white children would suffer, voiced by opponents of integration, proved groundless. Between 1970 and 1990, the black-white gap in educational attainment shrank -- not because white youngsters did worse but because black youngsters did better.

Not only were they more successful in school, they were more successful in life as well. A 2011 study by the Berkeley public policy professor Rucker C. Johnson concludes that black youths who spent five years in desegregated schools have earned 25 percent more than those who never had that opportunity. Now in their 30s and 40s, they're also healthier -- the equivalent of being seven years younger.  

Kirp calls for a return to integration. "If we're serious about improving educational opportunities, we need to revisit the abandoned policy of school integration."

I haven't seen those studies. I would like to see how they controlled for certain factors. Was there something different about the parents of African-American children who got their kids into those integrated schools? Did white students maintain their education advantage, because their parents put them in private schools or relocated to another town? Still, I'm pretty sure that their findings are accurate. Many other studies have shown the importance of peer group influences and the impact of wealth of a community on education outcomes. 

Kirp is right in some ways. Creating larger, more diverse schools would definitely improve outcomes of more children. However, he has little sympathy or understanding for the forces that stymie the efforts of reformers.  

There's no way to go back to busing or 70s integration methods. Racism might be a factor, but the biggest problem is self-interest. People worry that integration will harm their kids and their property values. 

It's a natural parental drive to provide your kids with the best things in life -- a nice home, good food and an excellent education, even if it comes at the expense of others or it flies in the face of political ideology. Our last two Democratic presidents sent their children to private schools, while at the same time having lunch with the teachers' unions. Parents want their kids in the Gifted and Talented Programs and don't want the special education kids to suck up too many resources. 

While there's little evidence that a diverse student body in terms of income, ethnicity, or cognitive abilities creates a worse learning environment for the most privileged kids, any threat to a child drives a parent insane. Protecting one's child is a natural instinct, and school reformers must deal with this instinct with compassion. 

When our public schools were gerry rigged a hundred years ago, few would have predicted the value ofone's home would be so tied to school quality. If my house was hoisted by a crane and dropped in Newark, NJ, the value of the home would plummet. If an influx of new kids cause overall test scores to drop, my property value would most likely drop, too. For most people in this country, their home is their biggest investment. A loss of property value makes even the most well meaning individuals to hyper-ventilate and worry about eating cat food during their Golden Years.

This natural instinct to protect property and children has undone more than integration efforts. School vouchers proposals were shot down in state after state in part because of the strength of the teachers unions, but also because there was huge resistance from suburban voters to open their schools to other children. Republicans, who ideologically support vouchers, voted it down in state legislatures, because their suburban constituents did not want urban kids using a voucher to attend their schools. 

So, how do we create more diverse schools without stepping on the natural instinct to protect children and property?  Baby steps and compassion.

Thanks to our terrific stable of outgoing guest bloggers--though they're not all leaving you; Scott Winship decided he wasn't done talking, so he'll be staying over.

Guest post by Jim Manzi, founder and Chairman of Applied Predictive Technologies, and the author of Uncontrolled: The Surprising Payoff of Trial-and-Error for Business, Politics and Society.

A commenter to one of Gabriel's posts made the point that it's hard for Megan's regular readers to have a sense of where each of the guest posters is coming from in general, and therefore how to see our various posts in context. This makes sense to me

For my final guest post, I'll try to provide context. Rather than going through some long framework, however, I'll give a very quick summary, and then tell a story that I hope illustrates what I'm trying to say.

In his review of my book at The New Republic, Eric Posner made a great overarching point:

The book is less interested in the RFT than in the limits of empirical knowledge. Given these limits, what attitude should we take toward government?

Guest post by Jim Manzi, founder and Chairman of Applied Predictive Technologies, and the author of Uncontrolled: The Surprising Payoff of Trial-and-Error for Business, Politics and Society.

I often criticize social scientists for making overly-aggressive claims for understanding causality in complex systems by building regression and other pattern-finding models. This is not evidence of some unique weakness of social scientists The same thing happens in business all the time, but business analyses tend not to be published for obvious reasons. A good example of one that has been published is in the current Harvard Business Review. This matters a lot, because HBR holds a unique position as the most important serious business publication in America.

Anne Marie Knott, professor of strategy at Washington University's Olin Business School, has written an article called "The Trillion-Dollar R&D Fix." The article proposes a new measurement of R&D effectiveness: RQ. In her words, RQ is a measure of "how effective your company is at R&D."

What is so striking to me about this article is how unvarnished Knott is in claiming that she has discovered a tool to do exactly what I say is so hard: make useful, reliable and non-obvious predictions for the effect of interventions in social systems. She writes that "Using standard regression analysis, the calculation tells us in a very precise way how productive each of the inputs is in generating output. It tells us, for instance, how much a 1% increase in R&D spending would increase a firm's revenue." Knott asserts that RQ allows the management if a company "to see how changes in your R&D expenditure affect the bottom line and, most important, your company's market value." She even names names: providing a table of what she thinks each of the top 20 public corporations in America should have spent on R&D, and how much more each would be worth if they followed her recommendations.

For example, Knott claims that she knows that Apple would have maximized its market value by spending $9.5 billion on R&D in 2010 They actually spent $1.8 billion. That's a fairly incredible claim. She thinks that what is generally conceded to be a management team that is pretty savvy about innovation underspent on R&D by more than 400 percent -- Apple ought to have quintupled its R&D spending in 2010. As another example, Knott claims that Dow Chemical could have roughly doubled its total market capitalization by increasing its R&D spending by 10 percent. That's a lot of money for them to leave on the table. And a very easy fix.

Knott claims that if just the top 20 American corporations had followed her recommendations, they would have collectively increased their market capitalization by more than $1 trillion. Consider this assertion for a moment The current total market capitalization of the top 20 U.S. public companies is a little over $4 trillion. Knott claims that she has outsmarted the entire system of management teams, investors, equity analysts, hedge funds, large-scale private equity firms and everyone else who is trying to change management practices to increase share price, and knows how to increase the total value of the most-closely followed companies in the world by almost 25 percent....by building a regression model using publicly-available data.

If you could rely on Knott's predictions, you could raise capital, buy these companies, change R&D spending in line with her model, and then sell them again at an enormous profit. You could start with Dow, because you know how to double its share price.

Maybe Knott has discovered an incredible, remediable market inefficiency, and somebody is about to get very, very rich. Or maybe there's a problem with her model.

The HBR article describes the calculation of RQ conceptually, and references a journal article (ungated) in which Knott describes the mechanics of it. In it, effectiveness in managing R&D is explicitly analogized to IQ, and refers to what is normally termed among businesspeople a competence for managing the R&D function. Knott contrasts her theory with an existing body of research on the topic:

Theories of innovation typically assume that firm R&D behavior is endogenously determined by industry conditions. If all firms in an industry share these conditions, and behave optimally then in equilibrium all firms should have identical R&D investment. Accordingly increases in R&D beyond the optimum should decrease market value. However the empirical record consistently demonstrates the opposite. Firms with higher R&D investment have higher market value.

We proposed that the inconsistency between theory and empirics stems from the assumption of homogenous firms. If instead firms have heterogeneous R&D elasticities (IQ), then a) the optimal levels of investment will differ across firms (firms with higher IQ invest more), and b) the market value per dollar of investment will differ across firms (firms with higher IQ have higher value per R&D dollar). This gives the empirical finding of increasing market value for increased R&D spending theoretical grounding: It is not that investing more in R&D increases market value, it is that higher IQ yields both higher returns (and market value) and therefore stimulates greater investment.

I'm sure she's correct that there are a lot of academic analyses that assume all firms have equal competence in R&D. Such studies may have some utility for some purposes. But the idea that competence in research management varies across firms is a belief that is universally held among relevant senior executives. I mean this literally. I doubt you could find three COO / CEO level executives among all large public U.S. companies who spend significant amounts on research that disagree.

So the non-obvious claim is that she has built a model which quantifies this effect with sufficient precision to reliably change the decision about how large R&D budgets ought to be, as compared to current management practices.

Knott uses a dataset of annual data for 610 publicly-traded American companies from 1981 - 2006. The primary data elements by company are annual numbers for: market value, revenues, Property, Plant and Equipment (PPE), number of employees, advertising spending, and R&D spending. (This was merged with data on patents by company in order to build a separate model.). From this primary data, Knott builds a time-series regression equation to estimate the causal effects of R&D spending.

The problems with this should be apparent.

I'm confident that exactly the effect Knott describes is real. Some companies are better at managing research, and they will spend more, all else equal, and create greater returns for it.

But causality also runs in the opposite direction. For example, when management teams rationally foresee a good year coming, they tend to relax spending discipline. So we will see R&D spending go up in year X, and profits rising in year X+1. The expectation of future profits cause R&D spending to rise today This effect is unobserved in Knott's model, because we have no data on executive anticipations. Some other variables will proxy for it, but the correlation between the actual unobserved variable and the proxy won't be close to 1.

Further, there are many confounding variables. For example, different firms that are called competitors will actually face different landscapes of potential R&D opportunities, independent of R&D effectiveness. IBM and HP, as examples, have different mixes of end-use markets, different customer bases, different installed technologies and so on that means that each is looking at a different list of potential relevant projects when deciding what to fund. This changes over time. Who were Apple's competitors in 1995? 2000? 2010? Who will be their competitors in 2015? This is referenced conceptually in Knott's paper, but how do we segregate this effect from RQ and everything else when the model has no data on it.

As another, firms have different "general management IQs" to use Knott's language. We could be cute and call this MQ. This will lead some firms to modify their RQ over time, and to better perceive potential opportunities than others, independent of effectiveness in going after them. This will also have some consistency and some changes over time, and the model is blind to it.

And yet further, there are also relevant interactions between each of these example variables. For example, higher MQ management teams will tend, all else equal, to get firms into a position with a better portfolio of possible R&D investments, but will also tend to lead them to exert more consistent cost discipline in good and bad years. These will tend to be persistent effects, but high MQ teams will also likely react better to changing external circumstances. MQ will correlate with RQ, but the correlation will be materially greater than 0 and materially below 1.

Knott's model considers none of this. I have written extensively, and more technically, here at The Atlantic about why attempts to use methods like those Knott employs (e.g., two-step Instrumental Variable models), to try to isolate the causal impact of variable X on corporate performance can't perform the magic of somehow overcoming the problem of never including so much of the relevant data. I summarized the conclusion as:

There's just no way out of the problem that what makes companies do well or badly is very, very complicated, and therefore isolating the impact of any one variable by lining up some descriptors for a few hundred companies and looking for patterns is like trying to grab liquid mercury.

Think of what Knott's advice means in practice. We would sit down with Apple's management team and say that they should quintuple their R&D spend. To avoid getting laughed out of the room, we would actually say "OK, we think there is high unexploited opportunity for R&D spend, so bump it up 10 percent." What this boils down to is some combination of taking the prioritized list of projects that have been considered, and move the "green light" line down further, and of rethinking our prioritization scheme somewhat, so as to increase spend by 10 percent. Presumably, afterward we would want to go back and try to evaluate whether these extra projects we therefore funded actually created market value. This is usually tricky, and requires judgment, but for many projects, we can evaluate the process cost reductions, or number of units the new product line sold at what margins, or whatever.

But all good executive teams do this already. They constantly attempt to evaluate how wide the choke on the R&D budget ought to be set by looking at actual performance after the fact.

If the advice is "in general, companies ought to try out getting a little looser with the R&D budget and see how it works." Fair enough, but nothing new. If the advice is "set your R&D budget each yer using this formula, and don't draw any subsequent conclusions based on the actual performance of the extra projects you funded," then it's not really very useful.

More »

Guest Post by Gabriel Rossman-- Professor Rossman is a sociologist at UCLA. His work applies economic sociology to media industries. He blogs at Code and Culture and is the author of Climbing the Charts

In the course of a discussion about Mormons, a friend pointed me to a religious testimony offered by Clayton Christensen (who is best known for his work on disruptive innovation). In his testimony, Christensen describes his belief in the Book of Mormon through a religious epiphany reminiscent of St. Augustine's "tolle lege" experience. However this is in the second half of the essay, the first half being devoted to a description of the strength of the LDS community and an argument that this social capital is directly related to the lay priesthood ecclesiastical structure of LDS. One story from this part of the testimonial struck me in particular:

[O]ur family had out-grown our small home, so we found a larger one and put the word out that we would appreciate any help in loading and unloading our rented moving truck. Among those who showed up that morning was Mitt Romney, now the governor of Massachusetts, who had just completed his unsuccessful campaign for the U.S. Senate in Massachusetts. Mitt had a broken collarbone, but for two hours traipsed between our home and the truck, carrying out whatever he could manage with his one good arm.

From a purely utilitarian perspective this is ridiculous. You have a man with a vast fortune and a (temporary) physical impairment. It would be an obvious gain from trade if rather than providing his hobbled physical labor, Romney were instead to give Christensen $100 and tell him to hire a day laborer. Of course if it happened like that the story would be the plot to an episode of Curb Your Enthusiasm with the upshot that Larry David is an obtuse misanthrope, not a religious testimonial with the upshot that Mormons have a strong community. Indeed this story is in the middle of a paragraph which begins and ends by talking about how as a general matter Mormons are eager to help one another and is part of a broader argument about how Mormons provide both mundane and ecclesiastical services to one another directly rather than through professionals. In the context of the essay, the practical value of the impaired labor that Romney provided is clearly secondary to the affirmation of moral community implied by his willingness to provide it. In this sense, that Romney was injured makes his contribution more significant, not less, which is why Christensen chose to draw attention to it.

Similarly, consider Joel Waldfogel's AER article "The Deadweight Loss of Christmas" (which he later adapted into Scroogenomics). The article basically demonstrates that people don't especially like the gifts grandma gives them for Christmas. I like Waldfogel a lot* and think this article makes a real contribution in showing how gifts are a deadweight loss when viewed from the perspective of market pricing. However treating this as a problem and normatively asserting that people are irrational to give gifts is like an astronomer chastising a comet for not having the right orbit. (This is not an uncommon issue with economists). The conclusion suggests the policy proposal that replacing in-kind welfare benefits with cash transfers would increase the poor's utility. (Again, not unheard of). In related news, if my grandmother had balls she'd be my grandfather. There is a certain logic to replacing in-kind programs with cash transfers that is very compelling on its own terms, but in practice few people would agree to it. One of our biggest transfer programs is Medicaid, and converting it to a cash transfer would mean that especially sick poor people would go without heath care, something the left would find unacceptable. (You can see this understanding implicit in the individual mandate, which not only serves the wonkish goals of avoiding the death spiral and cross-subsidizing the sick, but perhaps more importantly the political goal of including in universal coverage those people who would rather spend their money on something other than insurance premiums). Likewise, the right has a habit of objecting when welfare recipients spend transfers frivolously on either an isolated or widespread basis. In the 1990s it was a common trope to complain about welfare recipients who had cable television. More recently we've seen complaints about (and restrictions against) people drawing transfer payments from ATMs at casinos and strip clubs or using food stamps to buy junk food. That is to say, there is an implicit, pan-ideological consensus that transfers are about society providing the poor with that which we deem it appropriate for them to have and not that which they would purchase themselves if they had the money. A cash transfer welfare state would be politically untenable even though it is probably true that cash would be more efficient (as assessed by the utilitarian logic of market pricing).

Human beings have a variety of ways to exchange goods and services and the ways we do so both reveals and structures the nature of our relationships. Alan Fiske's relational models theory describes four types of exchange:

• communal sharing -- people are effectively a common unit and can freely draw resources from communal property, as with households
• authority ranking -- people have asymmetric duties and obligations to one another, as with patron-client ties
• equality matching -- people match actions on a like-for-like and tit-for-tat basis, as with friends
• market pricing -- people commensurate across categories on the basis of ratios (with prices being a special case of these ratios when we have money as a unit of account), as with traders in a market

In this schema we would say that Mormons have a communal sharing relationship with each other (at least for some services) whereas welfare recipients are in an authority ranking relationship with the state, as are Wharton students in authority ranking relationships with their grandmothers but in equality matching relationships with one another.

The interesting thing about equality matching is how central reciprocity is to it. It is a common trope in the gift literature to note the impossibility of the "pure" (that is, unreciprocated) gift. In Debt,** Graeber notes that some religious traditions emphasize that because anonymous gifts cannot be reciprocated (either in-kind or with clientalism) they are the highest form of charity. Graeber uses the impossibility of reciprocity with Santa Claus as a familiar example, and I would add that traditional iconography of St. Nicholas depicts him furtively tossing gold through a window to serve as dowries for three poor sisters who would otherwise be driven to prostitution. The social scientific point is not to argue that we ought to avoid reciprocity, only that it is so core to gifts that avoiding it requires special circumstances and in the normal course of things a gift will be reciprocated. The normal way not to enter into a reciprocal relationship is not to gift at all. For instance, last Christmas I deliberated giving a friend an album that I thought would resonate deeply with him, but I refrained from doing so precisely because I didn't want to impose an obligation to reciprocate. This same friend and I have bought one another meals, but reciprocity is achieved in-kind and is not commensurable with other types of gifts. In equality matching a meal cannot be easily reciprocated with an album, but only like for like.

A major source of social conflicts and scandals is when people disagree about what relational model does or ought to characterize an interaction. For instance, suppose a woman is attracted to a man at a party and they end up sleeping together. As they part, he says "that was great" and hands her $100. In blackboard economics this is just lovely. We know the utility to be derived from the hookup was sufficient to motivate her to go home with this guy. If x utility exceeded her reservation then surely x + $100 exceeds it by that much more. Surplus! Of course no human being (including economists) actually believes this. We all know intuitively that she would not view the $100 as income but as an accusation. In contrast if the man did not hand her cash at their parting but rather a day or two later sent her flowers and similar gifts worth $100 she would not be insulted. We can imagine her refusing either the cash or the gifts, but in the one case her refusal would mean "I'm offended that you think I'm a whore" whereas in the other it would mean "I'm sorry but I don't want to get into a relationship."

In the last week we've seen a fair amount of outrage over Facebook billionaire Eduardo Saverin renouncing his US citizenship to reduce his capital gains tax liability. The controversy is premised on an understanding that the citizen's proper relationship to the state is one of authority ranking where the state has such obligations to the citizen as to provide physical security and the citizen has such reciprocal but asymmetric obligations as to pay taxes. This understanding is betrayed by Saverin's tax exile, which treats citizenship as in the realm of market pricing. Not surprisingly people who tend to be skeptical of authority ranking relationships view Saverin's actions more sympathetically.

For another illustration of how we can get into trouble with conflicting understandings of relational models, consider another episode from Mitt Romney's life. In 1981, Romney was arrested for launching a boat after a police officer warned him that his boat's license number was inadequately displayed and he faced a $50 fine. Romney launched the boat anyway and the cop arrested him. What seems to have gone on is a conflict in how to understand the interaction up unto that point. The cop seems to have seen himself as giving an order which was then disobeyed. That is, a violation of an authority relationship which requires the lower party to show deference. Conversely, Romney described the situation as "I was willing to pay the fine. But if he had said don't launch the boat and not mentioned the fine, I would not have done it." That is he was operating under the understanding that, as Gneezy and Rustichini later put it, "a fine is a price." In this model a naked demand carries the weight of the relationship behind it, whether it be between daycare workers and parents who are late to pick up their kids or between a cop and citizen whose boat's tags are marred by some stray paint. This moral weight will often be enough to prevent transgression. In contrast a fine puts a price on the action and this finite price may be less inhibiting than the unpriced, and thus in some sense infinitely priced, demand. Romney's understanding was that the fine was a price and that if he was willing to pay the price this would fully settle the matter. In other words, it was a matter of market pricing. In contrast the police officer did not seem to be worried that Romney would skip out on the fine but that, as Eric Cartman would say, he had failed to "respect my authorité" ranking.

__________________________________

* I am familiar with and admire Waldfogel's work because we both study mass media. My favorite of his articles is a QJE on chain ownership in radio to which I devote an entire lecture in my undergraduate course.

** Graeber uses a similar typology as Fiske but with slightly different nomenclature: communism, hierarchy, and exchange. The main difference between their respective typologies is that whereas Fiske has separate categories for equality matching and market pricing, Graeber treats both of them as subcategories of "exchange."

More »

Guest post by Gabriel Rossman -- Sociologist at UCLA. His work applies economic sociology to media industries. He blogs at Code and Culture and is the author of Climbing the Charts.

I study media markets and one of the interesting things about the entertainment industry is there's a lot of complex pricing. This includes both simple bundling (eg, basic cable) and two-part tariffs (eg, HBO). These pricing practices are forms of price discrimination, which is to say they are ways to customize the price point so the seller doesn't leave much money on the table relative to each particular consumer's willingness to pay. It's kind of like haggling but it works at scale over a large number of consumers.

Price discrimination is a mixed bag. On the downside it pushes consumer surplus close to zero, meaning you always feel like you're getting ripped off but not so much that you balk. On the upside it increases total revenues and quantity supplied. The effects can be pretty substantial. For instance, the record labels' main problem isn't decreased quantity supplied but the end of bundling with the switch from albums to singles. Likewise one of the most popular explanations for the decline of the Hollywood studio system is that the Paramount decision banned a form of bundling called block-booking and this decreased revenues sufficiently that the studios couldn't maintain a vertically-integrated production system.

The thing is, that price discrimination is only supposed to work under certain very narrow circumstances. Suppose we have a two-part tariff seller, say, a movie theater selling tickets for $10 and popcorn for $5. If a competing theater opens across the street charging $12 for tickets and $2 for popcorn, you'd expect to see everybody who doesn't like popcorn stay at the first theater and everybody who does like popcorn go to the second theater. That is, a price discrimination scheme should very quickly break down in the face of perfect competition and in fact this problem is so well understood that monopoly is understood to be a scope condition. For instance, the word "monopoly" is in the title of one of the major cites on two-part tariffs.

So what about when you don't have a monopoly or perfect competition, but something in between? In theory, you don't need a perfectly competitive market with innumerable infinitesimally small price-takers in order to get something that looks a lot like a Walrasian auction. This is why industrial-organizational econ loves game theory. Once you apply a prisoner's dilemma model to price competition in a market with two sellers (duopoly) or a handful of sellers (oligopoly), it looks a lot more like a market with an infinite number of sellers (perfect competition) than it does like a market with exactly one seller (monopoly).

The thing is though that we have lots of cases of price discrimination and most of these cases occur in reasonably competitive markets, with multiple sellers and no apparent price-fixing. For instance, I previously noted that movie theaters practice two-part tariffs but let's reflect on the fact that this is a competitive industry. This raises the puzzle of why we haven't seen a chain of movie theaters compete by giving up the two-part tariff business model, which would mean cheaper popcorn but higher ticket prices.

These kinds of issues are why I was so interested when a colleague recently sent me Xavier Gabaix and David Laibson's QJE paper "Shrouded Attributes, Consumer Myopia, and Information Suppression in Competitive Markets" (ungated version). The reason the paper is important is that last phrase about "competitive markets." It shows how all sorts of stuff we already knew could go on with monopolies can also occur under competition. "Shrouded attributes" refers to hidden costs like the marked-up component of the two-part tariff. The "consumer myopia" phrase explains that this works if you make the reasonable assumption that many consumers aren't reasonable.

The logic goes that we imagine two types of consumers, myopic and sophisticated, and two types of products, "no hidden fees" (but with a high base price) versus "low price" (but which nickel and dimes you to death). The myopic customers will flock to the low price provider because they don't realize that they'll wind up buying $10 peanuts from the minibar. The sophisticated customers will go to either the "no hidden fees" or the "low price" provider based on who gives a better deal when you tally up the total cost of the basket they expect to consume. What this means from the provider's perspective is there are no customers who will pay more for a given basket of goods under a "no hidden fees" plan than they would in a "low price" plan. As such, the "low price" plan can crowd out the "no hidden fees" plan.*

The upshot is that some of the things we thought could only exist given the rare scope condition of monopolies or collusion can also exist given the all too plausible assumption of bounded rationality. This is pretty cool in an empirical sense because when our theories told us this sort of thing could only exist with monopolies it was kind of anomalous to go through life constantly coming across $150 printers that take $100 toner cartridges, smart phones which cost $200 but which lock you into a two-year contract at $80/month, hotels that charge $100/night but add a mandatory $15 "resort fee," etc.

My favorite example of how consumer myopia works this way is how rental cars now offer you prepaid gas for about 10% less than the price you'd pay at the pump. The consumer may be thinking, wow, $3.90 a gallon is actually pretty cheap compared to $4.30 at the Exxon station, I should prepay for this full tank of gas. What doesn't occur to this consumer is that this is only cheap if you use the full tank. If you bring it back with half a tank you're effectively paying $7.80 a gallon. The consumers who appreciate this don't buy the plan but those who don't see the hitch may well buy it. As long as the base rental price isn't too low it makes sense for the rental company to more or less break even on people who pass up this offer and make an easy $20 or so profit on those who sign up for it.

There are also some big policy implications. Under the old model, as long as you have a modest antitrust policy in place, the market will sort things out so consumer protection can be limited to outright fraud. Adding bounded rationality into the mix suggests that consumers really can sign up for a bad deal. This then makes it somewhat facile to claim that by definition any freely entered exchange is mutually beneficial and from this we can imagine a variety of consumer protections. You may still have reasons to be skeptical of intervention, but it's not tenable to say "markets work just great, thank you very much." 

* You may have seen the argument that even if individuals are irrational, markets can still be rational because irrationality gets arbitraged out. Let's accept this for the sake of argument and just note that it has scope conditions that basically mean it only works on Wall Street. With retail markets it's difficult to establish a secondary market and so you don't see the sophisticated people doing arbitrage that pushes us all back to predictions consistent with rational actors.

More »

Guest post by Jim Manzi, founder and Chairman of Applied Predictive Technologies, and the author of Uncontrolled: The Surprising Payoff of Trial-and-Error for Business, Politics and Society.

Gabriel, your very deep post that, in passing, requested my comment was fascinating.  My family thanks you for the weekend I just spent staring off into space.

You open with this:

Sampling error? Omitted variable bias? Bah, that's for first-year grad students. What I find really interesting is there are some fairly basic principles for how analysis can get really screwy but which can't be fixed by adding more control variables, increasing your sample size, or fiddling with assumptions about the distribution of the dependent variable.

I spend an enormous amount of time in my book arguing that that this problem is pervasive and significant, and that exactly this triptych of remedies will fail to enable us to build models that make useful, reliable and non-obvious predictions for the effects of our interventions in human social systems.  In it, I take apart some celebrated social science models for failing in this respect.  But in the spirit of what's sauce for the goose is sauce for the gander, I then take apart a model that I built to estimate the effect of changing the name of a convenience store, to show how all three together can't put Humpty Dumpty back together again.

Start at the most foundational level: What is causality?  I have an engineer's perspective on this.  What I care about is my ability to predict the effect of my interventions better than I can without the model. 

Consider two questions:

1.     - Does A cause B?

2.    - If I take action A, will it cause outcome B?

I don't care about the first, or more precisely, I might care about it, but only as scaffolding that might ultimately help me to answer the second.

For example, in your shoes story, I don't care whether the characteristic of discomfort cause shoes to be considered attractive.  I care about whether, for example, if I take an existing type of shoes and narrow the toes, this will cause them to get more coverage in fashion magazines, sell more units or whatever.

In general, the best way to determine this is to take some comfortable shoes, narrow the toes, and then see what happens to sales.  That is, to run an experiment.

There are big problems with this approach.  One obvious one is that it is often impossible or impractical to run the experiment.  But even if we assume that I have done exactly this experiment, I still have the problem of measuring the causal effect of the intervention.  In a complicated system, like shoe stores, I have to answer the question of how many pairs I would have sold in the, say, three months after changing my design to narrow toes - I can't just assume that I would have sold the same number of wide-toed shoes that I did in the prior three months.  For reasons well-known to you, and that I go through at length in the book, the best way to measure this in a complicated system is a randomized field trial (RFT) in which I randomly assign some stores to get the new shoes and others to keep selling the old shoes.  In essence, random assignment allows me to roughly hold constant all of the "screwy" effects that you reference between the test and control group.

But what many cheerleaders for randomized experiments gloss over is that even if I have executed a competent experiment, it is not obvious how I turn this result in to a prediction rule for the future (the problem of generalization or external validity).  Here's how I put this in an article a couple of years ago:

In medicine, for example, what we really know from a given clinical trial is that this particular list of patients who received this exact treatment delivered in these specific clinics on these dates by these doctors had these outcomes, as compared with a specific control group. But when we want to use the trial's results to guide future action, we must generalize them into a reliable predictive rule for as-yet-unseen situations. Even if the experiment was correctly executed, how do we know that our generalization is correct?

A physicist generally answers that question by assuming that predictive rules like the law of gravity apply everywhere, even in regions of the universe that have not been subject to experiments, and that gravity will not suddenly stop operating one second from now. No matter how many experiments we run, we can never escape the need for such assumptions. Even in classical therapeutic experiments, the assumption of uniform biological response is often a tolerable approximation that permits researchers to assert, say, that the polio vaccine that worked for a test population will also work for human beings beyond the test population.

But as we climb a ladder of phenomenological complexity from physics to biology to sociology, this problem of generalization becomes more severe.  As I put it in Uncontrolled:

We can run a clinical trial in Norfolk, Virginia, and conclude with tolerable reliability that "Vaccine X prevents disease Y." We can't conclude that if literacy program X works in Norfolk, then it will work everywhere. The real predictive rule is usually closer to something like "Literacy program X is effective for children in urban areas, and who have the following range of incomes and prior test scores, when the following alternatives are not available in the school district, and the teachers have the following qualifications, and overall economic conditions in the district are within the following range." And by the way, even this predictive rule stops working ten years from now, when different background conditions obtain in the society.

We must have some model that generalizes.  What we really need to do is to build a distribution of results of "experiments + model" in predicting the results of future experiments.  An example of what I mean applied to criminology is the following from the article I referenced above:

One of the most widely publicized of these [criminology RFTs] tried to determine the best way for police officers to handle domestic violence. In 1981 and 1982, Lawrence Sherman, a respected criminology professor at the University of Cambridge, randomly assigned one of three responses to Minneapolis cops responding to misdemeanor domestic-violence incidents: they were required to arrest the assailant, to provide advice to both parties, or to send the assailant away for eight hours. The experiment showed a statistically significant lower rate of repeat calls for domestic violence for the mandatory-arrest group. The media and many politicians seized upon what seemed like a triumph for scientific knowledge, and mandatory arrest for domestic violence rapidly became a widespread practice in many large jurisdictions in the United States.

But sophisticated experimentalists understood that because of the issue's high causal density, there would be hidden conditionals to the simple rule that "mandatory-arrest policies will reduce domestic violence." The only way to unearth these conditionals was to conduct replications of the original experiment under a variety of conditions. Indeed, Sherman's own analysis of the Minnesota study called for such replications. So researchers replicated the RFT six times in cities across the country. In three of those studies, the test groups exposed to the mandatory-arrest policy again experienced a lower rate of rearrest than the control groups did. But in the other three, the test groups had a higher rearrest rate.

Why? In 1992, Sherman surveyed the replications and concluded that in stable communities with high rates of employment, arrest shamed the perpetrators, who then became less likely to reoffend; in less stable communities with low rates of employment, arrest tended to anger the perpetrators, who would therefore be likely to become more violent. The problem with this kind of conclusion, though, is that because it is not itself the outcome of an experiment, it is subject to the same uncertainty that Aristotle's observations were. How do we know if it is right? By running an experiment to test it--that is, by conducting still more RFTs in both kinds of communities and seeing if they bear it out. Only if they do can we stop this seemingly endless cycle of tests begetting more tests. Even then, the very high causal densities that characterize human society guarantee that no matter how refined our predictive rules become, there will always be conditionals lurking undiscovered. The relevant questions then become whether the rules as they now exist can improve practices and whether further refinements can be achieved at a cost less than the benefits that they would create.

We can then then compare the accuracy of such a theory this to analogous distributions of predictions made by non-experimental methods (that can vary from sophisticated regression models to newer machine learning techniques to prediction markets to the judgments of experts, and so on) for predicting the results of future experiments.  As I put this in the book:

The job of experimentation in business is to put rounds on target. Abstract discussion of causality is a means to the end of using prior experimental results to more accurately predict the shareholder value impacts of various alternative potential courses of action.

As I go into, there is no absolutely secure philosophical resting place.  That is, even if I have such a distribution of results for the predictions made by various methods, I can't ever be absolutely certain that this distribution won't suddenly change.  (I expend a lot of effort trying to unify the problem of induction and the reference class problem to show that this is always a risk, no matter what.)  But I think this is as close as you can get. 

What this demands, of course, is a lot of experiments.  This is why lowering the cost per test is so critical.  Not just as an efficiency measure, but because in practice in enables me to get to much more reliable predictions of the effects of my proposed interventions.

To come back to where we started, I think this this is the way to evaluate whether some model, tool, guru or whatever has "really" discovered a causal relationship.  A statement about causality only has operational meaning as a predictor of future results of rigorous tests of the causal theory for the outcome of an intervention.

Guest post by Scott Winship, Brookings Institution. Follow him on Twitter: @swinshi

I promised that this was the last post I would write this week dwelling on rising inequality at the top, and I do want to shift to the comparatively under-appreciated lack of rising inequality in the bottom half of incomes.  But bear with me, as this turned into two separate posts. 

To review, in my first post on high-end inequality, I showed how outsized gains at the top are mostly concentrated in the top half of the top one percent and noted that these gains came even as the poor and middle class became significantly better off.  In my last post, I demonstrated that some potential shortcomings of these estimates do not seem to actually alter conclusions about the rise in inequality.  In my next two posts, however, I want to nevertheless flag some important sources of ambiguity about the data on top incomes that are available.

First, there is some question as to how robust some of the key results for early decades in the Piketty/Saez series are. You can use the figures they have made available to compute the average income of the bottom 90 percent or 99 percent of tax returns over time. In the chart below, the red line gives the trend in the bottom 90 percent's average income, pegged to 1917 levels.  It shows an implausible 91 percent increase over the three-year period from 1940 to 1943.  As the chart indicates, a lot happened during these years that might affect the Piketty/Saez estimates.  From 1939 to 1946, federal income taxes went from being something only the rich paid to something nearly everyone paid.  This fact matters because the low filing rates in the first part of the decade force Piketty and Saez to compute their figures differently than they do in later years.  Until 1944, Piketty and Saez determine the share of income received by the top ten percent (or top one percent) of tax returns by comparing the income they report to an aggregate figure drawn from national statistics collected outside the IRS.  They are forced to do so rather than compute total income received from the tax return data itself, which isn't informative in years when few people filed.

The purple line in my chart attempts to correct the average income estimate for the bottom 90 percent.  Let me get into the weeds in a moment to say what exactly I did, but first note what happens to the share of income received by the top ten percent when I make this correction, conveyed by comparing the blue and green lines.  Rather than showing pre-1940 income concentration at the top to rival that in the last 30 years, and rather than showing a big decline in income concentration in the early 1940s, the revised trend indicates hardly any change in income concentration from 1930 to 1980.  Since the basic assumption among researchers who study income inequality trends is that inequality has followed a big U-shaped trend over the past 100 years, this is kind of a big deal (but of course, it makes the recent run-up in inequality that much more striking).

ALL ABOUT THE LAST 30 YEARS

Now, before anyone runs too much with this revisionist take, I don't want to make the strong claim that inequality didn't change much until the past 30 years. For starters, if you do this same exercise for the top one percent rather than the top ten percent, you still get a big decline in inequality between 1930 and the mid-1970s, though smaller than before.  The 1929 peak drops from 24 percent to 19 percent, and the early 1940s decline that Piketty and Saez show shrinks dramatically. It would certainly require a change in thinking about historical income patterns if half the drop in the share received by the top one percent from 1928 to the mid-1970s accrued to the next richest 9 percent.  In the Piketty/Saez data, that next-richest 9 percent didn't receive any of the bounty.

More importantly, however, other data sets using other measures of income and earnings inequality also show big declines in the 1940s (though note that the trend for the top one percent doesn't have to mirror what happens to inequality among the 99 percent).  The basic point is just that relatively innocuous-seeming decisions can produce pretty dramatically different numbers in this earlier period. 

Furthermore, marginal tax rates rose dramatically over the period.  In 1929, a taxpayer with $100,000 in today's dollars paid a 3 percent marginal tax rate, and it was just 10 percent in 1940.  In 1941, however, it jumped to 21 percent, rising to 30 percent in 1942 and to 38 percent by 1948.  For a taxpayer with a million dollars in income in today's dollars, the rise was from 23 percent in 1929 to 39 percent in 1939, jumping to 51 percent in 1940, 63 percent in 1941, 78 percent in 1942, and 89 percent by 1948.  You think these taxpayers had strong incentives to keep their incomes off of federal income tax returns (and out of the Piketty/Saez data)?  Not only were the incentives strong (and options for avoidance plentiful), but the IRS was less equipped to keep up with tax avoidance practices in the days when the federal government was much smaller than it was today.

What about the high-end income concentration estimates for the past thirty years? Anything left to worry about there?  You'll just have to read the next post to find out.  In the meantime, here are the deets for the "correction" I made above.

From 1943 onward, I use the Piketty/Saez numbers (though they show up in my chart below the red line because they are now pegged to a higher 1917 estimate).  From 1940 to 1943, I rely on the annual percent change in average earnings (not total income) for all workers (not just the bottom 90 percent of tax returns) from a separate data source.  Specifically, Saez has a paper with two other coauthors using Social Security Administration data on earnings, and to his credit, they make lots of their figures publicly available too.  In general, the annual growth in SSA average earnings tracks the annual growth in the Piketty/Saez bottom-90-percent income very well over the decades for which both data sets provide estimates--except from 1940-41 to 1946-47. 

After making this correction, from 1917 to 1940 I go back to the annual growth rates in the Piketty/Saez data (the SSA data only goes back to 1937).  In other words, I go backward from 1943 to 1917, correcting the 1940-43 trend, which raises the pre-1940 average incomes when I go backward from 1940 to 1917. You can see that this fixes the big 1940-43 increase that the red line shows. The big assumption here is that the top 10 percent incomes are measured well prior to 1943 (or at least as well as they are after 1943) but the bottom 90 percent incomes are not (because they are residuals from the separate aggregate national income data used).

Guest post by Scott Winship, Brookings Institution. Follow him on Twitter: @swinshi

My last post provided an initial look at the inequality figures focusing on "the top one percent," which show sharply rising income concentration over the past 30 years.  But those figures rely entirely or heavily on tax return data from the IRS.  This creates some issues that warrant skepticism about the magnitude of the increase I described.

Piketty and Saez rank order and compare tax returns to isolate "the top one percent."  But that is not the same thing as rank-ordering and comparing households.  Piketty and Saez themselves note that even after a small adjustment to add in non-filers, there are 30 percent more tax returns than there are households, and average tax-return income is 25 percent smaller than average household income.  Research by the Federal Reserve Board looking at tax returns in 2000 estimated that the number of tax returns filed exceeded the number of tax returns filed by household heads and their partners by 25 percent. 

There are two reasons for the discrepancy.  First, unmarried couples (and many married couples) file separate tax returns, as do teens with summer jobs and college kids on work-study. You can see why this might be a problem: the "top 1 percent" of tax returns ends up being a bigger group than the top 1 percent of households because the bottom "99 percent" is padded with these extra people.  That's fine as far as it goes, and if the ratio of tax returns to households hasn't risen appreciably, then the trend might be the same for households even if the share of income received by the top one percent of households is overstated by the share received by the top one percent of tax returns.

The other reason that average tax-return income is smaller than average household income is that the IRS data excludes income from nontaxable sources, the most important of which include nontaxable amounts of government benefits and of pensions and employer-provided health insurance. Including these sources of income would tend to lower the share of income going to the top, and it might reduce the increase too.

CBO's figures are theoretically based on households, and they include all public benefits and employer-provided health coverage as income.  But in some sense, the CBO figures are even trickier to interpret than the Piketty/Saez figures.  That's because CBO analysts start with households in the Current Population Survey data, attempt to disaggregate households and their incomes to tax-return-like units, and append actual IRS tax return data from similar-looking tax-return-like units to the disaggregated household data.  They then aggregate the incomes back into households and combine income from the tax return data with income from the household survey.  Finally, they rank people (not households or tax returns) on the basis of incomes adjusted to account for household size differences, but report household incomes and shares going to the top and going to other groups in non-adjusted dollars.

All of this is much less straightforward than going to a household survey and just ordering households by household income to see what you get. The problem is that nearly all household surveys are incapable of giving reliable information about the top 1 percent, simply because in order to sweep up members of, say, the top one percent of the top one percent in a survey, you have to go to great lengths--either interviewing a massive number of people or developing a distinct strategy for finding and interviewing the richest of the rich. Even when massive surveys are available--the census that is taken once every ten years, for instance--because of privacy concerns, the incomes of the very rich are camouflaged. Only one survey has taken the second approach of strategically focusing on the rich separately in collecting income data: the Survey of Consumer Finances (SCF). The best one can do with other household surveys is to make some assumption about how the camouflaged incomes at the top are really distributed and to assign those households new incomes.

I recently analyzed the 1982 and 2006 income data from the SCF, which are the earliest and latest years for which comparable estimates are available. All of the figures in this paragraph compare incomes before taxes that include realized capital gains. The share of the top one percent in the SCF rose from 11 to 21 percent from 1982 to 2006, which is startlingly close to the Piketty/Saez figures (11 to 23 percent) and the CBO pre-tax figures (10 to 19 percent).  That is despite the fact that the income levels in the data sets differ notably. In the SCF, the entry point to the top one percent was $317,500 in 1982 (expressed in terms of what that income would have purchased in 2010).  It rose to $698,700 by 2006 -- an increase of 120 percent. In the Piketty and Saez data those numbers are $206,600 and $407,100 -- significantly lower, as expected, but the 97 percent increase is similar to that in the SCF. In the CBO data, the entry point to the top one percent is defined in terms of size-adjusted income, so we can't compare it to the others.  But the increase in the size-adjusted threshold was 113 percent. Even the percent of real income gains that accrued to the top one percent from 1982 to 2006 is similar across the data sets: 45 percent in the SCF, 53 percent in Piketty/Saez, and 39 percent in the CBO data.

In short, despite the shortcomings of the IRS-based top-one-percent measures in terms of units of analysis and income definitions, the basic conclusions about the rise in inequality hold up awfully well using the very different -- and in some ways preferable -- SCF. So why can't I learn to stop worrying and love the top one percent inequality estimates? One last post on trends in high-end inequality, and then I'll turn to low-end inequality.

Guest post by Gabriel Rossman -- Sociologist at UCLA.  His work applies economic sociology to media industries. He blogs at Code and Culture and is the author of Climbing the Charts.

[Since our hostess requested that I talk a bit about my forthcoming book during my guest-blogging stint I'm posting an excerpt describing the two fundamental patterns through which things get popular.]

This book's substantive concern of how songs become hits on the radio is part of a more general class of problems in social science known as the diffusion of innovation. This literature covers a wide variety of substantive areas where actors within a population each decide if and when to adopt an innovation. The seminal studies in this field were about such eclectic phenomena as:

• farmers in Iowa planting a new kind of corn 
• firms in heavy industry adopting various new production technologies 
• small-town doctors prescribing the antibiotic tetracycline 
• postwar households purchasing such appliances as televisions and washing machines

The innovations described in the literature range from drastic changes that reorder the actor's cultural and economic experience to fairly minor variations on incumbent practices for which "innovation" is perhaps too grandiose a term. In current sociology, one of the main applications of diffusion analysis is asking such questions as when firms adopt new business practices or how activists adopt new tactics.

At the most basic level, one can study diffusion simply by drawing a graph and looking at its shape to see whether it is more concave or more s-shaped. The graph shows typical curves of each ideal type. The shape of the graph is informative because different processes create differently shaped graphs; thus, seeing the shape of the graph gives very strong clues as to the process that created it. In a diffusion graph the x-axis is time, which can be denominated in whatever unit is appropriate. Many of the canonical studies measure time in years, but tetracycline spread in a matter of months, and pop songs usually spread even faster. The y-axis is how popular the innovation is at a particular time. Usually the y-axis is cumulative, showing how many actors have adopted the innovation to date, though sometimes they are plotted as instantaneous, showing how many actors are adopting in each period.

This implies that diffusion is about seeing how many actors adopt the innovation in each period, and it is, but this can be misleading. The reason is that it's quite a different thing for a hundred out of a thousand to adopt than for a hundred out of a hundred. The number of actors who have yet to adopt as of a time is the "risk pool," and the proportion of the risk pool who adopt in a time interval is the "hazard" rate. For a given hazard, the raw number of adoptions decreases as the risk pool shrinks. This is a case of Zeno's paradox, in which fleet- footed Achilles races a tortoise but allows the reptile a head start. If in each minute he closes half the remaining distance, then after the first minute he will have closed 1/2 the distance, after the second minute, 3/4 of the initial gap, then 7/8, 15/16, 31/32, etc. Returning to diffusion, imagine that a thousand doctors have a hazard rate of 10 percent for adopting tetracycline. In the first month 100 doctors (a tenth of 1,000) will write their first prescriptions for tetracycline; in the second month 90 will adopt, for a total of 190 doctors prescribing it; in the third month 81 will adopt, for a total of 271, and so on. In this example the hazard remains constant at one-tenth per month. Therefore, the proportion of the risk pool converted in each period is the same, but the raw volume decreases rapidly. This results in the concave-shaped curve labeled constant hazard" in the graph, which shows rapid growth initially and asymptotically limited growth thereafter.

So far we have assumed that the hazard is constant. This may be warranted if we imagine that there is some constant force acting in the population and encouraging actors to adopt the innovation, such as a marketing campaign with a fixed budget. For this reason these curves are often known as "external influence" in that the innovation is being spread by something outside of the population adopting it. However, imagine that the innovation is spread as an endogenous process within the population, perhaps by word of mouth. This might be because there is no marketing budget or because the actors simply don't trust advertisements or salesmen to provide impartial advice. For instance, imagine that farmers are deciding to plant a new type of maize that presents higher risk but offers higher reward. Most farmers are hesitant to make so radical a change, but one farmer is willing to experiment with the seed and, on seeing his higher crop yields, he tells two neighbors about his satisfactory experience and they try it. After their own satisfactory experiences they in turn each tell two others. If each person using the corn tells two new neighbors about it, then one farmer will plant it in the first year, three in the second, nine in the fourth, twenty-seven in the fifth, eighty-one in the sixth, and so on. This pattern shows slow diffusion at first, but follows exponential growth so that once the innovation reaches a critical mass of the population, it diffuses rapidly.

Of course there are a finite number of farmers, so the exponential growth can not continue forever. Once the innovation starts to become popular, many of the people who one might tell about it are in fact already using it, placing exponential growth for the hazard in tension with Zeno's paradox for the risk pool. Contagious diffusion can only occur when someone who has experienced the innovation encounters someone who has not. Diffusion is slow early on because there are too few adopters who can promote the innovation (a low hazard), and it is slow later on because there are so few potential adopters remaining (a small risk pool), but in the middle lies a "tipping point" of intense diffusion where many people are promoting the innovation to many who have yet to adopt it (a high hazard and large risk pool). The resulting graph is the s-shaped curve shown in the graph and labeled as "endogenous hazard."

Although the example of internal influence described above relies on direct word-of-mouth contagion, the same implications apply to "threshold" or "cascade" models where potential adopters are aware of how many others have adopted the innovation but don't directly communicate with them. For instance, many people who don't make a habit of smashing property and assaulting people on the street will nonetheless join in a sufficiently large riot because safety in numbers means they need be much less afraid of punishment than if they were alone to misbehave. In this model it doesn't matter whether the rioters directly communicate with each other, only that potential rioters have a sense of how large the riot has become. Although in the riot example the potential rioter is directly estimating the size of the mob, this miasmic sort of diffusion is often mediated by things like best- seller lists or website download counts that aggregate and make salient information on popularity. So you may be more likely to buy a book when it becomes a best-seller because the book's popularity gives it more conspicuous placement in bookstores, even if you don't personally know a single individual who has read the book or have even observed strangers reading the book in public.

Thus, we have two distinct patterns for how an innovation might diffuse across a population. In the second style, the proportion of holdouts who adopt in each period is determined by how many actors are already using the innovation. Because the hazard rate is a function of prior adoptions, this is an endogenous pattern or an "internal-influence" cycle. In contrast, in the first style a constant proportion of holdouts adopt in every period. Because a constant proportion cannot be a function of how many people have already adopted, it can be interpreted as reflecting an "external-influence" on the system, or an "exogenous" pattern. Of course these patterns are ideal-typical and real cases can approximate one or the other or even a compromise between them. For instance, the diffusion of tetracycline was mostly exogenous, the diffusion of hybrid corn almost perfectly endogenous, and the diffusion of postwar consumer appliances a compromise between the two patterns. Much of the literature brackets this issue of how different types of innovations spread and instead focus on a single innovation and then ask which actors adopted that innovation particularly early. However, in this book I emphasize the question of the nature of diffusion itself and focus on the question of under what circumstances songs follow the concave curve and under which circumstances they follow the s-curve. This is the type of question that can not be answered by studying a single innovation's diffusion history, but only in comparing those of many innovations, and seeing under what circumstances an innovation's trajectory will follow one path or the other. Such an endeavor requires data on many innovations, and this is a role for which radio singles are well-suited for they occur in such numbers, spread so rapidly, and are so well-documented as to serve the purposes of sociology as admirably as the fruit fly does for those of genetics.

More »

Guest post by Jim Manzi, founder and Chairman of Applied Predictive Technologies, and the author of Uncontrolled: The Surprising Payoff of Trial-and-Error for Business, Politics and Society.

A few months ago, I was asked to give a talk to the MIT Enterprise Forum in London on the rules for executing a successful new start-up company. Having listened to more than one such bloviation session myself, I began with some caveats.

First, even though every guy who has done a successful start-up somehow feels he's therefore become the philosopher-king of business, all experience is bounded. Any observations I make apply to venture-backed enterprise software targeting scale-up. Many of the things you would do for a biotech start-up or a consumer-oriented social media business, as examples, are probably very different. Further, there are companies that exist to sell products at a profit, and companies that exist to sell equity to investors. I only know about the former. The latter tend to flourish in the later stages of a bubble, and rely on a totally different set of skills related to promotions, networking and PR.

Second, even within the universe of relevant companies, all "rules for success" are either obvious or incomplete. Each suggestion in this post will be incomplete, in that it will ignore inevitable exceptions and complications. In other words, there are no rules for success. If there were, lots more people would do successful start-ups.

I'll divide these observations into those relevant for each phase of getting from a new business idea through about the second or third year of operations. After that (if you are the unusual start-up that makes it that far), things become quite different again. Caveat emptor.

IN THE BEGINNING..

You are selling a dream to prospective investors, employees and customers. Be ruthlessly honest with yourself at all times.

Have a co-founder. You're getting married to this person, so make sure you trust him or her, have great mutual respect, and can speak openly at all times. Drama is your enemy. Ideally, you should have high overlap in your view of the world, and only partial overlap in your skill sets.

Seek blue water. Do something innovative enough that nobody else is even trying. This is the best way to get around scale advantages that others have over you. Since you don't know what you're doing yet, it's better if nobody else does either.

Ignore capital markets and industry analysts until you are ready to exploit them; they are rear-view mirrors. If you are doing at start-up phase what the capital markets say is valuable, you're usually too late. You have to do what they say is stupid now (or are just not thinking about now).

In several years, you will either be proven wrong (in which case, they will say "told you so"), or you will be proven right (in which case, they will shamelessly ignore their prior opinions). It's nothing personal; it's just their business model.

Plans are basically useless, but a small amount of planning before you start can be valuable. A good business plan is done in Excel, not Word:

-- See if you can (A) make a list of the companies that might buy your offer, (B) estimate what each would pay for it. Multiply A by B. This, not "the CRM market" or whatever, is how to think about market size.

-- Guess at how many people at what comp level by person you would need to provide some early version of this offer. Gross up the headcount by a simple factor for rent and other costs, plus any other very expensive capital equipment you need to make a simple cash flow statement.

-- Figure two years of this cost run rate with no revenue, then some time with revenue at a loss, to get to break-even. This cumulative total is your capital requirement. Double it. This is still probably an under-estimate.

-- See if it looks like revenue might be greater than cost at some kind of a rough, early steady-state.

-- Always assume an average amount of luck in the long-run, and terrible luck in the short-run.

FUNDING IT

Fund it yourself for as long as you can stand it. Use your own money. Get some kind of revenue coming in the door to offset costs partially, but at least as important, to demonstrate that this is a business, not something on a cocktail napkin. However, don't make the mistake of becoming a consultant doing fee-for-service work, and not really become a technology company that can scale.

The more you are able to go to VCs with a business, rather than an idea, the less equity you will give up. This difference in A-round dilution is massive. Early dilution is the lever that moves the world. If you really are pursuing blue water, the slower time to market will be worth the trade-off.

Venture capitalists only provide money. I had a very good relationship with my VCs. I wasn't looking for a friend, strategy adviser or corporate recruiter. They gave me money, and I gave them an exit with a very good IRR. If Board members and investors can know better than you what should be done when they spend 1 - 2 days per month at the company, then you shouldn't be the CEO.

Always retain operational control. Your world will be complicated enough as it is. Keep things simple internally.

Don't trade equity for third-party services. It's so tempting to do this when you're cash-constrained, but you will regret it later.

10 TIPS FOR THE NEXT FEW YEARS

(1) Don't fantasize about your prestigious Board members, or partnerships, or being acquired, or anything else saving you. These can be very useful tools, but unless you win the lottery, they will help on the margin. You're on your own.

(2) Spend every nickel like it was your last one ...

(3) except for people. The team with the smartest guys usually wins. Pursue people quality beyond the point of apparent irrationality.

(4) Org charts, budgets and plans are mostly a waste of time at this stage.

(5) If you can't describe somebody's job to your mother, find something useful for this person to do.

(6) Run the business as if you are going to own 100% of the shares forever and live off the dividends.

(7) Ignore supposed competitors, whether incumbents or start-ups; ignore external advice; ignore time to market; ignore people like me; ignore capital markets. Focus on delivering value to customers at a foreseeable profit.

(8) Treat revenue and (especially) profit are the best possible feedback on your ideas.

(9) To a first-order engineering approximation, the only financial metric that matters is Free Cash Flow. Cash flow breakeven is the most important milestone your company will ever achieve. Your whole world becomes totally different when this happens. This will not necessarily be the view of your VC investors. Remember that they have chips all over the roulette table. You are the chip on 17 Red.

(10) If successful, the company is likely to end up focused on something different (and usually narrower) than you first thought. The trick is that this focus will only be discovered after 6 - 18 months in the market. This transition will be brutal. You need to be decisive about it, and some of the people there at the beginning won't make it.

My prediction for your new venture: Pain.

More »

There is no word for "cord cutter" in Dothraki
 

Scott Winship, Brookings Institution. Follow him on Twitter: @swinshi

Thanks in part to Occupy Wall Street, when people talk about inequality these days, they're typically referring to the extent to which the top 1 percent have pulled away from the bottom 99 percent.  I have previously expressed skepticism not about whether this has happened but about the magnitude of the increase in high-end inequality.  Over time, my skepticism has eroded, though I still believe that interpreting the figures that are commonly cited is complicated.  I'll use this post to start taking you behind the numbers you see all over the place about the top 1 percent.

The first thing to point out is that if marketing were no issue, a case could be made that Occupy Wall Street's slogan should be, "We Are the 99.5%!" The following chart shows that if you are in the top 10 percent of incomes, you command more of the income received in the U.S. today than your counterparts in the past.  But unless you are in the top one-half of one percent, the increase has not been startling.  If you look at the actual income shares that are behind the numbers in the chart, tax returns in the top one percent but not in the top one-half of the top one percent accounted for 3 percent of income in 1960 and 4 percent in 2010.  If you were in the top five percent but not the top one percent, your share increased from 12.5 percent of income to 16 percent.  These are not increases to inspire urban camping.

If you are in the top one-half of one percent, however, your peeps take home over twice the share of income that they would have in 1960--16 percent of all income received instead of 7 percent.  It gets more and more striking as you look at more stratospherically rich groups, but the trends tend to follow a similar pattern once you leave the bottom 99.5 percent behind.

 

These figures come from the research of Thomas Piketty and Emmanuel Saez and are based on IRS tax return data.  The Congressional Budget Office puts out its own estimates that are partly based on IRS data, combined with data from the household survey used for official unemployment figures (the "Current Population Survey," or CPS).  Like the Piketty and Saez figures I cite above, capital gains are included in their income estimates.  CBO, however, includes other sources of income that they cannot and estimates taxes for each household.  It also ranks people based on household income (adjusted to account for household size differences) rather than ranking tax returns based on the income they report.

CBO finds that the share of income received by the top one percent rose from 8 percent to 17 percent from 1979 to 2007 (the Piketty/Saez increase is from 10 percent to 24 percent).  It also finds that this "share" was taken from each of the four bottom fifths of the income distribution (but not from the other people in the top fifth).  When one sees the share of income received by the top rising and the share received by the bottom 80 percent falling, it is natural to think in zero-sum terms and to assume that the gains at the top must have come at the expense of the bottom.  This is a complicated issue, but what is not complicated is that if the economic pie grows enough, the top can take a bigger piece of it even as everyone gets more pie.  That is what has happened.  CBO reports that the bottom fifth of households saw their incomes increase by nearly 20 percent over this period, while the rest of the bottom 80 percent saw its income rise by nearly 40 percent.  To be sure, the income of the top one percent nearly quadrupled.  But it is still the case that everyone else is a lot better off in 2007 than in 1979.  (Hold your fire if you want to tell me that incomes haven't risen by much relative to the postwar era--which is true--or that they've not kept up with productivity gains--which is not.  I'll address these topics in future posts.)

There are several potentially important issues with these numbers, which is why I was skeptical of them for a while.  I remain somewhat wary of using them in certain ways, but in my next post I'll describe the analyses that have convinced me that skyrocketing inequality at the top is as real as these numbers indicate.

Scott Winship, Brookings Institution. Follow him on Twitter: @swinshi

Income inequality remains remarkably prominent in political and policy debates nearly four-and-a-half years since the start of the Great Recession and less than six months from the consequential 2012 elections. To flesh out this otherwise trite observation, consider what Lexis Nexus says about how often "inequality" has appeared in the national paper of record, The New York Times. During the three years from September 2008 -- when the collapse of Lehman Brothers ushered in the financial crisis -- through August 2011, the Times published one inequality piece every 1.8 days.  I wouldn't read too much into this number. I suspect that a more-careful analysis that honed in on prominent articles and columns that were clearly about economic inequality would produce a significantly smaller estimate.

The important thing here is the trend.  Before the financial crisis, from the start of 2001 through August of 2008, the Times published an article including "inequality" once every 2.1 days, up from once every 3.2 days during the 1990s and once every 4.8 days over the 1980s.  In other words, inequality became an increasingly visible topic in the Times over the long run, a rise that mirrored the increase in inequality over this period.

Beginning in September of last year, however, when Occupy Wall Street penetrated into the public consciousness and when President Obama began adopting a more populist and combative rhetoric and politics, inequality has been a constant presence in the news.  From September of last year through last week, the Times included pieces on inequality not once every 1.8 days or even once per day, but 1.5 times per day.  If one includes the Times's blogs, that figure goes up to 2.9 times per day.  Again, these levels probably overstate how much the Times has run prominent articles on inequality, but since last September, it has published nearly one article or blog post every three days mentioning "inequality" at least five times.*

I'll spend some of my time in my stint as guest blogger the next couple of weeks exploring the topic of inequality--what to make of the basic trends and whether they should generate as much attention and concern as they currently do.  I'll also focus on trends in other economic indicators, drawing from analyses I've been doing for the past six years.  It was around that time that I was inspired by the work of Jacob Hacker (in ways he surely didn't intend) and of Steve Rose (whose book you should buy) to start looking more deeply into the numbers that are waged as weapons of convenience in daily political debates.  These debates generally proceed from worldviews to the selective embrace and citation of evidence that reinforces them.  I have to fight off confirmation bias just like everyone else, but I'll try to at least tell you what the alternate arguments are and why I disagree with them.

If you think that economic problems are generally overstated, as I do, you pretty quickly get labeled a conservative. Ideology isn't a terrible thing in and of itself -- moral values related to freedom, equality, fairness, and justice do tend to cluster together in coherent ways that differentiate liberals and conservatives. I worry more about government inefficiency and unintended consequences and less about market failure compared with most liberals, but my views on justice and inequalities of opportunity align more closely with liberals than with conservatives. 

But my values and yours should be irrelevant when it comes to establishing what the available data says.  It's unfair to peg people ideologically based on their read of evidence. There's better and worse evidence, but we have to do the work of assessing evidence, not make ideological judgments based on the conclusions of this or that commentator.  (For those without the time or background to assess empirical claims, a good indicator of seriousness is the extent to which someone addresses the evidence behind counterarguments rather than casting aspersion on the service to which counterarguments are being put.)

What the ideally collected and measured data would say is more contentious and leaves more room for bias to creep in, but it is not difficult to find and assess the evidence cited by people with whom one disagrees.  It is reasonable to categorize people ideologically when they won't do this, won't change their views, and won't rigorously explain why they won't.  But pegging people simply based on their conclusions says at least as much about the ideology of the person doing the pegging than about the person doing the concluding.

*Times-haters and stats sticklers: I know, I may be saying more about how much the Times cares about inequality than how much a representative sample of newspapers does.  This is just a quick-and-dirty for illustration of the broader point that inequality is much newsier than it has been in the past. If the Wall St. Journal cooperated with Lexis Nexus, I'd have looked at the figures for them too. If someone else wants to fight with Factiva, I'd be curious to see the results.

Jim Manzi is founder and Chairman of Applied Predictive Technologies, and the author of Uncontrolled: The Surprising Payoff of Trial-and-Error for Business, Politics and Society.

I played in a lot of party bands in college, and there was a common expression among the soul and funk musicians I knew: "Never follow Aretha." In other words, it's rarely wise to walk onto a stage to perform right after the audience has just seen a virtuoso. Megan was very kind to invite me to help fill in here; but she was also pretty cruel, in that it will be natural to compare my writing to hers. It won't measure up. But at least she'll be back soon.

You are Megan's audience, so I'll try to stick to the kinds of economic topics that you come here to find. One of my co-bloggers for the week, Scott Winship, is a leading scholar on inequality and social mobility, so I'll start on that topic.

There is extensive blogospheric commentary right now about the article in the May 1st New York Times Magazine in which retired Bain Capital executive Edward Conard puts forward the view that current American income inequality is positive evidence that the market is correctly rewarding the innovators who drive economic growth. According to the article, he believes that America needs more technological innovation, and hence more income inequality, to motivate "art-history majors" and lawyers to become entrepreneurs.

I'm a technology entrepreneur. I think that Conard has an important point, but I also think that he is looking at one side of a trade-off. I argue in my book that America does need more technology-based innovation, and that this will likely require an even greater orientation toward markets and market-like mechanisms to accomplish. However, I think that this is likely to exacerbate social problems, and that those on the short end of the stick will not only react as Conard predicts by feeling "compelled to try to join" the successful entrepreneurial class, but also by attacking the innovation process.

Here's how I tried to put the problem concretely in one passage of Uncontrolled, after describing how the company I founded was one of a number that independently stumbled into the creation of Software-as-a-Service, or "cloud computing," as a method for delivering enterprise software:

Many entrepreneurs hold the opinion that "I did it all on my own," which may be well adapted to leadership success in certain situations, but it is objectively myopic. The entrepreneur relies on an ecosystem of venture capitalists, risk-taking purchasers, and so on. This ecosystem itself rests on a deeper foundation of collective, government-led enterprise. The delivery of our software, for example, depended on the existence of the Internet, which is the product of a series of government-sponsored R&D efforts, in combination with subsequent massive private commercial development. Government funding has been essential to much of the university science that entrepreneurs have exploited. Honest courts and police are required for functioning capital markets and protection of assets; physical infrastructure is required for the roads and running water without which we would not spend much time thinking about artificial intelligence software. At the absolute foundation, national armed forces protect the whole system against external aggression. All of our exciting technical and economic innovations ultimately require men to stand watch all night looking through Starlight scopes mounted on assault rifles--and die if necessary--to protect our commercial, law-bound society. Would you do this to protect a billionaire hedge-fund manager who sees his country as nothing more than lines on a map?

The vessel of the broader society must survive if social evolution of any kind is to take place within it.

Innovation forces change, while humans generally resist change. The pain of the change tends to be visible, while the benefits are usually diffuse and invisible, and clearly involve luck when they are visible. It is therefore natural that people will attempt to organize to prevent the spread of innovation. The original Luddites were English cotton weavers who responded directly to their displacement by automated weaving technology: they smashed looms. This is easy to mock ... as long as you're not the one who is going hungry. In the contemporary West, such people rarely assault property en masse; instead they form political coalitions to pass laws that restrict the use of the looms. Threats to cohesion are not met with violent overthrow of the government, but with use of the political process, broadly defined, to slow down disruptive innovation.

Thus the inherent tension built into the very structure of innovation manifests itself as the fundamental tension of democratic capitalism: winners in this scenario require shared resources produced by the losers. That is, the market economy requires broad social consent. Why should those who lose out in market competition give it?

People are as they are, and to pretend that they will willingly conform to some abstract notion of a pure market is to wish away human nature. It is a libertarian version of the old Soviet attempt to will the New Man into existence.

The whole third section of my book is an attempt to lay out this problem as clearly as I can, and then to propose some modest programs to alleviate it, very partially. I don't think there are even partial solutions that fit on a bumper sticker.

(h/t to David Frum who makes a similar point, very well and very succinctly)

More »

Thanks to Garrett Jones and Dave Ryan for a great two weeks of blogging. Today we have three new, brilliant bloggers for you:

Gabriel Rossman is a sociology professor at UCLA whose work focuses on mass media and the diffusion of ideas. I had the honor of blurbing his forthcoming book, which I hope he'll blog about.

Jim Manzi is an entrepreneur, writer, and all-around genius whose fantastic new book, Uncontrolled, covers the use of randomized controlled trials in discovering the truth -- especially, the truth about public policy.

Scott Winship of the Brookings Institution is one of my favorite scholars on the topic of income inequality and income mobility.  His work has inflected my opinions on the subject, and hopefully, will influence yours as well.

Be nice to them. I miss you all.

Garett Jones -- Economist at George Mason University.  Follow him on Twitter: @GarettJones

One of the major schools of thought in macroeconomics rarely makes it into mainstream discussions: Real Business Cycle Theory. 

RBC, as it is known, claims that a lot of year-to-year economic volatility is caused by changes to the supply side of the economy: perhaps tax and regulatory changes, bad weather in farm economies, spikes in oil prices, and above all, the mysterious force known as the "technology shock."  Finn Kydland and Ed Prescott shared a Nobel, partly for creating RBC theory.

Let me tell the basic story of RBC and technology shocks using a story famously ridiculed by Larry Summers: Robinson Crusoe is on his island, toiling away alone.  He spends some effort on catching fish to eat (consuming) and some time making nets (investing, building capital).  When a storm hits, what does he do?  Well, it's a bad time to catch fish and a bad time to make nets.  But a person's got to eat, so what little time he does spend working he spends catching fish. 

If you watched Crusoe from a distance for a few years, what would you see? You'd see boom times, when the weather is good: High output, high consumption, lots of work, high investment in nets. And during stormy weather, the opposite: Low output, low consumption, low work, low investment. 

Comovement reigns on the island, and it's all driven by Crusoe's rational reaction to changing weather.  That's just what we see in actual business cycles: People don't switch from consuming to investing in a boom, they do more of everything.

So you can see where I'm going: A change in the weather really is a technology shock.  But can this possibly be relevant for explaining the boom-bust cycle in real-world economies?  Not always, for sure, but I'm no monocauser: Monetary policy seems to matter a lot, as Friedman/Schwartz and Romer/Romer  argued; and other business cycle stories abound. 

But I'd like to know whether good ideas come in clusters and whether these waves of innovation (positive technology shocks) are big enough to move the whole economy: And when these waves do come along, do they usually boost output, hours of work, and investment?  When the waves go away, do hours fall?  And are the effects big enough to matter, or do they just become background noise in an economy as big as the US?

Michelle Alexopoulous looked into this in two papers (one coauthored), one of which appeared in the prestigious American Economic Review.  She looked at technical books: When a lot of new books are published in a promising technical field, does that predict a boom?  And what about reverse causation: Even if books predict booms, is that just because people publish technical books when the economy is already doing well?

She found that books really do predict booms. In her paper looking at new books from 1955-1997, she found that new technical books predicted between 1/6 and 1/5 of all medium-term changes in business capital investment.  Total GDP and (to a more modest extent) hours of work moved together with new tech books, usually with a lag of a couple of years. 

Further, she found that a good economy didn't predict more tech books, and a bad economy didn't predict fewer.  So reverse causation isn't the story. 

Finally, as a placebo, she checked to see whether years when lots of history books were published tended to precede economic booms. They didn't. Alexopoulos made a good effort of kicking the tires on this hypothesis.  And remember: She only looked at technical books: There are surely a lot of other new ideas in fields like management, biotech, and accounting that matter for business productivity, and they also seem to come in waves. 

What does this mean for rich-country economies?  It means that real-world technology waves are big enough to shift the whole economy within a few years: Investment, hours, GDP, all moving together. Technology shocks can create real Real Business Cycles.  

Normally, macroeconomists assume that technological change matters for the long run--big ideas diffuse over a generation, too slowly to matter for the boom-bust cycle.  But a glance at the business media will demonstrate that executives and managers are constantly in the hunt for new ideas, the next big thing.  Key players always want that first-mover advantage. The race to be first--so obvious from the rise of the Internet economy -- pushes up the demand for excellent labor and new capital when a promising idea arises. 

And the downside is grim as well. Innovation droughts usually mean stagnation, great or small.  Less work, less investment, less output.  

If I taught a class on technological progress, I'd surely assign a Vonnegut novel, perhaps Cat's Cradle. Here's Vonnegut, putting words into the mouth of an executive at a private research lab:

"Nothing is generous. New knowledge is a valuable commodity." 

RBC takes Vonnegut's idea and uses it to explain some of the big shifts we see in real-world economies.  It's usually hard to fit RBC into partisan political narratives, which helps explain why we don't hear about it much.  But as long as big ideas come in waves, as long as energy supplies depend on the vagaries of global politics, and as long as politicians enact policies that weaken confidence in the health of a nation's economic institutions, RBC will matter.