Larry Solum's critique of J.B.'s hierarchy scores two crucial points. First, the very "idea of such a list, while amusing, assumes that there is a meaningful hierarchy." Second, J.B.'s "categories are conceptually odd." I attribute both of these implied faults to one flawed assumption underlying J.B.'s project: the belief that value of scholarship arises from the cost associated with this creation.
Although some of J.B.'s categories are distinguished by quality (such as the distinction between ordinary doctrinal scholarship and doctrinal scholarship on "interesting" questions), the overall arc of his progression from blog posts to empirical scholarship appears to rest on J.B.'s evaluation of the difficulty of different types of scholarship:
0, 1 Blog posts 2, 3, 4 Doctrinal scholarship 5, 6 Normative policy analysis 7 Legal theory 8 Interdisciplinary studies 9, 10 Empirical scholarship
Barbie's right: Empirical scholarship is hard!
Value ≠ CostA considerable amount of misery in law and in economics flows from the evidently natural tendency to equate value with cost. In the language of the law of regulated industries, for instance, J.B. has implicitly embraced cost-of-service ratemaking principles, even though real markets and astutely designed approaches to deregulation favor value-of-service ratemaking. (For diehards who want the technical details, I offer an overview in this paper and a more comprehensive critique in this paper, which explains how Greg Sidak and Christopher Yoo, notwithstanding their great intelligence, are irredeemably wrong about pricing rules.) It's intuitively appealing, but ultimately -- and sometimes gravely -- wrong to equate value with cost.
Consider the classic case, Federal Power Commission v. Hope Natural Gas Co., 320 U.S. 591 (1944) (discussed at greater length in this forum's lamentation on "Disaster's dead hand"). Hope upheld a ratemaking order based on the historic cost of natural gas extraction operations. Good constitutional law, terrible economics. What Justice Jackson said in his Hope dissent applies not just to natural gas production, but to the entire world of economic enterprise: the "service one renders to society ... is measured by what" the producer "gets out of the ground, not by what he [or she] puts into it." 320 U.S. at 649 (Jackson, J., dissenting). In other words, "there is little more relation between the investment and the results than in a game of poker." Id.; cf. Steven Lubet, Lawyers' Poker: 52 Lessons Lawyers Can Learn from Card Players (2006).
A proper hierarchy of legal scholarship, therefore, demands attention not merely to its cost (as measured by the effort invested in it) but also to its value. What Hope Natural Gas said of ratemaking makes sense here: "It is not theory but the impact of [scholarship] which counts." 320 U.S. at 605.
How then should we measure scholarly impact or value? Over at MoneyLaw, another team of bloggers to which I belong has plumbed the virtues and vices of empirical measures such as citations and SSRN downloads. With respect to the grandest questions of scholarly merit, I suspect that those measures, even if perfected, would fail to capture the overall sense of value at stake. For the time being, then, I propose an alternate hierarchy based loosely on the value ascribed to legal scholarship by its potential audiences:
One final note: Mathematically inclined readers will notice that J.B.'s hierarchy, as I reduced it above, and my hiearchy can both be measured on a six-point scale. It is therefore possible to combine J.B.'s cost-based hierarchy and my value-based hierarchy into a game played with dice. May all of us hope that our scholarship comes up boxcars every time.