Some Elaborations on J.B. Ruhl's Hierarchy of Legal Scholarship (by J.B. Ruhl)
A few days ago I posted a Hierarchy of Legal Scholarship and, unlike my usual practice, I forgot to place my byline at the beginning of the post, which appears to have led some readers to conclude Jim Chen posted it. I'm glad Jim has clarified that, and I hope the way I've titled this post avoids and further confusion. I have a few thoughts based on comments to my original post and on Jim's alternative.
First of all, the underlying rationale for the hierarchy is not cost of effort; rather it is the value added, using the average associate at a top law firm as the baseline. In other words, what can law professors supply the world that top-100 law firm associates usually could not. Indeed, a law professor might be very good and very efficient at churning out high quality legal theory or empirical work, so the cost is low to him or her. My interest is in the value added. Doctrinal work is low on my list because any good associate can, and largely does, produce that all day long. Although I did not explain this rationale in a separate sidebar, it is the theme that I use and mention throughout the list.
Maybe a thought exercise will help:
This is why I put empirical work high on the list. Yes, empirical work is also costly, but it is the type of work that a good scholar, because of the nature of the job and the demands of that kind of work, will be in a better position to provide. One of the comments to my original post suggested that by "empirical work" I mean just compiling numbers. I should clarify that what I mean is (as suggested by another comment) empirical work that tests data against theory, the way real social scientists do it. The reason I put it higher on the list than theory work, therefore, is because it puts the theory to the test. High quality empirical work of this sort is costly, but it is valuable because it illuminates flaws and strengths of the theory work.
To be sure, I recognize that scholarship fitting any particular category will fall on a range of quality, and we might use the categories Jim suggested to differentiate. Excellent doctrinal work is more valuable than half-baked legal theory. I think of my list as defining a typology of legal scholarship based on what law professors can add in terms of value to legal development and understanding. It is, in that sense, only a starting point for evaluation of any particular piece of work. There have already been, in addition to Jim's suggestions for different sets of criteria, and I am not holding mine out as the only one that has some usefulness by any means.
Lastly, talk of a hierarchy of scholarship is more difficult in law than it is in other disciplines, I suspect, because law enjoys the luxury of publication without peer review. But even in law, many members of hiring committees and promotion committees surely do apply some sense of what makes some kinds of scholarship more valuable than others as a general proposition. Whether they use mine, Jim's, or another, they are using one. So I think it is healthy to have a conversation about how we do it.
First of all, the underlying rationale for the hierarchy is not cost of effort; rather it is the value added, using the average associate at a top law firm as the baseline. In other words, what can law professors supply the world that top-100 law firm associates usually could not. Indeed, a law professor might be very good and very efficient at churning out high quality legal theory or empirical work, so the cost is low to him or her. My interest is in the value added. Doctrinal work is low on my list because any good associate can, and largely does, produce that all day long. Although I did not explain this rationale in a separate sidebar, it is the theme that I use and mention throughout the list.Maybe a thought exercise will help:
- First, gather 100 randomly selected law professors to select the top 10 examples of legal scholarship in each of my 10 categories (clearly, this is a thought exercise).
- Next, capture each of my categories in a short assignment, such as from "compile the law of X" to "develop a theory of the law of X and conduct empirical studies to test it in ways relevant to all disciplines interested in X."
- Assign the assignments to 1000 randomly selected 4th year associates from the American Lawyer top 100 firms.
- When their work product is returned, have the panel of law professors evaluate on average how it compares in each category to the "top 10" selected in step 1.
This is why I put empirical work high on the list. Yes, empirical work is also costly, but it is the type of work that a good scholar, because of the nature of the job and the demands of that kind of work, will be in a better position to provide. One of the comments to my original post suggested that by "empirical work" I mean just compiling numbers. I should clarify that what I mean is (as suggested by another comment) empirical work that tests data against theory, the way real social scientists do it. The reason I put it higher on the list than theory work, therefore, is because it puts the theory to the test. High quality empirical work of this sort is costly, but it is valuable because it illuminates flaws and strengths of the theory work.
To be sure, I recognize that scholarship fitting any particular category will fall on a range of quality, and we might use the categories Jim suggested to differentiate. Excellent doctrinal work is more valuable than half-baked legal theory. I think of my list as defining a typology of legal scholarship based on what law professors can add in terms of value to legal development and understanding. It is, in that sense, only a starting point for evaluation of any particular piece of work. There have already been, in addition to Jim's suggestions for different sets of criteria, and I am not holding mine out as the only one that has some usefulness by any means.
Lastly, talk of a hierarchy of scholarship is more difficult in law than it is in other disciplines, I suspect, because law enjoys the luxury of publication without peer review. But even in law, many members of hiring committees and promotion committees surely do apply some sense of what makes some kinds of scholarship more valuable than others as a general proposition. Whether they use mine, Jim's, or another, they are using one. So I think it is healthy to have a conversation about how we do it.
posted by Civis Americae at 2:04 PM
1 Comments:
Empirical work, if done correctly, means developing and testing hypotheses about human and organizational behavior. Good empirical scholarship, therefore, is quite difficult to do and bad empirical work suffers from a host of limitations and challenges. The main difficulty with getting it right in empirical scholarship is ensuring that you don't generalize from the particular and that you avoid "going native" (or its opposite) when you set out to study an organization. Thus, as I have tried to study hurricane response in Gulf Coast, it's particularly easy to start with the notion that Michael Brown's FEMA was a failure, based on a number of observations and conversations with people who suffered from Katrina. More difficult, however, is to study systemic causes and effects and then suggest ways of making change. Those of us who study High Reliability Organizations know that good system design is a key to system performance. I am tempted to say that legal systems are no exception, but in truth, good legal system design is absolutely necessary to effective and reliable organizational performance. At least from the HRO (and Law and Society) perspective, the goal of empirical scholarship is to learn something about the world in action and then, from this learning, to begin to make the argument for change.
Thus, studying FEMA’s response to Katrina can be informed by its responses to Gustav and Ike. When you start doing that, you realize that it’s all too easy to blame a scapegoat – and then argue about failures at the federal level and the changes that should flow from that – but that digging deep into the inter-jurisdictional challenges and legal mechanisms that move organizations from Mission-driven to Compliance-driven, the various reimbursement requirements, the contractual relationships between government and private parties, over-reliance on technological systems sold by private contractors for whom there is little contractual punishment for failure, the cascading effects of Louisiana emergency response decisions on surrounding states – including those that are not impacted directly by the storm, but by the influx of Louisiana residents into their states, and federal benefits programs – Medicare, Individual Assistance Declarations, and FEMA reimbursement request tracking systems.
If legal theory is not grounded in some amount of experience, it may speak to inapt concepts. Thus, we may be spending far too much time speaking about FEMA in disaster response (and all the corresponding federalism questions) and far too little time in speaking of the VOAD community – which seems to be in a far better position not only to deliver the "last mile" of disaster relief aid but also more able to do so while paying attention to the particular human element of it. If we were to focus on VOADs we would spend less time talking about federalism and more time talking about developing an appropriate contract model that engages non-profits and avoids the utter failure of the private bus delivery contracts during Gustav.
I can’t really speak to the hierarchy of legal scholarship but it does seem that doctrinal questions have their place. Though, in truth they may best be left to those who are in practice – or through “client alerts” and the like. It does seem that legal scholarship should be engaged with the world and though the Law and Society movement has made great progress in bringing social scientific methods to the study of law, there is still a great deal of work to be done.
http://www.bus.lsu.edu/directory/Intranet_Directory/facultywebpage.asp?logonid=jegan
Post a Comment
<< Home