No. 11-345

In the Supreme Court of the United States

ABIGAIL NOEL FISHER,

PETITIONER

v.

UNIVERSITY OF TEXAS AT AUSTIN, ET AL.

ON WRIT OF CERTIORARI

TO THE UNITED STATES COURT OF APPEALS

FOR THE FIFTH CIRCUIT

BRIEF OF EMPIRICAL SCHOLARS

AS AMICI CURIAE

IN SUPPORT OF RESPONDENTS

HARRY M. REASONER	THOMAS S. LEATHERBURY
VINSON & ELKINS LLP	Counsel of Record
First City Tower	KIMBERLY R. MCCOY
1001 Fannin St.,	VINSON & ELKINS LLP
Suite 2500	Trammell Crow Center
Houston, TX 77002	2001 Ross Ave.,
(713) 758-2222	Suite 3700
	Dallas, TX 75201
	(214) 220-7700
	tleatherbury@velaw.com

[Additional Counsel Listed On Inside Cover]

ERIC A. WHITE VINSON & ELKINS LLP

2200 Pennsylvania Ave., NW, Suite 500 West

Washington, DC 20037 (202) 639-6500

II
TABLE OF AUTHORITIES
Cases:	Page(s)
Grutter v. Bollinger,
539 U.S. 328 (2003)........................................	7, 9, 10
Daubert v. Merrill Dow Pharm.,
509 U.S. 579 (1993)................................................	16
Regents of Univ. of Cal. v. Bakke,
438 U.S. 265 (1978).................................................	9
Miscellaneous:
Sigal Alon & Marta Tienda, Assessing the
“Mismatch” Hypothesis: Differences in
College Graduation Rates by Institutional
Selectivity, 78 Soc. Educ. 294 (2005).....................	14
William G. Bowen et. al, Crossing the Finish
Line: Completing College at America’s
Public Universities (Princeton Univ. Press
2011).......................................................................	16
William G. Bowen & Derek Bok, The Shape
of the River (Princeton Univ. Press 2000).............	16
Ian Ayres & Richard Brooks, Does Affirmative
Action Reduce the Number of Black
Lawyers?, 57 Stan. L. Rev. 1807 (2005) ................	12
Gregory Camilli et al., The Mismatch
Hypotheses in Law School Admissions,
2 Widener J.L. Econ. & Race 165 (2011) ..............	12
David L. Chambers et al., The Real Impact
of Eliminating Affirmative Action in
American Law Schools: An Empirical
Critique of Richard Sander’s Study,
57 Stan. L. Rev. 1855 (2005) .............................	9, 12

III
Miscellaneous—Continued:	Page(s)
Kalena E. Cortes, Do Bans on Affirmative
Action Hurt Minority Students? Evidence
from the Texas 10% Plan, 29 Econ. of
Educ. Rev. 1110 (2010) ..........................................	14
Stacy Berg Dale & Alan B. Krueger,
Estimating the Payoff to Attending a
More Selective College: An Application
of Selection on Observables and
Unobservables, 117 Q.J. of Econ. 1491
(2002)......................................................................	26
Stacy Dale & Alan B. Krueger, Estimating
the Return to College Selectivity over the
Career Using Administrative Earnings
Data (Nat’l Bureau of Econ. Research,
Working Paper No. 17159, June 2011),
available at http://www.nber.org/papers/
w17159 .............................................................	26-27
Michele Landis Dauber, The Big Muddy,
57 Stan L. Rev. 1899 (2005) ..................................	12
Rogers Elliott et al., The Role of Ethnicity in
Choosing and Leaving Science in Highly
Selective Institutions, 37 Res. in Higher
Educ. 681 (1996) ....................................................	10
Mary J. Fischer & Douglas S. Massey, The
Effects of Affirmative Action in Higher
Education, 36 Soc. Sci. Res. 531 (2007) ................	14
Constantine Frankgakis & Donald B. Rubin,
Principal Stratification in Causal
Inference, 58 Biometrics 21 (2002) ........................	23

IV
Miscellaneous—Continued:	Page(s)
Andrew Gelman and Jennifer Hill, Data
Analysis Using Regression and
Multilevel/Hierarchical Models
(Cambridge Univ. Press 2007) ..............................	22
Cheryl I. Harris & William C. Kidder, The
Black Student Mismatch Myth in Legal
Education: The Systemic Flaws in
Richard Sander’s Affirmative Action
Study, J. Blacks Higher Educ. (2005).............	12-13
Daniel E. Ho, Affirmative Action’s
Affirmative Actions: A Reply to Sander,
114 Yale L.J. 2011 (2005) ......................................	13
Daniel E. Ho, Why Affirmative Action Does
Not Cause Black Students to Fail the Bar,
114 Yale L.J. 1997 (2005) ................................	13, 25

Daniel E. Ho et al., Matching as Nonparametric Preprocessing for Reducing Model Dependence

in Parametric Causal Inference,
15 Pol. Analysis 199 (2007) .............................	22-23
Paul W. Holland, Statistics and Causal
Inference, 81 J. Am. Stat. Ass’n 945 (1986) ..........	18
Guido W. Imbens & Donald B. Rubin, Causal
Inference in Statistics and Social Sciences
(forthcoming Cambridge Univ. Press 2012) ...	19, 22
Thomas J. Kane, Racial and Ethnic
Preferences in College Admissions,
59 Ohio St. L.J. 971 (1998)....................................	14
Mark C. Long, College Quality and Early
Adult Outcomes, 27 Econ. of Educ. Rev.
588 (2008).........................................................	14-15

V
Miscellaneous—Continued:	Page(s)
Tatiana Melguizo, Are Students of Color
More Likely to Graduate from College if
They Attend More Selective Institutions?
Evidence from a Cohort of Recipients and
Nonrecipients of the Gates Millennium
Scholarship (GMS) Program, 32 Educ.
Eval. & Pol’y Analysis 230 (2010) .........................		15
Tatiana Melguizo, Quality Matters: Assessing
the Impact of Attending More Selective
Institutions on College Completion Rates of
Minorities, 49 Res. Higher Educ. 214 (2008)		........15
Beverly I. Moran, The Case for Black
Inferiority? What Must Be True if
Professor Sander Is Right: A Response to
A Systemic Analysis of Affirmative Action
in American Law Schools, 5 Conn. Pub.
Int. L.J. 41 (2005)...................................................		13
Angela Onwuachi-Willig & Amber Fricke,
Class, Classes, and Classic Race-Baiting:
What’s in a Definition?, 88 Denv. U.L. Rev.
807 (2011)...............................................................		13
Paul R. Rosenbaum, The Consequences of
Adjustment for a Concomitant Variable
That Has Been Affected by the Treatment,
147 J. Royal Stat. Soc’y Series A (Gen.)
656 (1984).........................................................		22-23
Jesse Rothstein & Albert Yoon, Affirmative
Action in Law School Admissions: What
Do Racial Preferences Do?,
75 U. Chi. L. Rev. 649 (2008) ................................		15

VI

Miscellaneous—Continued: Page(s)

Donald B. Rubin, Bayesian Inference for
Causal Effects: The Role of Randomization,
6 Annals of Stat. 34, 38 (1978) ..............................	18
Donald B. Rubin, Causal Inference Through
Potential Outcomes and Principal
Stratification: Applications to Studies with
‘Censoring’ Due to Death, 21 Stat. Sci. 299
(2006)......................................................................	23
Donald B. Rubin, The Design Versus the
Analysis of Observational Studies for
Causal Effects: Parallels with the Design
of Randomized Trials, 26 Stat. in Med. 20
(2007)......................................................................	19
Donald B. Rubin, Estimating Causal Effects
of Treatments in Randomized and
Nonrandomized Studies, 66 J. Educ.
Psychol. 688 (1974) ................................................	19
Donald B. Rubin, For Objective Causal
Inference, Design Trumps Analysis,
2 Annals Applied Stat. 808 (2008) ........................	19
Richard H. Sander, A Systemic Analysis of
Affirmative Action in American Law
Schools, 57 Stan. L. Rev. 367 (2004).............	passim
Richard H. Sander, Mismeasuring the
Mismatch: A Response to Ho,
114 Yale L.J. 2005 (2005) ................................	20-21

VII
Miscellaneous—Continued:	Page(s)
Richard H. Sander & Jane Yakowitz
Bambauer, The Secret of My Success: How
Status, Prestige and School Performance
Shape Legal Careers (UCLA School of Law
Research Paper No. 10-26, July 2010),
available at http://papers.ssrn.com/sol3/
papers.cfm?abstract_id=1640058..........................	24

Richard Sander & Roger Bolus, Do Credential Gaps in College Reduce the Number of Minority Science Graduates? (Project SEAPHE Working Paper, July 2009), available at http://www.seaphe.org/working-

papers/).............................................................	11-12
Mario L. Small & Christopher Winship,
Black Students’ Graduation from Elite
Colleges: Institutional Characteristics and
Between-Institution Differences,
36 Soc. Sci. Res. 1257 (2007) .................................	15
U.S. News and World Report Best Law Schools,
http://grad-schools.usnews.rankingsand
reviews.com/best-graduate-schools/top-law-
schools/law-rankings (last visited August 9,
2012).......................................................................	21
Linda F. Wightman, User’s Guide: LSAC
National Longitudinal Data File (1999)...............	21
Doug Williams, Does Affirmative Action
Create Educational Mismatches in Law
Schools? (Working Paper, Apr. 2009),
available at http://econ.duke.edu/~hf14/
ERID/ Williams.pdf) ......................................	passim

VIII

Miscellaneous—Continued: Page(s)

Junni Zhang & Donald B. Rubin, Estimation
of Causal Effects via Principal Stratification
When Some Outcomes Are Truncated by
‘Death,’ 28 J. Educ. & Behav. Stat. 353
(2003)......................................................................	23

2

designing and evaluating economic policy interventions in areas such as education and labor. Along with Professor Rubin, he is co-authoring a highly anticipated textbook on principles for causal inference.

Donald B. Rubin is the John L. Loeb Professor of Statistics at Harvard University, where he served as chairman for 13 years of three decades there as full Professor of Statistics. He has authored over 350 publications (including several books), including pioneering work on causal inference in experiments and observational studies. His publications have generated well over 100,000 citations. Rubin is a Fellow of the American Statistical Association, the Institute for Mathematical Statistics, the International Statistical Institute, the Woodrow Wilson Society, the John Simon Guggenheim Society, the New York Academy of Sciences, the American Association for the Advancement of Sciences, the American Academy of Arts and Sciences, and the Alexander von Humboldt Foundation. He is also the recipient of four of the most prestigious awards available to statisticians: the Samuel S. Wilks Medal of the American Statistical Association, the Parzen Prize for Statistical Innovation, the Fisher Lectureship, and the George W. Snedecor Award of the Committee of Presidents of Statistical Societies. He is an Elected Member of the U.S. National Academy of Sciences and an Elected Fellow of the British Academy. Furthermore, he is the recipient of many other awards and honors, such as an Honorary Doctorate from the Faculty of Social Sciences and Economics, Otto Friedrich University, Bamberg Germany.

3

Gary King is the Albert J. Weatherhead III University Professor at Harvard University—one of only 24 with the title of University Professor, Harvard’s most distinguished faculty position. He is based in the Department of Government and serves as Director of the Institute for Quantitative Social Science. He has been elected Fellow in six honorary societies (National Academy of Sciences 2010, American Statistical Association 2009, American Association for the Advancement of Science 2004, American Academy of Arts and Sciences 1998, Society for Political Methodology 2008, and American Academy of Political and Social Science 2004) and has won more than 30 “best of” awards for his work, including leading contributions to the statistics of causal inference. He was listed as the most cited political scientist of his cohort. His research has been supported by the National Science Foundation, the Centers for Disease Control and Prevention, the World Health Organization, the National Institute of Aging, the Global Forum for Health Research, as well as numerous centers, corporations, foundations, and other federal agencies.

Richard A. Berk is a Professor of Statistics and Criminology at the University of Pennsylvania, where he has appointments in the Department of Statistics in the Wharton School and in the Department of Criminology in the School of Arts and Sciences. He was previously a Distinguished Professor of Statistics at UCLA, a Professor of Statistics and Sociology at UC Santa Barbara, and a Professor of Sociology at Northwestern University. He has held visiting appointments at the École Normale Supérieure in Paris in the Department of Earth, Atmosphere and

4

Oceans, and with the Statistics Group of the Los Alamos National Laboratories. He has authored over 190 publications (among them 15 books), including highly respected work on causal inference, regression analysis, machine learning, and forecasting. Professor Berk has been elected to the Sociological Research Association, and is an Elected Fellow to the American Association for the Advancement of Science, the American Statistical Association, and the Academy of Experimental Criminology. He has received the Paul S. Lazarsfeld Award for methodological contributions from the American Sociological Association. He is a founding editor of The Evaluation Review, a journal of applied social research. Professor Berk received his BA from Yale University and his Ph.D. from The Johns Hopkins University.

Daniel E. Ho is a Professor of Law and the Robert E. Paradise Faculty Fellow for Excellence in Teaching and Research at Stanford Law School, where he teaches statistics and administrative law. His scholarship centers on quantitative empirical legal studies, and he holds a J.D. from Yale Law School and a Ph.D. in political science from Harvard University. He was the Maurice R. Greenberg Visiting Professor of Law at Yale Law School and a Visiting Professor of Law at New York University School of Law. His work has been supported by the National Science Foundation, and his work has been awarded numerous prizes, including the Warren Miller Prize (for an article on causal inference), awarded to the best paper published in Political Analysis. He currently serves as President of the Society for Empirical Legal Studies.

5

Kevin M. Quinn is a Professor of Law at the UC Berkeley School of Law (Boalt Hall). Previously, he served as Associate Professor of Government at Harvard University, and as Assistant Professor of Political Science and Adjunct Assistant Professor of Statistics at the University of Washington. Quinn has written extensively on statistical methodology, and he teaches courses on applied statistics and empirical legal studies. He is a three-time winner of the Gosnell Prize for excellence in political methodology and his research has been supported by the National Science Foundation. He currently serves as an Associate Editor for the Journal of the American Statistical Association.

D. James “Jim” Greiner is a Professor of Law at Harvard Law School. He holds a J.D. from the University of Michigan Law School and a Ph.D. in Statistics from Harvard University. His research focuses on the development of rigorous quantitative methods, with a particular focus on causal inference in observational and experimental settings. His work has been widely published in law reviews (such as the Harvard Law Review and the Yale Law Journal), as well as in peer-reviewed statistics journals (the Review of Economics and Statistics, the Journal of the Royal Statistical Association, and the Annals of Applied Statistics). He teaches courses on civil procedure, voting rights, and expert witnesses.

Ian Ayres is the William K. Townsend Professor at Yale Law School, the Anne Urowsky Professorial Fellow in Law, and a Professor at Yale’s School of Management. Ian has published 11 books and over 100 articles on a wide range of topics, including leading work in empirical legal studies. In 2006, he

6

was elected to the American Academy of Arts and Sciences. Professor Ayres has been ranked as one of the most prolific and most-cited law professors of his generation. He holds a J.D. from Yale Law School and a Ph.D in economics from M.I.T. He previously taught at Harvard, Illinois, Northwestern, Stanford, and Virginia law schools and served as a research fellow of the American Bar Foundation. From 2002 to 2009, he was the editor of the Journal of Law, Economics and Organization.

Richard Brooks is the Leighton Homer Surbeck Professor of Law at Yale Law School. His scholarship centers on law and economics, often involving empirical components. He holds a J.D. from the University of Chicago and a Ph.D. in economics from UC Berkeley. He previously taught at Cornell University and Northwestern University.

Paul Oyer is the Fred. H. Merrill Professor of Economics at Stanford University’s Graduate School of Business, where he teaches the core Human Resources Management class in the MBA program as well as a Ph.D. class in Personnel Economics. Before moving to Stanford in 2000, he was on the faculty of the Kellogg School of Management at Northwestern University. In his pre-academic life, he worked for the management consulting firm of Booz, Allen, and Hamilton, as well as for 3Com Corporation and ASK Computer Systems. He holds a BA in math and computer science from Middlebury College, an MBA from Yale University, and an MA and Ph.D. in economics from Princeton University.

Richard Lempert is the Eric Stein Distinguished University Professor of Law & Sociology, emeritus, at

7

the University of Michigan. He is the former Chair of the University of Michigan’s Department of Sociology and a past president of the Law and Society Association. He holds a J.D. and Ph.D. (sociology) from the University of Michigan. He served as Division Director for the Social and Economic Sciences at the National Science Foundation and as Chief Scientist in the Human Factors/Behavioral Science Division in the Science and Technology Directorate of the Department of Homeland Security. He also served as Chair of the National Research Council (the research arm of the National Academy of Science) Standing Committee on Law Enforcement and the Administration of Justice (now Committee on Law and Justice). Professor Lempert is an elected member of the American Academy of Arts and Sciences and Secretary of Section K (Sociology, Economics and Political Science) of the American Association for the Advancement of Science. And he has published numerous articles in law reviews and peer-reviewed social science journals, including several articles on research design and the judicial use of statistical evidence.

SUMMARY OF ARGUMENT

In Grutter v. Bollinger, this Court held that a state has a compelling interest in attaining a diverse student body for the benefit of all students, and that this compelling interest justifies the consideration of race as a factor in university admissions. See 539 U.S. 306, 325, 328 (2003). In this, the latest case to consider the constitutionality of affirmative-action admissions policies, Professor Richard H. Sander, along with lawyer and journalist Stuart S. Taylor, Jr., filed a brief amici curiae arguing that social-

8

science research has shown affirmative action to be harmful to minority students. See Brief Amici Curiae for Richard Sander and Stuart Taylor, Jr. in Support of Neither Party (“Sander-Taylor Brief”) 2. According to them, a “growing volume of very careful research, some of it completely unrebutted by dissenting work” has found that affirmative-action practices are not having their intended effect. Id.; see also Brief Amici Curiae of Gail Heriot et al. in Support of Petitioner (“Three Commissioners Brief”) 14 (“The Commissioner Amici are aware of no empirical research that challenges [Sander’s] findings.”).

But, as amici will show, the principal research on which Sander and Taylor rely for their conclusion about the negative effects of affirmative action— Sander’s so-called “mismatch” hypothesis2—is far from “unrebutted.” Sander-Taylor Brief 2. Since Sander first published findings in support of a “mismatch” in 2004, that research has been subjected to wide-ranging criticism. Nor is Sander’s research “very careful.” Id. As some of those critiques discuss in detail, Sander’s research has major methodological flaws—misapplying basic principles of causal inference—that call into doubt his controversial conclusions about affirmative action. The Sander

2 In essence, “mismatch” is said to result when a minority student attends a more selective university than he would have without affirmative action, based upon a “very large” racial preference. The claim is that because the student’s test scores and high school or college grades indicate that he is not as academically qualified to attend the school at which he matriculates as other students, his admission there works to his detriment because “teachers would aim instruction at the median student, and those with weaker preparation would fall behind and learn less.” Sander-Taylor Brief 4.

9

“mismatch” research—and its provocative claim that, on average, minority students admitted through affirmative action would be better off attending less selective colleges and universities—is not good social science.

Sander’s research has “significantly overestimated the costs of affirmative action and failed to demonstrate benefits from ending it.” David L. Chambers et al., The Real Impact of Affirmative Action in American Law Schools: An Empirical Critique of Richard Sander’s Study, 57 Stan. L. Rev. 1855, 1857 (2005). That research, which consists of weak empirical contentions that fail to meet the basic tenets of rigorous social-science research, provides no basis for this Court to revisit longstanding precedent supporting the individualized consideration of race in admissions. Cf. Grutter, 539 U.S. at 334 (“Universities can * * * consider race or ethnicity more flexibly as a ‘plus’ factor in the context of individualized consideration of each and every applicant.”) (citing Regents of Univ. of Cal. v. Bakke, 438 U.S. 265, 315-316 (1978) (opinion of Powell, J.,)). In light of the significant methodological flaws on which it rests, Sander’s research does not constitute credible evidence that affirmative action practices are harmful to minorities, let alone that the diversity rationale at the heart of Grutter is at odds with social science.

ARGUMENT

This Court has held the use of narrowly tailored race-based admissions constitutional in light of the “substantial * * * educational benefits that flow from student body diversity.” Grutter, 539 U.S. at 330. In

10

their brief amici curiae—nominally filed in support of neither party (see generally Sander-Taylor Brief) but nonetheless calling for the judgment of the Fifth Circuit to be reversed, id. at 36—Sander and Taylor argue that “a growing volume of very careful research

* * * suggests that racial preferences in higher education often undermine minority achievement,” id. at 2; accord Three Commissioners Brief 13-14, 20- 21, 24, which in turn purportedly undermines the diversity holding in Grutter, see Sander-Taylor Brief 15-17. But that “growing volume of very careful research” consists mostly of Sander’s own “mismatch” research along with unpublished papers by several others. As recognized by numerous publications refuting Sander’s conclusions, that research contains basic methodological errors, thereby invalidating its conclusions. As a result, its conclusions are without value to the Court.

A. “Mismatch” Hypothesis In Brief

Although Sander was not the first researcher to use the term “mismatch” in discussing the effects of race-based admissions policies,3 he has been rightly credited as a leading proponent of that theory since the publication of his controversial Stanford Law Review article finding “mismatch” in American law schools. See Richard H. Sander, A Systemic Analysis of Affirmative Action in American Law Schools, 57 Stan. L. Rev. 367 (2004). In that article and in later

3 See Thomas Sowell, Black Education: Myths and Tragedies

(David McKay 1972); Rogers Elliott, A. Christopher Strenta, Russell Adair, Michael Matier, and Jannah Scott, The Role of Ethnicity in Choosing and Leaving Science in Highly Selective Institutions, 37 Res. in Higher Educ. 681 (1996).

11

work, Sander argues that when a minority student attends a college or graduate school as a result of race-based admissions, and his academic credentials are substantially below those of his classmates, he does not thrive. For example, as a result of being surrounded by students with stronger academic credentials, a minority student may end up in the bottom of the grade distribution in difficult classes, and end up opting out of difficult majors that he may have chosen otherwise (often called “science mismatch”). Sander-Taylor Brief 6. Similarly, as a result of race-based admissions, a minority student may choose not to pursue graduate-school education (“academic mismatch”), id. at 7-8, or in the case of a law school student, may fail the bar exam (“law school mismatch”), id. at 8; see generally Sander, A Systemic Analysis of Affirmative Action, supra. Ultimately, the mismatch hypothesis holds that affirmative action does more harm than good for minority students, whom Sander and Taylor term the “the intended beneficiaries” (Sander-Taylor Brief 2) of affirmative action. Hence, by curtailing affirmative action, minorities would end up at schools more properly “matched” to their skill sets, and minority academic performance and graduation rates would rise, see id. at 32-35.

Mismatch research is premised on a series of causal inferences. For example, the mismatch hypothesis is that African-American undergraduates have transferred out of rigorous science majors because of mismatch. Id. at 6-7; see also Richard Sander & Roger Bolus, Do Credential Gaps in College Reduce the Number of Minority Science Graduates?

(Project SEAPHE Working Paper, July 2009),

12

available at http://www.seaphe.org/working-papers/). Proponents of this research claim that mismatch at law schools has caused African-Americans to learn less in law school, earn lower grades, fail the bar, and fare worse in employment outcomes. Sander-Taylor Brief 8-9. And, most controversially, Professor Sander contends that affirmative action has had the effect of decreasing the total number of black lawyers. Sander, A Systematic Analysis of Affirmative Action, supra, at 473.

B.Sander’s “Mismatch” Research Does Not Represent A Consensus In Social Science

Since the initial publication of his “mismatch” article, Sander’s work has been subject to wide- ranging criticism for its methodological flaws. See, e.g., Ian Ayres & Richard Brooks, Does Affirmative Action Reduce the Number of Black Lawyers?, 57 Stan. L. Rev. 1807, 1809 (2005) (“[E]ven within his [Sander’s] framework, there is not persuasive evidence indicating that affirmative action is responsible for lowering the number of black attorneys.”); Gregory Camilli et al., The Mismatch Hypotheses in Law School Admissions, 2 Widener J.L. Econ. & Race 165, 207 (2011) (“[T]his study has shown that regression analyses of the kind conducted by Sander are incapable of producing credible estimates of causal effects.”); Chambers et al., supra, at 1857 (“The conclusions in Systemic Analysis rest on a series of statistical errors, oversights, and implausible assumptions.”); Michele Landis Dauber, The Big Muddy, 57 Stan L. Rev. 1899, 1902 (2005) (“Unfortunately, Sander has muddied rather than clarified the waters with a flawed and ultimately misleading contribution.”); Cheryl I. Harris &

13

William C. Kidder, The Black Student Mismatch Myth in Legal Education: The Systemic Flaws in Richard Sander’s Affirmative Action Study, J. Blacks Higher Educ. (2005) (“Regrettably, Sander significantly underestimates the harms of ending affirmative action, and seriously overestimates the benefits of ending affirmative action. Even his own data do not support the mismatch hypothesis.”); Daniel E. Ho, Why Affirmative Action Does Not Cause Black Students to Fail the Bar, 114 Yale L.J. 1997, 1997 (2005) (“[T]he [Sander] study draws internally inconsistent and empirically invalid conclusions about the effects of affirmative action. Correcting the assumptions and testing the hypothesis directly shows that for similarly qualified black students, attending a higher-tier law school has no detectable effect on bar passage rates.”); Daniel E. Ho,

Affirmative Action’s Affirmative Actions: A Reply to Sander, 114 Yale L.J. 2011, 2011 (2005) (“[T]he impressive-sounding points in Sander’s Response violate basic methodological principles and are incorrect.”); Beverly I. Moran, The Case for Black Inferiority? What Must Be True if Professor Sander Is Right: A Response to A Systemic Analysis of Affirmative Action in American Law Schools, 5 Conn. Pub. Int. L.J. 41, 58 (2005) (“In the end, Professor Sander’s arguments fail on their methodology, their logic, and their real-world application.”); Angela Onwuachi-Willig & Amber Fricke, Class, Classes, and Classic Race-Baiting: What’s in a Definition?, 88 Denv. U.L. Rev. 807, 834 (2011) (“[W]e perceive numerous defects in Sander’s methodology that raise serious questions about the results in his article

Class in American Legal Education.”).

14

The hallmark of reliable empirical work is that it can be validated by other researchers. A wide array of social scientists have studied the impact of elite educational institutions on student outcomes, reaching conclusions directly contrary to those of mismatch. See, e.g., Sigal Alon & Marta Tienda,

Assessing the “Mismatch” Hypothesis: Differences in College Graduation Rates by Institutional Selectivity, 78 Soc. Educ. 294, 309 (2005) (“Minority students’ likelihood of graduation increases as the selectivity of the institution attended rises.”); Kalena E. Cortes, Do Bans on Affirmative Action Hurt Minority Students? Evidence from the Texas 10% Plan, 29 Econ. Educ. Rev. 1110, 1122 (2010) (“[R]esults from the analysis run counter to the ‘mismatch’ hypothesis, which would have predicted both higher retention and college graduation rates for these lower-ranked minority students because they are now supposedly being better ‘matched’ to an institution under the Top 10% Plan.”); Mary J. Fischer & Douglas S. Massey,

The Effects of Affirmative Action in Higher Education, 36 Soc. Sci. Res. 531, 544 (2007) (“If anything[,] minority students who benefited from affirmative action earned higher grades and left school at lower rates than others, and they expressed neither greater nor less satisfaction with college life in general.”); Thomas J. Kane, Racial and Ethnic Preferences in College Admissions, 59 Ohio St. L.J. 971, 991 (1998) (“[E]ven if a student’s characteristics are held constant, attendance at a more selective institution is associated with higher earnings and higher college completion rates for minority students as well as white and other non-Hispanic students.”); Mark C. Long, College Quality and Early Adult Outcomes, 27 Econ. Educ. Rev. 588, 589 (2008)

15

(“[C]ollege quality does appear to have positive signi•cant effects on most of the outcomes studied

* * *.”); Tatiana Melguizo, Quality Matters: Assessing the Impact of Attending More Selective Institutions on College Completion Rates of Minorities, 49 Res. Higher Educ. 214, 232 (2008) (“[M]inorities bene•t from attending the most elite institutions.”); Tatiana Melguizo, Are Students of Color More Likely to Graduate from College If They Attend More Selective Institutions? Evidence from the First Cohort of Recipients and Nonrecipients of the Gates Millennium Scholarship (GMS) Program, 32 Educ. Eval. & Pol’y Analysis 230, 244 (2010) (“The results of this study suggest that the probability of attaining a bachelor’s degree increases [for minority students] with the selectivity of the institution attended.”); Jesse Rothstein and Albert Yoon, Affirmative Action in Law School Admissions: What Do Racial Preferences Do?, 75 U. Chi. L. Rev. 649, 707 (2008) (“Even overstating mismatch effects and understating the importance of preferences to enrollment, the effects of eliminating mismatch are dwarfed by the first-order effect of eliminating preferences: the reduction in the number of black students admitted.”); Mario L. Small & Christopher Winship, Black Students’ Graduation from Elite Colleges: Institutional Characteristics and Between-Institution Differences, 36 Soc. Sci. Res. 1257, 1257 (2007) (“[S]electivity improves black probabilities of graduation.”); Doug Williams, Does Affirmative Action Create Educational Mismatches in Law Schools? 42 (Working Paper, Apr. 2009), available at econ.duke.edu/~hf14/ERID/Williams.pdf) (“All of the previous papers that have conducted formal tests of the mismatch hypotheses have concluded that there is no evidence of mismatch

16

effects in the [Bar Passage Study].”); see also William G. Bowen & Derek Bok, The Shape of the River 259 (Princeton Univ. Press 2000) (“[T]he more selective the college attended, the lower the black dropout rate.”); William G. Bowen et. al, Crossing the Finish Line: Completing College at America’s Public Universities 210 (Princeton Univ. Press 2011) (“There is certainly no evidence that black men were ‘harmed’ by going to the more selective universities that chose to admit them. In fact, the evidence available strongly suggests that students in general, including black students, are generally well advised to enroll at the most challenging university that will accept them.”).

In short, those relying on mismatch research mischaracterize the state of social-science evidence and describe a consensus that does not exist. The Court should not rely on the mismatch research.4

For clarity, amici will explain why the research on which mismatch rests is dubious.

C.The “Mismatch” Research Violates Basic Principles Of Causal Inference

The chief empirical research offered by the Sander-Taylor Brief—little of which is peer-reviewed and most of which remains unpublished—violates basic principles of causal inference that are widely accepted in the scientific community. As amici explain, there is no evidence that affirmative action

4 Indeed, were this a district court proceeding, mismatch research should not pass the core Daubert tests of surviving peer review and being generally accepted by experts in the field. See generally Daubert v. Merrill Dow Pharm., 509 U.S. 579 (1993).

17

hurts minority students who attend school under such programs.5 Attending a more elite school does not appear to cause those students harm.

1. The Evidence

Although mismatch has been discussed in a variety of contexts, Sander posits that the law school setting is “uniquely appropriate for studying the mismatch effect,” because, unlike in other higher- education settings, the bar exam is “more or less [a] uniform test[] taken by graduates to measure their legal learning.” Sander-Taylor Brief 8. Much of the cited research has been in the law school context. Amici therefore focus the rest of their arguments on the methodological flaws contained in Sander’s and economics professor Doug Williams’s law-school mismatch research (which dominate the empirical findings of the Sander-Taylor Brief), although the same methodological challenges also affect mismatch research in other settings.

Law-school mismatch alleges that large racial preferences have “seriously damaged the academic performance of black law students, contributing to lower graduation rates and much lower success rates on bar exams.” Id. at 8 (citing Sander, A Systemic

5 This proposition is counterintuitive because it would imply that the very fact of giving minority students extra options, by admitting them to more selective institutions, harms them, and that these students would personally benefit from being prevented from attending such institutions. It rests on the presumption that these students are themselves not good judges of what is in their interest, and that given the option of attending a more selective institution they would fail to make the “right” choice of attending the less selective institution.

18

Analysis of Affirmative Action, supra, at 440-448). Finding that a student’s law school grades have a stronger association with bar passage than the tier of the law school the student attended, Sander concludes that less-qualified students are better off attending a less-selective school where they will perform better and thus be more likely to pass the bar exam. Sander, A Systemic Analysis of Affirmative Action, supra, at 426, 429. The central empirical claim is that going to a higher-tier law school causes less qualified students to learn less, as reflected through lower grades, and decreases bar performance by a greater factor than school quality increases it. See id. at 449-450; Williams, supra, at 9.

2. Research Principles For Causal Inference

A causal effect is the difference between two “potential outcomes.” For example, a law student may have one potential outcome of career trajectory if he attended a higher-tier law school and another potential outcome of career trajectory if he attended a lower-tier law school. The difference between these two potential outcomes is the causal effect of law school tier on that student. The “fundamental problem of causal inference” is that researchers never observe both potential outcomes. See Paul W. Holland, Statistics and Causal Inference, 81 J. Am. Stat. Ass’n 945, 947 (1986); Donald B. Rubin,

Bayesian Inference for Causal Effects: The Role of Randomization, 6 Annals of Stat. 34, 38 (1978). Causal inference thereby always involves estimating the counterfactual outcome with observed data, e.g., how the student at a higher-tier school would have fared had he gone to a lower-tier school. An experiment addresses this problem, by comparing

	19
students who are	similar in pre-existing

characteristics (e.g., ability), but are randomly assigned to different tier schools. Because the two experimental groups would differ only in tier of school attended, differences in the outcomes for the two groups would provide a valid estimate of the causal effect of law school tier. See generally Guido W. Imbens & Donald B. Rubin, Causal Inference in Statistics and Social Sciences (forthcoming Cambridge Univ. Press 2012); Donald B. Rubin, For Objective Causal Inference, Design Trumps Analysis, 2 Annals Applied Stat. 808 (2008); Donald B. Rubin,

The Design Versus the Analysis of Observational Studies for Causal Effects: Parallels with the Design of Randomized Trials, 26 Stat. in Med. 20 (2007); Donald B. Rubin, Estimating Causal Effects of Treatments in Randomized and Nonrandomized Studies, 66 J. Educ. Psychol. 688 (1974).

Although actually conducting such an experiment is obviously infeasible, the experimental model highlights the primary task of research with data in which students have not been randomly assigned. Specifically, to draw a causal inference, researchers should generate (a) comparison groups that are (b) as similar as possible in pre-existing characteristics, so that (c) differences in outcomes can be attributed to the selectivity of the institution. Imbens & Rubin, supra, at ch. 15. In each of these three regards, the Sander / Williams evidence falls short.

3. Methodological Flaws

The Sander empirical evidence consists of “regression analysis” that predicts bar passage for all students that graduated law school with the variables

20

of undergraduate GPA, LSAT score, gender, race, law school tier, and law school GPA. See Sander, A Systemic Analysis of Affirmative Action, supra, at 444-445. The association between law school tier and bar passage is positive, but smaller in absolute magnitude than the association between law school GPA and bar passage. Id. Based on this analysis, Sander concludes that less-qualified students should attend lower-tier law schools where they would presumably achieve higher grades and that affirmative action hurts minority students. Id. at 444 tbl.6.1.

That inference is invalid for three reasons: a. Invalid Comparisons

As to the broad claim about the causal effect of affirmative action, the current analyses are simply uninformative. All the schools in the bar-passage data employ some form of affirmative action. Because there is no comparison group of schools that do not practice affirmative action, no broad inference about the effects of affirmative action can be sustained.6

Moreover, the primary comparison that Sander and Williams employ is that of black and white students. See Richard H. Sander, Mismeasuring the Mismatch: A Response to Ho, 114 Yale L.J. 2005, 2006 (2005) (“The entire [Sander Stanford Law Review] paper is organized around a comparison of

6 The extent of preferential admissions may of course vary by school, and capitalizing on these differences may provide one approach to assess different types of implementations of affirmative-action programs.

21

‘treatment’ blacks * * * and ‘control’ whites * * *.”); Williams, supra, at 18 (“[O]ne approach is to use black as a proxy for being negatively mismatched and white as a proxy for being matched.”). This comparison assumes that black students at selective institutions would have fared similarly to white students at less-selective institutions in the absence of affirmative action. For example, the estimate for how a black student at Yale Law School would have performed at a lower-tier school might be based on a white student at the University of Alabama Law School.7 This comparison violates the principle of

7 Using Yale Law School and the University of Alabama Law School is consistent with the Sander and Williams coding of the first two tiers of law schools, although no specific schools are ever disclosed in the data. The LSAC Bar Passage Study clusters anonymized schools based on factors such as cost, size, selectivity, faculty/student ratio, percent minority, and average LSAT and undergraduate GPA. See Linda F. Wightman, User’s Guide: LSAC National Longitudinal Data File 8 (1999). Sander reorders these clusters by the median entering credentials of white students to create a tier system. Sander, A Systemic Analysis of Affirmative Action, supra, at 416. Sander’s top tier includes 16 schools that “are the most selective and the most expensive” with “the highest UGPAs and LSAT scores.” Wightman, supra, at 16; Sander, A Systemic Analysis of Affirmative Action, supra, at 430. Sander’s second tier includes “14 large, highly selective law schools that enroll student bodies that have UGPAs and LSAT scores that are among the highest in the country.” Wightman, supra, at 16; Sander, A Systemic Analysis of Affirmative Action, supra, at 430. Using the current U.S. News and World Report rankings (not tiers), Yale would likely be a tier-one school and the University of Alabama Law School would likely be a tier-two school under Sander’s coding. See U.S. News and World Report Best Law Schools, http://grad- schools.usnews.rankingsandreviews.com/best-graduate- schools/top-law-schools/law-rankings (last visited August 9, 2012).

22

creating groups that are comparable in all pre- existing respects except for law-school tier. Usual tenets of research design require that a study hold constant pre-existing attributes such as race and gender. See Imbens & Rubin, supra, at ch. 12. By comparing black students at higher-tiered schools with white students at lower-tiered schools, Sander and Williams violate these basic principles.

b. Adjusting For Pre-Existing Characteristics

Proper research design requires that we compare students with similar pre-existing characteristics, who nonetheless attend different law-school tiers. Sander and Williams, however, adjust not only for pre-existing characteristics, but also for outcomes. Specifically, their bar passage analyses (i) hold constant law school graduation (by examining only students that graduated), see Sander, A Systemic Analysis of Affirmative Action, supra, at 444; Williams, supra, at 14, and (ii) control for law school grades, despite the fact that a central component of the mismatch hypothesis is that law school tier affects both law school graduation and grades. Not only is this approach inconsistent with mismatch, but it will also fail to generate valid inferences about the causal effect of law-school tier. Adjusting for outcomes will generally not result in valid estimates of causal effects. See Andrew Gelman and Jennifer Hill, Data Analysis Using Regression and Multilevel/Hierarchical Models 188-190 (Cambridge Univ. Press, 2007); Imbens & Rubin, supra, at ch. 12; Daniel E. Ho et al., Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference, 15 Pol. Analysis 199, 202 (2007); Paul R. Rosenbaum, The Consequences of

23

Adjustment for a Concomitant Variable That Has Been Affected by the Treatment, 147 J. Royal Sta. Soc’y Series A (Gen.) 656 (1984).8

Suppose, for example, that we conducted the ideal experiment, randomizing 200 students to attend selective and less-selective institutions. Assume that the results are that 95 out of 100 graduate at the selective institution, and that 70 pass the bar exam. Meanwhile, at the less selective institution, 80 out of 100 graduate, and 60 pass the bar. That experiment suggests that students benefit from attending a selective institution, both in terms of graduation and bar passage. If we focus only on those who graduated, however, the bar passage rate is 0.74 at the selective institution (70/95), and 0.75 at the less selective institution (60/80). Referring to these findings would lead a reader to wrongly infer that a more selective law school harms students. The selective law school, possibly via improved teaching and better resources, manages to graduate more students. But the subsets of students graduating from either school are not fully comparable.

Adjusting for such outcomes (rather than pre-

8 Principled methods for addressing these issues exist. For accounting for intermediate outcomes, see Constantine Frankgakis & Donald B. Rubin, Principal Stratification in Causal Inference, 58 Biometrics 21 (2002). For accounting for law school students that do not graduate, see Junni Zhang & Donald B. Rubin, Estimation of Causal Effects via Principal Stratification When Some Outcomes Are Truncated by ‘Death,’ 28 J. Educ. & Behav. Stat. 353 (2003); Donald B. Rubin, Causal Inference Through Potential Outcomes and Principal Stratification: Applications to Studies with ‘Censoring’ Due to Death, 21 Stat. Sci. 299 (2006).

24

existing characteristics), as the Sander studies do, contaminates inferences about the causal effect of law-school tier. Indeed, controlling for graduation and grades leads Sander to claim that there is no economic return to attending an elite law school at all, regardless of ethnicity. See Richard H. Sander & Jane Yakowitz Bambauer, The Secret of My Success: How Status, Prestige and School Performance Shape Legal Careers (UCLA School of Law Research Paper No. 10-26, July 2010), available at http://papers.ssrn.com/sol3/papers.cfm?abstract_id= 1640058 (finding that law-school grades are more highly correlated than law-school tier in a regression of economic returns and that therefore elite law schools have no causal effect on income).

c. The Effect Of Law-School Tier

The credibility of a causal inference depends on the credibility of the assumptions. One natural way forward with the bar-passage data is to compare students with identical observed pre-existing characteristics (i.e., undergraduate GPA scores, LSAT scores, race, and gender) who attend different law-school tiers. The critical assumption is that holding constant these factors, there are no other systematic differences between students in the different law-school tiers. In other words, female black students with GPAs of 3.8 and LSAT scores of 168, some of whom attend Yale Law School and some of whom attend the University of Alabama Law School, are comparable, and any systematic differences in outcomes between those attending Yale versus Alabama are due to the difference in the law school attended. The existence of important unobserved differences between these students that

25

affect bar performance invalidates the estimates. If the Yale Law student, for example, is already predisposed to taking the bar in a jurisdiction with a tougher exam such as California or New York, the assumption is violated, and the researcher would draw an inappropriate inference about the effect of law-school tier.

Research that applies these principles has not found any substantially and statistically significant effects on bar passage.9 See Ho, Why Affirmative Action Does Not Cause Black Students to Fail the Bar, supra, at 2002-2004. Taking into account these principles of research design, there is simply no evidence of the harms of mismatch suggested by the Sander-Taylor Brief.

D.Better-Designed Studies Contradict “Mismatch”

It is possible to avoid the rather basic methodological problems underlying the mismatch research. To that end, amici draw this Court’s attention to examples of recent research that employ better-conceived research designs with observational data.

Stacy Dale and Alan Krueger have employed careful methodology in two papers examining the return from college selectivity over a student’s

9 Without proper research design, causal-effect estimates are biased, and conventional tests for statistical significance (and confidence intervals) do not address that bias. Put differently, the fact that a result is “statistically significant” does not overcome the first-order issues of research design that amici highlight.

26

subsequent career. The first study was published a decade ago, Estimating the Payoff to Attending a More Selective College: An Application of Selection on Observables and Unobservables, 117 Q.J. Econ. 1491 (2002). In 2011, Dale and Krueger returned to the topic in Estimating the Return to College Selectivity over the Career Using Administrative Earnings Data

(Nat’l Bureau of Econ. Research Working Paper No. 17159, June 2011), available at http://www.nber.org/ papers/w17159.

In the recent work, they utilize tax data to examine the earnings of students who attended college in both 1976 and 1989. The Dale and Krueger model considers the college the student attended, conventional characteristics, a plausible measure of what is typically unobserved (ability and motivation), and the monetary payoff of attending a more selective college against the student’s actual earnings. See Dale & Krueger 2011, supra, at 5-8.

In both studies, Dale and Krueger examine characteristics that are commonly used as proxies for college quality (average SAT score, the Barron’s index, and net tuition). They also adjust for certain “unobservable factors” by using a “self-revelation model.” This model assumes that students signal their potential ability, motivation, and ambition (typically unobserved) by the choice of schools to which they apply. Dale and Krueger estimate, somewhat surprisingly, that better colleges generally do not increase earnings for either the 1976 and 1989 cohort of students. Id. at 25. They find, however, “[n]otable exceptions * * * for racial and ethnic minorities (black and Hispanic students) and for students whose parents have relatively little

27

education; for these subgroups, our estimates remain large, even in models that adjust for [typically] unobserved student characteristics.” Id. at 5. In other words, the Dale and Krueger research suggests—contrary to mismatch—that attending more selective colleges may result in an increase in minority earnings.

They conclude their paper by mentioning the caveats of their work, most notably that their analysis does not pertain to a nationally representative sample of schools and that their selection-adjusted model is imprecisely estimated. Id. at 25. Nowhere do Dale and Krueger control for outcomes, and their analysis holds constant minority status. Unlike Sander’s mismatch research, Dale and Krueger exercise appropriate caution in making far- ranging conclusions from a limited set of data.

* * * * *

Whether one finds Sander’s conclusions highly unlikely or intuitively appealing, his “mismatch” research fails to satisfy the basic standards of good empirical social-science research. The Sander-Taylor Brief misrepresents the acceptance of his hypothesis in the social-science community and, ultimately, the validity of mismatch. Numerous examples exist of better ways to perform the type of research Sander undertook. See Part D, supra. Sander’s failure to set up proper controls to test his hypothesis and his reliance on a number of contradictory assumptions lead him to draw unwarranted causal inferences. At a minimum, these basic research flaws call into question the conclusions of that research.

28

CONCLUSION

In light of the many methodological problems with the underlying research, amici curiae respectfully request that the Court reject Sander’s “mismatch” research discussed in the Brief Amici Curiae for Richard Sander and Stuart Taylor, Jr.

Respectfully submitted,

HARRY M. REASONER	THOMAS S. LEATHERBURY
VINSON & ELKINS LLP	Counsel of Record
First City Tower	KIMBERLY R. MCCOY
1001 Fannin St.,	VINSON & ELKINS LLP
Suite 2500	Trammell Crow Center
Houston, TX 77002	2001 Ross Ave.,
(713) 758-2222	Suite 3700
	Dallas, TX 75201
	(214) 220-7700
	tleatherbury@velaw.com
	ERIC A. WHITE
	VINSON & ELKINS LLP
	2200 Pennsylvania Ave.,
	NW, Suite 500 West
	Washington, DC 20037
	(202) 639-6500

Counsel for Amici Curiae

AUGUST 2012