Critique of "No Basis" Part One: Their Appalling Double-Standards

Virtually all peer-reviewed academic research on same-sex parenting has come to one conclusion: there’s no evidence that being raised by same-sex parents harms children in any way. This result, which has been replicated in one form or another at least fifty times, drives sexists and homophobes up a wall. If for children, being loved and taken care of by two parents is what matters, then the cherished conservative belief that children “need” parents of both sexes for healthy development is unsupportable. Furthermore, the research undercuts the myth that children need protection from queers (a major plank of the anti-same sex marriage platform).

No Basis by Robert Lerner and Althea Nagai is frequently cited by anti-gay activists to argue against this body of research (and, by a logically dubious implication, same-sex marriage). Christianity Today’s take on No Basis is pretty typical:

Researchers Robert Lerner and Althea Nagai, coauthors of No Basis: What the Studies Don’t Tell Us About Same-Sex Parenting […] evaluated 49 studies on gay parenting, finding significant mistakes in all of them.

They particularly criticized “convenience sampling,” in which investigators select whoever is available, and “snowball sampling,” in which homosexual activists help researchers find volunteers willing to answer questions.

“These studies prove nothing,” Lerner and Nagai wrote.

From No Basis itself:

Do these 49 studies offer conclusive proof that there is “no difference” between heterosexual and homosexual households? We believe that these studies offer no basis for that conclusion…because they are so deeply flawed pieces of research. The reader is invited to make his or her own judgment.

Lerner and Nagai claim that studies of same-sex parenting don’t meet minimum standards of scientific respectability. But are the standards they put forward ones they genuinely believe in, or are they standards that Lerner and Nagai opportunistically take on for the specific purpose of rejecting same-sex parenting studies? (It is perhaps worth noting that No Basis was commissioned by The Marriage Law Project, an organization formed to oppose same-sex marriage). One way of answering this question is to see if Lerner and Nagai have held their own research to the rigorous standards they insist are mandatory in No Basis.

Before No Basis, Lerner was probably best known for a 1996 study, published by a right-wing think tank in the wake of the O.J. verdict, which claimed to show that American juries typically treat black defendants more gently than white defendants (the disadvantage of whites compared to blacks is a frequent theme in Lerner’s research). Although the study was never published in a peer-reviewed journal, Lerner’s spectacular findings – in particular, his claim that juries convict white rape defendants twice as often as black rape defendants – created a stir in the mainstream press. From U.S. News and World Report (Oct 14 1996):

Sociologist Robert Lerner, who wrote the report, speculates on the reasons. “Maybe blacks really are getting off easier,” he says, through the leniency of mostly black juries. But it’s also possible, he adds, that “the criminal justice system is a dragnet”–catching countless blacks in its wake–and “then the subsequent process acts as a sieve,” screening out the innocent. That, says Lerner, would be good news. “It suggests that one part of the system appears to be working the way we’d like it to work.”

The study was widely derided by academics (among other things, it ignored the disparity in sentencing, an area in which the judicial process is clearly easier on whites compared to blacks). For our purposes, what’s interesting is that this study flunks the standards advocated in No Basis. As the ACLU comments,

To reach this conclusion Lerner looked at a mere five jury trials involving black defendants. (Roger Parloff, “Speaking of Junk Science,” The American Lawyer, January 1997.) This is the same man who dismisses a study of over two-dozen gay parents for having an insufficient sample size.

Lerner and his associates thought a sample size of five was solid enough to trumpet to the national press; but samples many times larger are still too small, according to Lerner, when he needs an excuse to dismiss gay parenting studies.

What about Lerner and Nagai’s other standards? In No Basis, a major objection to many studies of same-sex parenting is the use of non-probability samples, and in particular “snowball” sampling, in which participants recruit other participants. Here’s a passage of recommendations from No Basis:

1) Use probability samples. There i s no substitute. Only these offer any basis for scientific generalization to larger, representative populations.

2) Ignore studies based on non-probability samples…

3) Especially ignore studies where participants recruit other participants. These are so subject to bias, that the limited results cannot be trusted.

That’s some very strong language. So, surely, this is a standard that Lerner and Nagai genuinely believe in – not just an opportunistic standard they’ve taken on to bash gay parenting studies? To answer that question, I’ll quote from a review of Lerner and Nagai’s book American Elites (the review was published in the prestigious American Journal of Sociology, September 1997):

The samples can only be described as conceptually dubious and methodologically unsound. […] The methodology could not pass a first-year research methods course. No standard set of procedures were used in drawing the 12 elite samples. Something approaching stratified random sampling was used to draw several of the elite samples, but the business sample was drawn exclusively from seven corporations. Top-ranking bureaucrats were purposely sampled to draw equal numbers from “activist” and “traditional” agencies. The sample of religious leaders was collected using snowball methods, which somehow failed to qualify any Jews or Muslims as religious leaders.

(Emphasis added). Again, it’s clear that Lerner and Nagai have altered their conceptions of what is and isn’t acceptable methodology.

These are by no means unique examples – see, for instance, this Lerner and Nagai study of affirmative action. Although Lerner and Nagai argue in No Basis that conclusions can never be drawn without extremely rigorous statistical controls or tests of significance, they didn’t bother using any such statistical tests here. Instead, Lerner and Nagai present only percentages, an approach they single out for harsh criticism in No Basis. Note as well that their study seems to have included only 37 black students – a sample size they’d deride as far too small in No Basis.

You may be now saying to yourself, “so Lerner and Nagai use the same bad methods that the gay parenting studies do. They’re still bad methods, right?”

To that I’ll say: Have patience, folks. I’ll get there.

Today, I’ve shown that Lerner and Nagai are not serious about the standards they used to reject gay-parenting studies in No Basis, as demonstrated by the fact that they’ve never taken these standards seriously in their own work. Tomorrow, I’ll show that – setting aside Lerner and Nagai’s double-standards – the standards they use to dismiss gay parenting studies are illogical, misapplied, and show a severe misunderstanding of social science norms and standards.

This entry posted in Families structures, divorce, etc, Same-Sex Marriage. Bookmark the permalink. 

13 Responses to Critique of "No Basis" Part One: Their Appalling Double-Standards

  1. Pingback: Web Dictionary - review-Words/review-PukiWiki

  2. 2
    Mikhaela says:

    Thanks for the excellent analysis of that bogus anti-gay parent crap, Amp! In a much less analytical vein, I did a cartoon on the tongue-in-cheek “no republican adoption” bill. I should probably note that I of course don’t believe in any such law–anything that can help kids in foster care be adopted by good parents of any party or orientation is good by me.

  3. 3
    RonF says:

    One way of answering this question is to see if Lerner and Nagai have held their own research to the rigorous standards they insist are mandatory in No Basis.

    The point is not how Lerner and Nagai have conducted their own research. The point is whether or not the issues they raise are valid regarding the studies they discuss. What is the general consensus in the scientific community regarding Lerner and Nagai’s issues?

  4. 4
    mangala says:

    What is the general consensus in the scientific community regarding Lerner and Nagai’s issues?

    As a student, I can’t speak for the psychological community, but my development professors have thus far all agreed that the evidence shows that two parents are better than one, for a variety of reasons, when it comes to socialization, but the gender of the parents does not seem to matter. My feeling would be that over 49 studies, a convergence of evidence pointing to the same results (outcomes for children of same-sex parents are generally the same as for children of heterosexual parents, and where there is a difference, it generally favours the children in same-sex-parent families), there are reasonable grounds for saying that there is no difference. Of course, having just read an extremely up-to-date literature on the effects of family diversity which reached the same conclusion, I may be a bit biased myself.

    I would also say that there are some cases where in order to get enough participants for a reasonable sample size, truly random polling may be impossible. For example, when I worked in a lab recruiting participants for research into early cognitive development, I worked from a list of new parents who’d submitted their names to be participants. The people who actually came in to participate were largely affluent and highly educated. This is a biased sample, but it was the most efficient use of limited resources (i.e. busy undergrad volunteers!). Other researchers in the lab had tried to use ads, but the response for recruiting parents with young children was so poor that it wasn’t worth the cost of putting them up.

    When you’re trying to get participants from a relatively small group – such as gay parents – I would say that putting up ads or making random phone calls would be a highly inefficient way to recruit participants, so getting participants to suggest other participants seems reasonable. (And I doubt that most psychology labs maintain a list of gay parents to contact for research purposes…) This may not be ideal, but this is why it’s important that results be replicated.

  5. Pingback: Tariqat

  6. 5
    Josh Jasper says:

    One wonders how many actual same sex parents Lerner and Nagai bothered to get to know.

    Research is important, but there’s a technical term for a sociologist who thinks he or she can make judgements based on nothing more than data points. It’s called “fucking asshole”.

  7. 6
    Lab Kat says:

    It’s the difference between conducting an actual scientific study and one with a hidden agenda.

    *shakes head*

  8. 7
    PDXNAG says:

    I am still waiting for the study that links male sterility to poor parenting skills, with, of course, a prerequisite finding that there is even a sufficient nexus to even examine the issue. And, the corollary argument that non-sterility is unassailably conclusive of quality parenting skills.

  9. 8
    spit says:

    mangala — Soc and psych stuff not being my particular realm other than a personal interest, I also wonder if the “snowballing” — participants finding other participants — would even present a clear problem in this sort of study. You’d be likely, I’d think, to get a subset of folks from similar socio-economic backgrounds and the like, and while this would certainly limit the breadth of the picture you’d get, it would also mean that you’ve to some degree controlled for other factors than queerness that may be involved in family dynamics, right?

    Not even to mention the number of studies or their convergence, which is certainly a big pointer to me that they’re at least finding something consistent.

    Dunno, I’m merely a chem geek. People aren’t quite like molecules, though the similarities are sometimes striking.

  10. We all assume these guys are hypocritical douche-bags so forget the ad hominem rabble-rousing stuff and bring on the real rebuttal!

    I look forward to it.

  11. 10
    Michael says:

    Given that Lerner and Nagai (as well as Nock and Wilcox) appear to be social-scientists-for-hire when it comes to finding socially conservative outcomes, it seems all of their results are in doubt. Bias is clearly a two-way street and there are obviously researchers who are biased on the other side, but we should at least try to be honest about the biases (and funding) of the researchers.

  12. Amp, excellent job. Looking forward to the rest.

  13. Pingback: All the Answers? | Reasonable Conversation