Yet Another Koss Critic Can't Get His Facts Right

[Crossposted on “Alas” and on “TADA.” Non-feminists aren’t allowed to leave comments on this post on “Alas,” but anyone can leave a comment on “TADA.”]

Jim Hopper makes a good point about Mary Koss’ rape prevalence research:1

Major publications like The New York Times Magazine have given cover-story treatment to people who have minimal understanding of social science methodology, and apparently even less interest in the truth about rates of abuse and assault in our country. These people have claimed that researchers “make up” abuse that never happened by labeling subjects’ experiences as abusive even though the subjects might not.

This charge has been made against Mary Koss, an accomplished researcher who has conducted studies on prevalence rates of rape among college women (and has found that one in four have experienced rape or attempted rape since age 14). In constructing her questionnaire items, Koss made a good faith effort to use language that fit the legal definition of rape in the state where she lived when she conducted the research. Yet she has been accused of irresponsibly mislabeling her subjects’ experiences and exaggerating rates of rape. […]

One way that Koss has answered this critique is by referring to an analogous situation. I will paraphrase her argument. Imagine yourself questioning an alcoholic: Do you have more than six alcoholic drinks in one sitting several times a week? Yes. Do you often wake up with such a hangover that you can’t go to work? Yes. Have friends and family members repeatedly tried to help you stop drinking? Yes. Do you suffer from withdrawal symptoms when you stop drinking? Yes. Are you an alcoholic? No.

The point here is that good prevalence research must use behavioral descriptions to which definitions like “alcoholic” or “sexual abuse” may be applied. Researchers should not rely on people defining themselves as alcoholics or defining their sexual experiences as abusive. Such definitions can only be uninterpretable and unreliable.

The Damned Olde Man (TDOM) disagrees with Dr. Hopper. Like so many Koss critics, TDOM is unable to get his facts straight:

For instance, many of Koss’ questions began “Have you ever given in (to some sexual behavior) when you didn’t want to…” (1985, p. 167). In other words, have you ever consented to sex without consenting to sex? Koss states that this meets the legal definition for rape in her state.

Koss doesn’t state that — because those questions weren’t used by Koss to measure rape prevalence.

Two of the ten questions Koss used to measure sexual victimization used the “have you ever given in” wording TDOM quotes. But Koss was measuring a “scale” of sexual victimization — from “sexual contact” to “attempted rape” to “sexual coercion” to “rape.” Neither of the “have you ever given in” questions were used by Koss to measure rape.2

This isn’t a minor point, hidden deep in the footnotes of Koss’ study. This is the core of how her survey works. If TDOM had read the study with any attention at all, he’d know this.

TDOM also writes:

Koss could have improved her study by … testing her questionnaire for reliability and validity, ensuring that it measured what it was designed to measure.

Gosh, that would have been a good idea. Which is why Koss did exactly that. She also referred to this several times in her study report, which I sincerely doubt TDOM bothered reading.

TDOM also claims that the “alcoholism” argument, made by Koss and paraphrased by Hopper, is a straw man argument.

The straw man is the comparison (originally made by Koss) of her conclusions to that of questioning an alcoholic about her drinking habits. However, the questions asked by Koss about the alcoholic are quantitative questions. The answers are very clear cut, concrete, and quantifiable. They ask for numbers and frequencies. The questions asked by Koss in her study regarding rape and attempted rape were qualitative questions and answers were very open to interpretation.

Of course, that isn’t what a straw man argument is; a straw man is when “the arguer is attempting to refute his opponent’s position, and in the context is required to do so, but instead attacks a position—the ‘straw man’—not held by his opponent.”3 The term TDOM should have used was “weak analogy fallacy.”

Terminology aside, TDOM is wrong on the substance. There were plenty of questions about alcoholism, in the example, that were open to interpretation. If I drink a bottle of wine from the bottle, was that one drink or six? What does it mean that I “can’t go to work” — if I could go to work if my life depended on it, but called in sick because of my pounding headache, does that qualify or not? What does it mean to “help me stop drinking” — if my husband says “don’t forget, you have to drive home” does that qualify as trying to help me stop?

Etc, etc. When dealing with human behavior, there is almost no such thing as a question with no ambiguity at all. Survey authors can, and should, work to make their questions clear and specific, and do validity tests to make sure subjects understand what they’re being asked. But Koss did all that. TDOM’s objections are unreasonable.

Finally, TDOM makes one reasonable argument — he suggests that question 8’s wording (“Have you had sexual intercourse when you didn’t want to because a man gave you alcohol or drugs?”) is too broad.

However, researchers Martin Schwartz and Molly Leggett tested the disputed question empirically back in 1999.4 They surveyed students with Koss’ survey, but substituted this question for Koss’ original alcohol and drugs question:

Have you engaged in sexual intercourse when you didn’t want to but were so intoxicated under the influence of alcohol or drugs that you could not stop it or object?

If TDOM is correct to believe that Koss’ question 8 created a significant “false yes” problem, then many fewer students would have answered “yes” to Schwartz and Leggett’s rewritten version. So what actually happened? Rewriting the question made no difference at all. 17% percent of students surveyed by Schwartz and Leggett were found to have been raped, a number basically identical to Koss’ 15%.

Finally, let’s not forget that Koss’ three most important findings — that around 10-15% of women have been victims of rape in their lifetime, that most rapes are never reported to police, and that rape is usually committed by someone known to the victim, not by a stranger — have been replicated again and again by other researchers.

  • The NIJ/CDC “National Violence Against Women Study” found that 14.8% of American women experience a completed rape at some time in their lifetime. A typical rape-defining question was worded like this: “Has a man or boy ever made you have sex by using force or threatening to harm you or someone close to you? Just so there is no mistake, by sex we mean putting a penis in your vagina.”
  • The Department of Justice’s Sexual Victimization of College Women study included a sub-study in which college women were asked about lifetime incidence of rape (most of the study asked about rape since the beginning of the school year,which isn’t directly comparable to Koss). 10% of the women interviewed reported having been raped at some point in their lifetime. Rape was defined as “unwanted completed penetration by force or the threat of force.”
  • There’s also The National Women’s Study (NWS), a large-scale national study which found that 13% of American women have been raped in their lifetime.

At least a dozen more studies have confirmed Koss’ results. It is no longer possible to attribute Koss’ results to the particular wording Koss chose for her questions, because the same results have been found multiple times, regardless of wording.

  1. I haven’t read the rest of Hopper’s essay, so can’t say anything about it one way or the other. []
  2. You can read all ten questions yourself (pdf file). The two questions TDOM quoted were questions #1 and #6. From the scoring instructions: “Women are classified as victims of rape if they answered “yes” to items 8, 9, or 10.” []
  3. For example, when TDOM claims that Mary Koss states that answering “yes’ to “Have you ever given in (to some sexual behavior) when you didn’t want to…” “meets the legal definition for rape,” that’s a straw man argument, because he’s falsely attributing a position to Koss and then attacking that position. []
  4. Schwartz, Martin D. and Molly S. Leggett (1999), “Bad Dates or Emotional Trauma? The Aftermath of Campus Sexual Assault,” Violence Against Women 5(3): 251-271. []
This entry posted in crossposted on TADA, Mary Koss controversy. Bookmark the permalink. 

6 Responses to Yet Another Koss Critic Can't Get His Facts Right

  1. 1
    AM says:

    Um, what if you choose not to use the ‘feminist’ label but agree with the post? Sorry, I just think it’s a little unfair to say ‘non-feminists aren’t allowed to reply’. It’s saying ‘you’re either ‘feminist’ or against us.’ I see the people you’re trying to keep away, but there are valid reasons for not using the label while still defending women’s rights. There are many feminists who will shout at me for saying this – and I think that’s also part of the problem. (But that’s a whole other subject and this isn’t the place for it.)

    Saying that, may I still post a more in-depth response later?

  2. 2
    Ampersand says:

    I didn’t say “non-feminists aren’t allowed to reply.” I said “non-feminists should reply at this other space, rather than here.”

    Please go ahead and post your more in-depth response, either here or at the TADA cross-post. If you post here and I don’t think it fits in here, I’ll move your comment to the cross-post. Fair enough?

  3. 3
    BN says:

    Um, nope, I can read ” Non-feminists aren’t allowed to leave comments on this post on “Alas,” but anyone can leave a comment on “TADA.” right at the top of your post.

    Can you read it too, or are my non-orthodox views causing highly localised dyslexia?

  4. 4
    Ampersand says:

    BN, I’m not understanding what your point is. This post exists on both TADA and Alas. The comments on “Alas” are feminist only; the comments on “TADA” are open to everyone. Is that hard to understand in some way?

    (Also, are you the same poster as AM earlier this thread, or are you a different poster?)

  5. Pingback: This Rape Denialism Was Brought To You By The Scaife Foundation | Change Happens: The SAFER Blog

  6. 5
    Hazel says:

    Gi’day Ampersand,

    I really appreciate your well considered post. I may be missing this, but I’d like to read Koss’s study for myself. I tracked down the methodology note you linked to, but not the study from which all this controversy stems. Can you also link to that research? I found a few articles that it could be, but I have limited access to the publications so I’d like to narrow it down a little.

    Also, on q8 (the question about having unwanted sex when intoxicated) I think there remains a distinction between having sex one didn’t want and having sex one’s partner knew one didn’t want. Neither Koss nor Schwartz & Leggett’s wording gets to that distinction. For better or worse, many men and women drink to the point of significant intoxication of their own volition. Many women in this state, myself included, behave in ways that reasonable observers would think flirty. In this state, obviously if there is any other indication that advances are unwelcome or the victim falls unconscious, if someone continues sexual activity, that isn’t acceptable, and if intercourse occurs, it is rape.

    But I’m not sure I could categorize the sex occurs because the victim voluntarily consumed drugs or alcohol that rendered them unable or unwilling to indicate that they didn’t want the sex as rape. Should one’s partner perhaps be more sensitive to one’s response or lack there of? Sure. But does their lack of sensitivity make it rape? Especially when, as often is the case, the perpetrator is equally intoxicated? And often somewhat inexperienced in reading a sex partner’s behavior?

    I also want to say that I’m not asking this to inflame. I’m asking the question for two reasons. First, because I’m a geek and I love research and I hate it getting misinterpreted and love looking for the next research to do. Second, because so many bad sexual experiences (some rape, some not rape) happen when under the influence of drugs or alcohol, I wonder if looking at the role of intoxication and how to minimize or recognize risk when one wants to have a few drinks might help save women a lot of suffering?