Alas, a Blog

[Trigger Warning! This post includes a sample question from a study which asked graphic questions about rape.]

(I know that some “Alas” readers looked at the title of this blog post and said “well, no kidding: isn’t that obvious?” But I still think it’s valuable to be able to point to proof. Perhaps this research will lead to the NCVS improving its design in the future.)

I just read an interesting new report about the design of surveys measuring rape prevalence. ((Bonnie S. Fisher, “The Effects of Survey Question Wording on Rape Estimates: Evidence From a Quasi-Experimental Design,” in Violence Against Women Volume 15 Number 2 February 2009 pages 133-147. Link.))

The author, Bonnie Fisher, conducted two almost identical surveys of women in college. (Both surveys were conducted in 1997). In the first survey, respondents were asked a series of 12 behaviorally specific sexual victimization screening questions, such as “Since school began in the Fall 1996, has anyone made you have oral sex by force or threat of harm? By oral sex, I mean did someone’s mouth or tongue make contact with your vagina or anus or did your mouth or tongue make contact with someone else’s genitals or anus.” These screening questions are built on the approach developed by Mary Koss in her influential rape prevalence studies in the 1980s, which have been oft-criticized by conservatives.

In the second survey, respondents were asked the sexual violence screening questions from the governments National Crime Victimization Survey (NCVS). The NCVS, along with the FBI’s compilations of police data, is the Federal government’s primary way of measuring crime in the US.

A bit of history: The NCVS used to be called the NCS (National Crime Survey), and didn’t ask about rape or sexual victimization at all; rapes were only counted if respondents brought up their rapes after being asked if they had been “attacked.” In the 1980s and 1990s, in response to feminist criticism, the government redesigned the NCS into the NCVS. Now, along with dozens of other crime-related questions, respondents are asked if they’ve experienced “Any rape, attempted rape, or other type of sexual attack.” They’re also asked:

Incidents involving forced or unwanted sexual acts are often difficult to talk about. Have you been forced or coerced to engage in unwanted sexual activity by—(a) Someone you didn’t know before—(b) A casual acquaintance—OR (c) Someone you know well?

This is a big improvement from not asking about rape or sexual violence at all — but it’s still pretty vague, compared to the behaviorally-specific questions.

In both surveys, each screening question “the respondent reports (answers “yes” to) is then followed up with a detailed incident report that contains multiple questions about what occurred during the incident. The responses to these questions are then used to classify the type of victimization that occurred.” ((Quoted from “Measuring the Sexual Victimization of Women: Evolution, Current Controversies, and Future Research”, by Bonnie S. Fisher and Francis T. Cullen. Link.))

But even though the two studies were mostly identical, apart from the questions used, ((There were two more significant differences between the two surveys. The first survey was described to respondents as a study of “unwanted sexual experiences that women may experience during college.” The second survey, like the National Crime Victimization Survey, was given a crime context, and described as a study of “criminal victimization that women many experience during college.”

The last difference is that the second survey, modeled after the NCVS, actually used a slightly broader definition of rape, including incidents of “psychological coercion as well as physical force.”)) the results were very different. Using the behaviorally specific screening questions, Fisher found that 19 out of 1000 female students had been raped since the beginning of the Fall term of school. ((The surveys were conducted in February through May 1997.)) In contrast, using the Federal government’s screening questions from the National Crime Victimization Survey, two out of 1000 female students had been raped since the beginning of the Fall term.

This makes it seem likely that the National Crime Victimization Survey, the USA’s primary method of measuring crime, severely underestimates how often rape occurs.