Last year, on CNN’s “Anderson Cooper 360” show1, Elizabeth Marquardt, author of Between Two Worlds – which is being re-released in a trade paperback edition this month – had this exchange with Cooper:
Anderson Cooper: Elizabeth, one of the questions that you asked the participants is whether or not they would describe their family as stressful. Fifty-one percent of children of — of what might be termed good divorces agreed with that statement, as compared to only 35 percent of those in unhappy marriages. What do you make of that?
Elizabeth Marquardt: That’s right. One of the striking findings to come out of our study was that children of good divorces often fare worse than those from unhappy marriages, so long as the marriage is low-conflict. And most marriages that end in divorce now are low-conflict.
This is fairly typical of how the Between Two Worlds study was reported – compared to the control group (young adults raised in intact marriages), young adults whose parents divorced fare worse. Elizabeth relies on this comparison often; in the Anderson Cooper interview, she criticizes The Good Divorce author Constance Ahrons’ research for having no control group of nondivorced parents.
Here’s another example, from a New York Times article about the Between Two Worlds study:2
The new survey, based on the first nationally representative sample of young adults, highlights the many ways that divorce shapes the emotional tenor of childhood.
For example, those who grew up in divorced families were far more likely than those with married parents to say that they felt like a different person with each parent, that they sometimes felt like outsiders in their own home and that they had been alone a lot as a child.
Those with married parents, however, were far more likely to say that children were at the center of their family and that they generally felt emotionally safe.
Unfortunately, the methodology of the Between Two Worlds study is fatally flawed, and the research cannot support any of these comparisons. Why? Because respondents were asked different questions depending on if their parents had divorced or not.
For instance, take the finding, reported in the Times, that adults raised by married parents were far more likely to sat “that they generally felt emotionally safe” when they were growing up. This finding is also reported in Between Two Worlds (page 59):
In our national survey, most young adults from intact families strongly agree that when they were grouping up, “I generally felt emotionally safe.” But fewer than half of those from divorced families say the same thing. Young people who grew up in “bad divorces” are less likely to say they felt emotionally safe, but I was surprised to find that even those of us from “good divorces” felt significantly less safe than our peers from intact families with unhappy but low-conflict marriages.”
This finding is based on asking young adults whose parents stayed married to agree or disagree with the statement “I generally felt emotionally safe.” In contrast, those whose parents divorced were instead asked “After my parents’ divorce, I generally felt emotionally safe.”3
So the respondents in the control group were asked about their childhood in general. In contrast, the respondents from divorced families were asked to focus specifically on a major family trauma. Given the biased questioning, it would have been a miracle if Elizabeth didn’t find major differences between the two groups. But it doesn’t tell us anything about outcomes, or about long-range trauma. The study design cannot distinguish between those who were unhappy for a while in the post-divorce period, but who on the whole recovered; and those who were left with long-term, ongoing trauma due to their parents’ divorce.
In the Times article, Robert Emory (of the University of Virginia’s “Center for Children, Families and the Law”) said ”The key is to separate pain from pathology.” This is the distinction that the Between Two Worlds study failed to make.
To see why this matters, imagine doing a similar study, this time dividing respondents into those who had pets who died, and those who didn’t. If you asked the “no pet” group if they felt emotionally safe as kids, but asked the “pet died” group if they felt emotionally safe after their pet’s death, probably many more of the latter group would say they didn’t feel emotionally safe. But would that tell us anything about the long-range outcomes for these two groups? Would a direct comparison of the two groups, as if they had been asked the same question, be appropriate? Of course not. A question about a particular, traumatic period in childhood cannot be used to characterize a respondent’s upbringing on the whole.
According to Elizabeth Marquardt, the respondents from divorced families were “periodically” reminded to focus on the period after their parents’ divorce throughout the survey. This means that almost none of the comparisons between divorced and non-divorced families in Between Two Worlds have any validity. For instance, the finding that Anderson Cooper focused on, that those from divorced families are more likely to “describe their family as stressful,” seems unremarkable when you keep in mind that the respondents were asked only about stress “after my parents divorce.” What child wouldn’t find that time stressful? But it doesn’t prove that their upbringing was more stressfel as a whole.
Even results from questions that didn’t include a “after my parents’ divorce” provision are dubious, in my eyes, because the context of the survey as a whole is changed by constantly asking those with divorced parents – and only those with divorced parents – to focus on a painful memory. The poor methodology taints the entire survey.
One finding from Between Two Worlds that Elizabeth emphasized is that “grown children from unhappy, low-conflict marriages generally fare better than those from ‘good’ divorces.” I doubt this finding would have been as strong — or existed at all — if this survey had asked the control group the same questions. Even the current, flawed results show some cases in which those raised by “good divorce” parents appear to be better off than those who parents stayed together. For instance, contrary to the New York Times’ reporting, those raised in “good divorce” families were slightly less likely to feel “like an outsider in my home,” than those whose parents stuck with unhappy marriages.
How much “extra” unhappiness, doubt and misery did Between Two Worlds measure by asking those with divorced parents, over and over, to think of “after my parents divorce?” There’s no way of knowing, but it could have been a large effect. Even a small effect could have seriously changed the finding that “grown children from unhappy, low-conflict marriages generally fare better than those from ‘good’ divorces,” because in many cases the statistical difference between those two groups was fairly small.
When I asked her about the difference in questions on her blog, Elizabeth responded:
Since we were studying post-divorce childhood, we asked those from divorced families to answer the question in regard to “after your parents’ divorce” and reminded them of that periodically through the interview. …If you want to study post-divorce childhood you have to ask people about, well, post-divorce childhood.
I thought they were studying the long-term effects of divorce. If it’s true that a person whose parents divorced when they were a child is significantly more likely to recall their upbringing as emotionally unsafe, stressful, etc., then that effect should show up without loading the question by asking subjects to focus specifically on the divorce.
If Elizabeth fears that subjects whose parents divorced when they were older – say, 10 or 15 – would have put too much weight on their pre-divorce time, then she should have asked her questions bounded by period (“When you were between 10 and 15 years old….” and so on). Then all respondents could have been asked the same questions, and the results examined for differences pre- and post- divorce. Failing to ask the control group the same questions is not a small design flaw; it’s a catastrophic error that invalidates all results based on comparisons to the control group.4
Between Two Worlds proves that many young adults whose parents divorced recall the period “after my parents divorce” as painful and confusing. However, nearly all5 of the study’s conclusions — including the conclusion that “grown children from unhappy, low-conflict marriages generally fare better than those from ‘good’ divorces” — are based on an invalid study design. In particular, most findings from Between Two Worlds based on comparisons to the control group should be regarded as invalid and unproved.
(Related material: See the STATS critique of Between Two Worlds, including a response by Marquardt and Glenn, here and here.)
[Crossposted at Creative Destruction, where co-bloggers stay together for the sake of the readers. If your comments aren’t being approved here, try there.]
UPDATE: Read comment #18, for another significant flaw in this studies’ methodology.
- “Anderson Cooper 360,” November 22 2005. [↩]
- “Poll Says Even Quiet Divorces Affect Children’s Paths,” New York Times, November 5 2005, p. A13. [↩]
- Source: Appendix “B” of Between Two Worlds, page 117, available online as a pdf file. [↩]
- It’s notable that this study was put together by an anti-divorce think tank, and has never been subjected to peer review. It’s natural to wonder if the peer review process could have forced the study’s authors, Elizabeth Marquardt and Professor Norval Glenn, to improve their methodology. [↩]
- I say “nearly all” because some of the findings — such as the greater likelihood of divorce among children of divorced parents — would not have been effected by the bad study design [↩]
Pingback: Pandagon
Pingback: Effects of divorce on child updates
Just a question: how can one even begin to make comparisons between marriages that end in divorce and those that don’t? How can one make an assumption that a “low conflict” marriage is unhappy; obviously it wasn’t sufficiently unhappy to lead to divorce. There’s also the issue that a low conflict unhappy marriage might artificially put children in the center of the family because . . . duh . . . the spouses aren’t putting each other first. I am sure there are ways in which social scientists try to adjust for these kinds of factors, but it must be really difficult, and, as you say, asking participants to focus on the divorce clearly stacks the deck. I think the only real way to get at this is to ask a lot of questions about adult functioning and perception of childhood and family from a wide sample of individuals and then analyze answers with reference to whether participants experienced the divorce of their parents. Even then, who’s to say that those who divorce are more likely to have mental health issues, which could be transmitted genetically or otherwise to their offspring.
That’s an excellent point, and one brought up by the STATS critique linked to at the bottom of my post. You can see the response from the “Between Two Worlds” authors by following the second of those links.
As I understand it, marriages were classified as “unhappy” or “happy” based on the respondents’ characterizations of their parents’ marriages. They don’t assume that all “low conflict” marriages are unhappy; there is also a category for “low conflict” but happy marriages.
I believe that the one author’s name is Constance “Ahrons.” I could be wrong about the spelling, but I’m pretty sure that is it.
The issue that Amp is discussing in #2 also comes from an earlier study by Paul Amato, which had dissimilar findings.
One of the other problems with the questions Amp raises in that they are retrospective. It would have been better to have a cohort of families and conduct longitudinal surveys. She does need to ask about before and after the divorce, specifically did the children’s happiness change. However, the best way to really examine this is to interview the kids at time 1 and then again at time 2 years later. Additionally, it would be best to ask children’s and individual parent’s perceptions. This would help bolster claims about parental conflict. They need several different views on this.
Hi Amp —
Thanks for all the time you’ve devoted to my work.
As you already noted, my opinion is that if you want to study people’s childhoods after their parents divorced, you have to ask them questions about, well, after their parents’ divorce. Perhaps this is a problem or even a limitation, as you suggest, but I think you’re hard-pressed to claim that it “invalidates” the whole study.
And while the children of good divorce *generally* fare worse than those from unhappy, low-conflict marriages, on some selected items they fared about the same or better. But in general (i.e., on more items) they fared worse and that is the only claim I’ve ever made in that regard.
This study does not claim to examine “outcomes” or “long-term effects.” It studies the quality of childhood itself by asking adults from divorced and intact families to reflect, retrospectively, on their childhoods.
The problem I have with making a distinction between pain and pathology is that is carries an assumption that pain doesn’t matter so long as it doesn’t result in pathology. I’m interested in both. We have a lot of studies documenting pathology among children of divorce. I’m interested in going deeper and looking at pain too. And I think most parents are interested not just in what might cause pathology in their children but also what might cause pain.
In response to Rachel S., yes, longitudinal studies are the gold-standard. Our study is cross-sectional. It is the first to take on the “good” divorce, the first to examine the moral and spiritual experiences of children of divorce, and the first national survey in the U.S. of grown children of divorce. It is also a beginning. There is much more work to be done. I sincerely hope that other researchers will also take on these questions, in particular the moral and spiritual development of children of divorce. If they are seeking funding for a longitudinal study I would be happy to write a letter of support.
Again, many thanks,
Elizabeth Marquardt
Pingback: Family Scholars Blog
Thanks for being so nice about this, Elizabeth.
My problem is, many of your claims are made based on comparisons between young adults whose parents stayed married, and those who don’t. But you simply didn’t ask them the same questions, making all your conclusions based on those comparisons questionable.
In particular, a claim that you yourself have emphasized as an important finding of the study – “grown children from unhappy, low-conflict marriages generally fare better than those from ‘good’ divorces” – is based on you asking two significantly different questions of the two groups. I therefore do not think this comparison is justified by your results.
Amp, to study a population of children of divorce is to study a population of children whose parents were once married. I don’t see how any researcher gets around this. Sure, a longitudinal study that isolates a group at one time and follows them over time as their parents divorce or stay married is ideal, but far from all research on children of divorce is longitudinal research.
I’d also like to correct you again: We *did* ask the two groups the same questions. We simply reminded the divorce sample periodically that these questions were “after your parents’ divorce.” But the questions the two groups were asked were identical (there were, additionally, some divorce-specific questions, such as about custody-arrangements, that we asked only of the divorce sample).
But to aske the question inserting the phrase “after your divorce” stacks the deck in ways that seem rather obvious. It would seem that if the idea is to study the impact of divorce on the perception of one’s childhood as happy or not, you would not (and in fact, would not want to) remind people of the divorce — presumably, if the divorce had an impact the participant would not need to be reminded — the impact would be incorporated into the participant’s response.
To amplify, if I were asked about whether my childhood was happy, I wouldn’t need to be reminded that my father was mentally ill for the last six years I lived at home. I would answer something like, “well it was probably average until I was 13, at which point my father was pretty clearly pointed on the road to a nervous breakdown. After that it was pretty dreary.” In other words, if the divorce were such a momentous event, it doesn’t seem like you need to be reminded.
Just to clarify an apparent misunderstanding, we did not ask them to focus on the time right after the divorce. Very early in the survey we said to those in the divorced family sample: “The following statements concern your experience growing up in your family after the divorce, that is, the period that begins with your earliest memories after your parents’ divorce and ends when you were 18 years old. Do you strongly agree, somewhat agree, somewhat disagree, or strongly disagree with each statement:..” Then, periodically, we reminded them that these statements concern “After your parents’ divorce…”
Barbara, in a quantitative survey (remember, this is not a qualitative exercise in which you can probe all these subtleties, though I did conduct qualitative interviews too), if you did not ask them to reflect on their childhood after their parents’ divorce, then how would you know whether they were talking about their experience as the child of married parents or their experience as the child of divorced parents?
Anyway, Amp, if the argument is that the divorce is of course too traumatic to mention without biasing the respondents, then I wonder if that isn’t in a sense proving my point?
If they were asked the same question wouldn’t you be saying that the study was flawed because the observed differences could well have been the consequence of an unhappy marriage prior to the divorce, and not of the divorce itself? And wouldn’t that be a far worse flaw than anything you point out above?
I can’t help but think Elizabeth is damned if she does and damned if she doesn’t.
Pingback: What You Should Read Since I’m Not Writing at Faux Real Tho!
Actually, that criticism exists regardless of if the same question is asked or not. You’re talking about two separate issues, not a “either it’s A or it’s B” issue.
My feeling is that the “is an unhappy marriage that doesn’t divorce really comparable to one that does?” is probably an unavoidable problem; there’s always a chance of unmeasured variables. In contrast, the problem I’m discussing here – failing to ask the control group the same questions – is avoidable.
Two ways have already been suggested in this thread, Elizabeth. First of all, you could simply have asked about upbringings as a whole (“considering your entire upbringing, would you agree or disagree that…”). If it’s true that divorce is as damaging to children as you say, then asking the question that way, without leading subjects to specifically think of “after my parents’ divorce,” would have still turned up statistically significant differences. (Indeed, if you think this method would NOT have revealed statistically significant differences, then I have to wonder how much harm you can really believe divorce does).
(By the way, it’s likely that most people who have read about your study in newspapers or heard it described on the news, assume you used the above methodology.)
Secondly, you could have bounded the questions by time period (“Thinking back on the times when you were between 11 and 15 years old…”), and then analyzed how results changed for the period before and after the parents’ divorce for the study subjects, compared to how the control group reported recalling those same time periods. That would have been much more work, but it would have allowed you to roughly separate post-divorce from pre-divorce.
Either method would have allowed you to meaningfully compare the children of divorce with the control group while still asking them the same questions. In contrast, your actual methodology makes such comparisons meaningless.
And I’m sorry, Elizabeth, but “Do you agree or disagree with this statement: It was stressful in my family” and “Do you agree or disagree with this statement: After my parents’ divorce, it was stressful in my family” are simply not the same question. No ordinary person would consider those two questions identical.
Of course, you wouldn’t know for any specific respondent. Being able to say that about any specific respondent is not what quantitative research is for.
But you would be able to legitimately say “65% of children of divorced parents said X, compared to only 45% of the control group.” As long as you controlled for other differences (economic, racial, etc) between the children of divorced parents and the control group, then by social science standards that would be a meaningful finding.
What your study actually shows is that when asked related but different questions, 65% of children of divorced parents say X, compared to only 45% of the control group. But since you didn’t ask them the same question, you’ve created doubt about if you’re really measuring a result of divorce, or if you’re just measuring a result of differently worded questions.
If your point is that divorce is a painful memory for many children of divorce, and reminding them of that memory over and over may bias their answers, then yes, your point is proved. But that’s such an obvious point that I don’t think anyone would really disagree with it.
If you surveyed me about my childhood, but reminded me periodically that you wanted me to focus on “after my dog died,” of course you’d get more negative answers than if you had interviewed someone just like me but never asked them to focus on a painful memory. That only proves that asking interview subjects to focus on a painful memory encourages them to give sad answers; it doesn’t say a thing about their upbringings as a whole.
* * *
Plus there’s the issue of age differences – which would have been controlled for in either of the methods I suggest at the top of this comment, but weren’t controlled for with your methodology. I can’t find what the median age of your respondents at the time of their parents’ separations was, but – just for example’s sake – let’s say the median age was 12.
Now imagine you give two groups your survey, except that one group was asked to consider only the time after they were 12 years old, and the other was told to consider their whole childhood. How can you be sure that this age difference, in and of itself, doesn’t cause significant differences? Many people might find their upbringing as a whole to be more or less pleasant than their upbringing after age 12. (Or age 9, or whatever the median age of parental separation was).
But since the children of divorce in your study were asked to focus only on the time after divorce, while the control group was asked about their whole upbringing, in effect you asked about two different time periods.
Elizabeth, thank you for being kind enough to respond here; I appreciate it.
While you’re here, I have another question, which I hope you don’t mind answering. You’ve often said something along the lines of this, from your letter published in the NY Times: “grown children from unhappy, low-conflict marriages generally fare better than those from ‘good’ divorces.”
However, looking at your data in detail, this seems to me to be a misreporting of your findings, because 85% of the parental marriages you classified as “unhappy” were in fact reported to be “pretty happy.” Only 14% were reported as “not too happy.” (I calculated those percentages from page 154 of the crosstabs you placed online).
Furthermore, if “high conflict” marriages were especially likely to be reported as “not too happy” (which seems plausible), then well over 85% of the respondents you placed in the “unhappy, low-conflict marriages” category may have actually described their parents’ marriages as “pretty happy, low-conflict.”
With this in mind, a more accurate statement of your findings would therefore be “grown children from pretty happy, low-conflict marriages generally fare better than those from ‘good’ divorces.”
My question is, am I correct in all I say above? Because if I haven’t misunderstood something, then the claim that “grown children from unhappy, low-conflict marriages generally fare better than those from ‘good’ divorces” seems unsupported by your results.
At first I thought Amp’s critique was valid. But on reflection, I don’t think it is.
What Amp wants, for neutrality’s sake, is for the study to have asked people “so, how was your childhood overall”, and then compare the results between populations. And that would indeed be an interesting question.
But what EM was attempting to study was a different question, which was: is the childhood of someone after divorce notably different than the childhood of the non-divorced. And to discover that, it’s necessary to phrase the question differently, between the two populations had a different experience. Since the study is interested in a specific post-contingency timeframe, it is necessary to explicitly assert that timeframe with the group that experienced the event. Elizabeth’s description of the process – telling the subjects at the outset what timeframe needed to be concentrated on, and then occasionally reminding them that there’s a specific timeframe to keep in mind – seems a fair way of handling this. It wasn’t “YOUR PARENTS DIVORCED, SAD LOSER – now answer this question!”
It’s a fair point that reminding the subject of the divorce introduces some uncertainty into the results, but it seems an unavoidable uncertainty given what the study covered, and I don’t think the uncertainty arises to the level of making the study invalid.
What I’d like to know is how does she get the idea that most marriages that end in divorce are low-conflict? What is low-conflict?
The problem, Robert, is twofold: first, by including the phrase “after divorce” it seems highly likely that the participants would be much more likely to focus on what they thought about the divorce rather than their childhood. And second, as Amp stated already, “after divorce” may cause participants to focus on a period of childhood that is later than average, which could again skew the results. I was a lot happier as a six year old than I was as a 12 year old quite apart from any trauma that occurred in my life. The same is true for a lot of children.
You could conduct the study thus: Ask participants to rank their happiness overall, and then rank it by age (on a scale of 1-5, from the time you can remember, before school, during elementary school, middle school and high school . . . ) and then separately ask them to report the marital status of their parents and at what point the divorce occurred. This would give you data points without asking the participants to focus on the time period “after the divorce occurred.”
I have to believe social scientists deal with this kind of problem — avoiding biased questioning — all the time. And I still think that if the divorce were such a momentous event, it would have colored your overall perception of your entire childhood, and if it didn’t, then I have to question whether it was really all that negative.
Elizabeth is an able defender of her own work.
I’d only like to say that Prof. Emory is right: It is important to separate pain from pathology.
But that doesn’t mean pain doesn’t count.
Parent have no right to inflict pain on their children, on the grounds that it doesn’t lead to pathology. Maggie
Amp, I think you are at least somewhat mistaken.
Unless you make the incremental “how did you feel during ____ time period?” units extremely small, you won’t necessarily be accurate. 5 year increments, for example, are way too broad. Memories generally work in slices that don’t necessarily fit neat time periods.
You might remember a swatch of life as “the time between when Bob was born and when Mary was born” that happened to be 2 years, but without some triggering incident you’re unlikely to keep track ofyour youth in handy 2 year increments. And of course, if you DO make the increment small “how did you feel in the first quarter of your ninth year?” you run into problems as well.
You and other commenter are also operating on the constantly-stated assumption that the mere mention of a divorce will significantly skew the results. Barbara’s comment is a good example:
“Seem highly likely” isn’t a great way to attack a study. At least not for me. You need to provide some sort of quantized data and a specific objection, that shows both 1) your protest is apposite, 2) what the magnitude of the effect is that you discuss, and 3) what effect your claim would have on the overall data, if true.
personally, I don’t think it’s necessarily true that the overall impressions of childhood would be fatally marred by a reference to divorce. If children of divorce are blocking or denying their overall sadness–as is well possible–then you would also have to demonstrate that they would fail to continue that denial or blocking during the subsequent questioning period. The claim that the children would focus on the divorce itself rathere than their life would best be addressed by ana analysis of the questions themselves.
I actually think you may be right.
However.
When I read this
Not incidentally, in this and your last critique of a scientific paper (the fat study) you appear to have some significant bias of your own. While in politics there may be no “real truth,” that is often not the case in science. You don’t get to vote off studies you don’t like.
I don’t want to presume too much because I haven’t read the study. However, Sailorman, I do not believe that your objection to my objection is correct. There’s no way to test post hoc — certainly not quantitatively — whether asking a question one way or another will provide different data. Social scientists have to work through potential bias all the time(including potentially asking the question in a way that misses effects, along the lines of what you suggested).
I also disagree about having to slice childhood into really narrow segments. The whole point was to try to tabulate impact on one’s entire childhood after divorce, not only immediately after divorce — so if your parents got divorced when you were 10, at the end of elementary school, it would seem that if their divorce made you less happy it would show up in your middle school or high school index of happiness. Certainly, with enough participants, some kind of trend would be reasonably apparent by plotting the data.
The whole exercise is imprecise, no matter whether the researcher calls it quantitative or what have you — it’s not the same as plotting temperature or rainfall levels or rate of cell death or what have you. Judgments have to be made, and in my judgment, it appears that EM was concerned about missing the negative impact of divorce to the point that, perhaps, she veered into asking the question in a way that overstated its impact.
The difference between pain and pathology is sufficiently distinct that I would have to disagree. If I decided to move my family I would definitely be inflicting pain on my children. The same would be true if I made all of us start attending a new church or if I decided to send them to a new school. Or if I dropped their music lessons, or made all sorts of other non-trivial changes in their life. Can I look forward to a study on these things? Or is it just presumed, to some degree, that parents take the impact these things have on their children into account and make decisions so as to balance the welfare of all members of the family? Why don’t they get that same benefit of the doubt when divorce is concerned?
Sailorman,
This is ridiculous.
Suggesting that non-peer reviewed studies are somewhat suspect, and that being subject to peer review is beneficial to study design bears no resembalence to claiming there is no “real truth.” In fact, it is the opposite. Suggesting that social science work can be gamed (consciously or unconscously) for the benefit of a financier or employer is still not arguing that there is no “real truth.”
What, in your opinion, is the function of peer review? Do you consider non-peer reviewed studies financed by advocacy groups to be generally the equal of peer reviewed research financed by scientific foundations? Is not the purpose of peer review to help design research (or ensure that research was designed) in order to better get at the “real truth?”
Getting at the real truth is hard (as you well know) and it is easy to mess up your results and go astray, particularly when you are trying to tease out subtle effects long after the fact.
I think Amp’s claim that the study results are rendered valueless is over strong, but the results are certainly tainted by methodology problems. Maybe this is the best methodology design for the problem, and the problem just can’t be well studied by these types of methods, maybe another design would have done better.
Certainly, a series of studies could be designed to test how much of an effect (if at all) there was from the “after the divorce” reminders. However, it is not Amp’s responsibility to perform those studies. Those studies are necessary in order to demonstrate the validity of Elizabeth’s research. Those studies are more likely to occur because people who notice flaws in the research point them out and question the validity of the research. Amp is suggesting the need for those studies (or rather pointing out a problem and raising an objection that could be answered by such studies). This is a legitimate and valuable function.
As Barbara pointed out, it is not viable for me to rerun the Between Two Worlds study but this time asking the control group the same questions. If we took the standards you suggest here seriously, we would conclude one can never make a post-hoc critique of how a study phrased its questions study unless one has the resources to re-reun it with different phrasing and then compare the outcomes. That’s nonsensical.
The idea that social scientists should strive to ask the control group, as nearly as possible, the exact same questions as one asks the other subjects, is not a post hoc rationalization on my part. Indeed, I feel certain that what I’m saying is in accord with what most social scientists believe to be best practices, as a general rule.
I don’t think it’s necessarily true, either. However, I don’t think it’s necessarily untrue, as you seem to be assuming. Because we cannot know if it’s true or not, it would have been better to design the study to avoid this question.
A good study design cannot eliminate all uncertainty, because that’s not possible. But a good study does eliminate what uncertainty can be eliminated, and phrases its conclusions modestly in deference to what uncertainty couldn’t be eliminated. The obvious way to eliminate the “is it the divorce or the different questions producing these results?” problem is to have asked the same questions of both groups. And if the researcher doesn’t do that, then the conclusions drawn from the study should be stated in an appropriately modest way, to take into account the extra uncertainty created by asking different questions.
This is an obvious ad hominem attack, Sailorman, and it should be beneath you.
As for my bias, surely everyone has some biases. However, contrary to your implication, my bias did not prevent me from accepting the results of Elizabeth’s study in the past. When I first read about Elizabeth’s study, I was so impressed that I emailed her to let her know how persuasive her work was to me. My bias has not changed between then (when I accepted this study’s results) and now; what did change is how closely I had examined the study’s methodology.
[Cross-posted with Charles’ post.]
Amp-
You raise an interesting issue. It certainly is possible that some of the “pain” that Elizabeth uncovers is limited to the immediate aftermath of the divorce. But her argument is congruent with work done by Robert Emory at UVa and Paul Amato and Alan Booth at Penn State that shows that the emotional and social consequences of divorce are especially negative for kids whose parents have a low-conflict marriage and go on to divorce.
This is an important point because about two-thirds of all parental divorces, according to their work, do not involve highly conflicted marriages. In other words, two-thirds of divorces do not happen because of spousal physical abuse and/or serious conflict; rather, they happen because spouses grow apart or unhappy or seek a new partner. “Unfortunately, these are the very divorces that most likely to be stressful for children.” (Amato and Booth 1997: p. 220)
The reason? Children value the love, support, and attention they receive from their parents even if their parents’ marriage isn’t particularly warm. And, as Elizabeth’s book suggests, they are less likely to get that support, and more likely to feel vulnerable being bounced from place to place, after their parents divorce.
In other words, her book helps to suggest the emotional mechanisms that account for the negative effects of divorce–effects that are particularly salient when couples in low-conflict marriages break their wedding vows.
Regarding pain and pathology, I want to endorse what was said by Barbara in comment #24.
Let me add, in response to Maggie Gallagher, that I certainly wouldn’t say children’s pain is irrelevant. But nor do I think that parents’ pain is irrelevant; if the only issue is pain, I can certainly imagine circumstances in which someone might decide that the pain to all parties of maintaining a horrible but low-conflict marriage is not as bearable as the pain of divorce.
Some pain to children is something that parents’ can legitimately decide is the least bad of bad alternatives; indeed, making decisions that children find painful is sometimes an unavoidable part of parenting. It’s only in a very extreme circumstance that I could imagine saying the same thing about pathology.
I’m curious: Does anyone – but in particular, the folks who have been defending the study – have any comment on my comment #17? If I’m correct about the people in “pretty happy” marriages being classified as “unhappy,” does that make any difference, or am I barking up an irrelevant tree?
I’m waiting to hear what the study author(s), who have way better understanding of their data than I’ll ever acquire, say. Either they made a huge error or you’re totally nuts.
False dichotomy. :-P
False dichotomy. :-P
There are other possible explanations but P1 and P2 cover >95% of the expected range, so fuck it. The dichotomy is close enough to reality to be workable.
Actually, the problem with Robert’s description isn’t the false dichotomy. The wrongness of his description is the problem.
There is an issue with the study design that raises some questions. It is possible that the responses were biased by the reminders of an unhappy experience.
Amp believes that this may have been a large influence, and considers the study results highly suspect unless the question of how much biasing influence this design issue had can be resolved.
Elizabeth (the study’s designer) argues that the biasing effect isn’t really that much of a biasing effect, and that if it is, that that is part of the valid results (if it is so upsetting to think about your parents divorce that it throws off how you remember your childhood, then it must have made your childhood unhappy).
She further argues that the study could not have been effectively designed to avoid repeatedly reminding the subjects of their parents’ divorce.
Others have suggested alternate designs that would have avoided doing so.
Sailorman has suggested that those would be weaker designs in other ways.
The question of whether the results are biased by the design and the question of whether the study could have been designed to avoid this bias are separate questions. If the study design is truly the best design possible (given the type of study being done), with the other suggested designs introducing even greater errors of different sorts, that doesn’t mean that the results are not biased. It just means that this type of study can’t answer this type of question with much confidence.
There are variants that could be run (or could have been included in the study design) that could start to answer the question of how much bias the study design introduced. Those variants weren’t included in the design of the study and haven’t been run separately, so the first question is hard to answer.
The third question that the first two questions raise is should a potential bias of unknown magnitude be simply ignored or should it be reason to ignore a particular study. The answer, I think, to both of these options is “No.” The study should be treated as a reason to do further study, both to try to either measure the bias, or to try to avoid introducing it. In the meantime, the study results should be treated with caution, as suggestive of a question worthy of further study, and not as a definitive result on which one should necessarily base policy or personal decisions.
The nuts vs huge mistake description reduces this situation to an idiotic cartoon and adds absolutely nothing to the discussion.
Charles, I believe that Robert’s “Either they made a huge error or you’re totally nuts” comment was a humorous take on the specific criticism in my comment #17, and not a response to the more general discussion that your most recent post (#33) comments on.
In other words, Robert’s comment #30 was a direct response to my question #29. Put in less less humourous terms, I think he was saying, in regard to comment #17, that either I’ve completely misunderstood the data I was reporting on, or the study authors did (in Robert’s opinion) make a major error.
I quite agree with this. (Do I contradict what I wrote earlier? If so, then what I said back then was mistaken).
However, like Robert, I’m waiting to hear Elizabeth’s comment on the “unhappy” versus “pretty happy” question.
Uh, yeah, it would seem that if “unhappy” and “pretty happy” were considered to be essentially synonymous (if I am reading your #17 correctly) then a person who considers him/herself to be actually unhappy might not want to base any major life decisions on the results of this study. On the other hand, perhaps you are confused about the definitions or the data — the way I get confused when I try to remember whether I am supposed to prefer a day that is partly sunny or partly cloudy.
Ah, that makes much more sense.
partly cloudy = partly sunny, except partly cloudy can be used to refer to either day or night.
mostly sunny -> partly sunny/partly cloudy -> mostly cloudy
mostly clear -> partly clear/partly cloudy -> mostly cloudy
So sayeth the National Weather Service
Charles, now I am totally confused.
Amp, where are the crosstabs for #17? I want to look at them before I comment on that.
In response to W. Bradford Wilcox. You said, “Paul Amato and Alan Booth at Penn State that shows that the emotional and social consequences of divorce are especially negative for kids whose parents have a low-conflict marriage….. In other words, two-thirds of divorces do not happen because of spousal physical abuse and/or serious conflict; rather, they happen because spouses grow apart or unhappy or seek a new partner. “Unfortunately, these are the very divorces that most likely to be stressful for children.” (Amato and Booth 1997: p. 220)
Okay, but if I remember that article, they also found that kids from divorced high conflict families faired better on social psychological adjustment in early adulthood than kids from high conflict married couple families.
Additionally, your operationalization of conflict is really restrictive, and I do not remember Amato and Booth having that narrow definition of conflict (I’m going to go back and check the operationalization, to double check.). Additionally, one weakness of the Amato study is that they have adult children’s retrospective perceptions of parental conflict, which by definition has it’s limitations, since many parents understandably try to hide conflict from children and the conflicts are well in the past and may not be easy to remember.
I personally, do think it is logical that divorce does produce some negative emotional consequences for children whose parents divorce when they are older, but there could also be negative consequences that can come to children from having married parents who do not get along. Sometimes the divorce may shelter the kids from greater problems down the road.
Rachel, the crosstabs are on Elizabeth’s book’s website – here’s the pdf link . The bit I referred to in comment #17 is on page 154.
The crosstabs alone don’t tell the whole story. You also need to know that a footnote on page 198 of Between Two Worlds says “Parental marriages not reported to be ‘very happy’ are considered unhappy.”
Finally, table one of appendix A online ( pdf link ) gives more numbers; comparing these numbers to the crosstabs, it seems to me that the numbers only add up if the “pretty happy” category was catagorized as “unhappy” by the researchers.
However, it’s always possible that I’ve gotten confused somewhere.
Having read Amp’s post and Elizabeth’s responses, I can’t help but wonder why Elizabeth didn’t transform the “How did you feel” question into both
a) How did you feel after the divorce and b) How did you feel before the divorce.
The experiment would still suffer from the difficulties inherent in restrospective studies, but at least we might discover whether the children were already experiencing more pain prior to the divorce. For all we know, parents whose children are suffereing as a result of the bad marriage may divorce at higher rate. So yes, those children may still be suffering more pain than others afterwards. The divorce didn’t instantly cure that!
May I add one more specific concern about the study? Because the wording is different for the children of divorced and non-divorced couples, because of the very specific reminders to the children of divorce that they are Children of Divorce, isn’t the stereotype threat evoked? Given that married parents are the norm (regardless of actual rates), it is impossible that all those children of divorce are not emphatically aware of the societal expectation that their childhood was wrecked, and that now they, emotionally, are also wrecked. By continually reminding this one group of adults about The Divorce, can anything useful be drawn from their responses?
Pointing out potential flaws in a study does not invalidate it. But Amp, your concerns are valid. I’d say that this single study doesn’t prove anything, and that other studies, correcting for these (possible) defects, would be necessary before anything could be made of it.
Amp –
In response to your question in comment number 17:
Most marriages are reported to be “very happy” in most surveys that ask about marital happiness. Very few say that they are “not too happy.” Therefore, the usual way of dichotomizing the responses is “very happy” vs. all others. It would perhaps be more accurate to call the second category “less than very happy” or “less happy” rather than “unhappy,” but in fact on surveys on which married persons are asked more specific questions about their marriages, those who say their marriages are “pretty happy” turn out to have marriages that are not very good–in fact most of them have some rather serious problems. These marriages might be called mediocre, at best. Of course, when someone other than the spouses is rating the marriages (in this case, the grown children) that could be a somewhat different issue. We don’t really know how good those marriages were, but there is reason to believe that any marriage that can’t be rated “very happy” was not all that great. Perhaps using the phrase “less than very happy” or “less happy” would have been advisable. Again, let me repeat, there are good reasons (based on other marriage surveys) for categorizing the “pretty happy” marriages as “unhappy” when the adults are reporting their own marital happiness, but it is perhaps the case that using this terminology based on the grown child’s report of their parents’ marital happiness is not quite as strong.
Best,
Elizabeth Marquardt
Elizabeth, it seems to me that in light of your response, it’s fairly safe for me to conclude that what I said in comment #17 was correct; the large majority of “low-conflict, unhappy” marriages refered to in your study were actually described by respondents as “low-conflict, pretty happy.”
How many of your readers would have assumed that the “unhappy” category included “pretty happy” marriages? Other than one oblique footnote on page 198, was the fact that 85% of “unhappy” marriages were ones respondents described as “pretty happy” ever made explicit in Between Two Worlds?
Elizabeth, can you give a couple of citations to the studies you’re referring to? And do these surveys classify “pretty happy” marriages as “unhappy” marriages when reporting their results?
And in those studies, are the “pretty happy” marriages ones in which the married people are seriously considering divorce? Because you’ve referred to your study, multiple times, to suggest that low-conflict parents who are so unhappy that they’re seriously considering divorce should stay together, because your results showed that “low-conflict, unhappy” marriages are better for the children than low-conflict divorces. But unless “pretty happy, low-conflict” marriages are ones in which divorce is being seriously contemplated, your study cannot legitimately be used to support the statements you’ve been making.
Finally – as you yourself point out, much to your credit – it’s extremely questionable whether studies of how adults describe their own marriages can be extrapolated to how children describe their parents’ marriages. I don’t see how you can ignore the possibility that the marriages your respondents described as “pretty happy” were, in fact, pretty happy.
In the study, those from “good” divorces in many ways fared worse than those from pretty happy/not too happy marriages, so long as those marriages were low-conflict, as most are that end in divorce.
Additionally, those from “good” divorces fared far worse than those from happy marriages.
In our rather generous estimation (generous in favor of the “good” divorce) only about half of divorces are “good” ones, and when you set about divorcing you have no guarantees whether your divorce will be “good” or not.
If your marriage is low-conflict, whether you consider it very happy (in which case you’re unlikely to be considering divorce), pretty happy, or not too happy, there are good reasons for your child’s sake to reach out and get help and try to keep your marriage together.
I would describe my parent’s marriage as somewhere between pretty happy and very happy. I would tend towards saying pretty happy mostly because my father is an alcoholic and my parents came very close to divorcing (and separated for several months) at one point, and not quite so close at several other points. But most of the time they are loving and supportive of each other, so certainly I wouldn’t rate their marriage worse than pretty happy.
Just an anecdote, but it leads me to tend to agree with Elizabeth’s supposition that when children of in tact marriages say that their parents marriages were pretty happy, their parents (openly or not) probably contemplated divorce at some point. The ideology of marriage makes people very hesitant to describe a marriage, particularly an in tact one, as unhappy. Someone who describes their parents marriage as actually unhappy is probably describing a marriage that was perpetually on the verge of divorce, one in which other people had a hard time figuring out why the marriage held together.
So Robert still loses! You’re not crazy and Elizabeth didn’t make a mistake! HA! :-p
He could still be crazy for other reasons. I cling to hope!
Say Amp, how do you comment on Elizabeth Marquardt’s blog? She said you were a frequent commenter there and I couldn’t figure out how to do it.
Since she never answered my question of where she gets the ridiculous notion that most divorces are from low-conflict marriages, I wanted to ask her there.
Hi Salty C,
About a week ago we closed comments on FSB due to staff shortages.
Regarding the low-conflict numbers, the most-widely cited source is a large longitudinal study reported by Paul R. Amato and Alan Booth in “A Generation at Risk: Growing Up in an Era of Family Upheaval” (Cambridge: Harvard University Press, 1997). One of their key findings was that one-third of divorces ended high conflict marriages, and children did better after those divorces. However, two-thirds of divorces ended low conflict marriages, and children did worse after those divorces. They are leading divorce researchers based at Penn State University.
Best,
Elizabeth
I’m still fuzzy: Is a marriage in which the husband hits the wife only every two months considered low-conflict or high-conflict? Or does any abuse at all put it in the high-confilct category? And is there a medium-conflict category?
Also, do you think it’s all-too-common for women to stay married when they shouldn’t, for instance if the father is beating the children?
SInce there’s apparently no easy definition for a low-conflict marriage, I will just guess and say that it means a marriage with less-than-average conflict. So at least half of marriages are high conflict, whereas only one third of divorces end high-conflict. Therefore, a good predictor for divorce is that a marriage is low-conflict????
[Snake-oil spam comment deleted by Amp.]
Why don’t you go hock your snake oil elsewhere?
Pingback: Alas, a blog » Blog Archive » Has Divorce Reached Its Natural Rate?
Pingback: Equality Loudoun » The facts of life
I am the child of a couple that stayed together, but I’m not sure what low conflict is. Does it mean that there is infrequent conflict, or that the conflict isn’t physical, or both? The conflict between my parents wasn’t physical – it was emotional warfare and I felt stuck in its crossfire. My father would get mad – yell at my mother – and if she stuck up for herself (which she didn’t always do) he would get madder or leave the house threatening suicide. This went on for years. I found out later that sometimes when my father slammed the door after a blow-up, some of my brothers and sisters wished he would never come back. Like me, they just wanted it to stop. A divorce would have been a relief. By the way, my brothers and sisters and I are all doctors, pharmacists, and lawyers. From the outside I guess the Christian conservatives would say my parents’ staying together was a big success – but we all have a lot of baggage.
Teresa,
Your parents mariage would have been described as high conflict…you would have been better off if they had divorced. High conflict marriages as defined by most studies are characterized by physical AND verbal abuse.
It doesnt matter very much about how “flawed” Elizabeth’s design is: her conclusion is very similar to many other quantitative studies done in this field….divorced children face more problems of delinquency than children from intact families. The most noteworthy statistic is the fact that divorced children are almost twice as likely to divorce than those from intact families.
The “good” divorce advocates often separate pain from pathology and argue that as long as the child turns out ok, divorce shouldnt be condemned. Well, victims of childhood sexual abuse or violence turn out “ok” too. They go on to have familes, careers…so why make a big deal of these problems?
Nor is divorce just a conservative issue. Read “The case for marriage” written by both a liberal (Linda Waite) and a conservative (M. Gallagher) that uses qualitative and quantitative research…it should be very illuminating.
kesari, changing schools when their parents find new jobs, especially multiple times, can be a painful experience for many children. To what extent should parents’ economic choices be restricted for the sake of their children?
sylphhead,
If you are equating the painful experience of frequent moves with the trauma of a family break-up–a trauma that leaves everlasting scars and affects their own chances of successful marriages– then we have more than a mere difference in opinion. Its a fundamental difference is values and I doubt if anything that I say will matter to you and many others in this forum.
You know, the interesting thing about what you said, besides the absurd tautology behind it, was that by dodging my question, you were tacitly agreeing with the parallel premise. The average person in your position would simply deny that he’s trying to limit anyone’s freedom and that he’s merely trying to make them mindful of the effects of their free choices – sort of like admonishing people against junk food, say – rather than go on some self-righteous huff. Because the average person values freedom enough to not take my accusation of legal fascism lightly.
But you did, apparently because you actually would have no problem with restrictions on people’s freedom being imposed on the outside on this issue. And so if your values include taking the country imposing such a legal arm on the family that no other developed, democratic nations share and a lot of two-bit dictatorships do, I suppose we do have a fundamental difference in values.
This is a long posting with background information about Marquardt’s other writings on gay marriage, sex, and adoption, and about the organization that funds Elizabeth Marquardt’s work. There is a final paragraph below on the lack of information known about her survey quality. Sources are cited below for the information given (and all sources are available online, with the exception of Edwin Black’s book).
Elizabeth Marquardt is funded by the Institute for American Values (IAV). IAV was at the forefront of the development in the late 1980s/early 1990s of the new conservative movement. The new movement did not attack feminism, gays/lesbians, or racial tolerance directly, but instead through the back-door of attacks on divorce, nontraditional families and welfare.
Marquardt wrote a “non-attack attack” on gay marriage for IAV called “The Revolution in Parenthood: The Emerging Global Clash Between Adult Rights and Children’s Needs.” The report is available on the IAV website, http://www.americanvalues.org. The report highlights what the author views as the problem of gay marriage, but also stresses the importance of children being raised by their own biological parents. (See report at pages 19-21.) The report is full of “what if” medical horror stories of nontraditional families gone awry.
Marquardt also wrote a report on sexually active college girls for IAV, available on their website, called “Hooking Up.” The premise of the 88-page report seems to be that while college girls claim to want to get married more than anything, no man is going to buy the cow if he can drink the milk for free. There are sections comparing white college girl sexual behavior to black college girl sexual behavior that do not seem to serve any identifiable purpose.
IAV is run by a small group of individuals focused on subtly (and sometimes not so subtly) advocating views on what is an appropriate family and an appropriate society. Much of their work takes a purely economic approach: they oppose spending taxpayer dollars on social programs for divorced people or welfare moms and believe that a woman’s economic security threatens the stability of marriage. (In 2004, IAV received a grant from the Bush Administration to advise on welfare reform.) See, for example, a posting on the IAV website dated April 9, 2009 titled “Husbands, Wives and Hard Times,” written by one of the advisors on Marquardt’s book, Barbara Dafoe Whitehead. Dafoe Whitehead contrasts the Great Depression with the current economic downturn: “The New Deal also gave priority to family men in distributing public works jobs, which kept husbands from deserting their families. … Today, couples don’t look to marriage for economic security alone; they expect marriage to fulfill their emotional needs for intimacy. Women are also less dependent on a husband’s payment than in the past. These conditions conspire to make marriages less stable.”
IAV has also devoted resources to a special project on the black family. There is a section of the IAV website devoted to the challenges faced by the black family, and dominated by reports on the low-income black family.
David Popenoe directs IAV’s “Council on Families in America” project; his reports are available on the IAV website. [See, eg, http://center.americanvalues.org/?p=77 ] David Popenoe is the son of the American eugenicist Paul Popenoe. Paul Popenoe ran a “family therapy” center in L.A. in the 1930s and was head of the California Human Betterment Society in the 1920s and 1930s. In the buildup to WWII, Popenoe corresponded frequently with Fritz Lenz, one of the architects of Hitler’s eugenics-based vision for cleansing the white race. Letters from Popenoe to Lenz focus on “protection of the Nordic race.” Popenoe advocated mass executions in America of the “feebleminded” and prohibiting those people with weakened immune systems from marrying and procreating. For detailed information on Paul Popenoe and his writings, see Edwin Black’s book “War Against the Weak: Eugenics and America’s Campaign to Create a Master Race.” [All Popenoe references in the book are catalogued in the author’s comprehensive index. For Popenoe’s statements advocating mass executions, see page 251 (chapter 13) of Black’s book] On the lives and work of David and Paul Popenoe, please see also the epilogue to Wendy’s Kline’s book, “Building a Better Race,” available online in preview from google books.
Rather than publicly repudiating his father’s views on racial purity and the somewhat sinister potention role of marriage in furthering a eugenics agenda, IAV’s David Popenoe seems to have continued the themes in his father’s work, albeit with the much greater subtly necessitated by modern times. David and Paul Popenoe ran an organization together in the 1970s focused on family values issues. David has also created a website lauding his father (although not mentioning his connections to Nazi Germany or his support for mass executions). [See http://www.popenoe.com/PaulPopenoe.htm%5D David Popenoe does choose to say some things on his website about his father that are of a rather sensitive nature, describing in particular his father’s extreme discomfort with masturbation and pornography.
Back to the author at hand. She has a masters degree in divinity and a bachelor’s degree in international relations. If she has a degree in child psychology, psychology or sociology, or any work experience in these fields, it has not been disclosed. Most reputable nonprofits would require their scholars on topics of child psychology to have studied and worked in the field. How did the author get 1500 young adults to submit to a phone interview involving the answering of 125 questions, half of which are about religion? Well, she didn’t. The author’s website states the work was conducted by a survey firm called SRBI, Inc., which asked 125 questions over the phone of 1500 individuals between the ages of 18-35. The author’s website also states that the survey was conducted “in New York City” between Feb. and May 2003, although Marquardt is not clear about whether the participants were all “in” New York City, or whether the survey was just completed in the New York City-based office of SRBI. In sum, it appears the research is purportedly based on data provided through the random selection by a for-profit research firm of 1500 NYC young adults, each of which agreed to answer 125 questions over the phone for no compensation.
If you have found this information about the source of Marquardt’s funding helpful at all, please forward on to others who are familiar with her book, particularly any media outlets that have featured the author in their programming.
-Laura Kiruna