Tuesday, October 9, 2007

Survey Says...

Family, not sex, is Pinoys’ No. 1 source of happiness (from the Inquirer)
How Happy are Pinoys with Sex? (by Dr. Romulo Virola, Secretary General, NSCB)

Despite the titles, note that the results cannot be generalised in any way. The data are from a pilot survey among 167 nonrandom respondents taken from participants in a meeting; therefore, one cannot generalise these results to the general Filipino population (or to the meeting participants' population, for that matter). So no need to be surprised. Yet.

What surprised me is the write-up in the Inquirer. If you only read the write-up, it would seem that the results are from a full-fledged National Statistical Coordination Board (NSCB) study, which would make the study laughable given the sample selection and size. However, going to the source article by Virola, you get the sense that the results are from an informal study done after a meeting-- equal in generalisability to a Best Dressed survey. He repeatedly points out that the results are from nonrandom respondents as a caution to the readers-- a caveat lost in the Inquirer article. More importantly, Virola attributes the results only to the sample, while the Inquirer write-up uses generalising language ("among Filipinos")-- a glaring misrepresentation.

The Inquirer reporter should have stressed that the results were in no way generalisable; however, doing so would trivialise the article. Which is exactly the problem-- they were trying to make news out of a non-event. They could have referred to the results in passing as part of a bigger story on Filipino sexuality or sexual taboos. Instead, the results were made the story.

Having worked in media, I know the pressures of putting some spin in your article to ensure that it makes the press, particularly on matters as staid as statistical data. However, as someone who also does survey work, I draw the line at misleading the readers in interpreting the data. Reporters have the responsibility to aid the readers in reading the data and lay out any and all caveats to interpretation. Statistics are difficult enough to understand and malleable enough to be spun; we don't need to add misrepresentation to the mix.
---
Below are the results of the nonrandom pilot survey, taken from NSCB:

2 comments:

Anonymous said...

the author of the inq article did explain what non-random was... i'm more confused about virola, for exactly the same reason you mentioned. when a stat expert writes for the public, you expect him to make inferences on the population on good surveys only. When he uses the gov't sample to extrapolate what filipinos are thinking... then thats something i don't understand...

xsaltire said...

While the Inquirer article did say what nonrandom sampling is, there was no mention about its implications, so the audience may assume that everything's ok and the results can be generalised. Virola mentioned the results in an article written for a group that is versed in what nonrandom sampling implies. His use of the results was within a specific context, which was lost in the Inquirer article.

I agree, though, that Virola (and the NSCB) should have been more careful when they posted the article in their website. They should have known that they were likely to be misinterpreted when they posted the article.