David Baxter PhD
Late Founder
Surprise! Students Find Inaccurate Health Information Online
by John Grohol, epatient
July 22, 2008
A new study in the Journal of Medical Internet Research (JMIR) suggests that inaccurate medical information is easily found and regurgitated by students. But the topic the study chose to study -- vaccines -- has been under increasing scrutiny and controversy, limiting the generalizability of the results.
The study was simply setup -- 34 high school students were asked to find websites about vaccines, and then rate their accuracy. Of the 40 sites found, 22 were described by the researchers as "inaccurate," including well-known sites such as CBS News. The search term "vaccine dangers" returned virtually all inaccurate search results out of the first 20, while the search term "vaccine safety" returned virtually all accurate search results.
I think the only conclusion we can draw from a study of this nature is that controversial medical topics -- whether the controversy is warranted and valid or not -- brings out a bias in search results as dozens of new sites pop-up to discuss and offer their own viewpoints on the controversy.
Less surprising is the fact that high school students couldn't differentiate an accurate site from an inaccurate site. There is no single indicator (or even set of reliable, research-proven indicators) that lets a user know a site is "trustworthy" and produces only trustworthy information 100% of the time -- even CBS News' article on this was deemed inaccurate by the researchers.
Sadly, the researchers noted no limitations of the current study (and surprisingly, JMIR apparently didn't require the researchers to do so before accepting their article for publication). So I'll list them for you:
Students, too, may be especially naive when it comes to understanding the complexity of health and medical information online, since few have any direct or sustained interaction with health care concerns or treatments. I would expect adults to be a little less naive and a little more skeptical of any information they find on a site that proclaims to offer "Uncensored vaccine information that the government doesn't want you to know!"
Reference: Kortum, P., Edwards, C. & Richards-Kortum, R. (2008). The Impact of Inaccurate Internet Health Information in a Secondary School Learning Environment. J Med Internet Res, 10(2):e17.
by John Grohol, epatient
July 22, 2008
A new study in the Journal of Medical Internet Research (JMIR) suggests that inaccurate medical information is easily found and regurgitated by students. But the topic the study chose to study -- vaccines -- has been under increasing scrutiny and controversy, limiting the generalizability of the results.
The study was simply setup -- 34 high school students were asked to find websites about vaccines, and then rate their accuracy. Of the 40 sites found, 22 were described by the researchers as "inaccurate," including well-known sites such as CBS News. The search term "vaccine dangers" returned virtually all inaccurate search results out of the first 20, while the search term "vaccine safety" returned virtually all accurate search results.
I think the only conclusion we can draw from a study of this nature is that controversial medical topics -- whether the controversy is warranted and valid or not -- brings out a bias in search results as dozens of new sites pop-up to discuss and offer their own viewpoints on the controversy.
Less surprising is the fact that high school students couldn't differentiate an accurate site from an inaccurate site. There is no single indicator (or even set of reliable, research-proven indicators) that lets a user know a site is "trustworthy" and produces only trustworthy information 100% of the time -- even CBS News' article on this was deemed inaccurate by the researchers.
Sadly, the researchers noted no limitations of the current study (and surprisingly, JMIR apparently didn't require the researchers to do so before accepting their article for publication). So I'll list them for you:
- Small sample size
- Unrepresentative sample (not randomized from the general population)
- Students in high school (e.g., results are not generalizable to the internet population as a whole, or adults).
- Controversial health topic in the news that has likely spurred a temporary increase in controversial sites
- Search terms may have been biased (why didn't the researchers include a more generic "vaccines" search term?)
Students, too, may be especially naive when it comes to understanding the complexity of health and medical information online, since few have any direct or sustained interaction with health care concerns or treatments. I would expect adults to be a little less naive and a little more skeptical of any information they find on a site that proclaims to offer "Uncensored vaccine information that the government doesn't want you to know!"
Reference: Kortum, P., Edwards, C. & Richards-Kortum, R. (2008). The Impact of Inaccurate Internet Health Information in a Secondary School Learning Environment. J Med Internet Res, 10(2):e17.
