Checking Claims in Science Reporting

In the Guardian on Saturday, Ben Goldacre's column was called "how to read a paper" and suggested three checks that you can do when presented with scientific claims. These are:

1. Check the numbers - If there is sufficient data in the article then you should redo the mathematics yourself, especially where statistics and chance are concerned because the subtleties in relating these areas of mathematics to the real world mean that these are the areas that most people get wrong most often.
2. Check the publications - The claims made should be supported by published, peer-reviewed research. If this is not the case then serious questions must be asked of the claims such as, is the claimant trying to sell me something? A lot of science is communicated to the general public not directly from the scientists but from the press offices of Universities or other research establishments either as a way to increase their esteem, to attract a better quality of researchers, or to provide buzz for a new spin-put venture. As a result, quite often the claims made via press release are quite divorced from the claims made in the underlying research. The reader must therefore go back to the underlying research publications to get a clear understanding of the claim and it's veracity. At this point I must make a plea for all science reporting to include citations to the academic literature where possible. That said, all university press releases should also include reference to underlying research as well.
3. Check the wider field - If there is an established concensus and the claims seek to overturn this concensus then there is a burden of proof associated with the new claims that has to be satisfied.

One problem with this approach is that checking claim one requires mathematical ability, and checking claims two and three requires the ability to read and understand academic writing. Reading academic papers is a skill that is not easy to pick up and generally takes most new Ph.D students their first year to really start getting the hang of. Partly this is due to the "academic style" that most papers are written in, and partly because scientific knowledge as gained from academic publications is more akin to a reef of information with each part supporting many other parts. To be able to read and completely understand one paper in a cutting edge area of science may necessitate the reading and understanding of tens or even hundreds of earlier papers that the current paper bullds upon. That said, science is not easy and most people are not scientists. This is not because they couldn't be, they just don't, as a rule, have the necessary background to verify scientific claims for themselves. Rather, the general public must trust the messenger. This is why trust is such an important aspect of science communication and why public engagement with science and scientists is so important. If the public are to take on trust what scientists say, so as to avoid that whole training in science so that they can do it themselves thing, then scientists must do what they can to build that trust. This can include research blogging or science blogging coming off campus once in a while and talking about what they do in normal everyday language with people who are at least sufficiently interested in finding out more to have turned up to a science centre or event.