Green Tea and Velociraptors

Why should people trust the published scientific record?

Imagine this hypothetical, but potentially very real, conversation with a non-academic:

1: “This research paper has been published, and therefore is scientifically valid.”
2: “But it’s paywalled, I can’t access it. How do I know it’s valid?”
1: “Because it has been peer reviewed.”
2. “Can you show me the peer reviews?”
1: “No. But it was done by two experts in the field.”
2: “Which experts?”
1: “We don’t know. But it’s in a top journal.”
2: “Why is it a top journal?”
1: “Because it has a high impact factor, so is highly cited.”
2: “Why does that make the research better?”
1: “Trust me. I’m a scientist.”

Imagine this is on a topic like climate change, and you can see why this might be an issue. You can play through the same conversation with almost any aspect of scholarly publishing.

1: “This research is based on rigorous data.”
2: “Can you show me the data and how it was collected?”
1: “Well, no. The researchers have not shared it.”
2: “So how do I know the results are valid?”
1: “Because the process is described in the methods.”
2: “But I can’t find the code or software they used to create those results. How do I know it’s valid?”
1: “Trust me. I’m a scientist.”

Trust needs transparency

We have a scholarly communication that is essentially built on trust, but beset on all sides by secrecy, opacity, and barriers like paywalls. The problem here is that it is easy to see how this can lead to an ‘anti-scientific’ skeptical mindset. We ask people to believe what is published, when we have no real way of verifying the process. This goes beyond Open Access too – that just enables people to read research, not to see how or why it should be considered to be verified.

Peer review, data, rigorous analysis – these are the things that are supposed to set science apart from other forms of publishing and knowledge creation. Except overwhelmingly these processes are secretive, and hidden away from us. We really have no idea what happens before a research paper gets published, yet we are simply expected to accept that it is more valid or more verified than any other source.

You can even go a step further. Being ‘pro-science’, as in believing that published research is verified research, requires you to trust the system. You have to take a leap of faith, and that is quite unscientific.

What can we do?

Well, that’s simple:

1. Publish your peer review reports. All of them. Anonymously if needed.
2. Publish your data. Raw and treated. If not, make sure you have a damn good reason not to.
3. Publish methodologies and protocols more clearly, in a stepwise manner if possible. Justify each step.
4. Publish all of your software and code, or at least make sure they are publicly accessible.
5. Publish your work Open Access, of course. (Before you respond with $$ related issues, read this.)
6. Publish your work as a pre-print if possible. Let people see the beginning, the transformation, and the end.

These are just a few of the larger steps, but there is a whole world out there of things you can do. Transparency breeds trust. Opacity breeds skepticism. Science needs the former, badly.

Why does all of this matter?

We are in a global information and knowledge-literacy crisis. That much is undeniable. What the scientific community need to do is come together and work harder to engender public trust in science again. There is simply too much at stake not to commit to this.