ERC IMPROVE

Innovative Methods for Psychology: Reproducible, Open, Valid, and Efficient

With numerous failures to replicate, common misreporting of results, widespread failure to publish non- significant results or to share data, and considerable potential bias due the flexibility of analyses of data and researcher’s tendency to exploit that flexibility, psychological science is said to experience a crisis of confidence. These issues lead to dissemination of false positive results and inflate effect size estimates in meta-analyses. This leads to poor theory building, an inefficient scientific system, a waste of resources, lower trust in psychological science, and psychology’s outcomes being less useful for society. The goal of this ERC project is to improve psychological science by offering novel solutions to five vexing challenges:

1. Countering misreporting of results by using our new tool statcheck in several studies on reviewers’ tendency to demand perfection and by applying it to actual peer review.

2. Countering the biasing effects of common explorations of data (p-hacking) by professing and studying pre-registration and by developing promising new approaches called blind analysis and cross-validation using differential privacy that simultaneously allows for exploration and confirmation with the same data.

3. Countering the common problem of selective outcome reporting in psychological experiments by developing powerful latent variable methods that render it fruitless to not report all outcome variables in a study.

4. Countering the problem of publication bias by studying and correcting misinterpretations of non-significance.

5. Developing and refining meta-analytic methods that allow for the correction of biases that currently inflate estimates of effects and obscure moderation. The innovative tools we develop have the potential to improve the way psychologists (and other scientists) analyse data, disseminate findings, and draw inferences. 

Team

• Jelte M. Wicherts

• Robbie van Aert, PostDoc

• Richard Klein, PostDoc

• Amir M. Abdol, PostDoc

• Esther Maassen, PhD Student

• Olmo van den Akker, PhD Student