NWO Veni Grant for the 4-Step Robustness Check

We are happy to announce that a €250,000 NWO Veni Grant was awarded to Michèle Nuijten for her proposed 4-Step Robustness Check.

She describes the project on her website:

To check the robustness of a study could replicate it in a new sample. However, in my 4-Step Robustness Check, you first verify if the reported numbers in the original study are correct. If they’re not, they are not interpretable and you can’t compare them to the results of your replication.

Specifically, I advise researchers to do the following:

1. Check if there are visible errors in the reported numbers, for example by running a paper through my spellchecker for statistics: statcheck

2. Reanalyze the data following the original strategy to see if this leads to the same numbers

3. Check if the result is robust to alternative analytical choices

4. Perform a replication study in a new sample

The 4-Step Robustness Check can be used to efficiently assess robustness of results

This 4-step check provides an efficient framework to check if a study’s findings are robust. Note that the first steps take way less time than a full replication and might be enough to conclude a result is not robust.

The proposed framework can also be used as an efficient checklist for researchers to improve robustness of their own results:

1. Check the internal consistency of your reported results

2. Share your data and analysis scripts to facilitate reanalysis

3. Conduct and report your own sensitivity analyses

4. Write detailed methods sections and share materials to facilitate replication

Ultimately, I aim to create interactive, pragmatic, and evidence-based methods to improve and assess robustness, applicable to psychology and other fields.

I would like to wholeheartedly thank my colleagues, reviewers, and committee members for their time, feedback, and valuable insights. I’m looking forward to the next three years!