Parallel Session 2 (14:45 - 16:15)
Location: CZ8
Session leader: Peder Isager & Marcel van Assen
Session name: How can meta-research improve the Psychological Science Accelerator (PSA) and how can the PSA improve meta-research?
Abstract:
The Psychological Science Accelerator (PSA) is a standing network of more than 500 laboratories that collect large-scale, non-WEIRD data for psychology studies (https://psysciacc.org). The PSA is continually working to adopt best practices in (multi lab) research, and is actively seeking to create policy and collect data for the benefit of researchers in the meta-research community.
In this session, Marcel van Assen and Peder Isager will discuss the importance of many lab studies and provide an overview of PSA’s organization, including all completed and ongoing projects. A large part of the session will be interactive and be devoted to how meta-research and PSA may advance each other. Specifically, how can meta-research facilitate the submission, review, construction, design, and analysis of the many lab studies of PSA, and how can the results of these many lab studies advance meta-research?
The goal is to write a blog post based on the ideas raised in the session, with the possibility of extending this into a paper.
Location: CZ9
Session leader: Ana Marušic
Session name: How can meta-research improve peer review?
Abstract:
Relative to studies regarding other parts of the research cycle studies on peer review are limited in quantity and scope. The main reason for that is the anonymity of the current system where peer reviews are done behind closed journal doors and we thus have limited information about the peer review process. In this session, we will discuss how we can get more information from peer reviewed journals and specifically focus on the question: What characteristics of the peer review (e.g., type of peer review, gender, language of the reviews, field of research) influence the editorial decision?
Location: CZ109
Session leader: Sophia Crüwell & Leo Tiokhin
Session name: How can meta-research improve our understanding of the effects of incentives on the efficiency and reliability of science?
Abstract:
Everyone’s talking about “the incentives”, but what does that mean? How can we move beyond our intuitions and towards a deeper understanding of how incentives affect the efficiency and reliability of science? In this session, Sophia Crüwell, Leonid Tiokhin and Maia Salholz-Hillel will explore the role of incentives in science, with the goal of facilitating a broader discussion of what important questions remain unanswered. Specific questions that we would like to discuss include: "What is the evidence about how specific incentives affectscientists’ behavior and the efficiency and reliability of science? What are the gaps in our knowledge and what research is still needed?”, "How can we identify the most effective points for interventions to our current incentive system?”, and “To what extent do we want to incentivise people doing e.g. replications?”.
Location: CZ118
Session leader: Hans van Dijk and Esther Maassen
Session name: Many Paths: A new way to communicate, discuss, and conduct (meta-) research
Abstract:
To improve the way in which research is currently conducted and communicated, Hartgerink and van Zelst (2018) have recently suggested a new research communication infrastructure. In their infrastructure, research output is communicated continuously “as-you-go” as opposed to the current system where research is only communicated after the entire cycle has been completed (i.e., “after-the-fact”). This “as-you-go” system offers a host of advantages, including more transparency and a more rapid dissemination of research output.
We have started a meta-research project called “Many Paths” to investigate the viability of their proposed infrastructure. In Many Paths, we starts with one research question (i.e. “Does power corrupt?) and allow an emergent process to occur thereafter. Given that results are often path dependent and many paths can be taken in a research process, we aim to examine what paths a research project initiated, pruned, and merged. The Many Paths model offers insight into how researchers from different disciplines approach and study the same research question in a different way, and how that may relate to different conclusions being reached. As such, the project offers insight into the value of the “as-you-go” form of research communication.
The aims of our session are twofold: First, given that the Many Paths project is in the initiation stage, we welcome any feedback and suggestion that you may have to improve the project. Second, we would like to invite you to join the Many Paths project.