Mixed-effects models

For better or worse, the bread and butter of psycholinguistic and other CSD research. Conceptually fantastic, but often tricky to implement well. My suggestion is to start with the first link below, and then pick an introductory paper to read and go through examples (e.g., Brown 2021). Then, if you’re feeling good about things, I highly suggest working through the examples in Debruine & Barr (2021). Then go apply what you’ve learned to your own data. Then, when you’re feeling really accomplished, write it up for publication, incorporating recommendations from Meteyard & Davies (2020) as you write. Or take a class at your institution. Or take one of the workshops linked below.

  • An Introduction to Hierarchical Modeling: http://mfviz.com/hierarchical-models/* (Fantastic visual demonstration)

  • Brown VA. An Introduction to Linear Mixed-Effects Modeling in R. Advances in Methods and Practices in Psychological Science. 2021;4(1). doi:10.1177/2515245920960351

  • DeBruine, L. M., & Barr, D. J. (2021). Understanding mixed-effects models through data simulation. Advances in Methods and Practices in Psychological Science, 4(1), 2515245920965119.

  • Meteyard, L., & Davies, R. A. (2020). Best practice guidance for linear mixed-effects models in psychological science. Journal of Memory and Language, 112, 104092.

  • Winter, B. (2013). Linear models and linear mixed effects models in R with linguistic applications. arXiv preprint arXiv:1308.5499.

Psychometrics

  • Revelle, W. (2009). An introduction to psychometric theory with applications in R. https://personality-project.org/r/book/

  • Ten Hove, D., Jorgensen, T. D., & van der Ark, L. A. (2022). Updated guidelines on selecting an intraclass correlation coefficient for interrater reliability, with applications to incomplete observational designs. Psychological Methods.

  • Ten Hove, D., Jorgensen, T. D., & Van der Ark, L. A. (2025). How to estimate intraclass correlation coefficients for interrater reliability from planned incomplete data. Multivariate Behavioral Research, 1-20.

  • Bürkner, P. C. (2019). Bayesian item response modeling in R with brms and Stan. arXiv preprint arXiv:1905.09501.

  • McNeish, D. (2018). Thanks coefficient alpha, we’ll take it from here. Psychological methods, 23(3), 412.

Bayesian statistics

You should learn how to do Bayesian stats. If you’ve never tried to use bayes, I promise it’s more intuitive than you think. And because of modern computing and R packages like {brms}, its arguably just as easy to implement as frequentist mixed models. If you know how to use lme4, then you’re already 75% of the way to using {brms}.

  • Get Started with Bayesian Analysis: https://easystats.github.io/bayestestR/articles/bayestestR.html*

  • Bayesian Basics: https://m-clark.github.io/bayesian-basics

  • An Introduction to Bayesian Data Analysis for Cognitive Science: https://vasishth.github.io/bayescogsci/book/

  • Vasishth, S., Nicenboim, B., Beckman, M. E., Li, F., & Kong, E. J. (2018). Bayesian data analysis in the phonetic sciences: A tutorial introduction. Journal of phonetics, 71, 147-161.

  • Nalborczyk, L., Batailler, C., Lœvenbruck, H., Vilain, A., & Bürkner, P. C. (2019). An introduction to Bayesian multilevel models using brms: A case study of gender effects on vowel variability in standard Indonesian. Journal of Speech, Language, and Hearing Research, 62(5), 1225-1242

  • Schad, D. J., Betancourt, M., & Vasishth, S. (2021). Toward a principled Bayesian workflow in cognitive science. Psychological methods, 26(1), 103.

  • Statistical Rethinking: https://xcelab.net/rm/statistical-rethinking/ (Statistical Rethinking by Richard McElreath)

Basics of causal inference

  • Regression, Fire, and Dangerous Things by Richard McElreath: https://elevanth.org/blog/2021/06/15/regression-fire-and-dangerous-things-1-3/

  • Rohrer, J. M. (2024). Causal inference for psychologists who think that causal inference is not for them. Social and Personality Psychology Compass, 18(3), e12948.

  • Wysocki AC, Lawson KM, Rhemtulla M. Statistical Control Requires Causal Justification. Advances in Methods and Practices in Psychological Science. 2022;5(2). doi:10.1177/25152459221095823

Longitudinal Data Analysis

  • Fantastic and accessible primer on longitudinal analysis: McCormick, E. M., Byrne, M. L., Flournoy, J. C., Mills, K. L., & Pfeifer, J. H. (2023). The Hitchhiker’s guide to longitudinal models: A primer on model selection for repeated-measures methods. Developmental Cognitive Neuroscience, 63, 101281. doi:10.1016/j.dcn.2023.101281

  • Rohrer JM, Murayama K. These Are Not the Effects You Are Looking for: Causality and the Within-/Between-Persons Distinction in Longitudinal Data Analysis. Advances in Methods and Practices in Psychological Science. 2023;6(1). doi:10.1177/25152459221140842

  • Mirman, D. (2017). Growth curve analysis and visualization using R. Chapman and Hall/CRC.

Mediation Analysis

These resources are intended to be useful to get you started with mediation, but I also recommend carefully reading the first article by Julia Rohrer and colleagues before deciding to undertake a mediation analysis.

  • Rohrer JM, Hünermund P, Arslan RC, Elson M. That’s a Lot to Process! Pitfalls of Popular Path Models. Advances in Methods and Practices in Psychological Science. 2022;5(2). doi:10.1177/25152459221095827

  • Rohrer JM. Thinking Clearly About Correlations and Causation: Graphical Causal Models for Observational Data. Advances in Methods and Practices in Psychological Science. 2018;1(1):27-42. doi:10.1177/2515245917745629

  • Imai, K., Keele, L., Tingley, D., & Yamamoto, T. (2011). Unpacking the black box of causality: Learning about causal mechanisms from experimental and observational studies. American Political Science Review, 105(4), 765-789. https://imai.fas.harvard.edu/research/files/mediationP.pdf

  • Nguyen, T. Q., Schmid, I., & Stuart, E. A. (2021). Clarifying causal mediation analysis for the applied researcher: Defining effects based on what we want to learn. Psychological Methods, 26(2), 255–271. https://doi.org/10.1037/met0000299

  • Kline, R. B. (2015). The Mediation Myth. Basic and Applied Social Psychology, 37(4), 202–213. https://doi.org/10.1080/01973533.2015.1049349

Sample size planning and Statistical Power

If you’re going to read anything on power, read this one

  • Hancock GR, Feng Y. nmax and the quest to restore caution, integrity, and practicality to the sample size planning process. Psychol Methods. 2025 Aug 11. doi: 10.1037/met0000776. Epub ahead of print. PMID: 40788705.

  • Ying, X., Freedland, K. E., Powell, L. H., Stuart, E. A., Ehrhardt, S., & Mayo-Wilson, E. (2025). Determining sample size for pilot trials: a tutorial. bmj, 390.

  • Powering Your Interaction: https://approachingblog.wordpress.com/2018/01/24/powering-your-interaction-2/

Random useful things

  • There is only one test: http://allendowney.blogspot.com/2016/06/there-is-still-only-one-test.html implemented in an R package here: https://infer.netlify.app

  • Contrast Coding: https://debruine.github.io/faux/articles/contrasts.html (At some point you will have questions about contrast coding for categorical variables, and the answer is inevitably going to be here)

  • Reaction time: https://lindeloev.github.io/shiny-rt/

  • Complex survey analysis: Zimmer, S. A., Powell, R. J., & Velásquez, I. C. (2024). Exploring Complex Survey Data Analysis Using R: A Tidy Introduction with {srvyr} and {survey}. Chapman & Hall: CRC Press. https://tidy-survey-r.github.io/tidy-survey-book/

  • Mize, T. D. (2019). Best practices for estimating, interpreting, and presenting nonlinear interaction effects. Sociological Science, 6, 81-117.