A method for calculating the repeated-measures correlation (r_rm or “rmcorr” from here) using a within-person design [1] has gained widespread use, particularly because it (1) provides a single correlation coefficient summarizing all within-person correlations, and (2) remains within the framework of the general linear model, meaning it can be calculated entirely from a table of sums of squared errors (an ANOVA table). To calculate the Bland–Altman rmcorr between X1 and X2, one can simply predict (using regression) X1 using X2 and a series of binary (“dummy”) variables representing each subject; for example, if there were 10 subjects, X1 would be predicted by X2 and nine binary variables (with one binary subject variable omitted as the reference category). The partial sums of squares (SS) of this model (usually presented in an ANOVA table) would then be broken down into portions for each independent variable, as well as for the error (residual SS). To find the rmcorr, one would simply divide the SS for the independent variable of interest (X2 in this case) by the sum of that value and the residual SS. The square root of this quotient is the rmcorr. Note the SS for the subjects (binary variables) is ignored, which is by design: the variance attributable to each person’s unique mean is considered a “nuisance” in this context. This procedure is quite similar to giving subjects random intercepts in a mixed-effects regression, though there are important differences [2]. The problem with the Bland–Altman approach is that subjects usually do not have the same slope, so estimating random intercepts while assuming the same slope for everyone is often misleading. Of course, reporting a separate correlation for each person defeats the purpose of having a single correlation coefficient to describe the within-person correlation patterns within a sample; therefore, any method summarizing within-person correlations in a single number will have to contend with varying slopes.

The method proposed here is motivated primarily by potential real-world applications of the Bland–Altman rmcorr. From a scientific perspective, the p-value obtained from the original method comes from a known distribution of a test statistic given some degrees of freedom, so it will always be “correct” in that sense, i.e., the probability of obtaining the rmcorr result by chance would be the p-value already offered by the method presented in the original Bland–Altman paper [1]. However, because statistical significance is often used as a threshold to indicate whether a model should be applied in the real world, we hope we have demonstrated how some significant rmcorr models could be applied unreasonably. Using the original data as an example, the significant unadjusted p-value might lead some to conclude that if someone’s PaCO₂ levels changed by one standardized unit, his/her pH would decrease by 0.507 standardized units. While this might be true on average, the results in Figure 1 demonstrate that the variability in expected pH is enormous; indeed, two of the eight subjects show an increase in pH, one quite substantially. Our alternative suggestion, wmcorr, provides a reasonable alternative to the Bland–Altman [1] method. Our hope is that cases in which the original and weighted average methods provide quite different results will alert researchers to data patterns that warrant closer inspection.

While wmcorr is a viable addition to the “correlation toolbox”, several limitations should be considered. First, our simulation conditions, though varied, do not capture all possible real-world data scenarios, particularly those with extreme outliers or highly non-linear relationships. Second, the method’s reliance on calculating individual correlations means it requires at least three observations per person to compute a correlation coefficient, potentially limiting its application in studies with sparse sampling or irregular measurement intervals. Third, the square root weighting approach, while theoretically justified, is just one possible weighting method; alternative schemes might be more appropriate in certain contexts but were not explored in this study. Fourth, while the examples used in the present study are compelling, they are likely to be quite rare in practice. Indeed, something as extreme as that shown in Figure 2 would likely only occur if there were serious problems in data entry or cleaning. However, this is also one of the strengths of the method: it will detect such problems when they might otherwise go undetected. Finally, while our method can identify potentially problematic data patterns, it does not provide specific guidance on how to address such issues beyond suggesting further data collection or inspection.