Skip to contents

Purpose and measurement scale

HSMD asks how conclusions change when studies use harmonized standardizers. It is a sensitivity analysis alongside conventional SMD, rather than a correction for publication bias or an automatic improvement in effect-size estimation.

For each arm, the coefficient of variation is CV=s/|m|. At a selected benchmark quantile, the harmonized SD is s_q=|m|CV_q. The outcome needs a meaningful ratio-scale origin and nonzero group means. Zero means are rejected; negative means trigger a warning to review the measurement scale. Changing measurement units by a positive multiplier preserves results. Adding a constant to scores can change CVs and HSMD, so arbitrary recentering is not a remedy for zero means. Very small nonzero means can give unstable CVs even without numerical errors; inspect their distribution and substantive plausibility.

The implementation estimates separate treatment and control benchmarks, as in the worked examples of Jung and Aloe (2025). Each available row contributes to its arm’s empirical CV distribution. With multiple effects per study, this gives studies with more rows greater benchmark representation. Use outcome-specific groups or externally justified benchmarks when that weighting is inappropriate.

Start with conventional and harmonized effects

out <- hsmd(hsmd_example_1, mT, sdT, nT, mC, sdC, nC, study=Study)
out$summary
#>    method   component        est         se      ci.lb       ci.ub       tau2
#> 1     SMD (Intercept) -0.3482182 0.10640132 -0.5567610 -0.13967543 0.11743967
#> 2 HSMD_25 (Intercept) -0.5182606 0.22512743 -0.9595023 -0.07701898 0.72841243
#> 3 HSMD_50 (Intercept) -0.2504580 0.10868797 -0.4634825 -0.03743353 0.12560799
#> 4 HSMD_75 (Intercept) -0.1918735 0.08000593 -0.3486822 -0.03506475 0.04819645
#>         avwW       I2 n_effects n_studies
#> 1 0.04191511 60.01581        16        16
#> 2 0.04546883 89.56394        16        16
#> 3 0.04186181 61.64808        16        16
#> 4 0.04132897 38.45138        16        16
out$cv_benchmarks
#> $treatment
#> [1] 0.2854079 0.5067338 0.8359677
#> 
#> $control
#> [1] 0.2166862 0.5257705 0.6481069
out$heterogeneity
#>          method     component       tau2
#> SMD         SMD between-study 0.11743967
#> HSMD_25 HSMD_25 between-study 0.72841243
#> HSMD_50 HSMD_50 between-study 0.12560799
#> HSMD_75 HSMD_75 between-study 0.04819645

Compare effect direction, magnitude, uncertainty, heterogeneity, and analysis counts across SMD and the three benchmark scenarios. A change across scenarios shows sensitivity to the standardizer assumptions. It does not identify which scenario is true. Differences between scenario estimates are not formal tests: the scenarios reuse observations and benchmarks.

The pooled SD is

s_p=\sqrt{\frac{(n_T-1)s_T^2+(n_C-1)s_C^2}{n_T+n_C-2}},\qquad d=(m_T-m_C)/s_p.

The selected variance approximation is v_d=1/n_T+1/n_C+d^2/[2(n_T+n_C)]. For measure="g", J=1-3/[4(n_T+n_C-2)-1], g=Jd, and v_g=J^2v_d. Other SMD software may use exact corrections or different variance approximations. These formulas are applied to the reconstructed SDs for each HSMD scenario. The resulting standard errors and confidence intervals do not propagate CV benchmark uncertainty. Interpret them conditionally within the sensitivity analysis, not as fully calibrated inference for a new estimator.

Custom quantiles and grouped benchmarks

custom <- hsmd_effect_sizes(hsmd_example_1, mT, sdT, nT, mC, sdC, nC,
                           probs=c(.251,.254,.5))
custom$prob_names
#> [1] "25p1" "25p4" "50"

Distinct probabilities retain distinct labels: 25.1% becomes 25p1. Duplicate probabilities are rejected. Grouping is useful when outcomes or domains have different defensible variability distributions. The following artificial groups illustrate the API; choose groups substantively in an actual analysis.

dat <- hsmd_example_1
dat$domain <- rep(c("A","B"), length.out=nrow(dat))
grouped <- hsmd_effect_sizes(dat, mT, sdT, nT, mC, sdC, nC,
                            cv_by=domain, probs=.5)
grouped$cv_benchmarks
#> $treatment
#>        [,1]
#> A 0.5067338
#> B 0.5606226
#> 
#> $control
#>        [,1]
#> A 0.5052311
#> B 0.5257705
fixed <- hsmd_effect_sizes(dat, mT, sdT, nT, mC, sdC, nC,
                          cv_by=domain, probs=.5,
                          cv_benchmarks=grouped$cv_benchmarks)
stopifnot(isTRUE(all.equal(fixed$effect_sizes, grouped$effect_sizes)))

Supplied grouped benchmarks need group names on rows and probabilities in the requested column order. Ungrouped benchmarks are numeric vectors named treatment and control within the supplied list. Benchmarks must be positive and finite. Each automatically estimated group needs observed CVs; supplying benchmarks allows effect-size calculation when SDs are entirely unavailable. Use hsmd_effect_sizes() if insufficient conventional effects prevent pooling.

Missing SDs and analysis counts

incomplete <- hsmd_example_1
incomplete$sdT[1] <- NA_real_
missing_fit <- hsmd(incomplete, mT, sdT, nT, mC, sdC, nC,
                    study=Study, probs=.5)
missing_fit$summary[, c("method","est","n_effects","n_studies")]
#>    method        est n_effects n_studies
#> 1     SMD -0.3198513        15        15
#> 2 HSMD_50 -0.2491451        16        16

The conventional fit has 15 effects; the harmonized fit has 16. Original missing SDs and unrelated missing columns no longer remove computable harmonized effects. Missing model predictors still follow the selected missing policy. Benchmark estimation precedes model-specific moderator filtering, so fitted models remain conditional on that chosen benchmark population. To separate changes in the standardizer from changes in study composition, repeat the analysis on an explicit common subset, ideally using the same supplied benchmarks.

Dependent outcomes

For correlated outcomes from the same participants, select the multivariate structure and provide the assumed within-study outcome correlation r2do. This example duplicates outcomes solely to demonstrate model routing.

multi <- hsmd_example_1[rep(seq_len(nrow(hsmd_example_1)), each=2), ]
multi$Study <- rep(seq_len(nrow(hsmd_example_1)), each=2)
multi$effect <- rep(1:2, nrow(hsmd_example_1))
multi$mT <- multi$mT + ifelse(multi$effect==2, .5, 0)
multi$N <- multi$nT + multi$nC
multi$r2do <- .5
mv <- hsmd(multi, mT, sdT, nT, mC, sdC, nC, study=Study,
           effectID=effect, sample_size=N, structure="DIAG1",
           varcov_type="outcome", measure="g", probs=.5,
           estimation_method="MLE")
mv$summary
#>    method component        est         se      ci.lb       ci.ub tau2
#> 1     SMD         1 -0.3269361 0.07725749 -0.4783580 -0.17551421   NA
#> 2     SMD         2 -0.2556310 0.07809070 -0.4086860 -0.10257607   NA
#> 3 HSMD_50         1 -0.2550980 0.08780626 -0.4271951 -0.08300088   NA
#> 4 HSMD_50         2 -0.1907958 0.08833032 -0.3639200 -0.01767151   NA
#>         avwW I2 n_effects n_studies
#> 1 0.04020362 NA        32        16
#> 2 0.04020362 NA        32        16
#> 3 0.04020291 NA        32        16
#> 4 0.04020291 NA        32        16
mv$heterogeneity
#>            method component       tau2
#> SMD.1         SMD         1 0.04305889
#> SMD.2         SMD         2 0.04507179
#> HSMD_50.1 HSMD_50         1 0.06806320
#> HSMD_50.2 HSMD_50         2 0.06952497

outcome assumes equal arm sample sizes across a study’s outcomes and a common correlation for outcome pairs. Unequal participant overlap needs a richer covariance specification and is rejected by this path. Singleton studies do not need a correlation. Sample sizes come from the explicitly selected nT/nC columns even if stale nt/nc columns exist in the input.

For outcomes i,j on the same participants, the working covariance is \operatorname{Cov}(d_i,d_j)=(1/n_T+1/n_C)r+d_i d_jr^2/[2(n_T+n_C)]. For g, both axes are multiplied by their correction factors, J_iJ_j. The diagonal uses the selected effect’s sampling variance. Non-positive-definite outcome covariance is rejected. varcov_type="multilevel" supplies diagonal sampling covariance; it does not represent correlated sampling errors merely because a random-effects model is fitted.

Multilevel summaries and reporting

Nested models use structure="multilevel" and a formula such as ~ 1 + (1 | district/StudyID), with study=district. Inspect out$heterogeneity for every named random-effect variance. Multivariate and multilevel summary$tau2 and summary$I2 are deliberately unavailable because variance components cannot be assigned to regression coefficients by position. Full model objects retain their covariance matrices and model-specific heterogeneity diagnostics. Inspect optimizer and Hessian warnings before using uncertainty estimates; the HSMD wrapper does not repair failed underlying fits.

A report should state the measurement scale, CV source/grouping, chosen quantiles, d/g convention, covariance assumptions, analysis counts, and changes across scenarios. State that uncertainty is conditional on benchmarks. Refit objects created before these repairs to obtain consistent covariance and summaries.