Skip to contents

Intended question: is there disagreement across treatment designs anywhere in the network? Assess whether treatment means can be described by a coherent network after accounting for the specified sampling covariance and treatment heterogeneity. Available for all three joint network interfaces.

Usage

network_design_incoherence(
  object,
  method = c("wald", "conditional", "ML"),
  adjust = c("none", "holm", "bonferroni"),
  local_covariance = c("gls", "observed")
)

# S3 method for class 'nma_design_incoherence'
print(
  x,
  ...,
  digits = getOption("digits"),
  estimable_only = TRUE,
  detailed = FALSE
)

# S3 method for class 'nma_design_incoherence'
summary(
  object,
  ...,
  digits = getOption("digits"),
  estimable_only = TRUE,
  detailed = FALSE
)

Arguments

object

A fitted joint network model from network_meta() or network_meta_coherent(); for summary(), a result returned by network_design_incoherence().

method

wald (default) for the Higgins/White design-by-treatment Wald test with heterogeneity refitted under the alternative by REML; conditional for a Wald test with fitted covariance fixed; or ML for a likelihood-ratio test with both models refitted by ML.

adjust

Multiplicity correction for the local design-deviation tests: none, holm, or bonferroni. All requested pairs form the family, including nonestimable hypotheses. The global test is separate and remains unadjusted.

local_covariance

Covariance for local symmetric side-splitting Wald tests: gls (default) conditions on fitted heterogeneity; observed uses the mean-coefficient block of the inverse joint observed information, including heterogeneity uncertainty, to reproduce Stata's network sidesplit default calculation (White, 2015). observed requires method="wald". It affects local SEs, confidence limits, Wald statistics and p-values; gamma estimates, heterogeneity fits and the global test are unchanged.

x

A result from network_design_incoherence().

...

Additional print arguments forwarded to print.data.frame(); reserved additional arguments for summary.

digits

Decimal places to display, an integer from 1 to 22. Defaults to getOption("digits"); stored values retain full precision. Variance-component labels are preformatted diagnostic strings. P-values below 0.0001 print as <0.0001; other p-values use at least four decimal places.

estimable_only

Logical; show only estimable tests (default TRUE).

detailed

Logical; show all diagnostic columns (default FALSE). Combine with estimable_only=FALSE to inspect unavailable tests and reasons.

Value

A list of class nma_design_incoherence containing:

global

One-row table with statistic, df, p_value, and estimable.

local

The same fields for each treatment pair, plus comparison. A local row can jointly test several mean-stratum parameters. p_adjusted contains multiplicity-adjusted p-values. For one estimable coefficient, gamma, se_gamma, ci_lower, and ci_upper give the signed departure, its model-based SE and pointwise 95% interval; contrast states direction.

local_coefficients

One row per pair and mean stratum, including gamma, se_gamma, ci_lower, ci_upper, and estimable. Unavailable coefficients have missing estimates.

local_covariance

Requested local coefficient covariance: gls or observed. Fixed heterogeneity makes the two calculations identical.

adjustment

Local multiplicity method, family description and size.

method, assumption

Descriptions of the calculation and interpretation.

With method="wald" or method="ML", test tables also contain null_heterogeneity, alternative_heterogeneity, convergence codes (zero indicates convergence), and reason. Scalar variance components are labelled; unstructured matrices are flattened in column-major basic-effect coordinate order. These strings are summaries, not confidence intervals. Nonestimable tests have estimable=FALSE and missing statistics/p-values; this is different from failing to reject a null. Printing shows the global test and estimable local tests, with shared settings reported once. Constant local df is printed above the results. With no local adjustment, the duplicate adjusted-p column is omitted. Alternative-model heterogeneity, convergence codes and nonestimability reasons remain in the returned tables. Use print(result, detailed=TRUE, estimable_only=FALSE) to inspect all diagnostic fields and rows. Display filtering does not change the multiplicity family. summary(result, digits=3) remembers display preferences; explicit print arguments override them. No stored values are rounded.

Details

Printed output states the research question and global null hypothesis that all identifiable design-by-treatment interaction parameters are zero. The local section separately states that each pair's identifiable targeted deviation parameters are zero under the symmetric side-splitting model (White, 2015).

The null mean model is \(E(y_i)=X_i d\). The alternative adds columns \(W_i\gamma\); the null hypothesis is that all identifiable added coefficients are zero. Covariance remains sampling covariance plus the selected signed random-treatment covariance. Only columns increasing the mean-design rank are retained. Thus df is the number of independently testable added parameters, not the number of graphical triangles.

Global test. A study design is the set of treatments observed in that study, across its rows. The function interacts design indicators with the basic treatment-effect design Z. Mean strata receive separate interactions when specified. Existing moderator slopes remain in the mean model; this extension does not add design-specific moderator slopes. The joint test assesses between-design disagreement. Within-study multi-arm contrast identities are preserved, so a triangle supplied solely by one common multi-arm design does not create an independent incoherence parameter.

Local tests. For each unordered treatment pair A/B, eligible designs contain both treatments. Treatment B is perturbed by +1/2 and A by -1/2 in those studies; other treatments are unchanged. The perturbation of a contrast is the second treatment's shift minus the first's. One added column is constructed per mean stratum. These are White's (2015, section 3.6, Table 1) symmetric side-splitting tests, preserving multi-arm identities. They are not independent direct/indirect splits, and a significant result does not uniquely identify a problematic comparison or study. Qin et al. (2024, equation 3) describe the symmetric side-splitting model and its restrictions relative to the full design-by-treatment interaction model.

The local mean model is \(E(y)=Xd+w_{AB}\gamma_{AB}\). The sign of gamma follows treatment B minus A, shown in contrast. For a single estimable coefficient, local reports gamma, se_gamma, ci_lower, and ci_upper. The intervals are pointwise 95% normal Wald intervals, regardless of adjust or the primary fit's confidence level. With augmented matrix A and fitted total covariance V, GLS coefficient covariance is \((A^T V^{-1} A)^{-1}\). Its added coefficient block supplies gamma's covariance with local_covariance="gls"; heterogeneity estimation uncertainty is then not additionally propagated. For multiple mean strata, local_coefficients reports each retained gamma separately while local retains the joint test. Scalar gamma fields are missing when more than one coefficient is tested. A likelihood-ratio test still reports Wald coefficient intervals, which need not agree with its p-value.

Refitted Wald calculation. Fit the augmented design-by-treatment mean model by REML and jointly test its identifiable added coefficients using their model covariance and a chi-squared reference. Heterogeneity is estimated under this alternative, unless fixed through tau2 or heterogeneity="none". This follows Higgins et al. and White et al. (2012). Failure to identify or estimate the alternative is reported as nonestimable, rather than as agreement.

Observed-information local covariance. With method="wald" and local_covariance="observed", let theta contain the free heterogeneity covariance parameters, V=V(theta), and A the augmented local mean matrix. The joint observed information is the Hessian, with respect to both beta and theta, of $$\tfrac12[\log|V|+(y-A\beta)^T V^{-1}(y-A\beta)+\log|A^T V^{-1}A|].$$ It is evaluated at the REML heterogeneity estimate and its GLS beta estimate. The beta block of its inverse supplies the local coefficient covariance, retaining the cross-information between mean and heterogeneity parameters. This is not the Hessian of the profiled REML criterion with respect to beta. gls instead inverts only the mean-information block. The observed option is model-based and asymptotic, not a small-sample correction or a sandwich variance. Pointwise 95% intervals still use normal quantiles and Holm still adjusts only local p-values. White's Stata mvmeta default includes heterogeneity uncertainty; its nouncertainv option describes the alternative conditional calculation.

The observed calculation supports the fitted linear variance-component kernels and interior unstructured heterogeneity covariance. Fixed tau2 and heterogeneity="none" have no estimated variance parameters, so both covariance options reduce to GLS. An estimated variance on a boundary, singular information, or non-positive-definite joint information makes the local test unavailable: estimable=FALSE and reason explains why. No automatic GLS fallback or positive-definiteness repair is performed. Use detailed=TRUE and estimable_only=FALSE to inspect such rows. The default global GLS Wald test is unaffected; conditional and ML tests reject the observed option explicitly.

Conditional calculation. Refit the augmented mean by generalized least squares using the original fitted total covariance. Compute \(Q=\hat\gamma^T\operatorname{Var}(\hat\gamma)^{-1}\hat\gamma\) and use a chi-squared reference distribution with df degrees of freedom. This conditions on fitted heterogeneity and does not propagate its estimation uncertainty.

ML calculation. Refit both null and alternative using ML, even when the primary fit used REML. Calculate \(D=2(\ell_{alternative}-\ell_{null})\) and compare with chi-squared(df). Heterogeneity is re-estimated unless the original model fixed it through tau2 or heterogeneity="none". REML likelihoods are not compared across different mean designs. Alternative optimization or identifiability failures are marked in the output.

Interpretation. A small p-value indicates evidence against the specified coherent mean model, conditional on its covariance and effect-modifier assumptions. It does not measure the size or clinical importance of disagreement. Use network_nodesplit() to examine direct-minus-indirect effects and intervals, and network_influence() to investigate sensitivity. Large p-values can reflect low power; they do not establish coherence or transitivity. Global and local reference distributions are asymptotic. Local p-values are not multiplicity adjusted unless adjust is supplied, and tests use model-based covariance even for a primary CR1 fit. Local tests overlap: their statistics and df cannot be summed to obtain within-design heterogeneity. Use network_q_decomposition().

References

Higgins, J. P. T., Jackson, D., Barrett, J. K., Lu, G., Ades, A. E., and White, I. R. (2012). Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies. Research Synthesis Methods, 3, 98–110. doi:10.1002/jrsm.1044 .

White, I. R., Barrett, J. K., Jackson, D., and Higgins, J. P. T. (2012). Consistency and inconsistency in network meta-analysis: model estimation using multivariate meta-regression. Research Synthesis Methods, 3, 111–125. doi:10.1002/jrsm.1045 .

White, I. R. (2015). Network meta-analysis. The Stata Journal, 15(4), 951–985. Section 3.6, Table 1. doi:10.1177/1536867X1501500403 .

Qin, L., Zhao, S., Guo, W., Tong, T., and Yang, K. (2024). A comparison of two models for detecting inconsistency in network meta-analysis. Research Synthesis Methods, 15(6), 851–871. doi:10.1002/jrsm.1734 .

Examples

 d <- data.frame(study=1:9, a=rep(c("A","A","B"),3),
   b=rep(c("B","C","C"),3), effect=rep(c(.2,.6,.2),3), variance=.04)
 fit <- network_meta(d,"study","a","b","effect","variance")
 network_design_incoherence(fit)$global
#>   statistic df   p_value estimable null_heterogeneity alternative_heterogeneity
#> 1         1  1 0.3173105      TRUE            study 0                   study 0
#>   null_convergence alternative_convergence reason
#> 1                0                       0       
 network_design_incoherence(fit,method="ML")$local
#>   statistic df   p_value estimable null_heterogeneity alternative_heterogeneity
#> 1         1  1 0.3173105      TRUE            study 0                   study 0
#> 2         1  1 0.3173105      TRUE            study 0                   study 0
#> 3         1  1 0.3173105      TRUE            study 0                   study 0
#>   null_convergence alternative_convergence reason comparison contrast gamma
#> 1                0                       0            A vs B    B - A  -0.2
#> 2                0                       0            A vs C    C - A   0.2
#> 3                0                       0            B vs C    C - B  -0.2
#>   se_gamma   ci_lower  ci_upper p_adjusted
#> 1      0.2 -0.5919928 0.1919928  0.3173105
#> 2      0.2 -0.1919928 0.5919928  0.3173105
#> 3      0.2 -0.5919928 0.1919928  0.3173105
 network_design_incoherence(fit,local_covariance="observed")$local
#>   statistic df   p_value estimable null_heterogeneity alternative_heterogeneity
#> 1         1  1 0.3173105      TRUE            study 0                   study 0
#> 2         1  1 0.3173105      TRUE            study 0                   study 0
#> 3         1  1 0.3173105      TRUE            study 0                   study 0
#>   null_convergence alternative_convergence reason comparison contrast gamma
#> 1                0                       0            A vs B    B - A  -0.2
#> 2                0                       0            A vs C    C - A   0.2
#> 3                0                       0            B vs C    C - B  -0.2
#>   se_gamma   ci_lower  ci_upper p_adjusted
#> 1      0.2 -0.5919928 0.1919928  0.3173105
#> 2      0.2 -0.1919928 0.5919928  0.3173105
#> 3      0.2 -0.5919928 0.1919928  0.3173105