
Publication-Bias Selection Models and Small-Study Diagnostics
publication_bias.RdFits a normal selection likelihood to an ordinary fitted mars() model.
Estimates describe sensitivity to a specified selection mechanism. A changed
estimate or significant asymmetry test does not establish publication bias.
Usage
publication_bias(
mars_object,
method = c("beta_selection", "beta_binomial", "logistic_selection"),
p_value_tail = c("two.sided", "right", "left"),
n_bins = 10,
p_value_min = NULL,
maxit = 2000,
integration_control = list(),
starts = 3L,
gradient_tol = 1e-04,
level = 0.95,
funnel_plot = FALSE,
funnel_args = list(),
control = list(),
comparison_vcov = c("model", "robust"),
inference = c("model", "sandwich"),
egger_robust_args = list(type = "CR1S", inference = "naive-t"),
...
)Arguments
- mars_object
Ordinary fitted mars object. Penalized fits are unsupported.
- method
beta_selection, beta_binomial or logistic_selection; see Details for their exact weight functions. The names do not imply equivalence to similarly named models in other software.
- p_value_tail
With \(z=y/\mathrm{SE}\) and standard normal CDF \(\Phi\),
two.sideduses \(2\Phi(-|z|)\). When weights decrease with \(p\),rightfavors significant positive effects using \(1-\Phi(z)\), andleftfavors negative effects using \(\Phi(z)\). Prespecify direction.- n_bins
Integer at least three. Beta-binomial bins are \([j/m,(j+1)/m)\), where \(j=0,\ldots,m-1\) and \(m\) is
n_bins. The last bin includes \(p=1\). This controls selection-shape discretization, not the number of free bin parameters. Ignored by the continuous weight functions.- p_value_min
P-value censoring threshold: \(p\) is replaced by \(\min(\max(p,p_{\min}),1-p_{\min})\) in both observed weights and normalizers. NULL uses 1e-5 for beta_selection and 1e-12 otherwise. Beta selection requires a positive value with numerically distinct bounds; other models permit zero. Treat censoring as a sensitivity assumption, not only a numerical setting.
- maxit
Positive integer iteration limit for each optimization start. control$maxit overrides this value.
- integration_control
Named list: rel_tol (default 1e-10) and subdivisions (1000) control scalar adaptive integration; bivariate_rel_tol (1e-9) and bivariate_subdivisions (500) control outer conditional-normal integration. Inner integrals use scalar settings. Tolerances are in (0,1); subdivision limits are positive integers. Beta-binomial scalar integrals use exact normal interval probabilities and do not depend on quadrature controls. Full infinite normal tails are retained. These settings are saved and reused by profiling.
- starts
Integer from one to three. The default tries three deterministic initial selection shapes and heterogeneity values. One uses the parent means, parent positive tau2 (or 0.01 when zero), and shapes (1,1) for beta methods or (0,1) for logistic. Additional beta shapes are (0.5,2) and (2,0.5); logistic starts are (-2,0.5) and (2,2). Estimated initial tau2 is multiplied by 0.5 and 2 in those starts. Valid converged candidates take precedence; the smallest negative log likelihood among them is selected. If none converges, the best finite candidate is returned with unavailable Wald inference.
- gradient_tol
Positive stationarity threshold, default 1e-4. The diagnostic is max(abs(gradient))/(1+abs(negative_log_likelihood)), where numerical study scores give the gradient on the optimizer scale. Failing it suppresses Wald inference even when optim reports convergence. It is sensitive to units and parameterization and does not prove a global optimum.
- level
Confidence level in (0,1), default 0.95, for adjusted and unadjusted coefficient intervals in the comparison. Robust unadjusted intervals retain the parent's normal/t reference and coefficient-specific degrees of freedom.
- funnel_plot
Logical; draw a funnel plot after fitting.
- funnel_args
Uniquely named list of arguments passed to
funnel_plot().- control
Uniquely named BFGS controls: trace, REPORT, fnscale, parscale, ndeps, maxit, abstol and reltol. Positive fnscale retains minimization; parscale and ndeps must be positive vectors matching optimizer parameters. These control optimization, not quadrature.
- comparison_vcov
model uses the parent model covariance and normal intervals; robust uses its stored cluster-robust covariance and reference distribution, requiring a parent fit with robustID. This does not change the selection-model fit or its inference setting.
- inference
model uses the inverse observed Hessian; sandwich uses independent study score contributions. Nonconvergence, failed stationarity or invalid curvature produces unavailable SEs and Wald tests.
- egger_robust_args
Named robust covariance controls for the separate Egger-type test. Defaults to CR1S with a G-1 t reference. Other controls follow
mars()robust_args. This does not choose selection-model inference.- ...
Unsupported arguments are rejected.
Value
A pub_bias object. coefficients contains adjusted estimates, SEs, normal-reference z tests and p-values; adjusted_vs_unadjusted contains intervals, signed differences, percent change relative to abs(unadjusted), direction changes and interval overlap. Overlap is descriptive, not a test of the difference. selection contains shape parameters and estimated tau2 only when re-estimated. covariance_mode records covariance treatment; integration_control records quadrature settings. optimization_diagnostics stores each start's objective, convergence and selection, numerical gradient, stationarity, extreme nuisance parameters and near-zero estimated tau2. model_data stores aligned effects, sampling SEs, censored p-values, relative observed weights and mean predictions. egger_test is a separate diagnostic. summary() reports estimates, comparison, optimization and inference status.
Details
For study g, the selected density is $$f_{sel}(y_g)=\phi(y_g;X_g\beta,V_g) \prod_j w(p_{gj}) / E_{\phi}\{\prod_j w(p_{gj})\}.$$ Studies are independent. Within a study the product of outcome-specific weights defines the joint selection mechanism, while V_g retains dependence. This is not a general model of publication based on any one significant outcome, selective outcome omission or an unknown decision rule. Missing outcomes handled by the parent are not modeled as selective missingness. P-values test a zero effect using sampling SEs, without subtracting the fitted mean or including heterogeneity in the SE. Every reported and fitted p-value uses the same positive sampling variance, without a variance floor.
beta_selection uses \(w(p)=p^{\alpha-1}(1-p)^{\beta-1}\), with \(\alpha,\beta>0\). Setting \(\alpha=\beta=1\) gives constant selection. The shape may be nonmonotone. beta_binomial uses the bin weights $$w_j=\binom{m}{j}\frac{B(j+\alpha,m-j+\beta)}{B(\alpha,\beta)},$$ where \(m=n_{\mathrm{bins}}-1\). Setting \(\alpha=\beta=1\) yields constant bin weights. These are relative weights with a beta-binomial shape; their sum-one normalization does not make them publication probabilities or observed p-value-bin probabilities. logistic_selection uses \(w(p)=\operatorname{plogis}(a-b\log p)\), with \(b>0\). It is decreasing in p; b approaching zero gives constant weights and can weaken identification of a. This two-parameter function differs from metafor's model named logistic. The beta kernel agrees with the Citkowicz–Vevea formulation when censoring, tails, data and covariance assumptions match. Beta-binomial and logistic functions here are specified sensitivity models, not interchangeable defaults.
Absolute weight scale cancels from the density. Relative displayed weights cannot identify the fraction of unpublished studies. Fitting uses log weights and normalizers. Independent covariance components factor exactly. Connected components of size two use deterministic conditional-normal integration; larger components are unsupported. Do not discard correlations to bypass this limit. Bivariate fitting can be expensive, particularly with three starts.
Univariate constant heterogeneity is re-estimated jointly by ML unless the parent fixed tau2. Fixed tau2, including zero, is preserved. Multivariate, multilevel and scale-location models condition on fitted parent covariance; scale coefficients and covariance parameters are not re-estimated. Their estimation uncertainty is omitted. Observation-local sampling matrices retain observation order; outcome-category indices select heterogeneity separately. Inconsistent covariance metadata are rejected. Inputs and numerical predictors should be expressed on a well-conditioned scale.
Means are unrestricted. Positive nuisance parameters are optimized on log scales. Estimated tau2 approaches zero but is not represented as an exact zero boundary. Multiple starts and a zero optim code do not guarantee a global maximum. Absolute log nuisance values exceeding 20 and estimated tau2 below 1e-8 are descriptive flags, not formal identifiability tests. Inspect curvature, start agreement, gradient, profiles and changes under tighter quadrature and alternative censoring thresholds; changed censoring fits are distinct models.
Model covariance is \(H^{-1}\) for the negative log likelihood. Sandwich covariance is \(H^{-1}(\sum_g s_g s_g')H^{-1}\), with no small-sample correction and more studies than parameters required. Both require identifiable positive curvature and use asymptotic normal Wald inference. Failure is reported as unavailable inference, not a zero SE. No general finite-sample coverage is guaranteed.
Egger fits y/SE on an asymmetry intercept and the parent mean design divided by SE, without extra inverse-variance weighting. With a mean intercept only, this equals raw-effect regression on sampling SE with inverse-variance weights. Moderators are retained and inference uses the parent's stored robustID cluster when available, otherwise studyID. A rank-deficient design or invalid inference makes the check unavailable. Diagnostics flag fewer than ten studies and little SE variation. Ten studies is a rule of thumb, not a guarantee. Effect-SE coupling can create artefactual asymmetry, particularly for SMDs and odds ratios; assess suitability rather than automatically treating Egger as a bias test. Nonsignificance does not exclude missing evidence. See the Cochrane Handbook, Chapter 13.
References
Citkowicz, M., & Vevea, J. L. (2017). A parsimonious weight function for modeling publication bias. Psychological Methods, 22(1), 28-41. doi:10.1037/met0000119 .
Examples
# \donttest{
fit <- mars(
data = teacher_expectancy,
studyID = "study",
effectID = NULL,
sample_size = NULL,
formula = yi ~ 1,
variance = "vi",
varcov_type = "univariate",
structure = "univariate"
)
bias_fit <- publication_bias(fit, method = "beta_selection")
summary(bias_fit)
#> Publication Bias Model (MARS)
#> Method: beta_selection
#> Structure: univariate
#> Effects: 19 | Studies: 19
#> P-value tail: two.sided
#> Convergence: 0 - Converged
#> Objective: -0.1457
#>
#> Optimization starts:
#> start objective convergence valid selected message
#> 1 1.2326066 0 TRUE FALSE
#> 2 1.2326174 0 TRUE FALSE
#> 3 -0.1457375 0 TRUE TRUE
#> Scaled gradient: 0.000179 | Stationary: FALSE
#> Covariance mode: publication_bias_tau2
#> Comparison interval level: 0.95
#>
#> Selection Parameters:
#> parameter estimate
#> alpha 1.6081
#> beta 1.2475
#> tau2 0.6713
#>
#> Selection Weight Profile (p-value grid):
#> model p_value weight
#> beta_selection 0.0100 0.1019
#> beta_selection 0.1189 0.4461
#> beta_selection 0.2278 0.6411
#> beta_selection 0.3367 0.7831
#> beta_selection 0.4456 0.8882
#> beta_selection 0.5544 0.9611
#> beta_selection 0.6633 1.0000
#> beta_selection 0.7722 0.9957
#> beta_selection 0.8811 0.9185
#> beta_selection 0.9900 0.5343
#>
#> Bias-Adjusted Coefficients:
#> term estimate std_error z_value p_value
#> (Intercept) 1.4677 NA NA NA
#>
#> Adjusted vs Unadjusted Summary:
#> term unadjusted_estimate unadjusted_se unadjusted_ci_lower
#> (Intercept) 0.0837 0.0516 -0.0175
#> unadjusted_ci_upper estimate std_error adjusted_ci_lower adjusted_ci_upper
#> 0.1849 1.4677 NA NA NA
#> estimate_diff percent_change direction_changed ci_overlap
#> 1.384 1653.516 FALSE NA
#>
#> Small-Study Effects Check (Egger-type):
#> Model: SND ~ 1 + mean_design / SE
#> Method: Egger-type small-study-effects regression with study-cluster robust SE (CR1S, naive-t)
#> Intercept: 1.6243 SE: 0.786 t: 2.0666 df: 18 p: 0.0535
#>
#> SE note:
#> Wald inference unavailable: numerical stationarity check failed. Wald inference uses an asymptotic normal reference; finite-sample coverage is not guaranteed.
# }