Skip to contents

Below is a listing of tests available internally within gtsummary.

Tests listed with ... may have additional arguments passed to them using add_p(test.args=). For example, to calculate a p-value from t.test() assuming equal variance, use tbl_summary(trial, by = trt) %>% add_p(age ~ "t.test", test.args = age ~ list(var.equal = TRUE))

tbl_summary() %>% add_p()

aliasdescriptionpseudo-codedetails
't.test't-testt.test(variable ~ as.factor(by), data = data, conf.level = 0.95, ...)
'mood.test'Mood two-sample test of scalemood.test(variable ~ as.factor(by), data = data, ...)Not to be confused with the Brown-Mood test of medians
'oneway.test'One-way ANOVAoneway.test(variable ~ as.factor(by), data = data, ...)
'kruskal.test'Kruskal-Wallis testkruskal.test(data[[variable]], as.factor(data[[by]]))
'wilcox.test'Wilcoxon rank-sum testwilcox.test(as.numeric(variable) ~ as.factor(by), data = data, conf.int = TRUE, conf.level = conf.level, ...)
'chisq.test'chi-square test of independencechisq.test(x = data[[variable]], y = as.factor(data[[by]]), ...)
'chisq.test.no.correct'chi-square test of independencechisq.test(x = data[[variable]], y = as.factor(data[[by]]), correct = FALSE)
'fisher.test'Fisher's exact testfisher.test(data[[variable]], as.factor(data[[by]]), conf.level = 0.95, ...)
'mcnemar.test'McNemar's testtidyr::pivot_wider(id_cols = group, ...); mcnemar.test(by_1, by_2, conf.level = 0.95, ...)
'mcnemar.test.wide'McNemar's testmcnemar.test(data[[variable]], data[[by]], conf.level = 0.95, ...)
'lme4'random intercept logistic regressionlme4::glmer(by ~ (1 \UFF5C group), data, family = binomial) %>% anova(lme4::glmer(by ~ variable + (1 \UFF5C group), data, family = binomial))
'paired.t.test'Paired t-testtidyr::pivot_wider(id_cols = group, ...); t.test(by_1, by_2, paired = TRUE, conf.level = 0.95, ...)
'paired.wilcox.test'Paired Wilcoxon rank-sum testtidyr::pivot_wider(id_cols = group, ...); wilcox.test(by_1, by_2, paired = TRUE, conf.int = TRUE, conf.level = 0.95, ...)
'prop.test'Test for equality of proportionsprop.test(x, n, conf.level = 0.95, ...)
'ancova'ANCOVAlm(variable ~ by + adj.vars)
'emmeans'Estimated Marginal Means or LS-meanslm(variable ~ by + adj.vars, data) %>% emmeans::emmeans(specs =~by) %>% emmeans::contrast(method = "pairwise") %>% summary(infer = TRUE, level = conf.level)When variable is binary, glm(family = binomial) and emmeans(regrid = "response") arguments are used. When group is specified, lme4::lmer() and lme4::glmer() are used with the group as a random intercept.

tbl_svysummary() %>% add_p()

aliasdescriptionpseudo-codedetails
'svy.t.test't-test adapted to complex survey samplessurvey::svyttest(~variable + by, data)
'svy.wilcox.test'Wilcoxon rank-sum test for complex survey samplessurvey::svyranktest(~variable + by, data, test = 'wilcoxon')
'svy.kruskal.test'Kruskal-Wallis rank-sum test for complex survey samplessurvey::svyranktest(~variable + by, data, test = 'KruskalWallis')
'svy.vanderwaerden.test'van der Waerden's normal-scores test for complex survey samplessurvey::svyranktest(~variable + by, data, test = 'vanderWaerden')
'svy.median.test'Mood's test for the median for complex survey samplessurvey::svyranktest(~variable + by, data, test = 'median')
'svy.chisq.test'chi-squared test with Rao & Scott's second-order correctionsurvey::svychisq(~variable + by, data, statistic = 'F')
'svy.adj.chisq.test'chi-squared test adjusted by a design effect estimatesurvey::svychisq(~variable + by, data, statistic = 'Chisq')
'svy.wald.test'Wald test of independence for complex survey samplessurvey::svychisq(~variable + by, data, statistic = 'Wald')
'svy.adj.wald.test'adjusted Wald test of independence for complex survey samplessurvey::svychisq(~variable + by, data, statistic = 'adjWald')
'svy.lincom.test'test of independence using the exact asymptotic distribution for complex survey samplessurvey::svychisq(~variable + by, data, statistic = 'lincom')
'svy.saddlepoint.test'test of independence using a saddlepoint approximation for complex survey samplessurvey::svychisq(~variable + by, data, statistic = 'saddlepoint')
'emmeans'Estimated Marginal Means or LS-meanssurvey::svyglm(variable ~ by + adj.vars, data) %>% emmeans::emmeans(specs =~by) %>% emmeans::contrast(method = "pairwise") %>% summary(infer = TRUE, level = conf.level)When variable is binary, survey::svyglm(family = binomial) and emmeans(regrid = "response") arguments are used.

tbl_survfit() %>% add_p()

aliasdescriptionpseudo-code
'logrank'Log-rank testsurvival::survdiff(Surv(.) ~ variable, data, rho = 0)
'tarone'Tarone-Ware testsurvival::survdiff(Surv(.) ~ variable, data, rho = 1.5)
'petopeto_gehanwilcoxon'Peto & Peto modification of Gehan-Wilcoxon testsurvival::survdiff(Surv(.) ~ variable, data, rho = 1)
'survdiff'G-rho family testsurvival::survdiff(Surv(.) ~ variable, data, ...)
'coxph_lrt'Cox regression (LRT)survival::coxph(Surv(.) ~ variable, data, ...)
'coxph_wald'Cox regression (Wald)survival::coxph(Surv(.) ~ variable, data, ...)
'coxph_score'Cox regression (Score)survival::coxph(Surv(.) ~ variable, data, ...)

tbl_continuous() %>% add_p()

aliasdescriptionpseudo-code
'anova_2way'Two-way ANOVAlm(continuous_variable ~ by + variable)
't.test't-testt.test(continuous_variable ~ as.factor(variable), data = data, conf.level = 0.95, ...)
'oneway.test'One-way ANOVAoneway.test(continuous_variable ~ as.factor(variable), data = data)
'kruskal.test'Kruskal-Wallis testkruskal.test(data[[continuous_variable]], as.factor(data[[variable]]))
'wilcox.test'Wilcoxon rank-sum testwilcox.test(as.numeric(continuous_variable) ~ as.factor(variable), data = data, ...)
'lme4'random intercept logistic regressionlme4::glmer(by ~ (1 \UFF5C group), data, family = binomial) %>% anova(lme4::glmer(variable ~ continuous_variable + (1 \UFF5C group), data, family = binomial))
'ancova'ANCOVAlm(continuous_variable ~ variable + adj.vars)

tbl_summary() %>% add_difference()

aliasdescriptiondifference statisticpseudo-codedetails
't.test't-testmean differencet.test(variable ~ as.factor(by), data = data, conf.level = 0.95, ...)
'wilcox.test'Wilcoxon rank-sum testwilcox.test(as.numeric(variable) ~ as.factor(by), data = data, conf.int = TRUE, conf.level = conf.level, ...)
'paired.t.test'Paired t-testmean differencetidyr::pivot_wider(id_cols = group, ...); t.test(by_1, by_2, paired = TRUE, conf.level = 0.95, ...)
'prop.test'Test for equality of proportionsrate differenceprop.test(x, n, conf.level = 0.95, ...)
'ancova'ANCOVAmean differencelm(variable ~ by + adj.vars)
'ancova_lme4'ANCOVA with random interceptmean differencelme4::lmer(variable ~ by + adj.vars + (1 \UFF5C group), data)
'cohens_d'Cohen's Dstandardized mean differenceeffectsize::cohens_d(variable ~ by, data, ci = conf.level, verbose = FALSE, ...)
'hedges_g'Hedge's Gstandardized mean differenceeffectsize::hedges_g(variable ~ by, data, ci = conf.level, verbose = FALSE, ...)
'paired_cohens_d'Paired Cohen's Dstandardized mean differencetidyr::pivot_wider(id_cols = group, ...); effectsize::cohens_d(by_1, by_2, paired = TRUE, conf.level = 0.95, verbose = FALSE, ...)
'paired_hedges_g'Paired Hedge's Gstandardized mean differencetidyr::pivot_wider(id_cols = group, ...); effectsize::hedges_g(by_1, by_2, paired = TRUE, conf.level = 0.95, verbose = FALSE, ...)
'smd'Standardized Mean Differencestandardized mean differencesmd::smd(x = data[[variable]], g = data[[by]], std.error = TRUE)
'emmeans'Estimated Marginal Means or LS-meansadjusted mean differencelm(variable ~ by + adj.vars, data) %>% emmeans::emmeans(specs =~by) %>% emmeans::contrast(method = "pairwise") %>% summary(infer = TRUE, level = conf.level)When variable is binary, glm(family = binomial) and emmeans(regrid = "response") arguments are used. When group is specified, lme4::lmer() and lme4::glmer() are used with the group as a random intercept.

tbl_svysummary() %>% add_difference()

aliasdescriptiondifference statisticpseudo-codedetails
'smd'Standardized Mean Differencestandardized mean differencesmd::smd(x = variable, g = by, w = weights(data), std.error = TRUE)
'svy.t.test't-test adapted to complex survey samplessurvey::svyttest(~variable + by, data)
'emmeans'Estimated Marginal Means or LS-meansadjusted mean differencesurvey::svyglm(variable ~ by + adj.vars, data) %>% emmeans::emmeans(specs =~by) %>% emmeans::contrast(method = "pairwise") %>% summary(infer = TRUE, level = conf.level)When variable is binary, survey::svyglm(family = binomial) and emmeans(regrid = "response") arguments are used.

Custom Functions

To report a p-value (or difference) for a test not available in gtsummary, you can create a custom function. The output is a data frame that is one line long. The structure is similar to the output of broom::tidy() of a typical statistical test. The add_p() and add_difference() functions will look for columns called "p.value", "estimate", "statistic", "std.error", "parameter", "conf.low", "conf.high", and "method".

You can also pass an Analysis Results Dataset (ARD) object with the results for your custom result. These objects follow the structures outlined by the {cards} and {cardx} packages.

Example calculating a p-value from a t-test assuming a common variance between groups.

ttest_common_variance <- function(data, variable, by, ...) {
  data <- data[c(variable, by)] %>% dplyr::filter(complete.cases(.))
  t.test(data[[variable]] ~ factor(data[[by]]), var.equal = TRUE) %>%
  broom::tidy()
}

trial[c("age", "trt")] %>%
  tbl_summary(by = trt) %>%
  add_p(test = age ~ "ttest_common_variance")

A custom add_difference() is similar, and accepts arguments conf.level= and adj.vars= as well.

ttest_common_variance <- function(data, variable, by, conf.level, ...) {
  data <- data[c(variable, by)] %>% dplyr::filter(complete.cases(.))
  t.test(data[[variable]] ~ factor(data[[by]]), conf.level = conf.level, var.equal = TRUE) %>%
  broom::tidy()
}

Function Arguments

For tbl_summary() objects, the custom function will be passed the following arguments: custom_pvalue_fun(data=, variable=, by=, group=, type=, conf.level=, adj.vars=). While your function may not utilize each of these arguments, these arguments are passed and the function must accept them. We recommend including ... to future-proof against updates where additional arguments are added.

The following table describes the argument inputs for each gtsummary table type.

argumenttbl_summarytbl_svysummarytbl_survfittbl_continuous
data=A data frameA survey objectA survfit() objectA data frame
variable=String variable nameString variable nameNAString variable name
by=String variable nameString variable nameNAString variable name
group=String variable nameNANAString variable name
type=Summary typeSummary typeNANA
conf.level=Confidence interval levelNANANA
adj.vars=Character vector of adjustment variable names (e.g. used in ANCOVA)NANACharacter vector of adjustment variable names (e.g. used in ANCOVA)
continuous_variable=NANANAString of the continuous variable name