Analysis Results Datasets (ARDs) in R

Daniel D. Sjoberg

Questions

Please ask questions at any time!

What are ARDs?

• Dataset that stores key metadata and raw results from analysis

• Long dataset that is 1 record per result value

• May contained formatted values as well

• The ARD can be used to to subsequently create tables and figures.

• The ARD does not describe the layout of the results

Analysis Results Data (ARD)

• After the initial creation of an ARD, the results can later be re-used again and again for subsequent reporting needs.

A few notes about ARDs

• ARDs give us the opportunity to rethink QC

  • QC can be focused on the raw value, as well as the formatted display

    • You don’t have to waste time trying to match formatting to match QC

• ARDs can be flexibly saved to different file types

  • For example: a dataset (rds, xpt, etc) or json file

Zooming Out: The Analysis Results Standard (ARS)

Objectives include:

• To leverage analysis results metadata to drive the automation of results
• To support storage, access, processing, traceability and reproducibility of results
• Learn more at https://www.cdisc.org/events/webinar/analysis-results-standard-public-review

Proposed Metadata Framework in the ARS

Proposed Metadata Framework in the ARS

• The ARS provides a metadata-driven infrastructure for analysis

Proposed Metadata Framework in the ARS

• The ARS provides a metadata-driven infrastructure for analysis

• {cards} serves as the engine for the analysis

ARDs using {cards}

{cards}: Introduction

• Part of the Pharmaverse

• Collaboration between Roche, GSK, Novartis, Eli Lilly, Pfizer, and Clymb Clinical

• Contains a variety of utilities for making ARDs

• Can be used within the ARS workflow and separately

• 51K downloads per month 🤯

Data used in examples

ADSL from pharmaverseadam

adsl <- pharmaverseadam::adsl |> 
  dplyr::filter(SAFFL=="Y") |> 
  dplyr::mutate(ARM2 = ifelse(startsWith(ARM, "Xanomeline"), "Xanomeline", ARM))

ADAE from pharmaverseadam

adae <- pharmaverseadam::adae |> 
  dplyr::filter(SAFFL=="Y") |> 
  dplyr::mutate(ARM2 = ifelse(startsWith(ARM, "Xanomeline"), "Xanomeline", ARM)) |> 
  dplyr::filter(AESOC %in% unique(AESOC)[1:3]) |> 
  dplyr::group_by(AESOC) |> 
  dplyr::filter(AEDECOD %in% unique(AEDECOD)[1:3]) |> 
  dplyr::ungroup()

{cards}: ard_tabulate()

• includes n, %, N by default
• Any unobserved levels of the variables will be present in the resulting ARD.

library(cards)

adsl |> 
  ard_tabulate(
    variables = AGEGR1
  ) 

# An ARD data frame: 6 × 9
  variable variable_level context  stat_name stat_label    stat fmt_fun warning error 
* <chr>    <list>         <chr>    <chr>     <chr>       <list> <list>  <list>  <list>
1 AGEGR1   18-64          tabulate n         n           33     0       <NULL>  <NULL>
2 AGEGR1   18-64          tabulate N         N          254     0       <NULL>  <NULL>
3 AGEGR1   18-64          tabulate p         %            0.130 <fn>    <NULL>  <NULL>
4 AGEGR1   >64            tabulate n         n          221     0       <NULL>  <NULL>
5 AGEGR1   >64            tabulate N         N          254     0       <NULL>  <NULL>
6 AGEGR1   >64            tabulate p         %            0.870 <fn>    <NULL>  <NULL>

{cards}: ard_tabulate()

• includes n, %, N by default
• Any unobserved levels of the variables will be present in the resulting ARD.

adsl |> 
  ard_tabulate(
    by = ARM2,
    variables = AGEGR1
  ) 

# An ARD data frame: 12 × 11
   group1 group1_level variable variable_level context  stat_name stat_label    stat fmt_fun warning error 
 * <chr>  <list>       <chr>    <list>         <chr>    <chr>     <chr>       <list> <list>  <list>  <list>
 1 ARM2   Placebo      AGEGR1   18-64          tabulate n         n           14     0       <NULL>  <NULL>
 2 ARM2   Placebo      AGEGR1   18-64          tabulate N         N           86     0       <NULL>  <NULL>
 3 ARM2   Placebo      AGEGR1   18-64          tabulate p         %            0.163 <fn>    <NULL>  <NULL>
 4 ARM2   Placebo      AGEGR1   >64            tabulate n         n           72     0       <NULL>  <NULL>
 5 ARM2   Placebo      AGEGR1   >64            tabulate N         N           86     0       <NULL>  <NULL>
 6 ARM2   Placebo      AGEGR1   >64            tabulate p         %            0.837 <fn>    <NULL>  <NULL>
 7 ARM2   Xanomeline   AGEGR1   18-64          tabulate n         n           19     0       <NULL>  <NULL>
 8 ARM2   Xanomeline   AGEGR1   18-64          tabulate N         N          168     0       <NULL>  <NULL>
 9 ARM2   Xanomeline   AGEGR1   18-64          tabulate p         %            0.113 <fn>    <NULL>  <NULL>
10 ARM2   Xanomeline   AGEGR1   >64            tabulate n         n          149     0       <NULL>  <NULL>
11 ARM2   Xanomeline   AGEGR1   >64            tabulate N         N          168     0       <NULL>  <NULL>
12 ARM2   Xanomeline   AGEGR1   >64            tabulate p         %            0.887 <fn>    <NULL>  <NULL>

{cards}: ard_summary()

# create ARD with default summary statistics
adsl |> 
  ard_summary(
    variables = AGE
  )

# An ARD data frame: 8 × 8
  variable context stat_name stat_label   stat fmt_fun warning error 
* <chr>    <chr>   <chr>     <chr>      <list>  <list> <list>  <list>
1 AGE      summary N         N          254          0 <NULL>  <NULL>
2 AGE      summary mean      Mean        75.1        1 <NULL>  <NULL>
3 AGE      summary sd        SD           8.25       1 <NULL>  <NULL>
4 AGE      summary median    Median      77          1 <NULL>  <NULL>
5 AGE      summary p25       Q1          70          1 <NULL>  <NULL>
6 AGE      summary p75       Q3          81          1 <NULL>  <NULL>
7 AGE      summary min       Min         51          1 <NULL>  <NULL>
8 AGE      summary max       Max         89          1 <NULL>  <NULL>

{cards}: ard_summary() by variable

by: summary statistics are calculated by all combinations of the by variables, including unobserved factor levels

adsl |> 
  ard_summary(
    variables = AGE,
    by = ARM2         # stats by treatment arm
  ) 

# An ARD data frame: 16 × 10
   group1 group1_level variable context stat_name stat_label   stat fmt_fun warning error 
 * <chr>  <list>       <chr>    <chr>   <chr>     <chr>      <list>  <list> <list>  <list>
 1 ARM2   Placebo      AGE      summary N         N           86          0 <NULL>  <NULL>
 2 ARM2   Placebo      AGE      summary mean      Mean        75.2        1 <NULL>  <NULL>
 3 ARM2   Placebo      AGE      summary sd        SD           8.59       1 <NULL>  <NULL>
 4 ARM2   Placebo      AGE      summary median    Median      76          1 <NULL>  <NULL>
 5 ARM2   Placebo      AGE      summary p25       Q1          69          1 <NULL>  <NULL>
 6 ARM2   Placebo      AGE      summary p75       Q3          82          1 <NULL>  <NULL>
 7 ARM2   Placebo      AGE      summary min       Min         52          1 <NULL>  <NULL>
 8 ARM2   Placebo      AGE      summary max       Max         89          1 <NULL>  <NULL>
 9 ARM2   Xanomeline   AGE      summary N         N          168          0 <NULL>  <NULL>
10 ARM2   Xanomeline   AGE      summary mean      Mean        75.0        1 <NULL>  <NULL>
11 ARM2   Xanomeline   AGE      summary sd        SD           8.09       1 <NULL>  <NULL>
12 ARM2   Xanomeline   AGE      summary median    Median      77          1 <NULL>  <NULL>
13 ARM2   Xanomeline   AGE      summary p25       Q1          71          1 <NULL>  <NULL>
14 ARM2   Xanomeline   AGE      summary p75       Q3          81          1 <NULL>  <NULL>
15 ARM2   Xanomeline   AGE      summary min       Min         51          1 <NULL>  <NULL>
16 ARM2   Xanomeline   AGE      summary max       Max         88          1 <NULL>  <NULL>

{cards}: ard_summary() statistics

statistic: specify univariate summary statistics. Accepts any function, base R, from a package, or user-defined.

cv <- function(x)  sd(x, na.rm = TRUE)/mean(x, na.rm = TRUE)

adsl |> 
  ard_summary(  
    variables = AGE,
    by = ARM2,
    statistic = ~ list(cv = cv) # customize statistics
  )

# An ARD data frame: 2 × 10
  group1 group1_level variable context stat_name stat_label   stat fmt_fun warning error 
* <chr>  <list>       <chr>    <chr>   <chr>     <chr>      <list>  <list> <list>  <list>
1 ARM2   Placebo      AGE      summary cv        cv          0.114       1 <NULL>  <NULL>
2 ARM2   Xanomeline   AGE      summary cv        cv          0.108       1 <NULL>  <NULL>

{cards}: ard_summary() statistics

Customize the statistics returned for each variable

adsl |> 
  dplyr::mutate(AGE2 = AGE) |> 
  ard_summary(
    variables = c(AGE, AGE2),
    by = ARM2,
    statistic = list(AGE = list(cv = cv),
                     AGE2 = continuous_summary_fns(c("mean","median")))
  )

# An ARD data frame: 6 × 10
  group1 group1_level variable context stat_name stat_label   stat fmt_fun warning error 
* <chr>  <list>       <chr>    <chr>   <chr>     <chr>      <list>  <list> <list>  <list>
1 ARM2   Placebo      AGE      summary cv        cv          0.114       1 <NULL>  <NULL>
2 ARM2   Placebo      AGE2     summary mean      Mean       75.2         1 <NULL>  <NULL>
3 ARM2   Placebo      AGE2     summary median    Median     76           1 <NULL>  <NULL>
4 ARM2   Xanomeline   AGE      summary cv        cv          0.108       1 <NULL>  <NULL>
5 ARM2   Xanomeline   AGE2     summary mean      Mean       75.0         1 <NULL>  <NULL>
6 ARM2   Xanomeline   AGE2     summary median    Median     77           1 <NULL>  <NULL>

{cards}: ard_summary() fmt_fun

• Override the default formatting functions
• Can also update later via update_ard_fmt_fun()

adsl |> 
  ard_summary(
    variables = AGE,
    by = ARM2,                               
    fmt_fun = ~list(mean = 0)                
  ) |> 
  apply_fmt_fun() # add a character column of rounded results

# An ARD data frame: 16 × 11
   group1 group1_level variable context stat_name stat_label   stat stat_fmt fmt_fun warning error 
   <chr>  <list>       <chr>    <chr>   <chr>     <chr>      <list> <list>    <list> <list>  <list>
 1 ARM2   Placebo      AGE      summary N         N           86    86             0 <NULL>  <NULL>
 2 ARM2   Placebo      AGE      summary mean      Mean        75.2  75             0 <NULL>  <NULL>
 3 ARM2   Placebo      AGE      summary sd        SD           8.59 8.6            1 <NULL>  <NULL>
 4 ARM2   Placebo      AGE      summary median    Median      76    76.0           1 <NULL>  <NULL>
 5 ARM2   Placebo      AGE      summary p25       Q1          69    69.0           1 <NULL>  <NULL>
 6 ARM2   Placebo      AGE      summary p75       Q3          82    82.0           1 <NULL>  <NULL>
 7 ARM2   Placebo      AGE      summary min       Min         52    52.0           1 <NULL>  <NULL>
 8 ARM2   Placebo      AGE      summary max       Max         89    89.0           1 <NULL>  <NULL>
 9 ARM2   Xanomeline   AGE      summary N         N          168    168            0 <NULL>  <NULL>
10 ARM2   Xanomeline   AGE      summary mean      Mean        75.0  75             0 <NULL>  <NULL>
11 ARM2   Xanomeline   AGE      summary sd        SD           8.09 8.1            1 <NULL>  <NULL>
12 ARM2   Xanomeline   AGE      summary median    Median      77    77.0           1 <NULL>  <NULL>
13 ARM2   Xanomeline   AGE      summary p25       Q1          71    71.0           1 <NULL>  <NULL>
14 ARM2   Xanomeline   AGE      summary p75       Q3          81    81.0           1 <NULL>  <NULL>
15 ARM2   Xanomeline   AGE      summary min       Min         51    51.0           1 <NULL>  <NULL>
16 ARM2   Xanomeline   AGE      summary max       Max         88    88.0           1 <NULL>  <NULL>

{cards}: Other Summary Functions

• ard_tabulate_value(): similar to ard_tabulate(), but for dichotomous tabulations

• ard_hierarchical(): similar to ard_tabulate(), but built for nested tabulations, e.g. AE terms within SOC

• ard_mvsummary(): similar to ard_summary(), for multivariate summaries. The function accepts other arguments like the full and subsetted (within the by groups) data sets.

• ard_missing(): tabulates rates of missingness

The results from all these functions are entirely compatible with one another, and can be stacked into a single data frame. 🥞

{cards}: Other Functions

In addition to exporting functions to prepare summaries, {cards} exports many utilities for wrangling ARDs and creating new ARDs.

Constructing: bind_ard(), tidy_as_ard(), nest_for_ard(), check_ard_structure(), and many more

Wrangling: get_ard_statistics(), replace_null_statistic(), etc.

{cards}: Stacking utilities

• data and .by are shared by all ard_* calls

• Additional Options .overall, .missing, .attributes, and .total_n provide even more results

• By default, summaries of the .by variable are included

adsl |> 
  ard_stack( 
    .by = ARM2,      
    ard_summary(variables = AGE, statistic = ~ continuous_summary_fns(c("mean","sd"))), 
    ard_tabulate(variables = AGEGR1, statistic = ~ "p")
  )  

# An ARD data frame: 14 × 11
   group1 group1_level variable variable_level context  stat_name stat_label    stat fmt_fun warning error 
   <chr>  <list>       <chr>    <list>         <chr>    <chr>     <chr>       <list> <list>  <list>  <list>
 1 ARM2   Placebo      AGE      <NULL>         summary  mean      Mean        75.2   1       <NULL>  <NULL>
 2 ARM2   Placebo      AGE      <NULL>         summary  sd        SD           8.59  1       <NULL>  <NULL>
 3 ARM2   Placebo      AGEGR1   18-64          tabulate p         %            0.163 <fn>    <NULL>  <NULL>
 4 ARM2   Placebo      AGEGR1   >64            tabulate p         %            0.837 <fn>    <NULL>  <NULL>
 5 ARM2   Xanomeline   AGE      <NULL>         summary  mean      Mean        75.0   1       <NULL>  <NULL>
 6 ARM2   Xanomeline   AGE      <NULL>         summary  sd        SD           8.09  1       <NULL>  <NULL>
 7 ARM2   Xanomeline   AGEGR1   18-64          tabulate p         %            0.113 <fn>    <NULL>  <NULL>
 8 ARM2   Xanomeline   AGEGR1   >64            tabulate p         %            0.887 <fn>    <NULL>  <NULL>
 9 <NA>   <NULL>       ARM2     Placebo        tabulate n         n           86     0       <NULL>  <NULL>
10 <NA>   <NULL>       ARM2     Placebo        tabulate N         N          254     0       <NULL>  <NULL>
11 <NA>   <NULL>       ARM2     Placebo        tabulate p         %            0.339 <fn>    <NULL>  <NULL>
12 <NA>   <NULL>       ARM2     Xanomeline     tabulate n         n          168     0       <NULL>  <NULL>
13 <NA>   <NULL>       ARM2     Xanomeline     tabulate N         N          254     0       <NULL>  <NULL>
14 <NA>   <NULL>       ARM2     Xanomeline     tabulate p         %            0.661 <fn>    <NULL>  <NULL>

Quick recap!

• Let’s compute summaries for a demography table that includes age (AGE), age group (AGEGR1), and sex (SEX) by treatment (ARM2)
• First, we compute the continuous summaries for AGE by ARM2

ard_summary(
  data = adsl,
  by = ,
  variables = 
)

Quick recap!

• Let’s compute summaries for a demography table that includes age (AGE), age group (AGEGR1), and sex (SEX) by treatment (ARM2)
• First, we compute the continuous summaries for AGE by ARM2

ard_summary(
  data = adsl,
  by = ARM2,
  variables = AGE 
)

# An ARD data frame: 16 × 10
   group1 group1_level variable context stat_name stat_label   stat fmt_fun warning error 
 * <chr>  <list>       <chr>    <chr>   <chr>     <chr>      <list>  <list> <list>  <list>
 1 ARM2   Placebo      AGE      summary N         N           86          0 <NULL>  <NULL>
 2 ARM2   Placebo      AGE      summary mean      Mean        75.2        1 <NULL>  <NULL>
 3 ARM2   Placebo      AGE      summary sd        SD           8.59       1 <NULL>  <NULL>
 4 ARM2   Placebo      AGE      summary median    Median      76          1 <NULL>  <NULL>
 5 ARM2   Placebo      AGE      summary p25       Q1          69          1 <NULL>  <NULL>
 6 ARM2   Placebo      AGE      summary p75       Q3          82          1 <NULL>  <NULL>
 7 ARM2   Placebo      AGE      summary min       Min         52          1 <NULL>  <NULL>
 8 ARM2   Placebo      AGE      summary max       Max         89          1 <NULL>  <NULL>
 9 ARM2   Xanomeline   AGE      summary N         N          168          0 <NULL>  <NULL>
10 ARM2   Xanomeline   AGE      summary mean      Mean        75.0        1 <NULL>  <NULL>
11 ARM2   Xanomeline   AGE      summary sd        SD           8.09       1 <NULL>  <NULL>
12 ARM2   Xanomeline   AGE      summary median    Median      77          1 <NULL>  <NULL>
13 ARM2   Xanomeline   AGE      summary p25       Q1          71          1 <NULL>  <NULL>
14 ARM2   Xanomeline   AGE      summary p75       Q3          81          1 <NULL>  <NULL>
15 ARM2   Xanomeline   AGE      summary min       Min         51          1 <NULL>  <NULL>
16 ARM2   Xanomeline   AGE      summary max       Max         88          1 <NULL>  <NULL>

Quick recap!

• Let’s compute summaries for a demography table that includes age (AGE), age group (AGEGR1), and sex (SEX) by treatment (ARM2)
• Next, we compute the categorical summaries for AGEGR1 and SEX by ARM2

ard_tabulate(
  data = adsl,
  by = ,
  variables = 
)

Quick recap!

• Let’s compute summaries for a demography table that includes age (AGE), age group (AGEGR1), and sex (SEX) by treatment (ARM2)
• Next, we compute the categorical summaries for AGEGR1 and SEX by ARM2

ard_tabulate(
  data = adsl,
  by = ARM2,
  variables = c(AGEGR1, SEX) 
)

# An ARD data frame: 24 × 11
   group1 group1_level variable variable_level context  stat_name stat_label   stat fmt_fun warning error 
 * <chr>  <list>       <chr>    <list>         <chr>    <chr>     <chr>      <list> <list>  <list>  <list>
 1 ARM2   Placebo      AGEGR1   18-64          tabulate n         n          14     0       <NULL>  <NULL>
 2 ARM2   Placebo      AGEGR1   18-64          tabulate N         N          86     0       <NULL>  <NULL>
 3 ARM2   Placebo      AGEGR1   18-64          tabulate p         %           0.163 <fn>    <NULL>  <NULL>
 4 ARM2   Placebo      AGEGR1   >64            tabulate n         n          72     0       <NULL>  <NULL>
 5 ARM2   Placebo      AGEGR1   >64            tabulate N         N          86     0       <NULL>  <NULL>
 6 ARM2   Placebo      AGEGR1   >64            tabulate p         %           0.837 <fn>    <NULL>  <NULL>
 7 ARM2   Placebo      SEX      F              tabulate n         n          53     0       <NULL>  <NULL>
 8 ARM2   Placebo      SEX      F              tabulate N         N          86     0       <NULL>  <NULL>
 9 ARM2   Placebo      SEX      F              tabulate p         %           0.616 <fn>    <NULL>  <NULL>
10 ARM2   Placebo      SEX      M              tabulate n         n          33     0       <NULL>  <NULL>
# ℹ 14 more rows

Quick recap!

Let’s compute summaries for a demography table that includes age (AGE), age group (AGEGR1), and sex (SEX) by treatment (ARM2) in a single ard_stack() call, including:

• summaries by ARM2 as performed above

• continuous summaries from part A for AGE

• categorical summaries from part B for AGEGR1 and SEX

ard_stack(
  data = adsl,
  .by = ARM2,
  
  # add ard_* calls here
  
)

Quick recap!

Let’s compute summaries for a demography table that includes age (AGE), age group (AGEGR1), and sex (SEX) by treatment (ARM2) in a single ard_stack() call, including:

• summaries by ARM2 as performed above

• continuous summaries from part A for AGE

• categorical summaries from part B for AGEGR1 and SEX

ard_stack(
  data = adsl,
  .by = ARM2,
  ard_summary(variables = AGE),
  ard_tabulate(variables = c(AGEGR1, SEX))
)

# An ARD data frame: 46 × 11
   group1 group1_level variable variable_level context  stat_name stat_label   stat fmt_fun warning error 
   <chr>  <list>       <chr>    <list>         <chr>    <chr>     <chr>      <list>  <list> <list>  <list>
 1 ARM2   Placebo      AGE      <NULL>         summary  N         N           86          0 <NULL>  <NULL>
 2 ARM2   Placebo      AGE      <NULL>         summary  mean      Mean        75.2        1 <NULL>  <NULL>
 3 ARM2   Placebo      AGE      <NULL>         summary  sd        SD           8.59       1 <NULL>  <NULL>
 4 ARM2   Placebo      AGE      <NULL>         summary  median    Median      76          1 <NULL>  <NULL>
 5 ARM2   Placebo      AGE      <NULL>         summary  p25       Q1          69          1 <NULL>  <NULL>
 6 ARM2   Placebo      AGE      <NULL>         summary  p75       Q3          82          1 <NULL>  <NULL>
 7 ARM2   Placebo      AGE      <NULL>         summary  min       Min         52          1 <NULL>  <NULL>
 8 ARM2   Placebo      AGE      <NULL>         summary  max       Max         89          1 <NULL>  <NULL>
 9 ARM2   Placebo      AGEGR1   18-64          tabulate n         n           14          0 <NULL>  <NULL>
10 ARM2   Placebo      AGEGR1   18-64          tabulate N         N           86          0 <NULL>  <NULL>
# ℹ 36 more rows

Quick recap!

We can also add:

• Overall summaries for all variables
• Total N

ard_stack(
  data = adsl,
  .by = ARM2,
  ard_summary(variables = AGE),
  ard_tabulate(variables = c(AGEGR1, SEX)),
  .overall = TRUE,
  .total_n = TRUE
)

# An ARD data frame: 67 × 11
   group1 group1_level variable variable_level context  stat_name stat_label   stat fmt_fun warning error 
   <chr>  <list>       <chr>    <list>         <chr>    <chr>     <chr>      <list>  <list> <list>  <list>
 1 ARM2   Placebo      AGE      <NULL>         summary  N         N           86          0 <NULL>  <NULL>
 2 ARM2   Placebo      AGE      <NULL>         summary  mean      Mean        75.2        1 <NULL>  <NULL>
 3 ARM2   Placebo      AGE      <NULL>         summary  sd        SD           8.59       1 <NULL>  <NULL>
 4 ARM2   Placebo      AGE      <NULL>         summary  median    Median      76          1 <NULL>  <NULL>
 5 ARM2   Placebo      AGE      <NULL>         summary  p25       Q1          69          1 <NULL>  <NULL>
 6 ARM2   Placebo      AGE      <NULL>         summary  p75       Q3          82          1 <NULL>  <NULL>
 7 ARM2   Placebo      AGE      <NULL>         summary  min       Min         52          1 <NULL>  <NULL>
 8 ARM2   Placebo      AGE      <NULL>         summary  max       Max         89          1 <NULL>  <NULL>
 9 ARM2   Placebo      AGEGR1   18-64          tabulate n         n           14          0 <NULL>  <NULL>
10 ARM2   Placebo      AGEGR1   18-64          tabulate N         N           86          0 <NULL>  <NULL>
# ℹ 57 more rows

{cards}: Hierarchical Summary Functions

Following hierarchical summary functions aid in nested tabulations (e.g. AE terms within SOC):

• ard_hierarchical(): calculating nested subject-level rates

• ard_hierarchical_count(): calculating nested event-level counts

{cards}: ard_hierarchical

This function specializes in calculating subject-level rates.

• Rates computed on lowest level variables, nested within others

• id helps to check that no duplicate rows exist within the c(id, variables) columns

• denominator dictates the denominator for the rates

adae |> 
  dplyr::slice_tail(n = 1L, by = c(USUBJID, ARM, AESOC, AEDECOD)) |> 
  ard_hierarchical(
    variables = c(AESOC, AEDECOD),
    by = TRT01A,
    id = USUBJID,
    denominator = adsl
  )

# An ARD data frame: 81 × 13
   group1 group1_level group2 group2_level                                         variable variable_level            context      stat_name stat_label    stat fmt_fun warning error 
   <chr>  <list>       <chr>  <list>                                               <chr>    <list>                    <chr>        <chr>     <chr>       <list> <list>  <list>  <list>
 1 TRT01A Placebo      AESOC  GASTROINTESTINAL DISORDERS                           AEDECOD  DIARRHOEA                 hierarchical n         n           9      0       <NULL>  <NULL>
 2 TRT01A Placebo      AESOC  GASTROINTESTINAL DISORDERS                           AEDECOD  DIARRHOEA                 hierarchical N         N          86      0       <NULL>  <NULL>
 3 TRT01A Placebo      AESOC  GASTROINTESTINAL DISORDERS                           AEDECOD  DIARRHOEA                 hierarchical p         %           0.105  <fn>    <NULL>  <NULL>
 4 TRT01A Placebo      AESOC  GASTROINTESTINAL DISORDERS                           AEDECOD  HIATUS HERNIA             hierarchical n         n           1      0       <NULL>  <NULL>
 5 TRT01A Placebo      AESOC  GASTROINTESTINAL DISORDERS                           AEDECOD  HIATUS HERNIA             hierarchical N         N          86      0       <NULL>  <NULL>
 6 TRT01A Placebo      AESOC  GASTROINTESTINAL DISORDERS                           AEDECOD  HIATUS HERNIA             hierarchical p         %           0.0116 <fn>    <NULL>  <NULL>
 7 TRT01A Placebo      AESOC  GASTROINTESTINAL DISORDERS                           AEDECOD  VOMITING                  hierarchical n         n           3      0       <NULL>  <NULL>
 8 TRT01A Placebo      AESOC  GASTROINTESTINAL DISORDERS                           AEDECOD  VOMITING                  hierarchical N         N          86      0       <NULL>  <NULL>
 9 TRT01A Placebo      AESOC  GASTROINTESTINAL DISORDERS                           AEDECOD  VOMITING                  hierarchical p         %           0.0349 <fn>    <NULL>  <NULL>
10 TRT01A Placebo      AESOC  GENERAL DISORDERS AND ADMINISTRATION SITE CONDITIONS AEDECOD  APPLICATION SITE ERYTHEMA hierarchical n         n           3      0       <NULL>  <NULL>
# ℹ 71 more rows

{cards}: ard_hierarchical_count

This function specializes in calculating event-level frequencies.

adae |> 
  ard_hierarchical_count( 
    variables = c(AESOC, AEDECOD),
    by = TRT01A
  )

# An ARD data frame: 27 × 13
   group1 group1_level         group2 group2_level                                         variable variable_level            context            stat_name stat_label   stat fmt_fun warning error 
   <chr>  <list>               <chr>  <list>                                               <chr>    <list>                    <chr>              <chr>     <chr>      <list>  <list> <list>  <list>
 1 TRT01A Placebo              AESOC  GASTROINTESTINAL DISORDERS                           AEDECOD  DIARRHOEA                 hierarchical_count n         n              10       0 <NULL>  <NULL>
 2 TRT01A Placebo              AESOC  GASTROINTESTINAL DISORDERS                           AEDECOD  HIATUS HERNIA             hierarchical_count n         n               2       0 <NULL>  <NULL>
 3 TRT01A Placebo              AESOC  GASTROINTESTINAL DISORDERS                           AEDECOD  VOMITING                  hierarchical_count n         n               3       0 <NULL>  <NULL>
 4 TRT01A Placebo              AESOC  GENERAL DISORDERS AND ADMINISTRATION SITE CONDITIONS AEDECOD  APPLICATION SITE ERYTHEMA hierarchical_count n         n               3       0 <NULL>  <NULL>
 5 TRT01A Placebo              AESOC  GENERAL DISORDERS AND ADMINISTRATION SITE CONDITIONS AEDECOD  APPLICATION SITE PRURITUS hierarchical_count n         n              10       0 <NULL>  <NULL>
 6 TRT01A Placebo              AESOC  GENERAL DISORDERS AND ADMINISTRATION SITE CONDITIONS AEDECOD  FATIGUE                   hierarchical_count n         n               2       0 <NULL>  <NULL>
 7 TRT01A Placebo              AESOC  SKIN AND SUBCUTANEOUS TISSUE DISORDERS               AEDECOD  ERYTHEMA                  hierarchical_count n         n              13       0 <NULL>  <NULL>
 8 TRT01A Placebo              AESOC  SKIN AND SUBCUTANEOUS TISSUE DISORDERS               AEDECOD  PRURITUS                  hierarchical_count n         n              11       0 <NULL>  <NULL>
 9 TRT01A Placebo              AESOC  SKIN AND SUBCUTANEOUS TISSUE DISORDERS               AEDECOD  PRURITUS GENERALISED      hierarchical_count n         n               0       0 <NULL>  <NULL>
10 TRT01A Xanomeline High Dose AESOC  GASTROINTESTINAL DISORDERS                           AEDECOD  DIARRHOEA                 hierarchical_count n         n               3       0 <NULL>  <NULL>
# ℹ 17 more rows

Hierarchical summaries many ways

• Displays for hierarchical data typically report on each level of the hierarchy (Any AE Overall, by System Organ Class, by Preferred Term)

• This can mean several calls to the ard_hierarchical_* functions

• Further, subject-level summaries require a different subset of the data each time. For example, to calculate Overall rates, we need to subset to 1 record per subject in ADAE.

• Is there an easier way?

{cards}: Stacking function for ard_hierarchical()

• ard_hierarchical stacking functions simplify this multi-step process into a single step

• The id argument is used to subset the data along the way

adae |> 
  ard_stack_hierarchical(
    variables = c(AESOC, AEDECOD),
    by = TRT01A,
    id = USUBJID,
    denominator = pharmaverseadam::adsl
  )

# An ARD data frame: 12 × 13
   group1 group1_level group2 group2_level               variable variable_level             context      stat_name stat_label    stat fmt_fun warning error 
   <chr>  <list>       <chr>  <list>                     <chr>    <list>                     <chr>        <chr>     <chr>       <list> <list>  <list>  <list>
 1 TRT01A Placebo      <NA>   <NULL>                     AESOC    GASTROINTESTINAL DISORDERS hierarchical n         n          12      0       <NULL>  <NULL>
 2 TRT01A Placebo      <NA>   <NULL>                     AESOC    GASTROINTESTINAL DISORDERS hierarchical N         N          86      0       <NULL>  <NULL>
 3 TRT01A Placebo      <NA>   <NULL>                     AESOC    GASTROINTESTINAL DISORDERS hierarchical p         %           0.140  <fn>    <NULL>  <NULL>
 4 TRT01A Placebo      AESOC  GASTROINTESTINAL DISORDERS AEDECOD  DIARRHOEA                  hierarchical n         n           9      0       <NULL>  <NULL>
 5 TRT01A Placebo      AESOC  GASTROINTESTINAL DISORDERS AEDECOD  DIARRHOEA                  hierarchical N         N          86      0       <NULL>  <NULL>
 6 TRT01A Placebo      AESOC  GASTROINTESTINAL DISORDERS AEDECOD  DIARRHOEA                  hierarchical p         %           0.105  <fn>    <NULL>  <NULL>
 7 TRT01A Placebo      AESOC  GASTROINTESTINAL DISORDERS AEDECOD  HIATUS HERNIA              hierarchical n         n           1      0       <NULL>  <NULL>
 8 TRT01A Placebo      AESOC  GASTROINTESTINAL DISORDERS AEDECOD  HIATUS HERNIA              hierarchical N         N          86      0       <NULL>  <NULL>
 9 TRT01A Placebo      AESOC  GASTROINTESTINAL DISORDERS AEDECOD  HIATUS HERNIA              hierarchical p         %           0.0116 <fn>    <NULL>  <NULL>
10 TRT01A Placebo      AESOC  GASTROINTESTINAL DISORDERS AEDECOD  VOMITING                   hierarchical n         n           3      0       <NULL>  <NULL>
11 TRT01A Placebo      AESOC  GASTROINTESTINAL DISORDERS AEDECOD  VOMITING                   hierarchical N         N          86      0       <NULL>  <NULL>
12 TRT01A Placebo      AESOC  GASTROINTESTINAL DISORDERS AEDECOD  VOMITING                   hierarchical p         %           0.0349 <fn>    <NULL>  <NULL>

{cards}: Stacking function for ard_hierarchical_count()

• Below is the stacking function for event-level summaries, aligned with ard_hierarchical_count()

adae |> 
  ard_stack_hierarchical_count(
    variables = c(AESOC, AEDECOD),
    by = TRT01A, 
    denominator = pharmaverseadam::adsl
  )

# An ARD data frame: 4 × 13
  group1 group1_level group2 group2_level               variable variable_level             context            stat_name stat_label   stat fmt_fun warning error 
  <chr>  <list>       <chr>  <list>                     <chr>    <list>                     <chr>              <chr>     <chr>      <list>  <list> <list>  <list>
1 TRT01A Placebo      <NA>   <NULL>                     AESOC    GASTROINTESTINAL DISORDERS hierarchical_count n         n              15       0 <NULL>  <NULL>
2 TRT01A Placebo      AESOC  GASTROINTESTINAL DISORDERS AEDECOD  DIARRHOEA                  hierarchical_count n         n              10       0 <NULL>  <NULL>
3 TRT01A Placebo      AESOC  GASTROINTESTINAL DISORDERS AEDECOD  HIATUS HERNIA              hierarchical_count n         n               2       0 <NULL>  <NULL>
4 TRT01A Placebo      AESOC  GASTROINTESTINAL DISORDERS AEDECOD  VOMITING                   hierarchical_count n         n               3       0 <NULL>  <NULL>

Exercise 🏃‍➡️

1. Navigate to Workshop Website‍➡️ Exercises‍➡️ 01-ARD.R

2. Compute the nested AE tabulations as described.

{cardx}

• Extension of the {cards} package, providing additional functions to create Analysis Results Datasets (ARDs)

• The {cardx} package exports many ard_*() function for statistical methods.

{cardx}

• Exports ARD frameworks for statistical analyses from many packages

  - {stats}
  - {car}
  - {effectsize}
  - {emmeans}
  - {geepack}
  - {lme4}
  - {parameters}
  - {smd}
  - {survey}
  - {survival}

• This list is growing (rather quickly) 🌱

{cardx} t-test Example

• We see the results like the mean difference, the confidence interval, and p-value as expected.

• And we also see the function’s inputs, which is incredibly useful for re-use, e.g. we know that we did not use equal variances.

adsl |> 
  cardx::ard_stats_t_test(by = ARM2, variables = AGE)

# An ARD data frame: 14 × 9
   group1 variable context      stat_name   stat_label          stat                    fmt_fun      warning      error       
   <chr>  <chr>    <chr>        <chr>       <chr>               <named list>            <named list> <named list> <named list>
 1 ARM2   AGE      stats_t_test estimate    Mean Difference     0.1854928               1            <NULL>       <NULL>      
 2 ARM2   AGE      stats_t_test estimate1   Group 1 Mean        75.2093                 1            <NULL>       <NULL>      
 3 ARM2   AGE      stats_t_test estimate2   Group 2 Mean        75.02381                1            <NULL>       <NULL>      
 4 ARM2   AGE      stats_t_test statistic   t Statistic         0.1660687               1            <NULL>       <NULL>      
 5 ARM2   AGE      stats_t_test p.value     p-value             0.8683091               1            <NULL>       <NULL>      
 6 ARM2   AGE      stats_t_test parameter   Degrees of Freedom  162.6425                1            <NULL>       <NULL>      
 7 ARM2   AGE      stats_t_test conf.low    CI Lower Bound      -2.020129               1            <NULL>       <NULL>      
 8 ARM2   AGE      stats_t_test conf.high   CI Upper Bound      2.391114                1            <NULL>       <NULL>      
 9 ARM2   AGE      stats_t_test method      method              Welch Two Sample t-test <NULL>       <NULL>       <NULL>      
10 ARM2   AGE      stats_t_test alternative alternative         two.sided               <NULL>       <NULL>       <NULL>      
11 ARM2   AGE      stats_t_test mu          H0 Mean             0                       1            <NULL>       <NULL>      
12 ARM2   AGE      stats_t_test paired      Paired t-test       FALSE                   <NULL>       <NULL>       <NULL>      
13 ARM2   AGE      stats_t_test var.equal   Equal Variances     FALSE                   <NULL>       <NULL>       <NULL>      
14 ARM2   AGE      stats_t_test conf.level  CI Confidence Level 0.95                    1            <NULL>       <NULL>      

{cardx} t-test Example

• What to do if a method you need is not implemented?

• It’s simple to wrap existing frameworks to customize.

• One-sample t-test example utilizing cards::ard_summary().

adsl |> 
  cards::ard_summary(
    variables = AGE,
    statistic = everything() ~ list(t_test = \(x) t.test(x) |> broom::tidy())
  ) |> 
  dplyr::mutate(context = "t_test_one_sample")

# An ARD data frame: 8 × 8
  variable context           stat_name   stat_label  stat              fmt_fun warning error 
  <chr>    <chr>             <chr>       <chr>       <list>            <list>  <list>  <list>
1 AGE      t_test_one_sample estimate    estimate    75.08661          1       <NULL>  <NULL>
2 AGE      t_test_one_sample statistic   statistic   145.1188          1       <NULL>  <NULL>
3 AGE      t_test_one_sample p.value     p.value     1.333805e-245     1       <NULL>  <NULL>
4 AGE      t_test_one_sample parameter   parameter   253               1       <NULL>  <NULL>
5 AGE      t_test_one_sample conf.low    conf.low    74.06763          1       <NULL>  <NULL>
6 AGE      t_test_one_sample conf.high   conf.high   76.1056           1       <NULL>  <NULL>
7 AGE      t_test_one_sample method      method      One Sample t-test <fn>    <NULL>  <NULL>
8 AGE      t_test_one_sample alternative alternative two.sided         <fn>    <NULL>  <NULL>

{cardx} t-test Example

• How to modify if we need a two-sample test, or more generally accessing other columns in the data frame.

adsl |> 
  cards::ard_mvsummary(
    variables = AGE,
    statistic = 
      ~ list(t_test = \(x, data, ...) t.test(x ~ data$ARM2) |> broom::tidy())
  ) |> 
  dplyr::mutate(group1 = "ARM2", context = "t_test_two_sample") |> 
  cards::tidy_ard_column_order()

# An ARD data frame: 10 × 9
   group1 variable context           stat_name   stat_label  stat                    fmt_fun warning error 
   <chr>  <chr>    <chr>             <chr>       <chr>       <list>                  <list>  <list>  <list>
 1 ARM2   AGE      t_test_two_sample estimate    estimate    0.1854928               1       <NULL>  <NULL>
 2 ARM2   AGE      t_test_two_sample estimate1   estimate1   75.2093                 1       <NULL>  <NULL>
 3 ARM2   AGE      t_test_two_sample estimate2   estimate2   75.02381                1       <NULL>  <NULL>
 4 ARM2   AGE      t_test_two_sample statistic   statistic   0.1660687               1       <NULL>  <NULL>
 5 ARM2   AGE      t_test_two_sample p.value     p.value     0.8683091               1       <NULL>  <NULL>
 6 ARM2   AGE      t_test_two_sample parameter   parameter   162.6425                1       <NULL>  <NULL>
 7 ARM2   AGE      t_test_two_sample conf.low    conf.low    -2.020129               1       <NULL>  <NULL>
 8 ARM2   AGE      t_test_two_sample conf.high   conf.high   2.391114                1       <NULL>  <NULL>
 9 ARM2   AGE      t_test_two_sample method      method      Welch Two Sample t-test <fn>    <NULL>  <NULL>
10 ARM2   AGE      t_test_two_sample alternative alternative two.sided               <fn>    <NULL>  <NULL>

{cardx} Regression

• Includes functionality to summarize nearly every type of regression model in the R ecosystem:

betareg::betareg(), biglm::bigglm(), brms::brm(), cmprsk::crr(), fixest::feglm(), fixest::femlm(), fixest::feNmlm(), fixest::feols(), gam::gam(), geepack::geeglm(), glmmTMB::glmmTMB(), glmtoolbox::glmgee(), lavaan::lavaan(), lfe::felm(), lme4::glmer.nb(), lme4::glmer(), lme4::lmer(), logitr::logitr(), MASS::glm.nb(), MASS::polr(), mgcv::gam(), mice::mira, mmrm::mmrm(), multgee::nomLORgee(), multgee::ordLORgee(), nnet::multinom(), ordinal::clm(), ordinal::clmm(), parsnip::model_fit, plm::plm(), pscl::hurdle(), pscl::zeroinfl(), quantreg::rq(), rstanarm::stan_glm(), stats::aov(), stats::glm(), stats::lm(), stats::nls(), survey::svycoxph(), survey::svyglm(), survey::svyolr(), survival::cch(), survival::clogit(), survival::coxph(), survival::survreg(), svyVGAM::svy_vglm(), tidycmprsk::crr(), VGAM::vgam(), VGAM::vglm() (and more)

{cardx} Regression Example

library(survival)

# build model
mod <- pharmaverseadam::adtte_onco |> 
  dplyr::filter(PARAM %in% "Progression Free Survival") |>
  coxph(ggsurvfit::Surv_CNSR() ~ ARM, data = _)

# put model in a summary table
tbl <- gtsummary::tbl_regression(mod, exponentiate = TRUE) |> 
  gtsummary::add_n(location = c('label', 'level')) |> 
  gtsummary::add_nevent(location = c('label', 'level'))

Characteristic	N	Event N	HR	95% CI	p-value
Description of Planned Arm	254	6
Placebo	86	3	—	—
Xanomeline High Dose	84	2	3.00	0.39, 22.9	0.3
Xanomeline Low Dose	84	1	1.27	0.11, 14.3	0.8
Abbreviations: CI = Confidence Interval, HR = Hazard Ratio

{cardx} Regression Example

The cardx::ard_regression() does a lot for us in the background.

• Identifies the variable from the regression terms (i.e. groups levels of the same variable)
• Identifies reference groups from categorical covariates
• Finds variable labels from the source data frames
• Knows the total N of the model, the number of events, and can do the same for each level of categorical variables
• Contextually aware of slopes, odds ratios, hazard ratios, and incidence rate ratios
• And much much more.

When things go wrong 😱

What happens when statistics are un-calculable?

ard_gone_wrong <- 
  adsl |> 
  ard_summary(
    by = ARM2,
    variable = AGEGR1,
    statistic = ~list(kurtosis = \(x) e1071::kurtosis(x))
  ) |> 
  cards::replace_null_statistic()
ard_gone_wrong

# An ARD data frame: 2 × 10
  group1 group1_level variable context stat_name stat_label stat   fmt_fun warning                                          error                                  
* <chr>  <list>       <chr>    <chr>   <chr>     <chr>      <list> <list>  <list>                                           <list>                                 
1 ARM2   Placebo      AGEGR1   summary kurtosis  kurtosis   NA     <fn>    argument is not numeric or logical: returning NA non-numeric argument to binary operator
2 ARM2   Xanomeline   AGEGR1   summary kurtosis  kurtosis   NA     <fn>    argument is not numeric or logical: returning NA non-numeric argument to binary operator

cards::print_ard_conditions(ard_gone_wrong)

• Where is the statistic? AGEGR1 is character

• Even when there are errors or warnings, we still get the ARD with the expected structure returned.

  • THIS IS BIG! There are MANY circumstances, when you are designing TLGs early in a study when you do not have all the data required to calculate every statistic.

  • This allows you to design everything up-front.

• We can also report these warnings and errors back to users. <!CLICK!>