# Regulatory Reporting

#### Ndoh Penn

#### 2026-08-19

# 1 Regulatory Context

The FDA’s 2026 draft guidance, Use of Bayesian Methodology in Clinical Trials of Drug and Biological Products, calls for Bayesian prior distributions used in confirmatory clinical trials to be:

1. Explicitly specified — family, hyperparameters, and elicitation method
2. Justified — documented basis (expert opinion, historical data, or both)
3. Conflict-checked — evidence that prior and data are compatible
4. Sensitivity-tested — conclusions hold under plausible alternative priors

The EMA has not yet published guidance specific to Bayesian methods: as of 2026, a concept paper proposing the development of a reflection paper on the use of Bayesian methods in clinical development (EMA/CHMP/1813/2026) has been released for consultation, with the draft reflection paper itself not expected until 2027 and final publication in 2028. Separately, EMA’s existing reflection paper on extrapolation in paediatric drug development already requires sponsors incorporating external or historical data to quantify the prior’s influence relative to the observed data and to investigate the resulting Type I error properties — themes consistent with points 2-4 above. 5. Pre-specified — agreed with regulators before unblinding

`prior_report()` generates a self-contained document addressing all five requirements.

# 2 The Prior Justification Report

## 2.1 What It Contains

A bayprior report includes:

- Executive summary — one-page overview for non-statistical reviewers
- Trial information — protocol number, sponsor, statistician, date
- Prior specification — distribution, hyperparameters, summary statistics, density plot, and elicitation method
- Conflict diagnostics — Box p-value, surprise index, KL divergence, Bhattacharyya overlap, severity classification, and recommendation
- Prior-likelihood-posterior overlay — the canonical visual for conflict
- Sensitivity analysis — tornado plot and influence heatmap
- Robust and sensitivity priors — robust mixture, sceptical, and power prior summaries and plots (included automatically when computed in the session)
- Regulatory compliance checklist — maps to FDA/EMA requirements, including a dedicated row for robust/sceptical prior computation
- Session information — R version, bayprior version, Quarto CLI version

## 2.2 Output Formats

Three formats are supported:

Format | Use case
--- | ---
HTML | Internal review, version-controlled documentation
PDF | Regulatory submission appendix
Word (.docx) | Co-authoring and track-changes review

# 3 Generating a Report

    # 1. Elicit the prior
    prior <- elicit_beta(
      mean      = 0.30,
      sd        = 0.10,
      method    = "moments",
      expert_id = "Expert_1",
      label     = "Objective response rate"
    )

    # 2. Run conflict diagnostics
    cd <- prior_conflict(
      prior        = prior,
      data_summary = list(type = "binary", x = 18, n = 40),
      alpha        = 0.05
    )

    # 3. Run sensitivity analysis
    sa <- sensitivity_grid(
      prior        = prior,
      data_summary = list(type = "binary", x = 14, n = 40),
      param_grid   = list(alpha = seq(1, 8, 0.5), beta = seq(2, 20, 1)),
      target       = c("posterior_mean", "prob_efficacy"),
      threshold    = 0.30
    )

    # 4. Build robust and sceptical priors (optional — appear in report if supplied)
    rob  <- robust_prior(prior, vague_weight = 0.20)
    scep <- sceptical_prior(null_value = 0.20, family = "beta", strength = "moderate")

    # 5. Generate the report
    # NOTE: requires devtools::install(), not just devtools::load_all()
    # Quarto spawns a fresh R session that needs the installed package.
    prior_report(
      prior           = prior,
      conflict        = cd,
      sensitivity     = sa,
      robust_prior    = rob,      # included in "Robust and Sensitivity Priors" section
      sceptical_prior = scep,     # ditto
      output_format   = "pdf",
      output_file     = "prior_justification_report",
      trial_name      = "TRIAL-001",
      sponsor         = "BioPharma Ltd",
      author          = "J. Smith, Principal Biostatistician",
      notes         = paste0(
        "Prior elicited from two independent oncologists (Expert_1, Expert_2) ",
        "using the SHELF structured elicitation protocol. Experts were blinded ",
        "to interim data at the time of elicitation. Prior was pre-specified in ",
        "the Statistical Analysis Plan dated 2025-09-01."
      )
    )

# 4 The Notes Field

The `notes` argument is the statistician’s narrative — the most important section for regulatory reviewers because it provides the scientific rationale that no automated output can supply. It should address:

1. Basis for the prior

    "Prior elicited from three independent clinical experts in non-small cell
    lung cancer using SHELF quantile matching (O'Hagan et al., 2006). Experts
    were asked to specify the 10th, 50th, and 90th percentiles of the ORR
    distribution based on knowledge of similar agents in this indication."

2. Expert independence and blinding

    "All experts were blinded to any interim results at the time of elicitation.
    No expert had a financial interest in the trial outcome."

3. Pre-specification

    "The prior distribution and elicitation protocol were pre-specified in the
    Bayesian Statistical Analysis Plan (BSAP), version 2.1, submitted to FDA
    on 2025-09-01 as part of the IND amendment."

4. Conflict action plan

    "In the event of mild conflict (Box p-value 0.01-0.05), a sensitivity
    analysis using the robust mixture prior (vague weight 0.30) will be added.
    In the event of severe conflict (Box p-value < 0.01), the primary analysis
    will be repeated using the sceptical prior as a co-primary analysis."

# 5 The FDA/EMA Compliance Checklist

The report automatically generates a compliance checklist that maps each section to specific regulatory requirements:

    library(knitr)
    kable(data.frame(
      Requirement = c(
        "Prior elicitation method documented",
        "Distribution family and parameters specified",
        "Expert / source of prior identified",
        "Prior density plot provided",
        "Prior-data conflict assessed",
        "Conflict diagnostic statistics reported",
        "Sensitivity analysis performed",
        "Sensitivity visualisations provided",
        "Alternative priors considered",
        "Robust / sceptical prior computed",
        "Regulatory report generated"
      ),
      `FDA Guidance Section` = c(
        "Section V.A", "Section V.A", "Section V.D",
        "Section V.A", "Section V.D", "Section VIII.B",
        "Section V.F", "Section V.F", "Section V.F",
        "Section V.C", "Section VIII"
      ),
      check.names = FALSE
    ), align = "ll")

Requirement | FDA Guidance Section
--- | ---
Prior elicitation method documented | Section V.A
Distribution family and parameters specified | Section V.A
Expert / source of prior identified | Section V.D
Prior density plot provided | Section V.A
Prior-data conflict assessed | Section V.D
Conflict diagnostic statistics reported | Section VIII.B
Sensitivity analysis performed | Section V.F
Sensitivity visualisations provided | Section V.F
Alternative priors considered | Section V.F
Robust / sceptical prior computed | Section V.C
Regulatory report generated | Section VIII

# 6 Session Reproducibility

The Session Information section of every report records the full computational environment:

Item | Purpose
--- | ---
R version | Core language version
bayprior version | Package version — pin this in your SAP
quarto R package | Report rendering R interface
Quarto CLI version | Actual rendering engine version
Platform | Operating system and architecture
Date | Date of report generation

For fully reproducible regulatory submissions, pin the bayprior version in your `renv.lock` file and archive the complete `renv` snapshot alongside the submission documents.

# 7 Working with the Shiny App

The `run_app()` function launches the full interactive Shiny interface. Every analysis conducted in the app — elicitation, pooling, conflict diagnostics, sensitivity, and robust/sceptical/power priors — is preserved in the session state and flows directly into the report when the Download button is clicked.

Figures in the report reflect exactly what the analyst saw in the app at the time of download. Specifically:

- The prior density plot comes from the fitted prior in the Elicitation panel
- The conflict overlay comes from the Conflict Diagnostics panel
- The tornado and heatmap come from the Sensitivity Analysis panel
- The robust, sceptical, and power prior plots come from the Robust Priors panel — these appear in the report only if those analyses were run in the session

The app workflow mirrors the analytical steps:

    Welcome → Prior Elicitation → Expert Pooling → Conflict Diagnostics →
    Sensitivity Analysis → Robust Priors → Export Report

# 8 Pre-Submission Checklist

Before submitting a bayprior report to a regulatory agency:

- Prior was pre-specified in the Statistical Analysis Plan
- Expert elicitation was conducted blind to interim data
- Expert identities and conflicts of interest are documented
- Conflict action plan is pre-specified (what happens at each severity)
- Sensitivity range was agreed with regulators a priori
- At least one alternative prior is reported (robust or sceptical)
- bayprior version is pinned in the renv.lock
- Report generated from installed package (not `load_all()`)
- PDF and HTML versions both generated and cross-checked
- Notes field contains full scientific rationale

# 9 References

FDA (2026). Use of Bayesian Methodology in Clinical Trials of Drug and Biological Products (Draft Guidance for Industry).

EMA (2026). Concept Paper for the Development of a Reflection Paper on the Use of Bayesian Methods in Clinical Development (EMA/CHMP/1813/2026).

O’Hagan, A. et al. (2006). Uncertain Judgements: Eliciting Experts’ Probabilities. Wiley.

Spiegelhalter, D. J. & Freedman, L. S. (1994). Bayesian approaches to clinical trials. JRSS-A, 157, 357–416.