Package {coreval}

coreval: Check Clinical Trial Data Against 'CDISC' Open Rules

Description

Finds conformance problems in clinical trial data without leaving R, using the openly published 'CDISC' Open Rules ('CORE'). Check a single dataset while you are still writing the code that builds it, or a whole study folder once it exists, and get the findings back as a tidy data frame pointing at the exact row and variable. Reads transport ('XPT'), 'SAS', comma-separated and 'Dataset-JSON' files, plus 'Define-XML' when present, and 'USDM' study-design documents. Covers rules for the 'SDTM', 'SEND', 'TIG' and 'USDM' standards. The rules are bundled inside the package, so nothing is downloaded and your data never leaves your machine: no internet, no API key, no account. When a rule cannot be checked - because it needs a dataset you did not supply, for instance - it is reported as skipped with the reason, never counted as a pass. Meant as a quick first pass before a qualified validation system, never as a replacement for one. An independent project: not affiliated with or endorsed by 'CDISC', and not a 'CORE'-certified conformance engine.

Details

coreval finds CDISC conformance problems in clinical trial data without leaving R. Run it early and often, while you are still writing the code that produces the data, so problems turn up while they are cheap to fix.

Two ways in, depending on what you have:

• check_dataset() - one dataset, either a data frame you already have open or a single file. Use this while writing code.

• read_study() then check_study() - a whole study folder. Use this once the datasets exist, since the cross-dataset rules need everything present.

Both entry points give you two tables, and both matter. ⁠$findings⁠ is what is wrong. ⁠$skipped⁠ is what could not be checked, with a reason for each. An empty ⁠$findings⁠ can mean clean data or rules that never ran, and those look identical if you only read the first table.

write_findings() saves both to Excel or CSV. vignette("coreval") walks through all of it.

Status

coreval is a personal open-source project. It is not a CDISC product, is not affiliated with or endorsed by CDISC, and is not qualified or validated software. The rules are bundled inside the package, so nothing is downloaded and your data never leaves your machine: no internet, no API key, no account.

Treat every result as a hint, not a verdict. A qualified validation system and your own review are still what decide whether data is good to submit. coreval does not remove that step - it just leaves that step less to find.

Author(s)

Maintainer: Hrach Gevorgyan hrach.gevorgyan@yandex.com [copyright holder]

Authors:

• Hrach Gevorgyan hrach.gevorgyan@yandex.com [copyright holder]

See Also

Useful links:

• https://github.com/hrach-gevorgyan/coreval

• Report bugs at https://github.com/hrach-gevorgyan/coreval/issues

Check one dataset, without needing a study folder

Description

For when you are writing the code that builds a domain and want to know what is wrong with it right now. Give it the data frame you already have open, or the path to a single file.

Usage

check_dataset(
  x,
  domain = NULL,
  standard = NULL,
  version = NULL,
  use_case = NULL,
  max_records = 1000,
  include_deprecated = FALSE,
  ct_package = NULL
)

Arguments

Value

An object of class coreval_result: a list of the three tables findings, skipped and truncated, the same shape check_study() returns, so write_findings() and filter_findings() work on it unchanged. Because it has a class, typing the result's name prints a readable report rather than dumping the list.

skipped carries a reason for every rule that did not run - a dataset you did not supply, a missing Define-XML, or (for 9 rules) CDISC's controlled terminology, if you did not say which package the study follows - see ct_package. Nothing skipped is ever counted as a pass.

Provenance rides along as attributes: checks_run (how many rules were evaluated), domains, and excluded_by_standard (how many rules the standard/version filter set aside). write_findings() writes these into the file it saves.

What it cannot check on its own

Plenty of CDISC rules compare one dataset against another - an adverse event date against the subject's reference dates in DM, a visit against the trial design. Hand over a single dataset and those questions cannot be answered.

coreval does not guess. Those rules are skipped, and ⁠$skipped⁠ names the dataset each one wanted. Running them anyway would compare your data against columns that are not there and report problems that do not exist.

Most rules still run - across AE, DM, LB and VS, 76-84% of the applicable ones work on a single dataset. But the ones that cannot are the cross-dataset checks, which are often the ones that matter.

So a short ⁠$findings⁠ table here does not mean the data is clean. It is a quick first pass, not a verdict. Run check_study() on the whole folder before drawing conclusions.

See Also

check_study() to check a whole study folder.

Examples

ae <- data.frame(
  STUDYID = "S1", DOMAIN = "AE", USUBJID = c("01", "01"),
  AESEQ = c(1, 2), AETERM = c("Headache", "Rash"),
  AESTDTC = c("2024-01-10", "2024-02-30") # 30 February is not a date
)
result <- check_dataset(ae)
result$findings[result$findings$Value == "2024-02-30", ]

# Always look at what could not run:
nrow(result$skipped)

Check a whole study against CDISC Open Rules

Description

Runs every rule that applies to every dataset in the study, including the ones that compare datasets against each other. Use this once the datasets exist as files; to check a single dataset while you are still writing the code that builds it, see check_dataset().

Usage

check_study(
  study,
  standard = NULL,
  version = NULL,
  use_case = NULL,
  max_records = 1000,
  include_deprecated = FALSE,
  ct_package = NULL
)

Arguments

Details

Findings come back one row per (dataset, record, variable), pointing at the exact spot. Some rules ask about a dataset as a whole rather than a particular row - those leave Record blank. A few ask about the study as a whole, such as "is DM present at all?"; those are answered once and reported under Dataset = "STUDY" rather than repeated for every domain.

Rules comparing against a define.xml do run, as long as the study has one and the xml2 package is installed. Without both, they are skipped with a reason instead of being run against columns that are not there, which would report problems that do not exist. The same goes for any rule needing an operator or join coreval does not implement yet.

Value

An object of class coreval_result, holding three tables. Because it has a class, typing the result's name prints a readable report rather than dumping the list, and provenance rides along as attributes: checks_run, domains and excluded_by_standard.

• findings - what is wrong. One row per affected record, with Dataset, Record, Variable, Value, the issue in words, and its triage. ⁠Not in dataset⁠ under Value means the rule wanted a variable you do not have, which is usually the finding itself.

• skipped - what could not be checked, with a reason for each. Read this one: an empty findings table can mean clean data or rules that never ran, and they look identical otherwise. Reasons include a dataset you did not supply, a missing Define-XML, and - for 9 rules - CDISC's controlled terminology when no ct_package was given, which is not bundled. Nothing skipped is ever counted as a pass.

• truncated - rules that flagged more records than max_records kept, with how many they really found.

Progress

A large study takes long enough that silence looks like a hang, so an interactive session shows a progress bar naming the domain being checked and how far through the study it is:

  AE        4/7  |=====================       |  75%

The percentage is weighted by how many records each domain holds, not by a plain count of rules, because a check against a 161,600-row AE costs hundreds of times one against a 200-row SJ. It tracks elapsed time closely but is still an estimate - rules differ in cost among themselves too - so treat it as "roughly how far through", not a clock.

It is off in scripts and non-interactive runs, where it would only clutter a log. Turn it on or off with options(coreval.progress = TRUE) or FALSE.

Examples

dir <- tempfile("coreval_study_")
dir.create(dir)
haven::write_xpt(data.frame(USUBJID = c("1", "2"), AGE = c(30, 65)), file.path(dir, "dm.xpt"))

result <- check_study(dir)
result$findings
unlink(dir, recursive = TRUE)

Narrow a result to the findings you care about

Description

Saves writing subset() over ⁠$findings⁠ by hand, and - because it returns a result rather than a plain table - what comes back still prints as a readable report.

Usage

filter_findings(
  result,
  triage = NULL,
  dataset = NULL,
  rule = NULL,
  variable = NULL
)

Arguments

Value

A coreval_result holding the matching findings. ⁠$skipped⁠ is left alone: what could not be checked does not become less true because you narrowed what you are looking at.

Examples

ae <- data.frame(
  STUDYID = "S1", DOMAIN = "AE", USUBJID = c("01", "01"),
  AESEQ = c(1, 2), AETERM = c("Headache", "Rash"),
  AESTDTC = c("2024-01-10", "2024-02-30")
)
result <- check_dataset(ae)

# Just the things that are definitely wrong:
filter_findings(result, triage = "wrong value")

Controlled Terminology packages coreval bundles

Description

Which version of CDISC's Controlled Terminology a study follows decides whether a value is a legal term, and the answer changes between releases - SEX gained INTERSEX and lost UNDIFFERENTIATED. So coreval never picks one for you: pass the one your study declares as check_study(ct_package =) and this is the list to pick from.

Usage

list_ct_packages(family = NULL)

Arguments

Details

Only what a conformance rule can ask about is bundled - each codelist's submission value and C-code, its terms' submission values and C-codes, and whether it is extensible. Definitions and synonyms are not, which is how 438 MB of CDISC's own caches becomes half a megabyte here.

Value

A character vector of package names, oldest first.

Examples

head(list_ct_packages("sdtm"))

Look up CORE rules

Description

One way in for every question about the rule set: what rules exist, what a particular one checks, and which of them apply to a domain.

Usage

list_rules(
  id = NULL,
  domain = NULL,
  standard = NULL,
  version = NULL,
  use_case = NULL,
  include_deprecated = TRUE
)

Arguments

Details

The columns are the same whatever you ask for, so the result is safe to filter, join and script against.

Value

A data.table::data.table(), one row per rule: id, the one-line issue it reports, its fuller description, the guidance sentence from the Implementation Guide it enforces, the legacy_ids Pinnacle 21 uses for it, standard and standard_version, authority, rule_type, sensitivity, executability, source and status.

The commit the bundled rules came from is on the result as attr(x, "rules_version"); write_findings() records it in every exported file.

What source tells you

Not every bundled rule carries the same weight:

• "published" - ⁠Published/⁠ upstream, fully tested. The trusted core.

• "deprecated_dir" - superseded by a published replacement. Not returned unless you ask for it, since running both reports the same defect twice.

• "fda_business_rules_draft" - FDA drafts that already ship test data.

Examples

# Everything
nrow(list_rules())

# What does the rule the report just named actually check?
list_rules(id = "CORE-000547")$issue

# What applies to AE under SDTMIG 3.4?
nrow(list_rules(domain = "AE", standard = "SDTMIG", version = "3.4"))

# Which snapshot of CDISC's rules is this?
attr(list_rules(), "rules_version")

Print a coreval check result as a readable report

Description

Describes each problem in the rule's own words, worst first, with the rows and values that caused it. A whole-study result is grouped by dataset, with a summary first, so you can see where the trouble is before reading detail.

Usage

## S3 method for class 'coreval_result'
print(x, n = 10, rows = 3, guidance = FALSE, ...)

Arguments

Value

x, invisibly.

Examples

ae <- data.frame(
  STUDYID = "S1", DOMAIN = "AE", USUBJID = c("01", "01"),
  AESEQ = c(1, 2), AETERM = c("Headache", "Rash"),
  AESTDTC = c("2024-01-10", "2024-02-30")
)
check_dataset(ae)

Read a study into coreval's internal representation

Description

Detects whether path is a directory of XPT datasets (a real study) or a CORE test-case ⁠data/⁠ directory (⁠_variables.csv⁠ + one CSV per dataset, usually also .env and ⁠_datasets.csv⁠), and reads either into the same internal representation, so the evaluator never has to know which one it got.

Usage

read_study(path)

Arguments

Details

Character columns use "" for blank/missing (never NA) to match how SAS XPT round-trips blanks; numeric columns use NA. Column types are taken from the source (XPT's own types, or ⁠_variables.csv⁠'s declared Char/Num) rather than guessed from the data, so numeric-looking identifiers (e.g. "007") are never silently coerced.

Value

A study object: ⁠list(datasets = <named list of domain -> list(data, meta, label)>, define, ct = NULL, standard = list(product, version))⁠. define is the parsed Define-XML if one was found in path and the xml2 package is installed, and NULL otherwise; ct is always NULL (controlled terminology is not bundled). Each data is a data.table::data.table(); each meta is a data.table with columns variable, label, type; label is the dataset's own label (e.g. "Adverse Events"), or NA if unavailable. standard is the study's declared standard/version (e.g. list(product = "SDTMIG", version = "3-4")), read from a CORE test case's .env file - both NA for a real XPT-based study (no .env).

Examples

dir <- tempfile("coreval_study_")
dir.create(dir)
haven::write_xpt(data.frame(USUBJID = c("1", "2"), AGE = c(30, 65)), file.path(dir, "dm.xpt"))
study <- read_study(dir)
study$datasets$DM$data
unlink(dir, recursive = TRUE)

Summarize a check result in a few lines

Description

The short form of print.coreval_result(), for when you have run a check inside a script, or just want to know whether the last fix helped.

Usage

## S3 method for class 'coreval_result'
summary(object, ...)

Arguments

Value

A one-row data.table::data.table(), invisibly, with the counts it printed - so it can be logged or compared.

Examples

ae <- data.frame(
  STUDYID = "S1", DOMAIN = "AE", USUBJID = c("01", "01"),
  AESEQ = c(1, 2), AETERM = c("Headache", "Rash"),
  AESTDTC = c("2024-01-10", "2024-02-30")
)
summary(check_dataset(ae))

Write conformance findings to a file

Description

Saves the result of check_study() to CSV or Excel, so findings can be shared with people who don't use R, tracked in a spreadsheet, or attached to a data-review document.

Usage

write_findings(result, path, tracking = TRUE)

Arguments

Details

Both tables are always written, never just the findings. A short findings table can mean clean data, or it can mean many rules were skipped, and those two situations look identical if the skipped table is dropped:

• Excel (.xlsx) - one workbook with a sheet per table: findings, skipped, about, and truncated when any rule matched more records than were kept.

• CSV (.csv) - one file per table, since CSV has no notion of sheets. Findings go to path; the others go to sibling files with a suffix before the extension, so issues.csv gives issues_skipped.csv and issues_about.csv.

Excel output needs the writexl package. It is a Suggests, so if it isn't installed you get a clear message telling you to install it or use .csv instead, rather than a failure part-way through writing.

Value

The paths actually written, invisibly. One element for Excel (a single workbook). For CSV, one path per file written: the findings, plus siblings for skipped and about, plus one for truncated when any rule flagged more records than were kept.

Columns for tracking

Three empty columns are added to the findings - Status, Owner and Notes - for you to fill in by hand once the file is open. They exist so a finding you have looked at and decided not to act on ("expected, see protocol deviation log") can be recorded next to the finding itself, rather than in a separate document nobody reads.

Examples

dir <- tempfile("coreval_study_")
dir.create(dir)
haven::write_xpt(data.frame(USUBJID = c("1", "2"), AGE = c(30, 65)), file.path(dir, "dm.xpt"))
study <- read_study(dir)
result <- check_study(study)

out <- file.path(dir, "findings.csv")
written <- write_findings(result, out)
basename(written)

unlink(dir, recursive = TRUE)