Help for package simpr

Type:

Package

Title:

Flexible 'Tidyverse'-Friendly Simulations

Version:

0.2.6

Description:

A general, 'tidyverse'-friendly framework for simulation studies, design analysis, and power analysis. Specify data generation, define varying parameters, generate data, fit models, and tidy model results in a single pipeline, without needing loops or custom functions.

License:

GPL-2

Encoding:

UTF-8

URL:

https://statisfactions.github.io/simpr/, https://github.com/statisfactions/simpr/

BugReports:

https://github.com/statisfactions/simpr/issues/

Depends:

R (≥ 4.0.0)

Imports:

broom, dplyr, furrr, generics, purrr, lifecycle, magrittr, rlang, tibble, tidyr (≥ 1.2.0), tidyselect

RoxygenNote:

7.2.3

Suggests:

MASS, testthat, ggplot2, knitr, rmarkdown

VignetteBuilder:

knitr

NeedsCompilation:

Packaged:

2023-04-26 02:47:37 UTC; brow3821

Author:

Ethan Brown

[aut, cre], Jeffrey Bye [ctb]

Maintainer:

Ethan Brown <brow3821@umn.edu>

Repository:

CRAN

Date/Publication:

2023-04-26 06:20:02 UTC

Run a given function or formula expression on a simpr_mod object and tidy the output.

Description

This function applies a given function to all fit objects and returns the result in a tidy tibble. Any function or purrr-style lambda function can be used.

Usage

apply_fits(obj, .f, ..., .progress = FALSE, .options = furrr_options())

Arguments

obj

a simpr_tibble with repetition number, metaparameters, simulated data, and fitted models, from fit

.f

A function or purrr-style lambda function (see as_mapper) used for computing on the fit object

...

Additional arguments to .f.

.progress

A logical, for whether or not to print a progress bar for multiprocess, multisession, and multicore plans .

.options

The future specific options to use with the workers when using futures. This must be the result from a call to furrr_options().

Value

A tibble with columns .sim_id, rep, Source (which contains the name of the fit column), any metaparameters from define, and additional columns containing the results of .f applied to each fit object.

Examples

set.seed(100)
logit_fit = specify(a = ~ sample(0:1, size = n, replace = TRUE),
        b = ~ a + rlnorm(n)) %>%
  define(n = c(40, 50)) %>%
  generate(1) %>%
  fit(logit = ~ glm(a ~ b, family = "binomial"))

logit_fit %>%
  apply_fits(broom::augment)

## Arguments to the function can be passed in ...
logit_fit %>%
  apply_fits(broom::augment, se_fit = TRUE)

## Using a purrr-style lambda function
logit_fit %>%
  apply_fits(~ summary(.)$cov.scaled)

Define metaparameters to vary in simulation

Description

Takes the output of specify (a simpr_spec object) and defines the metaparameters (i.e. simulation factors).

Usage

define(.x = NULL, ..., .list = NULL, .suffix = "_index")

Arguments

.x

a simpr_spec object (the output of specify)

...

metaparameters: named arguments containing vectors or lists of objects to be used in the simulation.

.list

additional parameters passed to define() as a list. Useful if you already have desired metaparameters already in list format or created by other functions.

.suffix

name of suffix to append onto index column for list metaparameters, "_index" by default. See Details.

Details

This is the second step in the simulation process, after specifying the simulated data using specify. The output of define is then passed to generate to actually generate the simulation.

Metaparameters are named arguments, passed to ..., that are used in the simulation. A metaparameter is some kind of vector or list, representing something that is to be systematically varied as a part of the simulation design. Any metaparameter should also appear in the formulas of specify, and thus the simulation changes depending on the value of the metaparameter.

When creating the simulation, simulations for all possible combinations of metaparameters are generated, resulting in a fully crossed simulation design. If only a subset of the fully crossed design is needed, use the filtering options available in generate.

When one of ... is a list, a new column is generated in the output to generate to serve as the index of the list. This new column will be the name of the list argument, with the suffix argument appended onto the end. So if Y = list(a = 1:2, b = letters[2:3]), and suffix = "_index", the default, a column named Y_index would be added to the output of generate with values "a" and "b".

Value

a simpr_spec object to pass onto generate for the simulation.

Examples

# Simple example of setting a metaparameter
simple_meta = specify(a = ~ 1 + rnorm(n)) %>%
  define(n = c(5, 10)) %>%
  generate(1)

simple_meta # $sim has a 5-row tibble and a 10-row tibble

multi_meta = specify(a = ~ mu + rnorm(n)) %>%
  define(n = c(5, 10),
       mu = seq(-1, 1, length.out = 3)) %>%
  generate(1)

multi_meta # generates simulations for all combos of n and mu


# define can handle lists which can contain multiple matrices, etc.
meta_list_out = specify(a = ~ MASS::mvrnorm(n, rep(0, 2), Sigma = S)) %>%
  define(n = c(10, 20, 30),
       S = list(independent = diag(2), correlated = diag(2) + 2)) %>%
  generate(1)

meta_list_out # generates S_index column

# define can also take arguments as a list using the .list argument
meta_list_out_2 = specify(a = ~ MASS::mvrnorm(n, rep(0, 2), Sigma = S)) %>%
  define(.list = list(n = c(10, 20, 30),
       S = list(independent = diag(2), correlated = diag(2) + 2))) %>%
  generate(1)

Fit models to the simulated data

Description

Takes simulated data from generate and applies functions to it, usually model-fitting functions.

Usage

## S3 method for class 'simpr_tibble'
fit(
  object,
  ...,
  .quiet = TRUE,
  .warn_on_error = TRUE,
  .stop_on_error = FALSE,
  .debug = FALSE,
  .progress = FALSE,
  .options = furrr_options()
)

## S3 method for class 'simpr_spec'
fit(
  object,
  ...,
  .quiet = TRUE,
  .warn_on_error = TRUE,
  .stop_on_error = FALSE,
  .debug = FALSE,
  .progress = FALSE,
  .options = furrr_options()
)

Arguments

object

a simpr_tibble object–the simulated data from generate–or an simpr_spec object not yet generated.

...

purrr-style lambda functions used for computing on the simulated data. See Details and Examples.

.quiet

Should simulation errors be broadcast to the user as they occur?

.warn_on_error

Should there be a warning when simulation errors occur? See vignette("Managing simulation errors").

.stop_on_error

Should the simulation stop immediately when simulation errors occur?

.debug

Run simulation in debug mode, allowing objects, etc. to be explored for each attempt to fit objects.

.progress

A logical, for whether or not to print a progress bar for multiprocess, multisession, and multicore plans .

.options

The future specific options to use with the workers when using futures. This must be the result from a call to furrr_options().

Details

This is the fourth step in the simulation process: after generating the simulation data, apply functions such as fitting a statistical model to the data. The output is often then passed to tidy_fits or glance_fits to extract relevant model estimates from the object.

Similar to specify, the model-fitting ... arguments can be arbitrary R expressions (purrr-style lambda functions, see as_mapper) to specify fitting models to the data. The functions are computed within each simulation cell, so dataset names are generally unnecessary: e.g., to compute regressions on each cell, fit(linear_model = ~ lm(c ~ a + b). If your modeling function requires a reference to the full dataset, use ., e.g. fit(linear_model = ~lm(c ~ a + b, data = .).

Value

a simpr_tibble object with additional list-columns for the output of the provided functions (e.g. model outputs). Just like the output of generate, there is one row per repetition per combination of metaparameters, and the columns are the repetition number rep, the metaparameter names, the simulated data sim, with additional columns for the function outputs specified in .... If per_sim was called previously, fit returns the object to default simpr_tibble mode.

Examples

## Generate data to fit models
simple_linear_data = specify(a = ~ 2 + rnorm(n),
                               b = ~ 5 + 3*a + rnorm(n, 0, sd = 0.5)) %>%
  define(n = 100:101) %>%
  generate(2)

## Fit with a single linear term
linear_fit = simple_linear_data %>%
  fit(linear = ~lm(b ~ a, data = .))

linear_fit # first fit element also prints

## Each element of $linear is a model object
linear_fit$linear

## We can fit multiple models to the same data
multi_fit = simple_linear_data %>%
  fit(linear = ~lm(b ~ a, data = .),
      quadratic = ~lm(b ~ a + I(a^2), data = .))

## Two columns, one for each model
multi_fit

## Again, each element is a model object
multi_fit$quadratic

## Can view terms more nicely with tidy_fits
multi_fit %>%
  tidy_fits

## Can view model summaries with glance_fits
multi_fit %>%
  glance_fits

## Fit functions do not actually need to be any particular kind of model, they
## can be any arbitrary function. However, not all functions will lead to useful
## output with tidy_fits and glance_fits.
add_five_data = simple_linear_data %>%
  fit(add_five = ~ . + 5)  ## adds 5 to every value in dataset

add_five_data

Generate simulated data from specification

Description

Use specification from specify or define to produce simulated data.

Usage

## S3 method for class 'simpr_spec'
generate(
  x,
  .reps,
  ...,
  .sim_name = "sim",
  .quiet = TRUE,
  .warn_on_error = TRUE,
  .stop_on_error = FALSE,
  .debug = FALSE,
  .progress = FALSE,
  .options = furrr_options(seed = TRUE)
)

Arguments

x

a simpr_spec object generated by define or specify, containing the specifications of the simulation

.reps

number of replications to run (a whole number greater than 0)

...

filtering criteria for which rows to simulate, passed to filter. This is useful for reproducing just a few selected rows of a simulation without needing to redo the entire simulation, see vignette("Reproducing simulations"),

.sim_name

name of the list-column to be created, containing simulation results. Default is "sim"

.quiet

Should simulation errors be broadcast to the user as they occur?

.warn_on_error

Should there be a warning when simulation errors occur? See vignette("Managing simulation errors").

.stop_on_error

Should the simulation stop immediately when simulation errors occur?

.debug

Run simulation in debug mode, allowing objects, etc. to be explored for each generated variable specification.

.progress

A logical, for whether or not to print a progress bar for multiprocess, multisession, and multicore plans.

.options

The future specific options to use with the workers when using futures. This must be the result from a call to furrr_options(seed = TRUE).

Details

This is the third step in the simulation process: after specifying the population model and defining the metaparameters, if any, generate is the workhorse function that actually generates the simulated datasets, one for each replication and combination of metaparameters. You likely want to use the output of generate to fit model(s) with fit.

Errors you get using this function usually have to do with how you specified the simulation in specify and define.

Value

a simpr_sims object, which is a tibble with a row for each repetition (a total of rep repetitions) for each combination of metaparameters and some extra metadata used by fit. The columns are rep for the repetition number, the names of the metaparameters, and a list-column (named by the argument sim_name) containing the dataset for each repetition and metaparameter combination. simpr_sims objects can be manipulated elementwise by dplyr and tidyr verbs: the command is applied to each element of the simulation list-column.

Examples

meta_list_out = specify(a = ~ MASS::mvrnorm(n, rep(0, 2), Sigma = S)) %>%
  define(n = c(10, 20, 30),
       S = list(independent = diag(2), correlated = diag(2) + 2)) %>%
  generate(1)

 ## View overall structure of the result and a single simulation output
 meta_list_out

 ## Changing .reps will change the number of replications and thus the number of
 ## rows in the output
 meta_list_2 = specify(a = ~ MASS::mvrnorm(n, rep(0, 2), Sigma = S)) %>%
  define(n = c(10, 20, 30),
       S = list(independent = diag(2), correlated = diag(2) + 2)) %>%
  generate(2)

 meta_list_2

 ## Fitting, tidying functions can be included in this step by running those functions and then
 ## generate.  This can save computation time when doing large
 ## simulations, especially with parallel processing
 meta_list_generate_after = specify(a = ~ MASS::mvrnorm(n, rep(0, 2), Sigma = S)) %>%
  define(n = c(10, 20, 30),
       S = list(independent = diag(2), correlated = diag(2) + 2)) %>%
  fit(lm = ~ lm(a_2 ~ a_1, data = .)) %>%
  tidy_fits %>%
  generate(1)

  meta_list_generate_after

Create tibble of model "glances" (summaries)

Description

Turn fitted models of simulated data (from fit) into a tidy tibble of model summaries, each with one line (via broom::glance).

Usage

glance_fits(obj, ..., .progress = FALSE, .options = furrr_options())

Arguments

obj

tibble with repetition number, metaparameters, simulated data, and fitted models, from fit

...

Additional arguments to broom::glance.

.progress

A logical, for whether or not to print a progress bar for multiprocess, multisession, and multicore plans .

.options

The future specific options to use with the workers when using futures. This must be the result from a call to furrr_options().

Details

This the fifth step of the simulation process: after fitting the model with fit, now tidy the model output for further analysis such as evaluating power. All model objects should be supported by broom::glance.

The output of this function is quite useful comparing overall model fits; see Examples. For looking at specific features of the model such as tests for individual parameter estimates, use tidy_fits.

Value

a tibble with the output of the broom::glance method for the given object.

Examples

simple_linear_data = specify(a = ~ 2 + rnorm(n),
          b = ~ 5 + 3 * x1 + rnorm(n, 0, sd = 0.5)) %>%
  define(n = 100:101) %>%
  generate(2)

## Can show tidy output for multiple competing models,
compare_degree = simple_linear_data %>%
  fit(linear = ~lm(a ~ b, data = .),
      quadratic = ~lm(a ~ b + I(b^2), data = .)) %>%
  glance_fits

compare_degree

## Models can be anything supported by broom::tidy.
cor_vs_lm = simple_linear_data %>%
  fit(linear = ~lm(a ~ b, data = .),
      cor = ~ cor.test(.$a, .$b)) %>%
  glance_fits

cor_vs_lm # has NA for non-matching terms

Work directly with simulation results with dplyr and tidyr

Description

Allows applying data transformations to every simulation result with syntax as if dealing with a single simulation result using dplyr and tidyr verbs

Usage

per_sim(obj)

Arguments

obj

A simpr_tibble or simpr_spec object.

Details

After producing simulation results (a simpr_tibble object), it is sometimes needed to do some data transformation to prepare for analysis. This can always be specified in specify through custom functions, but per_sim allows you to also easily specify this in your pipeline. After running per_sim, you can use the dplyr and tidyr verbs you would use on a single simulation result and it will be applied to all results.

If, after running per_sim, you wish to return to the default behavior to access simpr_tibble results as a tibble with a list_column for simulation results again, run whole_tibble.

Value

A simpr_sims object for use with dplyr and tidyr verbs.

Examples

## Often most convenient to specify simulations for 'wide' data
data_wide = specify(a = ~ runif(5, min = 0, max = 1),
                    b = ~ runif(5, min = 0, max = 2)) %>%
  generate(2)

data_wide

## Any dplyr or tidyr verbs can be applied after per_sim()
data_long = data_wide %>%
  per_sim() %>%
  pivot_longer(everything(), names_to = "name",
               values_to = "value")
data_long

## Now, ready for analysis
data_long %>%
  fit(lm = ~lm(value ~ name)) %>%
  tidy_fits

Methods for simpr_spec class

Description

Accessor & display methods for simpr_spec class

Usage

## S3 method for class 'simpr_spec'
print(x, ...)

new_simpr_spec()

is.simpr_spec(x)

Arguments

x

a simpr_spec object

...

ignored

Details

Class simpr_spec is created by specify and/or define to specify the simulation variables, which is produced by generate. The print method provides an overview of the specification, including the conditions.

Value

print.simpr_spec has no return value and is called for its side-effects. new_simpr_spec returns an empty simpr_spec object. is.simpr_spec returns a length-1 logical vector, TRUE or FALSE, which indicates whether an object is a simpr_spec.

Examples

empty = new_simpr_spec()
print(empty)

## Easiest to create a simpr_spec with specify
simple_spec = specify(a = ~ rbinom(n, size, prob))
print(simple_spec)

## Adding on define adds all possible combinations
## of conditions and more info in output.
defined_spec = specify(a = ~ rbinom(n, size, prob)) %>%
  define(n = c(10, 20),
         size = c(20, 40),
         prob = c(0.2, 0.4))
print(defined_spec)

Objects exported from other packages

Description

These objects are imported from other packages. Follow the links below to see their documentation.

dplyr: add_count, anti_join, arrange, arrange_, as.tbl, auto_copy, collect, compute, count, distinct, distinct_, do, do_, dplyr_col_modify, dplyr_reconstruct, dplyr_row_slice, filter, filter_, full_join, group_by, group_by_, group_data, group_indices, group_indices_, group_keys, group_map, group_modify, group_nest, group_size, group_split, group_trim, group_vars, groups, inner_join, intersect, left_join, mutate, mutate_, n_groups, nest_by, nest_join, pull, relocate, rename, rename_, rename_with, right_join, rows_delete, rows_insert, rows_patch, rows_update, rows_upsert, rowwise, same_src, sample_frac, sample_n, select, select_, semi_join, setdiff, setequal, slice, slice_, slice_head, slice_max, slice_min, slice_sample, slice_tail, summarise, summarise_, tally, tbl_vars, transmute, transmute_, ungroup, union, union_all
furrr: furrr_options
generics: fit, generate, specify
magrittr: %>%
tidyr: complete, complete_, drop_na, drop_na_, expand, expand_, extract, extract_, fill, fill_, gather, gather_, nest, nest_legacy, pivot_longer, pivot_wider, replace_na, separate, separate_, separate_rows, separate_rows_, spread, spread_, unite, unite_, unnest, unnest_legacy

Specify data-generating mechanisms

Description

Specify the data-generating mechanisms for the simulation using purrr-style lambda functions.

Usage

## S3 method for class 'formula'
specify(x = NULL, ..., .use_names = TRUE, .sep = "_")

Arguments

x

leave this argument blank (NULL); this argument is a placeholder and can be skipped.

...

named purrr-style formula functions used for generating simulation variables. x is not recommended as a name, since it is a formal argument and will be automatically assumed to be the first variable (a message will be displayed if x is used).

.use_names

Whether to use names generated by the lambda function (TRUE, the default), or to overwrite them with supplied names.

.sep

Specify the separator for auto-generating names. See Column naming.

Details

This is always the first command in the simulation process, to specify the actual simulated variables, which is then passed to define to define metaparameters and then to generate to generate the data.

The ... arguments use an efficient syntax to specify custom functions needed for generating a simulation, based on the purrr package. When producing one variable, one can provide an expression such as specify(a = ~ 3 + runif(10)); the expression is preceded by ~, the tilde operator, and can refer to previous arguments in specify or to metaparameters in define. This is called a lambda function.

Order matters: arguments are evaluated sequentially, so later argument can refer to an earlier one, e.g. specify(a = ~ rnorm(2), b = ~ a + rnorm(2)).

generate combines results together into a single tibble for each simulation, so all lambda functions should produce the same number of rows. However, a lambda function can produce multiple columns.

Value

A simpr_specify object which contains the functions needed to generate the simulation; to be passed to define for defining metaparameters or, if there are no metaparameters, directly to generate for generating the simulation.

Also useful is the fact that one can refer to variables in subsequent arguments. So, one could define another variable b that depends on a very simply, e.g. specify(a = ~ 3 + runif(10), b = ~ 2 * x).

Finally, one can also refer to metaparameters that are to be systematically varied in the simulation study. See define and the examples for more details.

Column naming

Because functions can produce different numbers of columns, there are several options for naming columns. If a provided lambda function produces a single column, the name given to the argument becomes the name of the column. If the lambda function already produces column names, then the output will use these names if .use_names = TRUE, the default. Otherwise, simpr uses the argument name as a base and auto-numbers the columns. For instance, if the argument a generates a two-column matrix and .sep = "_" (the default) the columns will be named a_1and a_2.

Custom names can also be directly provided by a double-sided formula. The left-hand side must use c or cbind, e.g. specify(c(a, b) ~ MASS::mvrnorm(5, c(0, 0), Sigma = diag(2))).

Note

This function is an S3 method for specify from the generics package. Because x is a formal argument of specify, if you have a variable in your simulation named x it will be automatically moved to be the first variable (with a message). It is therefore safest to use any other variable name besides x.

Examples

## specify a variable and generate it in the simulation
single_var = specify(a = ~ 1 + rnorm(5)) %>%
  generate(1) # generate a single repetition of the simulation
single_var

two_var = specify(a = ~ 1 + rnorm(5),
                    b = ~ x + 2) %>%
  generate(1)
two_var

## Generates a_01 through a_10
autonumber_var = specify(a = ~ MASS::mvrnorm(5, rep(0, 10), Sigma = diag(10))) %>%
  generate(1)
autonumber_var

# alternatively, you could use a two-sided formula for names
multi_name = specify(cbind(a, b, c) ~ MASS::mvrnorm(5, rep(0, 3), Sigma = diag(3))) %>%
  generate(1)
multi_name

# Simple example of setting a metaparameter
simple_meta = specify(a = ~ 1 + rnorm(n)) %>%
  define(n = c(5, 10)) %>% # without this line you would get an error!
  generate(1)


simple_meta # has two rows now, one for each value of n
simple_meta$sim[[1]] # n = 5
simple_meta$sim[[2]] # n = 10

Tidy fits into a tidy tibble

Description

Turn models fit to simulated data (from fit) into a tidy tibble of model estimates (via broom::tidy).

Usage

tidy_fits(obj, ..., .progress = FALSE, .options = furrr_options())

Arguments

obj

a simpr_tibble with fitted models, from fit

...

Additional arguments to the broom::tidy method.

.progress

A logical, for whether or not to print a progress bar for multiprocess, multisession, and multicore plans .

.options

The future specific options to use with the workers when using futures. This must be the result from a call to furrr_options().

Details

The output of this function is quite useful for diagnosing bias, precision, and power. For looking at overall features of the model (e.g., R-squared), use glance_fits.

Value

a tibble with the output of the broom::tidy method applied to each model fit and then bound into a single tibble.

Examples

simple_linear_data = specify(a = ~ 2 + rnorm(n),
          b = ~ 5 + 3 * x1 + rnorm(n, 0, sd = 0.5)) %>%
  define(n = 100:101) %>%
  generate(2)

## Can show tidy output for multiple competing models,
compare_degree = simple_linear_data %>%
  fit(linear = ~lm(a ~ b, data = .),
      quadratic = ~lm(a ~ b + I(b^2), data = .)) %>%
  tidy_fits

compare_degree

## Models can be anything supported by broom::tidy.
cor_vs_lm = simple_linear_data %>%
  fit(linear = ~lm(a ~ b, data = .),
      cor = ~ cor.test(.$a, .$b)) %>%
  tidy_fits

cor_vs_lm # has NA for non-matching terms

Simpr methods for tidyverse verbs

Description

These are simpr-compatible methods for generic dplyr and tidyr verbs. The user is not expected to call these methods directly.