Open Scoring API Client for R

R-CMD-check CRAN/METACRAN Version Codecov test coverage

Creativity research involves the need to score open-ended problems. Usually done by humans, automatic scoring using AI becomes more and more accurate. This package provides a simple interface to the ‘Open Scoring’ API, leading creativity scoring technology by Organiscak et al. (2023). With it, you can score your own data directly from an R script.

Installation

Install the released version of openscoring from CRAN with:

install.packages("openscoring")

You can install the development version of openscoring from GitHub with:

# install.packages("pak")
pak::pak("jakub-jedrusiak/openscoring")

Example

library(openscoring)

df <- tibble::tibble(
  stimulus = c("brick", "hammer", "sponge"),
  response = c(
    "butter for trolls",
    "make Thor jealous",
    "make it play in a kids show"
  )
)

df
#> # A tibble: 3 × 2
#>   stimulus response                   
#>   <chr>    <chr>                      
#> 1 brick    butter for trolls          
#> 2 hammer   make Thor jealous          
#> 3 sponge   make it play in a kids show

scored_df <- ocsai(df, stimulus, response, model = "2-xs")

scored_df
#> # A tibble: 3 × 4
#>   stimulus response                    .originality .confidence
#>   <chr>    <chr>                              <dbl>       <dbl>
#> 1 brick    butter for trolls                   2.5         0.49
#> 2 hammer   make Thor jealous                   3.34        0.47
#> 3 sponge   make it play in a kids show         2.97        0.48

The "1.5" model works for multiple languages:

df_polish <- tibble::tibble(
  stimulus = c("cegła", "młotek", "gąbka"),
  response = c(
    "masło dla trolli",
    "wywoływanie zazdrości u Thora",
    "postać w programie dla dzieci"
  )
)

ocsai(df_polish, stimulus, response, model = "2", language = "Polish")
#> # A tibble: 3 × 4
#>   stimulus response                      .originality .confidence
#>   <chr>    <chr>                                <dbl>       <dbl>
#> 1 cegła    masło dla trolli                      2.55        0.54
#> 2 młotek   wywoływanie zazdrości u Thora         3.08        0.84
#> 3 gąbka    postać w programie dla dzieci         3.21        0.62

Confidence scores

By default, logprob_scoring is TRUE, which adds a .confidence column alongside the originality scores. Set logprob_scoring = FALSE to disable it, or rename the column with confidence_col:

ocsai(df, stimulus, response, model = "2-xs", confidence_col = "confidence")
#> # A tibble: 3 × 4
#>   stimulus response                    .originality confidence
#>   <chr>    <chr>                              <dbl>      <dbl>
#> 1 brick    butter for trolls                   2.5        0.49
#> 2 hammer   make Thor jealous                   3.34       0.47
#> 3 sponge   make it play in a kids show         2.97       0.48

API key

The API works without authentication for moderate usage. To increase rate limits, pass your key with the api_key argument (it is sent as the X-API-KEY header):

ocsai(
  df,
  stimulus,
  response,
  model = "2",
  api_key = Sys.getenv("OPENSCORING_API_KEY")
)

For safety, keep keys out of scripts and set them as environment variables.