Categorical variables

library(elicitr)
#> Registered S3 method overwritten by 'car':
#>   method           from
#>   na.action.merMod lme4

Many of the concepts introduced in vignette("continuous_variables") are also applicable to categorical variables, and the name of the functions are the same but have the prefix cat instead of cont. However, there are some differences in the workflow for loading and analysing data collected during the elicitation of categorical variables. This vignette will guide you through the process of loading and analysing categorical data.

Datasets

There are three datasets included in the package for demonstration purposes: ?topic_1, ?topic_2, and ?topic_3:

topic_1
#> # A tibble: 120 × 5
#>    name            option   category   confidence estimate
#>    <chr>           <chr>    <chr>           <dbl>    <dbl>
#>  1 Derek Maclellan option_1 category_1         66     0.57
#>  2 Derek Maclellan option_1 category_2         66     0.18
#>  3 Derek Maclellan option_1 category_3         66     0.02
#>  4 Derek Maclellan option_1 category_4         66     0.02
#>  5 Derek Maclellan option_1 category_5         66     0.21
#>  6 Derek Maclellan option_2 category_1         86     0.06
#>  7 Derek Maclellan option_2 category_2         86     0.04
#>  8 Derek Maclellan option_2 category_3         86     0.12
#>  9 Derek Maclellan option_2 category_4         86     0.42
#> 10 Derek Maclellan option_2 category_5         86     0.36
#> # ℹ 110 more rows

topic_2
#> # A tibble: 100 × 5
#>    name              option   category   confidence estimate
#>    <chr>             <chr>    <chr>           <dbl>    <dbl>
#>  1 Christopher Felix option_1 category_1         86     0.28
#>  2 Christopher Felix option_1 category_2         86     0.13
#>  3 Christopher Felix option_1 category_3         86     0.55
#>  4 Christopher Felix option_1 category_4         86     0.04
#>  5 Christopher Felix option_1 category_5         86     0   
#>  6 Christopher Felix option_2 category_1         81     0.06
#>  7 Christopher Felix option_2 category_2         81     0.26
#>  8 Christopher Felix option_2 category_3         81     0.02
#>  9 Christopher Felix option_2 category_4         81     0.47
#> 10 Christopher Felix option_2 category_5         81     0.19
#> # ℹ 90 more rows

topic_3
#> # A tibble: 90 × 5
#>    name            option   category   confidence estimate
#>    <chr>           <chr>    <chr>           <dbl>    <dbl>
#>  1 Derek Maclellan option_1 category_1         61     0.12
#>  2 Derek Maclellan option_1 category_2         61     0.5 
#>  3 Derek Maclellan option_1 category_3         61     0.16
#>  4 Derek Maclellan option_1 category_4         61     0.02
#>  5 Derek Maclellan option_1 category_5         61     0.2 
#>  6 Derek Maclellan option_2 category_1         91     0.02
#>  7 Derek Maclellan option_2 category_2         91     0.76
#>  8 Derek Maclellan option_2 category_3         91     0.15
#>  9 Derek Maclellan option_2 category_4         91     0.06
#> 10 Derek Maclellan option_2 category_5         91     0.01
#> # ℹ 80 more rows

In each dataset the first column contains the name of the expert, the second the options considered and the third the categories of the categorical variable. The fourth column contains the expert’s confidence, and the fifth the expert’s estimate. Expert estimates for each option should represent probabilities or percentages, and should thus sum up to 1 or 100. Both are accepted and data scaled to 1 will be automatically rescaled to 100.

Load data

We start by creating the ?elic_cat object with the function cat_start(). As for the continuous variables, this objects stores the metadata of the elicitation process:

my_categories <- c("category_1", "category_2", "category_3",
                   "category_4", "category_5")
my_options <- c("option_1", "option_2", "option_3", "option_4")
my_topics <- c("topic_1", "topic_2", "topic_3")
my_elicitation <- cat_start(categories = my_categories,
                            options = my_options,
                            experts = 6,
                            topics = my_topics)
#> ✔ <elic_cat> object for "Elicitation" correctly initialised
my_elicitation
#> 
#> ── Elicitation ──
#> 
#> • Categories: "category_1", "category_2", "category_3", "category_4", and
#> "category_5"
#> • Options: "option_1", "option_2", "option_3", and "option_4"
#> • Number of experts: 6
#> • Topics: "topic_1", "topic_2", and "topic_3"
#> • Data available for 0 topics

This elicitation process is for a categorical variables with 5 categories estimated for four options and three topics by six experts.

As we did for continuous variables, we can load the data with the function cat_add_data():

my_elicitation <- cat_add_data(my_elicitation,
                               data_source = topic_1,
                               topic = "topic_1") |>
  cat_add_data(data_source = topic_2, topic = "topic_2") |>
  cat_add_data(data_source = topic_3, topic = "topic_3")
#> ℹ Estimates sum to 1. Rescaling to 100.
#> ✔ Data added to Topic "topic_1" from "data.frame"
#> 
#> ℹ Estimates sum to 1. Rescaling to 100.
#> ✔ Data added to Topic "topic_2" from "data.frame"
#> 
#> ℹ Estimates sum to 1. Rescaling to 100.
#> ✔ Data added to Topic "topic_3" from "data.frame"

As mentioned before, estimates can also sum up to 100.

topic_1_percent <- dplyr::mutate(topic_1,
                                 estimate = estimate * 100)

my_elicitation <- cat_add_data(my_elicitation,
                               data_source = topic_1_percent,
                               topic = "topic_1")
#> ✔ Data added to Topic "topic_1" from "data.frame"

Expert anonymisation is automatic but optional. If expert names are not to be anonymised, the argument anonymise can be set to FALSE in the cat_add_data() function.

my_elicitation <- cat_add_data(my_elicitation,
                               data_source = topic_1,
                               topic = "topic_1",
                               anonymise = FALSE)
#> ℹ Estimates sum to 1. Rescaling to 100.
#> ✔ Data added to Topic "topic_1" from "data.frame"

Again, metadata are used to validate the data. If the data is not consistent with the metadata, an error message will be displayed. For example, if we try to load data with a category not defined in the metadata:

malformed_data <- topic_1
malformed_data[1, 3] <- "category_6"
cat_add_data(my_elicitation,
             data_source = malformed_data,
             topic = "topic_1")
#> Error in `cat_add_data()`:
#> ! The column with the name of the categories contains unexpected values:
#> ✖ The value "category_6" is not valid.
#> ℹ Check the metadata in the <elic_cat> object.

Get data

Data can be retrieved from the elic_cat object with the cat_get_data() function:

cat_get_data(my_elicitation, topic = "topic_1")
#> # A tibble: 120 × 5
#>    id              option   category   confidence estimate
#>    <chr>           <chr>    <chr>           <dbl>    <dbl>
#>  1 Derek Maclellan option_1 category_1         66       57
#>  2 Derek Maclellan option_1 category_2         66       18
#>  3 Derek Maclellan option_1 category_3         66        2
#>  4 Derek Maclellan option_1 category_4         66        2
#>  5 Derek Maclellan option_1 category_5         66       21
#>  6 Derek Maclellan option_2 category_1         86        6
#>  7 Derek Maclellan option_2 category_2         86        4
#>  8 Derek Maclellan option_2 category_3         86       12
#>  9 Derek Maclellan option_2 category_4         86       42
#> 10 Derek Maclellan option_2 category_5         86       36
#> # ℹ 110 more rows

Notice that the name of the expert is not anonymised in this case and assigned to the column id.

Data can also be retrieved only for given options:

cat_get_data(my_elicitation, topic = "topic_2", option = "option_1")
#> # A tibble: 25 × 5
#>    id      option   category   confidence estimate
#>    <chr>   <chr>    <chr>           <dbl>    <dbl>
#>  1 e51202e option_1 category_1         86       28
#>  2 e51202e option_1 category_2         86       13
#>  3 e51202e option_1 category_3         86       55
#>  4 e51202e option_1 category_4         86        4
#>  5 e51202e option_1 category_5         86        0
#>  6 e78cbf4 option_1 category_1         71        5
#>  7 e78cbf4 option_1 category_2         71       20
#>  8 e78cbf4 option_1 category_3         71        7
#>  9 e78cbf4 option_1 category_4         71       47
#> 10 e78cbf4 option_1 category_5         71       21
#> # ℹ 15 more rows

Here, the data is anonymised, following what was specified when loading the data.

Data analysis

Contrary to continuous variables, there is not yet a function for plotting the raw data. However, we can plot the distribution of the sampled data.

Sample data

Data can be sampled using the function cat_sample_data() (see the variable documentation for the explanation of the sampling methods). Two sampling methods are available, which sample data from a Dirichlet distribution using the expert estimates as parameters. When only one estimate is provided, i.e. 100 % for one category, the method assigns 100 % to all votes for this category.

The unweighted method: This method samples data based on the expert estimates without accounting for their confidence.
The weighted method: This method samples data based on the expert estimates accounting for their confidence. The confidence is used to weight the number of votes assigned to each expert.

Here we sample 100 values for each expert in each option (all options in topic_1 are sampled). Each expert are basically given 100 votes.

samp <- cat_sample_data(my_elicitation,
                        method = "unweighted",
                        topic = "topic_1",
                        n_votes = 100)
#> ✔ Data sampled successfully using "unweighted" method.
samp
#> # A tibble: 2,400 × 7
#>    id              option category_1 category_2 category_3 category_4 category_5
#>    <chr>           <chr>       <dbl>      <dbl>      <dbl>      <dbl>      <dbl>
#>  1 Derek Maclellan optio…      0.564      0.210    0.00780    0.0148       0.203
#>  2 Derek Maclellan optio…      0.517      0.119    0.101      0.0176       0.245
#>  3 Derek Maclellan optio…      0.527      0.256    0.0134     0.0213       0.183
#>  4 Derek Maclellan optio…      0.545      0.103    0.0288     0.0139       0.310
#>  5 Derek Maclellan optio…      0.651      0.105    0.0321     0.00616      0.206
#>  6 Derek Maclellan optio…      0.611      0.161    0.0102     0.00310      0.214
#>  7 Derek Maclellan optio…      0.611      0.134    0.0181     0.0123       0.224
#>  8 Derek Maclellan optio…      0.585      0.182    0.0112     0.0135       0.208
#>  9 Derek Maclellan optio…      0.611      0.152    0.0134     0.0150       0.209
#> 10 Derek Maclellan optio…      0.595      0.146    0.0311     0.0119       0.216
#> # ℹ 2,390 more rows

Here we sample 100 values for each expert in each option (all options in topic_1 are sampled). Each expert gets a portion of the total number of vote based on their self-expressed confidence level.

samp <- cat_sample_data(my_elicitation,
                        method = "unweighted",
                        topic = "topic_1",
                        n_votes = 100)
#> ✔ Data sampled successfully using "unweighted" method.
samp
#> # A tibble: 2,400 × 7
#>    id              option category_1 category_2 category_3 category_4 category_5
#>    <chr>           <chr>       <dbl>      <dbl>      <dbl>      <dbl>      <dbl>
#>  1 Derek Maclellan optio…      0.495      0.197    0.0129     0.0136       0.282
#>  2 Derek Maclellan optio…      0.439      0.271    0.0267     0.0155       0.249
#>  3 Derek Maclellan optio…      0.577      0.186    0.0660     0.0175       0.153
#>  4 Derek Maclellan optio…      0.485      0.185    0.00543    0.0420       0.283
#>  5 Derek Maclellan optio…      0.521      0.189    0.0355     0.0150       0.240
#>  6 Derek Maclellan optio…      0.566      0.220    0.00549    0.00738      0.201
#>  7 Derek Maclellan optio…      0.588      0.191    0.0149     0.0204       0.185
#>  8 Derek Maclellan optio…      0.537      0.210    0.0263     0.00874      0.218
#>  9 Derek Maclellan optio…      0.539      0.177    0.0116     0.0209       0.251
#> 10 Derek Maclellan optio…      0.576      0.175    0.00651    0.00906      0.234
#> # ℹ 2,390 more rows

Data can be sampled using the function cat_sample_data() (see the variable documentation for the explanation of the sampling methods). Here we sample 100 values for each expert in each option:

samp <- cat_sample_data(my_elicitation,
                        method = "unweighted",
                        topic = "topic_1",
                        n_votes = 100)
#> ✔ Data sampled successfully using "unweighted" method.
samp
#> # A tibble: 2,400 × 7
#>    id              option category_1 category_2 category_3 category_4 category_5
#>    <chr>           <chr>       <dbl>      <dbl>      <dbl>      <dbl>      <dbl>
#>  1 Derek Maclellan optio…      0.501      0.230   0.0198      0.0309       0.218
#>  2 Derek Maclellan optio…      0.559      0.143   0.0149      0.0178       0.266
#>  3 Derek Maclellan optio…      0.620      0.124   0.000985    0.0186       0.236
#>  4 Derek Maclellan optio…      0.587      0.174   0.00663     0.0347       0.198
#>  5 Derek Maclellan optio…      0.588      0.201   0.0113      0.0366       0.164
#>  6 Derek Maclellan optio…      0.544      0.181   0.0488      0.0201       0.205
#>  7 Derek Maclellan optio…      0.614      0.134   0.0116      0.0126       0.227
#>  8 Derek Maclellan optio…      0.644      0.184   0.0139      0.00510      0.153
#>  9 Derek Maclellan optio…      0.495      0.202   0.0327      0.00879      0.262
#> 10 Derek Maclellan optio…      0.516      0.250   0.0232      0.0158       0.196
#> # ℹ 2,390 more rows

Sampled data can be summarised for any option:

summary(samp, option = "option_1")
#> $option_1
#> # A tibble: 5 × 7
#>   category        Min     Q1 Median  Mean    Q3   Max
#>   <chr>         <dbl>  <dbl>  <dbl> <dbl> <dbl> <dbl>
#> 1 category_1 0.0742   0.177   0.376 0.386 0.561 0.815
#> 2 category_2 0        0.0551  0.118 0.121 0.182 0.384
#> 3 category_3 0.000985 0.0472  0.104 0.129 0.187 0.422
#> 4 category_4 0.00175  0.0476  0.233 0.220 0.308 0.603
#> 5 category_5 0.0192   0.0869  0.145 0.145 0.198 0.366

And plotted as violin plot:

plot(samp)

Violin plot of the sampled data for all options.

Or as beeswarm plot, which can be adapted as needed:

plot(samp, type = "beeswarm")

Beeswarm plot of the sampled data for all options. Each point represents a sampled value.

plot(samp, type = "beeswarm",
     beeswarm_cex = 0.9, beeswarm_corral = "wrap")

Beeswarm plot of the sampled data for all options. Each point represents a sampled value.

We can also plot the distribution for a specific option:

plot(samp, option = "option_2")

Violin plot of the sampled data for option 2.