Skip to contents

Model Validation

Source: vignettes/model-validation.Rmd
model-validation.Rmd

NOTE: Example df is small and simulated, which causes non-convergence errors in some models.

library(rebl)
pacman::p_load(
  reactable,
  dplyr
)

Start by getting our CML model:

rebl_items <- id_rebl_items(
  df = example, 
  pattern = '^(?!res).*', 
  perl = TRUE
)

model_cml <- get_rasch_model(
  df = example, 
  id = 'respondent_id', 
  rebl_items = rebl_items,
  type = 'cml'
)

model_con <- get_rasch_model(
  df = example, 
  id = 'respondent_id', 
  rebl_items = rebl_items,
  type = 'mml_con'
)

Measurement Invariance

Unidimensionality

Person and Item Fit

Person Fit

Use the get_person_fits function for either CML or MML models. For eRm model:

person_fits <- get_person_fits(model_cml)
head(person_fits)
#>   id    p.fit p.df p.outfitMSQ p.infitMSQ  p.outfitZ   p.infitZ
#> 1 p1 26.25577   24   1.0939906  1.0820874  1.1567754  1.0703092
#> 2 p2 21.71436   24   0.9047649  0.9092663 -1.1926572 -1.1981892
#> 3 p3 25.16591   24   1.0485795  1.0443576  0.6855344  0.6581565
#> 4 p4 20.21121   24   0.8421338  0.8573618 -1.5843117 -1.5596026
#> 5 p5 20.72360   24   0.8634835  0.8707686 -1.3609121 -1.3812978
#> 6 p6 27.76184   24   1.1567434  1.1295124  0.9355978  0.8631123

TODO: Interpretation

Same thing for ltm models:

person_fits <- get_person_fits(model_con)
#> Warning: There was 1 warning in `dplyr::arrange()`.
#> ℹ In argument: `..1 = as.numeric(id)`.
#> Caused by warning:
#> ! NAs introduced by coercion
head(person_fits)
#>    id        L0          Lz          p
#> 1 p77 -17.56102 -1.49687625 0.06721273
#> 2 p37 -15.66684  0.50445151 0.69302794
#> 3 p45 -14.78634  0.25237416 0.59962406
#> 4 p90 -15.05879  0.42184543 0.66343108
#> 5 p79 -15.86456  0.07596912 0.53027817
#> 6 p85 -14.14752  2.37984952 0.99134015

Here, L0L_0 is the Levine and Rubin (1979) test statistic, LzL_z is the standardized version (Drasgow et al., 1985), and pp is the test of person fit where the null hypothesis is good fit.

Item Fit 

fits <- get_item_fits(
  model = model_cml,
  df = example,
  rebl_items = rebl_items
)
fits %>% 
  reactable::reactable(
    resizable = TRUE,
    filterable = TRUE,
    searchable = TRUE,
    outlined = TRUE,
    bordered = TRUE,
    striped = TRUE,
    compact = TRUE,
    defaultColDef = colDef(format = colFormat(digits = 3)),
    columns = list(
      i.df = colDef(format = colFormat(digits = 0))
    )
  )

Note that the item fit function only works for CML models currently. More coming soon.