1334. sredin seminar: Plamen Mirazchiyski: RALSA: The R Analyzer for Large-Scale Assessments

RALSA: The R Analyzer for Large-Scale Assessments

International large-scale assessments and surveys (ILSA) are conducted for more than 60 years. The era of modern ILSA started with the Third International Mathematics and Science Study (TIMSS) in 1995 which is conducted in every four years. Other prominent modern ILSAs are the Progress in International Reading Literacy Study (PIRLS) and Programme for International Student Assessment (PISA). Their aim is to assess and evaluate student achievement in different domains on population level in participating countries.

Modern ILSAs use complex sampling and assessment designs. Sampling is with probability proportional to the size (PPS) of the units. The assessment design uses multiple matrix sampling (MMS) of items where no tested individual takes all items and no item is taken by all individual. Instead, ILSAs use the so-called “plausible values” (PVs) which in their essence are imputed variables and, as a result, each individual is assigned more than one achievement score in the assessed domain. Different studies use different number of PVs - TIMSS, PIRLS and other use five PVs, PISA uses 10.

Analysis of ILSAs data require analysis techniques that differ from the standard statistical procedures - using resampling and imputation techniques. ILSAs’ different assumptions and variety in the implementation of the aforementioned methods results in variation of the implementation of the analysis routines. There are different software solutions to handle the varying statistical complexities across ILSAs’ datasets. Different R packages are available for analyzing ILSAs’ data, but they require fluent knowledge in R to specify the design by hand in order to analyze different studies’ dataset which can be daunting for the users and is prone to errors. The R Analyzer for Large-Scale Assessments (RALSA) has been released recently. This is an open-source and free of charge R package that, different from other packages, can handle all statistical complexities and apply the statistical routines automatically as per the ILSA in scope, preventing the user from common mistakes. RALSA is the only R package that has a graphical user interface (GUI). The GUI is written in the R shiny package and works on all operating systems where R can operate.

PS. Anyone who would like to present something at the next seminars should let me know the title of the topic and add a short summary.