Small n Logo

Publication Details

Crowdsourced Analysis of Single-Case Experimental Design Data

Stevenson, Nathan and Peltier, Corey and Cosottile, David and Pollack, Marney and Ridden, Benjamin and Kang, Veronica and Cox, David and Farmer, Ryan and Robertson, Rachel and Bennett Eyler, Paige and Winchester, Claire and Knowles, Christen and Tincani, Matt and Dueñas, Ana and Haq, Shaji and Hirsch, Shanna and Sallese, Mary Rose and Wolfe, Katie and Alresheed, Fahad and Kittelman, Angus and Dunkel, Sarah and Todt, Mollie and Gilroy, Shawn and Ousley, Ciara and Lambert, Joseph and Nese, Rhonda and Bundock, Kaitlin and Kirkpatrick, Marie and Seven, Yagmur and Vannest, Kimberly and Ruiz, Salvador and Ledford, Jennifer and Bruhn, Allison and Ferron, John and Dowdy, Art and Dueker, Scott and Kirby, Megan and Witts, Ben and Tuck, Kathleen and Biggs, Elizabeth and Ennis, Robin and Royer, David and Joslyn, P. and McCammon, Meka and Rodriguez, Billie Jo and Jimenez, Bree and Travers, Jason and Coleman, Heather and Bhana, Naima and Majeika, Caitlyn and Kunze, Megan and Dart, Evan and Lüke, Timo

Resources:

Share:

Abstract:

Replicable data analysis procedures are a cornerstone of empirical scientific research. Data from single-case experimental designs (SCEDs) are typically analyzed using visual and/or quantitative analysis. Visual analysis is a structured examination of graphical displays of participant data, which involves evaluating level, trend, and variability within each phase of the experiment and then evaluating changes across phases to determine the confidence in stating a functional relation exists. Quantitative analyses are used to quantify intervention effectiveness. The following study examined visual analysis decisions and procedures for consistency across 51 independent expert raters through crowdsourced analysis. The consistency of quantitative analyses was also examined across 26 analysts who elected to conduct quantitative analysis following visual analysis. Results show 80.3% of analysts determined data were insufficient to support a functional relation between the intervention and dependent variable. The selection of metrics and results of quantitative analyses varied widely. Implications and limitations are discussed.

Keywords:

crowdsourcing
single-case experimental design
single-subject design
visual analysis
crowdsource analysis

Citation:

Stevenson, Nathan, Peltier, Corey, Cosottile, David, Pollack, Marney, Ridden, Benjamin, Kang, Veronica, Cox, David, Farmer, Ryan, Robertson, Rachel, Bennett Eyler, Paige, Winchester, Claire, Knowles, Christen, Tincani, Matt, Dueñas, Ana, Haq, Shaji, Hirsch, Shanna, Sallese, Mary Rose, Wolfe, Katie, Alresheed, Fahad, Kittelman, Angus, Dunkel, Sarah, Todt, Mollie, Gilroy, Shawn, Ousley, Ciara, Lambert, Joseph, Nese, Rhonda, Bundock, Kaitlin, Kirkpatrick, Marie, Seven, Yagmur, Vannest, Kimberly, Ruiz, Salvador, Ledford, Jennifer, Bruhn, Allison, Ferron, John, Dowdy, Art, Dueker, Scott, Kirby, Megan, Witts, Ben, Tuck, Kathleen, Biggs, Elizabeth, Ennis, Robin, Royer, David, Joslyn, P., McCammon, Meka, Rodriguez, Billie Jo, Jimenez, Bree, Travers, Jason, Coleman, Heather, Bhana, Naima, Majeika, Caitlyn, Kunze, Megan, Dart, Evan, Lüke, Timo (2026). Crowdsourced Analysis of Single-Case Experimental Design Data. Research in Special Education, 3. DOI: 10.25894/RISE.2840.

Crowdsourced Analysis of Single-Case Experimental Design Data