Research

This page is organized by research area:

  1. Bias, Fairness, and Equity
  2. Affective Computing in Education
  3. Predictive Modelling Applications: Games, Identity, Affect
  4. Methodological Improvements: Statistics, Machine Learning

For the full list of papers, please visit my Google Scholar page. If the data for a paper is not made public yet, the access is restricted under IRB - reach out to discuss options.


Bias, Fairness, and Equity


Adaptive systems in education need to ensure that their pedagogical decisions meet the needs of all students for an equitable outcome. Recent research highlights how these systems encode societal biases leading to discriminatory behaviors towards specific student subpopulations. However, the focus has mostly been on investigating bias in predictive modeling, particularly its downstream stages like model development and evaluation.

Upstream Bias

My research on upstream bias hypothesizes that the upstream sources (i.e., theory, design, training data collection method) in the development of adaptive systems also contribute to the bias in these systems, highlighting the need for a nuanced approach to conducting fairness research. By empirically analyzing student data collected from various virtual learning environments, we investigated demographic disparities in three cases representative of the aspects that shape technological advancements in education:

1) differing implications of technology design on student outcomes

2) non-conformance of data to a widely-accepted theoretical model of emotion

3) varying effectiveness of methodological improvements in annotated data collection

Historical Bias

I argue that some biases are also rooted deeply in the fundamental principles of data and algorithms. By sociohistorically contextualizing three illustrative scenarios, we assert in this paper that historical injustices perpetuated by algorithmic systems in education are the result of the colonial epistemologies that continue to shape them.


Affective Computing in Education


Affect Dynamics

Student affect in adaptive systems has been shown to correlate with a range of important educational constructs. Affect dynamics, the study of how affect develops and manifests over time, has become a popular area of research in affective computing for learning. Few empirical studies, however, have matched the predictions of the most commonly-cited theoretical model of affect dynamics. We first analyzed the prior empirical studies, elaborating both their findings and the contextual and methodological differences between these studies. We also addressed some methodological concerns that have not been previously addressed in the literature, discussing how various edge cases should be treated.

Next, we presented mathematical evidence that several past studies applied the transition metric incorrectly - leading to invalid conclusions of statistical significance - and provided a corrected method.

Using this corrected analysis method, we reanalyzed ten past affect datasets collected in diverse contexts and synthesized the results, determining that the findings do not match the most popular theoretical model of affect dynamics. Instead, our results highlights the need to focus on cultural factors in future affect dynamics research.

For my work on methodological improvements in affect dynamics research, see:

Affect Analysis

I analyzed student affect data to better understand its role in student learning and experience, including a randomized controlled study on an affective intervention and a study on the relationship between affect and game design.

Affect Detection

I designed and developed algorithms for automated affect detection in diverse learning systems using physiological sensors and interaction log data. I also developed the server-side software for an app that identifies and alerts field interviewers to critical affective moments during a student’s learning with an artificially intelligent learning system.


Predictive Modelling Applications


Learning Games

When a student is struggling in a learning game, a relevant and timely intervention could keep the student motivated and prevent frustration from leading the student to give up. However, it may be undesirable – even demotivating and harmful to learning – if the student is provided with scaffolding when they do not need it. As such, we developed a model to detect whether a student is likely to give up and quit a level in progress to identify opportunities to intervene meaningfully with supports aimed at improving student learning and engagement.

While predicting when a student is likely to quit is important, it is also crucial to understand why the student is likely to quit in order to inform the design of supports that address students’ individual needs. Using automatically generated events in the interaction log as codes, we study how the temporal interconnections between the events are different for students who quit and those who did not. Our analysis revealed a set of themes that point at some potential root causes for why students quit a game level unsolved.

Math Identity and Success

Math identity — the degree to which one considers oneself a “math person” — has been researched to better understand what drives students to enter STEM fields. Using text mining, click-stream analysis, and temporal analysis, we developed models of students’ math success and math identity.

More importantly, we demonstrated that the relationship between a predictor variable (e.g., number of hints used) and the outcome of interest (math self-concept, which is an affective measure of students’ perception of their own cognitive ability) varies significantly based on the context. Such demographic differences are likely to limit the generalizability of the models of math identity.

Affect Detection


Methodological Improvements


Part of my research also focuses on methodological innovations in statistics and machine learning. Most of this work is inspired by the challenges I encountered in using existing methods to conduct my primary research with education data.

Transition and Sequence Analysis

For around a decade, the L statistic has been used to evaluate the probability of transitions between states (or events). However, we found that a minor pre-processing step (excluding self-transitions), used in many papers, leads to a violation of the assumption of independence in L. We provided a simple correction to fix this statistical bias.

Although the previous solution attended to the primary statistical error, it made the statistic difficult and non-intuitive to interpret. Motivated by this challenge, we proposed a modified version of the statistic (L*) that fixed the bias by definition.

However, our previous study also discovered further issues with the L statistic involving states with high base rates. Another simulation study (Bosch & Paquette, 2021) reported issues with shorter sequences. These continuing issues with the L statistic suggested that an alternate approach may be warranted. We presented two alternative procedures to conduct transition analysis that attempt to address these problems using: 1) epistemic network analysis and 2) marginal models.

Active Machine Learning

Active (machine) Learning (AL) methods have been explored in education to improve the data labeling efficiency. However, due to the complexity of educational constructs and data, AL has suffered from the cold-start problem where the model does not have access to sufficient data yet. We experimented the use of past data to overcome this issue and found that it could be effective based on the target student population.

Multiple Comparisons

Research studies with education data often involve several variables related to student learning activities making it necessary to run multiple statistical tests simultaneously. We investigated the validity of methods used to adjust for false discoveries, showing that a frequently used procedure (Benjamini-Hochberg) may not be appropriate for two common scenarios in educational data mining, and recommend an alternate procedure (Benjamini-Yekutieli).