Horbach et al., 2019

The influence of variance in learner answers on automatic content scoring

Andrea Horbach and Torsten Zesch

Variance

Sources of variance

• Conceptual variance:

  • when there are multiple separate right answers to a question.
  • bigger issue is number of variants of incorrect answers. why not focus on modelling correct answers? Could you use an approach that allows you to rely more on how close this answer is to the correct answers I saw in training (if generative, I'm not sure how this would work for discriminative) could you model correct/wrong questions as anomaly detection?

• Variance in realization

  • different ways of forming the same conceptual answer
  • Linguistic variation
    • language provides lots of possibilities to express the same meaning what if you did reparsing or something to map variant forms to roughly the same meaning