Intro

• essay writing : good for logical thinking, critical reasoning, creative thinking
• human essay writing?
  • too expensive
  • not sufficiently accurate
    • raters characteristics (strictness, inconsistency)
• Automated Essay Scoring(AES)
  • statistical classification problem : large set of testual entities into a small number of discrete categories
  1. feature-engineering
     1. manually tuned features (length, spelling errors)
     2. advantage: interpretability, explainability
     3. extensive feature redesign required
  2. automatic feature extraction
     1. based on DNN
     2. requires a large dataset of essays graded by human raters as training data
     3. problem: bias effect
        1. performance of DNN rapidly drops when biased data are used for model training
     4. to solve: used IRT
• DNN-AES
  • IRT to estimate reliable scores that remove effects of rater bias
  • DNN-AES (conventional)
  • suited to educational context, low- and medium-stakes tests

Data

• J: examinees
• R: raters
• e: essay
• U: score
• V: vocab list for essay collection

Neural Automated Essay Scoring Models

배경지식

• DNN vs. CNN vs. RNN

  • DNN : 기본적인 1차원 데이터를 활용하기 때문에 공간적, 지역적 정보 손실 + 학습시간과 능률의 효율성 저하

  • CNN : 이미지 raw input으로 공간적, 지역적 정보 유지. 전체보다는 부분에 집중

    

    • convolutional layer : convolution + activation

      

    • pooling layer : feature map의 차원 축소

    • vectorization(flatten) : 이제 특이점 데이터들이니 1차원으로 변형해도 됩니다.

      

  • RNN : Recurrent Neural Network(앞을 되짚어 생각하는 모델) 중에서 장기 기억이 가능한 LSTM

    • 기존의 RNN

      

    • LSTM (컨베이어 벨트 같은 cell state, 수정하는 gate들과 연산)

      

본론

• CNN-LSTM-Based Model
  • Lookup table layer _ why?
  • Convolution layer : n-gram, zero padding
  • Recurrent layer
  • Pooling layer : MoT(mean-over-time) pooling for stable accuracy
  • Linear layer with sigmoid activation
• BERT-Based Model

  • pretraining

    • masked language modeling
    • next-sentence prediction

  • fine-tuning(CLS, SEP)