tojohere/goemotions-ja-xlm-roberta-base

GoEmotions Japanese — xlm-roberta-base (ONNX INT8)

28-emotion sentiment classifier for Japanese text, fine-tuned on GoEmotions + WRIME native Japanese data.

Model Details

• Base model: FacebookAI/xlm-roberta-base (270M params)
• Training data: GoEmotions EN+JA bilingual (87K) + WRIME native Japanese (24K) = 111K examples
• Translation: Helsinki-NLP/opus-tatoeba-en-ja (English → Japanese)
• Native data: WRIME v2 — 35K Japanese social media texts with Plutchik emotion annotations, mapped to GoEmotions labels
• Format: ONNX INT8 quantized (266MB)
• Tokenizer: SentencePiece (tokenizer.json, Unigram)

Performance

Metric	Score
F1 Macro (validation)	0.376
Top-1 accuracy (28×3 test)	56%
Top-3 accuracy (28×3 test)	74%
Sanity checks	8/8
Perfect emotions	8/28
Missed emotions	3/28 (grief, nervousness, pride)

Labels (28 GoEmotions)

admiration, amusement, anger, annoyance, approval, caring, confusion, curiosity, desire, disappointment, disapproval, disgust, embarrassment, excitement, fear, gratitude, grief, joy, love, nervousness, optimism, pride, realization, relief, remorse, sadness, surprise, neutral

Usage (Python + ONNX Runtime)

import onnxruntime as ort
import numpy as np
from tokenizers import Tokenizer

session = ort.InferenceSession("goemotions.onnx")
tokenizer = Tokenizer.from_file("tokenizer.json")
tokenizer.enable_padding(length=128, pad_id=1, pad_token="<pad>")
tokenizer.enable_truncation(max_length=128)

text = "今日はとても幸せで感謝しています"
enc = tokenizer.encode(text)
input_ids = np.array([enc.ids], dtype=np.int64)
attention_mask = np.array([enc.attention_mask], dtype=np.int64)

logits = session.run(None, {"input_ids": input_ids, "attention_mask": attention_mask})[0][0]
probs = np.exp(logits - logits.max()) / np.exp(logits - logits.max()).sum()
top_idx = probs.argmax()

Training Details

• Epochs: 10
• Learning rate: 1e-5
• Batch size: 32
• Warmup: 10%
• Optimizer: AdamW (weight_decay=0.01)
• Hardware: RTX 4090 (RunPod)
• Training time: ~50 minutes
• WRIME mapping: Joy→joy, Sadness→sadness, Anticipation→excitement+optimism, Surprise→surprise, Anger→anger, Fear→fear, Disgust→disgust, Trust→approval+caring (intensity ≥ 2)

Why WRIME?

Machine-translated emotion data often misses cultural nuances in Japanese emotional expression. Adding 24K native Japanese texts from WRIME improved results significantly:

Version	Translation Model	WRIME	F1	Top-1	Missed
v1	opus-mt-en-jap	No	0.309	51%	7/28
v2	opus-tatoeba-en-ja	Yes	0.376	56%	3/28

Limitations

• Rare emotions (grief, nervousness, pride) have very few examples — accuracy is low
• Japanese is linguistically distant from English — translated training data has inherent quality ceiling
• ONNX only — PyTorch checkpoint not included (contact for reproduction details)

Citation

@inproceedings{demszky2020goemotions,
  title={GoEmotions: A Dataset of Fine-Grained Emotions},
  author={Demszky, Dorottya and others},
  booktitle={ACL},
  year={2020}
}

@inproceedings{kajiwara2021wrime,
  title={WRIME: A New Dataset for Emotional Intensity Estimation with Subjective and Objective Annotations},
  author={Kajiwara, Tomoyuki and others},
  booktitle={NAACL},
  year={2021}
}

Author

Built by tojohere for SentiLog — an AI-powered journaling app.