NLPTransformersClassificationDeployed
LLM Preference Classifier
Predicting which of two LLM responses a human judge would prefer
CV log-loss1.07 (baseline)
Built on the LMSYS Chatbot Arena dataset (57K human preference judgments across model_a / model_b / tie). The flagship approach fine-tunes DeBERTa-v3-large with 5-fold CV and A/B swap augmentation; the deployed baseline uses TF-IDF + Logistic Regression, trained and served on Cloud Run in the time it takes to read this sentence. Try it live below.
Try It Live
Paste a prompt and two candidate responses — the model scores which one a human judge would more likely prefer.