Claire Kim

Claire Kim

Ph.D. Candidate · Trustworthy LLM / NLP

Hi,

I am Claire (Nayoung) Kim, a Computer Science Ph.D. candidate at the School of Computing and Augmented Intelligence (SCAI) at Arizona State University, graduating in 2026. I build and evaluate trustworthy NLP and LLM systems, with research spanning fairness, bias mitigation, hallucination, robustness, and human-centered AI evaluation.

My work is strongest at the intersection of rigorous research and practical AI systems: defining measurable failure modes, designing mitigation methods, running careful evaluations, and building usable ML/LLM prototypes.

  • FocusTrustworthy LLMs · Fairness · Hallucination mitigation · LLM evaluation
  • Core stackPython · PyTorch · HuggingFace · Transformers · LoRA / PEFT
  • LLM systemsRAG · Multi-agent · LLM-as-a-judge · Vector DBs (FAISS, Chroma)
  • Cloud & MLOpsAWS (SageMaker, S3, Lambda) · GCP · Docker · MLflow / WandB
  • Most recentApplied Scientist Intern @ Amazon (Fall 2025)
  • Looking forApplied / Research Scientist · ML / AI Engineer — Fall 2026 start
Applied Scientist Research Scientist AI/ML Engineer NLP / LLM Systems

Download Resume Research Projects Dev Projects Full CV

Experience

  • Applied Scientist Intern, Amazon — Bellevue, WA · Sep – Dec 2025
  • AI/ML Intern, AMD — Austin, TX · May – Aug 2025
  • Software Development Intern, AMD — Austin, TX · Aug – Dec 2024
  • Research Assistant, DHS-CAOE — Tempe, AZ · Aug 2022 – May 2025
  • Research Assistant, ASU — Tempe, AZ · Aug 2021 – Aug 2022
  • Research Assistant, ASU × Mathpresso — Tempe, AZ · Jan – May 2021
  • Research Assistant, Korea University DMIS Lab — Seoul, Republic of Korea · Mar 2017 – Feb 2019

For full role details (PIs, scope, responsibilities), see the Full CV.

Skills

Research focus
Trustworthy LLMs Fairness & bias mitigation Hallucination evaluation Robust AI Human-centered evaluation
LLM systems
RAG Multi-agent LLM-as-a-judge Synthetic QA generation LlamaIndex Vector DBs (FAISS, Chroma) Elasticsearch Prompt engineering Inference-time scaling
ML & training
PyTorch JAX HuggingFace Transformers SFT RLHF LoRA / PEFT Pandas / NumPy
Cloud & MLOps
AWS (SageMaker, S3, Lambda) GCP Docker MLflow WandB CI/CD
Engineering
Python SQL JavaScript Streamlit Flask Node.js Git Linux Algorithms & data structures System design
Methods
Bayesian inference Uncertainty quantification Statistical analysis Human-subject study design

Research Projects

A selection of research projects spanning trustworthy LLMs, fairness and bias mitigation, hallucination evaluation, and human-centered AI — grouped chronologically, with the most concrete outcome leading each entry.

Ongoing

Reasoning-Level Fairness in LLMs — Under submission

  • Reduced bias scores on the BBQ benchmark while preserving accuracy within 1% across GPT-3.5-Turbo and LLaMA-2-13B baselines.
  • Built a reasoning-aware framework that models reasoning as Q → {R} → A to identify how stereotypes emerge during intermediate steps, and introduced a new metric quantifying unsupported demographic assumptions in reasoning traces — a complementary signal beyond answer-level bias.
  • Designed hybrid mitigation: process supervision with counterfactual augmentation, fairness-aware RL rewards, and inference-time guided decoding with fairness verifiers (built on OpenR).
Tech Python PyTorch OpenR Inference-time scaling Fairness-aware RL Benchmark

Bayesian Learning for Uncertainty-Aware Hallucination Mitigation

  • Designing a Bayesian, uncertainty-aware LLM system to detect and mitigate hallucinations in production, paired with evaluation workflows that demonstrate hallucination detection at scale.
Tech Python PyTorch Bayesian methods LLM evaluation

2025

MASTOPIA: Transparency in LLM-Assisted Intelligence Analysis

  • Showed in a 2³ factorial human-subject study (n = 304) that high-transparency LLM outputs did not improve performance and in marginal conditions decreased it — evidence of overreliance from information overload, motivating adaptive / on-demand transparency design.
  • Built MASTOPIA, a multi-agent RAG system (supervisor → retriever → generator agents) powered by GPT-4 / GPT-3.5 that operationalizes Multisource AI Scorecard Table (MAST) tradecraft standards through prompt engineering.
  • Shipped an interactive Streamlit demo with model conditions, evidence retrieval, and behavioral logging of verification activity.

Try the demo Code

Tech Python GPT-4 / GPT-3.5 RAG Multi-agent LLM Vector DB Prompt engineering Flask Zero-inflated Poisson regression Ridit analysis Prolific / Qualtrics human-subject design

2024

Towards Fair Language Modeling via Parameter-Efficient Methods by Machine Feedback

  • Mitigated social biases in T5, BERT, and LLaMA-2 for toxicity and hate-speech detection by combining reinforcement learning with parameter-efficient tuning (LoRA, P-tuning).
Tech Python PyTorch Hugging Face LoRA RL

MEGAWATT: MAST for Evaluating Generative AI in Worker–Automation Team Tasks

  • Ran human-subject studies on whether off-the-shelf or improved GPT-4 outputs lead to appropriate use — including correct rejections — for intelligence-analysis (I&A) tasks.
  • Applied the MAST trust-assessment framework to evaluate baseline performance and inform adoption decisions for GPT-4 in I&A workflows.
  • Improved response quality with prompt engineering and RAG across summarization, NER, and conversational tasks.
Tech Python GPT-4 API RAG Human-subject study design

Automated Evaluation of Machine-generated Summaries using RLHF

  • Trained an LLM classifier to score document–summary pairs via multi-class classification + RLHF on a handcrafted human-preferences dataset; validated with expert evaluation to confirm the learning method.
Tech Python PyTorch RLHF LLM evaluation

2023

PADTHAI-MM: Designing Trustworthy, Human-Centered AI Systems Using the MAST Methodology — Published in AI Magazine, 2025

  • Designed and validated a principled AI design framework (PADTHAI-MM) for trustworthy decision-support systems; demonstrated effectiveness through a deployed AI system that positively impacted user trust perceptions.
  • Conducted association analysis between user ratings and trust-impacting factors, providing a theoretical basis for the framework.
  • Released open-source implementation artifacts for the READIT and Facewise prototypes supporting the MAST-based design workflow.

Code

Tech Python Decision-support system design User study & evaluation

2022

READIT: Reporting Assistant for Defense and Intelligence Tasks

  • Built a Transformer-based summarization system for intelligence analysts, deployed via a Node.js + Google Cloud web interface for production access to summarized reports.
  • Public implementation available through the PADTHAI-MM research-code repository.

Code

Tech Python Transformers Node.js Google Cloud Platform

Facewise: AI-based Face ID Verification System

  • Built a face ID verification system for security screening, using CNN + ResNet face matching with fine-tuning to optimize verification performance.
  • Public implementation available through the PADTHAI-MM research-code repository.

Code

Tech Python PyTorch CNN ResNet

2021

Interpreting Text Classifiers with Counterfactual Explanation

  • Final project for CSE 472 (Social Media Mining).
  • Implemented counterfactual explanations for a multi-layer neural network used in text classification.

Project Report

Tech Python PyTorch Explainable AI

2017

Biomedical Entity Relation Extraction

  • Extracted biomedical entities and identified relations using the Comparative Toxicogenomics Database (CTD) via distant supervision.
  • Implemented and trained a tree-RNN model (SPINN) combined with a word–character embedding model.
Tech Python TensorFlow Tree-RNN Distant supervision

Dev Projects

These projects translate my trustworthy AI research direction into interactive systems, demos, and product-quality prototypes — connecting model behavior, transparency, evaluation, and user interaction in real systems while also showing the engineering judgment needed to build and deploy them.

MASTOPIA — Transparency-Aware LLM Analysis Demo

MASTOPIA is a Streamlit demo for LLM-assisted intelligence analysis. It operationalizes MAST-style transparency ideas through an interactive workflow with model conditions, evidence retrieval, model information, session management, and activity logging.

Try the demo View code

System highlights

  • Multi-agent architecture with supervisor, retriever, and general AI agents.
  • Streamlit chat interface with model information controls and sidebar state.
  • Session management and activity logging for research evaluation.
  • RAG-oriented stack using LangChain / LangGraph, OpenAI APIs, FAISS, and Google Cloud / Firestore components.
Tech Python Streamlit LangChain / LangGraph OpenAI API FAISS Google Cloud / Firestore RAG Activity logging

Sprout — Product-Quality Engineering Build

Sprout is not a research project, but it shows my interest in building usable real-world systems with care for interaction quality, reliability, and user-facing product details.

Try Sprout View code

  • Full-stack bilingual social app with authentication, profiles, public sharing, responsive UI, dark mode, and Supabase-backed persistence.
  • Built with a high-quality agentic coding workflow while retaining ownership of product direction, architecture, testing, and final code quality.
  • Included here as engineering evidence alongside my primary research work.