I like researching and building
self-learning systems.

Currently MS in Computer Science
& RL research @ UC San Diego

Harshit Dhankhar

News

Flip through the latest. Click a page or use the arrows to turn it.

  1. Summer at Amazon Web Services wraps up Twelve weeks in Seattle as an SDE intern: an operations agent that reports pipeline health to Slack, and an auto-incident responder that posts root causes within fifteen minutes.
  2. ProNMT accepted at WMT 2026 “Balancing Global Quality and Pronoun-Specific Feedback for Context-Aware Machine Translation” will appear at the Conference on Machine Translation.
  3. EV routing paper published in Elsevier’s SEGAN Deep reinforcement learning for charging-station routes under driver preferences, out in Sustainable Energy, Grids and Networks.
  4. Off to Seattle Started as a Software Development Engineer intern at Amazon Web Services.
  5. HiGym submitted to NeurIPS 2026 A benchmark for reinforcement learning in high-dimensional action spaces, built with Nicklas Hansen in Prof. Xiaolong Wang’s lab.
  6. Joined Prof. Xiaolong Wang’s lab Graduate research assistant at UC San Diego, working on learning to act in high dimensions.
  7. MS in Computer Science begins at UC San Diego
  8. Degree in hand Received the undergraduate degree at IIT Patna’s 12th convocation.
  9. Microsoft Research Academic Summit Attended the summit in Bangalore, India.
  10. Gittins paper presented at WiOpt 2025 “Tabular and Deep Reinforcement Learning for Gittins Index”, presented in Linköping, Sweden.
  11. AI Engineer at AnyFeast Started building agentic pipelines and a recommender for the London food-tech startup.
  12. Admitted to UCSD and NYU Courant Offers for the MS in Computer Science starting Fall 2025.
  13. Quantum routing agents accepted at AAAI ’25 Nested Monte Carlo Search for qubit routing, to appear at the QC+AI workshop.
  14. Inter IIT Tech Meet 13.0 at IIT Bombay Led a problem statement set by Dream11, 11–14 December.
  15. Quality-estimation feedback for MT submitted to ARR Reward training from quality-estimation models, sent to ACL Rolling Review.
  16. Indexable policies via RL submitted to AAAI
  17. RL4SN workshop, Toulouse Presented online learning of the Gittins index.
  18. Summer at PSL Dauphine, Paris Started the research internship at LAMSADE with Prof. Tristan Cazenave.
  19. Deep-RL EV routing submitted Sent to IEEE Transactions on Reliability.
  20. Paper accepted at IEEE BigData Attention-based LSTMs for carbon-emission forecasting, Sorrento, Italy.
  21. TEDxIIT Patna Organized the institute’s TEDx chapter.
  22. Summer at IIIT Hyderabad Research internship at the Computer Systems Group with Dr. Tejas Bodas.
  23. Silver at Inter IIT Tech Meet 11.0
  24. Sophomore coordinator, NJACK ML Joined the institute’s machine-learning society.
  25. Runner-up at Smart India Hackathon 2022 ProfessionPro, built with the team in Visakhapatnam.
  26. The journey begins at IIT Patna First day of the B.S. in Mathematics and Computing.

Experience

Professional

Amazon Web Services

Software Development Engineer Intern

Jun 2026 – Sep 2026 · Seattle, WA

  • Engineered an operations agent hosted on OpenClaw that autonomously generates pipeline-health, dashboard-anomaly and ticket-queue reports and delivers them to Slack, cutting on-call review time from 4+ hours to 30 minutes.
  • Deployed an auto-incident-response system for Sev-2 tickets: an RCA agent runs a multi-hypothesis investigation loop and posts root cause and customer impact within 15 minutes.

AnyFeast

AI Engineer

Feb 2025 – Aug 2025 · London, UK (remote)

  • Shipped a zero-touch ETL pipeline that extracts recipe attributes with agentic AI and enriches instructions through a LangGraph-powered RAG module, reducing manual effort by over 90%.
  • Replaced popularity sorting with a hybrid-filtering recommender, lifting click-through rate by 30%, validated with A/B tests and Azure Monitor.

Research

UC San Diego · Prof. Xiaolong Wang's lab

Graduate Research Assistant, with Nicklas Hansen

Oct 2025 – present · San Diego, CA

  • HiGym: a high-dimensional continuous-control benchmark built on DMControl that isolates action-space scaling from 2 to 512 degrees of freedom by splitting joints while keeping objectives, dynamics, mass and collision geometry fixed, across six embodiment families.
  • Found that model-based agents (TD-MPC2, TD-M(PC)²) degrade far less than model-free PPO as dimensionality grows, with a policy-constraint loss keeping value estimates in check. Under review at NeurIPS 2026.

LAMSADE, Université Paris Dauphine – PSL

AI Research Intern

May 2024 – Oct 2024 · Paris, France

Advisor: Prof. Tristan Cazenave

  • Developed a Nested Monte Carlo Search router for qubit routing in quantum circuits, cutting circuit depth by 10.55% against IBM Qiskit and Google Cirq baselines.
  • Mutex-locked qubit scheduling, combinatorial SWAP-set construction and action-masked playout pruning keep exploration sample-efficient and cut average runtime by 37.17%. First-author paper presented at AAAI 2025 (QC+AI), Philadelphia.

AI-NLP-ML Lab, IIT Patna

Research Affiliate · Bachelor's thesis

Aug 2023 – May 2025 · Patna, India

Advisors: Prof. Asif Ekbal, Dr. Y. M. Tripathi

  • ProNMT: a reward framework that combines quality-estimation models with pronoun generation likelihood for feedback training of English→German translation, improving pronoun translation accuracy and overall COMET (24.06% over the baseline) without human feedback.
  • Accepted at WMT 2026; an earlier version became my bachelor's thesis.

Advanced Computing Lab, IIT Patna

Undergraduate Researcher

Jul 2023 – Sep 2024 · Patna, India

Advisor: Prof. Rajiv Misra

  • Deep RL for EV charging-station routing: a custom Gym environment of Xi'an and a PPO agent that cut cumulative driver cost by 74.19% across baseline strategies, published in Sustainable Energy, Grids and Networks (2026).
  • Attention-LSTM forecasting of CO₂ emissions across SAARC countries (IEEE BigData 2023) and a transformer + DRL scheduler for in-home EV charging.

Computer Systems Group, IIIT Hyderabad

Research Intern

Dec 2022 – Dec 2023 · Hyderabad, India

Advisor: Dr. Tejas Bodas

  • QGI: tabular and deep Q-learning algorithms that learn Gittins indices online for multi-armed bandits with ~50% less memory and ~10% lower runtime than prior methods, applied to pre-emptive job scheduling with unknown service times. Published at WiOpt 2025.

Publications

Hover or tab to a row to see the paper's figures.

Balancing Global Quality and Pronoun-Specific Feedback for Context-Aware Machine Translation accepted

Harshit Dhankhar, Baban Gain, Asif Ekbal, Y. M. Tripathi

WMT 2026 · Conference on Machine Translation

[code][earlier version][thesis]

HiGym: Learning to Act in High-Dimensional Spaces under review

Authors withheld during double-blind review

NeurIPS 2026 submission

[project page][code][OpenReview]

Optimizing Electric Vehicle Charging Routes with Deep Reinforcement Learning Considering Driver Preferences

Shivendu Mishra, Anurag Choubey, Harshit Dhankhar, Sri Vaibhav Devarasetty, Rajiv Misra

Sustainable Energy, Grids and Networks (Elsevier), 2026

[paper][code]

Tabular and Deep Reinforcement Learning for Gittins Index

Harshit Dhankhar, Kshitij Mishra, Tejas Bodas

WiOpt 2025 · 23rd International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks

[paper][code]

Nested Qubit Routing

Harshit Dhankhar, Tristan Cazenave

AAAI 2025 · Workshop on Quantum Computing and Artificial Intelligence (QC+AI)

[paper][code]

Analysis and Forecasting of Carbon Emission in SAARC Countries using Attention-based LSTM

Anil Verma, Harshit Dhankhar, Rajiv Misra, T. N. Singh, Om Prakash Dhakal

IEEE BigData 2023 · IEEE International Conference on Big Data

[paper][code]

Enhancing In-Home EV Charging Optimization with Time Series Transformers and Policy-based DRL: A Multi-Timeframe Approach for Cost Reduction and User Satisfaction under review

Shivendu Mishra, Anurag Choubey, Harshit Dhankhar, Sri Vaibhav Devarasetty, Rajiv Misra

Manuscript, 2024 · under review

[manuscript][code]

Key projects

Hover or tab to a row for screenshots and figures.

Agents & LLM systems

job-hunting-buddy

Phase 1 polls SimplifyJobs every 15 minutes (JobSpy hourly) for new-grad SWE and ML roles in the US and emails each genuinely new posting, with an idempotent ledger, conditional GETs and burst guards. Phase 2 prepares applications in the candidate's own browser through Greenhouse, Ashby, Workable and Lever adapters, and keeps every submission behind an explicit per-draft approval in a private Next.js + FastAPI dashboard with receipt-confirmed daily counts.

Sep 2026 – present · Python, Browser Use, Next.js, FastAPI

[code]

Insurance SOP agent

A claims-support agent that follows a fixed SOP (verify identity → resolve intent → process case → post-process) yet converses naturally. The SOP is a declarative YAML spec run by a generic engine: code owns phases, gates and side effects, the model reasons freely only where the spec allows, with read-only session-scoped tools, prompt-injection guards, emotion-aware de-escalation and human handoff. Runs on Claude, or keyless on a local Laya decision model with templated replies.

Oct 2026 · Python, FastAPI, Claude API, Docker

private repo

Gr8r

AI grading of handwritten answer sheets. Teachers set up classes, exams and rubrics; a Celery worker transcribes each scan with a vision model and LangGraph agents grade it against the rubric with per-question feedback. Criterion-level grading with an evaluator loop and teacher review is in progress. Multi-tenant Next.js 16 + FastAPI + PostgreSQL app with a model router across Claude, OpenAI and self-hosted vLLM/SGLang.

Jan 2026 – present · Next.js, FastAPI, PostgreSQL, Celery, LangGraph

private repo

Schema linking

Fine-tuned Qwen2.5-1.5B-Instruct with a LoRA adapter to map a natural-language question plus database schema to the tables and columns the answering SQL would touch, over 17 Spider-format databases. Training sweeps on RapidFire AI, gold schema links extracted from SQL for augmentation. CSE 234, UC San Diego.

Spring 2026 · PEFT, TRL, RapidFire AI

private repo

RapidFire RAG

End-to-end retrieval-augmented generation over a documentation corpus, swept across 30+ chunking, retrieval and reranking configurations: RST-aware chunking with image captions, FAISS retrieval, cross-encoder reranking and an LLM judge. Final pipeline scores F1@5 0.737, faithfulness 0.98 and correctness 0.87 on the golden set. CSE 234, UC San Diego.

Spring 2026 · FAISS, cross-encoders, LLM-as-judge

private repo

Reinforcement learning & control

HiGym

A benchmark that isolates action-space scaling in continuous control: six DMControl embodiment families split from 2 to 512 degrees of freedom while objectives, dynamics and mass stay fixed. Shows that model-based TD-MPC2 and TD-M(PC)² degrade far less than PPO as dimensionality grows. With Nicklas Hansen, Prof. Xiaolong Wang's lab.

Oct 2025 – present · DMControl, TD-MPC2, PPO

[project page][code]

EV charging routes

A custom Gym environment of Xi'an's road and charger network where a PPO agent picks charging stations under queue, placement and price uncertainty, weighing distance against cost by driver preference. Cuts cumulative driver cost by 74.19% versus baseline routing strategies. Published in Sustainable Energy, Grids and Networks.

2023 – 2026 · PPO, DQN, DDQN, OpenAI Gym

[paper][code]

In-home EV charging

Transformer forecasters (PatchTST, Autoformer, Informer) read electricity-price history over several timeframes and feed DQN, DDPG and PPO schedulers that decide when to charge or discharge, cutting cost while keeping drivers satisfied in a vehicle-to-grid setting. Manuscript with Prof. Rajiv Misra's group at IIT Patna.

2024 · PyTorch, Stable-Baselines3

[manuscript][code]

Products & platforms

Alloc8

Led a team of 8 to build the dorm-room allocation portal for 3,000 students across 1,400 rooms at IIT Patna, as Senior-Year Technical Secretary. MERN stack on a load-balanced VPS with 95%+ uptime under launch traffic, Redis locks for contention-free room picks, spherical k-means roommate pairing from free-text responses. Saves an estimated 200+ hours a year.

2024 · MongoDB, Express, React, Node.js, Redis

[website][demo]

Privy Notion

A collaborative workspace with real-time multi-user editing over Socket.IO: users sign in, share a workspace and edit the same document together. Next.js and Tailwind on the front end, Prisma with PostgreSQL behind it, Zod validation, JWT auth and NextAuth sessions.

2025 · Next.js, Socket.IO, Prisma, PostgreSQL

[code]

QTWAYS storefront

E-commerce site for the QTWAYS skincare brand (Shilpex Pharmysis): shop by category and skin concern, product pages with reviews and stock states, cart and login, newsletter and WhatsApp support. React + Tailwind CSS, deployed on Vercel.

Jan 2026 · React, Tailwind CSS, Vercel

[live site]private repo

ProfessionPro

End-to-end NLP career guidance: parses and tokenizes résumés, classifies them into job domains with KNN from skills, education and experience, and pairs the result with a questionnaire for personalized suggestions in a web portal. Runner-up at Smart India Hackathon 2022, Visakhapatnam.

Aug 2022 · Python, NLP, Flask

[code]

Vision & perception

Vitals from patient-monitor photos

Reads vitals off photographs of patient monitors: YOLOv5 finds the monitor, InceptionV3 classifies its layout, XGBoost labels each reading and EasyOCR reads the digits, robust to viewing angle, monitor shape and background. Confidence 0.90–0.97 across vital types. Built for Inter IIT Tech Meet 11.0 (Cloud Physician problem statement).

Feb 2023 · YOLOv5, InceptionV3, XGBoost, EasyOCR

[code]

About

I'm a second-year MS student in Computer Science at UC San Diego, where I work on reinforcement learning in high-dimensional action spaces with Nicklas Hansen in Prof. Xiaolong Wang's lab. I like taking a problem end to end: diving into theory, building the system, and shipping something people actually use.

In summer 2026 I was an SDE intern at Amazon Web Services, building operations and incident-response agents. Before that I was an AI engineer at AnyFeast in London, built Monte Carlo search routers for quantum circuits at LAMSADE (PSL Dauphine), and learned indexable bandit policies with IIIT Hyderabad. I graduated from IIT Patna in Mathematics & Computing in 2025.

Outside research: Senior-Year Technical Secretary at IIT Patna, overall organizer of TEDxIIT Patna 2023, silver medal at Inter IIT Tech Meet 11.0 and runner-up at Smart India Hackathon 2022.

Education

  • 2025 – 2026UC San Diego · MS Computer Science3.925 GPA
  • 2021 – 2025IIT Patna · BS Mathematics & Computing3.75 GPA

Toolbox: Python, C++, Java, TypeScript · PyTorch, LangGraph, FastAPI, Next.js, PostgreSQL, Redis, Docker, Kubernetes, AWS

Experience

  • 2026Amazon Web ServicesSDE Intern
  • 2025 – presentUC San DiegoResearch Assistant
  • 2025AnyFeastAI Engineer
  • 2024PSL DauphineAI Research Intern
  • 2023 – 2025IIT PatnaResearch Affiliate
  • 2022 – 2023IIIT HyderabadResearch Intern
Résumé ↗