Gen AI Contractor

POSITION SUMMARY

EarnIn's Gen AI Team is on a mission to deliver the best agentic experience for our customers — accurate, fast, and set up for success. At the heart of that mission is Evaluation-Driven Development (EDD): a closed loop of human judgment and calibrated LLM scorers that lets us ship new chatbot capabilities with confidence, measure performance in real time, and tie every change to business outcomes.

To help us execute on this vision, we are hiring a Gen AI Contractor for the Analytics Engineering Team. This position is based in Bengaluru, India, requires two days a week in-office as part of our expanding site location, and is a 6-month contract engagement starting in January 2027.

WHAT YOU'LL DO

Help build and run evaluations for our LLM-powered chatbot systems, contributing to our goal of making evals a standard practice across all teams at EarnIn

Support the design and calibration of LLM-as-judge scorers, working closely with subject matter experts (SMEs) to align automated scoring with human ground truth

Contribute to conversation simulation and replay infrastructure — a core building block of our automated research loop

Assist with real-time observability tooling to detect chatbot performance degradation before customers feel it

Help analyze production conversations and surface insights that drive chatbot improvements

Work alongside ML engineers and analysts to iterate on prompt mutations, model changes, and evaluation pipelines

WHAT WE'RE LOOKING FOR

Currently pursuing a Bachelor's or Master's degree (B.E., B.Tech, M.Tech, or equivalent) in Computer Science, Software Engineering, or a related field from an Indian university

Genuine curiosity about large language models, AI evaluation, and building reliable Gen AI systems

Familiarity with Python and comfort working with APIs, data pipelines, or scripting

Basic understanding of NLP concepts or hands-on experience with LLMs (even through personal projects or coursework) is a plus

Strong understanding of probability theory and stochastic systems is a significant plus

Strong analytical thinking — you ask "how do we know this works?" before shipping

Clear written and verbal communication; you can translate findings for both technical and non-technical audiences

Self-directed and thorough — you take ownership, follow through, and don't leave things half-done

#LI-Hybrid