Senior Data Scientist, Search & NLP

About the Role

As a Senior Data Scientist on our Search & Algorithms team, you'll own NLP analytics for our next-generation semantic search experiences. You'll apply NLP and LLM-based techniques to a proprietary corpus of millions of real consumer search interactions—extracting signals like buyer vs. seller intent and conversational patterns to improve how our models rank and personalize results. You'll also prototype data products that turn those signals into features product and engineering can ship.  This is an AI-native role: LLM evaluation tooling and AI-assisted workflows are core to how you'll work, not afterthoughts.

What You'll Do

Measure and evaluate semantic search models using NLP and LLM-based evaluation frameworks on real user queries—surfacing buyer vs. seller context, exploration vs. transaction signals, and conversational patterns that inform ranking and personalization.

Prototype data products that leverage conversational query data and behavioral signals to drive personalized search experiences, translating NLP insights into actionable features for product and engineering teams.

Design tests for search model changes; define primary and guardrail metrics; communicate results clearly to both technical and non-technical stakeholders.

Instrument and analyze conversational AI data using LLM observability and evaluation tooling—tracking model outputs, scoring response quality, and surfacing patterns across large volumes of AI-generated interactions to inform model and product improvements.

Partner with search product and engineering leadership to define success metrics, shape the analytics roadmap for search, and present findings to senior leadership.

Work in an AI-native way: apply agentic AI workflows, LLM APIs, and AI-assisted development to accelerate your work and help set the standard for how our team uses AI. Actively evaluate and adopt new tooling as the space evolves.

What You'll Bring

5+ years in data science, with 2+ years focused on NLP, text analytics, or search analytics.

Hands-on NLP experience: query classification, intent detection, semantic similarity, entity extraction, or LLM-based evaluation.

Strong Python skills and proficiency with NLP libraries (Hugging Face, spaCy, scikit-learn, or similar).

Experience analyzing conversational and LLM-generated data using observability or evaluation tooling (e.g., Langfuse, Braintrust, or similar platforms for tracing, scoring, and evaluating AI outputs).

Robust SQL skills and comfort working with large-scale clickstream or query log datasets.

Solid experimentation fundamentals: A/B testing, power analysis, and statistical inference.

Ability to build rapid prototypes and translate analytical findings into product-ready recommendations.

Excellent communication skills—able to present NLP and AI concepts clearly to product and executive stakeholders.

Bachelor's or Master's degree in Computer Science, Statistics, Linguistics, Information Science, or a related field.

Preferred Qualifications

Experience with search ranking analytics, query log analysis, or information retrieval measurement.

Hands-on experience with LLM APIs (OpenAI, Claude, Gemini, or similar) and prompt engineering for evaluation, scoring, or analytical tasks.

Experience working with LLM observability platforms (Langfuse, Braintrust, or similar) to instrument, trace, and evaluate AI product performance at scale.

Prior work in consumer-facing search products or two-sided marketplace analytics.

Experience with Snowflake or similar cloud data warehouses.

Ph.D. in Computer Science, Computational Linguistics, NLP, or a related field.

How We Work:

We balance creativity and innovation on a foundation of in-person collaboration. For most roles, our employees work three or more days in our offices, where they have the opportunity to collaborate in-person, adding richness to our culture and knitting us closer together.

How We Reward You:

Realtor.com is committed to investing in the health and wellbeing of our employees and their families. Our benefits programs include, but are not limited to:

Inclusive and Competitive medical, Rx, dental, and vision coverage

Family forming benefits

13 Paid Holidays

Flexible Time Off

8 hours of paid Volunteer Time off

Immediate eligibility into Company 401(k) plan with 3.5% company match

Tuition Reimbursement program for degreed and non-degreed programs

1:1 personalized Financial Planning Sessions

Student Debt Retirement Savings Match program

Free snacks and refreshments in each office location