Senior Machine Learning Engineer

About the Role

Healthcare's back office still runs on faxes, PDFs and phone calls. AI can take that work off people's plates, but only if it is accurate enough to trust with patient and financial data.

At Plenful, we build agents and models that do that work in production, with people in the loop wherever judgment matters. As a Senior Machine Learning Engineer, you own the pieces that make those systems reliable: the agents, the models behind them, and the data and evaluation loop that improves both every week.

What You'll Do

You Ship Agents That Do Real Work

• Design, build and run production agents that use tools, retrieval and multi-step reasoning to finish healthcare workflows end to end.
• Build the orchestration, tracing and failure recovery that keep agents dependable when inputs are messy.
• Decide when a problem needs an agent, a model or a deterministic rule, and route low-confidence cases to a person.

You Make Models Better, Not Just Call Them

• Fine-tune and post-train LLMs and vision-language models on healthcare tasks, using methods like supervised fine-tuning, preference optimization and distillation.
• Choose between prompting, retrieval and fine-tuning for each problem, and prove the choice with data.
• Balance quality, latency and cost when you serve models.

You Own the Loop That Makes IT All Improve

• Build and version datasets: labeled sets, golden sets and synthetic data drawn from production signal.
• Write evals that catch regressions before customers do: automated graders, LLM-as-judge calibrated against human labels, and regression suites that run in CI.
• Instrument production with tracing, drift detection and failure taxonomies, then feed what you learn back into training data.
• Keep releases reproducible with solid MLOps: Docker, Kubernetes, MLflow or equivalent, and CI/CD.

You Handle Patient Data with Care

• Use the minimum data necessary, and de-identified or synthetic data wherever it does the job.
• Follow HIPAA and Plenful's security and AI usage practices in how you build, test and log.

What We're Looking for

• 5+ years building and shipping software or ML systems, including 2+ years with LLM-based systems in production.
• Strong Python, plus PyTorch and Hugging Face Transformers.
• Hands-on agent experience with a framework like LangGraph, and with RAG and vector search (for example Pinecone).
• You have fine-tuned an LLM or VLM and can explain what improved and how you measured it.
• You have built an evaluation harness and know why accuracy alone misleads and how to calibrate an LLM judge.
• Comfort with large-scale datasets, databases and data pipelines.
• Experience with traditional ML, and the judgment to know when a simpler model beats an LLM.

You Stay Current, and You Can Show IT

• You use the newest models and coding agents every week, form opinions, and test them on real problems.
• You go from idea to working prototype in hours, then bring back what works.
• You have proof: shipped projects, open source, write-ups or your own evals of new releases.

Nice to Have

• Healthcare data experience: EHR, FHIR, claims, prior authorization.
• Document AI or vision model experience.
• Serving and training at scale with tools like vLLM, FSDP or DeepSpeed.
• Work in another regulated, high-stakes domain.

What Your First 90 Days Look Like

By day 30, you have shipped a change to a production agent or model. By day 90, you own the eval suite and improvement loop for one workflow, and its quality numbers are moving.

Benefits & Perks

• Healthcare Coverage - Full medical, dental, and vision insurance for you and participation for your family
• 401(k) with Company Match - Plenful matches 50% of your first 3% contributed
• Equity - Every full-time employee shares in our success
• Unlimited PTO - Take the time you need, when you need it
• Daily Lunch Stipend - $100/week to cover your midday meals
• Wellness Stipend - $100/month to support your health and well-being
• Commuter Benefits - $100/month for SF and NYC-based employees
• Parental Leave - Paid leave to support growing families