Staff Software Engineer, Applied AI

About the Role

• Backend for LLMs - Architect and implement scalable, low-latency APIs and services that wrap, orchestrate, and optimize LLMs for healthcare use cases.
• Data & Retrieval Pipelines - Build ingestion, preprocessing, and retrieval-augmented generation (RAG) pipelines to ground LLMs in clinical and revenue-cycle data.
• LLMOps & Observability - Design systems for model monitoring, evaluation, cost tracking, and guardrails, ensuring reliability and responsible use.
• Performance & Optimization - Engineer solutions for caching, batching, load balancing, and scaling LLM workloads across cloud and containerized environments.
• Security & Compliance - Implement HIPAA-ready infrastructure, data governance, and auditability for LLM-powered applications.
• Cross-Functional Collaboration - Partner with product, ML engineers, and healthcare experts to translate business workflows into robust backend systems.
• Technical Leadership - Drive end-to-end delivery of LLM backend projects, establish engineering best practices, and mentor peers in LLM system design.

About You

• 5+ years of backend or full-stack software engineering experience, with 3+ years working on ML/LLM-enabled applications.
• Strong coding skills in Python (and ideally one statically typed language such as Go, Java, or TypeScript).
• Experience with LLM integration frameworks (Hugging Face, LangChain, LlamaIndex, OpenAI APIs, Anthropic, etc.).
• Deep knowledge of distributed systems, service-oriented architecture, and building APIs at scale.
• Cloud-native expertise: AWS/GCP/Azure, Kubernetes, Docker, Terraform, etc.
• Familiarity with MLOps/LLMOps practices: CI/CD for models, evaluation harnesses, monitoring, and reproducibility.
• Excellent system design skills and the ability to align technical architecture with product goals.

Preferred Qualifications

• Experience applying LLMs in healthcare or other regulated industries (FHIR, HL7, HIPAA).
• Hands-on experience with RAG pipelines, vector databases, and structured-output orchestration.
• Background in enterprise SaaS or mission-critical platforms where uptime, latency, and scale matter.
• Knowledge of responsible AI, safety, and privacy-preserving ML techniques.

What’s in IT for You?

• The opportunity to apply cutting-edge AI to one of the world’s most important industries.
• A leadership role with ownership over core ML/LLM systems and influence on technical direction.
• Competitive salary, equity options, and benefits, including health, dental, and vision coverage.

Compensation

The base salary for this role ranges from $200,000 to $275,000, depending on experience and qualifications. We also offer equity and benefits as part of our total compensation package.