Applied Reinforcement Learning Engineer

Role

Location: Palo Alto, CA or Seattle, WA (Hybrid/Remote)

Role Overview

As an Applied RL Engineer, you will design and build RL environments that simulate complex enterprise workflows and train intelligent agents within them. You'll work at the intersection of RL research and production systems, translating customer requirements into bespoke simulation environments and post-training pipelines that deliver measurable improvements to AI agent performance.

This role requires deep expertise in both classical RL methodologies and modern LLM-based agent architectures. You'll shape our product direction and help make RL accessible to enterprise customers who need safe, compliant ways to improve their AI systems.

Core RL Competencies

Foundational RL

• MDPs & value methods: State/action spaces, Q-learning, DQN, Double DQN, Dueling DQN
• Policy gradient methods: REINFORCE, Actor-Critic, A2C/A3C, variance reduction
• Advanced optimization: PPO, TRPO, SAC, trust regions, entropy regularization
• TD learning: TD(0), TD(λ), eligibility traces, bootstrapping methods

LLM Alignment & Post-Training

• RLHF pipelines: Reward model training, preference learning, human feedback integration
• Direct optimization: DPO, IPO, KTO, offline preference optimization
• Group-based methods: GRPO, RLOO, sample-efficient policy improvement
• Reward modeling: Bradley-Terry models, reward hacking mitigation, KL constraints

Environment Design

• Gymnasium/OpenAI Gym: Custom environments, observation/action spaces, wrapper patterns
• Reward engineering: Sparse vs. dense rewards, potential-based shaping, intrinsic motivation
• Verifier design: Programmatic reward functions, outcome verification, ground-truth evaluation
• Simulation: Sim-to-real transfer, domain randomization, multi-agent dynamics

Advanced Techniques

• Offline RL: CQL, BCQ, IQL for learning from fixed datasets without environment interaction
• Model-based RL: World models, Dreamer, MuZero, learned dynamics
• Hierarchical RL: Options framework, goal-conditioned policies, temporal abstraction
• Imitation & exploration: Behavioral cloning, GAIL, curiosity-driven exploration, UCB

Key Responsibilities

• Design and build custom RL environments (digital twins) simulating enterprise workflows: document processing, compliance, onboarding, support automation
• Post-train LLM-based agents on domain-specific tasks using PPO, GRPO, DPO, and RLHF
• Build end-to-end pipelines converting human-labeled traces into RL training data
• Architect multi-step reasoning agents with tool-calling and closed learning loops
• Design reward functions, verifiers, and validation frameworks for pre-deployment testing
• Translate cutting-edge RL research into production systems; contribute to publications

Required Qualifications

• Deep RL expertise: 3+ years hands-on experience with environment design, reward engineering, policy optimization
• LLM post-training: Experience fine-tuning LLMs using RLHF, DPO, PPO, or similar
• Production skills: Software engineering beyond research with scalable pipelines and training infrastructure
• Agentic AI: Experience with LLM-based agents, tool use, multi-step reasoning
• Technical stack: Strong Python; Gymnasium, RLlib, Stable Baselines; PyTorch/JAX/TensorFlow
• Education: MS/PhD in CS, ML, or related field (or equivalent experience)

Preferred Qualifications

• Publications at NeurIPS, ICML, ICLR, ACL, or similar venues
• Enterprise workflow experience in healthcare, finance, logistics, or compliance
• Open-source contributions to CleanRL, TRL, veRL, or agent frameworks
• Experience with world models, synthetic data generation, and simulation
• Distributed training and large-scale RL experimentation

Salary

$150K - $300K Annually