Hiring: AI Engineer | Agentic AI, AWS Bedrock AgentCore & Langfuse | 15+ Years
- Location
- Los Angeles, CA
- Listed
- 10 days ago
About the role
- Job Title: AI Engineer / Generative AI Engineer
- Location: Los Angeles, CA (Hybrid)
- Duration: 3+ Months (Extension Possible)
- Experience: 15+ Years
Job Summary
We are seeking a hands-on AI Engineer with expertise in Python, Agentic AI, AWS Bedrock AgentCore, and LLM evaluation to build, deploy, monitor, and continuously improve production-grade AI agents.
Key Responsibilities
- Develop AI agents using Python and Claude Agent SDK.
- Deploy agents using AWS Bedrock AgentCore Runtime.
- Design golden datasets and implement LLM-as-a-Judge evaluations.
- Integrate automated evaluations into CI/CD pipelines as quality gates.
- Implement Langfuse/OpenTelemetry for AI observability and tracing.
- Evaluate agent accuracy, task completion, tool execution, and performance.
- Support RAG pipelines and AWS data processing workflows.
Required Skills
- Python and Agentic AI development.
- AWS Bedrock AgentCore Evaluations / Bedrock evaluation.
- Golden-dataset design and curation.
- LLM-as-a-Judge design and calibration against human labels.
- Automated evaluation and CI gate integration.
- Langfuse and/or OpenTelemetry.
Good to Have
- RAGAS or equivalent evaluation frameworks.
- Trajectory evaluation and agentic red-teaming.
- Bedrock Guardrails, Strands Agents SDK, and GitHub Actions.
Qualifications
- 6-8+ years of software engineering, cloud engineering, or AI/ML experience.
- Hands-on experience with production AI agents and continuous evaluation.
- Strong problem-solving and troubleshooting skills.
Additional Information
All your information will be kept confidential according to EEO guidelines.
From TMS LLC's careers page, first listed on Rolev.ai on October 31, 2026. Rolev.ai is not the employer.
How Rolev.ai helps with this application
Rolev.ai is a job search and application tool for US job seekers. For this Hiring: AI Engineer | Agentic AI, AWS Bedrock AgentCore & Langfuse | 15+ Years role at TMS LLC, it shows how your resume fits, tailors your resume to the posting, and prepares the application for you to review.
See how you fit
Openings from employer career sites are ranked against your whole resume—and searchable inside Claude, ChatGPT, and Cursor via Rolev.ai MCP.
Tailor without inventing
Every change to your resume is checked against your original. New experience goes in only with your yes—zero fabricated claims.
Draft the full kit
Tailored and Full Match resumes, role-specific cover letters, and answers to employer screening questions are prepared in seconds.
1-Click Extension Autofill
The Chrome extension fills the employer's Greenhouse, Workday, Lever, or Ashby form in 1 click. You review it and press Submit.
Common questions
Where is the Hiring: AI Engineer | Agentic AI, AWS Bedrock AgentCore & Langfuse | 15+ Years job at TMS LLC located?
This Hiring: AI Engineer | Agentic AI, AWS Bedrock AgentCore & Langfuse | 15+ Years position at TMS LLC is located in Los Angeles, CA. Always check TMS LLC's application page for specific state eligibility or relocation policies.
How do I apply for the Hiring: AI Engineer | Agentic AI, AWS Bedrock AgentCore & Langfuse | 15+ Years position at TMS LLC?
This listing links directly to TMS LLC's SmartRecruiters application page. You can apply directly on TMS LLC's site, or use Rolev.ai to check your resume fit, generate a zero-hallucination tailored resume and cover letter for this posting, and autofill the SmartRecruiters application form in 1 click with the Rolev.ai Chrome extension. You always review the form and press Submit yourself.