Data Engineer

The Role

You'll own the Spark pipelines on Databricks that power Monaco's analytics products, working directly with product teams to turn what they need into reliable, cost-conscious data. This is a hands-on individual contributor role for a mid-level to senior engineer (roughly 2-6 years of experience).

What You'll Do

• Build, deploy, and operate production Spark data pipelines on Databricks.
• Partner with product teams to turn analytics-product needs into pipeline requirements, including data quality and freshness expectations.
• Set up monitoring and observability so pipeline reliability is measured, not assumed.
• Diagnose performance and cost problems and fix their root causes.
• Recommend pragmatic standards for running pipelines as our data and platform scale.

What You'll Bring

• Hands-on experience building, deploying, and maintaining production Spark pipelines.
• Deep understanding of how concepts like clustering, shuffles, different join strategies, and batch versus stream processing create tradeoffs in performance, freshness, and cost.
• Experience with reliability, monitoring, and observability, and the ability to explain how you diagnosed a real pipeline failure or bottleneck.
• Strong SQL.
• Basic programming skills in any language (Python is a plus).
• Comfort working as an individual contributor, directly with product teams, and through ambiguity.

Nice to Have

• Familiarity with the Databricks ecosystem.
• Experience with streaming pipelines and logical replication, such as replicating Postgres into Databricks.
• Experience building RAG pipelines.
• AWS infrastructure experience.
• Platform administration experience: roles, permissions, access controls, and visibility management.

Location

• San Francisco. We're an in-person team - 5 days in the office. At this stage, proximity genuinely accelerates product quality and team cohesion.