Senior Software Engineer - Data Platform

Role and Responsibilities

• Data Pipeline & Ingestion: Design, build, and operate reliable, scalable, data-intensive pipelines carrying high-volume device and network telemetry - including ingestion and orchestration across network monitoring systems, device and satellite terminal telemetry, and third-party vendor APIs.
• Real-Time Event Processing: Build and evolve the stateful, real-time event processing that turns raw telemetry into meaningful operational events, applying detection logic at scale.
• Workflow & Rule Engine: Extend and enhance the detection and workflow engine to support client-specific flow paths and parameters through managed configuration that users can view and maintain from the product interface, along with the authoring, validation, and versioning that keeps those changes safe.
• Event & Incident Flow Visibility: Partner with product and front-end engineers to expose event and incident monitoring flow in the product experience - making it clear what was detected, what it triggered, and where it stands - for network operations engineers today and for customers as the capability expands. Own the data models, APIs, and services behind those views.
• Observability & Data Lineage: Instrument the platform so engineers can trace an event end to end and understand why the system behaved as it did - including per-job telemetry, retry semantics, and lineage back to the source.
• Data Lakehouse & Storage: Contribute to the design and evolution of our data lakehouse - table and partition design, schema evolution, storage lifecycle, and query performance for downstream analytics and reporting.
• Technology Stack Evolution: Contribute to the ongoing upgrade of our technology stack and architecture to meet the demands of an evolving product roadmap and customer needs.
• Cloud & DevOps: Build and operate services on our containerized infrastructure, deploying through CI/CD and following the team's deployment and infrastructure patterns.
• Design & Code Quality: Work with engineering leadership and business stakeholders to turn requirements into technical design and high-quality code. Write design docs and diagrams before significant changes, participate in code review, and share a light on-call rotation for the pipeline you own.
• AI-Augmented Engineering: Use AI-assisted engineering tools as a primary part of your development workflow, with the fundamentals and judgment to direct them effectively and validate what they produce.

Desired Qualifications

• A degree in Computer Science, Engineering, a similar field of study, or equivalent work experience.
• 5+ years of professional software engineering, with substantial time building production data pipelines and workflow or rule engines.
• Strong Java or Python - ideally both, as our stream processing is Java and our orchestration is Python.
• Hands-on production stream processing experience (Apache Flink, Kafka Streams, Spark Structured Streaming, or Beam), with the ability to reason about keyed state, watermarks, checkpointing, and event-time versus processing-time semantics.
• Practical Apache Kafka knowledge beyond consumer usage - partitioning, compaction, consumer group behavior, and schema evolution (Avro / Schema Registry).
• Experience with a workflow or ETL orchestration tool - Dagster preferred, or a comparable orchestrator such as Airflow or Prefect.
• SQL fluency and production experience with a columnar or lakehouse query engine (Trino, Presto, Spark SQL, Snowflake, BigQuery, ClickHouse, or StarRocks).
• Comfort operating in Kubernetes - reading logs, adjusting resources, and troubleshooting workloads.
• Demonstrated ability to debug distributed systems under pressure: serialization failures, GC pressure, backpressure, and stuck jobs.
• Familiarity with unit testing, automated testing, and CI/CD practices.
• Demonstrated proficiency with AI-assisted development tools (Claude Code, Copilot, Cursor, or similar) as a core part of your engineering workflow.
• Strong communication skills; able to articulate technical decisions to engineering peers, operations stakeholders, and non-technical audiences.

Pay and benefits

*The salary range reflected is a good faith estimate of base pay for the primary location of the position. Our compensation reflects the cost of labor across several U.S. geographic markets, and we pay differently based on those defined markets. The U.S. pay for this position ranges from $145,000.00 to $165,000.00 annually. Pay varies by work location and may also depend on job -related knowledge, skills, experience and abilities of the successful candidate. Your recruiter can share more about the specific salary range for the job location during the hiring process.

Keywords: #NYC, NYC, #NewYorkCity, New York City, NewYorkCity, Senior Software Engineer, #SeniorSoftwareEngineer, Senior Data Engineer, #SeniorDataEngineer, Data Engineer, #DataEngineer, #Manhatten, #DataPlatform, Data Platform, #DataPipeline, Data Pipeline, #DataEngineering, Data Engineering, #ApacheFlink, Apache Flink, Flink, #Kafka, Kafka, #Dagster, Dagster, #Airflow, Airflow, #Iceberg, Apache Iceberg, #Trino, Trino, #StarRocks, StarRocks, #Lakehouse, Lakehouse, #StreamProcessing, Stream Processing, #EventProcessing, Event Processing, #Java, Java, #Python, Python, #Kubernetes, Kubernetes, #ClaudeCode, Claude Code, ClaudeCode, #Engineering, #Telecom