Software Engineer II

Overview

Microsoft is a company where passionate innovators come to collaborate, envision what can be and take their careers further. This is a world of more possibilities, more innovation, more openness, and the sky is the limit thinking in a cloud-enabled world.

Microsoft’s Azure Data engineering team is leading the transformation of analytics in the world of data with products like databases, data integration, big data analytics, messaging & real-time analytics, and business intelligence. The products our portfolio include Microsoft Fabric, Azure SQL DB, Azure Cosmos DB, Azure PostgreSQL, Azure Data Factory, Azure Synapse Analytics, Azure Service Bus, Azure Event Grid, and Power BI. Our mission is to build the data platform for the age of AI, powering a new class of data-first applications and driving a data culture.

Within Azure Data, the big data analytics team provides a range of products that enable data engineers and data scientists to extract intelligence from all data – structured, semi-structured, and unstructured. We build the Data Engineering, Data Science, and Data Integration pillars of Microsoft Fabric.

Our team is part of Azure Data, building reliable, large-scale distributed systems for data engineering. We develop Fabric Materialized Lake Views, which automatically refresh analytical data in Microsoft Fabric so customers can query up-to-date results without manual pipelines. We also maintain Azure HDInsight in production, the PaaS platform running open-source Hadoop, Spark, HBase, and Kafka. You will get hands-on experience with both.

As a Software Engineer II, you will design and ship features for our Fabric-based SaaS platform with minimal supervision. You will write high-quality code, debug complex distributed systems, and work closely with Spark and Spark-based extension frameworks. You will own features through the full lifecycle: requirements, design, implementation, deployment, and production support. You will also maintain HDInsight components in production. This role will deepen your expertise in distributed systems, cloud platform engineering, and large-scale data infrastructure.

Successful candidates will have strong experience in one or more of these areas: designing and coding cloud-scale SaaS products; building data lakes or data engineering tools; working with big data technologies such as Spark, Kafka, Hadoop, or HBase; enterprise software development in C#, Scala, Java, or Python; and working with the Azure stack (Storage, Compute, Networking, Fabric, Purview, Synapse, AKS, DevOps, Data Factory, Power BI).

We do not just value differences or different perspectives. We seek them out and invite them in so we can tap into the collective power of everyone in the company. As a result, our customers are better served.

Responsibilities

• Write extensible, maintainable code in C#, Java, Scala, or Python for Fabric Materialized Lake View services and HDInsight components.
• Use AI tools and coding best practices across the development lifecycle.
• Produce code with few defects and clear test coverage.

• Design data refresh, scheduling, and query optimisation features with minimal supervision. Evaluate design options, document trade-offs, and begin owning or co-owning the architecture of Fabric data engineering components.
• Review code from teammates for correctness, test coverage, security risks, and adherence to team standards. Coach engineers through code reviews and provide feedback that raises code quality across the team.
• Debug complex issues in distributed systems running on Azure, Linux, and Windows. Use debugging tools, logs, and telemetry to verify assumptions before issues reach production. Conduct incident retrospectives for Fabric and HDInsight services; identify root causes, implement repairs, and prevent recurrence.
• Run live site operations on a rotational, on-call basis. Diagnose and fix both simple and complex issues across Fabric and HDInsight workloads. Improve troubleshooting guides, tests, and telemetry to make on-call better for the team.
• Integrate logging and instrumentation to gather telemetry on system health, performance, reliability, and security. Classify metrics, build dashboards, and create alerts that surface problems across the Materialized Lake View pipeline.
• Work with product managers, technical leads, and partners across geographies to define customer requirements for Materialized Lake View features. Incorporate customer feedback into designs. Understand and advocate for the security and privacy needs of customers.

Qualifications

Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, Apache Spark, Kafka, Spark SQL, PySpark, C, C++, C#, Java, JavaScript, or Python; OR equivalent experience.

Job Requirements: Other & Additional

Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include, but are not limited to the following specialized security screenings: Microsoft Cloud Background Check:

• This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.

Preferred/Additional Qualifications

• Master's + 4 years technical engineering experience; OR Bachelor's + 4 years technical engineering experience; OR equivalent experience.
• Experience with the Azure stack including Storage, Compute, Networking, Fabric, Purview, Synapse, AKS, DevOps, Data Factory, or Power BI.
• Experience with big data technologies such as Spark, Kafka, Hadoop, or HBase.
• Experience building data lake or data engineering products, tools, or pipelines.
• Familiarity with container-based architectures (Docker, Kubernetes).
• Ability to debug complex distributed systems on Linux and/or Windows platforms.

Additional information

Benefits/perks listed below may vary depending on the nature of your employment with Microsoft and the country where you work.

• #azdat
• #azuredata
• #Fabric #AzureData @BigData #Spark #Kafka

This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.