Data Engineer II

What You'll Do

• Design, build, and improve scalable data systems with a focus on performance, and reliability.
• Use Databricks and Spark to prepare, transform, and serve high-quality data to customers and downstream systems.
• Build, optimize, and maintain real-time and batch data pipelines using AWS technologies such as Lambda, S3, DynamoDB, ECS, EKS, and Terraform, as well as develop and maintain pipelines within the Databricks ecosystem.
• Identify and implement improvements to the cost and performance of data ingestion, transformation, and storage workflows.
• Be available for on-call support to troubleshoot and remediate data pipeline failures outside of standard business hours.
• Partner with Product Management, SpyCloud Labs, and Engineering teams to understand requirements and develop effective technical solutions.
• Evaluate and implement modern tooling for data processing, ML/AI data workflows, and observability.
• Contribute to architectural and technical design decisions and help implement scalable solutions aligned with engineering standards.
• Promote engineering best practices through collaboration, code reviews, documentation, and knowledge sharing.
• Identify and resolve complex data quality, performance, security, and reliability issues across production systems.
• Independently own complex data engineering initiatives from design and development through deployment and production support.
• Contribute to the continued evolution of SpyCloud's data platform as product, customer, and AI requirements grow.

Requirements

• 5+ years of professional experience in data engineering, software engineering, or a closely related discipline.
• Experience building and operating Lakehouse, Databricks, or large-scale distributed data architectures in AWS.
• Strong understanding of distributed systems as they relate to data storage, processing, and computing.
• Experience building and managing Elasticsearch/OpenSearch data stores.
• Hands-on experience designing, building, and maintaining ETL and data-processing pipelines.
• Experience preparing and managing data for machine learning and LLM workflows.
• Strong development skills in Python, particularly when building data-intensive systems.
• Practical experience with DynamoDB and other AWS-native data services.
• Experience with infrastructure-as-code tools such as Terraform.
• Experience troubleshooting complex data pipeline, performance, and data quality issues in production environments.
• Ability to independently own complex technical projects from design and development through deployment and production support.
• Ability to evaluate technical trade-offs and contribute effectively to architectural and design decisions.
• Strong communication and collaboration skills across technical and non-technical stakeholders.

Nice to Have

• Familiarity with AI/ML tools and technologies such as SageMaker, LangChain, or vector databases.
• Understanding of data governance, privacy, and compliance considerations within AI and ML environments.
• Experience with observability and monitoring tools for large-scale data platforms.
• Experience with Go or C programming languages.
• Experience working with cybersecurity or security-related datasets and technologies.
• Comfort working in fast-paced environments with evolving product and technical requirements.
• Bachelor's or Master's degree in Computer Science, Engineering, or a related STEM field.

Pay and benefits

The salary range reflects the expected base compensation for a fully qualified candidate at this level based on experience, qualifications, and market data at the time of posting.

Benefits

U.S.-Based Benefits + Perks (for Full Time Employees):

At SpyCloud, we are committed to working alongside individuals who are equally passionate about preventing cybercrime, regardless of their department or role. Guided by our core values in all business decisions, we prioritize unity in our mission and ensure all SpyCloud employees have the support and benefits they need to stay focused on our goals. In addition to our engaging workspace in South Austin, flexible and remote-friendly work options, and competitive salary package, we offer our employees a comprehensive benefits package that includes:

• 401(k) with Employer Contribution
• Health, Vision, and Dental Insurance
• Health Savings Account (HSA) available with Employer Contribution
• Employer Paid Life, Short-term, and Long-term Disability Insurance
• Generous PTO Plan and 16 paid holidays per year

Benefits

U.K.-Based Benefits + Perks (for Full Time Employees):

• Retirement Savings Plan with Employer Contribution
• Employer Provided Private Health Insurance and Healthcare Cashplan
• Employer Paid Life Insurance, Income Replacement, and Critical Illness Protection
• Employee Assistance Program
• 25 days annual accrued holiday plus 8 paid bank holidays and a paid company shutdown during winter holidays

SpyCloud's Recruitment Policy

We will never ask an applicant for sensitive or personal financial information during the recruitment process. We advise all applicants seeking employment with SpyCloud to review available information on recruitment fraud. Anyone who suspects that they have been contacted by someone falsely representing SpyCloud should email.

Compensation Transparency Policy

At SpyCloud, we believe in transparency and fairness in compensation. We strive to ensure that all employees are fairly compensated for their contributions, and we openly discuss our compensation philosophy and structure. We are committed to providing competitive salaries and benefits packages to attract and retain top talent, and we encourage open dialogue and feedback regarding compensation matters.

Learn more and apply: SpyCloud Careers

SpyCloud is not sponsoring visas at this time.

For applicants residing in California, please click here to read SpyCloud's CCPA Notice.

For applicants residing in the UK, please click here to read SpyCloud's Employee Privacy Notice.