Associate Director, Software Engineering Specialist

Tech leadership & delivery ownership: Lead end-to-end engineering delivery for big data ETL and scoring pipelines, from design through build, testing, release, and operational support. Architect scalable solutions on GCP: Design and evolve cloud-native architectures using GCP services (e.g., Dataproc/Spark, GCS, CloudSQL, GKE, VMs) aligned to non-functional requirements. Build high-performance data pipelines: Develop and optimise Spark/Scala jobs for large-scale batch processing, data quality controls, enrichment, and feature/scoring outputs. Integrate with Quantexa: Drive effective integration patterns and engineering standards around the Quantexa platform, ensuring robust data ingestion, entity resolution/analytics workflows, and downstream consumption. Search & investigative enablement: Design and implement efficient indexing/retrieval patterns using Elastic Search to support investigative and analytical use cases. Raise engineering standards: Establish best practices for code quality, CI/CD, testing strategies, observability, reliability, and cost/performance optimisation; mentor engineers and conduct technical reviews. Strong hands-on big data engineering background, with proven experience building production-grade ETL pipelines and data platforms at scale. Advanced Spark + Scala experience (performance tuning, partitioning strategies, handling skew, optimising job execution, production support). Strong GCP experience designing and running data workloads (e.g., Dataproc, GCS, CloudSQL, GKE/containers, IAM/security basics, networking considerations). Technical leadership experience (solution design ownership, estimation/planning, guiding teams through delivery, mentoring, setting standards and direction). Solid engineering fundamentals: distributed systems, data modelling, batch processing patterns, fault tolerance, testing and CI/CD discipline. Stakeholder and communication strength: able to translate complex technical topics to non-technical partners, navigate a complex organisation, and drive outcomes across teams.