A Tier 1 investment bank requires a specialist Spark Performance Engineer on an initial 6-month contract to diagnose and resolve critical latency issues in their real-time risk calculation pipelines.

You will profile and tune Spark jobs running on Databricks, implement adaptive query execution strategies, and document performance playbooks for the permanent team.

• Expert-level Apache Spark performance tuning (AQE, partitioning, caching)
• Databricks Runtime internals and Photon engine optimisation
• Experience with streaming workloads: Structured Streaming, Delta Live Tables
• Financial services or high-frequency data experience preferred
• Available to start within 2 weeks