01
- Rescue
- A job that will not finish, a cluster that keeps dying, or an inherited notebook nobody can read. We stabilise the pipeline and write down what changed.
02
- Performance tuning
- Shuffle sizing, broadcast thresholds, AQE configuration and executor memory tuned against real stage traces from your production runs.
03
- Skew and join rewrites
- Hot keys salted, many to many joins restructured, and skew handling verified with before and after measurements rather than claims.
04
- PySpark to Scala and back
- Hot paths rewritten in Scala or Spark SQL where Python UDF overhead is the real cost, with a plan for which code stays Pythonic for productivity.
05
- Delta, Iceberg and lakehouse layout
- Table partition layout, Z ordering, compaction and vacuum policies tuned so read and write workloads do not fight each other.
06
- Managed Tech
- An ongoing engagement covering job observability, cost reviews, cluster upgrades and a Spark engineer on call when an overnight pipeline misses its SLA.