Book a Call
Cuscal modernises its decentralised data environment by centralising systems on Databricks
Cuscal modernised its decentralised data environment by centralising systems on Databricks.
The Opportunity
Cuscal needed to centralise its data platform systems and automate the deployment of Data Engineering pipelines in a standardised, robust path to production to increase quality and speed.
The Delivery
Agile Insights designed and built a unified Databricks platform with a Medallion Architecture, Git Flow automation, and Infrastructureas- Code deployment replacing fragmented multi-platform tooling.
The Outcome
The Data Engineering team now runs a unified, touchless pipeline deployment process across all domains increasing quality, speed, and auditability while reducing manual intervention.
Book a free consultation
Download the full case study
About Cuscal
Cuscal Limited is one of Australia’s leading providers of payment and banking infrastructure, supporting a broad network of financial institutions, credit unions, and fintech partners. As a regulated entity operating under APRA oversight, Cuscal processes billions of dollars in transactions across card, BECS, NPP, and other payment schemes each year. Operating at this scale – and with obligations to clients, regulators, and payment schemes including Mastercard, Visa, and Eitpos, Cuscal requires timely, accurate, and consistent reporting across every aspect of its business.
The Challenge

Centralising data without losing speed or control Cuscal’s data platform had evolved into a decentralised, multi-platform environment. Data assets were spread across disparate tools, creating inconsistency, duplication, and significant operational risk. Conflicting internal tools made it difficult to integrate, deploy, and consume analytics in a coherent way. The absence of a unified platform meant there was no consistent audit trail a critical gap in a
regulated financial environment. Pipeline deployment was slow, manual, and error-prone.

Each new workflow required bespoke effort, limiting the team’s ability to scale. Cuscal needed a fundamentally different approach:

  • Shift from a decentralised multi-platform data asset to a standardised, centralised hub through Databricks
  • Resolve conflicting internal tools used to integrate, deploy and consume analytics
  • Establish a unified platform with consistency, quality and audit trail in a fast-moving, regulated Industry
  • Automate the process in the cloud with minimal human intervention
  • Apply Software Development best practices for robust pipeline development, test and release workflows
Client Testimonial
The foundation built with Agile Insights gives Cuscal the confidence to expand its data capability rapidly.
Download the full case study

Our Solution

A unified, automated data engineering platform

Agile Insights worked with Cuscal to design and deliver a centralised, enterprise-grade data platform on Databricks replacing fragmented systems with a single, governed operating model for pipeline development, deployment, and consumption. The solution was structured across three phases:

1. Design

  • Incrementally identified repeatable patterns in pipeline design to package delivery in a new ETL framework
  • Applied DevOps principles using infrastructure-as-code for better source control of pipelines and delivery
  • Designed a Git Flow to check quality and robustness of new pipelines, automatically deployed into all environments via Pull Request (PR) approval
  • Established a clear Medallion Architecture pattern to enrich and conform data for consumption

2. Build

  • Implemented Databricks Asset Bundles declaratively defining all Databricks components in source-controlled configuration files
  • Used Terraform to manage clusters and SQL endpoint deployment for consistency across all environments
  • Built a monitoring system of logs to oversee production pipelines
  • Created a reporting framework to deliver canned reports to internal and external users via flat file dumps and dashboards

3. Run

  • Automated, touchless pipeline that builds, tests, and deploys code from one environment to another via PR approval process
  • Collaborated with stakeholders in an Agile process to resolve bug fixes before releasing to production
  • Trained users on the tooling and delivered comprehensive documentation
Reporting
Before: Fragmented AI deployments created inconsistent data and unreliable reporting. After: One unified Databricks platform delivers consistent, governed reporting across all domains.
Decision Making
Before: Slow pipeline deployment and reliance on manual processes limited planning and innovation. After: Automated pipelines and a shared data foundation enable faster, more confident decisions.
Risk & Compliance
Before: No unified audit trail in a highly regulated environment created significant operational risk. After: Centralised, auditable data infrastructure enables effective risk management across all domains.
Delivery
Before: Each pipeline required bespoke setup, creating bottlenecks and quality inconsistencies. After: A unified environment replaces fragmented systems, allowing teams to deliver data solutions at scale.
Why It Worked

Agile Insights and Cuscal delivered this transformation through a structured, collaborative program focused on:

• Embedding DevOps and Software Development best practices directly into the pipeline architecture from day one
• Designing a scalable operating model using Infrastructure-as-Code and Databricks Asset Bundles for full source control
• Iterative engineering to identify and package repeatable pipeline patterns into a reusable ETL framework
• Ensuring quality and consistency through an automated Git Flow PR approval process across all environments
• Building internal team capability through training, documentation and hands-on collaboration throughout delivery

The Impact

Cuscal’s data platform transformation has moved the organisation from a fragmented, manually intensive environment to a governed, enterprise-wide capability built for speed and scale. The Data Engineering team now operates a unified, touchless pipeline deployment process across all domains with quality and auditability built in by design. The foundation built with Agile Insights gives Cuscal the confidence to expand its data capability rapidly, respond to regulatory requirements proactively, and deliver trusted insights to the business at pace, and without compromising control.

Download the full case study here.

Download

Get in touch