Success Stories

The Power of Frontline Data Quality Controls

Improving data quality was key to ensuring our financial services client retained data integrity and accuracy while also reducing time and money spent on after-hours issues.

A person types on a laptop. Above the laptop hovers icons of various checklists, representing data quality controls.

Home > Success Stories

Project Overview

Our client’s upstream source systems were causing failures in overnight data processing and required developers to be called after-hours to resolve issues for ETL (extract, transform, load) to successfully complete for the next business day.

As their Execution as a Service partner, it was critical for RevGen to design a solution that was simple enough to work with their current architecture, as the client was in the process of transitioning to a new platform and building an entirely new comprehensive solution wasn’t practical.

Client Challenge

Over a year and a half, our client had significant numbers of ETL outages caused by front-end entry data issues resulting in hours of business downtime and skyrocketing support time and costs.

Too many outages

Over the past 18 months, we documented 22 priority 1 ETL outages where business processes were down for approximately 6.25 hours.

Poor front-end data quality

Of these priority 1 outages, 91% of them were related to front-end data issues.

Significant resources used after hours

Approximately 150 hours had been dedicated to after–hours support needs to resolve these continual issues.

Approach

Given RevGen’s specific role as their execution partners, we had to find a resolution to the challenge without creating an entirely new solution design or purchasing a new system. So, we examined the current processes thoroughly and identified where we could insert a new quality control process into the existing ETL after the data is staged but before the ETL begins.  

Solution

Creating a new process required RevGen’s holistic knowledge of the current systems and processes. Our new schema could not interrupt current business flows, as this data was business critical.

New schema created

Our new schema organized the new process and identified necessary next steps and allowed us to align stakeholders on the proposed changes, heading off any technical or business issues before they could become bigger problems:

Creation of a data quarantine

New quarantining tables allowed us to hold any quarantined data separate from the scrubbed data, allowing the new procedures to run without interrupting other processes.

Scrubbing of data

We created new rules to identify bad values prior to the larger ETL process running. These rules scrubbed the data, moving any rows with bad values to the newly established quarantine for further analysis.

Alerts put in place

We added an extra notification to the ETL log using the existing alert process, which highlights any data quality issues found by the scrub.

Results

“I love the work life balance progress this effort boosts for our technical team members.” – Client Corporate Loan Accounting Director

“This is really great work and will save the company time and money.” – Client Enterprise Data Steward

Hours saved on support

Approximately 74 hours were used in 2023 to resolve these ETL outage issues, which are no longer needed thanks to RevGen’s implementation of the new schema.

Work-life balance restored

As the new process has nearly eliminated the need for after-hours support for these issues, both for the client-side team and RevGen’s team, employees are better able to relax and focus on their home lives during what should be off hours.

Saving money on expensive support

With the drastic cutdown on the after-hours data issues, the client is saving an estimated $7,000 or more on the high cost of expensive after-hours support.

Success Stories

Data Transformation to Enable Advanced Analytics and AI

Analytics & Insights

Our client needed a modern, transformative data architecture to reach the next level of analytics, including AI.

Three wooden blocks spelling out MQL. The "M" block tips over to reveal an "S"

Automating for Efficiency: Turning Marketing Leads into Sales Contacts

Technology ServicesAnalytics & Insights

The transfer of marketing leads to sales teams is critical for customer acquisition, however our client’s process was time consuming and heavily manual. RevGen built a proof-of-concept solution that automated the entire handoff.