Improving data quality was key to ensuring our financial services client retained data integrity and accuracy while also reducing time and money spent on after-hours issues.
Our client’s upstream source systems were causing failures in overnight data processing and required developers to be called after-hours to resolve issues for ETL (extract, transform, load) to successfully complete for the next business day.
As their Execution as a Service partner, it was critical for RevGen to design a solution that was simple enough to work with their current architecture, as the client was in the process of transitioning to a new platform and building an entirely new comprehensive solution wasn’t practical.
Over a year and a half, our client had significant numbers of ETL outages caused by front-end entry data issues resulting in hours of business downtime and skyrocketing support time and costs.
Over the past 18 months, we documented 22 priority 1 ETL outages where business processes were down for approximately 6.25 hours.
Of these priority 1 outages, 91% of them were related to front-end data issues.
Approximately 150 hours had been dedicated to after–hours support needs to resolve these continual issues.
Given RevGen’s specific role as their execution partners, we had to find a resolution to the challenge without creating an entirely new solution design or purchasing a new system. So, we examined the current processes thoroughly and identified where we could insert a new quality control process into the existing ETL after the data is staged but before the ETL begins.
Creating a new process required RevGen’s holistic knowledge of the current systems and processes. Our new schema could not interrupt current business flows, as this data was business critical.
Our new schema organized the new process and identified necessary next steps and allowed us to align stakeholders on the proposed changes, heading off any technical or business issues before they could become bigger problems:
New quarantining tables allowed us to hold any quarantined data separate from the scrubbed data, allowing the new procedures to run without interrupting other processes.
We created new rules to identify bad values prior to the larger ETL process running. These rules scrubbed the data, moving any rows with bad values to the newly established quarantine for further analysis.
We added an extra notification to the ETL log using the existing alert process, which highlights any data quality issues found by the scrub.
“I love the work life balance progress this effort boosts for our technical team members.” – Client Corporate Loan Accounting Director
“This is really great work and will save the company time and money.” – Client Enterprise Data Steward
Approximately 74 hours were used in 2023 to resolve these ETL outage issues, which are no longer needed thanks to RevGen’s implementation of the new schema.
As the new process has nearly eliminated the need for after-hours support for these issues, both for the client-side team and RevGen’s team, employees are better able to relax and focus on their home lives during what should be off hours.
With the drastic cutdown on the after-hours data issues, the client is saving an estimated $7,000 or more on the high cost of expensive after-hours support.
The transfer of marketing leads to sales teams is critical for customer acquisition, however our client’s process was time consuming and heavily manual. RevGen built a proof-of-concept solution that automated the entire handoff.
To improve sales efficiency, our client needed an “At-a-Glance” customer segmentation model that could evolve with their business.
Through a combination of data science and business expertise, RevGen devised a strategy that would grow our client’s revenues by more than 20%.
Get the latest updates and Insights from RevGen delivered straight to your inbox.
