Four Keys to Creating Value with DataOps

Few CFOs ask the IT department to opine on revenue realization. Nor would the strategy VP look to them for due diligence on an acquisition. Yet this happens all the time in data analytics. Businesses routinely bury IT teams with analytics requests that they have neither the time nor the expertise to answer. As data volumes and expectations of its value increase, so does this problem. DataOps can help.

What is DataOps?

DataOps is a set of practices and tools that assure the right people have the right data to make the right decisions. It pulls data from IT and puts it in the hands of decision makers across an organization. With DataOps, people have the authority and the incentives to make decisions with data. Most importantly they have access to the data, which is clearly defined and consistent across the company. It’s secure but available. Your CFO knows where to find profit margin by sales channel, she can see how it’s calculated, and is using the same numbers as a team of data scientists. All this is done without an onerous IT request form.

DataOps is a new development in the world of analytics and still loosely defined (mostly by nascent software companies selling a digital panacea). A good way to understand DataOps is to think of its component philosophies:

• Agile project management. This approach has semi-autonomous teams that complete chunks of work in short increments. It allows those teams to shift quickly as businesses learn new information and priorities change.
• Quality control / Data governance. DataOps calls for automated quality controls that detect data anomalies and overarching governance to catalogue and control data across the organization. Independent analysis is great – but companies need to make sure everyone agrees the same data means the same thing.
• DevOps Orchestration – DevOps is a methodology (the precursor to DataOps) that breaks down the barrier between development teams that write software and the operations teams that host the software. It does this by using version control systems and code repositories to foster parallel development and code reuse and through automated deployments, shortening the deployment timeline of software from months to days (or hours or minutes).

1) It’s about the people

“Empowering people with analytics [is] where the real value creation occurs.” 2018 McKinsey report on analytics.

People – not algorithms – are making decisions. An organization without a data-driven culture will struggle to derive any value from its data and often struggle to compete because of it.  DataOps is effective because it focuses on the people rather than the tools.

Focus on getting people the right data. Organizations that make this difficult find themselves with a web of shadow data pipelines. Motivated, data-savvy employees will find ways around a cumbersome process that squeezes an IT department to answer a barrage of requests. When different people find data on their own, they end up with different results. Even if a number is off a few percentage points, it can destroy confidence. A COO who is used to seeing product line revenue at $1.27 million won’t trust analysis based on $1.28 million – even if the error is nothing more than a difference in rounding.

If a company wants everyone using the same data, that data needs to be easily accessible. This means data requests can’t linger in a queue for two weeks until an IT analyst writes five lines of SQL code. A self-service data repository can shorten the request to 15 minutes.

2) Get the right people together on a team

DataOps emphasizes the use of self-organizing teams with business involvement and short (two to three week) development sprints that deliver analytics faster, with higher quality, accelerating time to value.

It also aims to break down the silos between data analytics teams, software development teams, and IT operations teams. In the same way DevOps puts developers and database operators in the same room on the same team, DataOps assures the business has representatives on the data development team. It’s the only way to ensure the data science stays focused on solving a problem that matters to the business.

Organizations waste countless millions on impressive analytics programming that predicts with spectacular accuracy something no one cares about. Conversely a team without the technical acumen often create faulty analysis that supports bad decisions.

It’s hard to drive real value from data without a team that boasts both technical and functional expertise.

3) Start Small

Avoid introducing too much change too quickly. Focus on a subset of the steps in the value chain rather than immediately introducing new approaches in every step.

For example, start building reliable data governance by cataloging the data. This lets users see what data is available, where it’s located, what it means, and the best practices for using it.  It also promotes self-service. This allows users to avoid time demands on IT and go right to the data they need.

 4) Employ the right tools

For organizations seeking a transformation within their data operations, automation technology can deliver a competitive advantage.  Automation can accelerate the delivery of changes and improve quality throughout the data delivery cycle.

In the world of data analytics, there are thousands of powerful applications that can streamline processes, automatically identify data discrepancies, and produce powerful insights quickly and effectively. But an expensive software package is never step one on a successful journey into data analytics. Building a data-driven culture is an essential precursor.

A $20,000 Martin guitar sounds bad in the hands of a beginner. Software is the same. Once you have the right people, arm them with the best tools. But not the other way around.

Regardless of your starting point, DataOps progression will always lead to more business value by driving speed and scale of operations and analytics throughout the data lifecycle.  The outcome is scalable, repeatable, and predictable data flows for data engineers, data scientists, and business users.

Regina Rider is passionate about delivering innovative solutions that democratize data, automate workflows, and evolve the practice of DataOps to create business value.