How can data analytics managers learn from the history of environmental management? In two previous posts, I’ve explored how big data is like the new oil in more ways than one and how forward-thinking industrial management practices can benefit big data projects and technologies. Now let’s look at what a big data management system approach would look like in reality.

A big data management system

For starters, a big data management system approach would be just that, a system. Rather than thinking about privacy concerns at the end of a project, privacy concerns can be accounted for in the overall process. This would have line of business owners, privacy professionals, data scientists, and programmers collaborating together to be aware of potential privacy and discriminatory impacts as they extract valuable insights from diverse data sets to test and develop their algorithms.

A big data management system would have—represented throughout the process—the organizational functions responsible for mitigating privacy and discriminatory impacts. This would ensure that product design, engineering, and operations teams see both the benefits and the potential privacy and discrimination issues as they design and implement algorithms and applications. In turn, this would also ensure that those responsible for privacy and data protection are aware of the engineering, economic, and business costs of designing for any given privacy protection. This would reduce the need for late-stage evaluation of the product, because societal and business implications—both those beneficial or potentially harmful—would be considered throughout.

Agile and DevOps methods

This approach fits naturally with the way that companies increasingly test, develop, and operate their applications and big data systems. Companies have increasingly moved from top-down, compartmentalized models such as ‘waterfall’ to adopt agile project management and DevOps software development methods that embrace an emergent and collaborative approach.

Originating from the same lean manufacturing roots as Environmental Management Systems (EMS), agile and DevOps seek to make continuous improvements throughout the process, not at the end of it. A "minimum viable product" is conceived, launched, and then rapidly iterated upon by teams of people to improve it. By making privacy leaders part of agile teams, privacy and discriminatory issues can be considered at the outset when defining the minimum viable product, and these leaders can be involved in identifying and making privacy-related improvements as issues arise.

A revolution in organizational management theory

In this time of rapid change, management systems with agility have a higher fitness than management systems striving only for efficiency. It's part of a wider agility revolution well underway in organizational management theory.

In his book XLR8, leading organizational change author John Kotter talks about the need for organizations to develop a dual operating system where a hierarchy acts as a superstructure for collaborative, self-forming teams to pursue big opportunities.

General Stanley McChrystal in his book Team of Teams explains how the hierarchical organization perfected last century for efficiency in the industrial revolution needs to give way in this century to a team of teams optimized for agility in the rapidly changing opening decades of this century’s information revolution.

Protecting personal data at scale

In the end, it’s clear that – like the need for environmental protection from fossil fuels at scale – there is a need to protect the ecology of personal data at scale. The data protection regulations and management models we choose today need to be aligned with emerging, collaborative project management and software development methodologies. This will allow organizations to accelerate taking advantage of big data’s many benefits while mitigating potential unintended consequences and, thus, support the the long-term sustainability of the big data economy.

If you want to explore this topic in more depth, please sign up below and get access to my paper: Big Data Sustainability: An Environmental Management Systems Analogy.

Get access to insights on data governance

Article written with Prof. Dennis Hirsch.

Data & Analytics Security

Jonathan H. King

Jonathan H. King is Head of Portfolio Management, PA Cloud Systems, for Ericsson. His responsibilities include product area cloud strategy, business development, mergers and acquisitions, alliance development and go to market strategy. Prior to joining Ericsson, Jonathan was Vice President, Platform Strategy and Business Development for CenturyLink, as well as SVP of WW Business Development at Joyent and a Client Partner Director with the Global Solutions group of Verizon Business. Jonathan holds a B.A. in History from Miami University, a J.D. from Loyola University Chicago School of Law and a LL. M. in Intellectual Property and Technology Law from Washington University School of Law. Jonathan remains an active legal scholar currently researching and publishing articles on big data and privacy as a Visiting Scholar at Washington University School of Law.

Follow Jonathan H. King:
Jonathan H. King