Data Collaboration is a new approach to building apps and systems that is enabled by network-based data management technology (such as a Data Fabric) and capabilities that support secure collaborations between people and systems.
Because the network architecture does not generate copies of data in order to integrate data, it offers unprecedented levels of control, data ownership, and transparency. But this is very different to how digital solutions are built today.
For decades, organizations have approached IT delivery with the mantra "an app for everything, and a database for every app" and this has served us pretty well. But recently, more and more organizations are finding themselves at an impasse.
The problem is that organizations maintain so many apps that data and its controls have become fragmented across hundreds or even thousands of individual databases. Even small and medium-sized organizations now face this challenge and the resulting cost and complexity has created a genuine barrier to innovation.
At the same time, new data protection laws in the USA and Canada are requiring all organizations to offer more control and transparency to the people and businesses who contribute data to their digital services.
But despite the efforts of regulators to get a handle on data, the demands for new apps just keep coming. In fact, they are escalating. But why?
First, many large organizations have made multi-billion dollar investments in IoT, 5G, AI/ML, and blockchain technologies which must now be translated into new apps and systems. Second, the virus-that-dare-not-speak-its-name has created urgent demand for new apps and systems that support contactless services for customers and remote workers.
Add it all up and we're heading towards a perfect storm of data complexity.
"So we need to deliver more tech, faster, with less money, and do it while offering end users more control of their data?"
Source: every IT and ops manager working in 2020
This scenario is precisely why the Data Collaboration approach offers such an enticing prospect and one that is already being adopted by some of the World's most pioneering organizations.
How building apps works today
Today, new apps are delivered by a process filled with silos, replication, high cost, and frustration.
Let's say that you work at an organization that wants to create a new app to give local citizens a safer way to enter and use facilities. Maybe it's for a company campus, school, or hospital.
Once the key features, business logic, control logic, and User Experience (UX) of the app have been identified, one of the next challenges is to identify the data that you'll need for your new app. The sources can include:
SaaS apps e.g. CRM, ERP, ecommerce
In-house systems
Databases
Spreadsheets
AI/ML tools
The next step will be to stand up a new database and connect the required data sources in a process known as integration which is based on generating copies of data.
So far so good, except now comes some tough news:
Building will take 9-24 months
Development will cost $1M-$5M
The project will over-stretch IT resources
Sourcing data will incur high legal costs
Integration (copies) will make true data ownership impossible
"A-ha", we hear you say, "Why wouldn't we just buy an off-the-shelf SaaS tool to solve the problem?"
Yes, you can buy instead of build, but here you will face other challenges:
The tool will be an imperfect match to your needs
You will have no control over new features
Your data is managed by the provider, not you
The service provider may not be compliant
App fees will force you to limit engagement
How Data Collaboration works
By contrast, Data Collaboration leverages a network-based data environment (e.g. Data Fabric) that supports capabilities for linking data and collaborative problem solving. The network effectively replaces the traditional app-specific database, app-specific access controls, and most, sometimes even all of the code.
By adopting the Data Collaboration approach, the delivery of your project would look something like this:
Establish a network-based Data Collaboration platform (cloud or on-premise)
Connect existing data sources to the network (just once)
Add new data directly on the network
Data owners set universal access controls
People and systems collaborate on data models
The data models power app-like experiences via 'Zero Copy' integration
It's the network-based architecture that makes the collaborative approach possible. Here's why that's so different:
One network can be used to power hundreds of solutions, not just one
The network gets more efficient as it grows, not more complex #networkeffects
The network can easily incorporate and data e.g. IoT, AI/Ml, 5G, SaaS tools
The network supports collaboration between people and systems
Zero Copy integration mans apps take weeks or months to deliver, not years
No copies means true consent, custodianship, deletion, and audit are possible
This is a very high-level description and there are still many best practices to be developed but the important thing is that the network gets operational data under CONTROL which is the only way to deliver more apps and systems faster while simultaneously offering meaningful data ownership.
If you would like to support a future for technology (and technologists) that is based on control rather than chaos then please consider getting involved in one of our key programs:
Free skills training (starting in 2021)
We hope your organization can join those already embracing Data Collaboration for its speed, efficiency, and data protection which no other approach can match. 👍