top of page
  • Chris McLellan

How does Data Collaboration work?

Updated: Jan 26, 2021

Data Collaboration is a new approach to building apps and systems that is enabled by network-based data management technology (such as a Data Fabric) and capabilities that support secure collaborations between people and systems.

Because the network architecture does not generate copies of data in order to integrate data, it offers unprecedented levels of control, data ownership, and transparency. But this is very different to how digital solutions are built today.

For decades, organizations have approached IT delivery with the mantra "an app for everything, and a database for every app" and this has served us pretty well. But recently, more and more organizations are finding themselves at an impasse.

The problem is that organizations maintain so many apps that data and its controls have become fragmented across hundreds or even thousands of individual databases. Even small and medium-sized organizations now face this challenge and the resulting cost and complexity has created a genuine barrier to innovation.

At the same time, new data protection laws in the USA and Canada are requiring all organizations to offer more control and transparency to the people and businesses who contribute data to their digital services.

But despite the efforts of regulators to get a handle on data, the demands for new apps just keep coming. In fact, they are escalating. But why?

First, many large organizations have made multi-billion dollar investments in IoT, 5G, AI/ML, and blockchain technologies which must now be translated into new apps and systems. Second, the virus-that-dare-not-speak-its-name has created urgent demand for new apps and systems that support contactless services for customers and remote workers.

Add it all up and we're heading towards a perfect storm of data complexity.

"So we need to deliver more tech, faster, with less money, and do it while offering end users more control of their data?"

Source: every IT and ops manager working in 2020

This scenario is precisely why the Data Collaboration approach offers such an enticing prospect and one that is already being adopted by some of the World's most pioneering organizations.

How building apps works today

Today, new apps are delivered by a process filled with silos, replication, high cost, and frustration.

Let's say that you work at an organization that wants to create a new app to give local citizens a safer way to enter and use facilities. Maybe it's for a company campus, school, or hospital.

Once the key features, business logic, control logic, and User Experience (UX) of the app have been identified, one of the next challenges is to identify the data that you'll need for your new app. The sources can include:

  • SaaS apps e.g. CRM, ERP, ecommerce

  • In-house systems

  • Databases

  • Spreadsheets

  • AI/ML tools

The next step will be to stand up a new database and connect the required data sources in a process known as integration which is based on generating copies of data.

So far so good, except now comes some tough news:

  • Building will take 9-24 months

  • Development will cost $1M-$5M

  • The project will over-stretch IT resources

  • Sourcing data will incur high legal costs

  • Integration (copies) will make true data ownership impossible