Data streaming: protecting consumers in the AI era

As AI grows, tough questions about data privacy are hitting the mainstream.

Here in the UK, the government’s Data (Use and Access) Bill has just entered its second reading in the House of Commons. At the same time, the government is navigating a legal challenge from Apple, which refuses to allow it to access a customer’s private data if required. Concerns around privacy, and the rules that guarantee it, are being examined at the very top.

The details are complex, but for most people it boils down to one simple question: “Is my data the cost of progress?”

From an enterprise perspective, we know that the technology used to house, transmit, and safeguard data isn’t necessarily built to prioritize privacy. Insufficient security, siloed data, and the pains of outdated infrastructure can all leave an organization exposed — in other words, those consumer concerns can be well founded.

Fixing this starts with how we collect and move data. If we can embed privacy straight into the data collection process, assessing and directing data in real time, we can minimize risk to both organization and consumer.

Tackling patchwork infrastructure

One of the reasons that fear around data privacy have risen in recent times is that it’s extremely difficult to keep up with new regulations – particularly given how quickly AI is evolving as a technology.

New and important legislation is constantly being introduced, with the UK’s Data Bill just one of many: the EU’s AI Act, China’s Deep Synthesis Provisions, and so on. But such laws are attempting to regulate technology as it develops, and can’t necessarily pre-empt the next big application of AI. The rule of law can’t always keep up with the blistering pace of change, often focusing on compliance at the expense of proactive security measures.

As a result, many organizations will continue to depend on a unique mix of hardware and infrastructure. The business can function, but it can’t robustly enforce a consistent standard of cybersecurity and data privacy.

Take, for example, the classic problem of data fragmentation. Spreading private data across multiple systems — each with their own capabilities, purposes, and cybersecurity measures — makes it incredibly difficult to standardize the use of data. It might be duplicated, accessible in some systems and not others, or simply absent from where you’d expect it.

The chinks in this armor are exacerbated by human intervention. Employees might have differing levels of access across different systems, retain permissions that they’re no longer supposed to, or not understand how to protect the data they’re accessing once they’ve used it. All of these things can cause a serious breach.

While regulations like GDPR and HIPAA are designed to mitigate these risks, insisting on the immediate execution of data deletion requests, the infrastructure that houses this data might not be capable of meeting these demands. Legacy technology both decelerates audits and response times and fails to paint a comprehensive picture of exactly what data needs to be deleted.

Introducing data streaming

All of these challenges make it clear that privacy can’t be an afterthought. If you don’t prioritize it at the points where data enters your ecosystem, it’s incredibly difficult to work back to a point of robust compliance.

This is where real-time data streaming excels. In processing data as it arrives, data streaming prevents the creation of vast datasets that demand slow, cumbersome batch processing en-masse. Being able to contextualize data and the protection it needs even as that data is in motion frontloads that security and organizational work, literally point by point.

Data streaming platforms (DSP) take this a step further. As a platform purpose-built to coordinate the streaming of data across a business, they offer a single point of access that integrates advanced security capabilities by design.

End-to-end encryption, for example, offers an additional layer of protection while the data is in transmission. Similarly, tokenisation can replace business-critical information with identifiers that make transcription impossible. Another option is differential privacy – the introduction of mathematical noise to datasets, protecting individual identities without stopping a business benefitting from analyzing that data.

All of these elements offer protection against the myriad of potential cybersecurity concerns across an organization — from accidental access to bad actors.

Security and the DSP

Research suggests that a vast majority of UK tech leaders — 91% — believe that data streaming improves cybersecurity and digital risk management.

Much of that value comes from the DSP acting as the organization's central nervous system, managing systems to keep everything in sync, and guaranteeing access to real-time data where it’s needed. Privacy and security are inherently baked into the system, from the point of entry to the point of use.

As AI continues to establish itself as a norm, this will only become more important. The business world is speeding up – and the technology we use to protect it has to accelerate, too.

We've featured the best productivity tool.

This article was produced as part of TechRadarPro's Expert Insights channel where we feature the best and brightest minds in the technology industry today. The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/news/submit-your-story-to-techradar-pro