AI is fundamentally eroding commercial confidentiality by placing sensitive corporate data into unmanaged environments. Erosion occurs through a combination of unauthorised employee use of public AI tools, and vulnerabilities in connected AI agents. These vulnerabilities compromise trade secrets, obviate legal privilege, and breach data governance.

How and where is this erosion of confidentiality occurring?

How Do Breaches Of Confidentiality Occur?

Sharing privileged information with a third-party AI platform legally acts as an implied waiver, which permanently destroys confidentiality protections.

Data ingested by public AI models cannot easily be "forgotten" or deleted upon termination of a contract, in consequence standard non-disclosure agreement (NDA) return-or-destroy clauses are impossible to fulfil.

Bad actors can deceive AI agents operating within internal systems, to extract or forward sensitive data to external, unauthorized destinations.

Traditional confidentiality agreements fail to define AI processing or AI-generated outputs as breaches, leaving organizations without legal recourse when proprietary data is synthesized and leaked by AI

Where Is Confidentiality Breached?

Employees routinely input sensitive business data, such as source code, financial forecasts, strategic plans, and client personal identifiable information into consumer-grade models. These “free” or standard platforms use this data to train their models, making it accessible to third parties.

Organizations link AI agents directly to internal enterprise systems in order to automate workflows. If compromised or misconfigured, these agents can expose, modify, or delete vast volumes of confidential data at speed.

Courts in the UK, US, and internationally have warned that uploading confidential documents to open-source or public AI tools, places that information in the public domain. This often voids legal professional privilege and trade secret protections entirely.

The Weaknesses Of Consumer Based AI Services

Consumer-based AI services have issues with data privacy, inaccuracy (sometimes known as hallucinations), lack of empathy, and algorithmic bias. All of these erode user trust and often create frustration when automated systems fail to handle complex or sensitive real-world situations effectively.

In order to function, most major AI platforms scrape large amounts of public data. They ingest personal user interactions by default to train their models. When consumers input personal, financial, or medical information into public AI tools, that data risks being exposed or reproduced in unstructured form. Strict data protection frameworks make managing consumer consent and right-to-be-forgotten requests a major challenge for AI developers.

As most publicly available AI services are Large Language Models (LLMs), and are using statistical probabilities to generate the next word in a sentence, rather than logic or deduction, they are prone to fabricating highly confident but completely false, or misleading information. The software delivers what appears to be authoritative statements, and naively, some consumers rely on the data for medical, financial, or legal advice.

With respect to customer service, reliance on AI chatbots often results in deflection rather than resolution leaving users stuck in automated loops, and AI agents struggle to recognise the emotional context of a conversation. As a result they respond with rigid template derived language which fails to identify a user distress or emotion.

When an AI is trained on skewed historical data sets, outputs can inadvertently produce discriminatory or exclusionary content, specifically in areas like automated screening and dynamic pricing. AI systems may unintentionally penalise certain demographics, reject job applications, or unfairly profile consumers based on inferred or proxy data.

As AI models become more complex, their rationalisation processes tend towards invisibility. Invisibility makes it difficult for users to understand why AI generated a specific output or made a decision affecting them.

Governments Are Taking Action To Protect Consumers By Regulation

In Europe the EU protects consumers through the EU AI Act. The Act applies stringent risk-based rules and completely bans harmful AI practices (like manipulative tools and non-consensual deep fake activities) as well as imposing strict audits on "high-risk" AI (e.g., credit scoring and healthcare), and mandating transparency so consumers know when they are interacting with AI

The EU's consumer protection strategy operates in a number of ways:

AI applications are classified into four risk tiers, with the two highest levels deserving special attention: The highest level comprises unacceptable risks which are banned outright, such as manipulative unconscious techniques, social emotion recognition in workplaces/schools, and untargeted scraping of facial images. The second category, “high risk” category, is subject to mandatory fundamental rights impact assessments, high cybersecurity, human oversight, and detailed documentation and include AI used in employment (CV scanning), credit scoring, and access to essential healthcare.

The regulations are intended to ensure that consumers are not misled by AI. Accordingly, AI systems must clearly disclose when content (text, audio, or video) is artificially generated or manipulated. Users must be explicitly informed when they are interacting with an AI chatbot or automate customer service systems.

To ensure accountability when AI causes harm, the EU enforces parallel legislation, such as the Product Liability Directive. Consumers are given clearer rights to seek compensation for damage caused by defective AI systems or algorithmic decisions. Rules for general-purpose AI (GPAI) require model providers to protect consumer data and uphold EU copyright laws.

Where Government Regulation Is Insufficient, Institutions Need To Take Their Own Steps To Combat Erosion Of Confidentiality

Combating the erosion of confidentiality requires a multi- faceted approach to establish safe, enterprise-grade AI platforms, implementing Data Loss Prevention (DLP) tools to stop accidental leaks, deployment of local or private AI models, and educating enterprise on strict data-handling policies:

Implement Enterprise-Grade AI Tools

Enterprises should not use consumer-grade AI platforms for work-related data, as they often use prompts and chats to train their models. They should adopt enterprise versions of major AI tools such as Claude, Microsoft Copilot.. These platforms deploy business-grade privacy guarantees- all data is isolated, is not used for AI training, and complies with technical standards like SOC 2. Settings that allow AI providers to use enterprise prompts and conversations for future model training should be reviewed and disabled.

Deploy Technical Safeguards

Solutions like Microsoft Purview or Cloudflare DLP scan prompts and file uploads in real-time to block, redact, or log sensitive information such as financial data, or trade secrets before the data leaves enterprise networks. Restrictions should be placed on accessibility to highly sensitive datasets. Logged-in users must be notified

Adopt Local & Private AI

To achieve absolute control over confidentiality, firms should use use private, on-premise, or controlled cloud environments (such as Microsoft Azure OpenAI). This guarantees your confidential information never leaves your personal or company servers.

Optimise AI Prompting Practices

Effective AI prompting practices are achieved by stripping out all sensitive information such as client names, real IP addresses before entering a prompt.

The Future Likelihood Of AI Further Eroding Confidentiality Is Extremely High

The future likelihood that AI will continue to erode confidentiality arises because AI systems will become more integrated if not embedded into daily life. Historic boundaries of privacy and data security face unprecedented stress:

Advanced AI models require vast datasets to train and improve. The demand for vast data sets creates a constant incentive for companies to scrape, aggregate, and analyse private communications, documents, and behaviours.

AI does not only read explicit data; but also reads between the lines. By analysing public or semi-private footprints, AI can accurately deduce highly confidential details about for example an individual's health, and other personal; information, or political beliefs.

The proliferation of AI-powered smart home devices, wearables, and workplace monitoring tools means audio, video, and biometric data are continuously processed. In this environment the risk of accidental exposure or unauthorized profiling.

Malicious agents use AI for phishing attacks and crack encryption, that make the technical storage of confidential data inherently less secure.

AI can recreate highly realistic deepfakes - photos, audio or video created by or changed by deployment of AI. These can undermine the security of identity verification systems.

Pressures On Confidentiality Will Continue To Be Severe But There Are Some Mitigating Factors

Regulatory constraints like the European Union's AI Act and evolving global privacy laws impose strict penalties for unauthorised data processing and require data minimization.

Development in fields like homomorphic encryption (computations made on encrypted data without having to de-encrypt the data), and federated learning, enable AI models to train on data without using confidential information.

Growing public awareness is driving a market for "privacy-first" AI tools that process data locally on user devices rather than in the cloud..

However, the key and overriding takeaway is that putting privileged or disputed facts into even an enterprise-grade AI tool can result in an irrevocable waiver of confidentiality and legal privilege.