AI vs privacy: How safe is your data?
We’ve entered an era where AI tools like ChatGPT and Microsoft Copilot are as commonplace as email. These days, it’s not unusual for students to write essays with AI tools or for professionals to use them to summarise meetings. Large language models - and similar models that can process different types of data beyond text, such as images, audio, and sometimes even video (known as multimodal models) - have found their way into everyday routines.
Considering how commonplace the use of AI tools has become, their privacy should be a pressing concern for individuals, and even more so for businesses.
The invisible data trail
Every time you type a prompt, upload an image, or share a document with an AI assistant, you’re creating a digital breadcrumb trail. Often, you might be sharing more than you realise.
Take an seemingly innocent example: say you upload a photo of your fridge to ask for recipe suggestions. That image reveals your dietary preferences, favourite brands; it even hints at your income bracket or household size.
Even though this is unstructured data, it's still DATA. And the right AI model, it can be interpreted, categorized, and potentially be used.
What happens to your data?
OpenAI, the company behind ChatGPT, has stated that they ‘do not share user content with third parties for marketing purposes’. However, they do work with a number of subprocessors: third-party companies that provide services like moderation, data storage, and infrastructure support.
Unless you opt out, your interactions with ChatGPT (outside of API, Enterprise, or Team plans) may be used to improve their models. This includes analysing prompts and responses to refine how these models understand and generates content.
And remember, it's not just about language - interactions with models like the ones behind ChatGPT can also be used to improve how they handle different types of data and help them evolve. And this can be text, images, audio, and so on.
In other words, your data might not be sold - but it could still be stored, studied, and used to train future versions of the model. While OpenAI offers features like “Temporary Chats” and data deletion requests, the default settings still lean toward data retention.
Regulatory frameworks are evolving
Recent legal developments have added complexity.
A U.S. court order in 2025 required OpenAI to preserve all ChatGPT logs, including deleted chats, as part of ongoing litigation. The case in question was brought by the New York Times and other media outlets, and was all about retaining the evidence of users potentially prompting AI to reproduce copyrighted material.
OpenAI pushed back, citing user privacy concerns - but the case highlights how quickly the rules around AI and data can change, and how fragile our assumptions about privacy may be.
The outcome: the order stands, and now affects users of ChatGPT Free, Plus, Pro, and API services. However, it does not apply to ChatGPT Enterprise, ChatGPT Edu, or users with a Zero Data Retention agreement
OpenAI is appealing the decision, arguing that it undermines user privacy and may conflict with international data protection laws like the GDPR .
For users, the reality is that, at least for now, even if you delete your chats, some version of that data is still stored somewhere.
So, what should we be asking?
The real issue isn’t just whether AI companies are acting responsibly or whether all their sub-processors are secure enough - it’s also whether users fully understand what they’re sharing in the first place.
Should there be stricter data protection laws for AI interactions? Possibly.
Should AI companies be transparent about how data is used? Absolutely.
But perhaps most importantly: should users be more mindful of what they input into these tools? Yes – and urgently.
We need to start treating AI prompts the same way we treat social media posts or emails - because they can be just as revealing, if not more so. That means thinking twice before uploading personal photos, sharing sensitive documents, or even describing private situations in detail.
A call for AI literacy
As AI becomes more embedded in our lives, we need a new kind of digital literacy - AI literacy.
This includes understanding how AI models work, what data they collect, and how that data might be used. It also means recognising that even unstructured data - like a casual chat or a blurry photo - can contain a surprising amount of information about you, and more!
In the age of ubiquitous AI use, privacy isn’t just about what companies do with your data. It’s about what you choose to share in the first place.
Explore our AI courses and skills training.
