The Shadow AI Problem: What Government Is Doing About Sensitive Data That Won't Stop Leaving

Picture a routine morning inside an Australian government agency. A policy officer working on a briefing pastes a paragraph of internal advice into ChatGPT to sharpen the phrasing. A project manager drops meeting notes into Copilot to summarise action items. A data analyst uploads a partially de-identified dataset into a free-tier AI tool to test a hypothesis before doing it “properly.” None of them intend harm. Most of them do not think they are doing anything unusual. All of them are contributing to a problem that public sector CIOs and CISOs are now actively trying to solve.

It has a name: shadow AI. And the numbers around it are hard to ignore.

The data

Global research paints a consistent picture. Around 77% of employees share sensitive company information through ChatGPT and similar AI tools. Roughly 18% of enterprise employees paste data into generative AI tools, and more than half of those paste events include corporate information. Nearly 40% of files uploaded to AI platforms contain personally identifiable or payment-card data.

IBM's 2025 breach reporting found that one in five organisations experienced a breach linked to shadow AI use. Organisations with high shadow AI exposure faced breach costs an average of US$670,000 higher than those with minimal exposure. A further 83% of organisations lack technical controls to detect or prevent confidential data being uploaded to AI platforms in the first place.

For government, those numbers land differently than they do for private industry. The data being pasted may include unreleased policy, protected personal records, briefings covered by cabinet convention, or classified information. The stakes attached to a “small” mistake are structurally higher.

Australia's policy response: decisive, but not yet operational

The Australian government has moved quickly on the policy side. The OAIC has published two AI-specific privacy guidelines. The National AI Centre's Guidance for AI Adoption (October 2025) sets out six essential practices for responsible AI governance. The DTA's updated Policy for the Responsible Use of AI in Government came into effect on 15 December 2025. The APS AI Plan 2025, structured around Trust, People, and Tools, is being implemented jointly by the Department of Finance, the DTA, and the APSC. Chief AI Officers will be appointed across every federal agency through 2026.

That is a comprehensive policy scaffold. What it does not do, and what no policy document can do on its own, is govern the specific decision an individual staff member makes at 10:47 on a Tuesday about whether to paste a paragraph into a free AI tool.

What operational governance actually looks like

The agencies closing the shadow AI gap fastest have moved past policy publication into the harder work of embedding governance into daily practice. Four disciplines stand out.

Sanctioned tooling that is governed by default. Rather than blocking AI outright (which drives usage further into the shadows), leading agencies are providing approved, enterprise-grade AI tools with data protection, audit trails, and clear guidance on what data classifications can be used with them. Availability plus governance beats prohibition every time.

Data classification made practical. Staff need to know at the moment of decision whether a given piece of information can go into a given tool. That means simple, visible classification prompts inside the tools themselves, not a forty-page policy document referenced once during induction.

Continuous audit and monitoring. The technical controls exist to see what data is moving into AI platforms. The 83% of organisations without those controls are choosing not to see. Government agencies are moving to close that gap, quickly.

Privacy impact assessments before deployment. The OAIC's guidance is clear: PIAs are not optional for AI systems handling personal information. They are the point at which risk is understood, not the point at which it is documented after the fact.

Together, these practices are what turn a policy scaffold into governance in practice.

Where the discipline lives

Shadow AI is fundamentally an identity, data governance, and cyber discipline problem. That is the same combination of disciplines that underpins the strongest public sector IT solutions being delivered in Australia today, and it is exactly the ground the strongest Government IT Consulting and Managed Services engagements are built on. Government Enterprise IT Partners with lived experience across identity assurance, data classification, and operational cyber are the ones best positioned to help agencies close the gap between what their policy says and what their people actually do.

How TSC Can Help

At The Services Company, we help government IT departments design and deliver citizen experiences that strike the right balance between technology and humanity. As a specialist public sector IT provider, we bring together intuitive portals, AI-assisted services and accessibility-first design to make complex systems simple, for the citizens who use them and the teams who manage them.

If your department is exploring how to bring AI and automation into your citizen experience strategy (without losing the human touch), let's talk.

Get in touch with our team at The Services Company, Australia’s Most Trusted NSW Government IT services partner.

The Services Company – Australia's Most Trusted Public Sector IT Provider

The Services Company (TSC) is a Sydney-based public sector IT provider delivering government IT consulting and managed services, cybersecurity consulting and ITSM to NSW state and federal departments.

We help departments run leaner, more transparent programs across Citizen Experience, Government Efficiency, Critical Services and Departmental Transformation, backed by 20+ years as the government enterprise IT partner Sydney teams rely on.

Whether you need an NSW Government IT managed services partner or public sector IT solutions for a single critical project, TSC delivers secure, compliant, citizen-first outcomes. Panel listings: SCM0020, SCM0005 and LGP ICT Products and Services.

Sources

Office of the Australian Information Commissioner (OAIC) — Guidance on privacy and the use of commercially available AI products and Guidance on privacy and developing and training generative AI models (oaic.gov.au); Digital Transformation Agency — Policy for the Responsible Use of AI in Government and APS AI Plan 2025 (dta.gov.au, digital.gov.au); National AI Centre / Department of Industry — Guidance for AI Adoption and Voluntary AI Safety Standard (industry.gov.au); IBM Cost of a Data Breach Report 2025; eSecurity Planet, “77% of Employees Leak Data via ChatGPT” (2025); Menlo Security State of Shadow AI Report (2025).