Making AI operational in constrained public sector environments

Public sector organizations are increasingly pressured to adopt artificial intelligence (AI), but face unique challenges that distinguish them from private sector counterparts. Government institutions must navigate strict security, governance, and operational constraints, particularly regarding sensitive data control and legal compliance. These factors complicate AI deployment, limiting many public sector AI initiatives to pilot stages without broader operationalization. Unlike private companies that rely on continuous cloud connectivity, centralized infrastructure, and less restrictive data movement, public sector agencies often operate in environments with limited or unreliable internet access. They must ensure data remains under strict control, maintain verifiable information, and minimize operational disruptions. Additionally, many government bodies lack experience managing the specialized hardware, such as graphics processing units (GPUs), required for training and running large AI models, creating further bottlenecks. To address these challenges, purpose-built small language models (SLMs) have emerged as a promising solution. SLMs are AI models with fewer parameters than large language models (LLMs), making them less computationally intensive and easier to deploy locally. This local housing enhances data security and operational control, aligning better with public sector requirements. Studies indicate that SLMs can perform as well or better than larger models while avoiding the complexities and risks associated with offsite, centralized AI systems. The adoption of SLMs could enable government agencies to leverage AI effectively within their operational constraints, supporting real-time data use and scalable AI growth without compromising security or continuity. As public sector organizations seek to modernize services and improve efficiency, these tailored AI models offer a viable path to overcoming longstanding barriers and accelerating responsible AI integration.