Municipalities are under growing pressure to adopt new technologies, including AI, at an unprecedented pace. Factors such as rising expectations for digital services, ongoing talent shortages, and rapid technological advancements are driving this accelerated shift.The truth is, there are many benefits to adopting AI or AI-enabled systems. These include supporting decision-making, improving workflows, creating efficiencies, generating new capabilities or increasing effectiveness. These are all powerful, game-changing reasons to embrace AI.
However, this increased dependency on technology is a double-edged sword. While there may be some noticeable gains, there are also privacy, security, and safety risks that need to be factored into costs.
Consider this scenario:
You and your team are eager to make council life a bit easier as it is quite time-intensive to record and transcribe council meetings. You also know that other internal and external meetings come with the same burden. You decide to investigate using a new AI-enabled note-taker to reduce the workload and increase efficiency. You assign your senior IT manager to investigate options. After a bit of shopping around, they find what you think will be a great option as it comes at a low cost, is easy to install within the network, and already has some level of security as users need to log in to gain access. You decide to move forward. Once installed, you try it out at the first council meeting, and it does in about a minute which would consume about three hours of the council secretary’s time. And, while the secretary hates to admit it, it was 99% accurate. It seems like it was a wise choice.
Understanding what’s at risk.
Returning to our scenario, let’s explore the implications of an incident.
One month after the implementation of the system, you reflect upon your purchase and think that all is going well and the efficiencies are being realized. Not long after, the supplier of the AI-enabled note-taker released a software update that was installed. Unknowingly, this change resulted in a vulnerability that was subsequently exploited by a cybercriminal who gained access not only to the AI software but the sensitive data that it was using. The system started to malfunction, but it wasn’t until the next council meeting that anyone noticed. When investigating, your tech team noted that not only was the AI system disrupted, but the data had been tampered with and much of it appeared to be missing. This was a massive problem as the data included at least two sensitive in-camera sessions and confidential union negotiation meeting minutes. While your tech team was dealing with this incident, you decided that, for now at least, the risks far outweighed the benefits, and you would stop using the system altogether.
This scenario does not necessarily end easily. All too commonly, there is no plan for backing up or retrieving the original data, no procedures for discontinuing use without additional data loss, and no process identified in the supplier agreement to safely and securely uninstall the system. All of this results in more damage than necessary.
As with other technologies, AI systems and the related processes are not necessarily secure when sold. Further, your employees may not know what to watch out for as they use AI or what to do when things go wrong. Finally, if you no longer want to use the technology, you should know how to safely and securely dispose of it.
These should all be considered before you adopt AI systems, as they will factor into not only your operating costs but also help inform you about risks that you need to manage throughout the lifecycle.
The following presents a framework for organizational risk-based decision-making that considers security and safety throughout the AI system lifecycle.
A risk-based approach to AI adoption is needed
Adoption of new technologies always carries some risk. Many decisions around adopting AI affect municipal operations, including protecting the privacy and safety of your employees and citizens, and ensuring security of your organization.
There are a few questions to help frame decisions.
- What are the risks and benefits of the AI or AI-enabled system?
- What are the risks and responsibilities throughout the lifecycle?
- How can we mitigate potential risks to an acceptable level?
While technology adoption is widely viewed to be in the purview of technical staff, a broader organizational risk-based approach ensures security, compliance, and ethical considerations are fully addressed. This necessarily requires the involvement of additional stakeholders from across the organization. This includes senior decision-makers and technical advisors, privacy officials, business line owners, data managers, and legal and human resources perspectives.
The figure below represents a simplified version of a system lifecycle that provides a framework identification of risk throughout a technology’s lifecycle. We’ve superimposed the discussion of AI and AI-enabled systems onto this framework.
From planning to disposal, risk must be identified and managed. The questions in the figure show that many are not technical in nature but involve organizational inputs and investments to help ensure safe and secure AI systems throughout their lifecycle.
Safe and secure AI is a whole-of-organizational responsibility
Like adoption of other technologies, it is a mistake to think that the responsibility for the safety and security of the organization’s AI systems lies only at the feet of your technical team. As shown above, many considerations and decisions must be made that involve several perspectives within your organization to mitigate risks, address stakeholder concerns, and ensure compliance throughout the AI system lifecycle. The following table presents just a few examples.
Role | Risk-based consideration |
Senior leaders | Identifying AI risks and allocating resources to mitigate those risks |
Technical advisors | Ensuring AI systems are properly installed, verified, and technical vulnerabilities are identified and remediated based on risk. |
Business line owners (Directors) | Understanding how the AI system operates with other systems and processes. Understanding how people interact with the AI. Identifying and managing business line risk within their authority. |
Legal advisors and procurement officials | Ensuring that third-party responsibilities for maintenance and incident response are clarified within agreements. |
Privacy officials | Ensuring that the AI operations that involve data covered under the Privacy Act are properly protected. |
Data managers | Ensuring their data is appropriately categorized and classified based on criticality and sensitivity. |
Users | Identify anomalies in AI operations and know to whom they report any issues. |
Conclusion
AI tools can streamline workflows, save time, and enhance productivity. Given the pace of change, it is difficult to operate within the digital economy without leveraging new and emerging technologies. This is not about avoiding AI but about adopting it responsibly.
It’s important to consider what AI can do for you, but blind adoption is a very risky move. As you consider acquiring and integrating new technologies like AI systems, you should also ensure that safety and security are considered not just at acquisition but throughout the lifecycle. As noted, this is not just a technical responsibility; everyone who has a stake in AI operations has a role to play.
Anita Schretlen is Manager, Training Programs & Partnerships at Rogers Cybersecure Catalyst, Toronto Metropolitan University’s national centre for training, innovation and collaboration in cybersecurity. Randy Purse is the Catalyst’s Senior Advisor, Cybersecurity Training and Education. For more information on municipal cyber training, visit: cybersecurecatalyst.ca/municipal.
