Responsible AI integration: Guidance from the Privacy Commissioner

In today's ever-evolving governance landscape, artificial intelligence (AI) is reshaping the way business operates, requiring directors to recognise the potential impact and implications of AI on their organisation. To ensure the responsible integration of AI within your organisation, thoughtful consideration and informed decision-making are paramount.

The Office of the Privacy Commissioner has issued its expectations regarding the use of AI by organisations. On its website, Privacy Commissioner Michael Webster has emphasised the need for businesses, and organisations to consider the implications of using generative AI systems that involve personal information.

The Commissioner notes that privacy risks already associated with generative AI include:

• that the information used to “train” generative AI contains information that is sensitive, may not be accurate or contains bias
• that information you enter is retained by the generative AI provider
• that the information created by generative AI is inaccurate
• that individuals are unable to correct inaccurate information held by generative AI

The use of AI and personal information falls under the regulation of the Privacy Act 2020. The Privacy Commissioner urges businesses and organisations to understand their obligations regarding generative AI before implementation. To assist in this regard, he has provided seven points of advice:

• Involve senior leadership and privacy officers in decision-making processes regarding generative AI implementation.
• Assess whether using a generative AI tool is necessary and proportionate, considering privacy implications.
• Conduct a Privacy Impact Assessment, seeking feedback from impacted communities, including Māori, and requesting information from providers regarding privacy protections.
• Be transparent with customers and clients about the use of generative AI, its associated privacy risks, and how those risks are managed.
   Establish procedures to ensure accuracy of information, respond to access and correction requests, and safeguard individual rights.
• Implement human review before taking action based on generative AI outputs to mitigate the risk of acting on inaccurate information.
• Avoid inputting personal or confidential information into generative AI tools, unless explicitly confirmed that such information is not retained or disclosed by the provider. Sensitivity and confidentiality should be carefully considered during training.

The Privacy Commissioner expects organisations to conduct due diligence and privacy analysis before using generative AI and confirms that his office will ensure compliance with the law through investigations.

What this means for directors

The Office of the Privacy Commissioner's guidance serves as a valuable resource for directors, offering a framework to effectively navigate the intricate AI landscape. If you haven't already, make sure to prioritise AI as a topic of discussion on your board's agenda.

To foster meaningful conversations within the boardroom, directors should consider the following key questions:

• Strategic Alignment: How does AI integration align with our organisation's mission, values, and long-term goals? What are the potential risks and benefits of implementing AI in our industry and organisation?

• Ethical Considerations: How can we ensure ethical and responsible AI development and deployment? What measures should we take to prevent bias and discrimination in AI algorithms?

• Data Governance and Privacy: How do we protect customers' data privacy and security when utilising AI technologies? Are we adequately addressing data protection and privacy concerns outlined in the Privacy Commissioner's guide?

• Human-AI Collaboration: How can we strike a balance between automation and human involvement? What roles should humans play in decision-making processes involving AI?

• Risk Management: Have we identified and assessed potential risks associated with AI implementation? How can we mitigate these risks and ensure the resilience of our AI systems?