Governed AI: what ClarityAILab adds

What is governed AI?

Governed AI is AI built so that people stay accountable for every decision it supports: its outputs can be traced and explained, people can overrule it, and records show what happened. For supply chain and procurement teams, where a purchase, a supplier choice or a customs declaration may have to be justified to an auditor, a regulator or a board, that is the difference between a useful tool and a liability.

At Building Resilience, governance is built into system design from the outset rather than added afterward. Our AI products are developed through ClarityAILab, our AI lab, on that basis.

Why does AI in supply chains need governance?

Supply chain AI is moving from reports to actions: drafting RFQs, qualifying suppliers, extracting data from invoices, approving small purchases. Each step touches money, contracts or compliance. When an AI tool gets one wrong, three questions follow: who decided, on what basis, and could anyone have stopped it? Governed AI is designed so those questions have answers before they are asked. Raji and colleagues describe the gap well: once AI systems are deployed, "emergent issues can become difficult or impossible to trace back to their source." Their answer is to build auditing into the whole development process, with each stage producing documents that form an overall audit record [1].

What do the main frameworks expect?

Two frameworks set the direction many organizations now follow.

The NIST AI Risk Management Framework. Released by the US National Institute of Standards and Technology in January 2023, AI RMF 1.0 is voluntary guidance [2]. It organizes AI risk management into four functions: Govern (a culture, policies and roles for managing AI risk), Map (establishing "the context to frame risks"), Measure (analysing, assessing and monitoring risk; NIST notes that "AI systems should be tested before their deployment and regularly while in operation") and Manage (allocating resources to the risks found, with plans to respond and recover). It describes trustworthy AI as "valid and reliable, safe, secure and resilient, accountable and transparent, explainable and interpretable, privacy enhanced, and fair with their harmful biases managed." In July 2024 NIST added a companion profile for generative AI, applying the same four functions to tools that draft text and documents [3].

The EU AI Act. Article 14 of Regulation (EU) 2024/1689 requires high-risk AI systems to be designed "in such a way, including with appropriate human-machine interface tools, that they can be effectively overseen by natural persons" [4]. The people overseeing them must be able to:

• understand the system's capacities and limitations;
• remain aware of automation bias, the tendency to over-rely on the system's output [5, 6];
• correctly interpret its output;
• decide not to use it, or to "disregard, override or reverse the output";
• intervene, or "interrupt the system through a 'stop' button or a similar procedure."

Whether a given system is high-risk under the Act depends on its use, and Canadian organizations are not directly bound by it unless they operate in the EU market. In Canada, the Treasury Board's Directive on Automated Decision-Making already applies to most federal institutions: it requires an Algorithmic Impact Assessment before an automated decision system goes into production, and more human involvement as the impact level rises [7].

Automation bias is not a theoretical worry. Parasuraman and Manzey's review found that it leads to errors of both omission and commission when decision aids are imperfect, and that the related problem of complacency "cannot be overcome with simple practice" [5]. A systematic review by Goddard and colleagues, which screened 13,821 papers and included 74, found that factors such as trust, experience and the design of the decision support all affect how much people over-rely on automation [6]. Good design, not good intentions, is what keeps people in charge. But the two frameworks agree on the core: people in charge, decisions traceable, risks tested before and during use.

What does ClarityAILab add?

ClarityAILab builds these principles into the product itself, so governance is how the tool works rather than a policy document beside it. In MaiMai, our supply chain AI developed through ClarityAILab and offered as Clarity Enterprise, that means:

Behind the products sits the same approach as all our work, set out on our Governance page: accountability for decisions and outcomes, outputs and processes that are understandable and traceable, AI that supports human decision-making rather than replacing it, and data that is entrusted, not owned. ClarityAILab's editorial board is chaired by our founder, Dr. Jonathan D. Linton, former Editor-in-Chief of Technovation.

How does governed AI fit with decision support?

Governed AI is one part of a larger system. Our AI work includes an early warning algorithm that identified a significant spike in deaths in early January 2020, weeks before the first official COVID-19 case was reported in the UK, and that same principle applies: the algorithm flags the change; people decide what to do about it. In decision support under uncertainty, models show the options and trade-offs with the assumptions visible, so the people accountable can challenge them before they commit.

What to do next

Before adopting any AI tool for supply chain or procurement work, ask the vendor five questions:

1. Who approves what the AI proposes, and where is that recorded?
2. Can we trace any output back to its source documents?
3. Can a person override or stop it, and how?
4. How was it tested before release, and how is it monitored now?
5. Who owns the data, and where does it go?

If the answers are vague, the governance is missing. To see how MaiMai answers them, visit ClarityAILab's Clarity Enterprise page or talk to us.

References

1. Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020). Closing the AI accountability gap: Defining an end-to-end framework for internal algorithmic auditing. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (pp. 33–44). ACM. doi:10.1145/3351095.3372873
2. National Institute of Standards and Technology (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1. doi:10.6028/NIST.AI.100-1
3. National Institute of Standards and Technology (2024). Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1. doi:10.6028/NIST.AI.600-1
4. Regulation (EU) 2024/1689 of the European Parliament and of the Council (Artificial Intelligence Act), Article 14: Human oversight. Official Journal of the European Union, L, 12 July 2024. eur-lex.europa.eu/eli/reg/2024/1689/oj
5. Parasuraman, R., & Manzey, D. H. (2010). Complacency and bias in human use of automation: An attentional integration. Human Factors, 52(3), 381–410. doi:10.1177/0018720810376055
6. Goddard, K., Roudsari, A., & Wyatt, J. C. (2012). Automation bias: A systematic review of frequency, effect mediators, and mitigators. Journal of the American Medical Informatics Association, 19(1), 121–127. doi:10.1136/amiajnl-2011-000089
7. Treasury Board of Canada Secretariat. Directive on Automated Decision-Making. tbs-sct.canada.ca/pol/doc-eng.aspx?id=32592