AI-Based systems need regulation. Part of the regulation is to ensure high quality characteristics.
ISO/IEC 25059 provides defines out characteristics that AI-based systems should have
- AI Functional correctness
- Functional adaptability
- User controllability
- Transparency
- AI robustness
- Intervenability
- Societal and ethical risk mitigation.
Every AI-based system should therefore aim at achieving the above quality characteristics. (Hold on we will discuss each of these in a few)
Before then, it is important to note that achieving each of these quality requirements is not easy. AI-based systems have a fair share of challenges which include;
- Vague specifications
- Non-determinism
- Self-learning
- Limited explainability
- Evolving standards
AI-specific Quality Characteristics
AI Functional Correctness
AI-based systems especially those that use probabilistic ML, cannot guarantee perfect accuracy. A certain error rate is expected in AI outputs. ISO/IEC 25059 evaluates functional correctness by considering both correct and incorrect outputs and defining acceptable thresholds for correct results, reflecting the inherent variability in AI-based system outputs.
AI Functional correctness relates to product quality
Functional adaptability
This is a new sub-characteristic of functional sustainability that also relates to product quality. Functional adaptability simply means the ability of an AI-system to autonomously adapt to changes in its operational environment after it has been deployed.
User Controllability
User controllability relates to product quality and is a new sub-characteristic of interaction capability. In this context, interaction capability is a new name for usability.
This quality characteristic refers to: Human/ external agent can intervene in its functioning in a timely manner.
Transparency
This relates to the degree in which appropriate information about the AI-based system is communicated to stakeholders.
Transparency is a new sub-characteristic of interaction capability and satisfaction and relates to both product quality and quality in use.
AI robustness
It is a product quality sub-characteristic of reliability.
Simply put, AI robustness is the ability of an AI-based system to maintain its level of AI functional correctness regardless of circumstances.
Such circumstances include;
- Presence of biased, adversarial or invalid data inputs
- External interference
- Adverse environmental conditions
- Operator misuse
Intervenability
Refers to ability of an operator to intervene in an AI based system functioning in a timely manner to prevent harm or hazard.
Intervenability is a product quality sub-characteristic of security.
Societal and ethical mitigation
Related to 'Quality in use' and is a new characteristic of 'Freedom from risk'
This quality characteristic considers many areas to mitigate both societal and ethical risks such as human centered designs, security, safety and fairness.
AI & Safety
Safety related systems have potential to cause injury or harm to people, property or environment. Developing and testing non-AI safety-related systems can take a lot of effort, but is feasible; however, for AI-based systems, there are several additional challenges:
Specifications
Unlike traditional safety-related systems that have requirements defined for a complete system upfront, requirements for AI-based systems often begin with vague goals and are then implicitly provided via the training data that encodes patterns rules and objectives. All details are not formalized upfront.
As a result, we often have inadequate requirements and implementation traceability.
Non determinism
Non-determinism makes it challenging to guarantee the precise behavior of AI systems. Even rigorously tested models exhibit unexpected behavior due to factors such as random number generation and slight variations in input values.
Self learning
AI based system self-learn and improve with new data. Therefore, systems behavior of AI-based systems often move from originally tested behavior.
- One way to deal with self-learning is managing how a model learns as well as the data it uses can help avoid emergence of new problematic behavior.
- Alternatively, use safety guards to prevent model from learning or making decisions that could compromise safety.
Explainability & Transparency
It is important to understand how and why systems make certain decisions. AI based systems can be complex - they use billions of parameters making it difficult for humans to comprehend. As a result, their decision making process is often not transparent.
AI techniques such as LIME (Local interpretable
model-agnostic explanations) can be used to provide insights into AI-based systems' reasoning. However, they are not widely available and may compromise system performance.
Evolving regulations
The regulatory landscape for safety-related AI-based systems is constantly evolving. In a recent development; The EU AI Act [EU AI Act] classifies AI systems used as safety components (such as in aviation, medical devices, or automotive) as high-risk and imposes strict requirements on their development and testing.
Acceptance Criteria for AI-Based Systems
When evaluating the quality of an AI-based system, it is essential to consider both functional and non functional quality characteristics. This helps confirm that the AI-based system functions as intended and satisfies broader quality requirements.
Top comments (0)