How to Run an AI Risk Assessment Before Deployment

Introduction

Learning how to run an AI risk assessment before deployment is now a basic requirement for any team that ships machine learning into the real world. IBM’s 2025 breach research found that 13 percent of organizations had already suffered a breach of an AI model or application. Of those breached organizations, 97 percent lacked proper AI access controls. Those figures describe companies that launched first and asked structural questions later. A risk assessment reverses that order by forcing the team to name what can go wrong, who gets hurt, and what evidence proves the system is safe enough to release. This guide walks through the full process, from scoping and inventory to testing, documentation, sign-off, and monitoring. You will also find a working scoring script, an interactive risk calculator, and real failures that show what skipped assessments cost. By the end, you will have a repeatable workflow that fits a small startup and still satisfies a regulator’s questions.

Quick Answers on AI Risk Assessment Before Launch

How do you run an AI risk assessment before deployment?

To learn how to run an AI risk assessment before deployment, define the system’s scope, inventory its data, identify harms, score risks, test the model, and require sign-off.

What should an AI risk assessment cover before launch?

An AI risk assessment should cover safety, bias, privacy, security, accuracy, legal exposure, third-party dependencies, and human oversight, each with an owner, a score, and a documented mitigation.

Who should approve an AI system before it goes live?

A cross-functional group should approve the AI system, usually including the product owner, security, legal or compliance, and an independent reviewer who did not build the system.

Key Takeaways

• Treat the assessment as a launch gate with named owners, written evidence, and the authority to delay release.
• Score risks on likelihood and severity, then rank them so scarce testing time goes to the harms that matter most.
• Test for bias, adversarial misuse, privacy leakage, and factual errors before users ever see the system.
• Reassess after every material change to the model, data, prompt, vendor, or intended use.

What Is an AI Risk Assessment Before Deployment?

Knowing how to run an AI risk assessment before deployment means following a structured review that identifies, scores, tests, and documents an AI system’s potential harms, then sets mitigations and approval conditions before real users depend on it.

Why AI Risk Assessment Belongs Before Launch Day

Every AI system carries a gap between how it behaves in a demo and how it behaves with real users, and that gap is where most expensive failures live. Teams usually discover the gap after launch, when a customer screenshot, a regulator letter, or a news story arrives. By then the model is wired into workflows, contracts are signed, and rollback means disrupting people who already depend on the product. Running the assessment first moves the discovery of those failures from public view into a controlled review where fixing them is cheap. It also gives leaders a defensible record showing that reasonable care was taken before release. The broader discipline of managing AI-related risks and challenges starts with exactly this kind of early review.

The evidence that this gap keeps widening continues to accumulate. The Stanford AI Index 2025 reported 233 harmful AI incidents logged in 2024, up from roughly 150 in 2023. Adoption is moving faster than oversight, since McKinsey's state of AI research says nearly nine in ten respondents now use AI regularly in at least one business function. More deployments with the same weak review habits produce more incidents in simple arithmetic. A pre-launch assessment is the cheapest control available against that trend.

There is also a legal argument that executives increasingly recognize. A tribunal in British Columbia held that an airline was responsible for what its own chatbot told a customer. That ruling means a company cannot disown the outputs of a system it chose to deploy. Regulators in hiring, lending, and consumer protection are applying the same logic to automated decisions. If your team cannot show what it tested, what it found, and what it changed, the safest assumption is that the system was released on hope. Knowing how to run an AI risk assessment before deployment replaces hope with evidence.

Mapping the Risk Landscape for Modern AI Systems

Before scoring anything, it helps to hold a clear picture of the risk categories that apply to modern AI. Practitioners usually sort them into safety, fairness, privacy, security, reliability, transparency, and legal exposure. Each category has its own failure modes, its own tests, and its own owners inside the company. A chatbot may carry little discrimination risk yet substantial misinformation and data leakage risk, while a resume screener reverses that profile. The landscape view prevents the common mistake of testing only for the risk the loudest stakeholder happens to fear.

Generative systems also add risk categories that classical predictive models never had. The OWASP Top 10 for LLM Applications lists prompt injection, sensitive information disclosure, supply chain weaknesses, data poisoning, improper output handling, and excessive agency. It also names system prompt leakage, vector weaknesses, misinformation, and unbounded consumption. Those ten items work well as a security checklist because each one maps to a concrete test. Agentic systems that call tools raise the stakes further, since an error becomes an action instead of a sentence. Our guide to AI security risks covers how those attack paths unfold in practice.

Risks also differ by lifecycle stage, and a good assessment follows the system from training data to retirement. Data risks include consent gaps, label errors, and unrepresentative samples that bake bias into the model. Model risks include overfitting, brittleness under distribution shift, and behaviors nobody intended. Deployment risks include weak access controls, poor logging, and users who apply the tool to jobs it was never built for. Operational risks include silent drift, vendor changes, and staff who stop questioning outputs they have learned to trust.

Finally, the landscape includes the people on the receiving end, who are often not the people paying for the system. A loan applicant never chose the underwriting model, and a job candidate never saw the screening rules. Mapping affected groups early keeps the assessment honest about who bears the harm when the model is wrong. It also reveals which risks are reversible, such as a bad product recommendation, and which are not, such as a wrongful denial of benefits. Reversibility becomes a major input when severity is scored later in the process.

Choosing Between NIST, ISO, and EU Frameworks

Building on that landscape view, the next decision is which framework gives your assessment its structure. Three references dominate the conversation: the NIST AI Risk Management Framework, the ISO/IEC 42001 management system standard, and the EU AI Act. They are not competitors in the usual sense, because each answers a different question. NIST describes how to organize risk work, ISO describes how to run a certifiable management system, and the EU law describes what you must legally do in each risk tier. Most mature programs borrow from all three, and the combination covers organization, evidence, and legal obligation.

The NIST AI Risk Management Framework was released on January 26, 2023, and organizes work into four functions named Govern, Map, Measure, and Manage. Govern covers policies and accountability, Map frames context and impacts, Measure analyzes and tests, and Manage prioritizes and treats risks. NIST later published a Generative AI Profile on July 26, 2024, which adapts the framework to generative systems. The framework is voluntary, which makes it a flexible backbone for teams that have no regulator dictating the format. Its vocabulary also appears in many customer security questionnaires, so adopting it saves explanation time.

The EU AI Act sets binding obligations, and its penalties are large enough to change budget conversations. Article 99 allows fines of up to 35 million euros or 7 percent of worldwide annual turnover for prohibited practices. Most other operator obligations carry fines of up to 15 million euros or 3 percent. Timing has shifted as well, because a provisional omnibus agreement would move standalone high-risk obligations to December 2, 2027 and product-embedded ones to August 2, 2028. Treat those dates as planning inputs rather than permission to wait, because the assessment work takes months to mature. Our overview of AI governance trends and regulations tracks how these rules are evolving.

In practice, a sensible small-team pattern is to use NIST's four functions as the outline. Borrow the management-system discipline of ISO 42001 for records and audits. Then use the EU risk tiers to classify each system you plan to release. State laws add another layer, and the Colorado AI Act compliance guide shows how a single US state can impose impact assessment duties. Do not let framework selection become a months-long debate over which standard is best. Pick one backbone, document the choice, and revisit it when your regulatory exposure changes.

Scoping the System and Its Intended Use

Shifting from frameworks to hands-on work, the first concrete step is a written scope statement for the system. The statement names the business purpose, the decision or output the AI produces, the users, and the people affected by the output. It also names what the system must never be used for, because misuse outside the intended context drives a large share of incidents. A scope written in two pages beats a scope held in five people's heads. Every later score in the assessment depends on this document being specific.

Good scoping separates the model from the product around it. The same language model can power a harmless internal summarizer or a customer-facing triage tool. The risk profile of those two products differs enormously even though the model underneath is identical. Record the autonomy level as well, noting whether a human reviews each output, samples a few, or never sees them. Record the deployment context, including geography, languages, regulated industries, and whether minors might use the tool. These details determine which laws, controls, and tests apply to the system.

Building a Complete Inventory of Data, Models, and Dependencies

Moving on from scope, you need a full inventory of everything the system touches. Start with data: training sets, fine-tuning data, retrieval corpora, prompt templates, user inputs, and logs. For each source record where it came from, the legal basis for using it, how sensitive it is, and who can access it. Teams are often surprised to find personal data in a retrieval index or customer text in a log file nobody planned to keep. An inventory turns those surprises into line items that can be fixed before launch.

Next, list every model and component, including the ones you did not build. That means base models, embedding models, rerankers, safety filters, open-source libraries, plugins, and tool integrations. Note the version of each, the supplier, the license, and how updates reach production. A silent model update from a vendor can change behavior overnight, so the inventory should record who is notified and who can pin versions. Our AI data readiness assessment framework offers a deeper checklist for the data half of this work.

Finally, map the dependencies around the system, including infrastructure, identity providers, downstream applications that consume its outputs, and the people who operate it. Draw the data flows so you can see where sensitive information crosses a trust boundary, such as leaving your network for a third-party API. This diagram becomes the foundation for security review, privacy review, and incident response planning. If the team cannot draw the diagram, that gap is itself a finding. Store the inventory in a place where it will be updated, because a stale inventory is worse than none.

Identifying Ethical Harms and the People They Reach

Turning to the people at risk, the assessment should list concrete harms rather than abstract worries about ethics. A harm is a specific adverse outcome for a specific group, such as a qualified applicant rejected for age or a patient given unsafe advice. Write each harm as a sentence that names who is affected, what happens to them, and how they would find out. This format exposes harms that stay invisible in generic risk lists, especially those that fall on people who never interact with your product directly. It also makes the later scoring step far more grounded.

Run structured workshops to generate the list, and include people who do not build the system. Customer support staff, frontline operators, compliance officers, and members of affected communities will name harms engineers overlook. Prompt each group with the same questions: who could be treated unfairly, who could be deceived, who could lose money, and who could be physically or psychologically hurt. Ask also who could be excluded entirely, since inaccessible or unsupported languages often lock people out silently. Capture every answer before debating which ones are realistic, because early filtering hides the uncomfortable cases.

Ethical review works best when it asks what the system does to people rather than what it intends to do. Intentions are easy to defend, while outcomes are what regulators, courts, and journalists measure. Consider consent and transparency, meaning whether people know they are dealing with an AI and whether they can contest a decision. Consider autonomy, meaning whether the system nudges or manipulates users beyond what they would accept if they saw the mechanism. Consider dignity and proportionality, meaning whether the level of automation matches the seriousness of the decision being made about a person. The responsible AI policy template on our site gives a starting set of principles to adapt.

Finally, classify each harm by reversibility and by who can remedy it. A wrong movie recommendation costs the user a few seconds, while a wrongful benefits denial can cost a family its housing. Harms that are hard to detect, hard to appeal, or hard to reverse deserve extra weight in the scoring step. Record also whether the affected person has any realistic path to complain, because a harm without a complaint channel can persist for years. This classification gives you the severity inputs you need for the next stage of the process.

Scoring Likelihood and Severity Without False Precision

From there, the team can rank the harms with a simple likelihood and severity matrix. Rate likelihood on a five-point scale from rare to almost certain, and rate severity on a five-point scale from negligible to catastrophic. Multiply the two numbers to produce a score between 1 and 25, then group scores into low, medium, high, and critical bands. A scoring scheme only needs to be consistent enough to rank risks, not accurate enough to predict them. Teams that chase decimal precision waste weeks on numbers that carry no real information.

Calibrate the scale with written anchors so two reviewers give the same rating to the same risk. For example, define severity four as harm to many people or a regulatory violation that would trigger an investigation. Define likelihood four as something you expect to see within the first month of operation based on testing. When there is no historical data, use expert judgment, red team findings, and incident records from similar systems. Document the reasoning behind each score in a sentence, since the reasoning is what auditors and later reviewers actually read.

Testing for Bias, Fairness, and Disparate Impact

Beyond scoring, the assessment needs direct evidence about how the model treats different groups. Start by defining the groups that matter for your use case, using legal categories as the minimum and adding context-specific ones. Gather or construct test data that represents each group with enough samples to measure meaningful differences. Choose fairness metrics that fit the decision, such as selection rate ratios for hiring or error rate gaps for risk prediction. No single metric captures fairness, and some well-known metrics cannot all be satisfied together, so the team must choose and justify.

Next, run the tests on realistic inputs and not only on clean benchmarks. Check intersectional groups, since a model can look fair for women and fair for older applicants while failing older women. Look at proxy variables like postal codes, school names, and writing style, which can encode protected traits even when those traits are removed. The AI bias audit guide walks through concrete testing methods and metric choices. Save the code, data version, and results, so the same test can be repeated after every model change.

Finally, decide what to do with the findings before you see them. Set thresholds in advance, such as a minimum selection rate ratio, so the decision is principled rather than negotiated under launch pressure. Mitigation options include rebalancing data, adjusting thresholds, adding human review for borderline cases, or narrowing the scope of the system. Sometimes the right answer is not to automate the decision at all. Record that outcome as a legitimate result of the assessment and not a failure of it.

Red Teaming and Adversarial Testing Before Launch

Stepping back from fairness metrics, the assessment must also test how the system behaves when someone is actively trying to break it. Red teaming means assigning skilled people to attack the system the way a malicious user, a careless user, or a curious teenager might. The goal is to find failures that normal testing never triggers, including jailbreaks, prompt injection, data extraction, and harmful content generation. Our overview of red teaming AI for safer models describes the main techniques and team structures. A small internal team with clear scope can find serious issues in a few days.

Design the exercise around the threats that match your scope statement. For a customer chatbot, test attempts to extract system prompts, trick the bot into making policy promises, and pull other customers' data. For an agent with tool access, test whether injected instructions in a web page or email can trigger an unauthorized action. Use the OWASP list as a starting scenario set, then add scenarios from your own harm workshop. Log every attempt with the exact input, the output, and a severity rating, because patterns across attempts often reveal one underlying weakness.

Treat red team findings as inputs to the risk register, not as a separate report that gets filed and forgotten. Each finding gets an owner, a mitigation, and a retest date. Retest after fixes, since patches for one attack frequently open another path. Do not launch with an open critical finding unless a named executive has accepted the risk in writing. That sentence belongs in your go-live policy, and the rest of this guide shows where it fits.

Privacy, Security, and Data Protection Checks

On top of adversarial testing, a pre-launch review must confirm that personal and confidential data are handled correctly end to end. Start from the data flow diagram built during the inventory step, and mark every point where personal data enters, moves, or leaves. Verify the legal basis for each use, the retention period, and the deletion process. Check whether prompts and outputs are logged, who can read the logs, and whether the vendor trains on your inputs. Many privacy failures come from logging and retention defaults nobody reviewed.

Security checks should cover identity, access, secrets, and the integration surface. Confirm that the model and its tools run with least privilege, so a compromised session cannot reach unrelated systems. Review how credentials are stored and rotated, and how third-party tool servers are authenticated, which our guide to MCP server security risks explains in detail. Remember the sobering IBM finding that 97 percent of breached AI organizations lacked proper access controls. Access control is the cheapest safeguard on the list, and it is the one most often skipped.

Evaluating Hallucination, Accuracy, and Reliability

Given the security and privacy checks above, the next test question is simply whether the system gives correct answers often enough to be useful and safe. For generative models, that means measuring hallucination, the tendency to state invented facts with confidence. Build an evaluation set from real questions your users will ask, with reference answers checked by a subject matter expert. Include hard cases, such as questions with no answer in your documents, ambiguous requests, and attempts to push the model beyond its knowledge. The size of the set matters less than its resemblance to production traffic.

Choose metrics that match the specific failure you fear most. For retrieval systems, measure whether answers are grounded in the retrieved passages and whether the right passages are retrieved at all. For classification systems, measure precision and recall separately, since the cost of a false positive rarely equals the cost of a false negative. Scaled evaluation often uses a second model as a grader, and our guide to evaluating outputs with LLM-as-a-judge explains how to calibrate that grader against human labels. Never accept a single aggregate accuracy number without reading the actual failures behind it. The failures tell you whether errors are random annoyances or systematic dangers.

Pair measurement with mitigation, since a measured problem needs a plan. Practical tools include retrieval grounding, citation requirements, confidence thresholds with fallback to a human, and refusal behavior for out-of-scope questions. Our walkthrough on reducing LLM hallucinations in production compares these approaches with their trade-offs. Set an acceptance threshold before testing, such as a maximum rate of unsupported claims on the evaluation set. Reliability also includes latency, availability, and behavior under load, so run those tests in the same cycle.

Third-Party Models and Vendor Due Diligence

Moving on to suppliers, most teams now build on models and platforms they do not control, which makes vendor review part of the assessment. Ask each vendor how the model was trained, what data it retains from your inputs, and whether you can opt out of training use. Request security attestations, penetration test summaries, and their incident notification commitments. Ask what happens when the vendor deprecates a model version, and how much notice you receive. Vendors who cannot answer plainly are telling you something about the risk you would inherit.

Contracts should translate those vendor answers into clear and enforceable obligations. Look for clauses on data processing, subprocessors, breach notification timing, audit rights, and indemnification for intellectual property claims. Check whether the vendor commits to model change notices, because an unannounced update can invalidate your test results. Our article on evaluating AI vendor partnerships offers a practical scorecard for this review. Keep a fallback plan, such as a second approved model, so a vendor failure does not become your outage.

Human Oversight and Escalation Design

Looking at oversight next, a risk assessment must show how people stay in control when the system errs. Define for each decision type whether a human approves outputs, reviews a sample, or only handles exceptions. Match the oversight level to the harm severity identified earlier, so high-severity decisions get a human check before they take effect. Write down who the reviewer is, what information they see, and how much time they realistically have. A reviewer who has two seconds per case is a rubber stamp and not a safeguard.

Design escalation paths for the moments when the system behaves unexpectedly. Users should have a visible way to report errors, request a human, and contest an automated decision. Internal staff should have a kill switch with a named owner who is allowed to use it without asking for permission. For agents that act on systems, add hard limits such as spending caps and allowlisted tools. Add confirmation steps for irreversible actions as well, which our piece on deterministic guardrails for AI agents explains. Oversight that exists only on an org chart will fail the first time a real incident arrives at three in the morning. Rehearse the escalation path before launch, and fix the gaps the rehearsal reveals.

Training and role clarity complete the oversight picture for most teams. People who supervise AI need to understand its typical failure modes, its confidence signals, and the situations where they must override it. The responsibilities described in our guide to the agent supervisor role show how organizations are formalizing this job. Explainability helps too, since reviewers decide faster and better when the system shows its reasoning, as covered in our explainer on SHAP and LIME methods. Measure the override rate over time, because a rate near zero may signal blind trust rather than perfect performance.

Showing Auditors Your Work Through Documentation

Rounding out the technical work, every finding and decision needs a written record that someone outside the team can understand. A useful package includes the scope statement, the data and model inventory, the risk register, the test plans and results, the mitigation log, and the approval record. Each document carries a date, an author, and the version of the system it describes. Auditors and regulators ask for exactly these items, and so do enterprise customers during procurement. Teams that write the records while doing the work spend a fraction of the time compared to those reconstructing them later.

Write for a reader who was not in the room. Explain why a risk was rated medium and not high, why a test threshold was chosen, and why an accepted risk was acceptable. Link every claim to the evidence behind it, such as a test report or a log extract. A decision without a recorded rationale is indistinguishable from a decision nobody made. Store the package in a controlled repository with change history, and assign an owner who keeps it current.

Go-Live Gates and Approval Workflows

With that evidence in hand, the assessment ends in a formal decision about whether the system may launch. Define the gate criteria in advance: no open critical risks, all high risks mitigated or formally accepted, required tests passed at their thresholds, and documentation complete. Define who votes, who can veto, and who can accept residual risk. A cross-functional group, often organized as an AI governance board, gives the decision legitimacy, and our guide on setting up an AI governance board covers membership and charters. Independence matters, so at least one approver should not report to the product owner.

Staged rollouts reduce the stakes of the launch decision considerably. Release first to internal users, then to a small percentage of customers, then more broadly, with predefined metrics that must hold at each stage. Keep the ability to roll back quickly, and test the rollback during the assessment and not during an emergency. Communicate limits to users honestly, including that the system can make mistakes and how to get help. Treat the approval as conditional, with explicit triggers that force a fresh review.

Record the outcome of the review whatever it turns out to be. A decision to delay launch is a success of the process, not a failure of the team. Capture the conditions attached to approval, such as monitoring requirements and review dates. Share a short summary with executives so accepted risks are visible at the level that owns them. This summary also becomes the baseline against which later changes are compared.

Monitoring, Incidents, and Continuous Reassessment

Next, remember that a pre-deployment assessment is a snapshot, and the system keeps changing after you ship it. Monitor the metrics that tie directly to your top risks, such as error rates by group, refusal rates, escalation volumes, and complaint themes. Watch for drift in inputs, since users often find new ways to use a tool. Set alert thresholds and name the person who responds when one fires. Monitoring that nobody reads is nothing more than a storage cost.

Prepare a written incident process well before the day you need one. Define what counts as an incident, who declares it, how affected users are notified, and when regulators must be told. Keep logs sufficient to reconstruct what the system did and why, while respecting privacy limits. After each incident, run a blameless review and feed the lessons back into the risk register. Triggers for reassessment should be explicit: a new model version, a new data source, a new user population, a changed vendor term, or a significant incident.

Common Mistakes That Undermine Risk Assessments

Despite the best intentions, teams repeat a short list of mistakes that hollow out their assessments. The first is treating the exercise as paperwork completed after the decision to launch has already been made. The second is letting the builders grade their own work, which removes the independent view that catches blind spots. The third is testing only the happy path, so the evaluation set looks nothing like real traffic. The fourth is scoring risks once and never revisiting them as the system, data, and users evolve.

Other mistakes are quieter and therefore harder to notice in review meetings. Shadow AI, meaning employees using unapproved tools with company data, sits outside most formal assessments, and IBM's 2025 findings tied high shadow AI use to $670,000 in added breach costs. Our explainer on the 2025 shadow AI threat shows how to bring those tools into scope. Teams also underestimate vendor risk, assuming a reputable supplier has already handled safety on their behalf. Another frequent error is writing mitigations without owners or dates, which turns a risk register into a wish list.

The most damaging mistake is allowing schedule pressure to override the gate criteria. When a launch date is fixed publicly, every finding starts to look negotiable. Protect the process by agreeing on blocking criteria before the schedule gets tight, and by giving the approvers real authority. Make the cost of skipping visible by reviewing past incidents with leadership. Remember that the chatbot, hiring, and pricing cases discussed later in this guide all involved teams who were confident in their launch decision.

The Future of AI Risk Assessment Before Launch

Looking ahead, assessment practice is moving from one-time reviews toward continuous, partly automated programs. Evaluation suites will run on every model or prompt change, in the same way unit tests run on every code commit. Agentic systems will push the field to assess chains of actions and not single outputs, since risk compounds across steps. Benchmarks are also maturing, and our report on a new AI risk assessment benchmark shows how standardized measurement is beginning to emerge. Expect customers to request evidence of these programs during procurement, as they already do for security certifications.

Regulation will keep shifting over the coming years, but the direction of travel is consistent. Even where deadlines move, the underlying expectations of documented risk management, human oversight, and transparency are staying in place. Organizations that build the assessment muscle now will adapt to new rules by updating mappings, while others will be starting from scratch under deadline pressure. Treat the process in this guide as infrastructure that you improve each quarter. The teams that learn how to run an AI risk assessment before deployment today will ship faster and with fewer surprises tomorrow.

How to Run an AI Risk Assessment Before Deployment: Seven Implementation Steps

The sections above explain the concepts, and this walkthrough turns them into a sequence your team can execute in a few weeks. Each step produces a specific artifact that feeds the next one, so nothing is wasted. You can run the steps with a spreadsheet, but the code samples below show how to keep the register in version control. Adapt the field names to your own tooling and review cadence. Small teams can finish steps one through four in a single workshop week. The remaining steps depend on how much testing your risk scores demand.

Step 1 - Write the scope statement

Open a document in your repository and write the scope statement before anything else. Name the system, the business purpose, the output it produces, the autonomy level, and the people affected by that output. Add a section titled out-of-scope uses so reviewers can see which misuse you expect and forbid. Keep it under 2 pages, because long scopes hide the choices that matter. Pro tip: have someone outside the project read the scope and explain it back to you. If their explanation differs from your intent, the scope is not yet clear enough.

system: support-assistant-v2
purpose: Answer billing questions for existing customers
output: Free-text answers shown in the web chat widget
autonomy: human-sampled
affected_people:
  - customers
  - support agents
out_of_scope:
  - legal or medical advice
  - changing account data
  - use with customers under 18
owner: product-lead
reviewers:
  - security
  - compliance

Step 2 - Inventory data, models, and dependencies

Create an inventory file that lists every data source, model, library, and external service the system touches. For each entry record the supplier, the version, the sensitivity level, and the person who can answer questions about it. Include retrieval indexes, prompt templates, and logs, since these hold personal data more often than teams expect. Draw 1 diagram showing where data crosses a trust boundary, and attach it to the file. Review the inventory against actual network traffic if you can, because documentation and reality frequently differ. Every unexplained connection is a finding to add to the register.

data_sources:
  - name: billing-faq-index
    sensitivity: internal
    contains_personal_data: false
  - name: chat-transcripts
    sensitivity: confidential
    contains_personal_data: true
    retention_days: 30
models:
  - name: base-llm
    supplier: external-vendor
    version: pinned-2026-09
    trains_on_our_inputs: false
dependencies:
  - identity-provider
  - ticketing-api

Step 3 - List harms and affected groups

Run a 2-hour workshop with product, engineering, legal, support, and at least one person who represents users. Ask each participant to write harms as full sentences naming who is affected, what happens, and how they would find out. Group duplicates, then add the reversibility and detectability notes described earlier. Seed the discussion with the OWASP categories and your own incident history to avoid blank-page paralysis. Aim for 20 to 40 distinct harms for a typical system. Do not filter harms during the workshop, because filtering is how uncomfortable but real risks disappear.

Afterward, assign each harm an owner and a draft likelihood and severity rating. Owners should be the people with the authority to fix the problem, not the people with the least workload. Capture the reasoning in one sentence per rating so the numbers can be defended later. Tag each harm with the risk category and the framework function it belongs to. This tagging lets you generate NIST-aligned reports without extra work.

Step 4 - Score and rank the risks

Load the harms into a register and compute the score as likelihood multiplied by severity. A short script keeps the arithmetic consistent and gives you a ranked list that anyone can regenerate. The sample below defines the bands used in this guide, where scores of twenty or higher are critical. Store the register file in version control, so each change has an author and a timestamp. Rerun the script whenever any score changes so the ranking stays current. Use a five-point scale for each factor so products range from 1 to 25, which separates the risks without implying false precision. Review the top 10 entries together before anyone debates the lower ones.

import csv
from dataclasses import dataclass


@dataclass
class Risk:
    risk_id: str
    description: str
    likelihood: int  # 1 (rare) to 5 (almost certain)
    severity: int    # 1 (negligible) to 5 (catastrophic)
    owner: str
    status: str = "open"

    @property
    def score(self) -> int:
        return self.likelihood * self.severity

    @property
    def band(self) -> str:
        if self.score >= 20:
            return "critical"
        if self.score >= 12:
            return "high"
        if self.score >= 6:
            return "medium"
        return "low"


def load_register(path: str) -> list[Risk]:
    with open(path, newline="") as handle:
        rows = csv.DictReader(handle)
        return [
            Risk(
                r["risk_id"],
                r["description"],
                int(r["likelihood"]),
                int(r["severity"]),
                r["owner"],
                r.get("status", "open"),
            )
            for r in rows
        ]


if __name__ == "__main__":
    register = sorted(load_register("risk_register.csv"), key=lambda r: -r.score)
    for risk in register:
        print(f"{risk.risk_id:8} {risk.score:3} {risk.band:9} {risk.owner}")

Step 5 - Run the pre-launch tests

Translate the top-ranked risks into concrete tests, one or more per risk, with pass thresholds written before the tests run. Typical suites include a bias comparison across groups, a red team session against the OWASP scenarios, a hallucination evaluation on real questions, and a privacy leakage probe. Assign each test an owner and a date, and store the inputs and outputs so results are reproducible. Run the suite in an environment that matches production, including the same model version and system prompt. Freeze the model version and prompt hash during testing, or your results will not describe the system you ship. Write at least 3 tests for every risk rated critical.

The configuration below shows one way to declare the suite so it can run automatically on every change. Thresholds belong in the file, not in a meeting where they can be negotiated. When a test fails, link the failure to its register entry and update the status. Retest after each fix to confirm the mitigation actually worked. Keep the failed runs, because they document that the process worked.

{
  "suite": "pre-launch-v1",
  "model_version": "pinned-2026-09",
  "tests": [
    {"id": "T1", "risk": "R-004", "type": "bias", "metric": "selection_rate_ratio", "min": 0.8},
    {"id": "T2", "risk": "R-007", "type": "red_team", "metric": "successful_injections", "max": 0},
    {"id": "T3", "risk": "R-011", "type": "hallucination", "metric": "unsupported_claim_rate", "max": 0.03},
    {"id": "T4", "risk": "R-013", "type": "privacy", "metric": "pii_leaks", "max": 0}
  ]
}

Step 6 - Document mitigations and residual risk

For every risk rated medium or above, record the mitigation, the owner, the due date, and the residual score after the mitigation is in place. Mitigations fall into four groups: remove the risk by narrowing scope, reduce it with controls, transfer it through contract or insurance, or accept it explicitly. Acceptance needs a name, a date, and a reason, and critical risks need executive acceptance. Link each mitigation to the test that proves it works. Avoid mitigations that rely only on user warnings, because warnings rarely change behavior. Set a due date no more than 30 days out for any mitigation tied to a critical risk.

Then assemble the full documentation package described earlier in this guide. It should contain the scope, the inventory, the register, the test results, the mitigation log, and a one-page summary. Date and version every file against the system release it covers. Give reviewers a single index page so they can find evidence quickly. A tidy package signals a disciplined process and speeds up customer security reviews.

Step 7 - Hold the go-live review and set monitoring

Convene the approval group with the package circulated at least 2 days ahead. Walk through the top risks, the test results, the open items, and the residual scores. Use an explicit gate check, like the script below, so the decision criteria are applied the same way every time. Plan a 60 minute meeting with a written agenda so the review stays focused on evidence. Record the decision, the conditions, and the date of the next review. Then configure the monitoring and alert rules described earlier, and confirm the on-call owner knows the escalation path.

def gate_check(register, tests_passed, docs_complete):
    """Return (approved, reasons) for the go-live review."""
    reasons = []
    for risk in register:
        if risk.status == "open" and risk.band == "critical":
            reasons.append(f"Open critical risk {risk.risk_id}")
        if risk.status == "open" and risk.band == "high" and not risk.owner:
            reasons.append(f"High risk {risk.risk_id} has no owner")
    if not tests_passed:
        reasons.append("Required pre-launch tests have not all passed")
    if not docs_complete:
        reasons.append("Documentation package is incomplete")
    return (len(reasons) == 0, reasons)

Treat a blocked gate as useful information instead of an obstacle. The reasons list tells the team exactly what work remains before approval. Rerun the check after each fix, and attach the final passing output to the approval record. If leadership wants to override a block, require a written acceptance from the executive who owns the risk. That friction is intentional, and it is what makes the gate meaningful.

Key Insights

• IBM's 2025 research found that 13 percent of organizations reported breaches of AI models or applications, and 97 percent of those breached lacked proper AI access controls.
• High shadow AI use added $670,000 to the average breach cost in IBM's 2025 data, which shows that unapproved tools belong inside every assessment scope.
• The AI Incident Database logged 233 harmful incidents in 2024, up from about 150 in 2023, so pre-launch testing now faces a steadily rising baseline of real failures.
• Article 99 of the EU AI Act allows fines up to 35 million euros or 7 percent of worldwide turnover, which makes a skipped risk review a board-level financial exposure.
• A provisional omnibus agreement would move standalone high-risk obligations to December 2, 2027, a longer runway that still requires the assessment evidence to be built well before then.
• The EEOC's first AI hiring settlement cost iTutorGroup $365,000 after software allegedly rejected about 200 older applicants, proving that automated screening carries ordinary employment law liability.
• Zillow wrote down $304 million in inventory and cut roughly 25 percent of its workforce after its pricing forecasts proved unreliable, showing how model error compounds into balance sheet losses.
• McKinsey reports that nearly nine in ten respondents use AI regularly in at least one business function, so oversight capacity has to scale at the same pace as adoption.

Taken together, these figures describe a market where adoption is nearly universal while governance and access control still lag behind. The breach and incident data show that failures are already common, and they cluster around missing controls rather than exotic attacks. Legal exposure has become concrete, with tribunals, regulators, and enforcement agencies treating automated outputs as the responsibility of the organization that deployed them. The financial cases show that the biggest losses arrive when a model's errors reach a balance sheet before anyone has stress tested them. A disciplined assessment before launch is therefore the lowest-cost control against every one of these patterns. Regulatory dates may shift, but the evidence requirements behind them are unlikely to disappear.

Dimension	NIST AI RMF	ISO/IEC 42001	EU AI Act
Nature	Voluntary risk management framework	Certifiable management system standard	Binding regulation
Primary focus	Organizing risk work across the AI lifecycle	Running a governed, auditable AI program	Protecting health, safety, and fundamental rights
Structure	Govern, Map, Measure, Manage	Policies, roles, controls, and continual improvement	Risk tiers with duties for each operator role
Evidence expected	Documented context, measurements, and treatments	Records, internal audits, and management review	Technical documentation, logging, and conformity steps
Certification	None; self-attested use	Third-party certification available	Conformity assessment for high-risk systems
Penalties	None directly	None directly; contractual consequences	Fines up to 35 million euros or 7 percent of turnover
Best fit	Teams needing a flexible backbone	Organizations wanting audit-ready proof	Anyone placing AI on the EU market
Timing	Available now; generative profile released July 2024	Available now	Phased dates; high-risk deadlines proposed for 2027 and 2028

Examples of Real Deployments That Skipped the Assessment

Air Canada's Bereavement Fare Chatbot

Air Canada deployed a website chatbot to answer customer questions about fares and policies. In November 2022 the bot told a passenger he could claim a bereavement discount after travel. That advice contradicted the airline's own policy page, which said such requests could not be made once the trip was complete. The British Columbia tribunal decision ordered Air Canada to pay a total of $812.02 CAD and rejected the argument that the airline was not responsible for its chatbot's statements. A simple test that compared chatbot answers against the published policy pages would likely have exposed the contradiction within hours. The financial amount was small, but the principle that companies own their chatbot's output is the lasting consequence. The ruling came from a small claims tribunal, so it is not binding on other courts, though the reasoning is still widely cited.

iTutorGroup's Automated Applicant Screening

iTutorGroup used application software that, according to the EEOC, automatically rejected female applicants aged 55 or older and male applicants aged 60 or older. The agency's May 2022 complaint said about 200 qualified applicants in the United States were turned away because of their age. In 2023 the company agreed to a $365,000 settlement split evenly between compensatory damages and back pay. The agreement also required a four-hour anti-discrimination training for everyone who screens or hires tutors, plus reporting and monitoring duties. A basic fairness test comparing acceptance rates by age would have caught the rule in minutes. The company denied the allegations, and the settlement still needed court approval, which is a reminder that these outcomes are negotiated rather than adjudicated.

New York City's MyCity Business Chatbot

New York City rolled out the MyCity chatbot in fall 2023 to help business owners check municipal rules and regulations. Testing by journalists in 2024 found it told users that employers could take a cut of workers' tips. It also said landlords could discriminate by income source and that businesses could refuse cash. Reporting by The Markup says the bot cost around half a million dollars. The city took it offline on February 4, 2026, and the new mayor cited the shutdown as a budget reduction. A red team exercise using real municipal law questions would have exposed these answers before launch. The problem persisted publicly for a long time after the first reports, which shows that fixing known failures is a governance question as much as a technical one.

Costly Lessons From Systems That Shipped Too Soon

Case Study: Zillow Offers and the Pricing Algorithm

Zillow wanted to buy homes at scale and resell them quickly, and the problem it needed to solve was predicting future sale prices accurately enough to earn a margin. The company built its Zillow Offers business around algorithmic forecasts of home values and began purchasing homes at scale. When the housing market moved in ways the forecasts missed, Zillow found itself holding homes it had bought for more than it could resell them. In November 2021 the company announced it would wind down the business and take a write-down of approximately $304 million in inventory. The decision also reduced the workforce by about 25 percent, and Zillow reported a third quarter loss of $328 million on $1.7 billion in revenue.

Executives said the unpredictability of home price forecasting far exceeded what they had anticipated. They also expected further losses of $240 million to $265 million from homes bought in the fourth quarter. The episode raised lasting questions about whether the models were validated against rapid market shifts and whether leadership paused purchases soon enough. A pre-launch assessment would have stress tested the forecasting error under rising and falling markets and set hard limits on exposure. It would also have defined a pause trigger so that losses were capped by policy rather than discovered in quarterly results. The lesson is that model error is a balance sheet risk, and risk scoring should include the financial exposure that a wrong prediction can create at scale.

Case Study: Amazon's Experimental Recruiting Engine

Amazon faced the problem of sorting an enormous volume of resumes and wanted a tool that could surface the best candidates automatically. Starting in 2014, a team built computer models that rated candidates from one to five stars. The effort produced roughly 500 models that each learned about 50,000 terms from resumes submitted over a ten-year period. Because most of those resumes came from men, the system learned to prefer male-coded language and penalized resumes containing the word women's. According to reporting on the project, the company realized by 2015 that the engine was not rating candidates in a gender-neutral way. Engineers edited the programs to ignore the flagged terms, and the team was disbanded by the start of 2017.

Edits to neutralize specific words did not increase confidence that the model would not discover other discriminatory patterns, which is the central limitation of patching bias after training. Recruiters reportedly never relied solely on the rankings, so the tool did not make final decisions on its own. Amazon later kept only a heavily watered-down version for tasks such as removing duplicate candidate profiles. The case remains a standard example of why representative training data, subgroup testing, and independent review belong in the assessment before a tool influences hiring. It also illustrates the legitimate outcome of a good process, because the company chose to stop and not to ship a system it could not trust.

Case Study: The Dutch Tax Authority's Risk Model

The Dutch Tax and Customs Administration faced the problem of detecting fraud in childcare benefit claims. It relied on a risk classification process to flag applications for closer scrutiny. As part of that solution, the agency used applicants' dual nationality as a risk indicator. That choice meant some people were treated as more suspect because of their background. The Dutch data protection authority later found the processing unlawful and discriminatory, and it issued a record fine of 2.75 million euros. Regulators said the data should have been deleted in 2014, yet nationality information on 1.4 million Dutch citizens was still registered in 2018.

The agency stopped using the data in its risk model in October 2018 and says it had fully deleted it by mid-2020. The regulator set aside its normal penalty policy, which usually caps such fines at one million euros per breach, because of the scale of the violation. The wider benefits scandal remains politically contested and raised lasting concern about algorithmic decision-making in public administration. For an assessment team, the lesson is that sensitive attributes and their proxies need explicit review. Each one requires a lawful basis, a retention limit, and an appeal path for affected people. A harm workshop that included affected communities would likely have surfaced the discrimination risk long before a regulator did.

Common Questions About Running an AI Risk Assessment Before Launch