Checklist · Technology

Challenging an Automated Decision and Auditing an AI System: A Practical Checklist

15 U.S.C. § 1691

Type
Checklist
Jurisdiction
National
Date
2024-11-26
Topic
Technology

This checklist serves the two sides of an automated decision. The first half is for the person on the receiving end. Get the adverse action notice, demand specific reasons when the notice is generic, and identify the data source it must name. Obtain the consumer file and dispute the errors that drive a large share of adverse outcomes. Request human review in writing, opt out of profiling where a state statute allows, and complain to the right regulator before the deadlines run. The second half is for the organization deploying the system. Build a model inventory, classify decisions by regulated domain, and justify every feature in writing. Test outcomes at the operating threshold, and conduct and document the search for a less discriminatory alternative that decides step three of the disparate impact framework. Validate that individualized adverse action reasons can be generated, build the human review and accommodation paths, contract properly with vendors, monitor in production, and retain the artifacts.

Challenging an Automated Decision and Auditing an AI System: A Practical Checklist

Technology > Automated Decisions and Artificial Intelligence | Published 26 November 2024 - Updated 6 July 2026 | UpLaw - uplaw.us

⭐ Claimants: the notice, the file, and the dispute right resolve more of these matters than any argument about the algorithm. ⚠ Deployers: the single most consequential omission is the undocumented search for a less discriminatory alternative.

Use with Automated Decisions and Artificial Intelligence: A Comprehensive Guide to Disclosure, Bias, and Accountability and The Automated Decision and AI Accountability Toolkit.

PART ONE — CHALLENGING A DECISION

Phase 1 — Get the notice

☐ Confirm the decision is an adverse action: a denial · a material change in terms · a revocation · a reduction · offering credit on terms materially less favorable than requested · a hiring or tenancy rejection.

☐ Demand the notice in writing if you did not receive one.

☐ ⭐ Read the reasons. ⚠ "You did not score well on our system" is NOT compliant. Under 15 U.S.C. § 1691 and 12 C.F.R. Part 1002, a creditor must state the specific principal reasons or disclose the right to obtain them.

☐ Request specific reasons in writing, citing the statute and the regulation, and give a deadline.

☐ Identify the data source — 15 U.S.C. § 1681m requires the notice to name the consumer reporting agency, state that the agency did not make the decision, and disclose your right to a free file copy and to dispute.

☐ For employment decisions, confirm the two-step process was followed: pre-adverse action notice with a copy of the report and a summary of rights, a reasonable waiting period, then the final notice. ⚠ Skipping the waiting period is a common and independently actionable violation.

☐ For a government determination, demand the specific factual basis. ⚠ A notice that says only "you are no longer eligible" is constitutionally inadequate, and automated systems produce exactly that.

Phase 2 — Get and fix the data

☐ ⭐ Request the full file under 15 U.S.C. § 1681g — free within sixty days of an adverse action. ☐ Request it from the specific agency named, not only the three nationwide bureaus. ⚠ Tenant screening and employment screening companies are separate agencies with separate files. ☐ Read for the errors that actually appear:

  • ☐ ⭐ A mixed file — someone else's information merged into yours
  • ☐ A judgment or eviction belonging to a person with a similar name
  • ☐ A satisfied debt still reported as open
  • ☐ An account that was never yours
  • ☐ A discharged debt reported as owing
  • ☐ Outdated information beyond the reporting period
  • ☐ A criminal record that was expunged, dismissed, or belongs to someone else ☐ Dispute in writing under 15 U.S.C. § 1681i, with documentation, ⭐ to both the agency and the furnisher — furnisher duties are at 15 U.S.C. § 1681s-2. ☐ Note the reinvestigation deadline, and that unverifiable items must be deleted. ☐ Request that corrected reports be sent to anyone who received the report in the relevant period. ☐ Check permissible purpose — 15 U.S.C. § 1681b. ⚠ A report pulled without a permissible purpose is its own violation. ☐ Check accuracy obligations — 15 U.S.C. § 1681e requires reasonable procedures to assure maximum possible accuracy. ☐ Ask the decision-maker to reconsider on the corrected record. ⚠ The opportunity may be gone; the correction still prevents the next rejection.

See Disputing a Credit Report Error and Defending a Debt Collection Suit: A Practical Checklist.

Phase 3 — Ask for a person

☐ ⭐ Request human review, in writing. Even where no statute requires it, manual review paths exist far more often than they are advertised. ☐ Where your state's privacy statute provides a right to human review of decisions with legal or similarly significant effects, cite it. ☐ Opt out of profiling where the statute allows. ☐ Ask what specifically would change the outcome. ⭐ The specific-reasons requirement exists so the answer is actionable. ☐ Request an accommodation if a disability affected an assessment — 42 U.S.C. § 12112 requires it, and an accessible alternative format is the usual remedy. ☐ Keep every request and response, with dates.

Phase 4 — Assess a discrimination theory

☐ Identify the protected characteristic and the statute:

☐ ⭐ Identify the SPECIFIC practice, not "the algorithm." ⚠ Texas Dept. of Housing & Community Affairs v. Inclusive Communities Project, 576 U.S. 519, requires robust causality.

☐ Establish the disparity — outcome rates by protected class at the operating threshold, not on raw scores.

☐ Anticipate the business necessity defense, and note that ⚠ "it predicted our past decisions accurately" is a description of the problem, not a justification — Griggs v. Duke Power Co., 401 U.S. 424.

☐ ⭐ Develop the less discriminatory alternative — the step that decides most of these cases, and the one the defendant's own development logs will illuminate.

☐ Note that scoring systems are not exempt because they are not formal tests — Watson v. Fort Worth Bank & Trust, 487 U.S. 977.

☐ ⚠ Watch the deadlines — employment charges have short windows. See Three Hundred Days: The Deadline That Ends Most Discrimination Cases Before They Start.

☐ Complain to the regulator — the consumer financial regulator, the housing agency, the employment agency, the insurance commissioner, or the state attorney general. ⭐ Free, and supervision in this area is driven substantially by complaint patterns.

Phase 5 — Government determinations

☐ Request a hearing immediately. ☐ ⚠ ⭐ Request continued benefits pending appeal where the program provides it — usually within ten to fifteen days of the notice. This deadline is the most commonly missed item in the entire area. ☐ Demand the specific factual basis and the data relied upon, including the match or score. ☐ Request agency records — 5 U.S.C. § 552a carries accuracy obligations and access rights for federal systems of records. ☐ Argue the notice was inadequate if it did not state what the discrepancy was. ☐ Argue no individualized determination was made if a match produced the outcome without human verification. See Applying for Benefits and Appealing a Denial, Reduction, or Overpayment: A Practical Checklist and The Public Benefits Appeal Toolkit.

PART TWO — AUDITING A SYSTEM

Phase 6 — Inventory and classification

☐ ⭐ Inventory every model that touches a person. ⚠ Most organizations cannot produce this, and nothing downstream works without it. ☐ For each, record: what it decides · who it affects · what data it uses · who built it · who owns it · when it was last validated · where it is deployed. ☐ Classify by risk:

Phase 7 — Pre-deployment review

☐ Purpose statement — what decision, why this approach, what alternatives were considered at the design level. ☐ Data provenance — sources, period, consent or licensing basis, and any restrictions on use. ☐ Population comparison — training population versus deployment population. ☐ ⭐ Feature justification, one sentence each. ⚠ Drop every feature nobody can justify — removal is free now and expensive later. ☐ Proxy analysis — ⚠ can a protected characteristic be predicted from the feature set? If yes, the model has access to it. ☐ ⭐ Target variable audit — what is the label actually measuring, and was that measurement itself biased? This is where the most serious problems hide. ☐ Subgroup performance, not just aggregate accuracy. ☐ Measurement error by subgroup — ⭐ a feature can be accurate for one group and noisy for another, producing disparity without any difference in definition. ☐ ⭐ Disparity testing at the OPERATING THRESHOLD, by protected class, on realistic data. ☐ ⭐ ⭐ The alternatives search: generate candidates, plot performance against disparity, record the frontier, and document the choice and the reason. ⚠ This is step three of the disparate impact framework. Its absence is close to conceding the case. ☐ Explainability validation — can the pipeline generate individualized principal reasons, and have they been tested against held-out cases? ⚠ A plausible narrative is not a validated reason code. ☐ Accommodation path for assessments — 42 U.S.C. § 12112. ☐ Data protection assessment where a state statute requires one. ☐ Jurisdiction check — bias audits, tool-use notices, consent requirements. ☐ Named approver and date.

Phase 8 — Process design

☐ Adverse action pipeline producing individualized, validated, actionable reasons — ⭐ built into the model's output at design time, not retrofitted. ☐ Human review with real override authority. ⚠ Measure override rates. ⭐ A reviewer who overrides in under one percent of cases is a rubber stamp, and automation bias is well documented. ☐ Give reviewers the underlying evidence, not just the recommendation, and give them time. ☐ Input dispute path connected to the consumer reporting obligations. ☐ Notice content — reasons, source, rights, and any jurisdiction-specific tool disclosure. ☐ Accessibility and an accommodation request route that a person can actually find. ☐ Logging — ⚠ for every decision: the exact inputs, the model version, the score, the threshold, the outcome, the reasons generated, and any human override. ⭐ A decision that was not logged cannot be explained, audited, or defended. ☐ For generative or non-deterministic systems: fixed prompts · constrained sampling · retrieval from a defined corpus · ⭐ full input, output, and version logging so a specific decision can be reconstructed.

Phase 9 — Vendors

☐ ⚠ "The vendor built it" is not a defense. ⭐ Duties run to the decision-maker. ☐ Contract for:

  • ☐ ⭐ Explainability sufficient to generate individualized adverse action reasons — as a warranty
  • ☐ Fairness testing results by protected class at your operating threshold
  • ☐ ⭐ Documentation of alternatives considered during development
  • ☐ Audit rights, including for an independent auditor where required
  • ☐ Data provenance representations and licensing warranties
  • ☐ ⚠ A prohibition on the vendor using your data for its own purposes, including model training
  • ☐ Notice of model changes and a right to re-validate before a new version goes live
  • ☐ Access to decision-level logs
  • ☐ Cooperation with regulator inquiries and litigation holds
  • ☐ Indemnity for discrimination and disclosure claims
  • ☐ Exit rights and data return or deletion ☐ ⚠ A vendor that will not disclose enough for you to explain a decision has sold you a compliance problem.

Phase 10 — Production and monitoring

☐ Define the metrics — performance, drift, and ⭐ outcome disparity by protected class at the threshold. ☐ Define the cadence — monthly or quarterly for significant decisions. ☐ ⭐ Define the threshold that triggers escalation, and the one that triggers suspension. ⚠ Decide this before launch, when it is a policy question rather than a crisis. ☐ Track override rates as a control health measure. ☐ Track complaint and appeal volumes by outcome. ☐ Re-validate on any material change — new data source, retraining, threshold change, population shift. ☐ ⚠ Do not adjust outputs by protected class after the fact without counsel — Ricci v. DeStefano, 557 U.S. 557. ☐ Have a model incident path: who is notified, who can suspend, and how decisions made under a defective model are remediated.

Phase 11 — Documentation retention

⭐ Retain, for every model, what a regulator or opposing expert will ask for:

☐ Purpose statement and design alternatives ☐ Data provenance and licensing ☐ Feature list with justifications and exclusions ☐ Target variable definition and validity analysis ☐ Validation results, overall and by subgroup ☐ ⭐ Fairness testing at the operating threshold ☐ ⭐ Candidate models considered, their performance and disparity, and the selection rationale ☐ Explanation method and its validation ☐ Human review design and override statistics ☐ Monitoring plan, reports, and any escalations ☐ Version history and change log ☐ Approvals with names and dates ☐ Vendor documentation and contracts

⚠ The item most often missing is the alternatives record. ⭐ The complete model file is the best defense available; its absence is the worst fact in the case.

Common mistakes

⚠ Accepting a generic adverse action notice. ⚠ Never requesting the consumer file — where a large share of these decisions actually go wrong. ⚠ Disputing only with the agency and not the furnisher. ⚠ Missing the continued-benefits deadline in a government appeal. ⚠ Deploying a model that cannot generate specific reasons. ⚠ Testing on raw scores rather than at the operating threshold. ⚠ Removing the protected variable and declaring the problem solved. ⚠ Never conducting or documenting an alternatives search. ⚠ Adjusting outputs after the fact to fix disparity. ⚠ A human review that never overrides. ⚠ No decision-level logging. ⚠ A vendor contract with no explainability warranty and no data-use restriction. ⚠ Assuming a national tool faces one jurisdiction's rules.

Authority quick reference

Authority Use it for
15 U.S.C. § 1691 ⭐ Credit adverse action; specific reasons required
12 C.F.R. Part 1002 The implementing regulation; ⚠ generic score explanations are insufficient
15 U.S.C. § 1681b Permissible purpose for a consumer report
15 U.S.C. § 1681e Reasonable procedures for maximum possible accuracy
15 U.S.C. § 1681g ⭐ Right to the full file
15 U.S.C. § 1681i ⭐ Dispute and reinvestigation
15 U.S.C. § 1681m Adverse action notice; identifies the agency
15 U.S.C. § 1681s-2 Furnisher duties on dispute
15 U.S.C. § 45 Unfair or deceptive practices; unsubstantiated capability claims
42 U.S.C. § 2000e-2 · § 2000e-5 Employment discrimination and enforcement
29 U.S.C. § 621 · § 623 Age
42 U.S.C. § 12111 · § 12112 ⭐ Disability; screening out; accommodation
42 U.S.C. § 3604 · § 3605 Housing and residential real estate transactions
5 U.S.C. § 552a Federal agency records used in determinations
Griggs v. Duke Power Co., 401 U.S. 424 ⭐ Effects, not intent
Watson v. Fort Worth Bank & Trust, 487 U.S. 977 Impact reaches subjective and non-test practices
Wards Cove Packing Co. v. Atonio, 490 U.S. 642 Burden allocation history
Inclusive Communities, 576 U.S. 519 ⚠ Robust causality required
Ricci v. DeStefano, 557 U.S. 557 ⚠ Do not adjust outputs after the fact

The one-page version

If a model just said no to you:

  1. ⭐ Get the notice and read the reasons. If generic, demand specific ones in writing.
  2. ⭐ Get the file from the agency named. Read it. A large share of these decisions rest on wrong data.
  3. Dispute the errors — agency and furnisher, in writing, with documents.
  4. Ask for human review, in writing.
  5. Ask what would change the outcome.
  6. Complain to the regulator. It is free.
  7. ⚠ Diary every deadline today.

If you are about to deploy one:

  1. ⭐ Is this a regulated decision? If yes, everything below is mandatory.
  2. Can you produce specific individualized reasons? Prove it before launch.
  3. Justify every feature in one sentence. Drop the rest.
  4. Test disparity at the operating threshold.
  5. ⭐ ⭐ Search for a less discriminatory alternative, and write down what you found.
  6. Build the human review and accommodation paths.
  7. Log every decision.
  8. Retain the file.

Domain supplements

Employment screening

☐ Confirm the two-step adverse action process — pre-adverse action notice with a copy of the report and a summary of rights, a reasonable waiting period, then the final notice. ☐ ⚠ Confirm the disclosure and authorization were standalone documents, not buried in an application. ☐ Check whether a jurisdiction requires notice that an automated tool will be used, and what it assesses. ☐ Check whether a bias audit is required, by an independent auditor, with published summary results. ☐ ⭐ Build the accommodation route into the assessment itself, with a visible way to request an alternative format — 42 U.S.C. § 12112. ☐ ⚠ Video, voice, timed-game, and keystroke assessments are the recurring accommodation problem. ☐ For candidates: ⭐ request the report before the decision is final; the waiting period exists precisely so an error can be corrected while the job is still open.

Tenant screening

☐ ⭐ A screening score is a consumer report. Request the file from the screening company, not only the nationwide bureaus. ☐ Check for a record belonging to another person with a similar name — ⚠ the single most common defect in this category. ☐ Check for a dismissed or sealed eviction still appearing. ☐ ⚠ Blanket criminal-record exclusions implemented by a tool are a recognized impact problem under 42 U.S.C. § 3604. ☐ Ask the landlord to hold the unit pending a dispute. ⚠ Often refused — but ask, and document the request.

Credit and pricing

☐ A worse-than-advertised rate is an adverse action. ⭐ Demand the specific reasons. ☐ Look for a risk-based pricing notice or the credit score disclosure alternative. ☐ Check for pricing disparity where the credit is housing-related — 42 U.S.C. § 3605 reaches it alongside 15 U.S.C. § 1691.

Insurance

☐ Demand the specific basis for a denial or a rating in writing. ☐ Request the claim file where state law permits. ☐ Exhaust the internal appeal, then ⭐ complain to the insurance commissioner — free, fast, and genuinely effective. ☐ Where a credit-based insurance score was used, adverse action notice obligations attach.

Related Documents