The Automated Decision and AI Accountability Toolkit

Technology > Automated Decisions and Artificial Intelligence | Published 25 June 2024 - Updated 9 June 2025 | UpLaw - uplaw.us

⚠ Jurisdiction-specific mandates — bias audits, tool-use notices, consent requirements — change every session. ⭐ The federal framework these templates rest on is stable; the local layer is not. Check it.

Companion documents: Challenging an Automated Decision and Auditing an AI System: A Practical Checklist · Automated Decisions and Artificial Intelligence: A Comprehensive Guide to Disclosure, Bias, and Accountability · The Model Said No: How Automated Decisions Are Made and How They Are Challenged

---

Tool 1 — Model inventory

⭐ Build this first. Nothing else in the toolkit works without it.

ID	System	What it decides	Population affected	⚠ Regulated domain?	Consumer report inputs?	Sensitive inputs?	Built by	Owner	Version	Last validated	Jurisdictions	Risk tier
				☐	☐	☐

Systems people forget to inventory: ☐ Vendor-hosted scoring tools embedded in a workflow ☐ Spreadsheet models used for eligibility or pricing ☐ Rules engines with thresholds that were never revisited ☐ Fraud and risk triage that routes to a slower queue — ⭐ a delay can be an adverse action in effect ☐ Scheduling and productivity systems that influence termination ☐ Prioritization tools in care management or case assignment ☐ Anything a business unit bought without procurement review

---

Tool 2 — Model card

Model Card — [Name], version ______

1. Decision. What is decided, about whom, with what consequence. ⚠ Regulated domain: ☐ credit ☐ housing ☐ employment ☐ insurance ☐ benefits ☐ health ☐ education ☐ none 2. Owner and approver, with dates. 3. Intended use and, ⭐ explicitly, out-of-scope uses. 4. Data. Sources, collection period, consent or licensing basis, restrictions on use. 5. Training population versus deployment population, with the comparison. 6. Features. Attach the justification log (Tool 3). 7. Target variable and the validity analysis (Tool 4). 8. Performance — overall and by subgroup. 9. Fairness testing — disparity at the operating threshold, by protected class (Tool 5). 10. Alternatives considered — attach the search record (Tool 6). ⭐ The most important attachment in this document. 11. Explanation method and its validation (Tool 7). 12. Human review design and authority (Tool 8). 13. Monitoring plan, thresholds, and escalation (Tool 11). 14. Known limitations. 15. Change log.

---

Tool 3 — Feature justification log

Feature	Source	⭐ Business justification, one sentence	Proxy risk (can it predict a protected class?)	Retained?	Reason if dropped
			☐ high ☐ medium ☐ low	☐

⚠ Drop every feature nobody can justify in one sentence. ⭐ Removal is free during development and enormously expensive in litigation.

Features to scrutinize hardest: ⭐ zip code and census tract · name · educational institution · employment gaps · credit history depth · device and browser · application timing · speech and video features · purchase categories · social graph features.

Proxy test: train a simple classifier to predict each protected characteristic from the feature set. ⚠ If it succeeds, the model has access to that characteristic regardless of whether you included it.

---

Tool 4 — Target variable audit

What is the model predicting? ______ What decision is it actually being used to make? ______ ⭐ Is the first a valid proxy for the second? Explain. ⚠ This is where the most serious problems hide.

How was the label created? ☐ Human decisions ☐ Observed outcome ☐ Proxy measure ☐ Vendor-supplied ⚠ If human decisions: whose, over what period, and is there reason to believe that process was biased? ⭐ A model predicting past hiring decisions is predicting the past hiring process. A model predicting arrests is predicting policing patterns. Was label quality uniform across subgroups? ⚠ A label that is noisier for one group produces disparity without any difference in feature definitions. What is the base rate by subgroup, and does it reflect reality or measurement?

---

Tool 5 — Disparate impact test plan

☐ Define the outcome being tested — the actual decision, at the actual threshold. ⚠ Not the raw score distribution. ☐ Define the population — applicants, not accepted applicants. ☐ Define the groups — race and ethnicity, sex, age, disability where determinable, national origin, familial status and disability for housing. ☐ Handle unknown demographics transparently, and document the imputation method if one is used.

Group	N	Selection rate	Ratio to highest group	Difference	Statistical significance	Practical significance

☐ ⚠ There is no safe harbor number. ⭐ Established rules of thumb are screening heuristics, not legal standards — courts weigh statistical and practical significance together. ☐ Test at every threshold under consideration, not just the chosen one. ☐ Test intersectionally where sample sizes permit. ☐ Test subgroup performance, not only selection rates — ⭐ a model can select at equal rates and be far less accurate for one group. ☐ Test measurement error by subgroup. ☐ Document the method, the data, the date, and who ran it.

---

Tool 6 — Alternatives search record

⭐ ⭐ This is step three of the disparate impact framework and the single most important document in the model file. Its absence is close to conceding the case.

Candidate	Description of the change	Performance metric	Disparity metric	Selected?	⭐ Reason
Baseline	Full feature set, threshold ___			☐
A	Removed feature(s) ______			☐
B	Constrained on ______			☐
C	Alternative threshold ___			☐
D	Alternative model family			☐
E	Alternative target definition			☐

☐ Plot the frontier — performance against disparity — and attach it. ☐ ⭐ Record the selection reason in business terms, not as a number: "Candidate B reduced the selection rate gap by 41 percent at a cost of 0.4 points of AUC, and was adopted." ☐ ⚠ If a lower-disparity candidate with comparable performance was rejected, record why in detail. That sentence is the case. ☐ Re-run the search on every retraining.

---

Tool 7 — Adverse action reason codes

Design requirement: ⭐ for every possible decision, produce the actual principal reasons, specific and actionable — 15 U.S.C. § 1691 and 12 C.F.R. Part 1002.

Code	Reason text given to the consumer	Underlying factor	⭐ Actionable?	Validated against held-out cases?
	"Length of credit history is too short"		☐	☐
	"Balances on revolving accounts are too high relative to limits"		☐	☐
	"Too many recent inquiries"		☐	☐
	"Insufficient number of accounts with satisfactory payment history"		☐	☐
	"Recent delinquency reported"		☐	☐

⚠ Unacceptable reason text: "did not meet our scoring criteria" · "model output below threshold" · "internal risk assessment" · ⚠ anything a consumer cannot act on.

Validation protocol: ☐ Generate reasons for a held-out sample. ☐ ⭐ Perturb the named factor and confirm the outcome actually changes. ⚠ If it does not, the reason is not the reason. ☐ Compare against a simpler interpretable benchmark model. ☐ Check for correlated-feature masking — ⚠ naming one factor when three jointly drove the outcome. ☐ Have a non-technical reviewer confirm each reason is comprehensible and actionable. ☐ Record the validation date and results in the model card.

---

Tool 8 — Human review protocol

Scope. Human review is available for [decision types], on request and automatically where [conditions]. Reviewer qualifications and training. ______ ⭐ What the reviewer sees: the underlying evidence, not the model's recommendation alone. A reviewer shown the score first anchors on it. Authority. The reviewer may confirm, modify, or reverse the decision without further approval. Time. Minimum ______ minutes per review. ⚠ A queue that requires ninety seconds per case is not review. Documentation. The reviewer records the basis for the decision and whether the model was overridden. Escalation. ______

Control health metrics:

Metric	Target	Actual	⚠ Flag
Override rate			⭐ Under 1 percent suggests a rubber stamp
Override rate by protected class			Disparities in who gets a second look
Median review time			Too short means not reading
Reversal rate on appeal after review			High means review is not working

⚠ Automation bias is well documented. ⭐ Measure the override rate and treat a near-zero figure as a control failure, not a validation.

---

Tool 9 — Vendor questionnaire

☐ What decision does the system make or influence, and in what regulated domains is it deployed? ☐ What data was it trained on? Sources, period, consent or licensing basis, restrictions. ☐ Is any input a consumer report? ☐ What features does it use? ⚠ If the answer is "proprietary," ask how we are expected to justify each feature under a business necessity showing. ☐ ⭐ Can it produce individualized principal reasons for each decision? Show us, on our data. Will you warrant it? ☐ What fairness testing has been performed? At what thresholds, on what population, by whom, and can we see the results? ☐ ⭐ What alternative models were considered during development, and why was this one chosen? ☐ How is the model monitored, and what triggers a retrain? ☐ How will we be notified of model changes, and can we re-validate before a new version is live? ☐ What decision-level logs will we receive? ☐ Will you support an independent bias audit where a jurisdiction requires one? ☐ ⚠ Will you use our data for any purpose other than serving us, including training? ⭐ The only acceptable answer is no, in the contract. ☐ What is your incident process if the model is found to be producing defective outcomes? ☐ Indemnity for discrimination and disclosure claims?

⚠ A vendor that cannot answer the explainability question has sold a compliance problem, whatever the performance claims.

---

Tool 10 — Vendor contract clauses

Explainability warranty. Vendor warrants that the System is capable of generating, for each individual decision, the specific principal factors that caused that decision, in a form sufficient for Customer to satisfy its obligations under 15 U.S.C. § 1691 and 12 C.F.R. Part 1002, and under 15 U.S.C. § 1681m where applicable. ⭐ Vendor shall provide the methodology and shall cooperate in Customer's validation of the generated reasons.

Fairness testing. Vendor shall provide, before deployment and annually thereafter, testing of outcome disparity by protected class at Customer's operating threshold, using a population representative of Customer's applicants, together with the underlying methodology.

⭐ Development record. Vendor shall provide documentation of the candidate models considered during development, their comparative performance and disparity, and the basis for the selection made.

Data use restriction. ⚠ Vendor shall not use Customer Data for any purpose other than providing the Services, and shall not use Customer Data to train, fine-tune, evaluate, or improve any model made available to any other party.

Change control. Vendor shall give Customer at least ______ days' notice of any material change to the System, including retraining, feature changes, and threshold changes, and Customer may re-validate before the change takes effect in Customer's environment.

Logging. Vendor shall make available decision-level logs including inputs, model version, score, threshold, outcome, and generated reasons, retained for not less than ______ years.

Audit. Vendor shall cooperate with audits by Customer and by an independent auditor engaged to satisfy any legal requirement, and shall provide access sufficient to complete such an audit under a protective order where Vendor asserts confidentiality.

Regulatory cooperation. Vendor shall cooperate promptly with any regulator inquiry, litigation hold, or discovery obligation relating to the System.

Incident. Vendor shall notify Customer within ______ hours of identifying any defect materially affecting the accuracy or fairness of decisions, and shall support remediation of affected decisions.

Indemnity. Vendor shall indemnify Customer against claims arising from the System's discriminatory operation, from inaccurate outputs, and from Vendor's failure to provide explanation capability as warranted.

---

Tool 11 — Monitoring dashboard specification

Metric	Cadence	Warning threshold	⚠ Suspension threshold	Owner
Model performance versus validation baseline	Monthly
Input drift by feature	Monthly
Population drift	Quarterly
⭐ Selection rate by protected class at the threshold	Monthly
⭐ Disparity ratio trend	Monthly
Subgroup accuracy	Quarterly
Override rate overall and by group	Monthly
Median human review time	Monthly
Adverse action reason distribution	Quarterly
Complaint and appeal volume by outcome	Monthly
Reversal rate on appeal	Quarterly
Decision log completeness	Monthly

⭐ Set the suspension threshold before launch, while it is a policy question rather than a crisis. ⚠ Naming who can suspend the system, and confirming they will not need three approvals to do it, is the part that fails.

---

Tool 12 — Decision log schema

⭐ A decision that was not logged cannot be explained, audited, or defended.

Field	Notes
Decision ID
Timestamp
Subject identifier	Pseudonymous where possible
Jurisdiction of the subject	⚠ Determines which rules apply
⭐ Model name and exact version
Complete input vector	Or a reference to an immutable snapshot
Data sources used	⚠ Flag any consumer report
Score or output
Threshold applied
Outcome
⭐ Reason codes generated
Notice sent — type, date, method
Human review — reviewer, time spent, decision, basis
Override — yes/no, and the reason
Appeal — filed, outcome, date
Dispute of inputs — filed, outcome, date

⚠ For generative or non-deterministic systems, also log: the exact prompt, the retrieval corpus version, sampling parameters, and the raw output. ⭐ Without these the decision cannot be reconstructed, and a reconstruction is what an audit requires.

---

Tool 13 — Claimant letters

Demand for specific reasons:

Re: Adverse action dated ______, application ______

I received your notice dated ______. The notice states only that ______. That is not a statement of specific reasons.

Under 15 U.S.C. § 1691 and 12 C.F.R. Part 1002, I am entitled to a statement of the specific principal reasons for the action. A reference to a scoring system or an internal model is insufficient.

Please provide, within thirty days: (1) the specific principal reasons for the decision; (2) whether an automated system was used and, if so, the factors it weighed in my case; (3) the identity of any consumer reporting agency whose report was used, as required by 15 U.S.C. § 1681m; and (4) whether human review is available and how to request it.

Consumer file request:

Re: Request for file disclosure following adverse action

On ______ I received notice of an adverse action based in whole or in part on a report from your company. Under 15 U.S.C. § 1681g and 15 U.S.C. § 1681j, I request my complete file at no charge, including all information in the file, the sources of the information, and identification of each person who procured a report about me.

Please also state whether any score was provided, the score, the key factors adversely affecting it, and the range of possible scores.

Dispute letter:

Re: Dispute of inaccurate information

The following information in my file is inaccurate: ______. The correct information is ______. Documentation is enclosed.

Under 15 U.S.C. § 1681i you must conduct a reasonable reinvestigation, forward all relevant information I have provided to the furnisher, and delete or modify any item that cannot be verified. Under 15 U.S.C. § 1681e you must follow reasonable procedures to assure maximum possible accuracy.

Please send written results and a corrected file, and provide corrected reports to everyone who received a report in the last six months, or two years for employment purposes.

⭐ Copy the furnisher, whose duties appear at 15 U.S.C. § 1681s-2.

Human review and profiling opt-out:

I request human review of the decision dated ______. I request that the review be conducted by a person with authority to reverse the decision, and that I be told the outcome and its basis in writing.

To the extent [state statute] applies, I also exercise my right to opt out of profiling in furtherance of decisions that produce legal or similarly significant effects concerning me, and my right to limit the use of my sensitive personal data.

---

Tool 14 — Regulator complaint outline

1. Who I am, and the decision at issue — date, entity, and outcome. 2. What I was told — quote the notice verbatim, and attach it. 3. Why the notice was inadequate — ⭐ cite 15 U.S.C. § 1691, 12 C.F.R. Part 1002, and 15 U.S.C. § 1681m as applicable. 4. What I requested and when, and what I received. 5. The data problem, if any — the inaccuracy, the dispute filed under 15 U.S.C. § 1681i, and the response. 6. The pattern, if known — ⭐ whether others similarly situated received the same outcome, which is what turns a complaint into an investigation. 7. What I am asking for — correction, reconsideration, and examination of the practice. 8. Attachments — notices, correspondence, file disclosures, and dispute results.

☐ Route it correctly: the consumer financial regulator for credit · the housing agency for housing · the employment agency for hiring · the insurance commissioner for coverage and claims · the state attorney general for state privacy statutes. ☐ ⭐ File even where an individual remedy is unlikely. ⚠ Supervisory attention in this field has been driven substantially by complaint patterns.

---

Tool 15 — Model incident response

⭐ Have this written before you need it.

Trigger. A monitoring threshold is breached, an audit finds a defect, a regulator inquires, or a pattern of complaints emerges.

Hour one. ⚠ Preserve — freeze the model version, the decision logs, and the monitoring data. Do not retrain or patch before preserving. Engage counsel. See Issuing a Litigation Hold and Preserving ESI: A Practical Checklist.

Day one. Decide whether to suspend the system, revert to a prior version, or route decisions to human review. ⭐ The named owner makes this call; it should not require a committee.

Days two through seven. Scope — which decisions, over what period, affecting whom, in which jurisdictions. ⚠ This requires the decision log; if the log is incomplete, that is the first finding.

Remediation. Identify affected individuals. Decide whether to re-decide, to notify, or both. ⚠ In a regulated domain, a decision made on a defective basis may require a corrected adverse action notice. ⭐ Do not silently re-decide adverse outcomes without documenting it.

⚠ Do not adjust outputs by protected class as the remedy without counsel — Ricci v. DeStefano, 557 U.S. 557.

Root cause and record. Document what failed — data, features, target, threshold, monitoring, or governance — and what changed.

Post-incident review, documented. ⚠ Its absence is itself a finding.

---

Tool 16 — Annual program review

Item	Done	Notes
Model inventory refreshed, including shadow systems	☐	⚠ Ask business units directly
Risk classification updated	☐	New jurisdictions, new populations
Feature justification logs current	☐
⭐ Fairness testing re-run at current thresholds	☐
⭐ Alternatives search re-run on any retrained model	☐
Reason codes re-validated	☐	Against held-out cases
Human review override rates reviewed	☐	⚠ Near-zero is a control failure
Accommodation requests and outcomes reviewed	☐	42 U.S.C. § 12112
Decision log completeness verified	☐
Vendor documentation refreshed; contracts checked	☐	Explainability warranty, data-use restriction
Data protection assessments current	☐
Jurisdiction-specific mandates re-checked	☐	⚠ Bias audits, notices, consent
Complaint and appeal patterns reviewed	☐	⭐ The earliest signal of a problem
Incident procedure tested	☐	Who suspends, and how fast

---

Tool 17 — Authority reference

Authority	Use
15 U.S.C. § 1691 · 12 C.F.R. Part 1002	⭐ Credit adverse action; specific reasons
15 U.S.C. § 1681b	Permissible purpose
15 U.S.C. § 1681e	Accuracy procedures
15 U.S.C. § 1681g · § 1681j	File disclosure; free disclosures
15 U.S.C. § 1681i · § 1681s-2	Dispute; furnisher duties
15 U.S.C. § 1681m	Adverse action based on a report
15 U.S.C. § 45	Unfair or deceptive practices
42 U.S.C. § 2000e-2 · § 2000e-5	Employment discrimination
29 U.S.C. § 621 · § 623	Age
42 U.S.C. § 12111 · § 12112	Disability; accommodation
42 U.S.C. § 3604 · § 3605	Housing
5 U.S.C. § 552a	Federal agency records
Griggs, 401 U.S. 424 · Watson, 487 U.S. 977 · Wards Cove, 490 U.S. 642 · Inclusive Communities, 576 U.S. 519 · Ricci, 557 U.S. 557	The impact framework

---

Tool 18 — Where this touches the rest of the file

Also in play	Read
Personal data governance generally	Data Privacy and Security: A Comprehensive Guide to Collection, Consent, Breach Response, and Enforcement · The Data Privacy and Breach Response Toolkit · Responding to a Data Breach and Exercising Privacy Rights: A Practical Checklist · The Data You Did Not Know They Had: How Privacy Law Actually Reaches Your Information
The inputs are consumer reports	Consumer Credit Reporting and Debt Collection: A Comprehensive Guide to the FCRA, FDCPA, and Identity Theft Remedies · Disputing a Credit Report Error and Defending a Debt Collection Suit: A Practical Checklist · The Consumer Debt and Credit Reporting Toolkit
The decision was about employment	Filing and Pursuing an Employment Discrimination Charge: A Practical Checklist · The Employment Discrimination Charge Toolkit · Three Hundred Days: The Deadline That Ends Most Discrimination Cases Before They Start
The decision was about housing	No Children, No Wheelchairs, No Comment: How Housing Discrimination Actually Shows Up · Defending an Eviction and Enforcing a Tenant's Rights: A Practical Checklist
The decision was about benefits	Applying for Benefits and Appealing a Denial, Reduction, or Overpayment: A Practical Checklist · The Public Benefits Appeal Toolkit · The Letter That Cuts Your Benefits: How Public Assistance Decisions Are Actually Made
The model or data is a trade secret	Trade Secrets and Restrictive Covenants: A Comprehensive Guide to Protection, Enforcement, and Defense · The Trade Secret and Restrictive Covenant Toolkit
Training data ownership is at issue	Copyright: A Comprehensive Guide to Ownership, Registration, Licensing, Fair Use, and Infringement · The Copyright Registration and Enforcement Toolkit

---

Related Documents

• Challenging an Automated Decision and Auditing an AI System: A Practical Checklist
• Automated Decisions and Artificial Intelligence: A Comprehensive Guide to Disclosure, Bias, and Accountability
• The Model Said No: How Automated Decisions Are Made and How They Are Challenged
• The Data Privacy and Breach Response Toolkit
• The Consumer Debt and Credit Reporting Toolkit