Bias Audits for Hiring AI: NYC Local Law 144 Practical Guide

What LL144 actually requires

The law has three load-bearing requirements:[1][2]

• Annual independent bias audit. The AEDT must have been audited within one year of the date you use it. Selection or scoring rates and impact ratios must be calculated for sex, race/ethnicity, and intersectional sex × race/ethnicity categories.
• Public summary publication.A summary of the most recent bias audit must be available on the employer's website, including the audit date, data source/explanation, selection or scoring rates, and impact ratios per category.
• Candidate notice. At least ten business days before AEDT use, candidates must be told the AEDT will be used, what qualifications and characteristics it assesses, and how to request an alternative selection process or accommodation.

The “AEDT” definition

An AEDT is any computational process that issues simplified output (a score, classification, recommendation, or ranking) that is then used to substantially assist or replace a human decision-maker in a hiring or promotion process.[3] This is broader than “AI” in the colloquial sense:

• A logistic regression that scores resumes — yes, AEDT.
• A rules-based knockout filter — DCWP guidance suggests yes if it materially substitutes for a human decision.
• An LLM-based interview scorer — yes, clearly.
• A simple pass-through Boolean search — usually not, because no “simplified output” is generated beyond a literal match.
• A scheduling tool that does not score candidates — no.

When in doubt, the operational test is whether removing the system would force a human to perform a non-trivial substitute analysis. If yes, treat it as an AEDT.

The four-fifths rule and selection-rate computation

The four-fifths rule (also called the 80% rule) comes from the EEOC's Uniform Guidelines on Employee Selection Procedures (29 C.F.R. § 1607.4) — DCWP explicitly aligned the LL144 calculation with this established federal standard.[3] The mechanic:

1. For each demographic category, compute the selection rate = (number selected by the AEDT) / (number assessed by the AEDT). For a scoring AEDT (no binary selection), use the scoring rate = (sum of scores in the category) / (count in the category).
2. Identify the category with the highest selection or scoring rate (the “most-selected category”).
3. For every other category, compute the impact ratio = (category's rate) / (most-selected category's rate).
4. An impact ratio below 0.80 in any category is the classic indicator of disparate impact and must be disclosed.

Worked example. Suppose your AEDT scored 1,000 candidates, with the following selection rates by sex × race intersection. The reference (highest-rate) category becomes the denominator for every impact ratio below it.

Two intersectional categories fall below 0.80, signalling disparate impact that must be disclosed in the published summary. The remediation conversation — model recalibration, feature audit, rubric review — happens after publication, not before; the law requires disclosure of the failing audit, not concealment.

Implementation note from the rule: an auditor may exclude a category that represents less than 2% of the data from the impact-ratio calculation, but must justify the exclusion and disclose the excluded category's selection rate in the summary.[3]

Who can perform the audit

The auditor must be independent, meaning none of the following applies:[1]

• Currently employed by the deploying employer or the AEDT vendor.
• Was previously involved in using, developing, or distributing the AEDT.
• Has a direct or material indirect financial interest in either party.

The law does not require auditor pre-approval by DCWP, and is notably silent on professional qualifications. In practice, this means quality varies dramatically. The defensible choice is an auditor with formal training in industrial-organizational psychology or applied statistics, ideally with a track record of EEOC-style adverse impact analyses prior to LL144.[4]

What the audit report must contain

Per the published rule, the bias audit report must include:[3]

• The date of the most recent bias audit.
• The source and explanation of the data used (historical, test, or vendor-provided).
• The number of applicants or candidates assessed.
• The number falling within an “unknown” demographic category.
• Selection or scoring rates per sex, race/ethnicity, and intersectional category.
• Impact ratios per category against the most-selected reference.
• Justification for any category excluded due to the <2% rule.

The summary must be on the employer's website (not the vendor's) and accessible without a login.

Annual cadence and the export pipeline

The audit must have been completed within one year of the date the AEDT is used. Operationally, this means a fresh audit every twelve months — not just at calendar year-end. To make that sustainable, your AEDT must continuously emit:

• Per-candidate AEDT score / classification / recommendation.
• Voluntary self-reported demographic data per the EEO-1 categories.
• Final hiring decision (where the AEDT's output was actually used as input).
• Timestamps for the assessment and decision.
• Audit log of any operator overrides.

Without this export pipeline, your auditor will resort to whatever the vendor happens to expose, which is rarely what they need. Best practice: define the audit-export schema once and treat it as a first-class product surface.

Penalties and the enforcement reality

Headline fines are modest at $500 for a first violation and $500–$1,500 for subsequent violations. The teeth come from two places: each day of continued non-compliance is a separate violation, and a published failing impact ratio is admissible evidence in federal Title VII discrimination litigation. The LL144 fine is rarely the most expensive consequence; the downstream civil rights exposure is.[5]

Demographic data collection: the hard part

To compute a four-fifths impact ratio you need demographic data on the people the AEDT assessed. This is the single biggest operational challenge in LL144 compliance, and it is regularly underestimated. The candidates self-reporting demographics are doing so voluntarily; participation rates run 40–70% in practice; and the “unknown” bucket can swamp the other categories if your collection workflow is bad.

Three operational principles tighten the collection rate. First, ask after the application is submitted, not before — pre-application demographic asks depress completion. Second, use the EEO-1 standard categories verbatim, including the “prefer not to disclose” option; deviating from the federal standard creates downstream interoperability problems for the auditor. Third, make explicit on the form that the data is used for bias audit compliance, not to influence the hiring decision — the audit is computed on the AEDT score, not the demographic field, and candidates are entitled to know that.

Even with these in place, you will see “unknown” accumulate. The DCWP rule lets you report it, but if more than 10–15% of your assessed pool is unknown the auditor will flag the data quality and recommend remediation. Treat collection rate as a first-class metric, not a side-effect.

The relationship to federal Title VII

LL144 sits on top of, not in place of, federal employment discrimination law. A passing four-fifths audit does not insulate you from Title VII liability if a candidate brings a discrimination claim; conversely, a failing four-fifths audit is admissible evidence and creates a presumption the employer must rebut. The dominant litigation pattern is therefore: a sub-0.80 ratio published per LL144 → discovery → a Title VII class action. The published audit becomes Exhibit A.

The defensible posture is to treat the LL144 audit not as a regulatory checkbox but as an early-warning system. Run it internally before the formal audit each year; use the internal run to recalibrate the AEDT, retrain on more representative data, or adjust the rubric. Then the formal audit is documenting an already- good outcome rather than discovering a bad one.

How Intrvio supports LL144

For Intrvio deployers in NYC scope, the platform exports an audit-ready CSV per pipeline that includes per-candidate AEDT rubric scores, voluntary self-reported demographics, the final hiring decision, and a per-decision audit log. The export schema maps directly to the LL144 four-fifths rule computation. Candidate notice templates and the alternative-process request flow ship in the platform; the bias audit itself is performed by an independent third party of your choosing.

Frequently asked questions