After passing a measure in 2021 to limit the use of artificial intelligence in employment-related decisions, New York City released proposed regulations ahead of the new law’s effective date on January 1, 2023.
The law regulates the use of Automated Employment Decision Tools (AEDT) for NYC residents and employees.
The law defines such tools as “any computational process derived from machine learning, statistical modeling, data analysis, or artificial intelligence that outputs simplified results, including a score, classification, or recommendation, used to support or replace judgmental decisions substantially to make employment decisions that affect individuals.”
An employer is prohibited from using an AI tool in hiring decisions unless the tool has been “bias audited” by an “independent auditor” in the previous year and a summary of the audit results and distribution data has been prepared for the tool be made publicly available on the website of the employer or employment agency.
Individuals have the ability to request accommodation so they are not subject to an AI tool and can request information about the data collected about them.
To provide clarity for employers, the Department for Consumer and Workplace Protection (DCWP) has published proposed rules to implement the new law, scheduled a public hearing for discussion and is open to public comments on the proposal until October 24.
The proposed rules provide new and expanded definitions for key terms, including those included in the law’s definition of “automated employment decision tool,” such as “machine learning, statistical modeling, data analysis, or artificial intelligence,” “simplified output,” and “substantially support discretionary decisions or substitute.”
For example, the DCWP said “substantially supporting or replacing discretionary decisions” means relying solely on a simplified output (e.g., score, tag, classification, ranking) without considering other factors to produce a simplified output to use as one of: a a set of criteria in which the output is weighted more heavily than any other criteria in the set, or to use a simplified output to override or modify conclusions drawn from other factors , including human decision-making. According to the proposed rule, a “simplified output” “may take the form of a score (e.g., rating a candidate’s estimated technical skills), a tag, or a categorization (e.g., categorizing a candidate’s resume using keywords, assigning a ability or trait) of a candidate), recommendation (e.g., whether a candidate should be invited for an interview), or ranking (e.g., compiling a list of candidates based on how well their cover letters match the job description) .”
Regarding the requirements for bias audits, the proposed rules define an independent auditor as “a person or group not involved in the use or development of an AEDT and who is responsible for conducting a bias audit of such AEDT”.
Depending on the type of AEDT used, there are two types of audits allowed by the proposed rules. When an AEDT selects individuals to advance in the hiring process or groups individuals, the bias audit must calculate the selection rate and impact ratio for each EEO-1 demographic category; and when an AEDT classifies individuals into groups, the calculations must be performed for each such classifica- tion.
Alternatively, when an AEDT evaluates applicants or candidates, the bias audit must calculate the average score for each individual in each category and the impact ratio for each category.
The proposed rules also explain what information must be made publicly available before an employer may use an AEDT (e.g. a summary of the last bias audit for the AEDT used).
As for legal notification requirements, employers must notify all New York City resident candidates and employees at least ten business days prior to using an AEDT that an AEDT will be used to evaluate their candidacy and indicate the professional qualifications and tool being evaluated .
The rules provide several options for notification, including posting a clear and conspicuous notification on a website, listing the notification in a job posting, or providing it by US mail or email. The notice must also include instructions on how to apply for an alternative selection process or accommodation under the proposal.
To read the proposed rules, click here.
Why it matters: Employers in New York City should keep a close eye on the proposed rules as the new law’s effective date approaches, and employers in other parts of the country should brace for similar regulations in their own states or even federal oversight on AI . Not long after the city released its proposed rules, the White House released a draft AI Bill of Rights tracking recent federal guidance from the Justice Department and the Equal Opportunity Commission on how employers can avoid violating the law on Americans with disabilities to violate the use of AI for employment-related decisions.