New York governor signs legislation addressing AI, consumer data protection

New York State has taken a significant step forward in ensuring that its use of AI is carried out responsibly. The recently enacted Senate Bill S7543B, known as the Legislative Oversight of Automated Decision-Making in Government Act (LOADinG Act), establishes a comprehensive framework for the state’s use of AI that emphasizes transparency, accountability, and the protection of individual rights.

Governor Kathy Hochul also signed into law a package of legislation that strengthens the protections of consumers’ personal data. Hochul said in a statement that “New Yorkers should never have to worry about their personal information being misused or falling into the wrong hands. Hochul said the legislative package she signed constitutes “bold action to hold companies accountable, strengthen protections, and give consumers the transparency and security they need and deserve.”

The enactment of the LOADinG Act positions New York as a leader in the responsible governance of AI and automated systems. By instituting rigorous oversight mechanisms, the state aims to harness the benefits of technological advancements while safeguarding the rights and well-being of its residents.

As AI continues to permeate virtually all facets of public administration, the principles outlined in the LOADinG Act could serve as a model for other states seeking to balance innovation with ethical responsibility. The act’s emphasis on human oversight, regular impact assessments, and transparency reflects a comprehensive approach to integrating technology into government operations without compromising individual rights.

The primary objective of the LOADinG Act is to regulate the deployment of automated decision-making systems by state agencies. These systems, defined as software utilizing algorithms, computational models, or AI techniques to make or assist in making decisions, have the potential to impact public assistance benefits, civil liberties, safety, and welfare. Recognizing this, the legislation mandates that any state agency employing such technology must ensure that it is subject to continuous and “meaningful human oversight.” The new law aims to prevent unchecked automated decisions that could adversely affect individuals’ rights.

The LOADinG Act “require[s] the disclosure of automated decision-making systems already in use, prohibit[s] the unauthorized use of automated decision-making systems by state agencies or entities on behalf of an agency without meaningful human review and, when authorized, require[s] the publication of an impact assessment.”

A cornerstone of the new law is the requirement for state agencies to conduct thorough impact assessments before implementing automated decision-making systems. State agencies planning to use automated decision-making systems, as allowed under Section 402 of the legislation, must ensure these systems are subject to consistent and meaningful human oversight.

Before deploying such a system, an impact assessment first must be completed and approved by individuals who are responsible for ensuring proper oversight and lawful application of the technology. After the initial assessment, agencies are required to conduct a new impact assessment at least every two years. If any significant changes are made to the system that could alter its outcomes or effects, a fresh assessment must be performed. These assessments serve as a detailed review of how the system operates and its potential effects.

The impact assessments must include a clear description of the system’s objectives, an evaluation of the system’s ability to achieve its goals effectively, and a detailed explanation of the system’s design and development. The assessment also must include a summary of the algorithms, computational methods, and AI tools that are used, as well as information about the data and processes involved in training and the design of the system.

Additionally, state agencies must conduct tests to ensure the system performs accurately, fairly, and without bias. They must also evaluate whether the system produces discriminatory outcomes based on factors such as race, ethnicity, religion, gender, sexual orientation, disability, or other protected characteristics. If disparities in outcomes are identified, the assessment must outline strategies to address and mitigate these issues. This rigorous process aims to ensure that automated decision-making systems are implemented responsibly, ethically, and in a manner that upholds fairness and equality.

To maintain ongoing oversight, the law stipulates that impact assessments be conducted at least once every two years. Furthermore, any significant modifications to an existing system that could alter its outcomes necessitate a new assessment. This continuous evaluation process is intended to ensure that as technology evolves, its application remains aligned with ethical standards and public interest.

Transparency is further reinforced by the requirement that each impact assessment be submitted to the Governor, the Temporary President of the Senate, and the Speaker of the Assembly at least thirty days prior to the system’s implementation. This provision facilitates legislative oversight and allows for informed decision-making at the highest levels of state government.

The act also addresses the procurement process by prohibiting state agencies from acquiring services or systems that utilize automated decision-making without ensuring meaningful human review. This measure prevents the adoption of potentially harmful technologies and encourages the selection of systems that adhere to the state’s ethical and legal standards.

Under the act, “meaningful human review” refers to the active involvement of one or more individuals in overseeing and controlling the automated decision-making process. These individuals must be knowledgeable about the system’s risks, limitations, and functionality, and must have received training on its use. They are empowered to intervene in the decision-making process, and have the authority to approve, deny, or modify any decision proposed or made by the automated system. This ensures that human judgment remains central to decisions influenced by such technology.

In instances where an impact assessment reveals that an automated decision-making system produces discriminatory or biased outcomes, the legislation mandates that the state agency immediately terminate its use. This decisive action underscores the state’s commitment to preventing harm and upholding the principles of fairness and equality.

Senate Bill S7543B represents a proactive and seemingly well-thought-out approach to the responsible use of AI by government. By establishing clear guidelines and oversight mechanisms, New York is demonstrating its commitment to leveraging technology in a manner that is effective and ethically sound. The LOADinG Act not only addresses current concerns, but it also sets a precedent for future technological governance, ensuring that as AI evolves, its application within the state remains aligned with the values of fairness, transparency, and accountability.

The act also mandates evaluations of cybersecurity vulnerabilities and privacy risks, along with the development of safeguards to mitigate any identified issues.

Complementing the LOADinG Act is the series of legislative measures Governor Hochul signed into law that address various aspects of digital privacy, security, and consumer protection. One law mandates that social media companies clearly display their terms of service to inform users and submit reports on these terms to the attorney general, with penalties for non-compliance. Another bill strengthens safeguards for medical and insurance data, aiming to reduce identity theft risks.

To enhance government cybersecurity, another new law requires state agencies to procure computing goods and services that are aligned with the National Institute of Standards and Technology’s Cybersecurity Framework. Additionally, businesses are now obligated to notify affected parties of data breaches within 30 days and to promptly inform the Department of Financial Services if New York residents are impacted.

A separate measure bans the use of social media platforms for debt collection, while regulations for online dating services have been tightened, requiring clearer disclosures and protocols to address fraudulent accounts.

Article Topics

biometric data  |  cybersecurity  |  data privacy  |  data protection  |  digital identity  |  legislation  |  New York  |  social media