AI in Insurance Update: NYDFS Adopts Insurance Circular Letter 7 on the Use of Artificial Intelligence and External Consumer Data and Information Sources in Insurance Underwriting and Pricing
I. Introduction
On July 11, 2024, the New York Department of Financial Services (“DFS”) adopted Insurance Circular Letter 7 on the Use of Artificial Intelligence and External Consumer Data and Information Sources in Insurance Underwriting and Pricing (the “Circular Letter”).[i] The Circular Letter is intended to provide DFS’s expectations for insurers authorized to write insurance in New York who develop and/or use external consumer data and information sources (“ECDIS”), artificial intelligence systems (“AIS”), and other predictive models in underwriting and pricing insurance policies and annuity contracts. As we have previously reported,[ii] on January 17, 2024, DFS released the Circular Letter in a draft form for public comment (the “Draft Circular Letter”),[iii] which has culminated with DFS adopting the Circular Letter with only minimal changes. In the summary provided below, we summarize the Circular Letter and highlight the material changes made to the Circular Letter in response to the comments received by DFS.
II. The Context
The Circular Letter joins other recent regulatory activity focused on the use of artificial intelligence in insurance, including prior action taken by Colorado, the National Association of Insurance Commissioners (the “NAIC”) and DFS’s own Insurance Circular Letter No. 1 (2019) (the “2019 Circular Letter”).
a. Colorado
In June 2021, Colorado passed a first-of-its-kind statute prohibiting insurers from using ECDIS, as well as algorithms or predictive models that use ECDIS, in a way that unfairly discriminates based on protected class.[iv] As required by the statute, the Colorado Division of Insurance (“CO DOI”) has engaged in a stakeholder process to effectuate this law, and in August 2023, it adopted a regulation requiring life insurers authorized to do business in Colorado to establish a governance and risk management framework to oversee the use of ECDIS, as well as algorithms and predictive models that use ECDIS.[v] In September 2023, CO DOI released for public comment a proposed quantitative testing regulation establishing a testing regime for life insurers authorized to do business in Colorado to ensure that use of ECDIS, algorithms and predictive models do not result in unfairly discriminatory outcomes.[vi] CO DOI is still working to finalize the quantitative testing regulation. CO DOI is also currently engaging in a stakeholder process to enact regulations specific to the use of artificial intelligence in private passenger auto insurance and health insurance.
b. NAIC
At its 2023 Fall National Meeting, the NAIC adopted a model bulletin, Use of Artificial Intelligence Systems by Insurers (the “Model Bulletin”), outlining how state regulators can use existing statutory authority to govern the development, acquisition and use of artificial intelligence technologies.[vii] The Model Bulletin lays out the types of information and documentation that regulators may request during an investigation or examination. It also requires insurers to develop, implement and maintain a written governance program to ensure that the use of AIS does not violate existing law. To date, 13 jurisdictions have adopted the Model Bulletin with no or minor changes.
c. New York
In 2019, DFS was a first mover among U.S. regulators to impose specific requirements on the use of artificial intelligence in insurance by releasing the 2019 Circular Letter. The 2019 Circular Letter, in short, provided guidance on the use of artificial intelligence in life insurance underwriting, by seeking (i) to mitigate unfair discrimination in life underwriting and (ii) to provide transparency and disclosure to consumers in the event of an adverse underwriting decision.
III. The Circular Letter
The Circular Letter applies to all insurers authorized to write insurance in New York.[viii] It only applies, however, to the use of ECDIS[ix] and AIS[x] in underwriting and pricing, unlike CO DOI’s governance and risk management regulation and the NAIC’s Model Bulletin which apply to all phases of the insurance life cycle. The Circular Letter also emphasizes that DFS has the right to audit and examine an insurer’s use of ECDIS and AIS, within the scope of regular or targeted examinations pursuant to New York Insurance Law.[xi]
a. Fairness Principles
The Circular Letter lays out several “fairness principles” which are meant to guide insurers’ use of ECDIS and AIS in underwriting and pricing.
Proxy Assessment. Insurers should demonstrate that ECDIS do not serve as a proxy for protected classes in a way that is prohibited by law. Insurers should evaluate the extent to which ECDIS are correlated with (i.e., a proxy for) status in a protected class that may result in unfair discrimination. Correlation should be determined using data currently available to the insurer or inferred using accepted statistical methodologies. If correlation is identified, insurers must consider if the use of ECDIS is required for a legitimate business purpose.
The Circular Letter adds to the Draft Circular Letter by specifying the data by which an insurer should determine if the ECDIS serve as a proxy for a protected class.
Unfair and Unlawful Discrimination. Insurers should only use ECDIS or AIS that are established to be not unfairly discriminatory against protected classes.[xii] Insurers should be able to demonstrate that the ECDIS used (i) are supported by accepted actuarial standards, (ii) are based on actual or reasonably anticipated experience and (iii) show a statistically significant, rational and not unfairly discriminatory relationship between the variables used and the relevant risk.
Insurers should use a comprehensive assessment to ensure that underwriting and pricing guidelines are not unfairly discriminatory including, at a minimum:
-
- Step 1: An assessment of whether the use of ECDIS or AIS produces disproportionate adverse effects in underwriting or pricing for similarly situated insureds. This assessment should be conducted for any protected class where membership in such protected class can be determined using data available to the insurer or can be reasonably inferred using accepted statistical methodologies. If there is no prima facie showing of a disproportionate adverse effect, the insurer may conclude its evaluation;
- Step 2: If there is a prima facie showing of a disproportionate adverse effect, the insurer must further assess whether there is a legitimate and lawful rationale for the differential effect. If there is no legitimate and lawful rationale, the insurer must modify its use of such ECDIS or AIS and reevaluate using Step 1; and
- Step 3: If there is a legitimate rationale for the differential effect, insurers should conduct a documented search for a less discriminatory alternative methodology that would reasonably meet the insurer’s legitimate business needs. If such an alternative exists, the insurer should modify its use of ECDIS or AIS accordingly and evaluate the modified use by beginning with Step 1. If no less discriminatory alternative exists, the insurer should conduct ongoing risk management consistent with the requirements in the Circular Letter and repeat Step 3 at least annually.
This requirement is notable compared to other existing regulations and guidance, which prohibit the use of AIS or ECDIS that has an unfairly discriminatory outcome. By contrast, the Circular Letter requires a pre-use assessment to establish that proposed ECDIS or AIS are not unfairly discriminatory.
Documentation. Insurers should document the processes and reasoning behind their testing methodologies and analysis for unfair discrimination, commensurate with the insurer’s use of ECDIS and the complexity and materiality of the ECDIS. Such documentation should be made available to DFS upon request.
Testing. Testing should be administered prior to putting AIS into production and on a regular cadence thereafter, and whenever material updates or changes are made to either ECDIS or AIS.
Testing should include both quantitative and qualitative assessments. For the quantitative assessment, which should demonstrate that internally developed and externally obtained ECDIS and AIS are not unfairly discriminatory, insurers are encouraged to use multiple statistical metrics in evaluating data and model outputs to ensure a comprehensive assessment.[xiii] There is no expectation that insurers collect additional data from, or about, individuals to perform this analysis. For the qualitative assessment, insurers must be able to explain how the AIS operates and a logical relationship between ECDIS and other model variables with an insured’s or potential insured’s individual risk.
The Circular Letter clarifies that the quantitative testing only applies when an insurer has data that it can use to reasonably impute whether an insured or potential insured is a member of a protected class. The Bayesian Improved Surname Geocoding is mentioned as a methodology, but the DFS explicitly does not endorse any particular methodology. The Circular Letter also clarifies that there is no expectation for insurers to collect additional data to perform the testing. The Circular Letter does not prescribe statistical thresholds by which to determine unfair discrimination, leaving it to insurers to make a determination based on the product and use. This contrasts with the CO DOI’s approach, which is grappling with how to prescribe suitable statistical metrics by which to determine unfair discrimination.
b. Governance and Risk Management
The Circular Letter requires existing corporate governance frameworks to be appropriately modified to encompass the insurer’s use of ECDIS and AIS as appropriate for the nature, scale and complexity of the insurer.[xiv]
Board Responsibility. The ultimate oversight responsibility of ECDIS and AIS use rests with the insurer’s board of directors (or equivalent governing body). The board of directors may delegate its management duties to a board committee or senior management. There must exist, however, adequate lines of reporting to meet the board’s information needs.
Senior Management Responsibility. Senior management is responsible for the day-to-day implementation, development and management of ECDIS and AIS. This includes establishing adequate policies, procedures, staff and other actions to ensure proper implementation and use of ECDIS and AIS. To this end, the Circular Letter recommends creating a cross-functional management committee with representatives from key constituencies.
Policies and Procedures. Insurers should formalize their development and management of ECDIS and AIS in written policies and procedures. The policies and procedures should include clearly defined roles and responsibilities, and monitoring, reporting and training requirements. The insurer’s board or senior management (if so delegated) should review and approve these policies and procedures at least annually to ensure they are kept current with the insurer’s use and industry best practices.
Training. Trainings should be in a manner that is appropriately tailored to the individual level of the trainee’s responsibilities, and employees should be held accountable for completing trainings in a timely manner.
Documentation. Insurers should maintain comprehensive documentation for their use of all AIS and ECDIS, whether developed internally or by a third party.[xv] This documentation should be made available to DFS upon request and should include:
-
- a description of the process to identify and assess operational, financial and compliance risks associated with an insurer’s use of ECDIS and AIS and associated internal controls designed to mitigate such risks;
- an up-to-date inventory of all AIS implemented for use, under development or recently retired;
- a description of how each AIS operates, including any use of ECDIS or other inputs and their sources, the purpose and products for which the AIS are designed, actual or expected usage, any restrictions on use and any potential risks and appropriate safeguards;
- a description of the process for tracking changes of an insurer’s use of ECDIS and AIS over time, including an explanation of any changes, rationale for such changes and parties responsible to approve such changes;
- a description of the process to monitor ECDIS and AIS usage and performance, including a list of any previous exceptions to policy and reporting;
- a description of testing conducted at least annually to assess the output of AIS models, including drift that may result from the use of machine learning or other automated updates;
- a description of data lifecycle management processes, including ECDIS acquisition, storage, usage and sharing, archiving and destruction; and
- records of consumer complaints in relation to the use of AIS and ECDIS.
Consumer Complaints. Insurers must implement a system for responding to and addressing consumer complaints and inquiries about the use of AIS and ECDIS.
Risk Management. Insurers should include AIS use within an enterprise risk management function to manage risks and formulate standards for each stage of the AIS life cycle. Insurers must consider risk from individual models as well as in the aggregate. Personnel must also be competent and qualified for their clearly defined roles and responsibilities with appropriate accountability.
Insurers should ensure that their internal audit function is appropriately engaged with the use of ECDIS and AIS, consistent with the relevant risks. The audit should assess the overall effectiveness of the AIS and ECDIS risk management framework, which includes to:
-
- verify that acceptable policies and procedures are in place and are appropriately adhered to;
- verify records of AIS use, that validations are performed in a timely manner and that AIS models are subject to controls that appropriately account for any weaknesses in validation activities;
- assess the accuracy and completeness of AIS documentation and adherence to documentation standards, including risk reporting;
- evaluate the processes for establishing and monitoring internal controls, such as limits on AIS usage;
- assess supporting operational systems and evaluate the accuracy, reliability and integrity of ECDIS and other data used by AIS;
- assess potential biases in ECDIS or other data that may result in unfair or unlawful discrimination against insureds or potential insureds; and
- assess whether there is sufficient reporting to the board and senior management to evaluate whether management is operating within the insurer’s risk appetite and limits for model risk.
c. Third-Party Vendors
The Circular Letter clarifies that compliance by insurers with anti-discrimination laws is irrespective of whether the insurer is itself collecting and using the underlying data or is contracting with external vendors of ECDIS and AIS. Insurers are responsible to understand any ECDIS or AIS used in underwriting and pricing even when developed or deployed by third-party vendors. To ensure appropriate oversight of such third-party vendors, insurers should develop written standards, policies, procedures and protocols for the acquisition, use of or reliance on third-party ECDIS and AIS. Insurers should implement procedures to report and remediate incorrect information from proprietary and third-party AIS. Insurers may not rely solely on the assurance of the third-party vendor or proprietary nature of the third party’s products to determine compliance.
Where appropriate and available, insurers should include terms in contracts with third-party vendors that (i) provide audit rights or entitle the insurer to receive audit reports by qualified auditors, and (ii) require the vendor to cooperate with the insurer regarding regulatory inquiries and investigations related to the insurer’s use of the third-party vendor’s products or services.
Encouraging the inclusion of these terms in contracts with third-party vendors is an addition that was made to the Circular Letter and was borrowed from the NAIC’s Model Bulletin.
d. Transparency
Insurers must provide specific reasons and explanations to insureds or potential insureds when they provide an adverse decision, which must be based on sound actuarial principles.[xvi] When an adverse underwriting or pricing decision is the result of ECDIS or AIS, the reason provided to the insured or potential insured must include in sufficient detail: (i) the specific source of the information upon which the insurer based its decision; (ii) whether the insurer uses AIS in its underwriting or pricing process; (iii) whether the insurer uses data obtained from external vendors; and (iv) that the insured or potential insured has the right to request information about the specific data that resulted in the decision. The Circular Letter specifies that an insurer cannot rely on the proprietary nature of a third-party vendor’s algorithm to justify a lack of specificity related to an adverse decision.
e. Clarification of the 2019 Circular Letter
The Circular Letter provides certain clarifications regarding the 2019 Circular Letter:
-
- If an insurer has threshold criteria for using an accelerated underwriting process based on ECDIS or AIS, the insurer should disclose that in a clear and prominent manner in all relevant advertisements and marketing materials.
- Applicants must be informed if they will not be approved through an accelerated underwriting process. Notice must be provided within 15 days of such a determination and should identify the reasons why. During the notice period, the insurer should continue the non-accelerated underwriting process.
- If an applicant will not be approved through an accelerated underwriting process, the insurer must provide the applicant with a process to review for accuracy the data that resulted in the decision. This process must be provided at the time that the applicant is notified of the decision.
f. Confidentiality
The Circular Letter explains that it does not guarantee confidentiality of information except as provided as an exception under the Freedom of Information Law, Public Officers Law Article 6. At the time an insurer submits information that it deems to be confidential in accordance with the relevant law, the insurer should request that DFS except the information from disclosure.
IV. Conclusion
The Willkie insurance team continues to monitor these efforts to adopt legislation, regulation and guidance on the use of artificial intelligence and big data in the business of insurance and stands ready to advise on the development of risk management, governance and testing structures compliant with these initiatives. Please contact any of the attorneys listed on this client alert if you would like to discuss further.
Click here to download this article.
___________________________
[i] The Circular Letter is available at: https://www.dfs.ny.gov/industry-guidance/circular-letters/cl2024-07#_ednref5.
[ii] Available at: https://www.willkie.com/media/files/publications/2024/01/aiininsuranceupdatenydfsreleases-proposedinsurancecircularletterontheuseofartificialintelligenceinins.pdf.
[iii] The Draft Circular Letter is available at: https://www.dfs.ny.gov/industry_guidance/circular_letters/cl2024_nn_proposed.
[iv] Colo. Rev. Stat. § 10-3-1104.9.
[v] 3 Colo. Code Regs. § 702-10.
[vi] The quantitative testing regulation is available at: https://communications.willkie.com/125/2263/uploads-(icalendars-pdf-documents)/draft-proposed-algorithm-and-predictive-model-quantitative-testing-regulation.pdf.
[vii] The Model Bulletin is available at: https://content.naic.org/sites/default/files/inline-files/2023-12-4%20Model%20Bulletin_Adopted_0.pdf.
[viii] The Circular Letter specifies that it applies to all insurers authorized to write insurance in New York, Article 43 corporations, health maintenance organizations, licensed fraternal benefit societies, and the New York State Insurance Fund but not Child Health Plus, Essential Plan and Medicaid managed care coverage.
[ix] The Circular Letter defines ECDIS as “data or information used – in whole or in part – to supplement traditional medical, property or casualty underwriting or pricing, as a proxy for traditional medical, property or casualty underwriting or pricing, or to identify ‘lifestyle indicators’ that may contribute to an underwriting or pricing assessment of an applicant for insurance coverage. ECDIS does not include an MIB Group, Inc. member information exchange service, a motor vehicle report, prescription drug data, or a criminal history search.”
[x] The Circular Letter defines AIS as “any machine-based system designed to perform functions normally associated with human intelligence, such as reasoning, learning, and self-improvement, that is used – in whole or in part – to supplement traditional health, life, property or casualty underwriting or pricing, as a proxy for traditional health, life, property or casualty underwriting or pricing, or to identify ‘lifestyle indicators’ that may contribute to an underwriting or pricing assessment of an applicant for insurance coverage.”
[xi] See N.Y. Ins. Law §§ 308; 309.
[xii] Protected classes are laid out in N.Y. Ins. Law Article 26 and include, in part, race, color, creed, national origin, disability, sex, marital status, mental disability, children, domestic abuse victims, past lawful travel and status as a living organ or tissue donor. See N.Y. Ins. Law §§ 2606; 2607; 2608; 2608-a; 2612; 2614; 2616.
[xiii] The Circular Letter provides examples of these metrics:
- Adverse Impact Ratio: Analyzing the rates of favorable outcomes between protected classes and control groups to identify any disparities.
- Denials Odds Ratios: Computing the odds of adverse decisions for protected classes compared to control groups.
- Marginal Effects: Assessing the effect of a marginal change in a predictive variable on the likelihood of unfavorable outcomes, particularly for members of protected classes.
- Standardized Mean Differences: Measuring the difference in average outcomes between protected classes and control groups.
- Z-tests and T-tests: Conducting statistical tests to ascertain whether differences in outcomes between protected classes and control groups are statistically significant.
- Drivers of Disparity: Identifying variables in AIS that cause differences in outcomes for protected classes relative to control groups. These drivers can be quantitatively computed or estimated using various methods, such as sensitivity analysis, Shapley values, regression coefficients, or other suitable explanatory techniques.
[xiv] See N.Y. Comp. Codes R. & Regs. tit. 11, § 90.2 et seq.
[xv] Such documentation must be compliant with the record retention requirements of N.Y. Comp. Codes R. & Regs. tit. 11, § 243.0 et seq.
[xvi] See N.Y. Ins. Law §§ 3425; 3426; 4224.
