In California’s 2023-24 legislative session, Governor Gavin Newsom signed 17 artificial intelligence (AI) bills into law, governing matters such as deepfakes, AI watermarking, digital replicas of entertainers, and training data disclosures.[1] While this means that California may now have more laws regulating AI than any other state, its approach stands in stark contrast to the comprehensive, industry-agnostic approach taken by other jurisdictions, such as Colorado, Utah, and the European Union. As a result, companies operating in California will need to take a close look at how and whether these varied provisions apply to them, and companies around the U.S. should be prepared as other states in the rest of 2024 and into 2025 consider taking an approach similar to what California has taken in 2024.
Below, we have highlighted three of the more notable AI-related laws signed by Governor Newsom that we expect to have a broad, though not necessarily immediately obvious, impact on the private sector.
Digital Replicas – AB 2602 (effective Jan. 1, 2025)[2]
Given the importance of the entertainment industry to California’s economy, and last year’s actors strike, it is unsurprising that one of the AI-related bills is designed to protect California actors and performers by ensuring they can control the use of their own digital replicas. AB 2602 amends California Labor Code § 927, and applies broadly to any contract “between an individual and any other person for the performance of personal or professional services.”[3] In particular, AB 2602 aims to protect performers by invalidating contracts that:
1) allow for the creation of a digital replica to perform work that could have been done by the performer;
2) fail to specifically describe the intended uses of the digital replica; and
3) were negotiated without legal and/or union representation.
While AB 2602 has been lauded by the actor’s union, SAG-AFTRA, it could also impact many companies outside Hollywood.[4] For example, these provisions may apply to companies seeking to use digital replicas for advertising, automated customer service bots, video games, or even something as mundane as corporate training videos.
Personal Information – AB 1008 (effective Jan. 1, 2025)[5]
One of the shorter AI bills signed into law in 2024, AB 1008 amends the California Consumer Privacy Act (“CCPA”) to clarify that personal information “can exist in various formats, including . . . artificial intelligence systems that are capable of outputting personal information.” Despite its brevity, AB 1008 could be the provision with the greatest impact. This update to the CCPA means that any company employing AI needs to be aware that its AI system could generate information that California would consider subject to the protection of the CCPA, and the consumer rights provided thereto. For instance, as the CCPA provides California residents with the right to request the correction of any inaccurate personal information, businesses will need to develop and implement means to correct AI hallucinations.
Training Data Disclosures – AB 2013 (effective Jan. 1, 2026)[6]
AB 2013 is designed to bring transparency to the development of generative AI systems. Specifically, it requires a developer of a generative AI system, or someone that substantially modifies such a system, to post a high-level summary on its website regarding the data used to train the generative AI system or service before such a system or service is made available to Californians. The new law requires that this high-level documentation disclose, for example:
-
- the sources or owners of the underlying datasets, and whether the developer purchased or licensed those datasets;
- whether the datasets include any data protected by copyright, trademark, or patent, or whether the datasets are entirely in the public domain;
- whether the datasets include personal information; and
- the period during which the data in the datasets were collected, or whether data collection is ongoing.
The law’s detailed disclosure requirements still leave a fair amount of room for interpretation, and it remains to be seen how helpful or comprehensive these disclosures will be in practice. That said, this law could provide a basis for businesses to request documentation from certain AI system vendors prior to agreeing to deploy a generative AI system, and could provide an additional vehicle for rights holders and plaintiffs’ attorneys to challenge unauthorized use of certain data.[7]
Other AI Laws Signed By Newsom
In addition to the laws discussed above, Governor Newsom signed a number of other AI-related laws which may impact the private sector:
Defining Artificial Intelligence
-
- AB 2885 (effective Jan. 1, 2025).[8] Defines AI, for the purpose of California law, as “an engineered or machine-based system that varies in its level of autonomy and that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.” This could be helpful insofar as it establishes a common definition across California law for this baseline concept.
Misinformation and Election Integrity
-
- AB 2355 (effective Jan. 1, 2025).[9] Requires that political action committees disclose whether political advertisements have been generated or substantially altered by AI.
- AB 2655 (effective Jan. 1, 2025).[10] Requires large online platforms, during specified periods leading up to elections, to: (i) block materially deceptive content related to California elections; (ii) label certain additional content as inauthentic or fake; and (iii) develop procedures for California residents to report content that has not been otherwise blocked or labeled.
- AB 2839 (effective immediately).[11] Prohibits knowingly distributing (e.g., posting or reposting on social media), with malice, materially deceptive content such as deepfakes within 120 days before an election in California and, in certain circumstances, 60 days after an election. AB 2839 took immediate effect on September 17, 2024, but on October 2, 2024, U.S. District Judge Mendez in the Eastern District of California ordered a preliminary injunction to temporarily block the enforcement of almost all aspects of AB 2839, because it was not sufficiently narrowly tailored.[12]
Deepfakes
-
- AB 1831 (effective Jan. 1, 2025).[13] Expands the scope of existing child pornography laws to include matter that is digitally altered or generated by AI systems.
- SB 926 (effective Jan. 1, 2025).[14] Criminalizes the creation and distribution of deepfake pornography that reasonably depicts another person.
- SB 981 (effective Jan. 1, 2025).[15] Requires social media platforms to establish channels for users to report sexually explicit digital replicas, and enable the content to be temporarily blocked while the platform determines whether permanent removal is required.
Digital Replicas / Entertainment Industry
-
- AB 1836 (effective Jan. 1, 2025).[16] Prohibits the creation of digital replicas of deceased performers in advertising or “expressive audiovisual works or sound recordings” (with limited First Amendment exceptions) without consent from their estates.
Healthcare
-
- AB 3030 (effective Jan. 1, 2025).[17] Requires healthcare facilities, clinics, physician’s offices, or healthcare practice groups that use generative AI systems to provide written or verbal communications regarding patient information to include (i) a disclaimer indicating that the communication was generated by AI, and (ii) clear instructions describing how a patient may contact a human, unless such communications are read and reviewed by a human licensed or certified healthcare provider. Utah’s recently-enacted AI Policy Act similarly requires healthcare providers to make disclosures to their patients when they interact with a generative AI system.[18]
- SB 1120 (effective Jan. 1, 2025).[19] Requires AI or other software that healthcare service providers and insurers use for utilization review or utilization management functions to comply with numerous requirements, including that the AI system bases its determinations on specified information that is fairly and equitably applied.
Robocalls
-
- AB 2905 (effective Jan. 1, 2025).[20] Requires all automatic dialing devices to inform the person called if the prerecorded messages uses an artificial voice.
Watermarks and AI Detection
- SB 942 (effective Jan. 1, 2025).[21] Requires covered providers of generative AI (those with more than 1,000,000 monthly visitors) to make tools to allow users to detect AI-generated content and to mark content as AI-generated, and include disclosures embedded in AI-generated content that identifies the system that generated the content. Along with Utah’s AI Policy Act, SB 942 marks a trend in state legislation requiring companies to notify individuals when they interact with AI systems or AI-generated content.[22] The Attorney General, a city attorney, or a county counsel may seek a civil penalty of $5,000 per violation of the bill.
Veto of the Foundation Model AI Safety Bill
While Governor Newsom signed numerous AI-related bills into law, he vetoed the most heavily lobbied bill, the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act (SB 1047). This bill would have imposed a series of safety measures on large AI models to mitigate potential “critical harms” like the creation of biological or chemical weapons, or a large-scale cyber-attack against critical infrastructure.[23] The bill divided some of California’s most powerful players: California’s Bay Area representatives in Washington, and leading AI Company OpenAI opposed the bill, while Tesla and SpaceX founder Elon Musk publicly endorsed the bill.[24] Ultimately, Governor Newsom vetoed it. As set forth in his veto statement, he concluded that the bill, as drafted, was ineffective because:
-
- it focused only on the most expensive and large-scale models, and could give the public a false sense of security about controlling AI;
- it did not take into account whether an AI system was deployed in high-risk environments, involved critical decision-making, or used sensitive data; and
- was not informed by an empirical trajectory analysis of AI systems and capabilities.[25]
Conclusion
This is only the beginning of AI regulation in California. California lawmakers introduced 65 AI-related bills during the 2023-24 legislative session–of which only 17 received Newsom’s signature.[26] While Governor Newsom vetoed SB 1047, a similar bill could resurface in the next legislative session as the Governor stated that he “will continue to work with the Legislature on this critical matter during its next session,” and he has asked leading experts on generative AI to help California develop workable guardrails for deploying an empirical, science-based trajectory analysis for frontier models and their capabilities and risks.[27] For now, there are plenty of AI-related laws to keep companies busy, and it will be critical for anyone developing or deploying AI systems or products to carefully watch the development of California’s piecemeal approach to AI legislation.
Click here to download this article.
________________________________
[1] Please note that two of the 17 bills signed by Governor Newsom apply only to the California government, so we do not discuss them herein.
[2] A.B. 2602, 2023-24 Sess. (Cal. 2024).
[3] A.B. 2602 § 927(a).
[4] Cristi Carras, SAG-AFTRA Celebrates Passage of California AI Bill Regulating Use of Digital Replicas, LA Times, (Aug. 28, 2024), https://www.latimes.com/entertainment-arts/business/story/2024-08-28/ab-2602-sag-aftra-ai-bill-california.
[5] A.B. 1008, 2023-24 Sess. (Cal. 2024).
[6] A.B. 2013, 2023-24 Sess. (Cal. 2024).
[7] Id; Assembly Committee on Privacy and Consumer Protection, AB 2013 – As Amended April 22, 2024, April 30, 2024 (Stating that the bill does not outline specific enforcement mechanisms for its provisions, and therefore enforcement will likely occur on the basis of California’s Unfair Competition Law. The bill does not preclude a private right of action.), https://apcp.assembly.ca.gov/system/files/2024-04/ab-2013-irwin-apcp-analysis.pdf#page=9.
[8] A.B. 2885, 2023-24 Sess. (Cal. 2024).
[9] A.B. 2355, 2023-24 Sess. (Cal. 2024).
[10] A.B. 2655, 2023-24 Sess. (Cal. 2024).
[11] A.B. 2839, 2023-24 Sess. (Cal. 2024).
[12] Christopher Kohls v. Rob Bonta, Case No. 2:24-cv-02527-JAM-CKD (E.D. Cal. 2024) (Finding that California has a valid interest in protecting the integrity and reliability of the electoral process, the court held that AB 2839 is unconstitutional because it lacks the narrow tailoring and least restrictive alternative required for laws that restrict political speech).
[13] A.B. 1831, 2023-24 Sess. (Cal. 2024).
[14] S.B. 926, 2023-24 Sess. (Cal. 2024).
[15] S.B. 981, 2023-24 Sess. (Cal. 2024).
[16] A.B. 1836, 2023-24 Sess. (Cal. 2024).
[17] A.B. 3030, 2023-24 Sess. (Cal. 2024).
[18] S.B. 149 Artificial Intelligence Amendments, §§ 13-2-12(1)(c), 13-2-12(5).
[19] S.B. 1120, 2023-24 Sess. (Cal. 2024).
[20] A.B. 2905, 2023-24 Sess. (Cal. 2024).
[21] S.B. 942, 2023-24 Sess. (Cal. 2024); A “covered provider” is defined as a person that creates, codes, or otherwise produces a generative AI system that has over 1,000,000 monthly visitors or users and is publicly accessible within the geographic boundaries of the state.
[22] S.B. 149 Artificial Intelligence Amendments, § 13-2-12(3).
[23] S.B. 1047, 2023-24 Sess. (Cal. 2024); Safety measures included the implementation of full shutdown mechanisms, written safety and security protocols, and detailed assessments.
[24] Elon Musk Voices Support for California Bill Requiring Safety Tests on AI Models, Reuters (Aug. 27, 2024), https://www.reuters.com/technology/artificial-intelligence/elon-musk-voices-support-california-bill-requiring-safety-tests-ai-models-2024-08-27; see Kyle Orland, California Legislature Passes Controversial “Kill Switch” AI Safety Bill, ARS Technica, (Aug. 29, 2024), https://arstechnica.com/ai/2024/08/as-contentious-california-ai-safety-bill-passes-critics-push-governor-for-veto.
[25] Governor Gavin Newsom, SB 1047 Veto Message, Office of the Governor (Sept. 29, 2024) https://www.gov.ca.gov/wp-content/uploads/2024/09/SB-1047-Veto-Message.pdf; While Governor Newsom stated that a California-only approach may be warranted absent Congressional action on AI, he highlighted that any regulatory decision should be based on empirical evidence and science. Such evidence may come from the U.S. AI Safety Institute, under the National Institute of Science and Technology, or agencies within Newsom’s Administration that are performing risk analyses on potential threats and vulnerabilities to California’s critical infrastructure.
[26] See Elon Musk Voices Support for California Bill Requiring Safety Tests on AI Models, Reuters (Aug. 27, 2024), https://www.reuters.com/technology/artificial-intelligence/elon-musk-voices-support-california-bill-requiring-safety-tests-ai-models-2024-08-27; Press Release, Governor Newsom Announces New Initiatives to Advance Safe and Responsible AI, Protect Californians (Sept. 29, 2024), https://www.gov.ca.gov/2024/09/29/governor-newsom-announces-new-initiatives-to-advance-safe-and-responsible-ai-protect-californians/.
[27] Press Release, Governor Newsom Announces New Initiatives to Advance Safe and Responsible AI, Protect Californians (Sept. 29, 2024), https://www.gov.ca.gov/2024/09/29/governor-newsom-announces-new-initiatives-to-advance-safe-and-responsible-ai-protect-californians/.
