1# CSAM guidance
2
3> For the complete documentation index, see [llms.txt](/llms.txt). Markdown versions of documentation pages are available by appending `.md` to the page URL.
4
5{/* This guide necessarily discusses sexual abuse, so these profanity heuristics don't apply. */}
6{/* vale alex.ProfanityMaybe = NO */}
7{/* vale alex.ProfanityUnlikely = NO */}
8{/* "Potentially" preserves uncertainty in classifier and policy language. */}
9{/* vale Microsoft.Adverbs = NO */}
10
11
12
13 {"OpenAI developed this resource with expert input from the "}
14 {", "}
15 {", the "}
16 {", and the "}
17 {"."}
18
19
20
21## Build with child safety in mind
22
23OpenAI has clear child safety expectations for developers:
24You are responsible for ensuring that your users use OpenAI services in
25compliance with applicable laws, including laws that criminalize child sexual
26abuse and exploitation. Never use OpenAI services to exploit, endanger, or
27sexualize anyone under the age of 18. See the OpenAI .
28
29Online child sexual exploitation and abuse affects a range of products and
30services, including those that don't target children. OpenAI wants to help
31developers understand what actions to consider taking to address this abuse.
32
33From the earliest possible stage, consider how people could misuse your product.
34Start early so child safety safeguards can scale with you, rather than becoming
35something you try to retrofit into an already complex product or system.
36Developer teams and organizations of all sizes should assess how people could
37misuse their products for a range of harms, including child sexual abuse material
38(CSAM), grooming, sexual extortion, the sexualization of children, livestreamed
39abuse, and trafficking—especially if their products support messaging, content
40uploads, image editing, livestreaming, discovery, or payments.
41
42This resource focuses on CSAM and offers developers practical guidance for
43protecting children.
44
45## Where to start
46
47It can be difficult to know where to start. The right solutions and
48implementation paths depend on your organization's size, maturity, and available
49resources.
50
51The following checklist is a good starting point for addressing CSAM. The
52important thing is to begin addressing the risk: Don't wait until you have every
53tool or step complete before taking action.
54
55## Prevent
56
57Set clear rules for your product or service, and establish mechanisms for
58hearing from your users about their experiences.
59
60- **Set clear rules.** Prohibit child sexual exploitation and abuse in your terms
61 of service, acceptable use policy, or community guidelines. Learn more from
62 or the Tech Coalition's free
63 for expert guidance and practical tools, including a resource on external
64 standards that prohibit online child sexual exploitation and abuse.
65- **Make reporting available to your users.** Give users a visible way to flag
66 potentially harmful content or behavior, and route those concerns to a monitored queue
67 or location with enough information to make policy decisions. For more
68 guidance, see the Australian .
69- **Track uploads and users through safety identifiers.** In your product or service,
70 associate every upload with a user. Sending
71 [safety identifiers with supported OpenAI
72 requests](https://developers.openai.com/api/docs/guides/safety-best-practices#implement-safety-identifiers)
73 can help OpenAI monitor and detect abuse. This can help OpenAI provide your team with more
74 actionable feedback if OpenAI detects policy violations in your
75 product or service. Safety identifiers can also help your team respond to abuse
76 faster. They create a stable way to trace activity back to an individual end
77 user and reduce the chance that one user's misuse disrupts access for your
78 broader organization. Use a unique string to represent each user. To protect
79 privacy, hash email addresses or usernames before sending them to avoid
80 disclosing personally identifiable information. The direct Images API uses a
81 different parameter for this purpose: Send the same stable identifier as
82 `user` for [image
83 generation](https://developers.openai.com/api/reference/resources/images/methods/generate) and [image
84 edits](https://developers.openai.com/api/reference/resources/images/methods/edit).
85
86When you are ready to do more, consider other prevention measures:
87
88- Add prevention-focused blocking or refusals for CSAM-related keywords
89 or URLs. Learn more from the .
90- Provide warnings or messages to users who are attempting to engage in
91 CSAM-related behavior. Learn more from .
92- If your service is available to children and a user tells you that sexual
93 images or videos of them are on your platform, direct them to support services
94 such as the IWF and NSPCC's program, which
95 enables children in the UK to report images and videos for removal or
96 blocking, or NCMEC's service.
97- If you become aware that a child is facing immediate or imminent harm:
98 - Refer the case to emergency services, such as local law enforcement.
99 - Give the child information on how to contact emergency services.
100
101## Detect
102
103Identify potentially violative content or conduct for review and action.
104
105Use accessible CSAM detection tools:
106
107- **Perceptual hash matching.** If your product supports uploading, storing, or
108 processing visual media, consider implementing perceptual hash matching. This
109 technology creates a digital fingerprint of an image or video and compares it
110 with fingerprints of known CSAM, helping to identify known
111 material even after someone resizes, compresses, or otherwise modifies a file.
112 You need access to both hash-matching technology, such as PhotoDNA, and CSAM
113 hash lists, which are repositories of known CSAM hashes. Not all services
114 provide both.
115 - **Recommended hash-matching technologies:**
116 - offers purpose-built detection for known CSAM in images and videos.
117 - The Tech Coalition offers eligible companies sublicenses to through its free
118 .
119 - .
120 - YouTube's technology
121 provides hash matching to identify known CSAM in videos.
122 - **Recommended CSAM hash lists:**
123 - NCMEC offers lists of known CSAM, exploitative content, and generative AI
124 CSAM. Contact its [Electronic Service Provider team](mailto:espteam@ncmec.org).
125 - IWF offers , a
126 hash-matching service for eligible small businesses and startups that
127 requires minimal technical expertise to integrate.
128- **Novel CSAM detection classifiers.** These tools can detect unknown or unseen
129 CSAM.
130 - offers
131 classifiers intended to identify potentially novel CSAM in images and
132 videos, as well as relevant text-based exploitation signals.
133 - Google's uses AI to
134 classify images and videos and assign their review priority. The higher the
135 priority assigned by the classifier, the more likely the media contains
136 CSAM.
137- **Bad actor detection and enforcement.** Using available identifiers and
138 information—such as usernames, email addresses, and device IDs—consider
139 permanently banning or otherwise disrupting accounts that people have used to
140 violate, or attempt to violate, CSAM policies. Watch for repeat offenders who
141 attempt to circumvent enforcement.
142
143## Respond and report
144
145Make sure your product and team can take appropriate action when you become
146aware of CSAM on your service.
147
148- **Register and prepare to report CSAM to the authorities.** or your . Consider what makes a report to NCMEC
149 actionable and how it can support child safeguarding. Include as much
150 information as possible to help route the report to the appropriate
151 jurisdiction and identify the suspect. NCMEC's includes IP addresses, device IDs, and other data.
152
153Local laws and reporting obligations may vary by jurisdiction.
154
155- **Keep usable records and identifiers.** When you make a report, maintain
156 documentation of the incident and any associated data that could help identify
157 violative actors, so you can respond to requests from law enforcement.
158- **Write a response playbook.** Define who reviews reports, how to escalate
159 urgent cases internally and externally, what actions to take against users
160 responsible for violations, and who can make those decisions. This can help you establish enforcement operations.
161- Consider establishing a network of trusted expert reporters, including
162 organizations such as IWF and other hotlines, that can use their expertise to
163 flag CSAM cases for you.
164- **Train and support the people involved.** It's important for humans to be in
165 the loop. Make sure reviewers, support teams, and on-call staff understand your
166 policy, escalation path, and the limits of any automated system.
167- **Use the tools you need to moderate content or respond to abuse.** Other
168 tools can help you address child safety risks and harms:
169 - The [Moderation API](https://developers.openai.com/api/docs/guides/moderation) detects potentially harmful
170 content in text and images. Learn more about the
171 [`omni-moderation-latest`](https://developers.openai.com/api/docs/models/omni-moderation-latest) model.
172 This isn't a substitute for dedicated CSAM detection. It still includes a
173 `sexual/minors` category covering sexual content involving people
174 under 18; this category is text-only. Use results to:
175 - Block or filter content.
176 - Send content for human review.
177 - Intervene on an account.
178 - Add friction to repeated misuse and apply product- or service-specific
179 enforcement.
180 - Consider other moderation tools that could help. For example, is an open-source review console for
181 triaging potential policy violations in text, multimedia, and profiles. It
182 supports human and automated review, takes a wellness-oriented approach for
183 reviewers, and enables end-to-end moderation workflows, including
184 NCMEC CyberTipline reporting.
185
186When you are ready to do more, consider other response and reporting
187measures:
188
189- **Support the people involved in tackling CSAM.** Organizations should invest
190 in training, support, and a well-being program for CSAM reviewers. Read
191 .
192- **Get specialist support.** You don't need a large trust and safety team to
193 start. The Tech Coalition offers ways for companies to build stronger child
194 safety systems:
195 - is a free capacity-building program designed especially for startups and
196 small and midsize platforms, while remaining open to companies of all sizes.
197 It provides practical resources, guidance, and support to help companies
198 establish strong child safety foundations. Eligible companies can also apply
199 for a PhotoDNA sublicense through Pathways.
200 - provides tailored consulting and implementation support for companies
201 seeking more hands-on help to strengthen their child safety programs and
202 respond to specific risks.
203 - enables companies to take part in the industry's global collaborative
204 response to online child sexual exploitation and abuse, engage with peers,
205 share expertise, and contribute to collective action. Contact the [Tech
206 Coalition team](mailto:md@technologycoalition.org) for an initial
207 consultation.
208 - Developers can use the Tech Coalition's for further guidance.
209
210### Scale safeguards
211
212The right controls and safeguards depend on the product, its development stage
213and maturity, its users and features, the regions in which it operates, and its
214available resources.
215
216Products with greater risk exposure—for example, those that support
217livestreaming, image generation or editing, file storage, or private
218connections—should consider implementing and strengthening safeguards such as:
219
220- Product risk assessments before launch and whenever high-risk features change.
221 See .
222- Layered detection appropriate to the service, which may include hash matching,
223 image or video classifiers, text signals, keyword detection, and URL blocking.
224- Human review of high-confidence or high-severity signals, using tools that
225 protect reviewer well-being and limit unnecessary exposure to harmful
226 material.
227- Rate limits, account controls, and abuse monitoring that make repeat misuse
228 harder.
229- Regular testing and measurement so you can find gaps, track outcomes, and
230 improve your controls.
231
232These recommendations are a starting point, not legal advice or a universal
233standard of care. Adapt them to your service, risk profile, and applicable law.