cli/resources/moderations/index.md +0 −638 deleted
File Deleted View Diff
1# Moderations
2
3## Create moderation
4
5`$ openai moderations create`
6
7**post** `/moderations`
8
9Classifies if text and/or image inputs are potentially harmful. Learn
10more in the [moderation guide](https://platform.openai.com/docs/guides/moderation).
11
12### Parameters
13
14- `--input: string or array of string or array of ModerationMultiModalInput`
15
16 Input (or inputs) to classify. Can be a single string, an array of strings, or
17 an array of multi-modal input objects similar to other models.
18
19- `--model: optional string or ModerationModel`
20
21 The content moderation model you would like to use. Learn more in
22 [the moderation guide](https://platform.openai.com/docs/guides/moderation), and learn about
23 available models [here](https://platform.openai.com/docs/models#moderation).
24
25### Returns
26
27- `ModerationNewResponse: object { id, model, results }`
28
29 Represents if a given text input is potentially harmful.
30
31 - `id: string`
32
33 The unique identifier for the moderation request.
34
35 - `model: string`
36
37 The model used to generate the moderation results.
38
39 - `results: array of Moderation`
40
41 A list of moderation objects.
42
43 - `categories: object { harassment, "harassment/threatening", hate, 10 more }`
44
45 A list of the categories, and whether they are flagged or not.
46
47 - `harassment: boolean`
48
49 Content that expresses, incites, or promotes harassing language towards any target.
50
51 - `harassment/threatening: boolean`
52
53 Harassment content that also includes violence or serious harm towards any target.
54
55 - `hate: boolean`
56
57 Content that expresses, incites, or promotes hate based on race, gender, ethnicity, religion, nationality, sexual orientation, disability status, or caste. Hateful content aimed at non-protected groups (e.g., chess players) is harassment.
58
59 - `hate/threatening: boolean`
60
61 Hateful content that also includes violence or serious harm towards the targeted group based on race, gender, ethnicity, religion, nationality, sexual orientation, disability status, or caste.
62
63 - `illicit: boolean`
64
65 Content that includes instructions or advice that facilitate the planning or execution of wrongdoing, or that gives advice or instruction on how to commit illicit acts. For example, "how to shoplift" would fit this category.
66
67 - `illicit/violent: boolean`
68
69 Content that includes instructions or advice that facilitate the planning or execution of wrongdoing that also includes violence, or that gives advice or instruction on the procurement of any weapon.
70
71 - `self-harm: boolean`
72
73 Content that promotes, encourages, or depicts acts of self-harm, such as suicide, cutting, and eating disorders.
74
75 - `self-harm/instructions: boolean`
76
77 Content that encourages performing acts of self-harm, such as suicide, cutting, and eating disorders, or that gives instructions or advice on how to commit such acts.
78
79 - `self-harm/intent: boolean`
80
81 Content where the speaker expresses that they are engaging or intend to engage in acts of self-harm, such as suicide, cutting, and eating disorders.
82
83 - `sexual: boolean`
84
85 Content meant to arouse sexual excitement, such as the description of sexual activity, or that promotes sexual services (excluding sex education and wellness).
86
87 - `sexual/minors: boolean`
88
89 Sexual content that includes an individual who is under 18 years old.
90
91 - `violence: boolean`
92
93 Content that depicts death, violence, or physical injury.
94
95 - `violence/graphic: boolean`
96
97 Content that depicts death, violence, or physical injury in graphic detail.
98
99 - `category_applied_input_types: object { harassment, "harassment/threatening", hate, 10 more }`
100
101 A list of the categories along with the input type(s) that the score applies to.
102
103 - `harassment: array of "text"`
104
105 The applied input type(s) for the category 'harassment'.
106
107 - `"text"`
108
109 - `harassment/threatening: array of "text"`
110
111 The applied input type(s) for the category 'harassment/threatening'.
112
113 - `"text"`
114
115 - `hate: array of "text"`
116
117 The applied input type(s) for the category 'hate'.
118
119 - `"text"`
120
121 - `hate/threatening: array of "text"`
122
123 The applied input type(s) for the category 'hate/threatening'.
124
125 - `"text"`
126
127 - `illicit: array of "text"`
128
129 The applied input type(s) for the category 'illicit'.
130
131 - `"text"`
132
133 - `illicit/violent: array of "text"`
134
135 The applied input type(s) for the category 'illicit/violent'.
136
137 - `"text"`
138
139 - `self-harm: array of "text" or "image"`
140
141 The applied input type(s) for the category 'self-harm'.
142
143 - `"text"`
144
145 - `"image"`
146
147 - `self-harm/instructions: array of "text" or "image"`
148
149 The applied input type(s) for the category 'self-harm/instructions'.
150
151 - `"text"`
152
153 - `"image"`
154
155 - `self-harm/intent: array of "text" or "image"`
156
157 The applied input type(s) for the category 'self-harm/intent'.
158
159 - `"text"`
160
161 - `"image"`
162
163 - `sexual: array of "text" or "image"`
164
165 The applied input type(s) for the category 'sexual'.
166
167 - `"text"`
168
169 - `"image"`
170
171 - `sexual/minors: array of "text"`
172
173 The applied input type(s) for the category 'sexual/minors'.
174
175 - `"text"`
176
177 - `violence: array of "text" or "image"`
178
179 The applied input type(s) for the category 'violence'.
180
181 - `"text"`
182
183 - `"image"`
184
185 - `violence/graphic: array of "text" or "image"`
186
187 The applied input type(s) for the category 'violence/graphic'.
188
189 - `"text"`
190
191 - `"image"`
192
193 - `category_scores: object { harassment, "harassment/threatening", hate, 10 more }`
194
195 A list of the categories along with their scores as predicted by model.
196
197 - `harassment: number`
198
199 The score for the category 'harassment'.
200
201 - `harassment/threatening: number`
202
203 The score for the category 'harassment/threatening'.
204
205 - `hate: number`
206
207 The score for the category 'hate'.
208
209 - `hate/threatening: number`
210
211 The score for the category 'hate/threatening'.
212
213 - `illicit: number`
214
215 The score for the category 'illicit'.
216
217 - `illicit/violent: number`
218
219 The score for the category 'illicit/violent'.
220
221 - `self-harm: number`
222
223 The score for the category 'self-harm'.
224
225 - `self-harm/instructions: number`
226
227 The score for the category 'self-harm/instructions'.
228
229 - `self-harm/intent: number`
230
231 The score for the category 'self-harm/intent'.
232
233 - `sexual: number`
234
235 The score for the category 'sexual'.
236
237 - `sexual/minors: number`
238
239 The score for the category 'sexual/minors'.
240
241 - `violence: number`
242
243 The score for the category 'violence'.
244
245 - `violence/graphic: number`
246
247 The score for the category 'violence/graphic'.
248
249 - `flagged: boolean`
250
251 Whether any of the below categories are flagged.
252
253### Example
254
255```cli
256openai moderations create \
257 --api-key 'My API Key' \
258 --input 'I want to kill them.'
259```
260
261#### Response
262
263```json
264{
265 "id": "id",
266 "model": "model",
267 "results": [
268 {
269 "categories": {
270 "harassment": true,
271 "harassment/threatening": true,
272 "hate": true,
273 "hate/threatening": true,
274 "illicit": true,
275 "illicit/violent": true,
276 "self-harm": true,
277 "self-harm/instructions": true,
278 "self-harm/intent": true,
279 "sexual": true,
280 "sexual/minors": true,
281 "violence": true,
282 "violence/graphic": true
283 },
284 "category_applied_input_types": {
285 "harassment": [
286 "text"
287 ],
288 "harassment/threatening": [
289 "text"
290 ],
291 "hate": [
292 "text"
293 ],
294 "hate/threatening": [
295 "text"
296 ],
297 "illicit": [
298 "text"
299 ],
300 "illicit/violent": [
301 "text"
302 ],
303 "self-harm": [
304 "text"
305 ],
306 "self-harm/instructions": [
307 "text"
308 ],
309 "self-harm/intent": [
310 "text"
311 ],
312 "sexual": [
313 "text"
314 ],
315 "sexual/minors": [
316 "text"
317 ],
318 "violence": [
319 "text"
320 ],
321 "violence/graphic": [
322 "text"
323 ]
324 },
325 "category_scores": {
326 "harassment": 0,
327 "harassment/threatening": 0,
328 "hate": 0,
329 "hate/threatening": 0,
330 "illicit": 0,
331 "illicit/violent": 0,
332 "self-harm": 0,
333 "self-harm/instructions": 0,
334 "self-harm/intent": 0,
335 "sexual": 0,
336 "sexual/minors": 0,
337 "violence": 0,
338 "violence/graphic": 0
339 },
340 "flagged": true
341 }
342 ]
343}
344```
345
346## Domain Types
347
348### Moderation
349
350- `moderation: object { categories, category_applied_input_types, category_scores, flagged }`
351
352 - `categories: object { harassment, "harassment/threatening", hate, 10 more }`
353
354 A list of the categories, and whether they are flagged or not.
355
356 - `harassment: boolean`
357
358 Content that expresses, incites, or promotes harassing language towards any target.
359
360 - `harassment/threatening: boolean`
361
362 Harassment content that also includes violence or serious harm towards any target.
363
364 - `hate: boolean`
365
366 Content that expresses, incites, or promotes hate based on race, gender, ethnicity, religion, nationality, sexual orientation, disability status, or caste. Hateful content aimed at non-protected groups (e.g., chess players) is harassment.
367
368 - `hate/threatening: boolean`
369
370 Hateful content that also includes violence or serious harm towards the targeted group based on race, gender, ethnicity, religion, nationality, sexual orientation, disability status, or caste.
371
372 - `illicit: boolean`
373
374 Content that includes instructions or advice that facilitate the planning or execution of wrongdoing, or that gives advice or instruction on how to commit illicit acts. For example, "how to shoplift" would fit this category.
375
376 - `illicit/violent: boolean`
377
378 Content that includes instructions or advice that facilitate the planning or execution of wrongdoing that also includes violence, or that gives advice or instruction on the procurement of any weapon.
379
380 - `self-harm: boolean`
381
382 Content that promotes, encourages, or depicts acts of self-harm, such as suicide, cutting, and eating disorders.
383
384 - `self-harm/instructions: boolean`
385
386 Content that encourages performing acts of self-harm, such as suicide, cutting, and eating disorders, or that gives instructions or advice on how to commit such acts.
387
388 - `self-harm/intent: boolean`
389
390 Content where the speaker expresses that they are engaging or intend to engage in acts of self-harm, such as suicide, cutting, and eating disorders.
391
392 - `sexual: boolean`
393
394 Content meant to arouse sexual excitement, such as the description of sexual activity, or that promotes sexual services (excluding sex education and wellness).
395
396 - `sexual/minors: boolean`
397
398 Sexual content that includes an individual who is under 18 years old.
399
400 - `violence: boolean`
401
402 Content that depicts death, violence, or physical injury.
403
404 - `violence/graphic: boolean`
405
406 Content that depicts death, violence, or physical injury in graphic detail.
407
408 - `category_applied_input_types: object { harassment, "harassment/threatening", hate, 10 more }`
409
410 A list of the categories along with the input type(s) that the score applies to.
411
412 - `harassment: array of "text"`
413
414 The applied input type(s) for the category 'harassment'.
415
416 - `"text"`
417
418 - `harassment/threatening: array of "text"`
419
420 The applied input type(s) for the category 'harassment/threatening'.
421
422 - `"text"`
423
424 - `hate: array of "text"`
425
426 The applied input type(s) for the category 'hate'.
427
428 - `"text"`
429
430 - `hate/threatening: array of "text"`
431
432 The applied input type(s) for the category 'hate/threatening'.
433
434 - `"text"`
435
436 - `illicit: array of "text"`
437
438 The applied input type(s) for the category 'illicit'.
439
440 - `"text"`
441
442 - `illicit/violent: array of "text"`
443
444 The applied input type(s) for the category 'illicit/violent'.
445
446 - `"text"`
447
448 - `self-harm: array of "text" or "image"`
449
450 The applied input type(s) for the category 'self-harm'.
451
452 - `"text"`
453
454 - `"image"`
455
456 - `self-harm/instructions: array of "text" or "image"`
457
458 The applied input type(s) for the category 'self-harm/instructions'.
459
460 - `"text"`
461
462 - `"image"`
463
464 - `self-harm/intent: array of "text" or "image"`
465
466 The applied input type(s) for the category 'self-harm/intent'.
467
468 - `"text"`
469
470 - `"image"`
471
472 - `sexual: array of "text" or "image"`
473
474 The applied input type(s) for the category 'sexual'.
475
476 - `"text"`
477
478 - `"image"`
479
480 - `sexual/minors: array of "text"`
481
482 The applied input type(s) for the category 'sexual/minors'.
483
484 - `"text"`
485
486 - `violence: array of "text" or "image"`
487
488 The applied input type(s) for the category 'violence'.
489
490 - `"text"`
491
492 - `"image"`
493
494 - `violence/graphic: array of "text" or "image"`
495
496 The applied input type(s) for the category 'violence/graphic'.
497
498 - `"text"`
499
500 - `"image"`
501
502 - `category_scores: object { harassment, "harassment/threatening", hate, 10 more }`
503
504 A list of the categories along with their scores as predicted by model.
505
506 - `harassment: number`
507
508 The score for the category 'harassment'.
509
510 - `harassment/threatening: number`
511
512 The score for the category 'harassment/threatening'.
513
514 - `hate: number`
515
516 The score for the category 'hate'.
517
518 - `hate/threatening: number`
519
520 The score for the category 'hate/threatening'.
521
522 - `illicit: number`
523
524 The score for the category 'illicit'.
525
526 - `illicit/violent: number`
527
528 The score for the category 'illicit/violent'.
529
530 - `self-harm: number`
531
532 The score for the category 'self-harm'.
533
534 - `self-harm/instructions: number`
535
536 The score for the category 'self-harm/instructions'.
537
538 - `self-harm/intent: number`
539
540 The score for the category 'self-harm/intent'.
541
542 - `sexual: number`
543
544 The score for the category 'sexual'.
545
546 - `sexual/minors: number`
547
548 The score for the category 'sexual/minors'.
549
550 - `violence: number`
551
552 The score for the category 'violence'.
553
554 - `violence/graphic: number`
555
556 The score for the category 'violence/graphic'.
557
558 - `flagged: boolean`
559
560 Whether any of the below categories are flagged.
561
562### Moderation Image URL Input
563
564- `moderation_image_url_input: object { image_url, type }`
565
566 An object describing an image to classify.
567
568 - `image_url: object { url }`
569
570 Contains either an image URL or a data URL for a base64 encoded image.
571
572 - `url: string`
573
574 Either a URL of the image or the base64 encoded image data.
575
576 - `type: "image_url"`
577
578 Always `image_url`.
579
580### Moderation Model
581
582- `moderation_model: "omni-moderation-latest" or "omni-moderation-2024-09-26" or "text-moderation-latest" or "text-moderation-stable"`
583
584 - `"omni-moderation-latest"`
585
586 - `"omni-moderation-2024-09-26"`
587
588 - `"text-moderation-latest"`
589
590 - `"text-moderation-stable"`
591
592### Moderation Multi Modal Input
593
594- `moderation_multi_modal_input: ModerationImageURLInput or ModerationTextInput`
595
596 An object describing an image to classify.
597
598 - `moderation_image_url_input: object { image_url, type }`
599
600 An object describing an image to classify.
601
602 - `image_url: object { url }`
603
604 Contains either an image URL or a data URL for a base64 encoded image.
605
606 - `url: string`
607
608 Either a URL of the image or the base64 encoded image data.
609
610 - `type: "image_url"`
611
612 Always `image_url`.
613
614 - `moderation_text_input: object { text, type }`
615
616 An object describing text to classify.
617
618 - `text: string`
619
620 A string of text to classify.
621
622 - `type: "text"`
623
624 Always `text`.
625
626### Moderation Text Input
627
628- `moderation_text_input: object { text, type }`
629
630 An object describing text to classify.
631
632 - `text: string`
633
634 A string of text to classify.
635
636 - `type: "text"`
637
638 Always `text`.