Create moderation
client.Moderations.New(ctx, body) (*ModerationNewResponse, error)
post /moderations
Classifies if text and/or image inputs are potentially harmful. Learn more in the moderation guide.
Parameters
-
body ModerationNewParams-
Input param.Field[ModerationNewParamsInputUnion]Input (or inputs) to classify. Can be a single string, an array of strings, or an array of multi-modal input objects similar to other models.
-
string -
type ModerationNewParamsInputArray []stringAn array of strings to classify for moderation.
-
type ModerationNewParamsInputModerationMultiModalArray []ModerationMultiModalInputUnionAn array of multi-modal inputs to the moderation model.
-
type ModerationImageURLInput struct{…}An object describing an image to classify.
-
ImageURL ModerationImageURLInputImageURLContains either an image URL or a data URL for a base64 encoded image.
-
URL stringEither a URL of the image or the base64 encoded image data.
-
-
Type ImageURLAlways
image_url.const ImageURLImageURL ImageURL = "image_url"
-
-
type ModerationTextInput struct{…}An object describing text to classify.
-
Text stringA string of text to classify.
-
Type TextAlways
text.const TextText Text = "text"
-
-
-
-
Model param.Field[ModerationModel]The content moderation model you would like to use. Learn more in the moderation guide, and learn about available models here.
-
string -
type ModerationModel string-
const ModerationModelOmniModerationLatest ModerationModel = "omni-moderation-latest" -
const ModerationModelOmniModeration2024_09_26 ModerationModel = "omni-moderation-2024-09-26" -
const ModerationModelTextModerationLatest ModerationModel = "text-moderation-latest" -
const ModerationModelTextModerationStable ModerationModel = "text-moderation-stable"
-
-
-
Returns
-
type ModerationNewResponse struct{…}Represents if a given text input is potentially harmful.
-
ID stringThe unique identifier for the moderation request.
-
Model stringThe model used to generate the moderation results.
-
Results []ModerationA list of moderation objects.
-
Categories ModerationCategoriesA list of the categories, and whether they are flagged or not.
-
Harassment boolContent that expresses, incites, or promotes harassing language towards any target.
-
HarassmentThreatening boolHarassment content that also includes violence or serious harm towards any target.
-
Hate boolContent that expresses, incites, or promotes hate based on race, gender, ethnicity, religion, nationality, sexual orientation, disability status, or caste. Hateful content aimed at non-protected groups (e.g., chess players) is harassment.
-
HateThreatening boolHateful content that also includes violence or serious harm towards the targeted group based on race, gender, ethnicity, religion, nationality, sexual orientation, disability status, or caste.
-
Illicit boolContent that includes instructions or advice that facilitate the planning or execution of wrongdoing, or that gives advice or instruction on how to commit illicit acts. For example, "how to shoplift" would fit this category.
-
IllicitViolent boolContent that includes instructions or advice that facilitate the planning or execution of wrongdoing that also includes violence, or that gives advice or instruction on the procurement of any weapon.
-
SelfHarm boolContent that promotes, encourages, or depicts acts of self-harm, such as suicide, cutting, and eating disorders.
-
SelfHarmInstructions boolContent that encourages performing acts of self-harm, such as suicide, cutting, and eating disorders, or that gives instructions or advice on how to commit such acts.
-
SelfHarmIntent boolContent where the speaker expresses that they are engaging or intend to engage in acts of self-harm, such as suicide, cutting, and eating disorders.
-
Sexual boolContent meant to arouse sexual excitement, such as the description of sexual activity, or that promotes sexual services (excluding sex education and wellness).
-
SexualMinors boolSexual content that includes an individual who is under 18 years old.
-
Violence boolContent that depicts death, violence, or physical injury.
-
ViolenceGraphic boolContent that depicts death, violence, or physical injury in graphic detail.
-
-
CategoryAppliedInputTypes ModerationCategoryAppliedInputTypesA list of the categories along with the input type(s) that the score applies to.
-
Harassment []stringThe applied input type(s) for the category 'harassment'.
const ModerationCategoryAppliedInputTypesHarassmentText ModerationCategoryAppliedInputTypesHarassment = "text"
-
HarassmentThreatening []stringThe applied input type(s) for the category 'harassment/threatening'.
const ModerationCategoryAppliedInputTypesHarassmentThreateningText ModerationCategoryAppliedInputTypesHarassmentThreatening = "text"
-
Hate []stringThe applied input type(s) for the category 'hate'.
const ModerationCategoryAppliedInputTypesHateText ModerationCategoryAppliedInputTypesHate = "text"
-
HateThreatening []stringThe applied input type(s) for the category 'hate/threatening'.
const ModerationCategoryAppliedInputTypesHateThreateningText ModerationCategoryAppliedInputTypesHateThreatening = "text"
-
Illicit []stringThe applied input type(s) for the category 'illicit'.
const ModerationCategoryAppliedInputTypesIllicitText ModerationCategoryAppliedInputTypesIllicit = "text"
-
IllicitViolent []stringThe applied input type(s) for the category 'illicit/violent'.
const ModerationCategoryAppliedInputTypesIllicitViolentText ModerationCategoryAppliedInputTypesIllicitViolent = "text"
-
SelfHarm []stringThe applied input type(s) for the category 'self-harm'.
-
const ModerationCategoryAppliedInputTypesSelfHarmText ModerationCategoryAppliedInputTypesSelfHarm = "text" -
const ModerationCategoryAppliedInputTypesSelfHarmImage ModerationCategoryAppliedInputTypesSelfHarm = "image"
-
-
SelfHarmInstructions []stringThe applied input type(s) for the category 'self-harm/instructions'.
-
const ModerationCategoryAppliedInputTypesSelfHarmInstructionText ModerationCategoryAppliedInputTypesSelfHarmInstruction = "text" -
const ModerationCategoryAppliedInputTypesSelfHarmInstructionImage ModerationCategoryAppliedInputTypesSelfHarmInstruction = "image"
-
-
SelfHarmIntent []stringThe applied input type(s) for the category 'self-harm/intent'.
-
const ModerationCategoryAppliedInputTypesSelfHarmIntentText ModerationCategoryAppliedInputTypesSelfHarmIntent = "text" -
const ModerationCategoryAppliedInputTypesSelfHarmIntentImage ModerationCategoryAppliedInputTypesSelfHarmIntent = "image"
-
-
Sexual []stringThe applied input type(s) for the category 'sexual'.
-
const ModerationCategoryAppliedInputTypesSexualText ModerationCategoryAppliedInputTypesSexual = "text" -
const ModerationCategoryAppliedInputTypesSexualImage ModerationCategoryAppliedInputTypesSexual = "image"
-
-
SexualMinors []stringThe applied input type(s) for the category 'sexual/minors'.
const ModerationCategoryAppliedInputTypesSexualMinorText ModerationCategoryAppliedInputTypesSexualMinor = "text"
-
Violence []stringThe applied input type(s) for the category 'violence'.
-
const ModerationCategoryAppliedInputTypesViolenceText ModerationCategoryAppliedInputTypesViolence = "text" -
const ModerationCategoryAppliedInputTypesViolenceImage ModerationCategoryAppliedInputTypesViolence = "image"
-
-
ViolenceGraphic []stringThe applied input type(s) for the category 'violence/graphic'.
-
const ModerationCategoryAppliedInputTypesViolenceGraphicText ModerationCategoryAppliedInputTypesViolenceGraphic = "text" -
const ModerationCategoryAppliedInputTypesViolenceGraphicImage ModerationCategoryAppliedInputTypesViolenceGraphic = "image"
-
-
-
CategoryScores ModerationCategoryScoresA list of the categories along with their scores as predicted by model.
-
Harassment float64The score for the category 'harassment'.
-
HarassmentThreatening float64The score for the category 'harassment/threatening'.
-
Hate float64The score for the category 'hate'.
-
HateThreatening float64The score for the category 'hate/threatening'.
-
Illicit float64The score for the category 'illicit'.
-
IllicitViolent float64The score for the category 'illicit/violent'.
-
SelfHarm float64The score for the category 'self-harm'.
-
SelfHarmInstructions float64The score for the category 'self-harm/instructions'.
-
SelfHarmIntent float64The score for the category 'self-harm/intent'.
-
Sexual float64The score for the category 'sexual'.
-
SexualMinors float64The score for the category 'sexual/minors'.
-
Violence float64The score for the category 'violence'.
-
ViolenceGraphic float64The score for the category 'violence/graphic'.
-
-
Flagged boolWhether any of the below categories are flagged.
-
-
Example
package main
import (
"context"
"fmt"
"github.com/openai/openai-go"
"github.com/openai/openai-go/option"
)
func main() {
client := openai.NewClient(
option.WithAPIKey("My API Key"),
)
moderation, err := client.Moderations.New(context.TODO(), openai.ModerationNewParams{
Input: openai.ModerationNewParamsInputUnion{
OfString: openai.String("I want to kill them."),
},
})
if err != nil {
panic(err.Error())
}
fmt.Printf("%+v\n", moderation.ID)
}
Response
{
"id": "id",
"model": "model",
"results": [
{
"categories": {
"harassment": true,
"harassment/threatening": true,
"hate": true,
"hate/threatening": true,
"illicit": true,
"illicit/violent": true,
"self-harm": true,
"self-harm/instructions": true,
"self-harm/intent": true,
"sexual": true,
"sexual/minors": true,
"violence": true,
"violence/graphic": true
},
"category_applied_input_types": {
"harassment": [
"text"
],
"harassment/threatening": [
"text"
],
"hate": [
"text"
],
"hate/threatening": [
"text"
],
"illicit": [
"text"
],
"illicit/violent": [
"text"
],
"self-harm": [
"text"
],
"self-harm/instructions": [
"text"
],
"self-harm/intent": [
"text"
],
"sexual": [
"text"
],
"sexual/minors": [
"text"
],
"violence": [
"text"
],
"violence/graphic": [
"text"
]
},
"category_scores": {
"harassment": 0,
"harassment/threatening": 0,
"hate": 0,
"hate/threatening": 0,
"illicit": 0,
"illicit/violent": 0,
"self-harm": 0,
"self-harm/instructions": 0,
"self-harm/intent": 0,
"sexual": 0,
"sexual/minors": 0,
"violence": 0,
"violence/graphic": 0
},
"flagged": true
}
]
}