AI Moderation

Quick Reference

// Send a message and check moderation status
CometChat.sendMessage(textMessage).then((message) => {
  const status = message.getModerationStatus();
  // CometChat.ModerationStatus.PENDING | APPROVED | DISAPPROVED
});

// Listen for moderation results
CometChat.addMessageListener("listenerId", new CometChat.MessageListener({
  onMessageModerated: (message) => {
    const status = message.getModerationStatus();
  }
}));

Overview

AI Moderation in the CometChat SDK helps ensure that your chat application remains safe and compliant by automatically reviewing messages for inappropriate content. This feature leverages AI to moderate messages in real-time, reducing manual intervention and improving user experience.

For a broader understanding of moderation features, configuring rules, and managing flagged messages, see the Moderation Overview.

Prerequisites

Before using AI Moderation, ensure the following:

Moderation is enabled for your app in the CometChat Dashboard
Moderation rules are configured under Moderation > Rules
You’re using a CometChat SDK version that supports moderation

How It Works

Step	Description
1. Send Message	App sends a text, image, or video message
2. Pending Status	Message is sent with `PENDING` moderation status
3. AI Processing	Moderation service analyzes the content
4. Result Event	`onMessageModerated` event fires with final status

Supported Message Types

Moderation is triggered only for the following message types:

Message Type	Moderated	Notes
Text Messages	✅	Content analyzed for inappropriate text
Image Messages	✅	Images scanned for unsafe content
Video Messages	✅	Videos analyzed for prohibited content
Custom Messages	❌	Not subject to AI moderation
Action Messages	❌	Not subject to AI moderation

Moderation Status

The getModerationStatus() method returns one of the following values:

Status	Enum Value	Description
Pending	`CometChat.ModerationStatus.PENDING`	Message is being processed by moderation
Approved	`CometChat.ModerationStatus.APPROVED`	Message passed moderation and is visible
Disapproved	`CometChat.ModerationStatus.DISAPPROVED`	Message violated rules and was blocked

Implementation

Step 1: Send a Message and Check Initial Status

When you send a text, image, or video message, check the initial moderation status:

JavaScript
TypeScript

const textMessage = new CometChat.TextMessage(
  receiverUID,
  "Hello, how are you?",
  CometChat.RECEIVER_TYPE.USER
);

CometChat.sendMessage(textMessage).then(
  (message) => {
    // Check moderation status
    const status = message.getModerationStatus();
    
    if (status === CometChat.ModerationStatus.PENDING) {
      console.log("Message is under moderation review");
      // Show pending indicator in UI
    }
  },
  (error) => {
    console.log("Message sending failed:", error);
  }
);

const textMessage = new CometChat.TextMessage(
  receiverUID,
  "Hello, how are you?",
  CometChat.RECEIVER_TYPE.USER
);

CometChat.sendMessage(textMessage).then(
  (message: CometChat.TextMessage) => {
    // Check moderation status
    const status: string = message.getModerationStatus();
    
    if (status === CometChat.ModerationStatus.PENDING) {
      console.log("Message is under moderation review");
      // Show pending indicator in UI
    }
  },
  (error: CometChat.CometChatException) => {
    console.log("Message sending failed:", error);
  }
);

Step 2: Listen for Moderation Results

JavaScript
TypeScript

const listenerID = "MODERATION_LISTENER";

CometChat.addMessageListener(
  listenerID,
  new CometChat.MessageListener({
    onMessageModerated: (message) => {
      if (
        message instanceof CometChat.TextMessage ||
        message instanceof CometChat.MediaMessage
      ) {
        const status = message.getModerationStatus();
        const messageId = message.getId();

        switch (status) {
          case CometChat.ModerationStatus.APPROVED:
            console.log(`Message ${messageId} approved`);
            // Update UI to show message normally
            break;

          case CometChat.ModerationStatus.DISAPPROVED:
            console.log(`Message ${messageId} blocked`);
            // Handle blocked message (hide or show warning)
            handleDisapprovedMessage(message);
            break;
        }
      }
    }
  })
);

// Don't forget to remove the listener when done
// CometChat.removeMessageListener(listenerID);

const listenerID: string = "MODERATION_LISTENER";

CometChat.addMessageListener(
  listenerID,
  new CometChat.MessageListener({
    onMessageModerated: (message: CometChat.BaseMessage) => {
      if (
        message instanceof CometChat.TextMessage ||
        message instanceof CometChat.MediaMessage
      ) {
        const status: string = message.getModerationStatus();
        const messageId: number = message.getId();

        switch (status) {
          case CometChat.ModerationStatus.APPROVED:
            console.log(`Message ${messageId} approved`);
            // Update UI to show message normally
            break;

          case CometChat.ModerationStatus.DISAPPROVED:
            console.log(`Message ${messageId} blocked`);
            // Handle blocked message (hide or show warning)
            handleDisapprovedMessage(message);
            break;
        }
      }
    }
  })
);

// Don't forget to remove the listener when done
// CometChat.removeMessageListener(listenerID);

Always remove message listeners when the component unmounts to prevent memory leaks.

CometChat.removeMessageListener("MODERATION_LISTENER");

Step 3: Handle Disapproved Messages

When a message is disapproved, handle it appropriately in your UI:

function handleDisapprovedMessage(message) {
  const messageId = message.getId();
  
  // Option 1: Hide the message completely
  hideMessageFromUI(messageId);
  
  // Option 2: Show a placeholder message
  showBlockedPlaceholder(messageId, "This message was blocked by moderation");
  
  // Option 3: Notify the sender (if it's their message)
  if (message.getSender().getUid() === currentUserUID) {
    showNotification("Your message was blocked due to policy violation");
  }
}

Best Practices

Always check getModerationStatus() after sending a message to show appropriate UI indicators (e.g., a pending badge)
Register the onMessageModerated listener early in your app lifecycle so you don’t miss moderation results
Provide clear feedback to users when their message is disapproved — avoid silently hiding content without explanation
Custom and Action messages are not moderated — if you need moderation on custom message types, implement your own server-side checks
Consider caching moderation status locally to avoid re-checking on message list re-renders

Troubleshooting

Moderation status always PENDING: Ensure moderation rules are configured in the CometChat Dashboard under Moderation > Rules.
onMessageModerated not firing: Verify you registered a MessageListener with the correct listener ID and that moderation is enabled for your app.
Custom messages not being moderated: This is expected — AI Moderation only applies to text, image, and video messages.
Disapproved messages still visible: Make sure your onMessageModerated handler updates the UI when a message status changes to DISAPPROVED.

Overview

Prerequisites

How It Works

Supported Message Types

Moderation Status

Implementation

Step 1: Send a Message and Check Initial Status

Step 2: Listen for Moderation Results

Step 3: Handle Disapproved Messages

Next Steps

Flag Message

AI Agents

​Overview

​Prerequisites

​How It Works

​Supported Message Types

​Moderation Status

​Implementation

​Step 1: Send a Message and Check Initial Status

​Step 2: Listen for Moderation Results

​Step 3: Handle Disapproved Messages

​Next Steps

Flag Message

AI Agents

Overview

Prerequisites

How It Works

Supported Message Types

Moderation Status

Implementation

Step 1: Send a Message and Check Initial Status

Step 2: Listen for Moderation Results

Step 3: Handle Disapproved Messages

Next Steps