Announcing Two New PubSub AutoMod Topics

Announcements
1

On Friday, May 21st, we deployed a change to the functionality of an existing PubSub topic chat_moderator_actions that caused it to no longer report AutoMod held messages. This was an intentional change, as these messages will now be sent over a new, dedicated PubSub topic automod_queue. Additionally we have created a new AutoMod PubSub topic for users to receive notification when a message has been blocked by AutoMod, user-moderation-notifications. You can find more information on these new topics in the PubSub documentation.

These changes were necessary to ensure long-term reliability and availability of our AutoMod systems. We understand these changes may have caused a service interruption, and we apologize for making these changes without warning. We will be working harder on improvements to our internal processes to prevent situations like this from occurring in the future.

What’s changing?

Today we are documenting two new PubSub topics: automod-queue and user-notifications.

Who will be impacted by these changes?

If you create chat or moderation tools that surface AutoMod held messages, you will need to update your implementation to use these new AutoMod PubSub topics.

What action needs to be taken?

See the newly documented PubSub topics here, to get more information on how to implement these new topics.

2 Likes
2

Thanks. One thing I noticed is that the example in the documentation doesn’t seem to quite match what I encountered in the wild.

The message payload (without the standard PubSub frame around it) in the example is:

{
    "id":"some-automod-caught-message-id",
    "content":{
        "text":"a bad word",
        "fragments":[
            {
                "text":"a bad word",
                "automod":{
                    "topics":{
                        "swearing":6
                    }
                }
            }
        ]
    },
    "sender":{
        "user_id":"559969259",
        "login":"msongvo222",
        "display_name":"msongvo222",
        "chat_color":"#8A2BE2"
    },
    "sent_at":"2021-05-24T18:46:20.964Z",
    "content_classification":{
        "category":"swearing",
        "level":2
    },
    "status":"PENDING",
    "reason_code":"",
    "resolver_id":"",
    "resolver_login":""
}

What I encountered is more something like this:

{
    "type":"automod_caught_message",
    "data":{
        "content_classification":{
            "category":"misogyny",
            "level":1
        },
        "message":{
            "content":{
                "text":"Stupid message you badword, swearword",
                "fragments":[
                    {
                        "text":"Stupid",
                        "automod":{
                            "topics":{
                                "bullying":7
                            }
                        }
                    },
                    {
                        "text":" message "
                    },
                    more fragments..
                ]
            },
            "id":"a-message-id",
            "sender":{
                "user_id":"12345",
                "login":"a_username",
                "display_name":"A_Username",
                "chat_color":"#FF0000"
            },
            "sent_at":"2021-05-25T14:46:42.597220201Z"
        },
        "reason_code":"",
        "resolver_id":"",
        "resolver_login":"",
        "status":"PENDING"
    }
}

Not a big deal, but if someone implements it sticking to the example it could be confusing.

2 Likes
3

Thanks for the heads up! We’ve updated the documented examples for AutoMod Queue and User Moderation Notifications.

1 Like