On June 20, 2022, an update to the underlying infrastructure for the “Check AutoMod Status” Twitch API endpoint will result in two changes that affect developers as described below.
What’s changing?
Rate limits
This endpoint will have its own set of rate limits per channel (i.e. broadcaster_id) based on the account type, rather than per access token, in addition to the standard Twitch API rate limits. See the table below for limits per-minute and per-hour for each account type. The documentation will be updated to include these numbers. The remaining rate limit is not included in the response headers, though a 429 response can be expected if any of these limits have been reached.
Limit per minute
Limit per hour
Non-affiliate/partner
5
50
Affiliate
10
100
Partner
30
300
Deprecation of user_id in the request
The AutoMod service will no longer take into account the status of a user to determine whether a string message meets the channel’s AutoMod requirements. For this reason, user_id will no longer be an acceptable body parameter and will be ignored after the date mentioned above. While current implementations will not experience errors after the change, we recommend updating requests to not include this parameter.
The following table illustrates a couple of examples of how the is_acceptable value will change when the AutoMod service no longer takes user status into account.
Message includes…
User
Current “is_acceptable”
New “is_acceptable”
a blocked term
Moderator
true
false
no blocked terms
Banned
false
true
In the first example, we see that the current version of the AutoMod service takes into account that the message with a blocked term was sent by a moderator and therefore was accepted. In the second example, we see that even messages that do not contain blocked terms return false if the user is banned in the given channel. Acceptance after this change is solely based on AutoMod settings and blocked/permitted terms; not user status.
Who will be impacted by these changes?
Any developer who is using the “Check AutoMod Status” endpoint will need to take into account the service-level rate limits and note the behavioral change for is_acceptable as it no longer considers the status of a user. We expect impact to be low and are actively emailing all developers who have requested this endpoint in the last 30 days.
To confirm the scope of this update, there will be no change to how AutoMod functions beyond this API endpoint.
My primary use case for this endpoint involes spike loads from asking viewers to respond to prompts in an extension. And all responses run via the Automod endpoint as per moderation policy of Extensions. (Policy 7.2 I auto mod before feeding to moderators to control)
Will there be additional headers added to monitor rate limit usage?
Please add headers for this so we can monitor and make our implementations auto respond
I agree that these rates are low, but it’s not far off what I need. It will just suddenly be tight when before it was fast and easily accessible without worry.
I created a channel (non-affiliate/partner) with specific AutoMod levels, blocked and allowed terms, etc. This channel is specifically used for checking against messages to be used in TTS in another channel (partner) that has looser AutoMod settings. These TTS messages often come fast (via cheering in the chat, so some AutoMod is already in use) and should be rapidly available without waiting for the rate limit bucket to fill. I created an endpoint that indirectly calls the AutoMod API for this purpose. I imagine now the requests could realistically wait 12-72 seconds which will timeout.
Unfortunately it seems like with these limits of max 300 requests per hour, it pretty much makes this endpoint unable to check for any sort of user input.
I’d imagine you might be able to check donation messages (due to the required payment), but anything without a paywall would be pretty much impossible.
If the endpoint is being abused so heavily, why not just remove non-affiliate/partner support and increase the other rate limits heavily?
We are gonna end up using Alca’s route, a second channel for automod tests and instead of using the API, making a bot send the message to chat and then seeing if the message passes the automod on the channel.
This API really should have the same rate limit as Twitch Chat.
IE there isn’t one beyond standard helix. Since this API is the same operation as Twitch Chat itself. Which uses a user_ID to limit chat sends.
But thats the other thing being removed from the endpoint.
Well, this is particularly unwelcome news. The existing set-up was perfect for my use-case; this change makes it worse than useless - the feature is now a detrimental limitation which I’ll have to abandon entirely. The entire purpose of this endpoint is to check whether a message can be safely posted in a channel - in what world is 50 messages in an hour a valid limit for any person or bot in a channel?
The user_id deprecation is also going to be particularly annoying, forcing more API calls and checks to get the state of a user manually.
My biggest question is - Why? Why were the limits chosen to be those numbers rather than something more along the lines of actual messaging limits? Why was user_id deprecated if the internalized calls will just have to be externalized, resulting in the same (or heavier) server-side processing anyway?
Why would you want to run every message in your chat through this API? If you’re using the API to actively moderated a chat then why not just enable automod in your chat moderation settings? It seems to be a misunderstanding of the use-case for this endpoint.
Sorry, I was replying to RealityRipple’s comment not Alca’s. But as a side comment on Alca’s post, wouldn’t messages that are apart of channel point redemptions (or anything that goes through twitch chat) already go through the channel’s automod if they have it enabled? I can understand the use-case of tips through a 3rd-party service, but if that’s the only case that doesn’t already go through the channel’s automod, the limits seem alright, at worst a little low.
My use case is a bot dubbed Robot Mailman. It receives messages from one user intended for another user via a !mail command (!mail username message). Users can check their messages using the !inbox command. This works across channels so users can mail each other and pick up their messages whenever and wherever they are. However, different channels have different chat rule cases, so holding back messages that won’t pass automod muster in one channel is advisable to prevent the bot from being flagged or the message lost. The number of messages the bot sends is a giant unknown to me, as I do not track any analytics from the system, however I’ve seen it used to spam emotes in channels that enjoy such events (with the broadcaster’s blessing) and the count has definitely gone far above 5 a minute or 50 per hour, even after I throttled the maximum messages per !inbox command limit to 5.
Just following up at this time to mention all of the comments have been shared with relevant teams. I wanted to make sure folks knew their feedback and uses cases were known.
My use case is also checking tip messages sent on other platforms (& mirroring the broadcaster’s chat on other platforms back into twitch chat). Currently I use the mod/bot’s user_id out of convenience, but with this change I’ll definitely be rate limited. It would be helpful if, like all of the other helix automod endpoints, a moderator’s oauth token would work for representing broadcaster_id’s that are not their own