RFC 0007 - Chat Capabilities in Extensions

Chat Capabilities in Extensions

Summary

Twitch chat is an important shared attention space on the channel page leveraged by both streamers and viewers to communicate and interact. Extensions facilitate the connection between streamers and their viewers by offering new ways to engage with the streamer and their content. By building new extension capabilities into chat, we can increase the effectiveness and reach of the interactions created by extensions.

Chat Capabilities in Extensions (CCE) will be an ongoing effort for the Extensions team to bridge Twitch chat and Extensions. In this iteration we aim to provide a very simple integration with chat allowing any extension to be able to send messages to chat. We’re also going to be introducing a new concept of “Feature Flags” allowing broadcasters to opt out/in of features within extensions.

Motivation

Channel Page Presence

Extensions have been successful as a way for broadcasters to provide their audience with more information about their stream or what game they’re playing. However, when we set out to create the Extensions product, one of our major goals was to increase interactivity between the broadcaster and the viewer. While many extensions are already doing this, we will make it much easier with the addition of Chat Capabilities in Extensions.

In order for an Extension to be noticed on the channel page, the user likely needs to be told to look at it by the broadcaster or they’ll stumble upon it by hovering the video player. We will provide developers a new way to communicate with the viewers through a familiar shared context - chat.

Chatbots

Looking a bit longer term, we are working towards expanding the set anchor locations available to extensions beyond just iframes. Chat Capabilities in Extensions sets us up perfectly to start creating features which make Chatbots as Extensions a possibility.

Chatbot developers face the same problems Extension developers face today with regards to scale, cost, and complexity. We’re aiming to solve all of these problems with Extensions and believe that bringing chatbots directly into the world of Extensions, we can solve this problem for both sets of developers.

Chat Message API Reference

Send Chat Message

Chat messages will appear in the chat as a normal message. The “username” of the chat message will be the name of your extension as specified in the Extension Manifest.

Authentication

Your request is authenticated by way of the extension JWT. The only authorized JWTs are those which are are signed with the external or broadcaster role. The channel_id inside the JWT must also match the channel_id in the URL of the request.

URL

POST /extensions/<extension_id>/channels/<channel_id>/chat

Example Request

curl -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE1MDMzNDM5NDcsImNoYW5uZWxfaWQiOiIyNzQxOTAxMSIsInJvbGUiOiJleHRlcm5hbCJ9.iN0uNLWuEq-qGn07NfDEsumB4Eii4arJdf-SX2RXtZsa' \
-H 'Content-Type: application/json' \
-d '{ "text": "This is a normal message." }' \
-X POST https://api.twitch.tv/extensions/<extension_id>/channels/1682620/chat

Example Response

204 No Content

Limits

Rate Limit: Chat messages will be rate limited to four messages per minute, per channel and will use the same rate limiting functionality which already exists for our API today. (Twitch API Concepts | Twitch Developers)

Message Size: The message must not exceed 280 characters.

Errors

Error Code Reason Likely Cause
401 Unauthorized This can happen for a few reasons. Either the broadcaster has disabled chat support in your extension (see Feature Flags below), you have not specified on the devsite that your extension supports chat (see Rollout Considerations below), the JWT you provided is invalid. Or, in the event of a 401 in your EBS, the extension is no longer installed and activated on the channel.
429 Rate limit exceeded You’ve made too many requests to the API.

Send Pinned Chat Message

Pinned messages will appear at the top of the chat space for ten seconds before automatically being removed. There will also be an option for the user to dismiss the message while it’s pinned to the top of chat. Pinned messages also appear as normal messages in chat but do not count against your normal message rate limit.

Authentication

Your request is authenticated by way of the extension JWT. The only authorized JWTs are those which are are signed with the external or broadcaster role. The channel_id inside the JWT must also match the channel_id in the URL of the request.

URL

POST /extensions/<extension_id>/channels/<channel_id>/chat/pinned

Example Request

curl -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE1MDMzNDM5NDcsImNoYW5uZWxfaWQiOiIyNzQxOTAxMSIsInJvbGUiOiJleHRlcm5hbCJ9.iN0uNLWuEq-qGn07NfDEsumB4Eii4arJdf-SX2RXtZsa' \
-H 'Content-Type: application/json' \
-d '{ "text": "This is a pinned message." }' \
-X POST https://api.twitch.tv/extensions/<extension_id>/channels/1682620/chat/pinned

Example Response

204 No Content

Limits

Rate Limit: Pinned chat messages will be rate limited to one message every five minutes and will use the same rate limiting functionality which already exists for our API today. (Twitch API Concepts | Twitch Developers)

Message Size: The message must not exceed 280 characters.

Errors

Error Code Reason Likely Cause
401 Unauthorized This can happen for a few reasons. Either the broadcaster has disabled chat support in your extension (see Feature Flags below), you have not specified on the devsite that your extension supports chat (see Rollout Considerations below), the JWT you provided is invalid. Or, in the event of a 401 in your EBS, the extension is no longer installed and activated on the channel.
429 Rate limit exceeded You’ve made too many requests to the API.

Feature Flags

With the release of Chat Capabilities in Extensions, we’re introducing the new concept in Extensions of “Feature Flags”.

The Extension Helper will now provide a features object within the Twitch.ext global. Within this object, we will provide whether or not a feature is allowed in the current context of your extension. For CCE, the key will be chat and the value will be a boolean representing whether or not the broadcaster has disabled the ability for your extension to send chat messages.

If the value of Twitch.ext.features.chat is false, you should expect any calls to the Chat Message APIs to return 401.

Helper API

features: Object
The features object will contain values for all supported feature flags. The object and the onChange method are guaranteed to exist when your extension loads. However, it is possible, and likely, for the feature flags to not exist because they are loaded asynchronously. Checking these fields before they are actually provided should not cause any runtime errors but you may get a false negative if you’re simply checking for support using if (Twitch.ext.features.flag) {} since it’s possible for flag to be undefined. The onChange function will be called once the features have been loaded into the global object which should happen before the first call to onAuthorized.

Properties:

Feature Flag Type Description
chat boolean Represents whether the ability to send chat messages is allowed in the current context. This may be false if the broadcaster has disabled chat for your extension in their channel or because you have not specified your extension supports chat through the devsite.
onChange function A listener function you can subscribe to to receive real-time updates to changes of the features object. If this callback is invoked, you should re-check the Twitch.ext.features object for a change to any of the feature flags your extension cares about. The callback will be called with an array of feature flags which were updated.

User Experience Considerations

Keep in mind, normal viewers do not have the ability to send a chat message from their front end using the Twitch JWT. This is because users are provided tokens which have role: "viewer". It is however, possible to make a call to your EBS which would invoke a message to be sent to the Twitch API with role: "external". In this case and in the case where you’d like the broadcaster to have an interface allowing them to send a message you should make sure the feature flag in Twitch.ext.features.chat is set to true rather than sending additional, unnecessary load to your EBS or the Twitch API.

As described above, we’ve provided an onChange listener to the features object which will notify you when any option inside the features object has changed. If you provide different user experiences based on the value of a feature flag, it is highly recommended that you subscribe to this callback.

Rollout Considerations

Adding CCE to Your Extension

The Developer Rig will be updated alongside the release of CCE to allow testing against these new APIs immediately upon release.

In order to use these new APIs in production, a new version of your extension must be created in the devsite with chat selected in the “Extension Capabilities” section. Once you’ve released your CCE-enabled extension, the chat feature flag will be default-enabled for new and existing installations.

Screenshots

Example pinned and normal messages:

Details about your extension are shown when a user clicks the extension name in chat.
ChatExt_inchat_MoreInfo

Example Code

// Closure to make sure none of my functionality is exposed to the Window
(function () {
 const CLIENT_ID = 'YOUR_CLIENT_ID';

 // Toast to show on success/failure of showing a message
 const toast = document.createElement('div');
 toast.style.display = 'none';
 document.body.appendChild(toast);

 // Button to invoke a message to chat which alerts viewers a poll has started. Default
 // the state of the button to disabled because we don't know if chat is actually supported
 // until we've received our first call to `Twitch.ext.features.onChange`.
 const startPollButton = document.createElement('button');
 startPollButton.innerText = 'Start Poll';
 startPollButton.disabled = true;
 document.body.appendChild(startPollButton);

 // Helper method to check the value of chat feature flag
 function isChatSupported() {
   return Twitch.ext.features.chat === true;
 }

 // Helper to show our test with a message
 function showToast(message) {
   toast.innerText = message;
   toast.style.display = 'block';
 }

 // Helper to clear out the text and hide our toast
 function clearTost() {
   toast.innerText = '';
   toast.style.display = 'none';
 }

 // If we get an update to features and the flag for chat has changed
 // we should update our button state
 Twitch.ext.features.onChange(() => {
   if (!isChatSupported()) {
     startPollButton.disabled = true;
   } else {
     startPollButton.disabled = false;
   }
 });

 // We'll want to be sure we're always using the latest version of the auth object
 Twitch.ext.onAuthorized((auth) => {
   currentAuth = auth;
 });

 // Chat functionality of our extension: send a message to chat to alert viewers a poll
 // has started
 startPollButton.onclick = (event) => {
   event.preventDefault();

   // If we don't have auth, just return out
   if (!currentAuth) {
     console.error('Auth has not been received');
     return;
   }

   // If chat isn't supported, our call to the chat endpoint will just 401, no reason to make the call
   if (isChatSupported()) {
     fetch({
       method: 'POST',
       url: `https://api.twitch.tv/extensions/${CLIENT_ID}/channels/${currentAuth.channelId}/chat/pinned`,
       body: {
         text: 'A poll has started! Scroll down to the panel section to participate!'
       }
     }).then(() => {
       showToast('Message sent!');
       setTimeout(clearToast, 3000);
     }).catch(() => {
       showToast('Message failed!');
       setTimeout(clearToast, 3000);
     });

   // Instead, we'll just alert the user that chat has been disabled and that this functionality
   // of our extension is not available
   } else {
     showToast('Chat not supported in context!');
     setTimeout(clearToast, 3000);
   }
 }
}());
3 Likes

Looks great!

Would it be possible to also get an API endpoint for the feature flags for a channel? Say I want to use the CCE to shoot regular call-to-actions into the chat from my EBS, but the extension doesn’t have a config or live panel. I don’t want to have every viewer that loads the extension report the results of Twitch.ext.features.onChange() back to my EBS to be sure. It would be ideal if I could simply check the flags from the EBS directly. Perhaps it could be incorporated into the live_activated_extensions response data?

oh this is cool! :open_mouth:

1 Like

I absolutely think that this is really cool, but some of the features would be really cool to see outside of extension limitations. As of now, I haven’t fully released an extension, but I manage a bot, so the pinned messages really caught my attention, since this isn’t something you can normally do in chat AFAIK.

POST /extensions/<extension_id>/channels/<channel_id>/chat/pinned

Seeing as how this proposed endpoint is part of ‘/extensions/’ is the ability to pin a message something we’ll be seeing later down the road of the new API, without an extension?

Looks very interesting and can think of many integrations with my existing Extensions. For example, one of my Extensions plays an animation when Bits are used with it - having a temporary pinned message at the top of the chat will be a perfect compliment to this animation “donation”.

Two wishes from my side:

  • Have the extension be able to also perform moderation actions, like timouts or bans.
  • This might be a bit far fetched, but I’d love to see this: A new anchor to place an extension someplace in/at the chat, i.e. make the extension experience part of the chat (maybe an area above the chat text input?)

@xzion - This is a great callout! I’m going to get back to you but I agree this is something we should provide!

@altoar - PogChamp

@TheElm - This functionality is going to be limited to Extensions. As we mentioned, we see a world where Chatbots are simply just Extensions. Once this future is realized, all the great chat features we add to Extensions will be available to you as a “chatbot” developer!

@tf2casperr - Awesome! I’m happy to hear developers already thinking about the cool ways to do this! Keep in mind the rate limit of once/5m on the pinned messages! You’ll want to make sure you’re not triggering the pinned messages more than that or your users might be :frowning:

@Hellcat - Moderation actions may be some of the capabilities we add in the future! We’re still ironing out what the future of CCE looks like which depends largely on the results of this addition. Regarding your second point, I would love to see that too! Hopefully a future RFC awaits :smiley:

First of all: sounds absolutely amazing!

My concern though: the rate limits seem incredibly low, especially if you have users interacting with your extension a lot and say, something special happens to multiple users and you want to announce it using chat messages or pinned messages. For that, the rate limit seems incredibly… limiting.

Other thing: Will extensions be able to read chat in the future? This would allow for much more stuff.

An example situation for my above comment:
I currently have my (Waifu) Trading Card Game and am using Streamlabs to alert for high rarity cards being drawn and if a bunch of people feel like opening booster pavks, alerrs tend to happen at a rate of maybe 1 every 5 seconds, more if a person draws multiple high rarity card in a single pack. I would love to utilise the pinned message for this, but… i would be exceeding the rate limit instantly.

the rate limits seem incredibly low

I don’t think these capabilities designed to be used for viewer interactions, they’re more to give existing (and future) extensions a way to draw attention to themselves with a call-to-actions in the chat area (where a lot of user eyeballs tend to be). Chatbots are definitely going to remain the way to go if your app wants to read/submit messages in Twitch chat, at least for now.

These kinda contradict each other

I think they see a future world where the kind of functionality you’re looking for will be available, but it’s not going to be in this first version.

2 Likes

What’s the use case for this exactly? Because if it’s once per 5 minutes (for max 10 seconds), I don’t see any meaningful viewer interaction going through this.

I imagine since the pinned area normally has cheers leaderboard in it. It’s to make sure that can stick around for enough time and to give space for other extensions that might be installed to use the feature.

6 extensions at once (1 video, 2 component, 3 panel), and people cheering, and the leaderboard, could make it very noisy with a lower rate [citation not provided]

Will this support the channel rooms?

Wouldn’t it be better to then instead queue up the pinned messages instead of restricting the use, solely based on the stream using every one of its extensions, and all those extensions also spamming the pinned messages?

What about broadcasters that use just one extension?

Shouldn’t it be up to the broadcaster to decide the experience, and disable any extensions they feel are spamming too much? With this new ability to turn off the specific pinned message feature for an extension, with “Feature Flags”, it would be a very simple thing to do for the broadcaster.

Thanks everyone for the great conversation so far!

@Marenthyu As @xzion pointed out – we’re definitely envisioning a future where extensions can do more with chat than just send messages. In the future, it will hopefully only make sense to build a chatbot as an extension. That, of course, won’t be the case once this RFC lands but this is just a stepping stone to get us there.

@moocat The main issue we’re trying to solve with this addition is the one of “channel page presence” for extensions. Right now, it’s really hard for an Extension to let a viewer know they should be paying attention. We’re hoping by adding these features extension developers will be able to provide some sort of “call to action” to the viewer informing them they should be looking at the extension. As I mentioned just above, we’re working towards a future with many more features and integrations with chat and this is the first step which we plan to grow and learn from. Regarding multiple extensions: we will be queuing up these messages on the front-end so 6 different extensions send a pinned message at the same time, they will appear one after each other in the order received.

@ehgoodenough For this version, we will only be supporting the main, Stream Chat.

1 Like

I like this idea a lot. I’d like to suggest making the Extension messages in the chat more apparent. Maybe similar to the clip links in the chat with an other background color. So its easier to distinguish between other users messages.

1 Like

I started using a chat bot for Sound Alerts to announce the name, sound name and amount of bits whenever a user triggers an alert.
Giving the viewer recognition and give the broadcaster the opportunity to thank the viewer who spent bits inside the extension is really important. And to do this through chat is a good way.

I tried out this native extension chat message feature on my other extensions and I found out that the extension chat messages are not really chat messages?
A lot of streamers are using the twitch.tv chat pop out feature or other software to read chat like “chatty”. But the extension chat messages are not visible there. Streamers reached out to me that they can’t see the chat notifications because they don’t have their own stream channel open while streaming

I hope there is something that can be done at least in the twitch chat pop out, because it’s important that everyone can see the chat notifications.

I’m currently using CCE and also had a streamer mention she couldn’t see the extension notification in chat when it’s popped out.

Additionally, I’ve noticed CCE does not show in the chat reruns.

@Altoar for clarification, does your chat bot fix these issues and also alerts a sound?