Webhooks Bulk Subscription

[DoctorLoktor]

Hello Everyone,

The webhooks which are currently implemented are really helpful to avoid hitting twitch with unnecessary calls.
One thing that is still somewhat of a problem. is that currently it is only possible to subscribe to one webhook (for one user) at a time. E.g. if i am offering a service for multiple streamers or a bot then i have to make one call to twitch per user, which can sum up quite a bit once you reach a certain amount of users.

So it would be nice if twitch would support bulk subscriptions for webhooks, it doesn’t have to be all at once if that is too much but it would be nice to atleast be able to subscribe to webhooks for users in batches of maybe 20/100?. This would drastically reduce the amount of calls needed to subscribe to a lot of webhooks for a lot of users. Currently i would have to make 1000 separate calls for 1000 users even if my increase limit of allowed webhooks is way higher.

Thoughts on that topic?

[Dist]

This was discussed quite a bit in the Twitch app chat room, and there are quite a few issues with doing something like this.

First off, Twitch want to keep to the specification, so it may not even be possible to implement something like this without breaking the specification.

Secondly, how would you even implement something like this? With an API request, say for example getting user data, it’s easy to make a request for multiple users as it’s just the addition of &id=... for each user. A webhook subscription on the other hand requires multiple parameters for each: hub.callback, hub.mode, hub.topic, hub.lease_second, hub.secret.

How would you send multiple sets of querystring parameters and have them relate to each other? One of the cleanest ways to handle notifications is to have a unique callback for each topic request, so simply having the hub.topic parameter be repeatable wouldn’t work, you’d need them all repeatable. Grouping together each set of subscription parameters in an object, and then sending them all in an array in the request body wouldn’t work either as again that breaks the specification.

Authentication in a bulk subscription can also get tricky. What if one token is invalid, so one subscription needs to return an error, while the others need to return success?

So overall, bulk subscription would be quite a mess, unless the devs have thought of a clean way to do it that they haven’t mentioned yet.

With a 10 day lease time, 1000 separate calls really isn’t much at al. You can create all those subscriptions over the first min or two of your apps startup, then once they’re all up, stagger the resubscriptions and you’ll never need to have a big burst of subscriptions again. The devs have put a lot of work into trying to make the process as painless as possible, even for devs with needs to subscribe to tens of thousands of topics, the long lease time and staggered subscriptions really helps even on that scale, and for large initial subscriptions the request for a higher rate-limit is processed much faster than when it was initially made available.

[DoctorLoktor]

This was discussed quite a bit in the Twitch app chat room, and there are quite a few issues with doing something like this.

First off, Twitch want to keep to the specification, so it may not even be possible to implement something like this without breaking the specification.

I might have overlooked that, that this topic was already discussed in detail

Secondly, how would you even implement something like this? With an API request, say for example getting user data, it’s easy to make a request for multiple users as it’s just the addition of &id=… for each user. A webhook subscription on the other hand requires multiple parameters for each: hub.callback, hub.mode, hub.topic, hub.lease_second, hub.secret.

Why not limit this one to allow only multiple ids? this wouldn’t be as flexible but would probably be the standard case because right now your basically duplicating that call for multiple users, but yeah since i don’t know the specification in detail this would probably still violate it. But i understand now why they decided against it.

Authentication in a bulk subscription can also get tricky. What if one token is invalid, so one subscription needs to return an error, while the others need to return success?

But is it really such a problem? at the moment every app access token is allowed to subscribe to any users on/off hook so why not allow requests with one app access token for multiple users?

So overall, bulk subscription would be quite a mess, unless the devs have thought of a clean way to do it that they haven’t mentioned yet.

With a 10 day lease time, 1000 separate calls really isn’t much at al. You can create all those subscriptions over the first min or two of your apps startup, then once they’re all up, stagger the resubscriptions and you’ll never need to have a big burst of subscriptions again. The devs have put a lot of work into trying to make the process as painless as possible, even for devs with needs to subscribe to tens of thousands of topics, the long lease time and staggered subscriptions really helps even on that scale, and for large initial subscriptions the request for a higher rate-limit is processed much faster than when it was initially made available.

Yeah i get now why they didn’t implement it yet (or won’t). A full solution would probably entail some problems, also i didn’t know yet that the lease was increased to 10 days, i think it was 3 before, this should make this less of a problem. I was just trying to understand why it is like it is since it seemed like a lot of unnecessary calls to me thanks for your write up/clarification