How to get follow status for a huge amount of viewers on many channels?

Hello,

im developing a loyality bot for twitch at the moment.

i want to reward more points to followers (and even more for subs)

lets mention my bot is in a lot of channels that have a lot of viewers.

this would generate a way to much requests to the twitch api, even if its “still” fair use i want to avoid this.

both ways even as the channel for its followers or to as the viewer for his follows are at least limited to 100 results per request.

i dont see a way when doing this for thousands of users without needing like “unlimited” requests to fetch this data and keep it up to date.

is there any other way to fetch this data without the need of so much requests in the case when my bot have to handle this for many channels and many viewers?

best wishes

daniel

For this to be viable for a large number of viewers/channels, you need to cache the follower list.

For example, for each channel your bot is on you could poll through the followers endpoint to get a full list of followers which you would store in a cache. Then when you award points, you only need to compare the user to the cache you have rather than making any queries to Twitch to check that.

To keep the cache up to date you could use Webhooks to receive notifications of new followers which you can then add to your cache. Periodically you’ll also want to query Twitch for the total number of followers and compare that to your own cache so you can check if there are any users who have unfollowed as the Webhook intentionally doesn’t notify of unfollows.

This isn’t 100% perfect, there may be times when your cache is inconsistent at the time of awarding points, but there will always be an inherent margin of error in systems like this, but with appropriate monitoring of the cache and updating as needed (and within the rate-limits) will help to keep the error rate to a minimum, and in the event of an error I don’t think awarding greater points to someone who isn’t actually a follower at that point in time isn’t a big deal.

1 Like

hello Dist,

thanks for your answer!

its not a big issue if sometimes a user will get probably too less or too much points.

helix have to following limits:

30 queries per minute (if a Bearer token is not provided) or 120 queries per minute

lets say the bot have to fetch this informaton for 10k+ channels there is no way to do it this way.
even when you queue them it will took like foever.

the same for webhooks.

im really wondering how services like streamlabs / streamelements and co handle this.

best wishes

Daniel

It wont take forever, but it will take some time, there’s no avoiding that.

The 120 limit is enough to get through 12k follows/min. If you actually have 10k channels using your app and you can show sufficient need then you could request a rate limit increase and significantly increase that limit which would allow you to build your cache faster.

You could also prioritise your requests, for example, if one of the channels that has authorised your bot is live you could get the follow status of the users in those channels first, and only then move on to building a full cache for offline channels.

Webhooks, and monitoring follower count, are ways to help maintain the caches accuracy, but there is simply no avoiding that it will take time to build the initial cache.

Hello Dist!

thank you a lot for this information.

thats really helpful.

the most calls will be based on getting all data for a “new” channel.

but after a channel is known i can use the api to get the latest x followers time by time. this should give me hopefully an accurate data as my bot will just get used on channels with less then 200 viewers.

awesome at least that you can sort the list by “desc” :slight_smile:

but its still a mysterious to me how big bots can handle so much channels and follow notification that fast!

in the worst case if a channel get raided with a huge amount of viewers this will probably not work anymore.

so this point would need a workaround somehow…

thank you again.

have a good time!

best wishes

Daniel

This is where webhooks can come in. If you have a complete cache you would already know which of the influx of people in chat are followers, and if those who aren’t following but then decide to follow that will fire a notification to your webhook subscription so you will know of new followers moments after they happen without needing to query the API. The only time webhooks will use up your rate limit is for the initial subscription, but the max lease time is 10 days so even with hundreds of thousands of channels, if you stagger subscription renewal it has a minimal impact on your rate-limit.

Yes webhooks are made for this, Of course you would need an inital set of followers to dont announce duplicates or to allow spamming with refollows but grabbing the follows once is actually not that bad. just grab them in batches of 100 (thats the max iirc) and you should be ok

1 Like

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.