I am about to track all concurrent liverstreamers for an app, which I am developing atm. At first, I tried it with requests (limit: 100 entries per response) walking through the pagination. When I analyzed the result, I noticed, that there were many duplicative entries. I guess, there is also a certain loss of entries, due to the shift of order, when all the API requests arrive sequentially/not at the same time.
I already contacted Twitch per E-Mail (integrationsuccess@twitch.tv) for help, unfortunately it is still unanswered since 15.11.2016. That’s why I’m asking you for help now !
My main questions are:
Is there a best practice to crawl all concurrent livestreamers (stable and lossfree solution)?
Can one avoid the pagination of the Twitch API (current limit is 100)?
How can I contact Twitch or more precisely the Integration Success staff directly?
Has anyone experiences with contacting Twitch/Integration Success staff and when did you received your answer?
What type of app are you building that needs all current livestreamers? I ask this because storing and resyndication of data from the API is against the Developer Agreement unless you have a specific agreement.
To answer your specific questions:
The best practice is with pagination.
No.
I’m right here. That email is specifically for help with the the Developer Agreement. Having said that, I don’t see any emails to that list.
Yep! They’re all awesome. Generally, responses are within 24 hours on the forums/Twitter. The email alias may or may not get a response since it is for matters relating to the Developer Agreement.
First, getting absolutely all channels may be difficult and useless.
Think about what you will do with all the data related to all those channels with no viewers at all.
I limit myself to the top 1000 channels.
Now, the super secret trick is to just compare the list of channels you got from the Twitch API to the previous one every time, and ask for channels missing in the new result with one or two additional requests, using limit=100&channel= followed by a comma-separated list of up to 100 channel names.
It depends on the period you use to get all those data, but with this technic you can get the top 1000 channels from the Twitch API every minute while being far from reaching rate limits.
!