Multiple bugs with /streams - it's pretty broken

API
1

I know API support isn’t exactly a top priority for Twitch, but I really wish something can be done about these issues.

  1. /streams often returns a user object instead of a channel object.
  2. /streams sometimes returns corrupt data (or an object for a completely unrelated user/channel).
  3. /streams sometimes does not return the correct number of arguments specified by the “limit” parameter.
  4. /streams often contains multiple entries for one user/channel.

Please feel free to add more in the comments.

1 Like
2

At least some sort of acknowledgement by the devs would be appreciated!

3

They are already fully aware of the broken state of the API, but there’s been little-to-no API upgrades since 2013. We might eventually see an improved API, but I wouldn’t count on that being anytime soon. Your best bet would be to throw out all the bad data you get.

4

Thank you, night. I guess I’ll have to come up with some ghetto methods to ensure that received data are reasonably correct.

5

Hi jkface,

If you can provide concrete examples, it would be immensely helpful in making sure these issues can be verified and resolved.

For example, when you say that /streams returns “corrupt data”, in what way is it corrupt? Is the response malformed?

Thanks!

6

You can see some concrete examples in this thread:

7

The issue the spawned that thread was temporary – it was introduced and fixed within about eight hours. Though it is interesting that 3ventic claims it’s been going around for a long time.

If you have examples outside of that window I’d love to see them. Thanks!

8

It’s been around for a long time in other endpoints where the user/channel object is part of the response, /streams for example. The whole response being incorrect was new and more serious

9

I have a script that repeatedly calls /streams and goes through the list of streams until the viewer count falls below a certain number.

Well, the script often ends prematurely because /streams returns data with faulty viewer counts. The following is a good example of what gets returned.

stream: viewers
riotgames: 22000
voyboy: 16000
trick2g: 12
anniebot: 8000

I feel like it’s trying to show the correct viewer count for trick2g but it cuts off abruptly, leaving only the first two digits.

I feel like all the other bugs I listed are self-explanatory.

10

Thanks, that information is helpful!

How often are you calling /streams and how often would you say that you see these types of issues?

11

I tried to recreate the problem just now and I’m beginning to think the above example isn’t quite representative of how the bug typically behaves. I configured my script to call /streams until it gets data for about 450 top streams. I ran it in terminal repeatedly with random breaks in between each execution until I found suspicious viewer counts. I’ve probably ran it about 30 times and came up with 2 instances where the viewer count for a stream shows up as 0. I’ve tried to recreate the problem again by repeatedly running the script with almost no breaks in between, and I haven’t been able to get the bug to show up. Maybe it’s my luck, but that’s what I have for now.

As an aside, I’m pretty sure you’re aware of this, but /streams very often returns objects out of order. For example, if Stream A has a higher viewer count than Stream B, it would make sense for Stream A to come before Stream B, but this is often not the case. In fact, about 10-20% of returned stream objects are out of order. Maybe the ordering thing is intentionally done for performance reasons, which I can understand.

12

It has to do with our multiple layers of caching. While the entire response to an API call is cached (as expected), we also cache individual components of the response (such as each stream object). So when we rebuild the /streams response, we fetch the ordered list of streams by viewer count fresh from our streams service, but then check to see each of the stream objects in that response to see if we have a cached copy pre-rendered and if so we use that instead. That cached copy can be out of date, and have higher or lower viewer numbers, causing the ordering to appear broken. I hope that explains this strange behavior!

tl;dr - yes its for performance reasons.

1 Like
13

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

14

/kraken/streams should no longer be returning user objects in place of channel objects in some rare cases. Viewer numbers may still not be 100% accurate based on caching or other factors (such as a stream resetting and everyone reconnecting).