Games/top returns Status:500 with high offset, inconsistent with other endpoints

[Benjamin_Dillinger]

Hi,

First of all, let me say thank you to everyone involved for making such a clean and easy-to-use API.

According to our app, sometime on Tuesday, Feb 21, the /games/top endpoint started returning an HTTP status 500 instead of the expected empty array when the offset is greater than the number of games (useful for signaling the end of traversing all of the live games).

Taken from Postman:

https://api.twitch.tv/kraken/games/top?limit=100&offset=2000
{
  "error": "Internal Server Error",
  "status": 500,
  "message": ""
}

As a separate example, here is the behavior for the /streams endpoint with a offset larger than the number of current streamers.

https://api.twitch.tv/kraken/streams?limit=100&offset=40000
{
  "_total": 23113,
  "streams": []
}

I’m wondering what we should expect the behavior to be going forward? We’ve made changes to allow for the 500 error, but should we apply those changes universally to all API calls? or would the 500 have a different meaning if it were to show up on a GET request to something that is not /games/top.

Cheers,
Ben

[Six]

There is a limit to how large offset can be, which is 1000. Any value above that max will not properly work.

[hangsu]

I’m running into the same issue. Wouldn’t returning 4xx be more appropriate here?

Also, @Six’s comment is both provably false and not relevant to the topic.

https://api.twitch.tv/kraken/games/top?limit=100&offset=1100
works just fine for me:

{
  "_total": 1488,
  "top": [
    {
      "game": {
        "name": "Mega Man X",
        "popularity": 2,
        "_id": 8676,
        "giantbomb_id": 9463,
...

[Six]

No, the title clearly states an issue with high offset values, so my statement is relevant. Also, no, offset does have a max value of 1000 and was confirmed by Dallas in this post. Make that same request again with 1000 offset and you’ll get the same result.

Edit, it seems like they may have altered the behavior of offset. Regardless, if you specify a value above its apparent max, you will get incorrect results or an internal server error.

[hangsu]

I believe the topic is about the inconsistency of out-of-bounds offset responses between different endpoints.

One endpoint returns an empty array. The other returns 500 (usually reserved for internal server errors) with an empty message which seems like an inappropriate response given the call.

The question isn’t about whether the high offset call should “work”. It’s about what the expected response of an out-of-bounds offset call should be.

[BarryCarlyon]

If you get a non 200 error, generally speaking, ignore the body of the response.

API’s will generally return something with a 4xx

But a 5xx error means “the server did something weird” so never expect a body response when you get a 5xx code, as thats a server error and it just dies…

[Benjamin_Dillinger]

It’s not that I’m expecting a response if I run into a 4xx or 5xx, but let’s say I ask for top games with offset 500, and there is a legitimate server error that prevents me from getting those games.

Do I assume that there are less than 500 games being streamed and not retry?
Or should I retry because it’s possible that a transient error prevented the request temporarily, and my next request may yield me the body?

This of course (potentially) applies to other requests if they are changed to return 500 when given out-of-bounds offset.

[DallasNChains]

Thanks for all the insight here! Endpoints are made by different teams, so there are inconsistencies in how they handle errors and edge cases (as you’re all aware ) Pointing these out helps me have a conversation with the teams to make them consistent.

I’ll work with the teams to make these consistent OR move to a cursor. The latter is the preferred method going forward (as you can see in most new APIs), but we should absolutely be consistent with our error codes and messaging if the offset is too high or it was an actual server error.

[Six]

Cursors for all endpoints would be amazing

[BarryCarlyon]

5xx codes you should retry. As the server gave up the request, so retrying after a moment should get the required data.

4xx codes refer to the Documentation for the code meaning and the body error message if present.

Again a 5xx error means the server fluffed up, ie failed DB connection, gave up trying to send a reply, went for tea, got lost in a maze, some other issue thats not apparent.

With a 5xx error, just retry the request