Duplicate webhook notifications

[cowhome]

I’m using the webhook API for stream change notifications, and am quite frequently receiving duplicate notifications.

In particular, if the initial notification for a stream has 0 viewers, a second notification will be generated a little later when there are non-0 viewers.

Most of the time the duplicate notification has the same notification ID, so it can be easily filtered out, but sometimes this isn’t the case. Some recent examples of notifications IDs where only the viewer count has changed:

2018-10-22 14:49:16,951 INFO  Notification: d63cc350-7963-5b43-aa61-9717b02cc550
2018-10-22 14:51:16,752 INFO  Notification: 1555266d-e54e-5145-9fcb-722a2c01a06d

2018-10-19 21:43:03,460 INFO  Notification: e7c1df85-b5c9-5c93-bfe0-8bc18003ab6b
2018-10-19 21:43:14,858 INFO  Notification: 66e554c0-aff2-5fcb-8dd5-00849a24c465

This wasn’t a problem until the changes to the stream webhook at the start of the month, so I don’t think it’s something I’m doing wrong (although I’m happy to be corrected. )

Let me know if there’s any more information I can provide on this issue.

[BarryCarlyon]

Just to check you are replying to a incoming webhook with a

HTTP 1.0/200 Ok

And/or

With the optional body text of

Ok

Which can help confirm a 200 Ok response to the webhook handler.

Normally a repeat is only sent if a 200 is not received or you took too long to reply

Normally you should 200 Ok as quickly as possible and pass the data/payload to a background processor.

So you reply before you start processing the payload not after

It’s possible your job is taking too log and the second/duplicate is in fact the first retry which clears normally

[cowhome]

Thanks for your reply Barry.

The response to an incoming webhook is:

$ curl -i https://xxx/cgi-bin/twitch_webhook.py
HTTP/1.1 200 OK
Date: Tue, 23 Oct 2018 20:57:35 GMT
Server: Apache/2.4.25 (Debian)
Content-Length: 0
Content-Type: text/plain; charset=UTF-8

The data processing is handed off to a separate process so that the webhook handler can return as soon as possible. Apache logs show the response time of the webhook handler is usually a 2ms or less.

On closer inspection, I’ve noticed a difference between the duplicate notifications - the language tag. On the first notification it’s

"language":""

On the second/duplicate:

"language":"en" or "en-gb"

This looks like the initial notification has not correctly picked up the language of the stream.

Another oddity is that it is inconsistent as to whether the notification ID changes, even though the language data has.

[BarryCarlyon]

@cowhome

This looks like the initial notification has not correctly picked up the language of the stream.

Yeah thats a known issue and it’s being looked at

@cowhome

Another oddity is that it is inconsistent as to whether the notification ID changes, even though the language data has.

if you get a second notification with an existing ID, then it’s a resend.

Try doing HTTP/1.1 200 Ok and actually printing Ok.

There could be something else awry but if the problem of dupes continues let us know I can poke someone to take a look. We’d need the clientID that you are using.

[cowhome]

For future reference: putting an OK in the body appears to have fixed the issue.

Note that it needs to be OK with a new-line, as opposed to the subscription response where the challenge must be sent without a new-line.