Do all IRC connections get all messages?

Just finished a Twitch.tv IRC bot (node.js + npm irc package). Simply put, it:

  1. Queries http://tmi.twitch.tv/servers?channel=CHANNEL (replace CHANNEL with the actual channel name) to get a list of server IPs (and stores the cluster, but that’s irrelevant).
  2. Stores all possible IPs for the channels it wants to join, then tries to get a unique IP to connect to (one connection per channel) that hasn’t been used yet. If there are none left, it joins one that’s already in use.
  3. Connects to the IP, port 443 (which is the port received in the query; using 6667 didn’t work on the event cluster when I first tried it).
  4. Receives messages and logs the last 5000.

So, it seems to work. But I say seems, because I’m not receiving all of the messages. When I open twitch chat in my browser (I used #esl_csgo as an example), I receive messages in my browser that never pass through the bot and I have absolutely no idea why.

The bot has a list of ~50 channels (I’m intentionally trying to increase the load, but I never go past 5% CPU usage), mostly on the main cluster but it includes esl_csgo which should be on the event cluster.

This is the last data the bot received from the query:

  • Channel: #esl_csgo
  • Cluster: event
  • IP: 199.9.255.149

A lot of messages pass through (up to 2.3k per minute according to logs), even messages from #esl_csgo, but somehow I’m missing messages. How, or why?

Any idea?

It’s probably a misconfiguration on your end. Each cluster runs independently of one another. In other words, since ESL is on the event cluster, only messages can be sent and received for ESL on the event cluster.

Hey night,

The bot receives messages from all channels - channels from both clusters. There was even a new highest recorded message count at 3.6k per minute on the 25th.

I also know that I need to connect to the proper cluster to receive all messages; this is fully automated since the bot queries Twitch’s API to receive the cluster and a list of IPs it can connect to. It does this for every single channel (even though this is technically not necessary).

I temporarily removed my message limit to make sure, and I’m currently at 306 819 stored messages from just the #esl_csgo channel.

My problem is not that I don’t receive any messages, it’s that there are some messages missing. I sometimes receive a message via Twitch’s website that my bot did not receive. There were also no errors on my side, as everything is logged (CPU/mem usage, message count per minute, uptime, disconnects, all errors).

What I haven’t checked yet is whether there’s a delay? If I receive a message via the Twitch.tv website, should the bot have received it as well (immediately) or is there some kind of fixed delay?

Thanks for your reply.

I’ve done some more tests over the past time, and it seems like the bot is receiving all messages. Strange.

I’m still curious about delays/filtering though. Is it possible that there was a small delay somehow, so that my bot received the message 5s+ after my web client (there’s no lag or recorded delay on my bot’s side)?

There’s no delays in receiving messages beyond the normal ping between you and the chat server.

Alright, thanks for the replies.

As I still can’t reproduce the missing message issue, I’ll just write this off as a brain malfunction rather than a bug in the bot. :slight_smile:

And after saying I can’t reproduce the problem, http://www.twitch.tv/twitchwatchespokemon starts streaming.

I’m getting messages, but some are getting skipped. I tested with a friend, I got 2 of his 3 messages that he sent during chat spam.

Sent messages can get dropped for a variety of reasons (too long, not allowed to send, processing timed out internally), but if one client sees the message, everyone else should receive it too. The only exception would be if you’re connected to different servers and sending the message to one edge timed out while another edge was able to process it in time.

If I want to guarantee that my bot sees all messages, does it need to connect to all the server IPs it receives via the API? Or should I just live with dropped messages?

These days it’s not worth bothering with it. Messages very rarely timeout anymore thanks to the great effort from Twitch in the past year or two.

Alright, thanks. Do we know how many messages timeout on average?

Honestly I don’t remember noticing any since the dark times of 2014.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.