Internal "Message trickling" behaviour

Hello,

while developing a twitch IRC library i’ve discovered that twitch appears to “trickle”/queue up messages internally before they are actually processed.
E.g. you can dump a bunch of commands over the TCP connections in a fraction of a second but they will take multiple seconds to actually appear in chat.

From empirical numbers it seems a message is taken from the stack and processed once every 100 ms, and it appears that the type of command does not matter (PING, PRIVMSG, JOIN, etc are all equal to this system).

Also, I’m suspecting rate limits apply after this internal throttle system.

Also it seems a connection’s “stack of messages” is dropped if the protocol is violated/some other error occurs, such as the rate limits being exceeded.

Is this how it works? It would be nice to see some offical word/explaination on this, or even a section on this in the documentation.

I have never observed this, except with anti flood protection in the client or server I was using to send.

Quite often in testing and fighting spam bots I’ve lost because I got globalled from Chat for flooding chat.

You really shouldn’t “see” this unless you are broken or deliberately trying to break rate limits. Sounds like an issue with your TCP sending stack, orother factors rather than Twitch itself.

Since the stack of messages getting dropped on a violation, means must be in a queue yourside rather than Twitch Side.

Nah. This is not even an edge case. You should try making a simple file along the lines of:

PASS oauth:xxxxxxxxxxxxxxxxxxxxxxxxxxxx
NICK yourchannel
CAP REQ :twitch.tv/membership
CAP REQ :twitch.tv/tags
CAP REQ :twitch.tv/commands
CAP REQ :twitch.tv/tags twitch.tv/commands
PRIVMSG #yourchannel :1
PRIVMSG #yourchannel :2
PRIVMSG #yourchannel :3
PRIVMSG #yourchannel :4
PRIVMSG #yourchannel :5
PRIVMSG #yourchannel :6
PRIVMSG #yourchannel :7
PRIVMSG #yourchannel :8
PRIVMSG #yourchannel :9
PRIVMSG #yourchannel :10
PRIVMSG #yourchannel :11
PRIVMSG #yourchannel :12
PRIVMSG #yourchannel :13
PRIVMSG #yourchannel :14
PRIVMSG #yourchannel :15
PRIVMSG #yourchannel :16
PRIVMSG #yourchannel :17
PRIVMSG #yourchannel :18
PRIVMSG #yourchannel :19
PRIVMSG #yourchannel :20
PRIVMSG #yourchannel :21
PRIVMSG #yourchannel :22
PRIVMSG #yourchannel :23
PRIVMSG #yourchannel :24
PRIVMSG #yourchannel :25
PRIVMSG #yourchannel :26
PRIVMSG #yourchannel :27
PRIVMSG #yourchannel :28
PRIVMSG #yourchannel :29
PRIVMSG #yourchannel :30
PRIVMSG #yourchannel :31
PRIVMSG #yourchannel :32
PRIVMSG #yourchannel :33
PRIVMSG #yourchannel :34
PRIVMSG #yourchannel :35
PRIVMSG #yourchannel :36
PRIVMSG #yourchannel :37
PRIVMSG #yourchannel :38
PRIVMSG #yourchannel :39
PRIVMSG #yourchannel :40
PRIVMSG #yourchannel :41
PRIVMSG #yourchannel :42
PRIVMSG #yourchannel :43
PRIVMSG #yourchannel :44
PRIVMSG #yourchannel :45
PRIVMSG #yourchannel :46
PRIVMSG #yourchannel :47
PRIVMSG #yourchannel :48
PRIVMSG #yourchannel :49
PRIVMSG #yourchannel :50
PRIVMSG #yourchannel :51
PRIVMSG #yourchannel :52
PRIVMSG #yourchannel :53
PRIVMSG #yourchannel :54
PRIVMSG #yourchannel :55
PRIVMSG #yourchannel :56
PRIVMSG #yourchannel :57
PRIVMSG #yourchannel :58
PRIVMSG #yourchannel :59
PRIVMSG #yourchannel :60
PRIVMSG #yourchannel :61
PRIVMSG #yourchannel :62
PRIVMSG #yourchannel :63
PRIVMSG #yourchannel :64
PRIVMSG #yourchannel :65
PRIVMSG #yourchannel :66
PRIVMSG #yourchannel :67
PRIVMSG #yourchannel :68
PRIVMSG #yourchannel :69
PRIVMSG #yourchannel :70
PRIVMSG #yourchannel :71
PRIVMSG #yourchannel :72
PRIVMSG #yourchannel :73
PRIVMSG #yourchannel :74
PRIVMSG #yourchannel :75
PRIVMSG #yourchannel :76
PRIVMSG #yourchannel :77
PRIVMSG #yourchannel :78
PRIVMSG #yourchannel :79
PRIVMSG #yourchannel :80
PRIVMSG #yourchannel :81
PRIVMSG #yourchannel :82
PRIVMSG #yourchannel :83
PRIVMSG #yourchannel :84
PRIVMSG #yourchannel :85
PRIVMSG #yourchannel :86
PRIVMSG #yourchannel :87
PRIVMSG #yourchannel :88
PRIVMSG #yourchannel :89
PRIVMSG #yourchannel :90
PRIVMSG #yourchannel :91
PRIVMSG #yourchannel :92
PRIVMSG #yourchannel :93
PRIVMSG #yourchannel :94
PRIVMSG #yourchannel :95
PRIVMSG #yourchannel :96
PRIVMSG #yourchannel :97
PRIVMSG #yourchannel :98
PRIVMSG #yourchannel :99
PRIVMSG #yourchannel :100

and then:

cat lines.txt | tee /dev/stderr | nc irc.chat.twitch.tv 6667

You will see that all the lines are easily dumped over the TCP connection before the server even responds with the login message, and then slowly but surely 100 USERSTATES will show up as the messages are delivered to chat. You can even terminate the connection (Ctrl+C) while the responses are flowing in and the messages will keep coming in the chat you have opened.

This is the reason why many high-volume bots use connection pooling to send messages over multiple connections.

Point is, there are legit use cases for sending high volumes of messages to chat, especially when you have a verified bot. With only one connection, it is literally not possible to exceed the high 7500 messages per 30 seconds limit, due to this behaviour.

I just want the behaviour to be documented.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.