The rate-limit could be sufficiently high to allow it to not be an issue. Whether connection limits will ever be a reality, I cannot say. It is something we may want to add, to avoid someone trying to DOS our servers by opening too many connections.
I love this idea! The Mods aspect should be easy to implement as the permissions module is already present. A much higher limit/no limit in channels the bot is modded would be a welcome addition and help reduce the amount of resources bot developers need to spend on working around these limits with additional connections and proxies.
However, the way elevated mod limits are currently implemented would also need to change. Currently, if 30 messages are sent to the modded channel and one message is sent to an un-modded channel, then the bot gets IP banned as it counts as 31 messages on an un-modded channel. This forces bot developers to restrict each connection to one channel.
I am also struggling with the limits on my bot (knoobot.com). I have all messages/commands/joins funneling through a processor for each that fires off at specific intervals to prevent the ban limit from being reached. Needless to say, that isn’t very scaleable.
@brildum - how soon do you think Twitch would be able to start the bot white-list application process?
For tracking moderators, Knoobot listens for the MODE +o and -o and maintains a local list per channel.
@brildum would it be possible to maintain a list in the kraken API?
Something like: api.twitch.tv/kraken/chat/moderators ?
@Knoober there are two existing solutions to this problem. You can call http://tmi.twitch.tv/group/user/channel/chatters which includes the list of moderators for a channel (but also a list of all other chatters in that channel) or you can send .mods to the channel that you are in and simply listen to the “The moderators of this room…” message.
One note about the latter Method: You need to be in TWITCHCLIENT 3 mode to be able to identify which channel the message with the moderators is sent to. Otherwise it will be PRIVMSG’d to your bot directly instead of to the channel.
@cerebellum42 thanks for the info. I think the chatters will work better since it is one less command to send to the twitch irc server. JSON objects are easier to work with as well.
little off topic
@brildum i made twitch chat directly into a game (StarCraft: Brood War) but for some users from Korea they get disconnected after few minutes. For rest all works stable. Any suggestion?
My guess would be some connection issues between our servers in San Francisco and those users’ ISP, since its happening to a subset of users from a very specific location. If the round-trip time between USA and Korea is long enough and the chat room is active enough, they may cross the 20 message buffer on the server and get disconnected.
You can skip the rate limit by connecting directly with the IRC server. Using some script servers like mIRC will cause the rate limit because there is a pause somewhere in the sending of the data.
If you can constantly stream data to the IRC server, there is no rate limit. This is how these bots are flooding channels. The only time you will see them drop is when there is a major packet loss from there end. This is how they create view bots.
To control those bots, you need a verification of actual viewers versus actual people connected to the Twitch IRC API. This is a very stupid design from my point of view. So one IP address can create thousands of users. Please fix this twitch!!! Allow only a limited amount of accounts from one IP address and you will stop a significant majority of view botters.
But in order to control the chat through moderation, you have to have a separate program connected to the IRC server that would handle all the chat commands coming through the channel. The twitch chat servers can’t handle all the API’s running then moderators trying to do there job.
IMO, if Twitch would limit IP’s, moderators would have it extremely easy
Where are you getting this information?
I doubt the chat has anything to do with viewbotting.
Stream views and the chat should be two completely separate systems.
Chat users by themselves don’t affect the view count at all
I have been programming for over 30 years and I have been working with the twitch IRC servers for over a year. Not that I do anything like what I am talking about. But it is there and I know about it. Twitch can check my IP logs on me if they want. I have nothing to hide, but I do know how they are doing the view botters.
Yes they do, that is the only way to do the view count. Think about this a second. What type of computer would you need to load 1000+ API’s at once just to load the Java API for chat, Not much really. All you would need is the memory.
Now how good of a computer would you have to have in order to load 1000’s of API’s for the video, that is IF the view count was based upon actual video users? Move over Watson, I have a 1000 computers in my bedroom. Not to be an ass about it, but it would physically not be possible. You could hijack a 1000 computers, log into their twitch account, load up the API for the video, and view a certain channel. The chances of this are probably smaller than the previous one.
So the way twitch is using the view counter, grabbing it from IRC users in your channel, seems more like what is happening. Please a twitch programmer notice this and correct me if I am wrong. I am trying to combat these assholes that are ruining a good thing.
IRC users have no affect on a channel’s viewer or views count. You have to be actively requesting the video feed to be counted as a viewer.
Question then: Why when I logon through an external IRC client, I can see all the viewers in the chat channel, but in the Web Browser API chat the count is different?
I only see a few of those users. Even waiting 5 minutes and refreshing the list in web browser does not show the total number of users that are in the IRC channel. Does not matter how long I wait, The IRC user count is a lot higher than what is shown in the chat list, and the count in IRC is close to viewer count.
Because the web client filters out “anonymous” irc users.
Concurrent viewers is based off of who is downloading the video. It has no relation to chat.
So how are people handling this? Just spreading out large channels evenly? Are there any cases that are so extreme that a single channel would require its own connection for receiving? My local dev server is struggling hard to even join 3 of the top 20 channels right now, sometimes not even 1 of them, and this is with TC3. I can join all 20 when I push this code to my Heroku production environment with TC3. I suspect that my local connection (likely the router) is responsible for this when developing locally, but it seems pretty insane that I’m being dropped after 1 or 2 channels. I’ll troubleshoot that on my own, but my original question is still valid.
I would imagine that with 1 or 2 channels you will not run into this case I mention. The reason the top 20 channels causes you to be disconnected is due to the NAMES replies being incredibly long. Additionally, the JOINs and PARTs of large channels can contribute to this issue as they are sent out in batches every ~20 seconds. With TC3, you really shouldn’t have any issue with channels unless you’re in multiple (more than 1 or 2) fast moving ones on the same connection.
Another suggestion: I am prepared to pay a small fee and accept additional terms and conditions to have a relaxed rate limit and to receive warnings instead of bans when my bot exceeds the rate limit. I imagine that if large bots are spending on infrastructure and development time to get around rate limits, then a fee for relaxed limits won’t be out of the question for them either.
For such whitelist to really be useful, they’d also have to make sure chat wouldn’t disconnect when receiving a high number of messages at once, so you could actually join a lot of channels on a single connection. I started seeing “random” disconnects any time I joined more than around 20 channels on normal chat and as few as 6 on event chat.
Right now, you have to spread the channels over connections for that reason alone, so worrying about sending rate limits is only really relevant for bots, which connect to a few very busy channels, and bots, which spam a lot (which is what rate limits are for anyway – limiting spam). My bot, which handles 10 channels per connection sending a maximum of 19 messages in 30 seconds has reached its limit only once in the past week.
Also, there is at least one 3rd party project being made that will allow bot developers to connect to that program, ideally running on the same computer with the bot(s), and have it handle the rate limits and keep the connection to Twitch alive even between bot restarts (to skip joining rate limits).