Legacy Chatters endpoint and detecting bots

[5cottFive]

Well, I guess I found out why the Twitch Insights Live Bot List is broken now.

That’s just great, twitch.

So, how are we supposed to track, and ban bots now if we can’t see who’s in a channel? You’re telling me I would have to get literally everyone on Twitch to mod me so I can track bots now? That’s obviously not feasible.

Twitch has a serious bot problem. And this change makes me think Twitch doesn’t care about protecting their users from bots.

Creators should have the ability to detect, and disallow bots from their channels. To me, that’s a bigger privacy concern. IMHO, this change harms creators by taking that power away from them, and makes Twitch less safe.

We desperately need a way to track, and ban bots.

[5cottFive]

How about this?

Can you add a tag to the IRC JOIN messages that would include the total number of channels that user is currently in?

This wouldn’t violate any users’ privacy by showing which channels they’re in, but would definitely show who is a bot and who is not.

Thoughts?

[BarryCarlyon]

@5cottFive

Can you add a tag to the IRC JOIN messages that would include the total number of channels that user is currently in?

IRC JOIN’s do not broadcast in channels with more than 1000 chatters in. So in a large channel JOIN’s are not sent.

This would also deviated from the RFC specification for IRC (I believe) which would cause issues for other users of the service, especially those on “regular” IRC clients.

The value would also be intantly out of date ot inaccuarte or unable to be correctly represented for a variety of reasons.

[5cottFive]

@BarryCarlyon

IRC JOIN’s do not broadcast in channels with more than 1000 chatters in. So in a large channel JOIN’s are not sent.

This would also deviated from the RFC specification for IRC (I believe) which would cause issues for other users of the service, especially those on “regular” IRC clients.

True.

How about adding a field to the user record which can be obtained by a call to GetUser?

Doesn’t really matter if it’s out of date. Just need to know if a user is regularly joining more than X number of channels.

[Dist]

@5cottFive

Twitch has a serious bot problem. And this change makes me think Twitch doesn’t care about protecting their users from bots.

Creators should have the ability to detect, and disallow bots from their channels. To me, that’s a bigger privacy concern. IMHO, this change harms creators by taking that power away from them, and makes Twitch less safe.

We desperately need a way to track, and ban bots.

How does this make Twitch less safe? The sort of bots that you’re talking about that join a large number of channels most often do absolutely nothing but sit in your channel to be in your channel list. They don’t harm your channel in any way, and if it wasn’t for 3rd party lists telling you they’re in multiple channels you would have no way to know they’re different from any other lurker, and for those who don’t pay such close attention to the chatters list they would never know the bots even exist.

The bigger issue here is a misunderstanding of what these bots do, and the impact they have. If it truly was an issue of safety then they’d be handled by Twitch, but they’re not violating Twitch’s ToS or Community Guidelines, as it’s allowable behaviour to connect to a large number of channels. This can be better handled through education, not needless tools to ban users.

It’s also worth noting that all ‘bot’ lists have false positives in them, so if you ban people just for being on some persons ‘bot’ list, you’ll also be causing more ‘harm’ to your own channel by banning legitimate users than any perceived ‘harm’ caused by the bots themselves.

[losfarmosctl]

@5cottFive

Creators should have the ability to detect, and disallow bots from their channels. To me, that’s a bigger privacy concern. IMHO, this change harms creators by taking that power away from them, and makes Twitch less safe.

You can always just connect to chat anonymously, I really don’t see the threat of anyone just sitting in a chat with an authenticated account, as long as it’s not a bot actually sending any messages (which of course you would be able to detect)

Additionally some number+ channels is not a good indicator if someone is a bot or not, there are a lot of just regular users with 150+ channels opened in something like chatterino.

[5cottFive]

Okay, wow. So, there’s a lot to unpack here.

I’m not sure this is the place for an extended discussion about the justifications for banning bots, so I’ll be brief. People can skip to the bottom heading if they want to stay on topic.

First, I take issue with your insinuation that because we disagree that I must somehow be uneducated on this topic.

The reality is that I, and many other streamers that I personally know, do not want these bots in our chats. You don’t get to decide what is harmful for us, or the myriad of reasons we want these bots removed. That should be enough to end the discussion.

I’m astounded that Twitch lets bots operate so unrestricted on their systems, and I would like to see Twitch take this seriously. There should be at least some level of responsible disclosure required such as the bare minimum of: 1) who owns/runs the bot, 2) what is the bot doing, exactly. If a bot collects any data at all, perhaps it should also have a posted privacy policy.

If you are scared that disclosing this information will make people ban your bots from their channels, then perhaps you need to rethink the validity of what you’re doing with your bot, and instead offer something that adds value to a community. The norm should not be that streamers are just expected to accept bots they don’t want. This mindset needs to change.

Back to the discussion about this endpoint:

There is NO harm in identifying accounts as bots, giving streamers that data, and letting them decide, with their own choice, who they want in their channel. That is very much in line with the ToS and Community Guidelines, and also the privacy directions in which Twitch is currently heading.

The reality is that streamers are going to continue banning accounts which they suspect to be bots. Taking down this endpoint will not change that. But it will significantly raise the false positive rate. And that will continue to rise until Twitch provides better data, and options, sadly.

Taking down this endpoint leaves a very large, noticeable gap that is not filled with the recommended solution. And Twitch needs to know that, and provide another, workable solution, IMHO.

[5cottFive]

@losfarmosctl

there are a lot of just regular users with 150+ channels opened in something like chatterino.

I’m curious – why would someone need to do this? How is this even manageable? I feel like it is Humanly impossible to be an active chatter in 150+ channels at the same time.

[5cottFive]

@losfarmosctl

I really don’t see the threat of anyone just sitting in a chat with an authenticated account

Let me remind you of the well-established rule in the infosec world: Never assume you can predict how a nefarious actor will abuse a system.

For example, I’ve run across command and control bots sitting in chats that then direct botnets against channels. I’d be willing to bet most people don’t know that that’s happening.

@losfarmosctl

Additionally some number+ channels is not a good indicator if someone is a bot or not

I agree, which is why I’d like to see Twitch implement better bot controls before taking down this endpoint.

I think Twitch might not be fully aware how people are using their systems, so I’m trying to point that out.

[Dist]

@5cottFive

First, I take issue with your insinuation that because we disagree that I must somehow be uneducated on this topic.

The reality is that I, and many other streamers that I personally know, do not want these bots in our chats. You don’t get to decide what is harmful for us, or the myriad of reasons we want these bots removed. That should be enough to end the discussion.

It’s a simple fact that there is a significant amount of misunderstanding and misinformation about these bots. As can be seen by scaremongering on social media, often resulting in people pushing ‘bot’ lists, and tools (which in some case negatively impact the users if misused) that aren’t really needed.

No one is saying you shouldn’t be able to ban whomever you want, but any ‘bot list’ you’ve been using with the now removed Chatters endpoint means you’ve been banning actual users who aren’t bots too and users need to be aware of that.

@5cottFive

I’m astounded that Twitch lets bots operate so unrestricted on their systems, and I would like to see Twitch take this seriously. There should be at least some level of responsible disclosure required such as the bare minimum of: 1) who owns/runs the bot, 2) what is the bot doing, exactly. If a bot collects any data at all, perhaps it should also have a posted privacy policy.

If you are scared that disclosing this information will make people ban your bots from their channels, then perhaps you need to rethink the validity of what you’re doing with your bot, and instead offer something that adds value to a community. The norm should not be that streamers are just expected to accept bots they don’t want. This mindset needs to change.

All Twitch bots must follow Twitch’s Terms of Service, Community Guidelines, the Twitch Developer Agreement, and all applicable privacy laws. None of these legal agreements consider a bot that idles in a channel and does nothing whatsoever to be ‘harmful’ to a channel. You may disagree with Twitch, in which case take it up with Twitch Legal to charge their legal agreements that such bots must follow.

@5cottFive

There is NO harm in identifying accounts as bots, giving streamers that data, and letting them decide, with their own choice, who they want in their channel. That is very much in line with the ToS and Community Guidelines, and also the privacy directions in which Twitch is currently heading.

Incorrect. All accounts are user accounts, there is no ‘bot’ indicator, which means that targeting users just because they are in a large number of channels will have false positives so it can harm those users falsely flagged as being a bot.

Additionally, a public chatters endpoint has been used for harassment and stalking users on Twitch, so there can be cause for harm and safety on the platform.

@5cottFive

The reality is that streamers are going to continue banning accounts which they suspect to be bots. Taking down this endpoint will not change that. But it will significantly raise the false positive rate. And that will continue to rise until Twitch provides better data, and options, sadly.

What data are you basing this on? If fewer sites put together ban lists, there’d most likely be a reduction in automated banning and so the channels that do continue to ban lurkers will do so manually so the number of bans in total (and false positives) will be reduced.

additionally, the Twitch site itself shows the list of users in chat as “Some active viewers and chatters in the community.”. Key word being “Some”, so you you’re not even going to notice all of the bots. There could be many bots sitting in your chat that you’ve been perfectly fine with because you just didn’t realise they were there, and so had no impact at all on your channel.

@5cottFive

For example, I’ve run across command and control bots sitting in chats that then direct botnets against channels. I’d be willing to bet most people don’t know that that’s happening.

If you think a bots are violating Twitch’s ToS by doing such acts, report them.

Also just to reiterate, this endpoint that has shut down has never been officially supported for 3rd party use and so you should never have been using it to begin with. So if you wish additional functionality on the new endpoint that is supported for 3rd party use perhaps you should submit your feature requests to UserVoice https://twitch.uservoice.com/forums/310213-developers/

[DevMcGee]

@BarryCarlyon

This would also deviated from the RFC specification for IRC (I believe) which would cause issues for other users of the service, especially those on “regular” IRC clients.

Tags are permitted in any kind of message (including with the JOIN command). Conforming implementations must not care about unsupported tags.

@5cottFive

I’m astounded that Twitch lets bots operate so unrestricted on their systems

As already mentioned, they do not.

@5cottFive

I would like to see Twitch take this seriously.

Same for lots of things. Alas…

@5cottFive

There is NO harm in identifying accounts as bots

Didn’t you just say people don’t get to decide what is harmful for others? There are potential harms in misclassifying users as bots who are not.

@Dist

there is no ‘bot’ indicator

There is but the public endpoint was removed.

@Dist

a public chatters endpoint has been used for harassment and stalking users

Removing the endpoint is not the correct solution, especially when the same data are available through other means (and much harder for Twitch to track).

@Dist

What data are you basing this on?

Where there’s a will, there’s a way. Even if the data are completely fabricated.

@Dist

If fewer sites put together ban lists, there’d most likely be a reduction in automated banning and… the number of bans in total (and false positives) will be reduced.

Or more streamers will rely on the same unreliable datasets, possibly increasing the rate of false positives.

@Dist

additionally, the Twitch site itself shows the list of users in chat as “Some active viewers and chatters in the community.”.

The chatters list was rarely complete and never reliable and the phrasing now partly reflects that. (Also, that information is provided publicly on the Twitch site but developers are forbidden from having the same information under the same conditions.)

@Dist

There could…

This is a dumb argument. Someone cares and wants it and you are essentially arguing that because the lists can’t be fully complete and correct, that they shouldn’t be done at all. The lists exist and people use them. Why do you care so much?

@Dist

this endpoint that has shut down has never been officially supported for 3rd party use and so you should never have been using it to begin with.

Twitch never officially supported it. Twitch did support it. At some point there was some reason for that and they continued supporting it well after they no longer had any interest in it. Only recently have they started removing these sorts of things.

At least this time there was a warning. Many other unofficially supported endpoints were removed without warning.

[losfarmosctl]

@5cottFive

Let me remind you of the well-established rule in the infosec world: Never assume you can predict how a nefarious actor will abuse a system.

For example, I’ve run across command and control bots sitting in chats that then direct botnets against channels. I’d be willing to bet most people don’t know that that’s happening.

I haven’t really brought my main point across, the focus was on it being an authenticated account. Because if you don’t want to get banned, just connect anonymously and you can join as many chats as you want, without anyone even noticing.

@5cottFive

I’m curious – why would someone need to do this? How is this even manageable? I feel like it is Humanly impossible to be an active chatter in 150+ channels at the same time.

Sure, you can’t actively chat in all of those chats, but you might want to see whats happening, get pings on someone mentioning you in there, etc. It is a thing and it’s something where I’ve seen quite a lot of people I know being added to some botlists.

[EchtkPvL]

@losfarmosctl

Sure, you can’t actively chat in all of those chats, but you might want to see whats happening, get pings on someone mentioning you in there, etc. It is a thing and it’s something where I’ve seen quite a lot of people I know being added to some botlists.

I’m such a user. I have a bot, that is in every channel I follow because I’m connected with the communitys. Everytime someone is mentioning me, I get a notification and can reply. In one community where I’m a mod this is also a method to contact me if the stream isn’t live.
Because of this “bot-lists” I’m now banned in some streams. But I think its a bigger loose for the streamers. They didn’t saved the chat from a harmful bot, they lost a potential viewer/supporter/donator/community member.

I can just repeat what I and others already said: Lurker (Bots) don’t need to be banned. Everything harmful can and will be done with anonymous login.

[5cottFive]

@DevMcGee

@5cottFive

There is NO harm in identifying accounts as bots

Didn’t you just say people don’t get to decide what is harmful for others? There are potential harms in misclassifying users as bots who are not.

I’ll rephrase: There is no harm in appropriately identifying bot accounts as bots.

[5cottFive]

@losfarmosctl

the focus was on it being an authenticated account.

I’ve found authenticated accounts acting as C&C, so I don’t think it matters whether they are authenticated or not.

Also, given Twitch’s direction, I am fully expecting they’re going to shut down the ability to connect anonymously at some point in the near future, a change I would support.

@losfarmosctl

It is a thing and it’s something where I’ve seen quite a lot of people I know being added to some botlists.

Noted. Clearly, I need to study this use case more. Thx!

[5cottFive]

@EchtkPvL

Because of this “bot-lists” I’m now banned in some streams. But I think its a bigger loose for the streamers. They didn’t saved the chat from a harmful bot, they lost a potential viewer/supporter/donator/community member.

I totally agree that false positive are something we want to avoid.

My algorithm relies in several factors before banning an account, not just “bot lists.” In the 3 years I’ve been doing this, I’ve only had 1 false positive, and that happened this week after this endpoint was taken down.

@EchtkPvL

Lurker (Bots) don’t need to be banned.

To be clear, that’s up the streamer to determine.

@EchtkPvL

Everything harmful can and will be done with anonymous login.

See above for more, but I expect anonymous access to be ending soon which puts the emphasis on policing authenticated accounts.

[EchtkPvL]

@5cottFive

I’ve found authenticated accounts acting as C&C, so I don’t think it matters whether they are authenticated or not.

How could you identify a C&C account? And if you can, why don’t you report the account. If it is so easy, twitch will ban them as they do with spam accounts.

@5cottFive

See above for more, but I expect anonymous access to be ending soon which puts the emphasis on policing authenticated accounts.

Anonymous access will never be removed as the whole concept of the site relays on accessing the site without an account. If the anonymous login will be shut down, then you could not read the chat on twitch.tv if you aren’t logged in.

[DevMcGee]

@EchtkPvL

Anonymous access will never be removed as the whole concept of the site relays on accessing the site without an account. If the anonymous login will be shut down, then you could not read the chat on twitch.tv if you aren’t logged in.

They won’t intentionally harm the user experience on their own site. I would not be remotely surprised if they removed unauthenticated access to tmi.

[5cottFive]

What makes you think I didn’t report the C&C and entire botnet?

@EchtkPvL

Anonymous access will never be removed as the whole concept of the site relays on accessing the site without an account.

I don’t think it does, but regardless, even so, there are ways to lock down data. Just off the top of my head, Twitch could allow anon access, and still show chat message, but limit the data that is available to anons. They could obfuscate, or totally mark registered chatter’s names as “Private,” or blurred, etc. when viewed by anonymous users.

That forces people to create an account to engage with the community. I’ve successfully done this with other sites. It drives signups because people want to be included in the fun.

I’m just saying, I see Twitch moving in the direction of being more privacy-minded, which I think is a good thing. I hope I’m right.

[losfarmosctl]

@5cottFive

I don’t think it does, but regardless, even so, there are ways to lock down data. Just off the top of my head, Twitch could allow anon access, and still show chat message, but limit the data that is available to anons. They could obfuscate, or totally mark registered chatter’s names as “Private,” or blurred, etc. when viewed by anonymous users.

This will at the very earliest happen when/if Twitch moves away from IRC. But even then I don’t see Twitch pulling anonymous chat for a livestreaming platform. The amount of actually valuable signups will not increase if you lock the main feature streaming platforms have over just normal VODs (Chat), compared to just giving people access and allow them to create an account for convenience if they do like the site. A forced signup isn’t what you want, you need people that will actually watch content, ads, pay for subs, the people that make you profitable. And if you drive away a portion of potentials there and instead get a ton of inactive forced singups, I can’t see the benefit there.

They might lock it behind some first-party endpoint and not document it, but like the follow and mods/vips endpoint, if anyone wants that data for malicious purposes they will just use GQL anyways, who cares about some TOS at that point. And the stuff you are talking about, like C&C bots, they will for sure just go ahead and use some 1st party stuff.