Bot whisper shadowbans

I built this bot for a caster. He requested that most of the bot’s responses be whispered so they don’t clog up the chat.

On particularly whisper heavy nights, the bot sometimes the ability to whisper entirely, which is really unfortunate because the bot uses whispers to send invites to chatters to play with the caster.

I can confirm that the bot is able to read and write to normal chat rooms, and that he’s sending the whisper messages to the irc socket. They work for awhile until they don’t, with no changes on my end.

I strongly suspect this is due to the whispers being misinterpreted as spam by some twitch chat algorithm due to their asymmetric nature.

My issue seems to match up pretty well with the issue outlined here

But that thread is locked with only a link to a form at the bottom. I filled it out a couple weeks ago and asked the twitch gods for clemency, but they continue to smite my bot in some pretty unfortunate ways when too many people want to interact with him.

Surely there has to be a better way of handling this. Does anyone know of good ways to avoid getting your bot’s whispering capabilities removed if they send too many whispers to too many people without having people whisper back? In the longer term, would it be possible to have a process that’s more official, effective and easy to find than filling out some form at the bottom of a locked forum thread?

We’re supposed to register applications and get these oauth tokens for any application that talks to the twitch API right? Would it be possible to add in some optional authentication with those when connecting to irc servers so that the the account is recognized as an account that a bot uses to chat, and it’s tentatively granted some leniency for whisper limits?

I mean, that’s obviously not perfect because spammers could just register applications for fake accounts and then be tentatively granted higher limits, but presumably there are other metrics to detect spam, like if an account gets blocked or banned or reported or what have you, and so leniency for those metrics could be lowered, plus it would be another step to take in the creation of spam accounts so that seems like a potentially meaningful offset as well.

Alternatively, if you think my issue is not related to whisper shadowbans, then please do let me know, as I don’t really know what else could be going wrong.

I can confirm that some of my bot users are experiencing the same thing. A response would be appreciated.

The short answer is that we have to do a better job defining how we want devs to use whispers. In general, we don’t encourage using whispers for direct communication by bots due to users not understanding why a bot is whispering them or viewing them as spam.

This was a very frequent topic with devs at TwitchCon and something that you all care about. I’m going to chat with the chat team about whispers and how we view it as a tool for developers. Fair warning: I’m on a two week vacation, so this will be delayed.

3 Likes

I’d avoid initiating whisper conversations with bots, i.e. the bot should not be the one to start the whispers. Replying to whispers sent to the bot shouldn’t cause problems.

1 Like

Thank you. I agree that conversation initiated by a user to a bot could be ignored and not considered spam.

What about a type of key like used for API calls that can be used when allowing whispers to tell good bots from bad bots.

I’d hate to lose the whisper capability altogether as it is an immense tool for reducing chat spam.

Or how about first authorizing/accepting whispers from someone who isn’t on your friends list before a conversation can start. Similar to what you do with messages from someone you’re not following.

Honestly this response doesn’t feel super uplifting to me.

In general, we don’t encourage using whispers for direct communication
by bots due to users not understanding why a bot is whispering them or
viewing them as spam.

When you say things like this, it feels pretty patronizing. It feels like you’re saying that you don’t trust developers and casters to come together to create good user experiences, where interaction with the bot is well defined and easily understood. But even if that were the case, where casters having bots whisper their viewers leads to unequivocally bad user experiences it feels like you don’t trust your users to be actual people with functioning brains who can block or report spam or even just vote with their feet to find a channel with a bot that’s more respectful.

I guess I can’t speak for everyone, but I feel like the community that I built the bot for and that I frequent most often does a really good job of ensuring that the viewers understand why the bot’s there and why it whispers them when it does. If they add a quote and it whispers them confirming that the quote was added, I feel like that’s a pretty straightforward interaction that’s easy to understand and hard to think of as spam. If a viewer joins the queue to play with the caster and the bot whispers them saying “I see that you joined the queue. I’ll whisper you again when it’s your turn to play.” I feel like it’s pretty clear why they’re receiving that whisper, and honestly if there are users who find that confusing, I’d go so far as to say that the confusion says more about the viewer than it does about the bot having whispered them.

And if a caster is worried about their bot’s whispers being viewed as spam, there’s a lot they can do about that. They can set up automatically repeated messages where the bot occasionally posts in chat about how it works and how to interact with it. They can just talk on stream to remind people of the bot and how it functions. They can ensure that the bot gets a fancy BTTV bot icon so that when people see the bot posting in chat they’ll know more immediately that it’s a bot, and when they see the bot whisper them later, they’re less likely to be confused.

Ultimately, in my experience, setting up this sort of environment where viewers know what’s up and see the bot as a valuable tool has been relatively straightforward. Saying that you don’t trust the use of whispers because it’ll confuse viewers seems like it’s selling casters, developers, and viewers all short.

I’m sure it’s difficult to differentiate between spam and legitimate whispers, and I’m sure that whisper spam is really irritating to your userbase. If there are technical reasons why it’s not really possible to differentiate between the two, or even if it’s just sufficiently difficult or expensive to do that twitch doesn’t see much value in doing it, that’s fine. I mean, that wouldn’t be my favorite answer the the world to this sort of question, but that’s an answer that at least makes sense to me. Your current answer leaves me scratching my head and questioning how willing I feel to put more time into developing my bot further. Which I realize in the grand scheme of things probably isn’t that important. There are better bots and cooler services out there. Still though, all I can do is honestly present my thoughts on your answer and hope that twitch is taking all (reasonable) perspectives into account when considering what it wants to do next.

1 Like

A public command getting a private reply makes the private reply look like spam, makes viewers unfamiliar with the channel become confused by the random command that didn’t get a reply and splits the user interface; an unexperienced user might not even notice the whisper until months later! I honestly don’t see a good reason to use that design, but I am biased due to my job.

2 Likes

Isn’t it also in an IRC Bot RFC about the fact that the channel bot SHOULD return output in the SAME medium that the message started in?

So

if you run a command publicly it should respond in public.
if you run a command privately it should respond privately.

It should not change mediums when replying to user in response to a command

That ONLY works if EVERYONE uses BTTV.
Which they don’t…

You have to go with the most common denominator, most users ARE idiots unfortunately. A bot can just of answered someones question and yet someone IMMEDIATELY ask the same question, because they don’t/didn’t read chat

Maybe, but I’m not familiar with it. In any case, an RFC is just that; a request for comments. A starting point for a discussion. They’re not written on stone tablets. (I mean, unless they’re ratified as an internet standard I suppose, but I don’t think there are any internet standards governing IRC bots.)

That ONLY works if EVERYONE uses BTTV.
Which they don’t…

It’s true, one of the tactics I mentioned won’t reach 100% of the viewership. That doesn’t feel like a very thorough refutation of my argument. Streamers still have a multitude of tools to help communicate with their audience about the channel bot. (or any other topic) I can’t think of one method of knowledge transmission that’s guaranteed to reach 100% of viewers, but despite this, communities still memetically develop unique knowledge. Some facts/jokes/responses/etc. always manage to become part of a channel’s culture, and I still don’t see a compelling reason why “This channel’s bot will sometimes whisper you its responses to your commands.” can’t be part of that knowledge for at least some channels.

You have to go with the most common denominator, most users ARE idiots
unfortunately. A bot can just of answered someones question and yet
someone IMMEDIATELY ask the same question, because they don’t/didn’t
read chat

I mean, the most common denominator is going to be community specific right? Depending on the size of the community and the kind of viewer it attracts. With that in mind, shouldn’t it ultimately be up to the caster what sort of demographic they’re aiming for, and what standards they think should be in place for their community?

The reason streaming is popular is because it builds communities and relationships. The primary way that happens on twitch is through twitch chat. If someone isn’t going to interact with it at all, why are they on twitch instead of watching youtube or netflix? Or perhaps if they want to write messages into the twitch chat, and expect to have them read, but don’t seem willing to spend time reading the messages of others, then surely at least some communities should have the right to say “This isn’t a viewer we’re interested in retaining.” right?

1 Like

That in itself would probably alienate a large amount of users, especially in larger channels - the same channels where bot whispers could be viewed as ‘spam’ due to the volume of whispers.

I think us dev’s are in the same position as Twitch, lots of ideas on how to handle whispers but…no real good way. In smaller channels, slow chat may not ‘warrant’ the use of whispers but then you can run into the chat being spammed by bot response. In high volume channels, chat is typically fast enough to spread bot responses but then people can miss it, so whispers may be more useful (which then leads to one of Twitch’s current problems; how do you discern valid whispers from spam whispers beyond the report tool).

I feel like these are some good points, and are definitely worth addressing.

Spam seems to be pretty widely regarded as irrelevant, unsolicited messages sent to large numbers of people. If a whisper is sent in response to a command, I feel it’s almost by definition guaranteed to be relevant, solicited, and sent to relatively few people. If the whisper clearly communicates why it was sent the only way to view it as spam would be to not read it, and if we’re allowed to categorize things as spam without reading them, then I feel like all bets are off. Now, is it possible that whispering users with bots could be spam? Of course, but you could just as easily post a message of random text to the public twitch chat every second and that would still clearly also be spam. I feel like whether or not a message counts as spam has less to do with the medium of that message and more to do with the value brought by the contents of that message.

As for your other point, I think I agree that splitting the request and response of a command across mediums is a little counterintuitive. The question becomes, “Are the benefits worth the cost.” I think they can be. I think a large part of that as I’ve said relies on explaining that the bot will sometimes send whispers in response to commands, so that people can get on board with that idea quickly. Another part of that I think just comes from having non-negligible benefits. The primary one of course is just removing unwanted and unnecessary messages from the channel’s twitch chat. I feel like maintaining a good signal to noise ratio in that chat is one thing that’s helped keep users paying attention to it. If it becomes mostly bot spam, it becomes harder to parse, with less return for doing so, and so fewer people are likely to read the messages that show up there. Another big point (and really the reason for posting this question) is that for some functionality, privacy is a fundamental feature of the message. On some nights the caster that I built the bot for plays quiplash, and having a bot with a !join command allows us to set up a priority queue where we can cycle in viewers in a fair and transparent way. This only works because the bot can whisper players automatically. They can get confirmation of joining the queue, the can ask the bot about their spot in the queue, they can leave the queue, and when it comes time to play with the caster, they can get the credentials they need to join automatically without the caster having to do anything. If the caster had to do that whispering himself it becomes difficult to maintain an entertaining vibe to the stream. If the bot puts the credentials in chat then obviously everyone can join and it doesn’t work. If every time someone joined, left and asked for their spot we put those responses in chat, it would quickly pollute chat with information that very few people care about, again making it difficult to parse, and taking the chat farther away from what we want it to be; a place for humans to interact with other humans.

I think I’m inclined to agree about bot spam in chat being primarily noticeable in smaller channels. If the channel I frequent more often had 600 concurrent viewers instead of 60, I may very well have different priorities in this discussion.

If twitch’s response is going to be, “Well, channels with less than a few hundred concurrent viewers aren’t really worth our time for this issue.” Then that feels bad but at least I understand it.

But even discounting the block and report tools, I feel like we have some mechanisms by which we can make inferences about the value of whispers being sent. I think the first and most important one would be verifying that there are one or more channels in common between whisperer and the person being whispered. While it’s possible that a spammer could get around that by joining a channel, spamming the viewers then leaving and joining a different one, there are ways to account for that. Looking at how long they’ve been in a channel’s list of chatters would probably be telling. It may also be worth checking if the whisperer is a mod in the channel(s) they have in common. Certainly not all bots are mods in their channels, but some certainly are and it seems like it might be meaningful data. It also seems unlikely for a single username to both be a helpful channel bot and a dastardly spammer. Once you can put a bot into one group or the other, you likely don’t need to check it as frequently. And while these checks sound computationally and temporally expensive, random sampling is probably your friend.

Again, I don’t know, I’m just some dude on the internet. I’m sure the twitch developers are a lot smarter than I am, and I feel like the odds are good that they could come up with something if it were important to them to do so.

Difference is one can be dealt with by numerous channel moderators, the other has very few people reviewing them globally. The current limits make for a balance that just barely works for both devs and admins.

Any news on this?

No update. I was on vacation for two weeks (as I mentioned in the first post) and just got back yesterday. :slight_smile:

Yea, I know. That’s why I gave you 1 day. Geez… :sweat_smile:

Update yet?

The current status is that we have an open issue to provide documentation, guidelines, and best practices around Whispers for developers. Nothing in flight yet, but I’ll let you know when we have something ready to go. :slight_smile:

2 Likes