Hi,
I have a problem with the “emotes” tag in some messages. In the following message (user info edited out) for example the emote index for the Kappa emote is 15-19, however the (unicode) character at index 15 is a (the correct indexes should be 14-18):
@badge-info=;badges=;color=;display-name=anahi_2334;emotes=58765:53-63;flags=21-25:P.6/S.6;id=c29fb36b-658b-4974-8206-de50af62b336;mod=0;room-id=175779148;subscriber=0;tmi-sent-ts=1596913673211;turbo=0;user-id=<snip>;user-type= :<snip>!<snip>@<snip>.tmi.twitch.tv PRIVMSG #luisormenoa27 :@zthegoin aya xd, yo pensé que era por la sala bebé NotLikeThis
The interesting thing is that this only happens in about one message out of every million.
Also, when I copy-paste those messages into my chat my program receives a message that is byte-identical (so no UTF8/16 conversion issues) but the indexes are correct.
What those three messages have in common is that they have a value for the “flags” attribute which is not documented anywhere. Also, when I copy-paste the message and I get the correct indexes, the flags attribute is empty. Unfortunately the flags attribute is not documented, so I don’t know if my problem is related to that.
I can rule out the following:
My program counts the characters incorrectly. (My program can extract the emotes just fine for all other messages that contain non-ASCII characters)
This is an issue caused by combining unicode characters or unicode normalization. (The first example contains no characters that are changed by normalization)
This only happens for messages that have non-ASCII characters followed by an emote at the end. (I have several messages containing <cyrillic text> Kappa where the indexes are correct)
In the examples I found flags=3-12:S.5, flags=0-6:S.6, and flags=21-25:P.6/S.6. What does this mean? Is this related to the incorrect emote indexes?