Extracting video data


#1

I wish to extract video as well as comments from a saved twitch video, say this one, can someone guide me the steps to do so,preferred as HTML objects.

If there is no such way whats the procedure to do some analysis on comment.

TIA
-Anurag


#2

So you wish to extract the Chat for a VOD?

There are no official end points to perform this action.

Any end points that may exist may break or change at any time as Twitch isn’t obligated to maintain them for third parties.

You could use inspector to investigate whats going on, but again it’s unofficial/unsupported


#3

So from the answer above I am assuming that there in no solution for me to work in my own machine locally, so for video source I have to be reliant on the uploader itself and use the browser JS console to do my tasks .
Correct me if you think I am wrong at any point

Thanks
Aurag


#4

um no…

lets start again,

from you OP, you seem to want to take a video url, and basically download the video file and the chat replay?

if that is so, there are no official endpoints to do this.


#5

sorry for a late reply, then whats the unofficial one.
Further once I noticed that there were some other HTML objects that told the stats of current game, is that some kind of extension


#6

No idea off hand I’ve not reverse engineered Twitch’s Website to find out.

What do you mean by HTML objects? Sounds like what you are trying to do is scrape data off the page instead of performing HTTP/API queries?

HTML doesn’t really have “objects”. HTML is a collection of Tags arranged in a XML type hierarchy and whilst that could be extracted into what Javascript would call and array or an object, depending how you key it. I’m not sure what you mean by “HTML Objects”.

Unless you mean the objects tag usually for embedding flash objects

https://www.w3schools.com/tags/tag_object.asp

Depends what stats you are on about…


#7

An example can be say a counter strike match between 2 teams so the net score of rounds won by each team can be considered a stat and i say it was being displayed separately than the video.

BTW is there some github repo which can provide a better view of the scraping ?


#8

Do you have an example of this?

The only CSGO stuff I know of on the Twitch website off hand is the automatic map detection to supplement the game directory filter.

Still trying to figure out what you are after…


#9

similar to OP.GG extension for LOL. It gives out current stats for the game.

extraction of comments with video and save it locally will work


#10

I believe LOL has a API that the OP.GG extension pokes to fetch the current game information and statistics.

There are no official end points to obtain chat from a VOD.


#11

Does RIG provide the feature to extract the comments ?


#12

The Developer rig is for the creation of a Twitch Extension.

There is no official end points to recover chat (comments as you call it) from a “saved twitch video” which we refer to as a VOD


#13

Waasn’t there an API for this ??


#14

There is no official API end points for obtaining the chat/comments present on a “saved twitch video” aka a vod


#15

The following undocumented API end point can load the first 59 messages if the VOD exist.
But this api call response with a json structure and this will change any time without warning.

Response the first 59 messages from the actual VOD and next page id:
https://api.twitch.tv/v5/videos/[VOD-VIDEO-ID]/comments?content_offset_seconds=0

https://api.twitch.tv/v5/videos/[VOD-VIDEO-ID]/comments?cursor=[next-page-id]


#16

since this got necro’ed with the unofficial endpoints answer…

Do make sure whatever you are doing abides by the “Twitch Developer Services Agreement”. Specifically XI. B


#17

So the using of undocumented API’s is prohibited? I can’t find any exact clear information about this in the Developer Agreement.


#18

hmm…
This all is so confusing, the other day I was looking into what they say official source but got stuck in the new python3.6 errors and documentation.
The other one I found was this one, it did work for the clips but didn’t try for the whole game video… maybe useful

Both of the above stated however are based on the API


#19

No.

Undocumented end points can and will change at any point or even break.

I was talking specifically about the extraction and storing of data.

Thats not an “official source”. There are no official API end points for obtaining the data you are after.


#20

Oh i understand. So if i want to develope a webapp which is store chat for the Twitch Vod which is uploaded to Youtube even if the vod doesnt exist anymore in Twitch is prohibited becouse of storing policy? :confused: