I was wondering if there was anyway to get a reference to the actual streamed video in an extension. For example, I’d like to pass the video as a texture to three.js or something similar to do fun video effects on the video. I tried embedding the stream through the iframe embed route, but it doesn’t appear that it’s working and I suspect it’s because iframes are blocked in general for extensions for security reasons.
Is there any way to achieve what I’m trying to do?
Not sure I’ve never tried to do what you want to do. Most of the things I’ve dabbled with are too heavy for client side, which means punting it server side and then out of date/way out of sync.
But depends what sort of interaction you want from viewers. I was just thinking you were making a pretty overlay which wouldn’t need/have any interaction (except maybe picking which filter)
On second thoughts what you describe sounds a little like the SnapChat extension, but that will be running a local compoenent to inject the “cat ears”/whatever and access the video stream locally and then thats in the live stream. Which is probably the most performant/works on all viewer computers.
I don’t think that an extension is able to break out of the iframe to see whats behind it, might means you will need a local component to capture the video/display on the streamers computer.
It seems like in general you can’t have an iframe in an extension at all, which may be the core issue. I think what I really need would just be the ability to get the stream from the channel the extension is running on, and basically pass that to the js code or use in a video tag or otherwise.
Hmm I definitely see some overlap of ideas there, but I think my usecase is still slightly different. The idea here is that any given viewer could have a different experience of the stream based on how they were interacting with the stream. For example if the stream video was mapped to a 3D cube, any given user could rotate the cube differently, independent of action by the streamer.
So said differently, no local data sending besides a stream.
Also looking at the rules I now see that you can’t use iframes (makes sense).