When trying to speed up video uploads by using multiple upload threads the API behaves in a multitude of strange ways. The upload guide (https://dev.twitch.tv/docs/v5/guides/video-upload/) described no restrictions on order or other things when uploading files, however there seem to be some that are not documented.
My experimentation on the subject has lead to quite a few questions:
- When using more than one thread to upload video fragments the results seems to be unpredictable:
- Some uploads run into a processing error.
- Some uploads are stuck on processing forever (14+ days).
- Some uploads show as finished, have a thumbnail but are corrupt and cannot be played after a certain point.
- The documentation states no restriction on individual framgent sizes. Can these be chosen arbitrarily within limits or do they have to be constant except for the last one? I assumed they have to be of equal size so far.
- Do the fragments need to be uploaded in order? If so, that would make multithreaded uploads pretty much impossible.
- Can a cancelled fragment upload be repeated at a later point without failing the upload? This is something I am unsure of, as all uploads I ever tried resuming just failed processing.
There is also some more things about the upload process I found out so far:
- documentation does not mention the needed create_premiere option to be able to actually publish videos uploaded via the API (Video upload via API broken after introduction of premieres)
- upload always seems to happen via an upload endpoint with a US domain. Is there just this one?
- uploads are rather slow to pick up speed. I assume this to be a result of the TCP sliding transmission window mechanism. Even when using maximum fragment size allowed it takes longer than the fragment upload time to reach peak upload speed. Speed is reset after a fragment upload is finished, capping the effective speed at around 1-2MB/s. This is the main reason I tried getting multithreaded uploads to work…
Has someone encountered similar problems and has a solution for them?
Can someone from Twitch maybe elaborate how the upload process works internally and how the problems mentioned should be addressed?