Extensions today are usable on Desktop Web only. This proposal seeks to expand the ability for viewers to see and interact with extensions on our Mobile App.
- Bring extensions to a new platform by offering mobile support for extensions
- Extensions with mobile support enabled will occupy the same area as chat in Mobile and Tablet Apps
- Developers provide a mobile-specific front end file for their extension
- The extension’s type on web does not impact how the extension is presented on mobile (for example a video overlay on web will show in the chat area on mobile)
- Users can interact with one extension at a time on mobile
While the majority of users on any given channel are on web, a large and growing number of users access Twitch via our mobile apps. Without the capability for these users to interact with extensions, broadcasters are left with the challenge of either foregoing the new interactive experiences that extensions provide or giving a confusing and lesser experience to their mobile viewers.
This represents our first of hopefully many projects on mobile. We recognize that not all use cases will be served by this initial design but our goal is to allow the majority of extension developers to start engaging mobile audiences.
While viewing a channel in an app, users will see an icon in the chat header to indicate that they can collapse chat.
By clicking on the arrow (iconography not final), they will be taken to a view of all extensions with a mobile view on the channel sorted by installed anchor type on web (Overlay, then Components, then Panels).
Tapping on another extension’s name will swap to that extension. Swiping left or right will also switch to the appropriate extension based on swipe direction.
Tapping on the currently selected extension’s name will bring up the extension settings menu, allowing users to share their identity or get more details about the extension.
Lastly, Twitch actions (follow and share identity) are supported and will work on mobile in much the same way as on web.
The broadcaster experience will be nearly identical to the experience today. The only changes are visibility changes. Extensions that support mobile will be badged specially in the Extension Manager and their details page will note that they support mobile. Specific of this badging are still TBD.
Developers, not broadcasters determine whether an extension is available on mobile. Broadcasters will not have the option to disable the mobile experience for extensions and thus impact on broadcaster setup is minimal.
When configuring an extension version, a new field will be available allowing you to specify a path to your mobile viewer HTML. Presence of a valid mobile path is all that is required to enable mobile for your extension.
API and Lifecycle changes
Visibility and Extension Load
On web, whenever a user visits a page, all extensions on the page are loaded as soon as video playback begins. On mobile, this paradigm is changed. Due to constrained resources on mobile, we only load extensions as they are requested by the user. As such, until a user tries to view your extension it will not be loaded.
Your extension may be unloaded once we determine users are no longer intending to view it (typically when they have stopped looking at it for 5 minutes or longer but potentially based on other factors as well)
When a user stops viewing your extension you will stop receiving onContext updates but when the user begins viewing again you will be sent a fresh context object with the latest data.
visibilityCallback will be fired whenever visibility changes and has two arguments
Property Type Description visible boolean true if the extension is visible, false if not context object Only given when visibility is true. The entire context object. Used to refresh your app if needed once visible
Changes to onContext
In general, we have endeavored to keep the context object unchanged but in the instances below that was not possible.
Property Type Description of Change isTheatreMode boolean Removed for Mobile only. Theatre mode does not exist on mobile theme string Added for all Platforms Indicates which theme the user is using. Valid values are
For most extensions, you should be able to tell what platform you’re on simply due to the file that is loaded. If, however, you are using the same file for all platforms you can detect your platform using a new query parameter we will place on your viewer html as we load it:
platform=web will be added to viewer html on web for consistency)
Adding Mobile to Existing Extensions
Developers will be able to convert all existing extensions to utilize mobile, regardless of what type of extension they are on web. The process for this will be identical to releasing a new version. No additional work needs to be done on the part of a broadcaster to upgrade to a mobile experience.
We aim to give all developers access to start working on Mobile extensions approximately 3 weeks before our public launch. During this time, we hope existing extension developers will investigate and start building out mobile experiences. This will be an invaluable time for the Dev community to provide feedback on the product before we release these extensions to broadcasters.
As the developer beta begins, we will publish design and technical recommendations for building mobile extensions.
Lastly, during this period, any extension with mobile enabled will not be able to enter review but all steps prior to that will be possible.
Following the developer beta, we will launch a new version of the app with extensions enabled. Consumer adoption should be expected to be slow as is the case with all app versions.
Developers will have to think responsively
Compared to panel extensions today which have static height and width, Mobile extensions will vary based on device. Rather than constraining to a specific size, we have opted to have extensions take up a percentage of screen real estate. This puts the burden on developers to present a good experience regardless of size.
Developers thinking mobile first, however, will have an easy path towards panel or component extensions on web if they develop responsively.
After early prototyping, we determined that the experience for video overlays on mobile was not going to be the smoothest either for end users or developers. Given limited screen real estate – especially when player controls are take into account – usable screen area was quite small.
We experimented with an “interactive mode” to toggle between player and extension interactions but ultimately felt this was too much of a departure from how the app worked today.
We will likely revisit this idea in the future as users grow more accustomed to having extensions on mobile and valuable video-relative use cases continue to evolve.