Support spatial video (MV-HEVC) #2011

3D video playback: MV-HEVC is a common format for distributing 3D content, especially in broadcast and streaming scenarios. Lack of support prevents ExoPlayer from being used in 3D video players.
Free-viewpoint video: MV-HEVC can be used to encode multiple viewpoints of a scene, allowing users to interactively choose their viewing angle. This is relevant for immersive media experiences and interactive video applications.
Immersive media and VR: While other formats like multiview extensions of H.264/AVC are sometimes used, MV-HEVC offers improved coding efficiency, which is crucial for high-resolution immersive experiences.

Demuxing: Implementing a demuxer that can correctly parse MV-HEVC container formats (e.g., MP4, TS) and extract the individual views and associated metadata (view identifiers, dependency information, etc.). This might involve extending existing HEVC demuxers or creating a new specialized demuxer.
Decoding: Ensuring that the underlying HEVC decoder (e.g., MediaCodec) can handle the MV-HEVC bitstream. This might require specific decoder configurations or handling of MV-HEVC specific NAL units. If MediaCodec doesn't fully support MV-HEVC, a software decoder might be necessary as a fallback.
Rendering: Implementing a renderer that can correctly display the multiple views. This would likely involve creating multiple Surface instances and managing the rendering of each view. For 3D playback, this would involve rendering the left and right views to create a stereoscopic image.
Metadata Handling: Parsing and exposing MV-HEVC specific SEI messages and other metadata to the application, allowing developers to access information about the different views and control the playback experience.

Extending or creating a new Extractor: To handle MV-HEVC specific container formats.
Modifying or creating a new SampleQueue: To manage the multiple views.
Potentially modifying MediaCodecVideoRenderer or creating a new renderer: To handle the rendering of multiple views.
Adding new classes or interfaces: To represent and manage MV-HEVC metadata.
Make Mediacodec support Hardware decoding.

Using a separate, external decoder: This would involve extracting the MV-HEVC bitstream from the container and passing it to an external decoder library. This approach was rejected because it would add significant complexity to the integration and would not be as efficient as native support within ExoPlayer.
Transcoding the MV-HEVC stream to a single-view HEVC stream: This would involve decoding the MV-HEVC stream and re-encoding it as a standard HEVC stream containing only one view. This approach was rejected because it would introduce significant processing overhead and would not allow for 3D or multiview playback.
Relying on future Android platform support: While it's possible that future Android versions might provide more comprehensive MV-HEVC support at the platform level, relying on this is not a viable solution in the short term, as it would limit the availability of MV-HEVC playback on older devices.