Secure decoder performance points might be invalid on API 35 devices #2856

Version

Media3 main branch

More details

A while ago the performance point workaround was disabled for devices running on API level 35 or later. According to the commit message and code comment, the reason why it shouldn't be required anymore, is that the CTS contain tests that ensure valid performance points.

Reference commit: 737bf08

We suspect that the performance points for secure decoders are still incorrect on some devices after API 34.

Specifically, we have been able to identify a list of devices that run on API 35 that still seem to suffer from the underlying problem with the reported performance points for secure decoders. Most of these devices though are not initially shipped with API 35, but received OS updates to get there.

Unfortunately, the change in the player does not mention which test(s) in the CTS specifically relate to this. I tried to find them, but I don't have a lot of knowledge about the CTS code base. So take my findings with a grain of salt.
I believe I have found that the tests might have some blind spots, especially in regard to secure decoders, which are not tested at all if my understanding of the tests is right.
Entrypoint to the probably related CTS tests

Even if there are tests and I was just not able to find the proper one, there is still the list of devices which are on API 35+ and seem to suffer from insufficient/invalid performance point reporting. (See the quite long list in the devices section).

It might make sense to open up the workaround for higher API levels again, but I would also very much appreciate your thoughts and insights on the topic.

 Decoder “c2.mtk.avc.decoder” supports
    aliases: [
      “OMX.MTK.VIDEO.DECODER.AVC” ]
    attributes: 0xa: [
      encoder: 0,
      vendor: 1,
      software-only: 0,
      hw-accelerated: 1 ]
    owner: “codec2::default”
    rank: 512
    profile/levels: [
          1/32768 (Baseline/5.1),
      65536/32768 (ConstrainedBaseline/5.1),
          2/32768 (Main/5.1),
          8/32768 (High/5.1) ]
    colors: [
      0x7f000789 (Surface),
      0x7f420888 (YUV420Flexible),
      0x13 (YUV420Planar),
      0x15 (YUV420SemiPlanar),
      0x14 (YUV420PackedPlanar),
      0x27 (YUV420PackedSemiPlanar) ]
    details: AMessage(what = 0x00000000) = {
        string bitrate-range = “1-60000000"
        string block-count-range = “16-14400”
        string block-size = “16x16"
        string blocks-per-second-range = “16-432000”
        int32_t feature-adaptive-playback = 0
        int32_t feature-can-swap-width-height = 1
        string max-concurrent-instances = “15"
        string measured-frame-rate-1280x720-range = “111-244”
        string measured-frame-rate-1920x1080-range = “69-152"
        string measured-frame-rate-320x240-range = “158-349”
        string measured-frame-rate-720x480-range = “156-343"
        string performance-point-1280x720-range = “120-120”
        string performance-point-1920x1080-range = “60-60"
        string performance-point-2560x1440-range = “30-30”
        string size-range = “64x64-2560x1440"
        int32_t feature-detached-surface = 0
      }
  Decoder “c2.mtk.avc.decoder.secure” supports
    aliases: [
      “OMX.MTK.VIDEO.DECODER.AVC.secure” ]
    attributes: 0xa: [
      encoder: 0,
      vendor: 1,
      software-only: 0,
      hw-accelerated: 1 ]
    owner: “codec2::default”
    rank: 512
    profile/levels: [
          1/32768 (Baseline/5.1),
      65536/32768 (ConstrainedBaseline/5.1),
          2/32768 (Main/5.1),
          8/32768 (High/5.1) ]
    colors: [
      0x7f000789 (Surface),
      0x7f420888 (YUV420Flexible),
      0x13 (YUV420Planar),
      0x15 (YUV420SemiPlanar),
      0x14 (YUV420PackedPlanar),
      0x27 (YUV420PackedSemiPlanar) ]
    details: AMessage(what = 0x00000000) = {
        string bitrate-range = “1-60000000"
        string block-count-range = “16-8100”
        string block-size = “16x16"
        string blocks-per-second-range = “16-243000”
        int32_t feature-adaptive-playback = 0
        int32_t feature-can-swap-width-height = 1
        int32_t feature-secure-playback = 1
        string max-concurrent-instances = “1"
        string performance-point-1920x1080-range = “30-30”
        string size-range = “64x64-1920x1080"
        int32_t feature-detached-surface = 0
      }

Devices that reproduce the issue

Device Models:
SM-A556B, SM-A528B, SM-A546B, SM-S928B, 2312DRA50G, SM-S918B, SM-S911B, SM-A536B, SM-S921B, SM-S901B, 22101316G, 23090RA98G, M2101K6G, SM-A346B, CRT-NX1, REA-NX9, SM-A336B, SM-A566B, 2201116SG, 2312DRAABG, SM-A236B, SM-S938B, SM-A356B, 24090RA29G, SM-A137F, SM-X216B, Redmi Note 9 Pro, ELI-NX9, SM-S908B, SM-X205, T Tablet, SM-A715F, SM-G991B, 2201117TY, Lenovo TB128XU, SM-S931B, SM-A725F, SM-A156B, 2406APNFAG, MAR-LX1A, 2109119DG, SM-A515F, SM-X200, SM-S926B, ALI-NX1, SM-A226B, RMO-NX1, SM-A525F, LLY-NX1, SM-A145R

Devices that do not reproduce the issue

No response

Reproducible in the demo app?

Yes

Reproduction steps

We could reproduce the issue with a DASH stream, that contains DRM protected tracks with a frame rate of 50 also for HD ready (720p) resolution.
The affected device has 'limited' performance point reporting for the secure decoder. e.g. there is only one performance point at the maximum resolution the decoder would support, which is 1080p for most of the affected devices.

Expected result

Performance points might be ignored on all API levels, if the devise seems to inaccurately report it's capabilities. Tracks should not are marked as not supported.

Actual result

Some tracks are marked as not supported, despite them being playable without issues.

Media

I don't have a particular media file I can share here right now. In case you find it necessary to have one, please let me know.

Bug Report

• You will email the zip file produced by adb bugreport to android-media-github@google.com after filing this issue.