Repository navigation
Write the frame packing SEI on every path that outputs 3D video, and find it in files that only signal 3D in the stream - #6389
Merged
SubJunk merged 3 commits intoOct 16, 2026
Conversation
UniversalMediaServer#6330 signalled 3D with x264 only. A video that is copied to MPEG-TS lost its Matroska StereoMode tag and no SEI took its place, x264 does not write the SEI for a right eye first video, and x265 does not write it at all, so renderers that auto-detect 3D showed these flat. FFmpeg's h264_metadata and hevc_metadata can write the frame packing arrangement SEI without touching the picture with the frame_packing option (FFmpeg patch series: avcodec/bsf/h264_metadata,hevc_metadata: add the frame_packing option). UMS adds it to the copy and to the x264 and x265 transcodes when this FFmpeg has the option, and keeps -x264-params frame-packing when it does not. Side by side and top and bottom, both eye orders; half and full resolution give the same message, as the SEI does not tell them apart.
MediaInfo reports the layout from the container, not the frame packing arrangement SEI of an H.264 or H.265 video, so a file that only has the SEI was not 3D to UMS: not in the 3D folder, not converted for Output3DFormat. Read it from the first frame with FFmpeg's showinfo filter, for H.264 and H.265 videos at least 1280 pixels wide that have no layout yet. Half or full is told from the size like it is for the file name.
danielcamposramos
force-pushed
the
stereo3d-every-path
branch
from
October 12, 2026 17:46
9d810ba to
b516036
Compare
Contributor
Author
|
Hi @SubJunk, and thank you again for merging #6330. |
Member
|
@danielcamposramos this is cool, I will merge it, thanks! Something I find hard about FFmpeg is that the syntax sometimes changes at seemingly random times, like in patch releases and they don't announce it. That has caused bugs for us before. We have a collection of small videos in Maybe you could help these 3D capabilities be more robust by adding some 3D files in there, with a test that spins up FFmpeg and makes sure the code in this PR continues to parse the output properly |
Contributor
Author
|
Thank you for merging it, @SubJunk, and for the idea. |
SubJunk
pushed a commit
that referenced
this pull request
Oct 16, 2026
…, and fix what they found (#6400) * Recognize half side by side and half top-bottom right eye first The frame packing check of #6389 names a right eye first video up to 1920 pixels wide "half side by side (right eye first)" and one up to 1080 pixels high "half top-bottom (right eye first)", but get3DLayout() had no mode for them and multiViewIsAnaglyph() took them for anaglyph. Such a video was not converted and not signalled, and for an H.265 video with Output3DFormat set, StoreItem called toString() on the missing mode. Map them to SBS2R and AB2R, like their left eye first twins, and draw 3D subtitles for them too. Assisted-by: Claude Opus 5.5 via Claude Code * Run the frame packing check with the FFmpeg of the FFmpeg engine FramePackingParser asked UmsConfiguration.getFFmpegPath() for FFmpeg, which is null on Linux when there is no FFmpeg on the PATH and UMS uses the one it bundles: the default type of that program info goes back to the installed one, which has no path. So the check of #6389 never ran there, and it did not follow the FFmpeg the user picked either. Ask the FFmpeg engine for its executable, as FFmpegParser does, and as FFMpegVideo already does when it writes the SEI. Assisted-by: Claude Opus 5.5 via Claude Code * Find the frame packing SEI in the output of FFmpeg before 7.1 too showinfo prints the Stereo 3D side data three ways: "stereoscopic information: type - side by side (inverted)" up to FFmpeg 5.1, "Stereo 3D: type - side by side (inverted)" in 6.0 to 7.0, and "Stereo 3D: type - side by side (inverted), view - packed, primary_eye - none" since 7.1. The check of #6389 read only the last, so with the FFmpeg of Ubuntu 22.04 (4.4) and 24.04 (6.1) or Debian 12 (5.1), which UMS runs on Linux when it is on the PATH, no file was found to be 3D. Read all three. The view is still checked when FFmpeg prints it. Assisted-by: Claude Opus 5.5 via Claude Code * Add 3D test videos, and test that FFmpeg and MediaInfo still find their layout Eighteen small videos made with FFmpeg from lavfi colour sources, LEFT and RIGHT written on each eye, five frames at 25 fps (eight at 50 fps, one eye per frame, for temporal interleaving), 112 KB in all. The frame packing arrangement SEI (H.264 D.2.26, H.265 D.3.16): side by side and top and bottom, left and right eye first, half and full size, in H.264 and H.265, in MP4 and MPEG-TS. The other arrangements of H.264 Table D-8 that x264 writes (checkerboard, column and row interleaving, temporal interleaving, 2D) and the temporal interleaving of H.265 Table D.8, which have no 3D mode in UMS and must stay 2D. And Matroska's StereoMode without the SEI, side by side and top and bottom, both eye orders. FramePackingParserTest parses them like any file, so MediaInfo reads StereoMode and FFmpeg's showinfo the SEI, and checks the layout and the 3D mode. FramePackingFilterTest copies the Matroska videos to MPEG-TS with the bitstream filter UMS adds, and checks that the copy is found to be 3D from the SEI alone; it runs when FFmpeg has the frame_packing option. Assisted-by: Claude Opus 5.5 via Claude Code
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#6330 made UMS signal 3D with x264. Three paths still send a flat picture to the renderers that auto-detect 3D from the SEI:
-x264-params frame-packing, and has no right eye first message (libx264.c logs "Ignoring unsupported inverted stereo value"), so it writes none.We wrote and tested a
frame_packingoption for FFmpeg'sh264_metadataandhevc_metadatathat writes the message while the picture is untouched. We held this pull request while we waited for confirmations and help on FFmpeg's side (our request FFmpeg #24531, and the related #24628 and #24643, are still open), and the option goes to FFmpeg as soon as they move; this pull request already works with any FFmpeg. When the FFmpeg that UMS runs has the option (UMS asks once withffmpeg -h bsf=..., and keeps the answer),FFMpegVideoadds it on the copy, on the x264 transcode (instead of-x264-params frame-packing) and on the x265 transcode, with the type and the eye order of the layout that leaves UMS. Without the option nothing changes: the x264 transcode keeps-x264-params frame-packing. Side by side and top and bottom, left and right eye first. The SEI does not tell half from full resolution, so both give the same message.The second commit finds the layout of files that signal 3D only with the SEI (MediaInfo reports the container, not the stream). For H.264 and H.265 videos at least 1280 wide and with no layout yet, UMS decodes the first frame with FFmpeg's
showinfofilter and reads the Stereo 3D line. Half or full is told from the size, the way the file name case does. These files were not 3D to UMS until now: they were not in the 3D folder andOutput3DFormatcould not convert them. It costs one FFmpeg run of a frame for each H.264 or H.265 video of 1280 pixels or more without a layout, with a 5 second limit. If you prefer a setting to switch it off, tell me the name you want and I will add it.How it was checked: a DLNA client in containers browsed UMS, played each title as four test renderers (copy H.264, copy H.265, re-encode H.264, re-encode H.265) and read the received MPEG-TS with FFmpeg's
trace_headersbitstream filter. The library has H.264 and HEVC files, side by side and top and bottom, left and right eye first, with the Matroska StereoMode tag only, plus a 2D file of each codec.Output3DFormat = sbsr): the 15.9.0 jar sends it unchanged; with the second commit UMS finds the layout, the eyes are swapped (PSNR of the received left eye against the source right eye: 45.9 dB, against the source left eye: 3.1 dB) and the message says right eye first.FramePackingFilterTest,FramePackingParserTest) and the existingFFmpegVideoTest,MediaInfoParserTest,FFmpegParserTest,FFmpegHlsSubtitlesTest,TranscodingSettingsTestpass.Not done: the renderer profiles are untouched, and nothing was tried on a real TV in this round (the earlier measurements on the KDL-46EX725 and KDL-46HX855 in #6329 are the reason for the SEI).
This text and the code were written with Claude (Anthropic) and checked by running them. I reviewed and submit it.