Skip to content

RenderingDevice.texture_get_data_async() silently returns another texture's data when one frame's downloads approach the staging pool size #124883

Open

Description

@A-Games-Dev

Tested versions

Reproducible in: v4.7.2.stable.official [ed1daf0]
Other versions not tested.

System information

Godot v4.7.2.stable - SteamOS 3.9.5 (build 20261023.100), Linux 7.2.7-valve2 - Wayland (KDE) - Vulkan (Forward+) - dedicated AMD Radeon RX 7700S (RADV NAVI33), Mesa 26.2.0-devel (git-5c65d4fc31), Vulkan 1.4.350 - AMD Ryzen 7 7435HS - 16 GB RAM (ASUS TUF Gaming A16 FA617NTR)

Issue description

When several RenderingDevice.texture_get_data_async() downloads are queued in the same frame, and their combined size gets close to the staging pool size (rendering/rendering_device/staging_buffer/max_size_mb, default 128), one of them comes back with the wrong contents. Most or all of its data belongs to the texture downloaded just before it.

No error or warning is printed, even with --verbose. The callback fires normally, and the buffer has the correct size, so the corruption is silent.
It happens below the pool size. At 2560×1440 RGBA16F (28.1 MB per texture), four textures totalling 112.5 MB already trigger it.
It's deterministic. The same texture index is hit on every run, with the same number of wrong texels.
The GPU data is correct. The synchronous texture_get_data() returns the same textures correctly, and so does texture_get_data_async() once the downloads are spread over several frames.
A bigger pool hides it. Setting max_size_mb=256 (via override.cfg) makes the same case come back correct.
Measured with the attached MRP. Every texture is RGBA16F, cleared to its own constant colour with texture_clear(), then read back in one frame:

Textures × size Total Result
3 × 2560×1440 (28.1 MB) 84.4 MB all correct
4 × 2560×1440 112.5 MB texture 3: 3,645,440 of 3,686,400 texels wrong, holding texture 2's value
5, 6 and 7 × 2560×1440 140.6–196.9 MB texture 3 wrong, identical counts; all others correct
7 × 2560×1440, sync texture_get_data() 196.9 MB all correct
7 × 2560×1440, max_size_mb=256 196.9 MB all correct
7 × 1920×1080 (15.8 MB) 110.7 MB all correct
8 × 1920×1080 126.6 MB texture 7: 229,376 of 2,073,600 texels wrong, holding texture 6's value
9 and 12 × 1920×1080 142.4–189.8 MB texture 7 wrong, identical counts; all others correct
2 × 4096×2048 (64 MB) 128.0 MB texture 1: 6,291,456 of 8,388,608 texels wrong, holding texture 0's value
What the numbers suggest (inference, not verified in the engine source):

One texture per frame: only one download per frame is affected, namely the one whose staging space reaches the end of the pool. Downloads after it come back correct.
Region-sized corruption: the wrong texels always come in whole multiples of 4,096 texels. That's one 64×64 download region (texture_download_region_size_px = 64): 3,645,440 = 890 regions, 229,376 = 56, 6,291,456 = 1,536.
Stale data: the wrong data is always the previous download's.
So my guess is that when the download staging pool fills up and its blocks are reused, the regions copied out for that download are read from blocks that still hold the previous download's data. Either the GPU-to-staging copy for those regions isn't waited on, or it's ordered wrongly against the reuse.

We hit this in a real project. A debug tool reads back about 200 MB of compute results in one frame, and one tensor came back partly overwritten by another, which our parity tests caught. Our workaround is to spread readbacks over several frames, staying well under the pool size.

Steps to reproduce

Unzip the MRP and run it from a terminal (it quits by itself): godot --path staging_overflow_mrp -s mrp.gd
It defaults to 7 textures of 2560×1440.
Options: -- --count=N --size=WxH changes the case, and -- --sync uses texture_get_data() instead.
Expected: RESULT: 0 of 7 textures read back wrong.
Actual: texture 3: CORRUPT, 3645440 of 3686400 texels wrong ... sampled texel value 0x3c003a8038003200 ... (texture 2's value) and RESULT: 1 of 7 textures read back wrong.
With -- --count=3 it passes; with -- --count=4 it fails the same way.
Add override.cfg next to project.godot containing:

[rendering]
rendering_device/staging_buffer/max_size_mb=256
With it, the default case passes.
The core of the MRP. Everything runs on the main RenderingDevice via RenderingServer.call_on_render_thread:

func _create() -> void:  # frame A
	for i in count:
		var f := RDTextureFormat.new()
		f.format = RenderingDevice.DATA_FORMAT_R16G16B16A16_SFLOAT
		f.width = size.x
		f.height = size.y
		f.usage_bits = RenderingDevice.TEXTURE_USAGE_SAMPLING_BIT \
				| RenderingDevice.TEXTURE_USAGE_CAN_COPY_FROM_BIT | RenderingDevice.TEXTURE_USAGE_CAN_COPY_TO_BIT
		var t := rd.texture_create(f, RDTextureView.new())
		rd.texture_clear(t, Color((i + 1) / 16.0, 0.5, 1.0 - (i + 1) / 16.0, 1.0), 0, 1, 0, 1)
		textures.append(t)

func _read_all() -> void:  # 5 frames later, all in one frame
	for i in count:
		rd.texture_get_data_async(textures[i], 0, _on_data.bind(i))

Each result is then compared byte for byte with an Image of the same size filled with the same colour in FORMAT_RGBAH.

Minimal reproduction project (MRP)

staging_overflow_mrp.zip (attached): project.godot (Forward+, no settings changed) and mrp.gd.

Not established:

Whether other GPUs or drivers are affected; only RADV on an RX 7700S was tested.
Whether uploads (texture_update and similar) have an equivalent problem.
The exact threshold. It falls between 110.7 and 126.6 MB with 15.8 MB textures, and between 84.4 and 112.5 MB with 28.1 MB textures, which suggests it depends on where each download lands in the pool rather than on the total alone.

staging_overflow_mrp.zip

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions