Jump to content
Lowepost

Using DWAA/DWAB compressed EXR


Recommended Posts

Hi, has anyone used DWAA/DWAB compressed EXR intead of uncompressed frames or prores/dnxhr as an intermediate codec? Could be a good solution for a frame-based workflow with UHD/4K material.

However With UHD footage and timeline I get 17 fps playback in Resolve 12.5.

With my current computer (4ghz quad i7) the processor usage is 80%ish, GPU is at 40%, Drive speed not a problem, 100MB/s off an SSD. I wonder what is the bottleneck with playback, probably the CPU?

 

 

  • Like 1
Link to comment
Share on other sites

1 hour ago, Emily Haine said:

Hi Jussi. DWAB is more efficient than DWAA because the algorithm is optimized for increased performance in most of the common post production tools. You should be able to notice the difference.

I get similar frame rate for both in Resolve. Piz compression gives me similar frame rate as well.

Edited by Jussi Rovanperä
typo
Link to comment
Share on other sites

Similar frame rates suprises me. It could mean that it is a bottle neck somewhere that restricts the full potential. It could also mean that the DWAA/DWAB compression algorithm is not well enough integrated in Resolve. In Nuke the difference is huge. 

  • Like 3
Link to comment
Share on other sites

10 hours ago, Thomas Singh said:

Similar frame rates suprises me. It could mean that it is a bottle neck somewhere that restricts the full potential. It could also mean that the DWAA/DWAB compression algorithm is not well enough integrated in Resolve. In Nuke the difference is huge. 

Thanks Thomas, that's good to know. I gotta check with the Resolve 14 beta, if there is a difference to 12.5.

Link to comment
Share on other sites

Resolve 14 b7:

PIZ 19.5fps

DWAA 45  15fps

DWAB 45 13.5fps

I tried different compression options between 0-45, but the files Resolve wrote are all similar in size, so something is wrong I guess...

 

 

Edited by Jussi Rovanperä
typo
  • Like 1
Link to comment
Share on other sites

Scratch Play can playback the PIZ / DWAA and DWAB files @25fps

 

EDIT: Sorry, that was a bit too optimistic, Scratch Play seems to cache single clip playback.

With Scratch I get realtime (25fps) for PIZ (cpu at 80%) , 22fps for DWAA and 20fps for DWAB (cpu at 100%).

With Resolve the cpu doesn't go to 100%, but stays at 80%.

Edited by Jussi Rovanperä
misinformation
  • Like 1
Link to comment
Share on other sites

5 hours ago, Jussi Rovanperä said:

With Scratch I get realtime (25fps) for PIZ (cpu at 80%) , 22fps for DWAA and 20fps for DWAB (cpu at 100%).

The way I thought Scratch worked was reading DWAB data by bundling all the lines which should mean increased speed. Can you comment on this @Mazze?

Edited by Emily Haine
  • Like 1
Link to comment
Share on other sites

On 23.8.2017 at 6:29 PM, Emily Haine said:

The way I thought Scratch worked was reading DWAB data by bundling all the lines which should mean increased speed. Can you comment on this @Mazze?

Not 100% like this, but we read the full file into memory as fast as possible and from there start the decode as fast as possible.

However, we don't need all the file data to be loaded already in order to start the decode :-) .

  • Like 1
Link to comment
Share on other sites

  • 3 weeks later...
On 8/31/2017 at 4:29 PM, Mazze said:

Not 100% like this, but we read the full file into memory as fast as possible and from there start the decode as fast as possible.

However, we don't need all the file data to be loaded already in order to start the decode :-) .

Does that mean DWAB is processed faster?

Link to comment
Share on other sites

Hi Guys.

I made some tests about this some time ago for our workflow.

I tested render speeds of the same shot from Scratch to different formats and compression's:

framestacks_01_rendertimes.thumb.jpg.2c3b526c947b9c76efa87b781f582ebf.jpg

Write speeds:

12s - DPX (16.56 fps)
18s - EXR - B44 (11.68 fps)
18s - EXR - B44A (11.72 fps)
19s - EXR - DWAA (10.66 fps)
20s - EXR - RLE (10.51 fps)
21s - EXR - none (10.05 fps)
22s - EXR - ZIP (9.61 fps)
26s - EXR - ZIPS (8.06 fps)
26s - EXR - PIZ (8.13 fps)
30s - EXR - DWAB (6.94 fps)
33s - EXR - PXR24 (6.28 fps)

 

-And read speeds in nuke of the frames from scratch: (the results here is only for one frame, but I did the test for 100 frames as well. For 100 frames the numbers are just higher all around, but the resulting order is the same, and the smaller numbers are easier to relate to) framestacks_03_nukeDragAndDropIn-oneFrameLoad_small.thumb.jpg.e09a56155071f66d143a41a9dad3837d.jpg

Read speeds:

53ms - EXR none
54ms - DPX
65ms - EXR RLE
87ms - EXR ZIP
302ms - EXR DWAA
380ms - EXR PXR24
398ms - EXR ZIPS
404ms - EXR B44
411ms - EXR B44a
471ms - EXR PIZ
847ms - EXR DWAB

 

-and file sizes also played a part in our evaluation:

framestacks_05_filesizes.jpg.1e2af53c4d0f4f455640c739b0c13252.jpg

 

We decided to use EXR DWAA as our all round got to format as it seems to suit our needs and is quite good all around.

If we need super high quality we go with PIZ or ZIP.

I know thees read and write speeds is not the same for all tools and it might also depend on hardware, but it is what I needed for our pipeline.

 

Cheers

Johs

 

 

Edited by Johannes Møgelvang
  • Like 4
Link to comment
Share on other sites

Just a little update on this - the latest SCRATCH build has a fix for EXR rendering, which also has an impact on reading EXRs that have been rendered with SCRATCH:

Until now, EXR renders did not always include the line offset table correctly, so when reading those renders back in, the table needed to be created manually, which could cause a playback performance drop for high resolution files.

Possibly, if you now re-run the tests with EXRs generated from the latest SCRATCH build, you might see an improvement on the playback side.

 

 

Cheers,

Mazze

 

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...