DV Pix - Multigeneration tests with different codecs
Pix: Sampling
Pix: Artifacts
Pix: Defects
Pix: Generations
> Pix: Codecs <
 
Text: The DV Format

[updated 11 June 2000]

Introduction

In July 1998, Joachim (Jack) Heden was working on a spot using DV compression, and was appalled at the difference in quality between graphics compressed using DV and the same graphics compressed using a JPEG codec of unspecified compression ratio. He went five generations and was horrified -- horrified enough to kindly provide his original graphic and his test results to several folks on the DV-L mailing list to see if we could reproduce his results.

Jack used the Mac version of Premiere (whether 4.2 or 5.0 is not known) and the FireMAX DVSoft codec. For each generation, he superimposed a small graphic in a corner of the screen, causing Premiere to decompress the clip, super the new graphic, and recompress it (essentially the same as Processing Inside the Editor).

I performed the same test on a PC using Premiere 4.2 and the Adaptec DVSoft codec (same basic code as the FireMAX DVSoft codec on the Mac) supplied with DPS Spark. I also more recently ran the test on a Mac using Final Cut Pro 1.2 and both the Apple DV - NTSC codec supplied with FCP 1.2 (and to included in QT 4.1) and the DV Plus codec in the ProMax DV Toolkit 3.0. DV Plus is derived from the DVSoft codec.

I also went five generations of making a DV-compressed clip, exporting one frame as a still, and then importing that still to make the next generation clip (see Import/Export), since that behaves quite differently from internal processing-induced decompression/recompression. In Import/Export (as you'll see below), the Premiere/DVSoft combo performs very well, whereas in internal processing the chroma information shows a left shift (about 2 pixels per generation). Codec guru Brad Pillow tells me this is a result of the simple 4-tap filter used for speed in the DVSoft/DV Plus codecs, and that a much better, but slower 9-tap filter exists in the code but has never been turned on. Perhaps in a future release of DV Plus... the DVSoft codec is no longer actively being developed, so it's unlikely we'd ever see the fix there.

Perry Mitchell ran a similar test using the Radius codec in AfterEffects on the Mac to generate his multigeneration tests. He repeatedly made an AE composition, forcing an internal decompress/recompress cycle each time.

Kent Borg ran multigeneration tests on the same image using AfterEffects with the DVSoft, Apple, and Radius codecs, which he comments on in detail in his DV Compression Experiments article at the Multimedia Workshop.

Dmitry Gromov has also tested a variety of DV codecs. Have a look at his page, then tell me: is it just me, or does anyone else think it odd that some of the MMX-enabled codecs cause a green shift? DVSoft and Microsoft both do, as does SoftDV (not yet posted on my site, but's noticeable, whereas the Mac version, like DVSoft without MMX, has no such color shift). Are we seeing a bug in the accuracy of the MMX engine? Or a bug in the reference implementation of DV on MMX that the codecs slavishly follow? The MainConcepts and Canopus codecs are clean, though... hmm...
 

Results

I have reproduced 1st and 5th generation results below. All are 128x60 pixel samples of the original image, enlarged by a factor of four. All were processed with 525/59.94 4:1:1 DV codecs.

All the images here are "high" quality JPEGs output with PhotoShop; there are very few visible artifacts added by this process and the JPEGs shown here are almost identical to the TIFF versions of the same pix.

The images labeled "1st" (2nd) and "5th" (6th) generations are really "half a generation more": these are generations following compression into a DV file on the PC using DVSoft, whereupon the AVI clip was imported into Mac QuickTime for testing. They show a bit of lateral chroma shift from that first compression, and are not directly comparable to the other clips as a result. The full images from those tests (including luma ramps and other details) are available here.

Likewise, Jack did not specify the location within the original frame of his original image (which is a 295x97 pixel chunk of the original 720x480 frame); Perry centered the image for rendering. My PC tests have it in the lower left corner of the frame; my FCP tests were run with the Stress Test image. This matters because differing positions will cause different alignments of image details with DCT block boundaries, so the comparisons below should be taken as general outcomes, not exactly comparable results. The exact positioning and extent of mosquito noise or chroma offsetting, for example, will differ depending on where the nasty edge is positioned within a DCT block.

Nonetheless, it's interesting to see how FireMAX (Mac DVSoft), PC DVSoft, and DV Plus all cause left chroma shifts. Also, the DVSoft codec shows no such shift when the export/import multigen testing is performed. What's the difference between cycling the codec by exporting/importing stills, and cycling the codec by decompressing/recompressing internally? As Click 'n' Clack would say, "How do it know?"

DV Plus performs just like DVSoft in the "clamped" mode; in the "full range" mode used to cope with white clip, it's very close, though there are minor errors in some luma / chroma values over multiple generations (some of which may be fixed in a later release, according to codec coder Brad Pillow).

The Radius (now Digital Origin, and a division of Media100, at that) SoftDV codec shows no chroma shift when performing internal processing in AfterEffects (Perry reports that occasionally he'll see a very slight vertical chroma shift when working in 4:2:0 with EditDV, but this may simply be the 4:2:0 sampling and decimation coming into play). There does seem to be a tiny fraction more mosquito noise buildup with SoftDV than with DVSoft/DV Plus. SoftDV, like DV Plus, offers a full range mode; it's not known if that mode was used in these images.

The Apple DV - NTSC codec supplied with QuickTime 4.0 (including the updated codec supplied with Final Cut Pro 1.2 and slated for general release in QuickTime 4.1) is a poor performer. Prior to FCP 1.2, the codec also showed odd striping on solid fields when played back through a Sony hardware codec, and many folks were rendering with the "Medium" setting instead of the "Best" setting to work around this (the FCP 1.2 release's update works around the striping by adding a slight dither pattern to solid areas). As the tests show, the Medium setting is much worse than the Best setting.

Apple is aware of the problems with the current codec and are working on an improved version. The QuickTime 4.1.2 and Final Cut Pro 1.2.5 releases (June 2000) use a codec that allows YUV processing, thus avoiding white clip issues and the rounding errors of the YUV-RGB conversion, but reports indicate that poor compression quality is still an issue.

I still need to test the Canopus codec (Yamada-san, want to lend me an eval unit, grin?), which is reportedly very good.

You can also display this page split into two windows side-by-side or one above the other so you can scroll the pix next to each other for comparison purposes. Your browser must support frames for this to work.

Original, uncompressed graphic
Original, uncompressed image
1st GENERATION: Mac, Premiere ?.?, FireMax/DVSoft  1.?
1st Generation with FireMAX codec
1st GENERATION: PC,  Premiere 4.2, DVSoft  1.1
1st Generation with DVSoft codec
"1st" (2nd) GENERATION: Mac, Final Cut Pro, DV Plus clamped luma/chroma
1st generation with DV Plus clamped luma / chroma
"1st" (2nd) GENERATION: Mac, Final Cut Pro, DV Plus full-range luma/chroma
1st generation with DV Plus full-range luma / chroma
"1st" (2nd) GENERATION: Mac, Final Cut Pro, Apple DV - NTSC, Best quality
1st generation with Apple DV - NTSC Best
"1st" (2nd) GENERATION: Mac, Final Cut Pro, Apple DV - NTSC, Medium quality
1st generation with Apple DV - NTSC Medium
1st GENERATION: Mac, AfterEffects, Radius
1st Generation with Radius codec
1st GENERATION: PC, Premiere 4.2, DVSoft 1.1, EXPORT/IMPORT
1st generation with DVSoft, export/import
5th GENERATION: Mac, Premiere ?.?, FireMax/DVSoft 1.?
5th generation with FireMAX codec
5th GENERATION: PC, Premiere 4.2, DVSoft 1.1
5th Generation with DVSoft codec
"5th" (6th) GENERATION: Mac, Final Cut Pro, DV Plus clamped luma/chroma
5th generation with DV Plus clamped luma / chroma
"5th" (6th) GENERATION: Mac, Final Cut Pro, DV Plus full-range luma/chroma
5th generation with DV Plus full-range luma / chroma
"5th" (6th) GENERATION: Mac, Final Cut Pro, Apple DV - NTSC, Best quality
5th generation with Apple DV - NTSC Best
"5th" (6th) GENERATION: Mac, Final Cut Pro, Apple DV - NTSC, Medium quality
5th generation with Apple DV - NTSC Medium
5th GENERATION: Mac, AfterEffects, Radius
5th Generation with Radius codec
5th GENERATION: PC, Premiere 4.2, DVSoft 1.1, EXPORT/IMPORT
5th Generation with DVSoft, exporting/importing images

Other Results / Resources

The "DV Stress Test" runs made for the January 2000 Digital Video Café can be seen here.

See Kent Borg's  DV Compression Experiments  for additional tests on this same image.

Dmitry Gromov has also tested a variety of DV codecs using naturalistic scenes.

Two other tests look at multigeneration performance using the hardware codecs in DVCPRO/DVCAM VTRs and compare them against other formats:

The European Broadcasting Union (EBU) ran a series of multigeneration tests, rating subjective quality of DV compared to uncompressed 601 digital video and to Betacam SP. It makes for interesting reading.

The May 1999 issue of DV Magazine used the Tektronix Picture Quality Analyzer to compare a variety of formats, including DV. You can download PDF files of their testing methodology and their results.

Copyright (c) 1998, 2000 by Adam J. Wilt.

Images from Jack Hegen and Perry Mitchell reproduced by permission.
The MasterCard and VISA names and logos are trademarks of their respective copyright holders.

You are granted a nonexclusive right to duplicate, print, link to, frame, or otherwise repurpose this material,
as long as all authorship, ownership and copyright information is preserved, and a link to this site is displayed.


Contact me via email

 
 
 
 
 
 
 
 

Last updated 11 June 2000.