|adamwilt.com > HD > Four Affordable HD Camcorders Compared, part 1||copyright © 2006-2007 Adam J. Wilt|
|HVR-Z1, GY-HD100, XL H1, and HVX200 compared E-E, with CineAlta and Varicam||search|
(An edited version of this article appeared online at www.dv.com in early 2006, and a condensed version appeared in DV Magazine later in the year. The text of this version is used with the kind permission of DV Magazine. Copyright © 2006 DV Magazine and Adam J. Wilt.)
On January 11th 2006, filmmaker Barry Green organized a side-by-side comparison of four low-cost HD camcorders: the Canon XL H1, JVC GY-HD100U, Panasonic AG-HVX200, and the Sony HVR-Z1U. Barry was prompted to do this by rampant speculation and widely varying reported performance figures for the various cameras; he wanted to know what the cameras did relative to each other. For reference, he added two "real" HD camcorders to the mix: the Panasonic HDC27F Varicam and the Sony HDW-F900/3 CineAlta. I had the privilege of crashing his party, and this is my report on how the tests were conducted and what we saw. I'm basing this both on impressions I had and notes I took on location, and on examining captured clips from all the cameras.
First, however, I need to provide some context and caveats. Camera comparisons are incredibly difficult to perform, to judge objectively, and to quantify. By their nature, they are open to errors of omission and commission, and to accusations of bias. At their best, they illuminate aspects of performance, but they can never completely encapsulate the entire scope of how a camera behaves and how it renders a scene, because there are simply too many variables to control.
When I write a camera review for DV Magazine, I normally spend ten to twenty hours over the course of several days just looking at the camera's output, both live and off-tape, on a picture monitor, waveform monitor, and vectorscope. Of that time, perhaps four or five hours are spent imaging test charts and working through the camera's rendering options--gamma, knee, detail, matrix, noise reduction, etc.--while changing exposure, frame rate, gain, aperture, and the like. In the rest of the time, I'll shoot real-world images, including interiors and exteriors with both subtle and coarse camera and subject motions, so I can see how the camera handles real-world scenes and also see what sorts of spatial and temporal artifacts appear in the recordings.
After all of this, I feel I have a fairly good understanding of how the camera will perform in various situations.
In a multi-camera comparison, it's virtually impossible to explore things with that degree of detail. Simply getting six cameras together along with tripods, test equipment, operators/observers, charts, models, signal routing/recording/monitoring gear, and a studio big enough to hold everything and everyone is a major accomplishment in itself. Barry was able to assemble the necessary constellation of players and equipment for a single day; clearly we weren't going to run exhaustive and definitive tests, but just as clearly there were valuable lessons to be learned by lining up all these cameras side-by-side. We could get a rough idea of how the cameras differed; we could get some baseline performance data, and we could figure out what we weren't able to explore in enough detail and what we screwed up, and thus should pursue in future tests.
There's also the danger of perceived bias. Much of the time when equipment is compared, it's in terms of a "shootout", with the goal of picking winners. Vendors often stage such shootouts with an eye towards making their own products look good, so many folks are suspicious of anyone claiming to perform an evenhanded comparison--after all, what's in it for them?
Indeed, within twenty-four hours of the test, discussion lists were ablaze with accusations of bias, mostly against the Canon XL H1. I state categorically that I saw no such bias at work: the entire crew appeared intent on making all the cameras look their best and on collecting honest results.
As it turns out, the Canon was disadvantaged slightly. Barry intended to have experienced operators for each camera, and he invited a couple of XL H1 owner/operators, but they weren't able to attend. Shannon Rawls, who kindly provided his XL H1 for the tests, freely admits to being a producer more than a cameraman, and my prior experience exploring the camera's menus was limited to about half an hour at DV Expo last December. Our collective lack of experience meant that we didn't choose optimal settings for knee and noise reduction, and this probably hurt the Canon's picture in the dynamic range test. I didn't realize this until after the fact, and I'll describe the problem in detail later.
We tested four low-cost HD camcorders, all of them using three 1/3" CCDs:
Canon XL H1 (http://www.canondv.com/), a 1080i-native HDV camcorder with interchangeable lenses, 1440x1080-pixel CCDs, and 24f and 30f modes in addition to 60i.
JVC GY-HD100U (http://pro.jvc.com/prof/), a 720p-native HDV camcorder with interchangeable lenses, 1280x720-pixel CCDs, and 24p as well as 30p recording.
Panasonic AG-HVX200 (http://www.panasonic.com/business/provideo/home.asp), a fixed-lens DVCPROHD machine recording on P2 cards in either 720p or 1080i/p modes. It is said to use 1080p chips with an unspecified horizontal pixel count (revealed: it actually uses 960x540p chips).
Sony HVR-Z1U (http://bssc.sel.sony.com/), a fixed-lens 1080i-native HDV camcorder using 960x1080-pixel chips.
We put this Gang of Four up against the 2/3" 3-CCD Panasonic HDC27F Varicam and the Sony HDW-F900/3 CineAlta. The Varicam has 1280x720p CCDs, while the CineAlta's chips are 1920x1080p-native. Both cameras had ENG-style HD zooms, but I didn't record model numbers.
We had hoped to use a 24" Sony HD CRT as our engineering monitor, but at the last moment it went out on a rental. We made do with two Sony Luma LMD-230W LCD monitors, which were clearly not 1080-line native displays, and a Panasonic HD plasma monitor of unknown, but better, resolution. Towards the end of the day, a small 1080i-capable CRT arrived, but as with the plasma, I failed to record the model number. A Leader LV 5750 HD-SDI WFM/vectorscope was used for signal monitoring. HD-SDI was taken directly from the Canon XL-H1 and the two 2/3" cameras; the others fed component analog into an AJA HD10A analog-to-HD-SDI converter.
Some material was recorded to the native recording format of the cameras under test: 720p or 1080i HDV, or DVCPROHD on a P2 card for the Panasonic. We also captured clips of every test in uncompressed 10-bit format to hard disk using a PowerMac G5 with AJA Kona LHe card, using Final Cut Pro as the capture application.
Barry's intent was to assemble a large group of folks to scrutinize the pix from all the cameras; furthermore he wanted each camera to have someone conversant with its features and operations and able to act as an advocate for that camera.
Evan Grant, DP / Director, scrutinized results and kept us honest.
Barry Green is the co-author (with Panasonic) of The DVX Book (www.dvxuser.com/articles/dvxbook/) and the upcoming HVX Book, and a beta-tester/owner of an HVX200 camcorder. Barry organized the tests and conducted them with Jay Nemeth. Barry set up the HVX200. Barry's writeup and the discussions it sparked are at www.dvxuser.com/V3/showthread.php?t=43409.
Jay Nemeth, Director / Cinematographer, served as DP/DIT (Digital Imaging Technician). He arranged the facilities, supervised the staging and technical operations, optimized the CineAlta and Varicam for the comparison, and provided the HVX200 used in the tests. Jay's commentary is at www.dvxuser.com/V3/showthread.php?t=43511
Shannon Rawls, Independent Producer, supplied the Canon XL H1 and Sony HVR-Z1 camcorders, and kept a close and interested eye on the tests.
Aaron Umetani, DP, scrutinized results and handled local transport and craft services.
Nate Weaver, Director / DP / GY-HD100 owner/operator, ensured the JVC camcorder was set up to look its best.
Adam Wilt, video geek, supplied a GY-HD100 on loan from JVC, scrutinized results, argued for "wobbles" in resolution testing, and set up the Sony, Canon, and to a lesser extent the JVC. In the interests of full disclosure, I own/operate an HVR-Z1 and I pre-ordered an HVX200 several months ago.
Big Vision Studios in Burbank (http://www.BigVision.com) supplied the facility, the Varicam and CineAlta, sticks, scopes, and monitors. A tip of the hat to Big Vision's Chuck Haifley and his staff for their support.
ProMax Systems (http://www.promax.com/) supplied a Quad G5 Mac with a 2 TB Huge Systems fibre channel array and AJA Kona LHe card to capture uncompressed output from all the cameras. Shane Sokolosky and Ernesto Sanchez drove the system up from Irvine, set it up, operated it, and supplied FireWire drives of the footage. Thanks, guys: the spirit of Charles McConathy lives on.
Thanks also to Michael Kent at DSC Labs (http://www.dsclabs.com/) who supplied a top-end ChromaDuMonde test chart (http://www.dsclabs.com/chromadumonde.htm), and Michael Bravin of Band Pro Film & Digital (http://www.bandprodigital.com/) who provided DSC's Ambi/Combi transilluminated resolution test chart (http://www.dsclabs.com/ambi_combi_system.htm).
Kacy Bult, actress, spent most of the day patiently waiting around, then served as the model in the dynamic range test scene. Thanks for your patience and good humor, Kacy.
Once we had all the equipment set up and had resolved various technical glitches, we had five hours left in which to play. Obviously we were not going to be able to perform exhaustive tests, so we settled on some basics: sensitivity, resolution, and dynamic range. Along the way we'd look at white balance, color response, and image tweaking, but we simply didn't have time to explore these in detail.
Since most of the folks involved were interested in film-style production, whether for film transfer or for "the film look" on video, the cameras were for the most part run in their 24fps modes, although we switched the Canon between 24f and 60i modes to ensure that we were getting the highest possible resolution from it ("24f" is not a true progressive mode, and I wanted to see if there was a difference in image quality between 24f and 60i). We also ran the Z1 in 60i mode, as its Cineframe24 (CF24) mode produces compromised results (see www.adamwilt.com/HDV/cineframe.html), and we also know that CF25 and CF30 modes result in a field-doubled image with half the normal vertical resolution.
Shannon argued against 60i: many of his films are shot in CF25 mode, simulating 25p, then slowed 4% to 24fps, downconverted to standard-def, and burned to DVD for distribution. This is a common "film-look" workflow amongst indies with this camera, and he wanted us to test it in the same way it would be used in the field. I agreed that this was a sensible workflow--the roughly 540 lines of vertical resolution downconvert to 480-line SD very nicely--but countered that testing in that mode would seriously compromise the Sony's performance, and we should see the best pictures the camera could make for all workflows, not just film-style work. As we were not planning to test motion rendering, Shannon agreed; the Sony remained at 60i the rest of the day.
Jay set up a white card illuminated by two tungsten-balanced Lowel Caselights (http://www.lowel.com/caselite/index.html) and we white-balanced the cameras.
All the cameras white balanced cleanly, but we saw some interesting differences on the vectorscope. The Varicam's vector display was much tighter than the CineAlta's, but both cameras showed even "pools of light" on the vectorscope: bright near the origin, with a smooth fade-out at the edges.
The Canon's display, by contrast, showed an obvious quantization, with a central square of intense data and a gridded penumbra on the 5x magnified display. The Z1 and HVX showed looser patterns, with regular horizontal and vertical bars of intensity (the photo shown doesn't do them justice). At first, we attributed the quantization to the AJA A/D converter, but when we hooked it up the Canon's analog outputs, the Canon's display was essentially unchanged, just very slightly fuzzier. Clearly, there's some sort of processing in the cameras that's causing the Cr and Cb signals to be coarsely quantized, but there was nothing detectably posterized on the picture display that we could detect as a result of this.
|XL H1 vector pattern||Z1 or HVX vector pattern|
I don't recall if the JVC showed a similar patterning or not.
The white card was swapped for a gray card, and the lights were adjusted to provide 120 foot-candles of illumination. Jay metered the card with his Spectra-Pro spotmeter set for ISO 320, 24fps, 180 degree shutter--hence 1/48 sec for the exposure--and got an aperture of f5.6.
All the cameras were set to their 24fps mode and 1/48 shutter save for the Z1, which we left at 60i, 1/60, because it lacks a true 24p setting. We selected "normal" or "standard" video gamma on all the cameras as it's the only gamma setting shared by all of them, so it was the only way to consistently compare exposures. We then adjusted each camera's iris until the gray card was reproduced at 50% brightness on the waveform monitor. We then read off the f-stop, and were able to determine the effective sensitivities of the cameras:
|Camera||f-stop||Effective ISO rating|
|HDW-F900/3 CineAlta||Just under 5.6||Just under 320|
|HDC27F Varicam||Just under 4||Just under 160|
|Canon XL H1||4+1/3||200|
|JVC GY-HD100||Just under 5.6||Just under 320|
We next shot the DSC ChromaDuMonde chart. It has an 11-step grayscale, white and black patches, resolution trumpets, and a sequence of precision color chips designed to map out a regular hexagon on the vectorscope. We didn't spend a lot of time on the color patches (ironically, since they are the main claim to fame for the ChromaDuMonde) since we could easily have spent the rest of the day just tweaking each camera for perfect color reproduction. Instead, we focused on the resolution trumpets to determine the native sharpness of the chips, and see what happens to detail too fine to be properly rendered.
The following test used the Combi-2.3 resolution chart. In addition to resolution trumpets, it has a frequency sweep consisting of a wide band of vertical lines of varying spacing, becoming more tightly bunched towards the right; large blocks of lines at fixed frequencies in both horizontal and diagonal directions; and "bullseye" patterns similar to a Putora sharpness indicator. The greater variety of resolution targets offered more chances to determine limiting resolutions, as well as chances to see what out-of-band detail looked like, and the bullseyes graphically illustrate H/V detail balance.
A (rather lengthy) digression: Measuring resolution on sensors with discrete photosites, like the CCDs in all these cameras, is a much more uncertain proposition than doing so on a tube camera with a continuous sensor. On a tube camera, one simply shoots the vertical patterns and looks for the point where the lines merge together; that's the limiting horizontal resolution. One can also shoot a "multiburst" pattern, showing a spatial frequency sweep, of light and dark lines and look at the WFM to see where along the multiburst pattern the waveform diminishes to zero. On a CCD, though, where that merger occurs depends on the alignment of the lines in the pattern with respect to the photosites on the chips.
When the spacing of the lines as projected onto the sensor exactly matches the spacing of the photosites, the spatial sampling frequency will exactly equal the detail frequency in the image; theory tells us we can't honestly do any better than that (strictly speaking, Nyquist's sampling theorem tells us the highest image frequency we can sample is half the sampling frequency, but "TV lines" consist of the black and white lines individually; it takes two TV lines to make a complete cycle as is normally measured by the theory).
Nyquist tells us the upper limit, and this holds when the lines exactly line up with the photosites on the chip: each photosite will be "painted" with a solid black line or a solid white line, and the image of the lines will have high contrast. But if one offsets the lines by half their spacing, one photosite will see half a white line and half a black one, the next photosite will see half a black line and half a white one, and so on; each photosite's average picture level will be a neutral gray:
What this tells us is that if we slowly pan across the resolution trumpets in the test chart, we'll see the trumpets going from contrasty to completely grayed out at or near the point of maximum usable resolution, with this dynamic change repeating as the pan continues (as in these two samples, mere frames apart, from the HVR-Z1).
Detail frequencies higher than the sampling frequency--image details finer than our CCDs can resolve--will result in aliasing if they aren't filtered out. Aliasing is the appearance of spurious or erroneous detail caused by high-frequency information masquerading as lower-frequency information. We've all seen filmed westerns where the wagon wheels appear to turn backwards; those are examples of temporal aliasing. In shooting the test charts, we can see an analogous artifact: as we pan across the chart, low-frequency detail in the resolution trumpets is accurately rendered, but detail finer than the CCDs can accurately capture appears as coarser detail that seems to shimmer and even move across the resolution trumpets in the direction opposite to the rest of the image. On the resolution trumpets, consisting of five black lines separated with white space, aliased detail appears as fewer dark lines, moving contrary to un-aliased detail--the backwards-wagon-wheels of the test chart world.
Aliasing means that the traditional practices of looking at a static image, or measuring the point where the waveform goes to zero on the WFM, are not useful. The initial extinction point of the waveform depends on the alignment between chart and CCD, and detail beyond the extinction point can and will cause spurious signals.
In real-world images, excessive aliasing shows up as spurious detail moving oddly or contrarily in the image, as crawling moiré on repetitive patterns, and as overly stepped edges along near-vertical lines (similar to the jaggies one sees on near-horizontal lines). Digital cameras employ an optical low-pass filter--essentially a bit of scratched-up glass just in front of the CCD or CMOS sensor--to blur out this excessive fine detail before it causes aliasing. However, it's nearly impossible to craft such a filter with a "brick-wall" response: one that passes all detail below the Nyquist frequency and blocks all frequencies above it. Camera manufacturers must choose between making the filter aggressive and blurring too much detail, or making the filter timid, preserving more scene detail at the expense of excessive aliasing; most bias their choice towards more detail and more aliasing.
Vertically speaking, the limiting resolution of each camera should be set by its format (1080 or 720), since the fixed scanning structure defines the line count. A 720p camera should show unaliased vertical detail to no more than 720 TV lines; above that count, aliasing of any visible detail is guaranteed. Likewise, a 1080-line camera's limit is 1080 lines for unaliased material, but interlaced cameras, with dual-row readout to avoid twitter, will typically lose detail above roughly 750 TV lines into a gray blur. If we see numbers below those theoretical limits, we'll know something else is happening. For example, the field-doubled CF25 and CF30 modes on the Sony Z1 result in a vertical resolution of about 540 TV lines, with aliased detail above that point.
The point of this digression is to explain why I insisted that we "wobble" the cameras during the resolution testing, so we could see how resolution varied depending on chart-to-chip alignment. I gently perturbed each camera while we recorded the charts, torquing the tripod head just enough to wobble the camera horizontally and vertically by a few percent. As you can see from the sample frame details, there's a considerable difference in the apparent resolution depending on alignment.
When I quote numbers, I'll use the point at which I see the resolution trumpets transitioning from unaliased to aliased. It's a somewhat subjective judgment, so don't use the numbers as absolutes, but it's a consistent standard that should make relative comparisons possible. For the Z1 samples shown, that turnover happens around 550 TVl/ph (TV lines per picture height).
(Those who read my FX1 and Z1 reviews may recall that I quoted an honest resolution of 650 TVl/ph for the Z1, not 550 TVl/ph. The resolution wedges on my 1972 PortaPattern chart appear to give slightly higher numbers than the trumpets on the DCS charts, possibly since the wedges on the PortaPattern have only four black lines compared to DSC's five: the more lines, the more easily aliasing and moiré are seen. Just now, I set up my Z1 with the PortaPattern chart, and found that in ramping the sharpness from zero to fifteen, the "honest" figures I saw on the PortaPattern ranged from around 550 TV lines to 650 TV lines. Yet when looking at the DSC wedges during a similar test of different sharpness settings, the five sharp lines retained their point of maximum usable resolution at 550 TVL/ph. At low sharpness settings, the lines blur into gray before decaying into aliasing, while at sharper settings the blurred-gray area vanishes, the trumpet simply transitioning from quiescent, properly rendered lines, to moving aliased ones, and fewer of them.
The additional resolution test patterns on the Combi-2.3 chart confirmed the results seen on the ChromaDuMonde. Clearly, I'm going to have to get a DSC chart or two before I quote any further resolution numbers in my camera reviews!)
The 1920x1080p CineAlta sets the sharpness standard: at the 1000-line limits of the charts, both horizontally and vertically, the resolution trumpets are trembling, but all five black lines are still present and fixed in place. On the Combi chart the frequency sweep decays into undifferentiated gray from just past 1000 all the way to 1200 lines, with only the slightest bit of aliasing visible. The fine detail in CineAlta images looks like images from a high-reolusion still camera sized to 1920x1080; there's almost no moiré or aliasing to be seen. Likewise, on real-world images like the dynamic range setup we did later, the pictures are crisp and natural with as much detail as a frame this size can hold.
The 1280x720p Varicam likewise has a clean and balanced 700+ line image in both H and V directions. It shows a little bit more aliasing and moiré past its detail limits than the CineAlta does, but it's not objectionably worse.
Of the 1/3" cameras, the 1440-pixel, 1080i Canon XL H1 was the clear winner in horizontal resolution, and it was second only to the 1920-pixel CineAlta. I see almost 800 TVL/ph from it, with minimal aliasing above that: an astonishingly good performance.
In 60i, vertical resolution was a solid 700+ lines; in 24f mode, we thought it looked like half-resolution material on the monitors, consistent with what you'd get with field-doubling. Looking at the captured clips, especially the Combi-2.3's bullseyes, I'd have to say that it certainly looks like field-doubled vertical sampling, with about 540 TV lines usable, although certain aspects of the image "feel" sharper. I'm not sure how to describe it, but perhaps Canon's secret sauce for creating 24f results in a perceptually sharper picture than plain field-doubling, even if the clues I can glean from the interference patterns on the bullseyes would indicate otherwise.
The XL H1's minimum sharpness value still showed some enhancement overshoot on the waveform monitor, so we weren't able to completely defeat its detail setting, but the detailing was minimal and mostly unobjectionable. Dialing sharpness up and down did little to the perceived limiting resolution, although it clearly affected the "crispness" of the image. At –7, the image was naturalistic; at +7 it was as edgy as a Sunday-morning talk show.
The 1280x720p JVC HD100 closely followed the Canon and the Varicam with almost 700 TVl/ph and 700 lines vertically. Everyone was very impressed (and some were more than a little surprised) at the JVC's strong showing (the JVC's recorded HDV resolution of 1280x720 exceeds that of the Varicam's 960x720 DVCPROHD; if recorded resolution were the only factor of interest, you'd probably buy the JVC!). The JVC, furthermore, showed the most pleasing rolloff in high frequencies of any of the 1/3" cameras; it had less visibly distracting aliasing than its stable mates, and less than the Varicam; more aggressive optical low-pass filtering may explain the lower aliasing along with lower perceived sharpness.
The JVC has a considerable range of detail settings, including "off". "Off" was truly off, while "min" was a bit edgy for the film folks among us; we agreed that we'd prefer a setting between "off" and "min". Even so, both were equally sharp at the limit, though we felt that the edginess of the images at JVC's default setting and higher were clearly in the electric-video realm.
The Sony Z1 and Panasonic HVX200 brought up the rear; as I look at the images I see them being equally coarse. Both cameras showed noticeable aliasing above the limiting resolution; to my eye they were equal in this respect. On the charts, I saw both cameras at 550 TVl/ph.
Changing "sharpness" settings on the Sony makes a large difference in perceived sharpness, with lower settings making the image very blurry and higher ones making it edgy, if no higher in resolution. The HVX200's corresponding control had a subtler effect; the machine holds its fundamental sharpness better with detail turned down, and it doesn't go quite as electric in its edginess when it's cranked up all the way. Sharpness on the Sony was at 5 and on the HVX we used –3 or –4, both to give the same apparent sharpness (as judged from edging artifacts) as the Canon did on its minimum setting.
P2 playback of 1080i from the Panasonic matched what we saw in E-E mode, but 720p playback was a bit softer and yet (to me) more pleasing, as the 960-pixel sampling of 720p DVCPROHD stepped on some of the near-Nyquist aliasing of the camera's sensors whereas 1080i DVCPROHD's 1280-pixel sampling let it pass. At 960 pixels per scanline, limiting resolution is 540 TVl/ph; at 1280, it's 720 TVl/ph. If we agree that the Panasonic, like the Sony, is a 550 TVl/ph camera, then it's very nicely matched to 720p DVCPROHD recording, but a bit on the coarse side for 1080i recording. In other words, it'll be a closer match, resolution-wise, for the Varicam than it will be for 1080i cameras.
Vertically, the 1080i Sony Z1 showed better than 720 lines, as expected, but the HVX in both 720p and 1080i showed 540 lines, as if its 1080p CCDs images were being field-doubled like the Z1's CF or the Canon's 24f modes. As far as we know, there's no setting on the HVX that causes such field-doubling (slow shutters aside), so this result is puzzling to say the least.
The HVX's anomalous vertical result also casts the horizontal numbers into doubt as far as I'm concerned; I'll want to test another sample of this camera a bit further into the production run to see if we saw a sample-based aberration or if it really is a 550-line camera horizontally and 540 vertically. Before you gnash teeth and rend clothing in despair at the camera's poor showing, wait until we have more data points!
1080-line camera limiting resolutions
|Camera||Chip details||H limit, TVl/ph||V limit, TV lines|
|HDW-F900/3 CineAlta||2/3" 1920x1080p||1000+||1000+|
|Canon XL H1||1/3" 1440x1080i||800-||700+ (540)|
|Sony HVR-Z1||1/3" 960x1080i||550||700+ (540)|
|Panasonic AG-HVX200||1/3" 960x540p||550?||540?|
720-line camera limiting resolutions
|Camera||Chip details||H limit, TVl/ph||V limit, TV lines|
|AJ-HDC27F Varicam||2/3" 1280x720p||700+||700+|
|JVC GY- HD100U||1/3" 1280x720p||700||700|
|Panasonic AG-HVX200||1/3" 960x540p||550?||540?|
I urge you to look at the source materials rather than taking my numbers as gospel; not everyone agrees with my methodology for determining resolution. I've made two DV clips (assuming that everyone has a computer that can play back DV using QuickTime Player or other media programs) that contain clips from all the 1080 cams and all the 720 cams side by side. I've grabbed resolution trumpets from the uncompressed 10-bit clips and reproduced them 1:1 in the DV clips (field order "None", so there's no interlace-induced blurring; the High Quality bit is set for clean desktop playback); the detail in these DV clips is equal to the originals. (Note that the Z1 is shown half the time at detail level 5, and half the time at detail level 10; in retrospect we should probably have run it at detail level 6 or 7.) Put them in loop mode (control-L on PC, Command-L on Mac) and let them play.
In any case it's the overall character of image rendering, not a single dry number, which determines the quality of a camera's image and its suitability for any given task.
Jay next set up a scene with 12 stops of dynamic range, from a black-draped box in the shadows to a white shipping case blasted with light. The gray card, the ChromaDuMonde, and actress Kacy Bult's skin tones were all illuminated to the same level, and the cameras were set to expose these subjects at 50%. The shadowed depths of the studio and Kacy's dark dress gave us more low-key information while shiny objects gave us specular highlights.
We were running out of time by this point; we were supposed to be out of the studio by 5pm, and here it was 4:45 and we were still setting up the test. As it was, we were there until after 7pm, but note-taking fell by the wayside as we switched between cameras, argued about settings, and pressed our eyeballs to the monitors in the rush to get all the tests done and all the footage captured.
We tried to optimize each camera for the best subjective image quality and the broadest latitude. I'll mention specific departures from the default setup (normal gamma, black stretch, knee, matrix, no gain boost, etc.) as I recall them.
Both Panasonics, as I recall, were set to Cine-Like_D gamma. On the HVX200 the knee is not separately settable in this mode, and I don't know what was set on the Varicam.
The Z1 was left at normal gamma; both its cinegamma settings depressed midtones too much for our tastes, although I turned on black stretch to reveal more shadow detail.
The JVC was kept in normal gamma, while its knee was set to manual at 80% for the strongest highlight compression (I don't recall if we used any black stretch on the JVC).
Here's where we missed an opportunity: the Canon XL H1 has a variety of menu settings we failed to exploit. Its knee was set to "middle" whereas "low" might have bought us a smidgen more highlight detail. The Canon also has two noise reduction options (NR1, which appears to perform recursive or "3D" noise reduction, and NR2, which is said to be like "applying the skin detail function over the entire picture") as well as coring (which reduces enhancement noise in the shadows), all of which we neglected to engage, so the Canon's image was a tad noisier than it otherwise might have been. The Canon was used with normal gamma and no black stretch.
As far as I know we used the normal color matrices on the Canon, JVC, and Z1. I'm not sure what was used on either Panasonic or on the CineAlta, nor do I know how any parameters were set on the CineAlta; I didn't have time to look at it in detail.
In general we tried to stay with the "equalized" sharpness settings across all the cameras we had agreed on for the resolution tests, and we tried to set the gray card at 50% exposure, but aside from that the cameras were individually optimized for this one scene. Even so, the Varicam wound up darker in the midtones. Take any qualitative comparisons with a grain of salt; what we saw in Burbank may not correspond with what you see on your set.
All the cameras did a very good job of rendering the scene. Everyone present was surprised at how little difference there was between the cameras when it came to putting a clean, pleasing picture on the screen. There were differences in resolution, colorimetry, noise, and dynamic range, but these differences were minor compared to the overall high quality and similarity of the images.
The CineAlta and Varicam took top honors overall; they had better resolution, lower noise, wider latitude, and more subtle highlight handling than all the 1/3" contenders--but their lead over the "toy cameras" was much less pronounced than anyone expected. The Varicam in particular yielded gorgeous images, and those who expressed a preference agreed that if they could take home one camera from the test, based on that one scene, it would be the Varicam.
Amongst the 1/3" cameras, all aside from the Sony Z1 had noticeably noisier images than the 2/3" cameras. The Z1 looked almost as clean as the 2/3" cameras. The Z1 lagged in sharpness and aliasing, tying (as I saw it) with the HVX200, and its highlight handling was a bit less accomplished than the others.
The Canon XL H1 was the resolution champ amongst the 1/3" cameras, with a crisper, visibly more detailed image than its compatriots. To my eye it showed slightly more noise than the HVX200, with the noise being a fine-grained luma noise compared to the HVX200's slightly softer, more chroma-oriented noise. I preferred the Canon's noise signature as being less video-like, but Barry preferred the HVX's noise for exactly the same reason--you'll want to judge for yourself. In any case, there was potential to reduce the visible noise in the Canon's image that we didn't explore; we'll have to do that in a later test.
The Canon clipped highlights a bit more harshly than the HVX did; it was comparable to the Sony as best I remember. Again, had we set the Canon's knee to "low", we might have eked out a small increment in usable highlight detail; how much so I can't say. Something else to test on another day…
The HVX200 was not the resolution winner, but it did a very good job on skin tones, handling highlights cleanly. It showed less noise than the Canon by a slight amount, and the noise was more colorful: more film-like according to Barry, more like color-under analog to my eye. This is one of those subjective things where taste is as important as numbers.
In post-test comparisons I found the HVX's skin tones to be slightly on the greenish side. I used my Channel Balance filter (http://www.adamwilt.com/downloads/) in Final Cut Pro to reduce Cb by 10-15%, and the resulting images were much more pleasing and a better match for the other cameras.
The real surprise of the test was the JVC HD100. It had the noisiest picture of the bunch, but in every other aspect we were all stunned at how good its images were. While the Canon was slightly crisper, the JVC rendered a more naturalistic, more alias-free image while yielding only a little ground in terms of raw resolution (and remember that the other cameras are 1080-line cams while the JVC is 720p-native). Its image detail clearly topped that of the Z1 and HVX200. The JVC also seemed to handle highlights more gently than the other cameras, and may (if I recall correctly, but this is disputable) have held onto detail for half a stop more overexposure than the other cameras did.
And the skin tones! After shooting the normally exposed images, we increased each camera's exposure by two stops to drive Kacy's skin up into the knee-affected area. While I don't recall specifics for the other cameras, I do recall that everyone was agog at how well the JVC handled skin tones. There was little of the hue shift that characterizes overexposed skin on so many cameras; the HD100 "looked more like film" in this test than the other 1/3" cameras did.
If a winner were to be picked on how well a camera did on the tests as compared to its expected showing, the HD100 would be that winner.
No, I don't have numbers for this test. All cameras shot the same scene from different angles, so it's difficult to adequately judge whether a visible highlight detail on one camera is blown out on another because of the angle of reflection or because of a difference in highlight handling. Also, the captures from the +2 stops test don't show consistency in midtone exposure; I know there were some issues involved in getting some of the cameras to match, and different gamma curves were used, so after the fact it's impossible for me to say what exactly is going on. I've simply provided frame grabs from the normal exposure test, and you can make your own judgments.
Note that FCP applies a gamma correction to exported stills to compensate for the default Mac screen gamma of 1.8 (PCs and video use 2.2). If you're viewing the stills on a PC or on a Mac calibrated to 2.2 gamma, try adding a gamma correction around 1.22 to see the tonal balance more the way we saw it on-set (in Photoshop's Levels control, slide the middle slider leftwards until its readout is 1.22).
All the 1/3" cameras clustered together more tightly than we expected. Each camera excelled at some aspect of image rendering, but all of them were more alike than different; none stood out as being clearly superior all around.
Furthermore, they all came a lot closer to the 2/3" cameras than we thought they would: while we could clearly see that the big cameras made superior images, the contrast between the 1/3" and 2/3" cameras was nowhere near what we expected to see.
DP and CineAlta operator Art Adams once characterized my Z1 pix as "half HD" for their horizontal softness, and that perception holds as I look at the 10-bit uncompressed clips: The 1/3" 1080i camera pictures are only about half as crisp as the CineAlta's. In 720p, the JVC comes close to the Varicam in raw detail, although its noise is quite a bit higher.
When you consider that none of these 1/3" cameras comes anywhere close to half the price of their 2/3" brethren, you'll see that "half HD" isn't bad for the money.
We came away convinced that any of the cameras would do a creditable job in the hands of a skilled user, and that the choice of camera should be made more on features and ergonomics than on image quality. This is not to say that people didn't pick favorites; people did. It's just that no one, not even the most partisan among us, would have claimed that any one of the cameras was unacceptable for doing serious work.
We also understood how sketchy and rudimentary our explorations were. We didn't look at motion rendering, or how the camera's different codecs and recording formats affected image quality. Do the cameras behave differently in daylight-lit exteriors than in tungsten-balanced interiors? How does each one handle handheld? What kind of work is each camera most suited for?
We didn't have time to answer these questions, so they remain subjects for part two...
on this page copyright
© 2006-2007 by Adam J. Wilt.
You are granted a nonexclusive right to reprint, link to, or frame this material for educational purposes,
as long as all authorship, ownership and copyright information is preserved and a link to this site is
retained. Copying this content to another website (instead of linking it) is expressly forbidden.
last updated 2007.01.07