How Canon Measured 15 Stops of Dynamic Range on the C300 Mark II

canon-c300-mark-ii-15-stops

In September we published our lab results of the Canon C300 Mark II dynamic range. Most interesting about this test was the fact that according to our own parameters the dynamic range was closer to 12 stops, while Canon markets 15-stops of dynamic range for their current flagship cinema camera. This caused some confusion among customers and earlier this month Canon replied to our article and offered an insight into how they measured dynamic range.

Dynamic Range is one aspect of a camera’s performance. Certainly there is more to a camera as a whole than just this one figure. Nevertheless, for many cinematographers a high dynamic range is an important attribute of a camera sensor.

About measuring dynamic range in general it is important to mention that at this moment there is no established “standard” to conduct such a measurement. This is why companies come up with their own standards and are able to market their cameras with very high dynamic range values, which of course is confusing to the customer. Here’s where we decided to measure all cameras with an equal and realistic standard in order to be able to compare them to each other.

First let’s look at how we measure dynamic range at cinema5D and later we will describe Canon’s testing methodology as presented to us.

How cinema5D measures dynamic range

As some of you know accurately testing dynamic range is a somewhat delicate task as each sensor has a very unique way it captures light. In order to assess how many of the stops of data the sensor outputs are “usable” for a normal production process, every tester has to come up with testing parameters they can apply to the cameras tested in order to compare them to each other. At cinema5D we strived to setup these parameters in a way, so that we get a realistic evaluation of each camera’s performance.

We test dynamic range of a camera at its optimal settings and film a DSC labs XYLA-21 transmissive test chart with a Zeiss 50mm CP2 T/2.1 makro lens. After that we use IMATEST evaluation software to measure the shots objectively. The aforementioned parameters are applied by using a fixed signal-to-noise threshold for all evaluations.

In our tests we rated the Canon C300 mark II at 12.3 stops, about 2 stops weaker than the Arri ALEXA that is rated by us and by the manufacturer at about 14 stops. We also pointed out the horizontal strip that appears on overexposed portions of images recorded with the Canon C300 mark II.

alexa-vs-c300-ii-exaggerated

For the image above the gamma has been raised to an extreme manually and is only for you to observe how the two mentioned cameras record light differently and where the noise levels surpass the information recorded. This version of the shot is not used by us to determine the dynamic range.

At the point where the noise is too strong in the dark areas of your image, it becomes hard to use that data for your film. When evaluating dynamic range every tester has to determine such a point, where the noise is too strong to effectively use the underlying data.

How Canon measured dynamic range

We’ve had an insightful conversation via e-mail with Senior Fellow of the Professional Engineering & Solutions Division at Canon, Larry Thorpe, who has also written Canon’s whitepapers on the Canon C300 mark II, which is an in-depth look at the technology behind the new camera.

In a document that Canon officially sent out to the marketplace on the topic of dynamic range Canon writes:

“Canon recognizes that some of our customers are commenting about the image quality results after shooting the Canon C300 Mark II with the new Canon Log 2 Gamma setting.

Canon Log 2 Gamma is designed to provide as much usable data and dynamic range as possible for professionals to make use of in post-production. Because of this, it is possible under certain shooting environments that there can be the appearance of increased noise in low-light areas and/or luminance banding. These two artifacts are intrinsic to the design of the CMOS sensor. To make a fair assessment of Canon Log 2 Gamma the image should not be judged by viewing the log output alone, but rather the final graded image. (…)”

In order to reduce the noise mentioned above Canon further recommends users should:

  • “Set camera ISO to 400.”
  • “Set the cameras internal noise reduction filter to 2.”
  • “Utilize the original Canon Log Gamma (To minimize Luminance Banding).”

Canon later says that the base sensitivity for Canon Log is ISO 400, while the base sensitivity for Canon Log 2 is ISO 800.
About their testing methodology Canon goes on to say:

“Measurement of Dynamic Range can be both subjective and objective and Canon realizes there can be a variance in this determination based on procedures used and the tolerances of the end user. Because of this we are presenting our methodology of testing and determination of dynamic range visually.”

For their tests Canon used the Canon EF 50mm F1.8 lens at F/5.6 with a shutter of 1/68th and the aforementioned chart. It turns out that Canon used the same DSC labs XYLA-21 transmissive chart that we at cinema5D are using for our evaluations. In theory the testing methods seem to be very similar to ours, with the only difference that we used the Zeiss 50mm CP2 T/2.1 makro lens and a PAL framerate with a 1/50th shutter and F/4.0,5.

The obvious difference though, is that Canon evaluates the dynamic range with a Waveform Monitor (Leader L5770 Multi monitor) and a subjective readout while cinema5D is using the IMATEST to evaluate the results objectively.

CanonLog2-c300-ii

Canon sent us a screenshot of the results on their waveform monitor. Each step represents a step on the chart. The first step represents overexposure, the second step  represents the first stop of dynamic range, which Canon also explains in their document.

Larry Thorpe further commented on the signal-to-noise ratio they determined as the threshold we described as “usable dynamic range”:

signal-to-noise-canon-c300-mark-ii“(…) on a Millivolt video amplitude scale (with Reference White at the traditional 700 mV) the linear signal output (…) – for those last two stops – are less than one millivolt in amplitude. Even with the impressive 67dB Luma signal to noise specification this means those two steps ARE noisy. Indeed, the definition of Dynamic Range is when the final lower step has a signal to noise of approximately 0 dB.”

In summary it looks like the testing methods of Canon and cinema5D are very similar if not mostly identical, with the only difference of where to apply the threshold in order to determine how far into the noise the dynamic range is counted. cinema5D’s evaluation software did not count the 13th stop, sitting inside the noise floor, as another stop of dynamic range, while Canon did, as they did for stop 14 and 15.

Conclusion

First of all we are happy that Canon decided to put their testing methods on determining the 15-stops of dynamic range for the Canon C300 mark II up for discussion. This shows us that Canon takes user feedback and concerns seriously and it supports the notion that the Canon C300 mark II is an important camera to them. Furthermore we felt a lot of detail and thought went into the scientific evaluations they presented so there is no question they understand the science behind sensor design and are willing to share details about their technology.

Finally, we want to leave the conclusion about the testing methods presented up to you. Is it simply a different way to evaluate dynamic range? Was Canon’s evaluation parameter of 0db as a threshold too loose or are cinema5D’s testing parameters maybe too strict?

My personal note: These tests are useful to us cameramen when there is a reference we can compare the results to. This is the mission of cinema5D, to give users an impartial comparison between cameras, free from any marketing agenda, so they can make an informed decision based on the performance needs of their shoot.

Leave a Comment

You are not subscribed to this post. Follow new comments

Login to comment

Philippe Claes Reply
Philippe Claes November 30, 2015

We show to our customers an final image and not an Oscilloscope signal, so for me this camera has 13 stop of real world dynamic range.

Andrea Seno Reply
Andrea Seno November 30, 2015

Nice job guys

Reply
Marcos Avlonitis November 30, 2015

Highly interesting and informative – thank you very much!

Reply
Joachim Hedén November 30, 2015

In the wake of the debate in the comments related to the original C5D blog post, I did some tests myself to see with my own eyes what’s what.

You can draw your own conclusions, but I think it’s fair to say that my test reveals the C300 mk2 to be substantially better than the original C100/300, and on par, if not better than Sony F5.

Bgd Videography Reply
Bgd Videography November 30, 2015

Thanks for that, it’s nice to know that Canon replied to these concerns. I have the camera now, and I’m happy with the DR. Not so happy with the noise CLOG2 produces when there’s not a great deal of light though. I suppose we’ll have to rely on CLOG in that kind of situation.

Derek Brandon Reply
Derek Brandon November 30, 2015

1 stop per thousand dollars.

 Anatoly DP Reply
Anatoly DP December 2, 2015

so the real world usable price should not be more then 12300 usd?!?!

Jeff Regan Reply
Jeff Regan November 30, 2015

I have to agree with C5D, those last stops are buried in the mud. I’m curious as to why the Mark II seems to be slower, with Thorpe quoting a base.ISO of 400 for C-Log, when the Mark I was 850? It’s supposed to have a much higher signal to noise ratio, but it seems like the preamps were gained up to make a base ISO of 800 for C-Log 2.

 Claudio Almeida Reply
Claudio Almeida November 30, 2015

Well done. Thanks for sharing this information!

Mux Neumeier Reply
Mux Neumeier November 30, 2015

Haven’t read the article yet but did they use another camera?

Reply
Joachim Hedén November 30, 2015

Hmm… strange. The link seems to have vanished from my post. Did admin remove? Tech error?

Posting again:

In the wake of the debate in the comments related to the original C5D blog post, I did some tests myself to see with my own eyes what’s what.

You can draw your own conclusions, but I think it’s fair to say that my test reveals the C300 mk2 to be substantially better than the original C100/300, and on par, if not better than Sony F5.

The test is here:
https://joachimhedenworkblog.wordpress.com/2015/10/22/some-dynamic-range-tests/

Reply
Joachim Hedén November 30, 2015

Oh, and by the way, real world performance is quite lovely. Below is a Youtube link to a Subaru spot I shot with the C300 mk2 in September. It’s Youtube, so you can’t really tell anything about the true image quality, but it should be obvious that there were some quite challenging shots from a DR-perspective and the camera did really well. The first and last shot have backlit clouds in the sky that is held nicely and still had plenty of detail on the shadow side. The shots of the blue car driving through the forrest is exposed for when the car is out in the open, so when it is running in the shade (the shots used), it is severely underexposed, but with a little bit of NR in Resolve it came up quite nicely from the dark. In the shot of the hunter with his dog, the sun is obviously clipping, but it is clipping ”nicely”.
My 2 cents
https://www.youtube.com/watch?v=jZ2wUPSbuXA

Sebastian Wöber Reply
Sebastian Wöber December 1, 2015

Hi Joachim,
Nice film. But hard to judge dynamic range or clipping in comparison to a different camera you didn’t shoot with on that day. That’s what the lab is for.
Yes the C300 II is much better than the C300, I agree.

Reply
Arash Ayrom November 30, 2015

It would be great to run, say, a Panasonic GH4 and a Sony FS7 through Canon’s methodology and then compare the results to the C300 MkII’s claim of 15 stops.

Reply
Leonardo Spagnol December 1, 2015

Yes.

 Fergus Cahill Reply
Fergus Cahill November 30, 2015

So with canons school of thought the Arri Alexa has 18 stops of range? :|

Reply
Adrian Bacon November 30, 2015

>> Is it simply a different way to evaluate dynamic range?

Yes.

>> Was Canon’s evaluation parameter of 0db as a threshold too loose or are cinema5D’s testing parameters maybe too strict?

Probably a little bit of both. It’s good to have a standardized test and report the range according to the standardized test, but perhaps cinema5D can also recognize that there is more range there that can be utilized if noise reduction is employed and how much more range is available depends on how much noise one is willing to deal with in post.

Personally for me, if you can see the steps on a scope, then there is detail there that can be put to use, though it may not be nearly as clean as other cameras. It’s not realistic to evaluate cameras without taking post processing into account because we all deal with lighting that we can’t control in one form or another and therefore regularly do things like noise reduction in post.

Reply
Crimson Son December 1, 2015

Strict or loose is likely irrelevant as long as your have a baseline or other tests results one can compare.

I prefer or at least any dynamic range test should be accompanied by a screenshot of a scope and image read out. This way informed readers do not need to depend on proprietary software and individual biases (we are all bias). We all have different thresholds for what we consider good, not good or good enough.

And part of dynamic range is making judgement based on personal preference.

Reply
Adrian Bacon December 1, 2015

Very true.

 Andrés Castañeda Reply
Andrés Castañeda December 1, 2015

So why don’t you make that measurement with the settings canon provided (ISO 400, noise reduction), so we can see if that really affects the final quality of the image

 Will Lodge Reply
Will Lodge December 1, 2015

Exactly what I was thinking…

Reply
Adrian Bacon December 1, 2015

That would certainly be an interesting exercise. It would be even more interesting if we saw the output from the various combinations of settings as part of the camera evaluation. These cameras all have different settings that apply to different types of light you’ll be dealing with, so it would be great to see how the image is affected by those settings.

Sebastian Wöber Reply
Sebastian Wöber December 1, 2015

We test them all and choose the ones where the camera performs best. In this case it was Canon Log 2.
The suggestion by Canon in the article is for users who want less noise in Canon Log 1.
And nobody makes a dynamic range tests with noise reduction enabled where it can be avoided. According to Canon they already have hardware noise reduction built into the sensor by the way, so I think we’re already seeing a “good” result.

Reply
Adrian Bacon December 1, 2015

I don’t doubt that we’ll see anything better than what you’ve already published, though, it would still be interesting to see how the picture is affected by those other settings since we don’t all have a Canon C300MkII. It’d be useful to those of us who may or may not be considering acquiring such a camera.

Personally, I would have preferred that Canon be getting a lot more aggressive in getting clean dynamic range in low light, however, I can also see why they haven’t been given the market that this camera is aimed at, being commercial productions where you’re largely on a set and can light it, in which case, you’ll almost never light more than 12 stops into it.

 Sean Mclennan Reply
Sean Mclennan December 1, 2015

So in other words…Camera manufacturers test their DR like auto manufacturers test their fuel mileage.

 Alex Chinnici Reply
Alex Chinnici December 1, 2015

Thank you for this, what a great article! Do you plan on measuring the Red Dragon? I know that its tough with the OLPFs and many different suggested ISOs. But I think its worth it simply because Red says 16 stops and anyone who has shot with it would agree that is simply not the case.

Sebastian Wöber Reply
Sebastian Wöber December 1, 2015

Actually we already scheduled renting it just last week and should be able to test it soon. I think it might actually perform really well. Let’s see.

 James Pittman Reply
James Pittman December 1, 2015

This all seems ridiculous to me. Too much fuss in being made about very small differences in how a camera performs in bad lighting conditions. This has very little to do with producing good content.

 Anatoly DP Reply
Anatoly DP December 2, 2015

It would be great to see same test for original C300! So we all could see if New C300II is trully better then the old one! in any case it would be great to know the reall DR number of original c300.

Reply
Joachim Hedén December 3, 2015

There is a direct comparison between the C100 (Same sensor as C300, different codec) and the C300MK2 here:

https://joachimhedenworkblog.wordpress.com/2015/10/22/some-dynamic-range-tests/

Reply
Phil P December 5, 2015

I grow tired of this conversation about dynamic range…it’s not hard to appreciate the price difference between the Alexa and the C300. What I will say after shooting with this camera today is that it was a real pleasure to use. The Dual Pixel Autofocus feature is a major game changer. Imagine shooting an interview and knowing that your focus is dead on without touching the camera. Incredible!

 Anatoly DP Reply
Anatoly DP December 13, 2015

yes i saw it, and thanks for your time making it, but i dont understand that, there is no description, or conclusions, i am not that tech smart….to understand these images…

 Andy Krucsai Reply
Andy Krucsai December 12, 2015

Breaking! Canon is NOT going to repeat the same sales success with Canon C300 Mark II. as they did with the original C300. The price point (after 2 weeks on the market) already dropped by € 1400 from € 14600 to € 13199 without VAT on two German sites BPM & AF Marcotec. Obviously the sales are super slow, otherwise they would keep the original price offer. My estimate is € 12000 without VAT mid 2016 or even earlier.

 Ruy De Faria Neto Reply
Ruy De Faria Neto January 13, 2016

Then, I will leave a simply question. Using Canon methodology what would be the DR of Arri Alexa? 17 stop?

 Olivier Sarbil Reply
Olivier Sarbil January 31, 2016

I have been filming with the Canon C300 for the past few years. I have just picked up a brand new C300 MkII and I am getting a terrible high level of noise… at 800 ISO, the picture is just unusable. CPS Europe AGREED that my footage is getting far too much noise and actually their technician was pretty embarrassed… but in the end, he just blamed it on using Wide DR Gamma. They asked me to do more tests which I have done… and I am still getting the most awful noise. I did more tests shooting C-log, BT709 Preset, with CP setting… the footage is just bad… Let’s speak frankly, the footage is just terribly crap. So far it’s 18k in the bin and still no help from Canon… Not to mention that the picture is also soft… If anyone had similar issues?

 Matt Garland Reply
Matt Garland February 4, 2016

Hi Oliver,
I also just made the purchase and noticed exactly what you are talking about. To me it seems the Canon Log 2 is incredibly noisy. I really thought the CLOG2 would be a better improvement over CLOG which I think is great, even if the improvement was minor. I loved being able to use the higher ISOs in the original C300 without worrying about noise, 6400 was my limit. I don’t think I can use CLOG2 past 800 ever and it’s really only acceptable in scenes that are shot in bright sunlight. Even once you put a LUT on the image it doesn’t mask the noise much. Maybe CLOG2 is really a last ditch effort to retain as much shadow information as possible for an experienced colorist. I think I’m forced to go back to CLOG and use ISO 400. I really hope they release some kind of firmware update in the near future that addresses these issues. I really think the noisiness of the CLOG2 is just unusable. Lets hope Canon will release a CLOG3 with improved signal to noise.

 Andy Krucsai Reply
Andy Krucsai March 11, 2016

Cinematographer Geoff Boyle confirms, that it was a very good decision for us to buy the Canon C300 Mark II. to be our 4 K cinema camera smile emoticon
https://vimeo.com/158017639

Reply
AlexAdrian Adrian March 20, 2016

GReat job Sebastian. Thank you and please do a test on the Mini Ursa 4.6k. I’m curious to see how much usable dynamic range you can get (without using RAW).

I have a feeling it is about 13 stops.