This week I had the pleasure of supervising a camera test with photographer and filmmaker Alfredo Chiarappa (Instagram!)and his new Sony FX9. As an accomplished visual artist with a scrutinizing eye for detail, Alfredo wanted to explore the exposure options and limitations with this new system in preparation for his new documentary feature project. One of our favorite services here at Magic Lantern is to provide technical consultation with cinematographers in order to determine the best color science, exposure strategy, and workflow for their camera system and project. Alfredo’s upcoming documentary project will be filmed in Cuba under challenging lighting situations. He’ll be filming his subjects using only the practical and natural lighting in the moment to include phone screens and low-wattage bulbs under the darkness of night. Under these sort of technical limitations, Alfredo wanted to prioritize a strong signal to noise ratio along with good color science in order to retain as much depth in the negative as possible for a robust color grade.

Introduction to the Sony FX9

Over-sampling for True 4k

I’ve been shooting and color grading Sony cameras for the last fifteen years and I can honestly say that the FX9 is a game changer for the company. By oversampling a 4k image using a 6k sensor, Sony is achieving multiple things at once. Due to the nature of a Bayer sensor, most 4k cameras on the market aren’t true 4k. Each RGB pixel on a sensor is spaced in such a way that the term “pixel resolution” is misleading. Below are two images that illustrate my point:

Example of a Bayer sensor (Bayer filter array) above. (courtesy of Wikipedia)

The above image is broken up into separate RGB pixels (courtesy of RED)

While a Bayer filter array has its technical advantages, one can see that each pixel does not, in fact, represent a true replication of color corresponding to the scene which is meant to be recorded. The image is spread out over RGB pixels diluting the actual resolution. Thus, one strategy to overcoming this loss in resolution is to “over-sample” an image by placing more RGB pixels on the sensor prior to the “de-bayer” process. By over-sampling the FX9 using a 6k sensor, a true 4k de-bayered image can be recorded.

Color Resolution and Accuracy

Not only does the pixel resolution increase (and thus apparent resolution), but the color resolution and signal to noise ratio improves as well. More RGB photo sites mean that there is more color detail recorded in the image per down-sampled pixel. Sony also decided to utilize higher quality color filters on each of the RGB photo sites thus increasing the color accuracy even further. When manufacturers use lower quality material to construct a red green or blue lens for a photo site, the spectral accuracy suffers. Camera engineers then have to play with the signals coming out of the sensor and into the digital to analog converter in order to try and approximate a true representation of a given color. Higher quality color filters give a camera sensor the ability to record colors within a more narrow band of precision tolerance. The reward is better skin tones and color in the dark and super bright areas! It’s all well and good to say that you’re recording in a giant color space like S-Gamut, but if the color that is coming out of a digital sensor and written into a color space is inaccurate, you will still have weird looking skin tones. There are multiple steps in the process of recording light into a digital camera. Camera manufacturers are constantly trying to improve one of these steps often at the expense of another. The Sony FX9 has somehow magically ticked a few of my favorite boxes!

Signal to Noise Ratio (SNR)

More photo sites on an over-sampled sensor also means that the spatial characteristics of sensor noise are smaller and averaged out during the down-sampling process. When the noise floor of a digital image is reduced, more detail can be quantified and recorded therefore the dynamic range is increased. In most professional digital cinema cameras that aren’t optimally exposed the shadows tend to be lumpy, chaotic, and an absolute mess. By increasing the fidelity of the lowest 20% of the waveform, we’re actually given a bit more freedom to underexpose. This flexibility gives us a more dense digital negative for post!

Camera Test

My natural inclination when working with a digital negative is to dismiss the shadows with trepidation. This lack of faith was forged over the last sixteen years out of consistent disappointment with the messy nature of digital compression and its chromatic chaos in the lower end of the waveform. We often have to crush and/or clean the shadows in order to hide the mess, thus losing depth in the darker areas of an image. I’ve always felt trapped by the noise floor because I can’t venture too far from the base without losing integrity in the image. I love grading with low noise negs because the shadow roll-off is smooth and the image effortlessly sinks into black areas with nuance like silk. No color cast and no macro blocking chunks simmering from the abyss.

Super 35mm Windowed

We also shot the same series of test images in the Super 35mm windowed mode in order to analyze the signal to noise ratio and color reproduction. These test images can be found along with the rest via the link at the bottom of this article. As expected, the S35 images have a significant increase in apparent noise. This is because the camera is no longer utilizing the full 6k down-sample. This “windowed” mode means that the camera is only recording the pixels from a centralized rectangle and scanned at standard 4k thus magnifying the already inherent noise.

Example of a “windowed” sensor offering full frame vs the APS-C sized Super 35mm crop.

Exposure Methodology

The first thing I did once I transformed and calibrated each FX9 test clip was to magnify and inspect the nature of the shadows. To my delight, the noise was mostly monochromatic and therefore kept the lower quarter of the waveform without an unnatural color cast. While both native exposure index modes (800 and 4000 EI) exposed to their corresponding 18% reflectance recommendation by Sony were remarkably clean, I found that over exposing by just 1/3 (N+ 1/3) of a stop significantly reduced the noise floor even further making for a lovely and rich negative without having to sacrifice much of the dynamic range. This, as many cinematographers and Log geeks will tell you, is a serious leap forward since this now gives us more headroom in the highlights and a bit more bravado in the dark. By maintaining proximity to the engineering team’s methodology for optimal color science, exposing for 18% reflectance helps me maintain consistency in proper color reproduction throughout a project. To over expose or under expose a camera is to change the characteristics of a sensor’s interpretation of color, so being able to rely on clean shadows actually encourages consistency in exposure methodology. To put it bluntly, our fear of the dark has led us to form sub-optimal exposure habits! Off-axis exposure means that all of the strange and annoying color characteristics that Sony’s cameras exhibit (especially in the skin) are amplified.

Recording Codec

I also chose to record using the internal XAVC-I codec as this is the format that most of our documentary collaborators will use in the field. Capturing the linear RAW sensor data directly from the sensor to an external recorder would be lovely, but not at all a realistic scenario for doc filmmakers. This camera test is for my scrappy documentary friends in the field!

*Please note that these tests were not executed using true laboratory equipment and would not stand up to a full diagnostic examination under the scrutiny of an engineer. While we do try to maintain good scientific practice in our approach to exposure and color science, these tests are more qualitative in nature and are designed to give the documentary community a good idea of the benefits of using an over-sampled camera with a high signal to noise ratio with the intent to color grade in post.

We shot a series of test clips under two different lighting scenarios:

Ground Truth

Our first test was under controlled full-spectrum lighting to provide a ground truth. Both our camera and lamps were set to a color temperature of 5500 Kelvin. We set our camera to capture within the S-Gamut3.cine because this is the color space that most of our collaborators will likely capture in.

Sony’s recommended exposure value for 18% reflectance for S-Log3

While on set with the camera, I changed the variable ND filter to tweak the exposure slightly in order to place our 18% reflectance gray card to about 420 on the IRE (snicker) as per Sony’s recommendations for S-Log3 outlined in the S-Log 3/S-Gamut3.cine Technical Summary. Once we got the footage into Davinci Resolve, I used the False Color 3 OFX tool by TimeinPixels in order to verify exposure again using the 18% reflectance card. Once you overlay the REC-709 transform onto the S-Log3 footage, your exposure values will shift and you’ll need to make some slight adjustments. I also utilized the false color tool to correct the over and under-exposure. This is why I LOVE gray cards! Such a simple tool to keep it real.

False color representation of a REC-709 image exposed to 18% reflectance as indicated by the color green.

Stress Test

The purpose of the second set of tests was to explore the specific spatial and temporal qualities of the internally recorded media when stressed outside of optimal exposure. We overexposed and underexposed each of the two native EI options by two stops in 1/3 stop increments. From N-2 to N+2 (N = exposure set for 18% reflectance). We also underexposed and overexposed each native exposure index by N-3 and N+3 respectively in order to see how the sensor would behave in an extreme situation.

Color Science

After importing the footage into our color system in our lab, I immediately noticed the quality of the skin tones and color reproduction. For our ground truth, I used the Color Space Transform OFX tool in Davinci Resolve in order to convert the image from S-Gamut3.cine to REC-709. If you’ve ever seen S-Gamut as an option in a Sony camera menu, there are multiple versions. S-Gamut3 is the latest iteration. As Sony continues to develop their digital cinema cameras, they are constantly re-mapping their color science to better reflect human perception of color accurately. As you can see from the illustration below, S-Gamut is a massive color space. S-Gamut3.cine was developed as a smaller and more manageable color space specifically for filmmakers who want to move quickly into a grade. Many LUTs, plugins, and fx tools have been specifically designed for S-Gamut3.cine, so most filmmakers tend to shoot with this in mind. S-Gamut is more like a digital negative and as a container, it’s able to hold much more color information. In fact, Sony recommends that this large color space is best used for long term cold storage of “original camera negatives”. Unfortunately, most cameras do not have a sensor that is able to capture the full range of this color space to begin with, so again, S-Gamut3.cine has an advantage. Think of it as a S-Gamut3 lite. Other Sony cameras like the FS7, FS5, and even the F5 do not have the technical ability to even capture S-Gamut3.cine accurately, so much work has to be done in the color grade to tweak and bend the colors to look real.

S-Gamut3.cine is significantly larger than DCI-P3

As I outlined in our recent article Skin Tones – The Guiding Star to Good Color, one tried and true method of assessing the color quality of an image is by looking at skin. Mother nature gave us a built-in color checker tool! Since we’re doing a qualitative examination of the Sony FX9’s images, we decided to focus our test images on the charming face of my friend and fellow documentary filmmaker Eric Jenkins-Sahlin. The color reproduction of this camera is wonderful! I can’t tell you how much of a white whale good skin tones have been for colorists working with Sony footage over the last decade. Right out of the box, the FX9 gave me good skin tones with lovely pink and orange hue.

Test Results

This camera gives a lot of bang for your buck. It’s a mini-Venice for $10k. Since this camera is able to record 15+ stops of dynamic range, it is utilizing most of the tonal curve of S-Log3 which means that it’s getting very close to matching the gamma curve of Cineon Log, the gold standard! The highlight roll-off is nice and smooth with little color shift even in extreme exposure issues. The detail and clarity of color with the over-sampled 4k is clean and natural. The shadows hold depth and even when pushed, the chromatic noise is a relatively uniform chromatic noise which prevents awful color cast. In short, I can’t wait to start color grading films with these negs. We’ve had S-Log, S-Gamut, and 4k for many years now, but this is the first time I’ve seen a cinema camera from Sony at this price point that can compete with the F65 and Venice.