Are DSLR darks a waste of time?

[MarkS]

Here's another technical investigation I got dragged into, following the discovery of the various kinds of spatial filtering (a.k.a. hot pixel suppression, HPS) on Nikon and Sony cameras.

There is a lot of anecdotal evidence (typically from Nikon users) that master dark subtraction makes very little difference to the final image.  This has always surprised me because I have always found darks to be useful on my Canon and Sony cameras.  It is an important part of image calibration.

I now have the answer.  Depending on which spatial filtering algorithm the Nikon camera uses, dark subtraction actually doesn't remove the thermal fixed pattern noise from the light frames, at least not when the light frames have a certain level of sky glow e.g. back-of-camera histogram 1/4 to 1/3 from the left.  Ironically it is not an issue for the really crude spatial filtering used by Sony cameras.  But it is certainly an issue for many Nikons. The commonly used Nikon D5300, D5500 are certainly affected.

I'm not going to repeat the details here but if you are interested, my explanation of why this occurs is here:
https://www.cloudynights.com/topic/647035-dark-subtraction-and-spatial-filtering/

The results of a practical experiment with my Nikon D5300 are here:
https://www.cloudynights.com/topic/647035-dark-subtraction-and-spatial-filtering/?p=9082246

Here's the comparison image from that experiment:



URL here: http://www.markshelley.co.uk/webdisk/NikonD5300_DarkSubtraction.png

In any case, here's a brief explanation of the above comparison image.

When taking darks, the spatial filtering removes most of the bright pixels caused by the dark current.  But when you take lights, the background sky glow protects those bright pixels from being removed by the spatial filtering.  The lights therefore have bright pixels caused by the dark current that the darks do not have.  So dark subtraction can never remove them!  On the other hand if you switch on long exposure noise reduction (LENR) then the dark subtraction is performed in-camera and this works quite well for removing the thermal fixed pattern noise because the camera subtracts a raw dark from the raw light instead of subtracting a spatially filtered dark from a spatially filtered light.

Mark

[Carole]

Interesting.  I wonder if it works the same with a Canon camera?
So leaving the in camera noise reduction ON will reduce noise but just takes twice as long.

Carole

[MarkS]

Interesting.  I wonder if it works the same with a Canon camera?
So leaving the in camera noise reduction ON will reduce noise but just takes twice as long.

Yes, LENR works the same on Canon cameras - it's very effective.  But master dark subtraction also works well on Canons (assuming sensor temperature matches) because Canon does not "cook" the raw data.

Mark

[RobertM]

I'm sure it took a while to get to the bottom of it but that makes perfect sense.

So is it a generalisation that to use darks with Nikon cameras is a waste of time and instead, potentially, a whole load more time has to be wasted using the in camera 'LENR'.

If Canon could just improve their sensors then they would clean up yet again !

Robert

[MarkS]

I'm sure it took a while to get to the bottom of it but that makes perfect sense.

So is it a generalisation that to use darks with Nikon cameras is a waste of time and instead, potentially, a whole load more time has to be wasted using the in camera 'LENR'.

If Canon could just improve their sensors then they would clean up yet again !

Robert

I don't know enough to be able to generalise because the spatial filtering algorithms differ so greatly from one Nikon model to another.  Certainly darks are a complete waste of time for some models (including the Nikon D5300 that I haven't yet sold) and I feel quite sorry for those folk who diligently build a library of darks at different temperatures because somebody recommended it on a forum!

The best workaround is probably not LENR but to use dithered exposures processed with sigma rejection stacking.  Anecdotally this seems to be the approach that most people take.

I now have doubts about my potential choice of the Nikon Z6 mirrorless camera.  Would the Canon EOS R mirrorless be better?  I certainly won't buy another Sony. Which is worse for final image quality - thermal fixed pattern noise (that can be dithered away) or Canon banding that is really difficult to treat? 

The Canon 200D (that I haven't yet sold) had the worse banding I have ever come across on a Canon.  The exposure latitude test in the DPReview's full review of the EOS R (https://www.dpreview.com/reviews/canon-eos-r/8) shows very obvious banding.  But that's with severe underexposure at ISO 100.  Would we see the same banding on sky glow subtracted data at a higher ISO?  Dunno but it's certainly a warning sign.

Mark