CMOS Blotch Problem

[The Thing]

Some of you may remember my M16 Eagle Nebula posted on Aug 09, 2018
.

It had strange colour blotches on it, very technicolour but definitely not an accurate representation of the target! There has been a discussion about this on Cloudy Nights https://www.cloudynights.com/topic/636301-asi294mc-calibration-%E2%80%93-testing-notes-thoughts-and-opinions/ and it sems to be a 'feature' off BSI (Back Side Illuminated) sensors. I have been battling with a second frame to the IC1805 Heart Nebula I posted recently, it has this problem in spades and I can't see a way round it. The other odd thing is if I stack all my 164 good subs (over 5 hours) I get a worse result than if I only stack the very best 80 subs. This image has been dark subtracted (only) which as per the article does seem to make matters worse.

I think the way CMOS camera subs are collected and processed is going to have to be different to the traditional CCD/DSLR processing as a result. I've spent hours trying to get the subs to stack and process in PixInsight to no avail. My best results are coming from DeepSkyStacker using no calibration frames or any other settings that affect the result (apart from Bilinear debayering) and loading the standard stacked Autosave file into StarTools which actually prefers to deal with uncalibrated stacks.

[The Thing]

Same but re-stacked without the dark. Processing in Star Tools was Linear Bayered not whitebalanced, Crop 75px, Wipe Default Temp AutoDev, screen dump to JPG, same for both pictures. Now there's a different blotch!

[ApophisAstros]

Much better Duncan,
Roger

[Carole]

Very odd Duncan. 

Is there any chance you could upload your stacked file/s (perhaps the best 80 and then separately the 164 subs stacked.  So we can have a "go" at them.  Perhaps you could also upload the best 80 with flats, darks and bias applied too.

I must say - although many people think it is wrong, I get better results often without using bias at all which seems to completely mess up the images, but not every time.

Can you upload to something like Dropbox as a Tiff file (i.e. not the pixinsight file name).

I'll meanwhile have a read of that link.

Carole

[The Thing]

Hi Carole,

That could be useful, they are Fits files from DSS. I'll let you know where they can be downloaded from later, I'll do various combinations.

Duncan

[Carole]

I look forward to having a "go".  I did try to do something with your first image but there were different coloured blotches all over it. 

I came unstuck with my OSC camera and I think it was the debayering.  I was fine all the time I used APT, but then APT stopped working and I tried using Nebulosity, I stacked using the correct debayering for the camera but you should see the results I got with that, I might post it up if I can find it.  In the end I decided it was too much faff and mono was easier so I got rid of it.

Carole

PS:  Found it - you can just about make out the target I was imaging.  This was back in December 2014.
At the time I had cleaned the sensor and thought I had got alcohol behind the sensor, but eventually someone tested it for me who had experience with this camera, and it was fine, so I sold it.  It was Robin of Sharpcap who finally diagnosed that the stacked software had got confused with the debayering, despite me using the recommended format. 

[MarkS]

I now remember doing some analysis on that sensor:
https://www.cloudynights.com/topic/596025-zwo-asi294-mc-pro/?p=8631836

The problem is that the pins on the rear of the sensor mean that the cooling is not in direct contact with the chip.  As a result the sensor can never be cooled evenly and the dark current varies across the sensor.

It's a great sensor for real time astronomy but unfortunately not so suitable for long exposure deep sky work.

If it's any consolation, I never managed to resolve the purple/green swathes problem on my Sony A7S.  I have to resort to sophisticated background subtraction.
https://www.cloudynights.com/topic/576265-sony-a7s-purplegreen-swathes/

Mark

[Carole]

I had a VERY vivid dream about this last night, (no accounting for dreams) and my conclusion in the dream was that darks should take away this colour noise, in the dream I first thought it needed flat darks, but no accounting for dreams lol.

But just to say this must very very disappointing after purchasing what should have been an improvement to DSLR imaging just to find you have acquired a new set of problems.  Some people seem to be having some good success with CMOS imaging, but others having lots of problems.

Is the pattern consistent in all images you do, if so maybe you could make some sort of extraction image, but not quite sure where you would need to apply it as yet.

What do stretched flats and stretched darks look like? 

Carole

[The Thing]

Hi Carole, what a thing to dream about!

I think the answer is not to calibrate but to use software to correct vignetting and noise. The sub's I get are very low noise and I've worked out if the main mirror is at the back end of its movement I get very little vignetting. Dust bunnies have never been a problem for my setups.

The other thing is to abandon the idea if lots of short sub's. More signal per low noise sub is going to swamp the blotches. One thing in the article is to alternate long and short sub's in he shooting plan, I tried this last night with M42 and have stacked 120s with 30s and 10s subs using entropy weighted (for HDR) in DSS and it looks good. Need to get the result into StarTools later after my two hour French lesson.

[Carole]

Did you take a look at the master dark and master flat to see if either of these contain the colour blotches?

Lol, yes my dream consisted of you standing up at an OAS  meeting and doing a presentation on it. 

Any chance of those download links yet?

Carole

[The Thing]

The smarty pants on CloudyNights (not Mark for a change!) has been through all the options. The chips don't produce consistent patterns. They were designed for low light video applications. Its a great planetary camera! Mark found similar issues with his Sony A7s.

[Fay]

I have never used Bias but always do Flat Darks, as you should if not using Bias

[Carole]

The chips don't produce consistent patterns. They were designed for low light video applications.

Yet they sold it as a Deep Sky imaging camera?

I think I'd be asking for my money back or an exchange of camera.

Carole

[MarkS]

The smarty pants on CloudyNights (not Mark for a change!) has been through all the options. The chips don't produce consistent patterns. They were designed for low light video applications. Its a great planetary camera! Mark found similar issues with his Sony A7s.

I'm dead impressed by the effort he has put into his analysis.  Maybe if I tried a bit harder I would make a bit more progress in understanding my Sony A7S!  For one reason or another there do appear to be calibration issues and colour gradients on many Sony CMOS sensors.

Mark

[NoelC]

Fascinating article, but I felt his position on dark frames wasn't consistent.
It starts by saying dark current is non-linear with temp and time, but concludes that it's OK to use scaled dark frames(?). The implication of his first bit of work was library darks would not work. Pretty conclusive on bias frames though.   Apart from the serious Amp glow issues, it sheds no light on the blotching, but suggests that problems may occur through flats (if taken using exposures sub 3 seconds - which all of mine are).  The suggestion that the response will vary dependent on what the chip is or has been doing is also quite worrying (and could account for blotches - but not I suspect your original ones).  I was quite interested in this camera, but am starting to go off the idea...  Did you look at the dark frames for blotching? 
Can you get round the problem?