DSS problems

[Carole]

Not sure whether this should be Astrophotography or technical, please move if necessary.

Been having some problems with colour - images are coming out very pale, and I am suspecting I might have changed a setting on DSS but can't think what, any ideas?  I have done the DSC processing using kappa Sigma Clipping which I have not used before as DSS recommended it (also Chris), can this cause such a problem?

Second problem and this has been going on for quite a while now:
DSS does not seem to be removing dust, it removes the vignetting OK, this is my master flat:


These are my flats showing the histogram.


This is the resulting image with dust not processed out (again any thoughts on this (BTW I have auto cleaning now switched off as we wondered previously if the dust was moving):

[The Thing]

I changed from using AHD debayering to bilinear (Settings - Raw/FITS DDP Settings),  that seemed to solve my flats problems. I use Kappa-Sigma with settings of 2 and 2 iterations. M51 from Friday night (which I will post shortly) has come out wonderfully using DSS.

[Carole]

Thanks Duncan, I will give that a try.

Carole

[MarkS]

Agreed - the dust is not being removed for one reason or another.  But I'm not a DSS user so I can't really advise.

But I do know that if it's not removing the dust, it won't be removing the vignetting either (it is not that the dust has moved - most of it, at least, is in the same place).

As I have commented on a couple of your recent images, I agree with you that something is not quite right in the processing.  If you post some crops then it might be possible to determine where things are going wrong.  Especially important is a crop of the TIFF that directly comes out of DSS - the crop also needs to be in the same TIFF format.

Mark

[RobertM]

Are you using bias with your flats ? If not then that would be the issue with flat calibration.

Robert

[Carole]

I am sure it is removing the vignetting Mark, as I did a "quickie" whilst away just to check whether I had the object in the FOV (without flats) and it was absolutely awful to try to stretch without flats similar to the experience I had in my early days when I hadn't got the hang of how to do them.

On this occasion and it's the first time I have done this I did not use Bias as the Bias caused horizontal lines on the image, without it they did not, I'll post an example up.

I have the same problem with the dust with or without using bias.

Mark, not quite sure what you mean about posting a CROP of the original Tiff - wouldn't the whole Tiff be better?  If not, which bit do you want me to crop?  PLUS, in order to post it I will have to reduce it to a Jpeg.  Unless you mean a crop to make the file smaller so I can E mail it to some-one.  Could you elucidate please Mark.

Thanks

Carole

[mickw]

You could stick the Tiff on your website and just post a link here - wouldn't clutter up the gallery either

[Carole]

My website doesn't accept Tiffs either, but what I can do is post it on Dropbox and post up a link.

I am just looking for the file calibrated with Bias but can't find it so will have to do it again.

Be back soon with the Tiff link and the version with Bias.

Carole

[MarkS]

We need a TIFF rather than a JPG in order to preserve the full dynamic range (16 bits / channel?).

But the whole TIFF would be rather large, which is why I suggested a crop.  Choose an area that includes nebulosity, dark areas and one of the brighter stars.

[Carole]

I have put the tiff files into Dropbox:

1. Stacked without Bias
2. Stacked with Bias
3. Stacked tiff WITH Bias then stretched showing horizontal banding.

1. This is the Tiff of the Pelican stacked without Bias.
http://dl.dropbox.com/u/5730788/Pelican%20files/Pelican%20Nebula%202h%2045mins%205min%20subs%20ED80%2027-5-12%20no%20BIAS.tif

2. with BIAS:
http://dl.dropbox.com/u/5730788/Pelican%20files/Pelican%20Nebula%2027-5-12%20%20WITH%20BIAS.TIF

3. With bias but stretched and shows horizontal banding, which was why I left out the Bias frames when I did my processing.  
http://dl.dropbox.com/u/5730788/Pelican%20files/Pelican%20nebula%2027-5-12%20with%20BIAS%20stretched%20showing%20banding.tif

There was a lot of discussion at SGL7 at a talk on processing, about DSS's ability to process Bias properly and whether you should or shouldn't use them (this was with a top imaging guy called Steppenwolf on the forums).  I also processed my Horsehead with and without Bias at SGL7 and the bias definitely messed up the image with horizontal banding.

I will be really interested to see what you all come up with.  

Thanks

Carole

[MarkS]

I'll take a look at your dropbox images tonight but I have some questions/observations in the meantime.

If you use the bias, does it correctly remove the dust bunnies (at least where the dust has not moved position)?  If so, it demonstrates that DSS is performing correctly.  If this also causes horizontal banding then it sounds like the infamous "Canon banding" problem.  PixInsight and (I think) Noel's Actions have scripts to deal with this.  I also wrote one for IRIS.  Other processing suites may also have functions to remove it.

My explanation of the banding is as follows: with Canon cameras (at least the 350, 400, 450 range) every exposure has low level horizontal and/or vertical banding - it is most easily seen on bias frames but it is there, all the same, on every image.  It is also true of dedicated astro cameras but calibration with bias and flat frames removes it.  However, with the Canon, the calibration frames do not remove it.  I've done a lot of experimentation with this issue.  It seems that every time you switch the Canon off and on, the position of the bands moves.  This makes it almost impossible to produce a consistent set of calibration frames to remove it - unless you take lights, darks, flats and bias all in a single session.  Even then I'm not 100% sure you will get a consistent set of frames.  The problem is that if you use a set of calibration frames unmatched to your lights (or worse still a set of clibration frames that are unmatched to each other), then it actually makes the banding worse than using no calibration frames at all.

My own compromise solution is to use a artificial master bias frame which consists of all pixels having ADU of 256 (or 512 as appropriate).  You can create one quite easily by applying an extreme Gaussian blur to your existing master bias frame.  This won't remove the banding in your light frames but at least it will stop it being aggravated by using an unmatched bias frame.

Astro-imaging really pushes our equipment to extremes.  The closer to the extreme you get the more work is involved in extracting out the data from the noise.

Mark

[Carole]

That was an interesting read Mark.  I have just examined the stacked file which has the bias subtracted and although there do seem to be less dust bunnies, but still some remain. 

I then had a go at Noel's actions horizontal band removal but that did not sem to improve it at all.

It makes me think from what you have written that maybe my Bias frames have got too out of date.  I was given to understand that you could use the same Bias frames for months on end, but I have to confess mine are about 10 months old now and I have only been experiencing the banding problem since March, but the dust bunnie problem since I came to High Halden on 1st October, and I only stopped using Bias frames for the horsehead (done with the Atik in March), and this set of images, otherwise I have always used them.  
So maybe I should try a new set of bias frames and see what happens.  

I would also be interested to see what you could make of my subs stacking them in Iris, if you don't mind having a go at it Mark as I would like to see what it would produce (not that I could manage to use Iris myself, it's far too technical for me).

Many thanks

Carole

[Mike]

If you are doing darks then surely there is no need to do separate bias frames as the bias information will be contained within the dark frames?

[Carole]

There seems to be divided opinion on this Mike.
I just do what tends to work for me, but at the moment things are not working as they should.

Carole

[MarkS]

If you are doing darks then surely there is no need to do separate bias frames as the bias information will be contained within the dark frames?

What you say is kind of true, but you would still need the bias frames for the flats.  And it really depends on what order your processing software applies the calibration frames to the lights.  You'll find there is a set sequence in which to create your master bias, dark and flat.  Some processing suites don't expect you to produce masters anyway - you simply feed in the raw calibration frames - it really depends on the processing suite you are using.

There seems to be divided opinion on this Mike.

There is really is no need for divided opinion.  If DSS is being used then follow the DSS instructions for creating/processing calibration frames.  If IRIS or PixInsight is being used then follow the relevant IRIS or PixInsight instructions.  If the instructions are ignored then expect to get it wrong!

[MarkS]

Carole,

To me visually, both the "bias" and "no bias" images have exactly the same amount of banding.  I even verified this by subtracting one form the other and the banding completely disappeared - this shows there is the same amount in each one.  Did you post the right images?

There is also absolutely no problem with the colour balance in those TIFF files - here's a quick process I did by setting the black point and boosting the red by 20% and the blue by 10% (all done in IRIS as usual):



Larger version here: http://www.markshelley.co.uk/webdisk/carolepelican_iris.jpg

There's a odd blue patch of light top centre that might be caused by an external light source or internal reflections.  The colouring is good overall and it is possible to preserve good star colouring also.

The banding is typical of Canon and but I didn't try to remove it.

Mark

[JohnP]

Now that does look a lot better - nice one Mark. The link to the larger image is not the larger image by the way...?

[Carole]

Thanks for that Mark,

Did you post the right images?

Hmm, I think I had better check this out.

There's a odd blue patch of light top centre

Yes I wondered what that was.

I find it strange that the horizontal banding has only occured in 2 images that I have done and doesn't seem to appear in other images and until recently never had a problem with it at all.  

Which of the two images did you process Mark, I guess since there are very few dust bunnies that it was the one WITH the bias subtracted.

You have done an excellent job Mark thanks very much, I just need to try to make it work with DSS.  

Any further thoughts on the dust removal problem?

Thanks
Carole

[MarkS]

The link to the larger image is not the larger image by the way...?

Now fixed.

[MarkS]

Which of the two images did you process Mark, I guess since there are very few dust bunnies that it was the one WITH the bias subtracted.

I processed the bias subtracted one.

[Carole]

I'll have another go at processing the bias subtracted one then.

Thanks

Carole

[Mike]

There is really is no need for divided opinion.  If DSS is being used then follow the DSS instructions for creating/processing calibration frames.  If IRIS or PixInsight is being used then follow the relevant IRIS or PixInsight instructions.  If the instructions are ignored then expect to get it wrong!

i.e. RTFM !!