Let's start at the very begining... (well the processing part anyway)

[Ivor]

Morning,

Well as discussed in the intro section before I try and process my next image I want to check the quality of my darks, flats, offset and dark flat to eliminate them from the processing issues I'm struggling with. I've put a  copy of my masters here
https://picasaweb.google.com/103066655176609881871/Astrophotography , my understanding of the definition of these files is:
 
Lights — frames taken with the imaging camera through the imaging scope, with the dust cap off.
Darks — frames taken at the same ISO, exposure time, and temperature as the Lights, but with the telescope cap on.
Offsets/Bias — frames taken at the same ISO, but with as short an exposure time as the camera allows (1/4000th in my case) and with the camera's body cap in place.
Flats — frames taken of an evenly illuminated target, through the telescope @ 1/250th temperature doesn't matter.
Flat Darks — frames taken at the same ISO, exposure timeas the Flats, but with the camera's body cap in place.
 
Please let me know if my understanding of these files is correct and the images look as expected.

Thanks

Ivor
  

[Carole]

Your understanding looks pretty much correct, but I had a look at your files - very nice M27 and M33 by the way.  
I am not sure if the Master flat is right as it doesn't appear to show any vignetting or dust particles when stretched.  It might be because of copying a file from your attached site.

The flats need to be half well depth, or roughly half way accross the histogram.  I was doing mine wrong for quite a while before some-one kindly pointed out to me that my flats were not doing anything to my images.  I had been over exposing them and washing them out.  

I now use 100ISO and switch to AV which gives a nice short exposure and point at a dull daytime sky.  There are other methods of doing this, depending on your circumstances and what suits you.  

I am sure the other more expert imagers will give you further advice.

Carole

[Mac]

Hi Ivor

Your understanding of the definition of the files is correct, but the one that always makes me chuckle is the flat dark, why? anyway.

A Light frame is your image file plus all the other junk.

The idea of the flat frame is to take an image of the optical patch and photograph all of the debris within the path,
i.e all the junk on your filters, ccd lenses, ect. so to this you need light. As carol said, this is normally taken so the well depth is half full, so check your histogram on the back of the camera, and make sure that you have not
overexposed your image, even underexposing by 2 stops should be ok.

A dark frame is an image that records all the noise present within the camera/ccd. As the camera heats up during the exposure it produces noise,
as you said this, is normally done at the same iso, and length of exposure, so these should be ok.

A bias frame is the frame that records the readout noise from the ccd/camera.

All you need are these four files to process your image,

however!!, you need to take your flats at the same time as your lights, so that any debris is in the same place optically.

If you take them at a later date, the debris will have moved and you will get strange processing problems.
If you look at this flat that has been stretched you can see the junk in the image train.



If you set your camera up next time, the junk will have moved, so any processing that takes place using the flat will affect the image.


Bias files contain only read noise.
Dark files contain read noise and exposure noise, (no illumination problems as there is no light)
flat files contain all the uneven illumination noise.
light contain all of the above plus your image

by subtracting each of the bias and dark above from your light frame, you are just left with the image and the illumination problems
by then adjusting the exposure with the flat frame, you are just left with your image.

Now by stacking your image files you reduce the ratio of signal to noise,
so the more frames you take, the better the final image, upto a point,
adding more frames past this limit will not increase the image noticeably.

then there is the subject of colours, but thats all down to you and artistic licence. 

Mac

[Fay]

I always do flat darks.

In DSS and Astroart, there is a place for them as well, when stacking or processing. I think you do not have to do bias, if you do flat darks, not sure.


Fay

[Ivor]

thanks for the replies I hadn't realised the flats had to be taken at the same time, I've been used a set I created at the begining, so they probably need redoing. Your comments have left me the three questions:

I don't understand what you mean by "well depth is half full" I've looked at the histogram of Mac's example and it is clearly broader than mine but I thought that was also down to the diameter of the telescope and the amount of light getting in.

My next question is logistical how do yo manage to do the lights as, At the time of year I'm home after dark so doing flats would provide difficult without a light box. Do you have a light box or a cunning alternative?

As it would appear my flats are rejects is there a way I can get around this and not have to throw away my lights?

[Carole]

Unfortunately you can't do flats after the event unless the camera and optics haven't been moved.
Mark has a way of reusing flats, but I am not sure what his method is.

You might be able to get away with doing flats purely for vignetting, provided you have the same orientation, and same focus etc, but the chances of you lining up the dust is pretty impossible.

Doing flats in the dark you can either use a light box, or a flats panel:
http://www.gerdneumann.net/v2/download/Pricelist%20No%2027%20Brutto_Netto%20EN.pdf

Or leave the optics assembled until the following day and take them in daylight.  Either put a cover over your set up, or disassemble the telescope and camera from the mount without moving any of the optical light path. 

I always do flat darks.

I keep getting conflicting advice about this, some people say it is not necessary and some say it is.  

Carole

[RobertM]

Ivor,

You can take flats against any evenly illuminated smooth white surface, I'm using a white card on a wall in the observatory - akin to what the professionals call dome flats.   I illuminate the white card with some white LEDs through a diffuser.  You can do them any time but, as Carole mentioned', it must be before you change the optical configuration in any way.

A light box or flats (ELP) panel isn't needed but they do make life much much easier if you haven't got a permanent setup.  If you have a smallish refractor then an option is to keep the camera/optics together as a unit then you could do the flats indoors in the warm.

Carole,
Dark Flats - take darks for your flats so you have another set of darks to process.  I've never done that as I take sufficient flats (30-50) to bring the noise down but technically dark flats should be done.  Trouble is I don't know anyone who uses them apart from Fay and perhaps JP and Chris ?  I would say that if your Flats are of short duration then noise due to dark current from the sensor is not an issue and can be ignored, thus Dark Flats are not needed.

Mac - 2/3rd well is best, half well will lead to more nose in areas with strong vignetting.

Robert

[Mac]

can some one explain how you take a dark flat, because they are of no use.

To take a dark you need to cover the optics, thats why its called a dark.
To take a flat you need light, so you cant take a light with the cover on.

you cant take a DARK WITH LIGHT,
you cant take a light with dark.

so it is IMPOSSIBLE to take a dark flat.
anyway.

You have the following.

Dark  all the noise from the CCD due to heating ect.
Flat   all the noise created by dust in the optical train.
Bias  all the noise created by reading out the data from the chip
Light everything above including your image.

dont forget though,

your dark will have bias noise.
your flat will also have bias noise.

To get a clean light, you need to remove the bias noise and the dark noise and then adjust the exposure to compensate for the dust by using the flat.
this is done automatically by most software.

If you use incorrect flats, then the adjustments made to your image will give you processing errors.

Mac.

ps. the image of my flat has been stretched so that you can see the debris.

[MarkS]

can some one explain how you take a dark flat, because they are of no use.

They are not "dark flats" but "flat darks".  

They are only required if your flats require you to use a long exposure.  Flat darks are darks taken with the same exposure time (and ISO setting) as the flats. I never use them.


Mark

[MarkS]

Ivor,

Flats - my take on flats is the following.

For a refractor and DSLR combination, just point the scope at your LCD monitor using a low ISO (ISO 100).  Adjust the exposure so the pixel values are around half the max possible.  Newtonians and SCTs tend to be more problematic.

I disagree with Carole:

You might be able to get away with doing flats purely for vignetting, provided you have the same orientation, and same focus etc, but the chances of you lining up the dust is pretty impossible.

The truth is that for a DSLR user, dust particles only show up in the flat only if they are on the CCD itself.  Any filters being used are too far away from the CCD to create a noticable shadow.  So there is no need to get stressed over the filters and optics all being in the same precise orientation as they were during shooting. Also, the exact position of the focuser makes almost no difference whatsoever.  However, the position of the dewshield might affect things so make sure this correctly extended/retracted.

Moreover, if you treat the camera carefully (i.e. no not leave it with the insides exposed) dust particles don't tend to move around much.  The dust specks on my Canon 350D have not moved for 18 months and the camera has travelled around literally thousands of miles in the boot of my car - so I'm still using the same flats I took 18 months ago.  If you are using a more modern camera that vibrates the CCD to remove dust then SWITCH OFF the vibration permanently because otherwise it will tend to move the dust around and you will need new flats each time.

Mark

[MarkS]

I always do flat darks.

I keep getting conflicting advice about this, some people say it is not necessary and some say it is.  

Well ignore the conflicting advice and just listen to me 
You don't need flat darks unless you are taking flats using long exposure times.  If so, your flat dark needs the same exposure time and ISO as your flats.

[Carole]

Thanks for clearing that up Mark regarding flat darks.

I am pleased to hear about the optical train regarding dust on filters etc with a DSLR I was just quoting what had been told to me when I was learning. 

BUT, as a prime example, if you remember when I came to High Halden on 1st October, my M31 which I took, I had left the camera in place to take the flats the following morning.  Unfortunately the camera was not tightened sufficiently to the flattener and in pressing the on/off switch on the DSLR it caused it to rotate.  I attempted to put the camera back to the same orientation, but did not succeed, with the result that the dust (which according to your explanation above must have been on the CCD) did not line up and I got two dust bunnies which have spoilt my image.

I plan to mark the flattener and camera positions for the future - AND make sure the camera and flattener are tight before imaging.  

Carole

[RobertM]

Carole, I'm afraid your're getting to be a bit of a Forum junkie.  If you will insist on getting advice from numerous forums on the same topics they you're bound to get conflicting advice.

In this case, because of the longer focal ratio, those dust specks will in fact cast a shadow and cause a dust bunny.  Mark is entirely correct at short focal ratios but at longer focal ratios you will still need to take flats to remove any bunnies.  Of course if your optics were free of dust then that would be be the ideal scenario 

Robert

[MarkS]

BUT, as a prime example, if you remember when I came to High Halden on 1st October, my M31 which I took, I had left the camera in place to take the flats the following morning.  Unfortunately the camera was not tightened sufficiently to the flattener and in pressing the on/off switch on the DSLR it caused it to rotate.  I attempted to put the camera back to the same orientation, but did not succeed, with the result that the dust (which according to your explanation above must have been on the CCD) did not line up and I got two dust bunnies which have spoilt my image.

I'm happy to be proved wrong but I need to see the evidence. 

My calculations tell me the following:
The Canon CCD is 44mm behind the front mating surface of the camera.
The focal reducer must therefore be at least 44mm in front of the camera.
Even at F/10 a dust particle on the reducer would create a dust bunny on the CCD 44/10 = 4.4mm across
The 450D pixel pitch is around 5 microns so your dust bunny would be nearly 1000 pixels in diameter at F/10

But you were probably imaging at F/6 which means the dust bunny would be 1500 pixels across.
I seriously don't believe a 1500 pixel dust bunny would be noticeable.

If the dust was on the clip-in filter or on the CCD then when you rotated the camera, the dust bunny would remain in the same place on the image.

Mark

[MarkS]

In this case, because of the longer focal ratio, those dust specks will in fact cast a shadow and cause a dust bunny.  Mark is entirely correct at short focal ratios but at longer focal ratios you will still need to take flats to remove any bunnies. 

See my calculations in previous post.  Would anyone notice a 1500 pixel dust bunny?

Mark