So why haven't I produced any images recently? It's because I currently have a processing backlog awaiting solutions to one or two Sony issues.
If you have a technical mindset then here is my latest discovery:
https://www.cloudynights.com/topic/557931-concentric-coloured-rings-caused-by-flats/?p=7853605
Unfortunately I have been using flats shot at ISO 100 to calibrate lights shot at ISO 10000. This has created lots of artifacts related to concentric banding. On any normal camera mixing ISOs would be no problem but this is Sony :(
I can't believe the amount of time I have wasted investigating and commenting on Sony issues. I seriously wish I hadn't bought the Sony A7S.
Mark
Haven't read the link as yet, but despite the problems you have had with it, you have managed to produce vastly superior images and detail than you could get from any other OSC camera.
Will try to look at that link if my non technical mind can cope with it.
Carole
Blimey Mark you are a real trooper with this camera - you figured out how many hours you worked trying to sort out the issues? What you need is someone that can download/ hack the camera software so you can see exactly what they are doing when processing the data in various Bulb/ non Bulb modes etc.
Like carole says you have still produced awesome images with this camera despite its issues.
John
I honestly don't want to think about the time I've spent investigating issues. It probably amounts to a whole month or two of solid evenings over the last 2 years. At the moment I am really, really despondent - I'm on the verge of ditching the camera altogether and buying Nikon. Nikon use the excellent Sony sensors but build a camera with fewer compromises.
On the other hand it is true that I've produced my best ever images with this camera. But push the camera too far and it produces a mess. I wanted to capture M81 M82 and the integrated flux nebula. But when I stretch the image enough to reveal the IFN I get this:
(http://www.markshelley.co.uk/webdisk/SonyA7s_M8182_PurpleBand.jpg)
Somehow the flats have led to purple and green swathes through the image. But why? I think it is related to the concentric banding issue. I'm running loads to tests to get to the bottom of it and again it's consuming enormous amounts of time.
It's now just over 2 years since I introduced the world to the problem of Sony's "star eater". It was a struggle back then with most people disbelieving me or not seeing it as an issue. However, in the last few weeks it has gone mainstream - some of the biggest bloggers have picked up the story:
http://www.sonyalpharumors.com/specific-a7sii-astrophotography-fix-request/
https://johnleathwick.wordpress.com/2017/04/29/sony-and-the-star-eater-algorithm-or-how-sony-crippled-the-worlds-possibly-best-cameras-for-astrophotography/
http://www.lonelyspeck.com/why-i-no-longer-recommend-sony-cameras-for-astrophotography-an-open-letter-to-sony/
The idea of mirrorless cameras is excellent in principle but Sony has given them multiple Achille's heels.
Mark
I can see your problem Mark, it looks like an uphill battle. I think if I was in your shoes I'd have thrown in the towel on that camera by now.
Haven't you got a OSC CCD camera? Why not get colour with that on one of your two rigs and get a Mono CCD camera on the other, with your skills you'd get some stonking results.
Carole
Quote from: Carole
I can see your problem Mark, it looks like an uphill battle. I think if I was in your shoes I'd have thrown in the towel on that camera by now.
If I don't solve this problem then I'll be buying a full frame Nikon.
I do have a OSC CCD but I never use it because its sensitivity was no better than the Canon 350D I was already using. The sensitivity of the Sony A7S was more than double and it's a full frame sensor so a typical exposure collected 4x as many photons as the Canon! That's why the image quality was so much better.
I've become far more demanding and want to extract really faint details and so I'm hitting a new set of issues.
However, if I ditch the Sony, I will really miss the incredible live view functionality.
Mark
defo looks very messy - wouldn't know where to start on programming this. What's the specific Nikon model that you mention? Cheers, John
Nikon D810. It's not cheap though :(
I'll need to modify it of course.
I also need to check that it doesn't have gappy histograms. Unfortunately the D610 does has gappy histograms that could lead to concentric banding. So it indicates that Nikon are heading the wrong way :(
Mark
Ripping a £2,500 camera to bits would fill me with utter dread...! I reckon that you & Mike should team up with your analytical, mathematical & Software skills & his electronic abilities you could probably build/ design the ultimate astro camera. I've got access to workshops (metal) & CAD stuff... ;-)
Hi Mark, sorry to bang on about this (as I have also suggested it to you at other times), but you don't need the detail in the colour camera (OSC CCD), the colour is just a "wash", the detail comes from the Luminance or Ha filters.
I was told this when I first switched over to CCD imaging, and I have seen the proof over and over again when processing. I have even used my old DSLR images to provide colour and added Luminance to them. You can just blur the DSLR images to smooth out the noise and then overlay the luminance on top. So should work the same with a OSC.
I agree the OSC CCD's do not give any more detail than a 350D, but used in combination with a Mono camera on a dual rig would give great images.
The amount of detail you can get from a Mono camera is much much better than a OSC CCD.
I will shut up now.
Carole
Quote from: Carole
Hi Mark, sorry to bang on about this (as I have also suggested it to you at other times), but you don't need the detail in the colour camera (OSC CCD), the colour is just a "wash", the detail comes from the Luminance or Ha filters.
I agree the OSC CCD's do not give any more detail than a 350D.
I was told this when I first switched over to CCD imaging, and I have seen the proof over and over again when processing. I have even used my old DSLR images to provide colour and added Luminance to them. You can just blur the DSLR images to smooth out the noise and then overlay the luminance on top. So should work the same with a OSC.
I've never believed that "washing" colour over deep luminance data produces the best results. Sure, LRGB has traditionally been a quick way of obtaining a half decent image with high read noise mono cameras. But the problem is that if the the colour data does not go as deep as the luminance data then the faintest structures can only be grey or blurred to death. The result looks unbalanced to my eye and I've seen the proof over and over again in countless LRGB images. That's one of the reasons I love OSC imaging - it always produces a balanced result with all visible structures having the correct colour and not fading to grey.
I'm not alone in thinking this way - Juan Conejero (the author of PixInsight) is a strong proponent of RGB imaging. Sure, he provides support for LRGB in PI but he argues against using it. There is a growing movement back to RGB imaging with mono cameras - especially with the current crop of low read noise CMOS cameras, since good noise-free R, G & B data is now more easy to obtain than previously. If you're in doubt look at some of results I get with the uncooled low read noise Sony A7S, despite its many failings. My style of imaging is a deliberate choice. Maurice Toet even surprised Olly Penrice by going back to a DSLR from mono imaging but I'm certain this will be an increasing trend.
Coincidentally there was a very interesting thread on this very subject (LRGB vs RGB) on the Cloudy Nights forum recently:
https://www.cloudynights.com/topic/575319-lrgb-versus-rgb-a-fascinating-debate-at-the-highest-level/
However, in general I think it will prove difficult to win converts away from the LRGB "camp" because their mythology is so ingrained.
Mark
OK Mark, fair enough. I suppose I was just keen to see what you would produce with a mono camera.
Carole
By the way, if you want to follow progress (or non-progress) on what I'm calling the "Purple and Green swathes issue" then follow it on Cloudy Nights: https://www.cloudynights.com/topic/576265-sony-a7s-purplegreen-swathes I can't update it in more that one place.
Mark
Nightmare.
I know it's a daft question but what does the image look like without flats applied, I know it will have dust bunnies and vignetting, but would be interesting to see the difference.
Also for arguments sake, have you tried applying the flats and no bias subs:
I know you technical people will shudder at this suggestion, but I have in the past stacked an image and it's come out really awful, perhaps something on the lines of what you are getting or worse. Then some-one suggested to me to leave out the bias as it can sometimes muck it up. I did exactly that and the image came out just fine. This has happened to me on 2 or 3 occasions over the years though I must confess not on every image. Just thinking "anything might be worth a try" if only for the sake of elimination.
Carole
Any chance of letting us have some unstacked files and calibration files so we can "have a go" at them.
Carole
Hi Carole,
Those aren't daft questions, nor are they daft suggestions. Everything is worth considering when someone has reached the point of ditching a camera.
Quote from: Carole
I know it's a daft question but what does the image look like without flats applied, I know it will have dust bunnies and vignetting, but would be interesting to see the difference.
It's impossible to do. I'm stretching the data by a factor of 200 or more to reveal the IFN. Without flats applied, the vignetting means that only a tiny area of the image would be visible, with the rest either saturated or black.
Quote from: Carole
Also for arguments sake, have you tried applying the flats and no bias subs:
I know you technical people will shudder at this suggestion, but I have in the past stacked an image and it's come out really awful, perhaps something on the lines of what you are getting or worse. Then some-one suggested to me to leave out the bias as it can sometimes muck it up. I did exactly that and the image came out just fine. This has happened to me on 2 or 3 occasions over the years though I must confess not on every image. Just thinking "anything might be worth a try" if only for the sake of elimination.
Good suggestion and I'm already doing that. The colour banding does not exist in the bias or darks so I've eliminated them from the ongoing inquiry and use a synthetic offset instead of the bias frame.
Quote from: Carole
Any chance of letting us have some unstacked files and calibration files so we can "have a go" at them.
No chance. I'm not precious with my data but you need hundreds of subs to show this problem. They wouldn't even fit on a DVD backup!
I've wasted most of my day on this but I think I've found something very surprising. I'm waiting for dusk when I can take various sets of sky flats to either confirm or refute a very surprising hypothesis.
Mark
Waiting with baited breath.
Carole
Last night's test results were inconclusive. I'll take another batch tonight. But I did get an image of the moon and Jupiter in a cloud gap.
Oh dear, I've just noticed how much a brand new Nikon D810A actually costs!
Mark
Last night's batch confirmed my worst fears: I can't even calibrate one master flat with another master flat of the same ISO when there is a brightness difference between them i.e. the back of camera histogram peak is in a different place. Both master flats were taken in the same twilight session but the result leaves coloured vertical swathes through the data.
I'll admit that I'm applying a huge stretch to the data but it's no different to the stretch I would apply to an image of M81 and M82 to bring out the Integrated Flux Nebulosity.
Mark
So if l am understanding this correctly, since sky flats could vary, what about an artificial light source which will be consistent.
Presume using a single flat will not work. Otherwise could you just duplicate the single flat to produce a master.
Dreadful scenario.
Carole
Yes an artificial light would definitely be constant. But it would be white light, so the relative positions of red green and blue histograms would not match the relative positions of red green and blue histograms of the orange sodium light pollution in my light frames. This is the actual problem - my flats don't calibrate my lights - I get purple and green vertical swathes through the data.
I'm just calibrating master flat against master flat to see if I can narrow down the problem in a simple test environment. If I can't even do that successfully then there is absolutely no hope of calibrating the lights.
Mark
Many people use EL panels for their flats, including OSC, but maybe they are not stretching the images so much.
I do know I had some very strange results with my QHY8L which was a OSC camera and could never work out what was going on there. I was not stretching to the extent that you do. It makes me wonder now whether that might have had anything to do with the flats, though not the same problem as you are faced with.
In the end I abandoned OSC and just stayed with my Mono camera.
This is one of the images I got, I don't think I saved the other one.(http://cdn.astrobin.com/images/thumbs/c60b0e65e401291343cc77e52ae6e611.1824x0_q100_watermark.jpg)
Another thought, though it might be a stupid one. What would happen if you changed your flats/master flat to mono before calibration?
It's only supposed to be dust and vignetting you are trying to eradicate?
Carole
Quote from: Carole
Another thought, though it might be a stupid one. What would happen if you changed your flats/master flat to mono before calibration?
It's only supposed to be dust and vignetting you are trying to eradicate?
That's my back up plan - I already tried it and it works pretty well. It's not an ideal solution because these stripes are encoded in both the lights and the flats. If they are not in the same place in both then they are not corrected in the final image. Similarly, a flat converted to mono cannot correct the stripes in the lights but it does reduce the problem by half.
I love that image you posted - it looks like the background cosmic signature from the big bang!
Mark
So my idea of changing flats to mono wasn't so daft then. Glad it seems to work better than the colour version.
QuoteI love that image you posted - it looks like the background cosmic signature from the big bang!
I got that on several images and thought at first I had buggered up the sensor when I cleaned it with Alcohol as it had some very stubborn bits of dirt. Solving the problem was beyond me and all the help I got on one of the forums, so I sent it to some-one who was used to the camera and the software I could not get on with and he managed to get it working OK, so I just sold it. I think it might have been something to do with debayering and the software getting confused (at least that was the analysis of one of the people trying to help me), but far too complex for me to sort out. I never had that problem when I used APT, but for some reason APT kept crashing, so it was when I was trying to use other capture software that I ran into this problem. Mono is much more straight forward albeit a bit of a learning curve to combine the data.
Well hopefully you are making some progress.
Carole
Would be interesting to see the results of an image calibrated with a coloured flat and then the same image calibrated with a Mono flat, if you already have "one you made earlier".
Carole
Quote from: Carole
Would be interesting to see the results of an image calibrated with a coloured flat and then the same image calibrated with a Mono flat, if you already have "one you made earlier".
I'm doing a whole batch of test runs but each run takes a few hours because there are hundreds of light frames. The idea of a mono flat doesn't actually work in practice because it doesn't correct the coloured horizontal banding on the left hand side of the split sensor.
Mark
Quoteit doesn't correct the coloured horizontal banding on the left hand side of the split sensor.
Bummer, forgot about that problem.
Carole
Quote from: Carole
Quoteit doesn't correct the coloured horizontal banding on the left hand side of the split sensor.
Bummer, forgot about that problem.
So had I ;)
Mark