From: "milton_aupperle" <milton@outcastsoft.com>
Date: September 1, 2010 3:11:10 PM MDT
To: Astro_IIDC@yahoogroups.com
Subject: Light Curve Bench Mark measuring
Hi Folks;
I've always wanted to know where all the estimation errors are for light curves and which ones are external and which are internal to the cameras. So I created a benchmark system to measure brightness changes over time for simulated stars. Nothing elaborate, just a battery powered flashlight covered over with aluminum foil and needle holes punched through for stars. I cooled the Grasshopper to 20°C below ambient temperature to +5°C and had it resting on a bench with a 150 mm focal length SLR camera lens stopped to to F22. Then I recorded about 100 frames at 15 fps using 7 ms exposures. This rig eliminates the issue of turbulence, focus, clouds, haze, moon or city lights, weather and mount tracking issues - leaving us with pure camera errors.
Here are the stars in my test rig with the C0 and C1 being my constant stars and the V0, V1 and V2 being my "variable" stars to measure:
http://tech.groups.yahoo.com/group/Astro_IIDC/files/Other/PhotomPickedStars_MJA.jpg
I measured the magnitude change using the "Measure Differential Photometrics.." feature of Astro IIDC.
My first test was done with the fan on for the cooling rig:
http://tech.groups.yahoo.com/group/Astro_IIDC/files/Other/FanOnTest.png
and a second test with the Fan off (briefly) is shown here:
http://tech.groups.yahoo.com/group/Astro_IIDC/files/Other/FanOffTest.png
I wanted to see if the fan vibration was causing any significant change in magnitude change.
Unfortunately, I bumped into the tripod holding the flashlight / star between tests, so it was skewed slightly off axis and the inner star brightnesses are not quite the same. I didn't notice until after I took the rig apart that the inner magnitude estimates had changed as the C0 C 1 star are the same.
The S/N mag error estimates for both test is exactly the same at +/-0.0025 mags or +/- 2.5 millimags of variability. The S/ N error compare star signal strength to background around the star in the outer annulus rings.
You can see in the graphs that the "Fan Off" case has less variability between samples, but I wanted to quantify this. To do that, I compare sample repeatability by calculating the difference (delta) between subsequent samples for each of the stars.
For the Fan On test, the delta samples show a maximum change of +/- 0.0295 mags between all samples. The Standard Deviation (at 68% Confidence Interval) for all the delta samples comes in +/- 0.0113 magnitudes. This means that the worse change between readings I can expect is +/-0.03 and 68% of the time the measured magnitude is +/- 0.011 mags of actual.
For the Fan Off test, the delta samples show a maximum change of +/- 0.022 mags between samples. The Standard Deviation (at 68% Confidence Interval) for all the delta samples comes in +/- 0.0086 magnitudes. This means that the worse change between readings I can expect is +/- 0.022 mags and 68% of the time the measured magnitude is +/- 0.0086 mags of actual.
When I step through the frames in the Fan On and Fan Off movies and examine the stars at 400% magnification, I can't see any motion of the stars between samples. So the fan must be causing a very slight sub pixel motion that is changing the brightness of the stars enough to add some variation to the tests.
What you can take away from this experiment is two fold.
The S/N error estimate doesn't really mean anything about magnitude accuracy between samples. It's really a measure of the variability of the background to the star and with good dark skies away from urban light can be very very low.
Secondly turbulence plays a huge role in the accuracy of your measurements. A very slight (not even perceptual) amount of sub pixel shifting due to a cooling fan is adding 30% variability to the measurements under static conditions. My guess is this would be in the < 0.5 arc second range for this test. So under real world conditions where turbulence is moving stars +/- 3 to 6 arc seconds, you can get really bouncy results.
One caveat to my measurements is that I'm using very short exposure times of 7 ms per frame. If there is even the slightest variance in the camera's shutter timing by as little as 0.07 ms (thats 70 micro seconds), that would account for 1% variance or 0.03 magnitudes in brightness between samples. To remove this possibility means adding in a neutral density or narrow band filter to reduce the light levels far enough to do 10+ second exposures. I may try that in the future if I get time.
Hope this helps someone..
Milton Aupperle