English Version
Thanks to Edmond Shum to help us to translate this article to English
Dr.Qiu's Talking about Astronomical Photography
Why Dark Field Is Darker Than Bias Field
and Overscan Calibration
Amateurs often encounter the same issue: dark field is even darker than the bias. In the our usual understanding, the bias field exposure time is zero or very short, while the dark field exposure time is long, we expect that exposure frame has more dark current, so the average value of the dark field should be higher than the bias field. But why is it lower? We may first suspect that it is the camera problem, or driver problem.
In fact, it is very normal and is known as overscan correction.
For CCD, this problem arises from the temperature drift of the reference voltage of the AD converter. CCD output is the analog signal, through the AD converter to convert to digital signals. The AD converter requires a voltage as a reference for conversion, which is called the reference voltage, as if we are measuring length and need a standard ruler. If the scale of the ruler changes, then the measured value will change.
Therefore, if the reference voltage changes, then the value of the conversion of the AD converter will follow and change the overall image brightness.
The reference voltage is supplied via a standard source. Usually these voltage sources will have a certain temperature drift. Taking the common high-precision reference source TL431 as an example, its temperature drift is 50ppm per degree. That is, every change in degrees Celsius will have 50 per million. At first glance this value is small. But in terms 16-bit ADC range, each degree will change several ADU. Ten degrees will result in dozens of ADU. This is very obvious.
In CCD, usually only the chip is being cooled and temperature controlled, but not the ADC. As the ambient temperature changes, the overall image will be slightly brightened or darkened.
This is easy to explain why the dark field will be darker than the bias field. Usually we shoot multiple dark field and bias field for superposition/averaging. Since the exposure time of the bias field is extremely short and mainly the time is spent on read out, the ADC is in continuous operation and its temperature rises. This causes the reference voltage to drift. While continuous shooting the darks , the dark field exposure time is much longer. In such case, ADC does not work as hard, only during read-out, so the temperature does not rise as much.
That’s why the average of the dark field appears to be darker than the bias field.
In addition, changes in ambient temperature has similar effect. For example, when we shoot the bias, the room temperature is 25 degrees, and shooting dark field when ambient termperature is -10 degrees, thus it results in considerable drift in the ADC.
How to use OVERSCAN calibration to solve the problem
There are several ways to solve this problem. We first introduce the manual calibration method. In order to let everyone understand the principle of calibration, the following method take the MAXIMDL software as an example.
First we can shoot a series of bias, darks, lights, and flats. Note that you need to save the OVERSCAN area during shooting, you *must* uncheck "ignore OVERSCAN area"
Create the masters for averaged the bias, dark, bright, flat.
Open the master bias field. You can see there is a black side in the image on the right, this is the OVERSCAN area. Use the mouse to draw a box. This box needs to include a small part of the black edge. Record the average of the pixels in this box, eg 500 from the INFO window.
step 1
Use the PIXELMATH tool. Set the image A to the master bias field. Select “Calculation Method” to NONE. Then in OFFSET, enter the negative value of the above average, -500
Execute PIXELMATH. Complete the OVERSCAN calibration of the master bias field.
Then do the OVERSCAN calibration in the same way for the masters of flat, light and dark.
Step 2
Use the PIXELMATH tool to set the image A to the master flat and the image B to the master bias. Select SUBTRACT for “Calculation Method”, OFFSET is set to 0. The result is a precisely calibrated flat field.
With the PIXELMATH tool, set the image A as the master light and the image B to the master dark. Select SUBTRACT for Calculation Method. OFFSET is set to 0. The result is a precisely calibrated light frame.
Use the PIXELMATCH tool to set the image A to the precisely re-calibrated light field and set the image B to a precisely re-calibrated flat. Select DIVIDE for Calculation method , OFFSET is set to 0. The result is a full set of calibrated images.
The above calculation method is based on the formula
The corrected image = (L-D) / (F-B)
From this formula we can observe that the bias is primarily for on the calibrating the flat field. Dark frame is mainly used to calibrate the light frame for the dark current.
If the flat field is not correct, we usually see 4 corners are brighter than the center ( over-correction ), or darker ( under-correction ).
If the light frame calibration is not accurate, then the heat noise deduction is not accurate. (Sometimes the CCD temperature is already set at fixed value, but result varies as it is quite possible).
Automatic calibration method
Today, some software has the automatic OVERSCAN calibration like SGP (Sequence Generator Pro). So you can set this in the software. Please consult the experts of those software.
Note: If the average of the overscan area is higher than or very close to the effective reading, what should we do ?
This will result in subtraction of the whole image to zero , or to be reduced to near zero. In this case, a constant should be added to -500 in step A, such as 1000. The result is +500, and then perform the overscan correction. Note that this is only applied to step 1, step 2 of OFFSET or 0
CMOS case
CMOS cameras also have such problems, and the impact on CMOS is more complicated, because some of the CMOS chips come with optical dark level calibration. It will automatically do an optical dark level calibration. Optical dark level calibration and OVERSCAN area calibration are different. The difference is that the optical black level region contains thermal noise, while the OVERSCAN calibration does not. So when the exposure time is longer, CMOS dark current of the dark area increases, resulting in increased optical dark value to be subtracted, more than that of the shorter exposure. This leads to the dark field darker than the bias field situation. For CMOS calibration problems, please refer to "from the SLR camera to QHY163M" article. The English version of the article is at http://www.alessiobeltrame.com/wp-content/uploads/2017/09/QHY163M_review_EN.pdf
Chinese version in the "QHYCCD astronomical photography" can be found in groups files under related QQ group.
references
1. CCD Theory http://astro.ufl.edu/~lee/ast325/handouts/ccd.pdf
2. using the Overscan Bias Correction http://www.mirametrics.com/tech_note_overscan_bias.php
3. Calibration http://www.astro.caltech.edu/~aam/science/thesis/total/node15.html
Q博談天文攝影之
暗場比偏置場還暗問題 與 過掃區矯正
經常會碰到一些同好詢問一個問題，拍攝的暗場比偏置還更暗，也就是說暗場的背景平均值比偏置場的平均值還低。在通常理解裡面，偏置場曝光時間為零或者非常短，而暗場的曝光時間是長曝光，大家會覺得長曝光下，熱噪聲在增加，因此暗場的平均值應該比偏置場更高才對。為什麼反而更低了呢。這時候大家首先可能會懷疑相機出問題了，或者驅動有問題。
其實這個現象是很正常的，而且這個現象還與一個名詞：過掃區校正 非常相關。
對於CCD而言，這個問題產生的原因來源於AD轉換器的參考電壓的溫度漂移。CCD輸出的是模擬信號，要通過AD轉換器轉換為數字信號。AD轉換器需要一個電壓作為基準進行轉換，這叫做參考電壓，就好比我們測量一個長度，必須有一把標準的尺子。如果這個尺子的刻度變化了，那麼測量出來的值也會跟著變化。
因此，參考電壓如果變化了。那麼AD轉換器轉換出來的值也會變化，就會引起圖像的整體亮度變化。
參考電壓是通過一個基準電源源提供的。通常這些電壓源都會有一定的溫漂。以常見的高精度基準電源源TL431為例，它的溫漂是50ppm每度. 就是每一攝氏度變化百萬分之50.乍一看這個值很小。但是對應到16位的ADC的範圍而言，每一度就會變化好幾個ADU. 十度就會是幾十個ADU。這個就很容易看出來了。
由於通常相機是CCD芯片製冷並且進行溫控，而不對ADC進行溫控。因此就會導致環境溫度變化了，圖像整體會略微變亮或者變暗。
這個就容易解釋為什麼暗場會比偏置場更暗的原因了。通常大家會連續拍攝多張暗場和偏置場進行疊加。由於偏置場的曝光時間極其短，因此主要時間都在讀出，ADC處於連續的工作狀態，自身溫度就升高。引起參考電壓發生漂移。而連續拍攝暗場的時候，由於暗場的曝光時間長，在曝光的時候，ADC不工作，只有讀出的過程才工作，因此溫升沒有那麼大。
所以就出現了暗場的平均值比偏置場還要暗的情況。
另外，環境溫度的變化也同樣會引起這個效應。比如我們拍攝偏置的時候，室溫是25度，而拍攝暗場的時候，室溫是-10度，那麼就會產生可觀的漂移。
如何使用OVERSCAN校準解決該問題
解決該問題的方法可以有若干種。這裡我們首先介紹手動校準方法。以便讓大家深入理解校準的原理。下述方法以MAXIMDL軟件為例。
首先我們可以拍攝一系列的偏置場，暗場，明場，平場。拍攝的時候注意需要保存OVERSCAN區域，不能選擇“忽略OVERSCAN區域”
分別將偏置場，暗場，明場，平場進行疊加，獲得主偏置場，主暗場，主明場，主平場。
打開主偏置場。可以看到在圖像右邊有一個黑邊，這個就是OVERSCAN區域，然後用鼠標拉一個框。這個框值需要取一部分黑邊即可。記錄下這個框內像素的平均值。例如500（用INFO窗口可以顯示）
步驟1
使用PIXELMATH工具。將圖像A設置為主偏置場。計算方法選擇NONE。然後在OFFSET裡面，輸入上述平均值的負數，-500
執行PIXELMATH. 完成主偏置場的OVERSCAN 校準。
用同樣方法，對主平場，主亮場，主暗場進行OVERSCAN校準。
步驟2
然後用PIXELMATH工具，將圖像A設置為主平場，圖像B設置為主偏置場。計算方法選擇SUBTRACT(減法）。OFFSET設置為0。運行。得到的結果就是經過了精確校準的平場。
用PIXELMATH工具，將圖像A設置為主亮場，圖像B設置為主暗場。計算方法選擇SUBTRACT(減法）。OFFSET設置為0。運行。得到的結果就是經過了精確校準的亮場。
用PIXELMATCH工具，將圖像A設置為精確校準的亮場，將圖像B設置為精確校準的平場。計算方法選擇DIVIDE(除法）。OFFSET設置為0。運行。得到的結果就是經過全套校準之後的圖像。
上述計算方法依據的公式為
校準後的圖像=（L-D)/(F-B)
從這個公式也可以看出來。偏置場主要是在平場校準的時候其作用。暗場主要是在對亮幀做暗場校準的時候其作用。
平場如果不準確，容易出現的問題是校正過度(四個角比中心還亮），或者欠校正。
亮幀校準不準確，就是熱噪點扣不準確。（有時候CCD溫度已經是相同了，還扣不準，這個是一個可能性）。
自動校準方法
目前一些軟件具備了OVERSCAN自動校準功能。比較典型的例如SGP軟件。因此可以對此進行設置。具體方法請諮詢相關軟件使用高手
注：假如出現了過掃區平均值比圖像有效像素去還要高，或者很接近怎麼辦？
這樣使用減法就會導致圖像被全部減成零。或者很接近導致有部分像素被減成零。這種情況下，對於步驟A中的-500，應該再加上一個常數，比如1000. 這樣結果是+500. 然後執行過掃區校正。注意這個僅限於步驟1，步驟2的OFFSET還是為0
CMOS的情況
CMOS相機也存在此類問題，並且影響CMOS的更為複雜，因為某一些CMOS芯片內有具有光學黑電平校準功能。會自動做一次光學黑電平校準。光學黑電平校準和OVERSCAN區域校準是有區別的。區別在於光學黑電平區域，是包含熱噪聲的。而OVERSCAN區域是不包含熱噪聲的。因此當曝光時間較長的時候。CMOS的光學黑電平區域的暗電流增加，導致光學黑電平校準的時候扣的值，比曝光時間短的時候扣得更多。從而也導致了暗場比偏置場還要暗的情況。關於CMOS的校準問題，請參考《從單反相機到QHY163M》一文。該文章的英文版在http://www.alessiobeltrame.com/wp-content/uploads/2017/09/QHY163M_review_EN.pdf
中文版在“QHYCCD天文攝影”QQ群的群文件中可以找到。
參考文獻
1. CCD Theory http://astro.ufl.edu/~lee/ast325/handouts/ccd.pdf
2. Using the Overscan Bias Correction http://www.mirametrics.com/tech_note_overscan_bias.php
3. Calibration http://www.astro.caltech.edu/~aam/science/thesis/total/node15.html
閱讀更多 QHYCCD光速視覺 的文章
