Например, Бобцов

Реализация в программируемой логической интегральной схеме (ПЛИС) алгоритма выделения точечных изображений в датчике волнового фронта адаптивной оптической системы

УДК 535
THE IMPLEMENTATION OF ADAPTIVE OPTICS WAVEFRONT SPOT EXTRACTION ON FPGA
РЕАЛИЗАЦИЯ В ПРОГРАММИРУЕМОЙ ЛОГИЧЕСКОЙ ИНТЕГРАЛЬНОЙ СХЕМЕ (ПЛИС) АЛГОРИТМА ВЫДЕЛЕНИЯ ТОЧЕЧНЫХ ИЗОБРАЖЕНИЙ В ДАТЧИКЕ ВОЛНОВОГО ФРОНТА АДАПТИВНОЙ ОПТИЧЕСКОЙ СИСТЕМЫ

© 2013 г. Yanyan Zhang*; Suting Chen*; Mei Li**
** Nanjing University of Information Science and Technology, College of Electronic and Information ** Engineering, Nanjing, Jiangsu, China ** Institute of Optics and Electronics, Chinese Academy of Sciences Laboratory on Adaptive Optics, ** Chengdu, China
** E-mail: zyy1983_gucas@yahoo.com.cn, sutingchen@163.com, limei@163.com

The skylight background and the noise generated by the detection device system in adaptive optical system have a great impact for the spot detecting precision. In order to eliminate their affects, a wavefront spot signal extraction system is designed and implemented in this paper, and the effect of the extraction and the precision are tested in the indoor adaptive optics precision tracking system. The experiment results show that the design is better in extracting the target spot; the centroid accuracy after processed is higher. In the other hand, the processing time delay is small, which can meet the adaptive optics system’s real time requirement.

Key words: adaptive optics, signal extraction, FPGA.

Фон, обусловленный свечением неба, и шумы, генерируемые в приемном устройстве адаптивной оптической системы, оказывают сильное влияние на точность обнаружения точечных объектов. Чтобы исключить эти эффекты была разработана система выделения точечных изображений в датчике волнового фронта, представленная в настоящей статье. И сам эффект выделения и достигнутая точность были проверены на лабораторной прецизионной адаптивной оптической следящей системе. Результаты эксперимента подтвердили высокую эффективность предложенной системы в выделении точечных объектов и лучшую точность определения центроида после обработки. Показано, что время, затрачиваемое на обработку информации, невелико, и устройство может удовлетворить требованию функционирования адаптивной системы в реальном времени.
Ключевые слова: адаптивная оптика, выделение сигналов, ПЛИС.

OCIS Codes: 070.4560, 070.6020

Submitted 04.05.2012

1. Introduction
Adaptive optics is a kind of system which can compensate the wavefront image distortion caused by atmospheric turbulence or other factors real-time to obtain high quality images [1–3]. The system is composed of wavefront sensor, wavefront controller and wavefront corrector. Adaptive optics system often work at night

due to the influence of the sky background, which greatly limits the system’s working hours and lead to discontinuous observational data. In  addition, adaptive optics is a complex system that the system’s own devices will inevitably introduce some additional noises which have a  greater influence on the detecting accuracy [4]. As a major component of adaptive optics system, the wavefront detector generates large numbers

68 “Оптический журнал”, 80, 1, 2013

of noises, in which the optical charge-coupled device (CCD) is the main noise sources [5–6]. Extract the wavefront signal from the background and noises can improve the adaptive optics system open-loop and closed-loop accuracy, as well as the detection capabilities.
The threshold is often adopted in adaptive optic system to reduce the impact of noise and extract the target at present. Each pixel is subtract the fixed threshold in the image as

f ¢(x, y) = ïìîïïíïf0(x, y) -T

f(x, y) > T f(x, y) £T,

(1)

where f(x, y) and f′(x, y) is the original pixel value and the value after processed in (x,  y) position, T is the threshold. It has also proved that the optimal threshold [7] is

Toptimal = m + 3s,

(2)

where m is the mean value of the background and σ is the noise variance.
In real detecting environment, the noise and the background are changing dramatic that the mean value of the background and the noise variance are hard to calculate. In other hand, the intensified charge-coupled device (ICCD) and the electron-multiplying charge-coupled device (EMCCD) are used in the system, which increase the difficulty of the noise and background calculation when adjust the gain. So the optimal threshold is suitable in theoretical analysis but not in real detecting. Manual threshold reduction method which not calculates the threshold automatically is used in practical applications. This method is too cumbersome which affects the system real-time and the noise cannot be subtracted cleanly due to the uneven distribution of the noise.
In this paper, we present a solution on hardware which can calculate the noises and the background value real-time. The implementation is based on field-programmable gate array (FPGA). Multiply image processing methods are  adopted. Pipelined and parallel processing technologies are used. The experiments are tested on the indoor adaptive optics precision tracking system and the results are analyzed.
The extraction platform is shown in fig. 1. The implementation process includes image acquisition part, wavefront target extraction part and centroid calculation part. The implementations of these parts are all under the control of synchronization signal of CCD (fv, lv, pv, the synchronized

signals) and the clock signal (clk). The image data after processed are stored in external storage and then transferred to the monitor host through peripheral component interconnect (PCI) interface. Then the image is displayed on the monitor. The centroid coordinates calculated a­ fter process are stored in FPGA internal memory (memory), and can also communicate through PCI interface with the host computer.
The main parts in fig. 1 are 1. The image acquisition part: the image acquisition methods are different according to the CCD type in the system [8–9]. This part can provide the correct timing for the following operations. 2. The wavefront target extraction part: this part completes the functions of filtering out the noise and extracting the target. It includs three modules: the median filter module, Top-hat operation module and the adaptive threshold module. Median filter can filter out spurious noise. Top-hat operation can estimate the background whose distribution is relatively stable. The adaptive threshold module subtracts the residual noise. Detail decomposition of these algorithms will be introduced in the following section. 3. The centroid calculation part: the centroid coordinates of the image after processed is calculated in this part. The CCD image acquisition part and the centroid calculation part are introduced in references [8–9]; the spot extraction part is described in the following sections in detail.
1.1. The design and implementation of wavefront spot extraction
There are three modules in the wavefront target extraction part: median filter module, Top-

fv, lv, pv Synchronized signals

FPGA

CCD_data [15…0]

1. Image acquisi-
tion

2. wavefront target extraction

Median filter

Top-hat operation

Adaptive Threshold segmentation

3. Centroid calculation memory

External

PCI Monitor

clk

storage

interface

host

Fig. 1. Real-time extraction platform based on FPGA.

“Оптический журнал”, 80, 1, 2013

69

hat operation module and adaptive threshold subtraction module. In the decomposition of these three algorithms, we found that both median filter and Top-hat operation are operated in a processing window based on the neighborhood data. The processing window can provide the gray value of the center pixel and its neighborhoods’ which are the prerequisite for the following processing. So the generation method of the processing window is first described.
1.1.1. 3×3 processing window generation module
We choose a 3×3 processing window in view of the spot’s size. There are nine adjacent pixels in the processing window. However, most of the image sensor is a serial output mode, which means that the pixels are progressive, so we need to build a suitable cache structure according to the camera’s output mode based on FPGA’s resource.
In this design, we use three dual-port RAMs in FPGA to store the pixel data in three adjacent lines, shown in fig.  2. The image data from CCD are transferred to dual-port RAM under the control of clock, address generators and read-write controller. One RAM caches a line of data. Synchronous logic is adopted in this design. Data signals, address signals and control signals are generated by the system clock and change with the clock. In order to ensure data synchronous and order, the three dual-port RAM share a set of read and write address counting circuits. And the data processed by the previous RAM transmit by the shift register to the next one. Therefore, the flow of pixel data is: the pixel data from the

Image data entry

Clock, Address gene­ rator &

Addr clk we1 re1

Reading

and Addr

writing clk

controller

we2 re2

Addr
clk we3 re4

RAM1 D flip-flop
RAM2 D flip-flop
RAM3 D flip-flop

D flip-flop D flip-flop D flip-flop

D flip-flop D flip-flop D flip-flop

3×3
Image acquisi-
tion window

Fig. 2. 3×3 processing window module.

camera→RAM1→RAM2→RAM3 thus can ensure the three RAMs producing the same column data simultaneously.
In a clock cycle, the nine output data in fig.  2 correspond to the three rows and three columns data in fig. 3. The data through D flip-flop from RAM1 are W11, W12, W13 in turn from top to bottom, the data from RAM2 are W21, W22, W23 from RAM3 are W31, W32, W33. In fig.  3, the process is neighborhood-based to the window center pixel W22, and the processing result is the output result of W22. All window-based processing in the paper are in this way.
The neighborhood operation on the two-dimensional image is like sliding of the window in the image. The center of the window will slide to the edge point of the image where doesn’t exist a complete neighborhood. Therefore, the output results of the processing window are meaningless. Typically, the edge information is relatively unimportant, so in this paper, the gray values of the edge points are assigned to be 0 without any operation.
1.1.2. 2-dimension median filter module
The median filter is a sliding-window spatial filter. It replaces the center value in the window with the median of all the pixel values in the window. As for the mean filter, the kernel is usually square but can be any shape. In this design, a single 3×3 window is adopted [10].
The scheme diagram of 3×3 median filter is shown in fig.  4, where the data W11, W33 correspond to the nine pixel points in fig.  3. After six three-input comparison circuits, the median value is the output result of the center pixel. The first-stage comparison circuit is composed of comparator 1 (com1), comparator 2 (com2) and comparator 3 (com3). The minimum data, the

W11 W21

W12 W22

W13 W23

W31

W32

W33

Fig. 3. 3×3 template corresponding to the window data.

70 “Оптический журнал”, 80, 1, 2013

W11 W12 W13 W21 W22 W23 W31 W32 W33

The first level comparison

com1

com2

com3

The second level comparison
Minimum value
The third level comparison
The second minimum value
The fourth level comparison

com4 com7 com9

com5 reg1

com6 com8 com10

Maximum value The second maximum value

The fifth level comparison

com11

reg2 com12

The sixth level comparison

com13

mid value
Fig. 4. Median filter.

middle data, and the maximum data of the three comparators are sent to com4, com5, com6 in the second stage comparator circuit respectively. The principle of the second stage comparators are the same as the first stage ones. After the second stage comparators, the min value output of com4 is the minimum value of W11, W33 (the minimum value), and the max value output of com6 is the maximum value of W11, W33 (the maximum value). The minimum and maximum values don’t participation in the lower-stage computing. There are seven data left in computing in the  third stage. The output values of comparator 7 (com7) and 8 (com8) are the second minimum value and the second maximum of W11, W33, which don’t participate in the next stage computing, too. Therefore, there are five data in  the four stages in computing after the third stage circuit. Then, after six stage comparison circuits, the output value is the median value of the median filter. In order to ensure the synchronization of the pipelining in the operation, the data registers (reg1, reg2) in the third and fifth comparator circuits are adopted to cache the data which don’t participate in computing in current stage.
1.1.3. Two-dimensional Top-hat transform for gray level image module
Top-hat transform processes images based on shapes and applies a structuring element to an in-

Open operation

Subtraction module

Input data flow

3×3
sample Window module

Erosion module

3×3
sample Window module

Erosion module

filamentous

Output data flow

minuend

cache
Fig. 5. Top-hat operation module.

put image, creating an output image of the same size [10]. According to the character of the spot, 3×3 square structuring element is adopted in the design, shown in equation 3.

b = êêëêêébbb123111

b12 b22 b32

b13 b23 b33

ûúúúùú

=

ëêêêêé111

1 1 1

111úúûùúú.

(3)

The process flow of Top-hat transform is shown in fig.  5. The source data is divided into two parts. In the first part, the data are cached as a minuend in the following subtraction module without any process. The data in the second part are implemented open operation, and the result is as the subtrahend in the following process. The minuend data cached in the first part minus the subtrahend cached in the second part and the v­ alues which are less than 0 is set to be 0. The output value is the Top-hat operation result.
Open operation is the most important step in Top-hat operation. The opening operation is an erosion-dilation process. Dilation adds pixels to the boundaries of objects in an image which the value of the output pixel is the maximum value of  all the pixels in the input pixel’s neighborhood, while erosion removes pixels on objects boundaries. The number of pixels added or removed from the objects in an image depends on the size and shape of the structuring element used to process the image. The implementation process of dilation is similar as the erosion operation. The implementation of erosion is shown in fig.  6. In the dilation operation, the corresponding operation block diagram replace the sub operation to the add operation, and the minimum comparator changes to the maximum value.

“Оптический журнал”, 80, 1, 2013

71

The entire process is parallel pipelined which can meet the real-time image processing requirements.
1.2. Adaptive threshold module
The data after Top-hat operation will enter the adaptive threshold module. As the sampling rate of the camera is fast, the noise and the background between the adjacent frame images change little, so we calculate the threshold in current frame and subtract the threshold in the next frame. The threshold is the mean value of the whole image. In calculating threshold, the sum operation are processed is in frame valid period of the camera; while the mean and the multiplication operation are in frame blank period when the frame valid signal is 0. This operation will not cause the output delay.
The output from the adaptive threshold module is the output of the whole spot extraction process.

W11 Sub b11 W12 Sub b11 W13 Sub b11 W21 Sub b11 W22 Sub b11 W23 Sub b11 W31 Sub b11 W32 Sub b11 W33 Sub b11

Minimum comparator Minimum comparator Minimum comparator

Fig. 6. Erosion module.

Minimum comparator

Open operation result

attenuator

lens

laser

EMCCD

monitor

spot extraction processor

Fig. 7. Optical path system.

2. Experimental results and analysis
2.1. The indoor experimental system
The experiments are tested in indoor adaptive optics system. The experimental optical path is shown in fig.  7. The devices include a laser, attenuator, lens, an EMCCD, spot extraction processor and a monitor. The lens and EMCCD are the real-time wavefront measurements as well as an imaging sensor. The spot extraction processor is the design in this paper which completes all the wavefront spot extraction process.
The incident light from the laser attenuates EMCCD. The output data of EMCCD is the input of spot extraction processor. The output result of the spot extraction processor is transferred to the monitor by PCI. And the result images are shown in the monitor interface. In the experiment, the laser power and the gain of EMCCD are changed to get different intensity spot. This can test the extraction efficiency of the processing platform under different conditions.
The Camera used in this system is CASCADE128+ which is 128  pixels×128  pixels of 16 bit and the read-out frequency is 12 MHz. The sampling frequency of the camera is 513fps that means the cycle is about 2000us. The entire process has been tested without loss of data. The processing delay is 60.8us which is much less than the sampling period of the camera. Therefore, the design can satisfy the real-time requirement of the adaptive optics system.
2.2. The experimental result analysis
Change the gain of the camera in different laser levels to compare the effect of the process. Adjust the gain scale value to 1500 and 2000 respectively which corresponds to EMCCD multip­ lication gain is about 5 and 7. The contrast images before and after extraction are shown in fig. 8.
It can be seen from the figure that the noise before process is strong and uneven. After the process, the noise is completely removed and the target spot is retained. The extraction effect is better when the gain is high in the certain laser power. While, in a certain gain, the extraction effect is better when the laser power is high and the gray value of the spot is significantly increased. The extraction process designed can better filter out the uneven strong noise and keep the spot signal. The shape of the spot is well

72 “Оптический журнал”, 80, 1, 2013

(а) (b) (c) (d)

Fig. 8. Spot extraction before and after contrast in a certain light. a  – the original spot image of gain calibration value 1500, b  – the original spot image of gain calibration value 2000, c  – the processed spot
image of gain calibration value 1500, d – the processed spot image of gain calibration value 2000.

centroid deviation/pixels

3,5 3
2,5 2
1,5 2

Mean centroid deviation RMS and PV compare 1 before and after the process

Mean centroid

Centroid deviation/
pixel

RMS/ pixel

Pv/pixel

Original image

2,8478 0,2263 1,0333

Optimal threshold 2,0205 0,3400 1,4762

Signal extraction

0,2397 0,1295 0,6078

1

0,5 3
0 0 10 20 30 40 50 60 70 80 90 100
Fig. 9. Centroid deviation rolling curve. 1  – original image, 2  – subtracting threshold, 3  – method of this paper.

after process in this paper is significantly lower and the up and down of the centroid deviation is smaller. This shows that our design is of better process effect when the spot is surrounded with strong uneven noise.
3. Conclusion

deserved. With the gain and the power of the laser increasing, the extraction effect gets better.
Calculate the centroid deviation of one hundred continuous images before and after the process, and compare with the optimal threshold algorithm. We use the root mean square (RMS) value and the peak value (PV) of the centroid deviation as an evaluation criterion. The contrast is shown in fig. 9 and table. It can be seen that, the accuracy of subtracting threshold is less effective due to the uneven strong noise. Though the centroid deviation is lower, but the up and down of the centroid is greater. The centroid deviation

A wavefront spot extraction system based on FPGA is designed and implemented. The algorithms are detailed analyzed and the implementations of key modules are described. We tested the extraction effect in indoor adaptive optics precision tracking system. The spot images before and after the extraction were compared, and the extraction accuracy were analyzed. The results show that the processing delay is 60.8us that is much less than the sampling period of the camera. The extraction effect is good and the uneven noise can be filter out. The spot is well extracted and the shape is well reserved. The accuracy after processed is also increased significantly.

*   *   *   *   *

REFERENCES
1. Chen Bo, Li Xinyang. Zernike mode close-loop correction of adaptive optics with the curvature sensor // Infrared and Laser Engineering. 2011. V. 3. P. 467–47.

“Оптический журнал”, 80, 1, 2013

73

2. Jiang Pengzhi, Ma Haotong, Zou Yongchao, Du Shaojun. Study of aberration correction in light path of adaptive optical system // Acta Optica Sinica. 2012. V. 31. № 12. P. 1214002–1:5.
3. Li Chaohong, Xian Hao, Jiang Wenhan, Rao Changhui. Performance analysis of field-of-view Shack-Hartmann wavefront sensor based on splitter // Appl. Phys. B. Laser and optics. 2007. V. 88. № 3. P. 367–372.
4. Li Chaohong, Xian Hao, Jiang Wenhan, Rao Changhu. Analysis of wavefront measuring method for daytime adaptive optics // Acta Physica Silica. 2007. V. 56. № 7. P. 4289–4296.
5. Xu Xiuzhen, Li Zitian, Xue Lijun. Analysis and processing of CCD noise // Infrared and Laser Engineering. 2004. V. 33. № 4. P. 343–347.
6. Zhang Yanyan, Rao Changhui, Li Mei, Ma Xiaoyu. The detection error analysis of Hartmann-Shack wavefront sensor based on electron multiplying charge-coupled devices // Acta Physica Sinica. 2010. V. 59. № 8. P. 5904–5913.
7. Jiang Wenhan, Shen Feng, Xian Hao. Detecting error of shack-hartmann wavefront sensor // Chinese Joural of Quantum Electronics. 1998. V. 15. № 2.
8. Zhou Luchun, Wang Chunhong, Li Mei, Jiang Wenhan. Real-time wave front reconstruction based on a control flow systolic array // Opto-Electronic Engineering. 2008. V. 35. № 4. P. 39–42.
9. Peng Xiaofeng, Li Mei, Rao Changhui. Design of correlating hartmann-shack wavefront processor based on absolute difference algorithm // Opto-Electronic Engineering. 2008. V. 35. № 12. P. 18–22.
10. Ruan Qiuqi. Digital image processing. Bei Jing, China: Publishing House of Electronics Industry, 2007.
74 “Оптический журнал”, 80, 1, 2013