IEEE IMAGE PROCESSING DEPARTMENT

Novel approach for limited-angle problems in EM

Marc Vil Oliva

based on CS
(mmvo@kth.se)

and HH Muhammed
[8].

(Hamed.Muhammed@sth.kth.se)

AbstractRecent work in thea eld of CS (Compressed Sensing) [1, 2, 3] has suggested great variety of have new been possibilities in the world of image reconstruction. We focused on a to novel approach using this kind of algorithms based on CS solve problems related to limited-angle data (e.g, computed tomography or electron microscope) or when we onlyhas dispose from few projections of the theRobbins-Monro data. This approach been based on a variation of stochastic approximation procedure developed byadaptive Egaziarian in [4] with regularization enabled by a spatially lter. The idea, explained in [4], consists in exciting an algorithm bythe injection of random noise in the unobserved portion of image spectrum and a spatially adaptive image denoising lter, working in the image domain, isunobserved used to attenuate the noise and reveal new features of the part of the image spectrum. datasets We applied this procedurearto dierent EM 3-dimensional with limited-angle tifacts (e.g, aof missing wedgeand or cone), in order to study the performance the method its viability to solve them. Index TermsCS, Compressed Sensing, problems. limited-angle data, electron microscope, EM, limited-angle
1
I.

norm which is exploited to enable the assumed sparsity Therefore, it results in parametric modeling of the solution and in problems that are commonly solved by mathematical algorithms. In [4] it is proposed to replace the traditional parametric modeling used in CS by a non-parametric one. The non-parametric modeling is implemented by the use of a spatially adaptive lter. The regularization imposed by the l0 or l1 norms is essentially only a tool to design of some non-linear ltering. This implicit regularization is replaced by explicit ltering, exploiting a spatially adaptive lter which is sensitive to the features and details of the image. If this lter is properly designed, there is a big chance to achieve better results than by the common method based on formulation of imaging. In imaging, the regularization with global sparsity penalties (e.g,

lp

norms in some domain) often

results in an inecient ltering. It is known that a higher quality can be achieved when regularization criteria are local and adaptive.

Introduction

view.

2 N EM, CT or other X-ray techniques a very common

The applied method to reconstruct the dead zones Via Recursive in a Spatially recursive Adaptive algorithm Filtering, based on [4],

3 is

carried out by Compressed Sensing Image Reconstruction that consists spatially

is missing or corrupted from a certain angle, and thus we can not trust in the results. problems are due to various technical and fundamental limitations on the minimum and maximum attainable tilt angles of the instrumentation that we use to acquire the data. Therefore, the data is conned to a limited angular range that is usually not enough to obtain a good representation of the model which leads to the situation that we are not able to read the data in a proper way.

adaptive image denoising lter.

Each iteration provides

to a block-matching and a spatially adaptive 3D ltering algorithm, BM3D, with data by the injection of random noise in the unobserved portion of the spectrum. To carry out the block-matching and the ltering block, we have used the lter implemented in [5] with some modications . This lter which is working in the image domain, attenuates the noise and reveals new details and features of the image that we are reconstructing. The algorithm is ruled by the following recursive system:

datasets which have some limited-angle distorsions (in form of a missing wedge, cone, etc.) tioned method in [4]. applying the menthe results In this paper, we will present and

HE goal

of

our

work

is from

to

estimate Electron

this

miss-

ing/corrupted

data

Microscope

y 2 =

y 2 = 0, y 2
(k )

(0)

=y 2

(k1)

y 2

(k1)

(1 S ). ((1 (y1 + y 2

(k1)

)))

explain which technique we have applied,

obtained and the conclusions extracted from the results.

Definition 1:

is a MatLab operator used as a

II.

Applied Method

point-wise multiplication between matrices.

In publications [1, 2, 3], it is shown that under CS assumptions, stable reconstruction of unknown data is possible and that in some cases the reconstruction can be exact. These techniques typically rely on convex optimization with a penalty expressed by the
1 2

l0

or

l1

Computerized Tomography

Electron Microscope

corrupted or missing data The wiener ltering is removed and Haar wavelet is used as the 3D transform. Also some parameters of the HT block were changed to speed up the reconstruction. N 1 = 4, N 2 = 12, N s = 31, tau_match = 1800.
3 4

IEEE IMAGE PROCESSING DEPARTMENT

Parameters:

III. The Data

y 2 estimation of unknown data in F ourier domain y1 known data in F ourier domain speed step of the algorithm (1 S ) mask to select the region of the unknown data F ast F ourier T ransf orm, Rn , C n , n = 2 f iltering block 1 InverseF astF ourierT ransf orm, Rn , C n , n = 2 k Gaussian noise

We used three diferent datasets: 1. It consists of a 3D articial crystallography of size 100 100 100 pixels in .mrc format, [9]. It was created by a simulator software, [16], to evaluate the performance of TEM5 reconstruction algorithms. In this case, we forced a missing wedge and a missing cone in the Fourier domain, Figure 3, to apply the procedure and study the quality of the reconstruction.
Hansandrey crystallography:

We divide the images as:

That procedure can be applied to reconstruct 2dimensional signals (e.g, Figure 1) and achieve a perfect reconstruction but we wanted to apply the method to 3-dimensional EM models (e.g, Figure 2). Therefore, we decided to split the 3D problem into several 2D problems and solve them separately. For example, in the case presented in Figure 2, if we took slices in the xy-plane (Fourier domain) we will have the same problem as in Figure 1, repeated several times.

Figure 3.

Hansandrey a.Missing wedge case b.Missing cone case

2.

It consists of a 3D model of a viral DNA gatekeeper, of size 100 100 100 pixels in .mrc format, obtained by cryoelectron microscopy technique. For this case, we only forced; like in the case above, a missing cone in the data to be able to apply the reconstruction method and evaluate its performance lling up the missing cone. This model consists in the acquisition of a protein in a TEM which presents a limited-angle problem in the obtained model. We could not obtain projections after tilting 45o degrees the specimen. Therefore, the 3D model has in the Fourier domain a missing/corrupted cone of 90o as in Figure 4. The size of this model is 80 80 80 pixels in .mrc format.
Philip's crystallography:

Viral DNA gatekeeper:

Figure 1.

2D limited-angle example a.Image domain b.Fourier do-

main

3.

Figure 4.

90o missing/corrupted cone present in the data

Figure 2.

3D limited-angle example a.Image domain b.Fourier do5

main

IEEE IMAGE PROCESSING DEPARTMENT

IV.

Experiments and Results

In

order

to

parameter dened as:

P SN R = 10log10
Parameters:

2 M AXI )2 ) mean((I I

Figure

5.

2 M AXI

maximum value of the 3D model

b.Reconstruction

Hansandrey reconstruction: a.Original model wedge case c.Original model d.Reconstruction

cone (xz) case

I original 3D model reconstruction of the 3D model I

The results obtained are shown in the next table

7 8:

XY-plane worst case to reconstruct a missing cone

artifact

Data Hansandrey DNA Philip's

Wedge

Cone (xy)

Cone (xz)

47.58dB
-

36.46dB
-

46.21dB 45.86dB
-

For the viral DNA gatekeeper, we obtained a good reconstruction as we can see with the PSNR value, to assure that we are right we are going to compare the original and the reconstructed models.

TABLE I
Results

As we can observe in the results table, we obtained good reconstruction values for Hansandrey wedge case and cone (xz-plane) case. We can visually evaluate the quality of those reconstructions in Figure 5 . Regarding the missing cone (xy-plane) case results, we attribute this poor PSNR value to the large dead zones present in the slices at the beginning and end of the model, Figure 6. The algorithm is not able to ll such kind of missing zones like in Figure 6, thus we do not obtain a good PSNR for this specic case. The algorithm is not able to give a good approximation at the beginning and at the end of the model. We only obtain a good representation of the model when we are near the middle of it, because the size of the dead zones has decreased. If we compare the original model with the recon-

Figure 7. Viral DNA gatekeeper a.Original model b.Modified model c.Reconstructed model

struction, it is fairly impossible to distinguish between both models. We can arm that we obtained a good reconstruction in terms of PSNR and visually.

For the latter case, Philip's crystallography, we only show the previous model and its reconstruction proposal
6 Peak Signal-to-Noise Ratio 7 CONE (XY) means that we took xy-slices from the 3D model 8 CONE (XZ) means that we took xz-slices from the 3D model 9 Visualized by Chimera

because, as we already mentioned, we do not have the original model, therefore the only way to assess the result is visually.

IEEE IMAGE PROCESSING DEPARTMENT

we commented in the subsections above, only half or even less of the spectrum is necessary to obtain a good reconstruction. Therefore, it could be a great advance to nd a method that could ll the necessary frequencies of the spectrum and then apply the algorithm to obtain the nal model. If it is possible to carry out this idea, a signicant dose and exposition time reduction can be achieved. Another project could be to apply this method to other kind of models like CT images, MRI, astronomy, b.Reconstruction proposal geophysical exploration or other type of electron microscope models. It would also be necessary to speed up the Visually we can not detect very big changes between algorithm, since a complete reconstruction of Hansandrey them but the PSNR value informs that the models above model takes about 32 hours. dier by 8% (P SN R = 21dB ).
Figure 8.
Philip's crystallography:

a.Previous

model

This paper would not have been possible without the The purpose of our work was to apply a new method to support and help of Dr. HH Muhammed and Dr. Philip solve limited-angle problems (e.g, missing cone eect or Koeck who supplied two of the datasets (Hansandrey and missing wedge eect) in electron microscope data. After Philip's crystallography). running some experiments and evaluating the results, we References can assure that this procedure is applicable to this type of problems and achieve good reconstruction results. If we [1] Donoho, D.L.,  Compressed sensing , IEEE Trans. Inf. Theory, vol. 52, no. 4, pp. 1289-1306, April 2006. compare the PSNR values obtained in this paper with the [2] Tsaig, Y., and D.L. Donoho,  Extensions of compressed sens10 commonly used techniques like POCS ([12], [13], [14]), ing , Signal Process., vol. 86, no. 3, pp. 549-571, March 2006. we can see that this procedure is a great breakthrough. [3] E. Candes, J. Romberg, and T. Tao,  Robust uncertainty principles: Exact signal reconstruction from highly incomplete freWhen data is limited to [45o , 45o ] POCS oers an error quency information, IEEE Trans. Info. Th., vol. 52(2), pp. 489 rate around 40%, meanwhile we obtain an error rate lower 509, February 2006. than 1% (P SN R 40dB ) for Hansandrey and viral DNA [4] K. Egiazarian, A. Foi, and V. Katkovnik,  Compressed sensing image reconstruction via recursive spatially adapgatekeeper cases. tive ltering, in IEEE Intl. Conf. Image Proc., 2007. This implementation is able to reconstruct spectra with half of the data missing, in form of a wedge, cone or other kind of artifacts. We can also reconstruct a model from multiple projections of the spectrum. We have to comment that the algorithm performance depends on great measure on the spectrum that we are working on. Thus, it is possible to reconstruct the same missing wedge or cone in dierent datasets and obtain totally dierent results. It is possible that a great part of the energy was in that missing zone, therefore it is much more dicult to reconstruct the model in that scenario due to the algorithm has not enough prior information. A study is needed to determine which spectra are valid or not to apply this method when they present a limited-angle artifact. Then we will be able to say if our reconstruction has a great similarity with the original model or not (e.g, we would be able to assure if Philip's crystallography is valid or not). One of the most excitement works that could emerge from this paper, is the idea to develop a new technique to acquire the data from the EM achieving a large radiation dose reduction applied to the specimen. Indeed, like
10 Projection Onto Convex Sets
http://www.cs.tut./~comsens [5] Dabov, K., A. Foi, V. Katkovnik, and K. Egiazarian,  Image denoising by sparse 3D transform-domain collaborative ltering , IEEE Trans. Image Process., 2007 (in press) [6] Chen G-H, Tang J. and Leng S. Prior image constrained compressed sensing (PICCS): a method to accurately reconstruct dynamic CT images from highly undersampled projection data sets Med. Phys. 2008 [7] Emil Y Sidky and Xiaochuan Pan Image reconstruction in circular cone-beam computed tomography by constrained, totalvariation minimization Phys. Med. Biol. 2008 [8] Donoho, D.L., and M. Elad,  Maximal sparsity representation via l1 minimization , Proc. Nat. Aca. Sci., vol. 100, pp. 21972202, 2003. [9] MRC specication: http://ami.scripps.edu/software/mrctools/mrc_specication.php [10] Igor Carron Compressed Sensing Sensing blog blog http://sites.google.com/site/igorcarron2/cs [11] Nuit-Blanche Compressed http://nuitblanche.blogspot.com/search/label/CS [12] H. Peng and H. Stark, J. Opt. Soc. Am. A 6 (1989), p. 844. [13] M.Ibrahim Sezan, An overview of convex projections theory and its application to image recovery problems, Ultramicroscopy, Volume 40, Issue 1, January 1992, Pages 55-67, ISSN 0304-3991, DOI: 10.1016/0304-3991(92)90234-B. [14] M. I. Sezan and H. Stark,  Tomographic Image Reconstruction from Incomplete View Data by Convex Projections and Direct Fourier Inversion, IEEE Transactions on Medical Imaging, vol. MI-3, no. 2, pp. 9198, June 1984. [15] TV minimization demo: http://www.ricam.oeaw.ac.at/people/page/fornasier/ [16] H. Rullgard , B. Danelholt, L.-G. fverstedt, S. Masich and O. ktem. Simulation of transmission electron microscope images of biological specimens Journal of Microscopy, 2011

V.

Conclusions and Future Work

VI.

Acknowledgements