“Combining frequency and spatial domain information for fast interactive image noise removal” by Hirani and Totsuka

Scratches on old films must be removed since these are more noticeable on higher definition and digital televisions. Wires that suspend actors or cars must be carefully erased during post production of special effects shots. Both of these are time consuming tasks but can be addressed by the following image restoration process: given the locations of noisy pixels to be replaced and a prototype image, restore those noisy pixels in a natural way. We call it image noise removal and this paper describes its fast iterative algorithm. Most existing algorithms for removing image noise use either frequency domain information (e.g low pass filtering) or spatial domain information (e.g median filtering or stochastic texture generation). The few that do combine the two domains place the limitation that the image be band limited and the band limits be known. Our algorithm works in both spatial and frequency domains without placing the limitations about band limits, making it possible to fully exploit advantages from each domain. While global features and large textures are captured in frequency domain, local continuity and sharpness are maintained in spatial domain. With a judicious choice of operations and domains in which they work, our dual-domain approach can reconstruct many contiguous noisy pixels in areas with large patterns while maintaining continuity of features such as lines. In addition, the image intensity does not have to be uniform. These are significant advantages over existing algorithms. Our algorithm is based on a general framework of projection onto convex sets (POCS). Any image analysis technique that can be described as a closed convex set can be cleanly plugged into the iteration loop of our algorithm. This is another important advantage of our algorithm.

1. FERREIRA, RJ.S.G. and PINHO, A.J. “Errorless Restoration Algorithms for Band-Limited Images”, Proc. IEEE Intl. Conf. Image Proc. (ICIP), III, 157-161, 1994.
2. FERREIRA, EJ.S.G. “Interpolation and the Discrete Papoulis- Gerchberg Algorithm”, IEEE Trans. Sig. Proc., 42, No. 10, 2596- 2606, Oct 1994.
3. FIENUR J.R. “Phase Retrieval Algorithms: a Comparison”, Appl. Opt., 21, No. 15, 2758-2769, 1982.
4. HEEGER, D.J. and BERGEN, J.R. “Pyramid-Based Texture Analysis/Synthesis”, Proc. SIGGRAPH 95, 229-238, 1995.
5. MALZBENDER, T. and SPACH, S. “A Context Sensitive Texture Nib”, Communicating with Virtual Worlds, Thalmann, N.M. and D. Thalmann (eds.), Springer-Verlag Tokyo, 1993.
6. PAPOULIS, A. “A New Algorithm in Spectral Analysis and Band- Limited Extrapolation”, IEEE Trans. Cir. & Sys., 22, No. 9, 735- 742, 1975.
7. PITAS, I. and VENETSANOPOULOS, A.N. “Order Statistics in Digital Image Processing”, Proc. IEEE, 80, No. 12, Dec 1992.
8. SEZAN, M. I. and STARK, H. “Image Restoration by the Method of Convex Projections: Part 2 – Applications and Numerical Results”, IEEE Trans. Med. Imag., 1, No. 2, 95-101, 1982.
9. STROHMER, T. “On Discrete Band-Limited Signal Extrapolation”, In Mathematical Analysis, Wavelets, and Signal Processing, Ismail, M. et. al. (eds.), AMS Contemporary Mathematics, 190, 323-337, 1995. Also available from http://tyche.mat.univie.ac.at, the home page of NUHAG in University of Vienna.
10. SUN, H. and KWOK, W. “Concealment of Damaged Block Transform Coded Images Using Projections onto Convex Sets”, IEEE Trans. Image Proc., 4, No. 4, April 1995.
11. YOULA, D.C. and WEBB, H. “Image Restoration by the Method of Convex Projections: Part 1 – Theory”, IEEE Trans. Med. Imag., 1, No. 2, 81-94, 1982.