Modulation Domain Image Processing

Abstract

The classical Fourier transform is the cornerstone of traditional linear signal and image processing. The discrete Fourier transform (DFT) and the fast Fourier transform (FFT) in particular led to profound changes during the later decades of the last century in how we analyze and process 1D and multi-dimensional signals. The Fourier transform represents a signal as an infinite superposition of stationary sinusoids each of which has constant amplitude and constant frequency. However, many important practical signals such as radar returns and seismic waves are inherently nonstationary. Hence, more complex techniques such as the windowed Fourier transform and the wavelet transform were invented to better capture nonstationary properties of these signals. In this dissertation, I studied an alternative nonstationary representation for images, the 2D AM-FM model. In contrast to the stationary nature of the classical Fourier representation, the AM-FM model represents an image as a finite sum of smoothly varying amplitudes and smoothly varying frequencies. The model has been applied successfully in image processing applications such as image segmentation, texture analysis, and target tracking. However, these applications are limited to \emph{analysis}, meaning that the computed AM and FM functions are used as features for signal processing tasks such as classification and recognition. For synthesis applications, few attempts have been made to synthesize the original image from the AM and FM components. Nevertheless, these attempts were unstable and the synthesized results contained artifacts. The main reason is that the perfect reconstruction AM-FM image model was either unavailable or unstable. Here, I constructed the first functional perfect reconstruction AM-FM image transform that paves the way for AM-FM image synthesis applications. The transform enables intuitive nonlinear image filter designs in the modulation domain. I showed that these filters provide important advantages relative to traditional linear translation invariant filters. This dissertation addresses image processing operations in the nonlinear nonstationary modulation domain. In the modulation domain, an image is modeled as a sum of nonstationary amplitude modulation (AM) functions and nonstationary frequency modulation (FM) functions. I developed a theoretical framework for high fidelity signal and image modeling in the modulation domain, constructed an invertible multi-dimensional AM-FM transform (xAMFM), and investigated practical signal processing applications of the transform. After developing the xAMFM, I investigated new image processing operations that apply directly to the transformed AM and FM functions in the modulation domain. In addition, I introduced two classes of modulation domain image filters. These filters produce perceptually motivated signal processing results that are difficult or impossible to obtain with traditional linear processing or spatial domain nonlinear approaches. Finally, I proposed three extensions of the AM-FM transform and applied them in image analysis applications. The main original contributions of this dissertation include the following.

I proposed a perfect reconstruction FM algorithm. I used a

least-squares approach to recover the phase signal from its gradient. In order to allow perfect reconstruction of the phase function, I enforced an initial condition on the reconstructed phase. The perfect reconstruction FM algorithm plays a critical role in the overall AM-FM transform.

I constructed a perfect reconstruction multi-dimensional filterbank

by modifying the classical steerable pyramid. This modified filterbank ensures a true multi-scale multi-orientation signal decomposition. Such a decomposition is required for a perceptually meaningful AM-FM image representation.

I rotated the partial Hilbert transform to alleviate rippling

artifacts in the computed AM and FM functions. This adjustment results in artifact free filtering results in the modulation domain.

I proposed the modulation domain image filtering framework. I

constructed two classes of modulation domain filters. I showed that the modulation domain filters outperform traditional linear shift invariant (LSI) filters qualitatively and quantitatively in applications such as selective orientation filtering, selective frequency filtering, and fundamental geometric image transformations.

I provided extensions of the AM-FM transform for image decomposition

problems. I illustrated that the AM-FM approach can successfully decompose an image into coherent components such as texture and structural components.

I investigated the relationship between the two prominent

AM-FM computational models, namely the partial Hilbert transform approach (pHT) and the monogenic signal. The established relationship helps unify these two AM-FM algorithms. This dissertation lays a theoretical foundation for future nonlinear modulation domain image processing applications. For the first time, one can apply modulation domain filters to images to obtain predictable results. The design of modulation domain filters is intuitive and simple, yet these filters produce superior results compared to those of pixel domain LSI filters. Moreover, this dissertation opens up other research problems. For instance, classical image applications such as image segmentation and edge detection can be re-formulated in the modulation domain setting. Modulation domain based perceptual image and video quality assessment and image compression are important future application areas for the fundamental representation results developed in this dissertation.