Modulation Domain Image Processing

Abstract

The classical Fourier transform is the cornerstone of traditional linear

signal and image processing. The discrete Fourier transform (DFT) and the

fast Fourier transform (FFT) in particular led to

profound changes during the later decades of the last century in how

we analyze and process 1D and multi-dimensional signals.

The Fourier transform represents a signal as an infinite superposition

of stationary sinusoids each of which has constant amplitude and constant

frequency. However, many important practical signals such as radar returns

and seismic waves are inherently nonstationary. Hence, more complex

techniques such as the windowed Fourier transform and the wavelet transform

were invented to better capture nonstationary properties of these signals.

In this dissertation, I studied an alternative nonstationary representation

for images, the 2D AM-FM model. In contrast to the

stationary nature of the classical Fourier representation, the AM-FM model

represents an image as a finite sum of smoothly varying amplitudes

and smoothly varying frequencies. The model has been applied successfully

in image processing applications such as image segmentation, texture analysis,

and target tracking. However, these applications are limited

to \emph{analysis}, meaning that the computed AM and FM functions

are used as features for signal processing tasks such as classification

and recognition. For synthesis applications, few attempts have been made

to synthesize the original image from the AM and FM components. Nevertheless,

these attempts were unstable and the synthesized results contained artifacts.

The main reason is that the perfect reconstruction AM-FM image model was

either unavailable or unstable. Here, I constructed the first functional

perfect reconstruction AM-FM image transform that paves the way for AM-FM

image synthesis applications. The transform enables intuitive nonlinear

image filter designs in the modulation domain. I showed that these filters

provide important advantages relative to traditional linear translation invariant filters.

This dissertation addresses image processing operations in the nonlinear

nonstationary modulation domain. In the modulation domain, an image is modeled

as a sum of nonstationary amplitude modulation (AM) functions and

nonstationary frequency modulation (FM) functions. I developed

a theoretical framework for high fidelity signal and image modeling in the

modulation domain, constructed an invertible multi-dimensional AM-FM

transform (xAMFM), and investigated practical signal processing applications

of the transform. After developing the xAMFM, I investigated new image

processing operations that apply directly to the transformed AM and FM

functions in the modulation domain. In addition, I introduced two

classes of modulation domain image filters. These filters produce

perceptually motivated signal processing results that are difficult or

impossible to obtain with traditional linear processing or spatial domain

nonlinear approaches. Finally, I proposed three extensions of the AM-FM

transform and applied them in image analysis applications.

The main original contributions of this dissertation include the following.

- I proposed a perfect reconstruction FM algorithm. I used a

least-squares approach to recover the phase signal from its

gradient. In order to allow perfect reconstruction of the phase function, I

enforced an initial condition on the reconstructed phase. The perfect

reconstruction FM algorithm plays a critical role in the

overall AM-FM transform.

- I constructed a perfect reconstruction multi-dimensional filterbank

by modifying the classical steerable pyramid. This modified filterbank

ensures a true multi-scale multi-orientation signal decomposition. Such a

decomposition is required for a perceptually meaningful AM-FM image

representation.

- I rotated the partial Hilbert transform to alleviate rippling

artifacts in the computed AM and FM functions. This adjustment results in

artifact free filtering results in the modulation domain.

- I proposed the modulation domain image filtering framework. I

constructed two classes of modulation domain filters. I showed that the

modulation domain filters outperform traditional linear shift

invariant (LSI) filters qualitatively and quantitatively in applications

such as selective orientation filtering, selective frequency filtering,

and fundamental geometric image transformations.

- I provided extensions of the AM-FM transform for image decomposition

problems. I illustrated that the AM-FM approach can successfully

decompose an image into coherent components such as texture

and structural components.

- I investigated the relationship between the two prominent

AM-FM computational models, namely the partial Hilbert transform

approach (pHT) and the monogenic signal. The established relationship

helps unify these two AM-FM algorithms.

This dissertation lays a theoretical foundation for future nonlinear

modulation domain image processing applications. For the first time, one

can apply modulation domain filters to images to obtain predictable

results. The design of modulation domain filters is intuitive and simple,

yet these filters produce superior results compared to those of pixel

domain LSI filters. Moreover, this dissertation opens up other research problems.

For instance, classical image applications such as image segmentation and

edge detection can be re-formulated in the modulation domain setting.

Modulation domain based perceptual image and video quality assessment and

image compression are important future application areas for the fundamental

representation results developed in this dissertation.