This book explains the motivation for using microphone arrays as opposed to using a single sensor for sound acquisition. The book then goes on to summarize the most useful ideas, concepts, results, and new algorithms therein. The material presented in this work includes analysis of the advantages of using microphone arrays, including dimensionality reduction to remove the redundancy while preserving the variability of the array signals using the principal component analysis (PCA). The authors also discuss benefits such as beamforming with low-rank approximations, fixed, adaptive, and robust distortionless beamforming, differential beamforming, and a new form of binaural beamforming that takes advantage of both beamforming and human binaural hearing properties to improve speech intelligibility. The book makes the microphone array signal processing theory and applications available in a complete and self-contained text. The authors attempt to explain the main ideas in a clear and rigorous way so that the reader can easily capture the potentials, opportunities, challenges, and limitations of microphone array signal processing. This book is written for those who work on the topics of microphone arrays, noise reduction, speech enhancement, speech communication, and human-machine speech interfaces.

This book introduces readers to the novel concept of variable span speech enhancement filters, and demonstrates how it can be used for effective noise reduction in various ways. Further, the book provides the accompanying Matlab code, allowing readers to easily implement the main ideas discussed. Variable span filters combine the ideas of optimal linear filters with those of subspace methods, as they involve the joint diagonalization of the correlation matrices of the desired signal and the noise. The book shows how some well-known filter designs, e.g. the minimum distortion, maximum signal-to-noise ratio, Wiener, and tradeoff filters (including their new generalizations) can be obtained using the variable span filter framework. It then illustrates how the variable span filters can be applied in various contexts, namely in single-channel STFT-based enhancement, in multichannel enhancement in both the time and STFT domains, and, lastly, in time-domain binaural enhancement. In these contexts, the properties of these filters are analyzed in terms of their noise reduction capabilities and desired signal distortion, and the analyses are validated and further explored in simulations.

This book presents and develops several important concepts of speech enhancement in a simple but rigorous way. Many of the ideas are new; not only do they shed light on this old problem but they also offer valuable tips on how to improve on some well-known conventional approaches. The book unifies all aspects of speech enhancement, from single channel, multichannel, beamforming, time domain, frequency domain and time-frequency domain, to binaural in a clear and flexible framework. It starts with an exhaustive discussion on the fundamental best (linear and nonlinear) estimators, showing how they are connected to various important measures such as the coefficient of determination, the correlation coefficient, the conditional correlation coefficient, and the signal-to-noise ratio (SNR). It then goes on to show how to exploit these measures in order to derive all kinds of noise reduction algorithms that can offer an accurate and versatile compromise between noise reduction and speech distortion.

This book focuses on the application of canonical correlation analysis (CCA) to speech enhancement using the filtering approach. The authors explain how to derive different classes of time-domain and time-frequency-domain noise reduction filters, which are optimal from the CCA perspective for both single-channel and multichannel speech enhancement. Enhancement of noisy speech has been a challenging problem for many researchers over the past few decades and remains an active research area. Typically, speech enhancement algorithms operate in the short-time Fourier transform (STFT) domain, where the clean speech spectral coefficients are estimated using a multiplicative gain function. A filtering approach, which can be performed in the time domain or in the subband domain, obtains an estimate of the clean speech sample at every time instant or time-frequency bin by applying a filtering vector to the noisy speech vector. Compared to the multiplicative gain approach, the filtering approach more naturally takes into account the correlation of the speech signal in adjacent time frames. In this study, the authors pursue the filtering approach and show how to apply CCA to the speech enhancement problem. They also address the problem of adaptive beamforming from the CCA perspective, and show that the well-known Wiener and minimum variance distortionless response (MVDR) beamformers are particular cases of a general class of CCA-based adaptive beamformers.

Recently, we proposed a completely novel and efficient way to design differential beamforming algorithms for linear microphone arrays. Thanks to this very flexible approach, any order of differential arrays can be designed. Moreover, they can be made robust against white noise amplification, which is the main inconvenience in these types of arrays. The other well-known problem with linear arrays is that electronic steering is not really feasible.  In this book, we extend all these fundamental ideas to circular microphone arrays and show that we can design small and compact differential arrays of any order that can be electronically steered in many different directions and offer a good degree of control of the white noise amplification problem, high directional gain, and frequency-independent response. We also present a number of practical examples, demonstrating that differential beamforming with circular microphone arrays is likely one of the best candidates for applications involving speech enhancement (i.e., noise reduction and dereverberation). Nearly all of the material presented is new and will be of great interest to engineers, students, and researchers working with microphone arrays and their applications in all types of telecommunications, security and surveillance contexts.

This book provides a systematic study of the fundamental theory and methods of beamforming with differential microphone arrays (DMAs), or differential beamforming in short. It begins with a brief overview of differential beamforming and some popularly used DMA beampatterns such as the dipole, cardioid, hypercardioid, and supercardioid, before providing essential background knowledge on orthogonal functions and orthogonal polynomials, which form the basis of differential beamforming. From a physical perspective, a DMA of a given order is defined as an array that measures the differential acoustic pressure field of that order; such an array has a beampattern in the form of a polynomial whose degree is equal to the DMA order. Therefore, the fundamental and core problem of differential beamforming boils down to the design of beampatterns with orthogonal polynomials. But certain constraints also have to be considered so that the resulting beamformer does not seriously amplify the sensors' self noise and the mismatches among sensors. Accordingly, the book subsequently revisits several performance criteria, which can be used to evaluate the performance of the derived differential beamformers. Next, differential beamforming is placed in a framework of optimization and linear system solving, and it is shown how different beampatterns can be designed with the help of this optimization framework. The book then presents several approaches to the design of differential beamformers with the maximum DMA order, with the control of the white noise gain, and with the control of both the frequency invariance of the beampattern and the white noise gain. Lastly, it elucidates a joint optimization method that can be used to derive differential beamformers that not only deliver nearly frequency-invariant beampatterns, but are also robust to sensors' self noise.

This book introduces readers to the novel concept of variable span speech enhancement filters, and demonstrates how it can be used for effective noise reduction in various ways. Further, the book provides the accompanying Matlab code, allowing readers to easily implement the main ideas discussed. Variable span filters combine the ideas of optimal linear filters with those of subspace methods, as they involve the joint diagonalization of the correlation matrices of the desired signal and the noise. The book shows how some well-known filter designs, e.g. the minimum distortion, maximum signal-to-noise ratio, Wiener, and tradeoff filters (including their new generalizations) can be obtained using the variable span filter framework. It then illustrates how the variable span filters can be applied in various contexts, namely in single-channel STFT-based enhancement, in multichannel enhancement in both the time and STFT domains, and, lastly, in time-domain binaural enhancement. In these contexts, the properties of these filters are analyzed in terms of their noise reduction capabilities and desired signal distortion, and the analyses are validated and further explored in simulations.

Recently, we proposed a completely novel and efficient way to design differential beamforming algorithms for linear microphone arrays. Thanks to this very flexible approach, any order of differential arrays can be designed. Moreover, they can be made robust against white noise amplification, which is the main inconvenience in these types of arrays. The other well-known problem with linear arrays is that electronic steering is not really feasible. In this book, we extend all these fundamental ideas to circular microphone arrays and show that we can design small and compact differential arrays of any order that can be electronically steered in many different directions and offer a good degree of control of the white noise amplification problem, high directional gain, and frequency-independent response. We also present a number of practical examples, demonstrating that differential beamforming with circular microphone arrays is likely one of the best candidates for applications involving speech enhancement (i.e., noise reduction and dereverberation). Nearly all of the material presented is new and will be of great interest to engineers, students, and researchers working with microphone arrays and their applications in all types of telecommunications, security and surveillance contexts.

Microphone arrays have attracted a lot of interest over the last few decades since they have the potential to solve many important problems such as noise reduction/speech enhancement, source separation, dereverberation, spatial sound recording, and source localization/tracking, to name a few. However, the design and implementation of microphone arrays with beamforming algorithms is not a trivial task when it comes to processing broadband signals such as speech. Indeed, in most sensor arrangements, the beamformer output tends to have a frequency-dependent response. One exception, perhaps, is the family of differential microphone arrays (DMAs) who have the promise to form frequency-independent responses. Moreover, they have the potential to attain high directional gains with small and compact apertures. As a result, this type of microphone arrays has drawn much research and development attention recently. This book is intended to provide a systematic study of DMAs from a signal processing perspective. The primary objective is to develop a rigorous but yet simple theory for the design, implementation, and performance analysis of DMAs. The theory includes some signal processing techniques for the design of commonly used first-order, second-order, third-order, and also the general Nth-order DMAs. For each order, particular examples are given on how to form standard directional patterns such as the dipole, cardioid, supercardioid, hypercardioid, subcardioid, and quadrupole. The study demonstrates the performance of the different order DMAs in terms of beampattern, directivity factor, white noise gain, and gain for point sources. The inherent relationship between differential processing and adaptive beamforming is discussed, which provides a better understanding of DMAs and why they can achieve high directional gain. Finally, we show how to design DMAs that can be robust against white noise amplification.

158 2. Wiener Filtering 159 3. Speech Enhancement by Short-Time Spectral Modification 3. 1 Short-Time Fourier Analysis and Synthesis 159 160 3. 2 Short-Time Wiener Filter 161 3. 3 Power Subtraction 3. 4 Magnitude Subtraction 162 3. 5 Parametric Wiener Filtering 163 164 3. 6 Review and Discussion Averaging Techniques for Envelope Estimation 169 4. 169 4. 1 Moving Average 170 4. 2 Single-Pole Recursion 170 4. 3 Two-Sided Single-Pole Recursion 4. 4 Nonlinear Data Processing 171 5. Example Implementation 172 5. 1 Subband Filter Bank Architecture 172 173 5. 2 A-Posteriori-SNR Voice Activity Detector 5. 3 Example 175 6. Conclusion 175 Part IV Microphone Arrays 10 Superdirectional Microphone Arrays 181 Gary W. Elko 1. Introduction 181 2. Differential Microphone Arrays 182 3. Array Directional Gain 192 4. Optimal Arrays for Spherically Isotropic Fields 193 4. 1 Maximum Gain for Omnidirectional Microphones 193 4. 2 Maximum Directivity Index for Differential Microphones 195 4. 3 Maximimum Front-to-Back Ratio 197 4. 4 Minimum Peak Directional Response 200 4. 5 Beamwidth 201 5. Design Examples 201 5. 1 First-Order Designs 202 5. 2 Second-Order Designs 207 5. 3 Third-Order Designs 216 5. 4 Higher-Order designs 221 6. Optimal Arrays for Cylindrically Isotropic Fields 222 6. 1 Maximum Gain for Omnidirectional Microphones 222 6. 2 Optimal Weights for Maximum Directional Gain 224 6. 3 Solution for Optimal Weights for Maximum Front-to-Back Ratio for Cylindrical Noise 225 7. Sensitivity to Microphone Mismatch and Noise 230 8.

Audio Signal Processing for Next-Generation Multimedia Communication Systems presents cutting-edge digital signal processing theory and implementation techniques for problems including speech acquisition and enhancement using microphone arrays, new adaptive filtering algorithms, multichannel acoustic echo cancellation, sound source tracking and separation, audio coding, and realistic sound stage reproduction. This book's focus is almost exclusively on the processing, transmission, and presentation of audio and acoustic signals in multimedia communications for telecollaboration where immersive acoustics will play a great role in the near future.

Microphone arrays have attracted a lot of interest over the last few decades since they have the potential to solve many important problems such as noise reduction/speech enhancement, source separation, dereverberation, spatial sound recording, and source localization/tracking, to name a few. However, the design and implementation of microphone arrays with beamforming algorithms is not a trivial task when it comes to processing broadband signals such as speech. Indeed, in most sensor arrangements, the beamformer output tends to have a frequency-dependent response. One exception, perhaps, is the family of differential microphone arrays (DMAs) who have the promise to form frequency-independent responses. Moreover, they have the potential to attain high directional gains with small and compact apertures. As a result, this type of microphone arrays has drawn much research and development attention recently. This book is intended to provide a systematic study of DMAs from a signal processing perspective. The primary objective is to develop a rigorous but yet simple theory

for the design, implementation, and performance analysis of DMAs. The theory includes some signal processing techniques for the design of commonly used first-order, second-order, third-order, and also the general "N"th-order DMAs. For each order, particular examples are given on how to form standard directional patterns such as the dipole, cardioid, supercardioid, hypercardioid, subcardioid, and quadrupole. The study demonstrates the performance of the different order DMAs in terms of beampattern, directivity factor, white noise gain, and gain for point sources. The inherent relationship between differential processing and adaptive beamforming is discussed, which provides a better understanding of DMAs and why they can achieve high directional gain. Finally, we show how to design DMAs that can be robust against white noise amplification.

Modern communication devices, such as mobile phones, teleconferencing systems, VoIP, etc., are often used in noisy and reverberant environments. Therefore, signals picked up by the microphones from telecommunication devices contain not only the desired near-end speech signal, but also interferences such as the background noise, far-end echoes produced by the loudspeaker, and reverberations of the desired source. These interferences degrade the fidelity and intelligibility of the near-end speech in human-to-human telecommunications and decrease the performance of human-to-machine interfaces (i.e., automatic speech recognition systems).

The proposed book deals with the fundamental challenges of speech processing in modern communication, including speech enhancement, interference suppression, acoustic echo cancellation, relative transfer function identification, source localization, dereverberation, and beamforming in reverberant environments.

Enhancement of speech signals is necessary whenever the source signal is corrupted by noise. In highly non-stationary noise environments, noise transients, and interferences may be extremely annoying. Acoustic echo cancellation is used to eliminate the acoustic coupling between the loudspeaker and the microphone of a communication device. Identification of the relative transfer function between sensors in response to a desired speech signal enables to derive a reference noise signal for suppressing directional or coherent noise sources. Source localization, dereverberation, and beamforming in reverberant environments further enable to increase the intelligibility of the near-end speech signal.

Single-channel hands-free teleconferencing systems are becoming popular. In order to enhance the communication quality of these systems, more and more stereophonic sound devices with two loudspeakers and two microphones are deployed. Because of the coupling between loudspeakers and microphones, there may be strong echoes, which make real-time communication very difficult. The best way we know to cancel these echoes is via a stereo acoustic echo canceller (SAEC), which can be modelled as a two-input/two-output system with real random variables. In this work, the authors recast this problem into a single-input/single-output system with complex random variables thanks to the widely linear model. From this new convenient formulation, they re-derive the most important aspects of a SAEC, including identification of the echo paths with adaptive filters, double-talk detection, and suppression.

Additive noise is ubiquitous in acoustics environments and can affect the intelligibility and quality of speech signals. Therefore, a so-called noise reduction algorithm is required to mitigate the effect of the noise that is picked up by the microphones. This work proposes a general framework in the time domain for the single and multiple microphone cases, from which it is very convenient to derive, study, and analyze all kind of optimal noise reduction filters. Not only that all known algorithms can be deduced from this approach, shedding more light on how they function, but new ones can be discovered as well.

This book focuses on a class of single-channel noise reduction methods that are performed in the frequency domain via the short-time Fourier transform (STFT). The simplicity and relative effectiveness of this class of approaches make them the dominant choice in practical systems. Even though many popular algorithms have been proposed through more than four decades of continuous research, there are a number of critical areas where our understanding and capabilities still remain quite rudimentary, especially with respect to the relationship between noise reduction and speech distortion. All existing frequency-domain algorithms, no matter how they are developed, have one feature in common: the solution is eventually expressed as a gain function applied to the STFT of the noisy signal only in the current frame. As a result, the narrowband signal-to-noise ratio (SNR) cannot be improved, and any gains achieved in noise reduction on the fullband basis come with a price to pay, which is speech distortion. In this book, we present a new perspective on the problem by exploiting the difference between speech and typical noise in circularity and interframe self-correlation, which were ignored in the past. By gathering the STFT of the microphone signal of the current frame, its complex conjugate, and the STFTs in the previous frames, we construct several new, multiple-observation signal models similar to a microphone array system: there are multiple noisy speech observations, and their speech components are correlated but not completely coherent while their noise components are presumably uncorrelated. Therefore, the multichannel Wiener filter and the minimum variance distortionless response (MVDR) filter that were usually associated with microphone arrays will be developed for single-channel noise reduction in this book. This might instigate a paradigm shift geared toward speech distortionless noise reduction techniques. Table of Contents: Introduction / Problem Formulation / Performance Measures / Linear and Widely Linear Models / Optimal Filters with Model 1 / Optimal Filters with Model 2 / Optimal Filters with Model 3 / Optimal Filters with Model 4 / Experimental Study

For the first time, a reference on the most relevant applications of adaptive filtering techniques. Top researchers in the field contributed chapters addressing applications in acoustics, speech, wireless and networking, where research is still very active and open.

In the past few years we have written and edited several books in the area of acousticandspeechsignalprocessing. Thereasonbehindthisendeavoristhat there were almost no books available in the literature when we ?rst started while there was (and still is) a real need to publish manuscripts summarizing the most useful ideas, concepts, results, and state-of-the-art algorithms in this important area of research. According to all the feedback we have received so far, we can say that we were right in doing this. Recently, several other researchers have followed us in this journey and have published interesting books with their own visions and perspectives. The idea of writing a book on Microphone Array Signal Processing comes from discussions we have had with many colleagues and friends. As a c- sequence of these discussions, we came up with the conclusion that, again, there is an urgent need for a monograph that carefully explains the theory and implementation of microphone arrays. While there are many manuscripts on antenna arrays from a narrowband perspective (narrowband signals and narrowband processing), the literature is quite scarce when it comes to s- sor arrays explained from a truly broadband perspective. Many algorithms for speech applications were simply borrowed from narrowband antenna - rays. However, a direct application of narrowband ideas to broadband speech processing may not be necessarily appropriate and can lead to many m- understandings.

Noise is everywhere and in most applications that are related to audio and speech, such as human-machine interfaces, hands-free communications, voice over IP (VoIP), hearing aids, teleconferencing/telepresence/telecollaboration systems, and so many others, the signal of interest (usually speech) that is picked up by a microphone is generally contaminated by noise. As a result, the microphone signal has to be cleaned up with digital signal processing tools before it is stored, analyzed, transmitted, or played out. This cleaning process is often called noise reduction and this topic has attracted a considerable amount of research and engineering attention for several decades. One of the objectives of this book is to present in a common framework an overview of the state of the art of noise reduction algorithms in the single-channel (one microphone) case. The focus is on the most useful approaches, i.e., filtering techniques (in different domains) and spectral enhancement methods. The other objective of Noise Reduction in Speech Processing is to derive all these well-known techniques in a rigorous way and prove many fundamental and intuitive results often taken for granted. This book is especially written for graduate students and research engineers who work on noise reduction for speech and audio applications and want to understand the subtle mechanisms behind each approach. Many new and interesting concepts are presented in this text that we hope the readers will find useful and inspiring.

A strong reference on the problem of signal and speech enhancement, describing the newest developments in this exciting field. The general emphasis is on noise reduction, because of the large number of applications that can benefit from this technology.

Modern communication devices, such as mobile phones, teleconferencing systems, VoIP, etc., are often used in noisy and reverberant environments. Therefore, signals picked up by the microphones from telecommunication devices contain not only the desired near-end speech signal, but also interferences such as the background noise, far-end echoes produced by the loudspeaker, and reverberations of the desired source. These interferences degrade the fidelity and intelligibility of the near-end speech in human-to-human telecommunications and decrease the performance of human-to-machine interfaces (i.e., automatic speech recognition systems).

The proposed book deals with the fundamental challenges of speech processing in modern communication, including speech enhancement, interference suppression, acoustic echo cancellation, relative transfer function identification, source localization, dereverberation, and beamforming in reverberant environments.

Enhancement of speech signals is necessary whenever the source signal is corrupted by noise. In highly non-stationary noise environments, noise transients, and interferences may be extremely annoying. Acoustic echo cancellation is used to eliminate the acoustic coupling between the loudspeaker and the microphone of a communication device. Identification of the relative transfer function between sensors in response to a desired speech signal enables to derive a reference noise signal for suppressing directional or coherent noise sources. Source localization, dereverberation, and beamforming in reverberant environments further enable to increase the intelligibility of the near-end speech signal.

Telecommunication systems and human-machine interfaces start employing multiple microphones and loudspeakers in order to make conversations and interactions more lifelike, hence more efficient. This development gives rise to a variety of acoustic signal processing problems under multiple-input multiple-output (MIMO) scenarios, encompassing distant speech acquisition, sound source localization and tracking, echo and noise control, source separation and speech dereverberation, and many others.

Acoustic MIMO Signal Processing is divided into two major parts - the theoretical and the practical. The authors begin by introducing an acoustic MIMO paradigm, establishing the fundamental of the field, and linking acoustic MIMO signal processing with the concepts of classical signal processing and communication theories in terms of system identification, equalization, and adaptive algorithms. In the second part of the book, a novel and penetrating analysis of aforementioned acoustic applications is carried out in the paradigm to reinforce the fundamental concepts of acoustic MIMO signal processing.

Acoustic MIMO Signal Processing is a timely and important professional reference for researchers and practitioners from universities and a wide range of industries. It is also an excellent text for graduate students who are interested in this exciting field.

A strong reference on the problem of signal and speech enhancement, describing the newest developments in this exciting field. The general emphasis is on noise reduction, because of the large number of applications that can benefit from this technology.

Audio Signal Processing for Next-Generation Multimedia Communication Systems presents cutting-edge digital signal processing theory and implementation techniques for problems including speech acquisition and enhancement using microphone arrays, new adaptive filtering algorithms, multichannel acoustic echo cancellation, sound source tracking and separation, audio coding, and realistic sound stage reproduction. This book's focus is almost exclusively on the processing, transmission, and presentation of audio and acoustic signals in multimedia communications for telecollaboration where immersive acoustics will play a great role in the near future.

The goal of this book is to provide, for the first time, a reference to the most relevant applications of adaptive filtering techniques. Top researchers in the field contributed chapters addressing their specific topic of study. The topics are limited to acoustics, speech, wireless, and networking applications where research is still very active and open. The book is roughly organized into two parts. In the first part, several applications in acoustics and speech are developed. The second part focuses on wireless and networking applications. Some chapters are tutorial in nature ,while others present new research ideas, and all have in common, the use of adaptive algorithms to solve real-world problems.