Deep Multilayer Convolution Frameworks for Data-Driven Learning of Nonlinear Dynamics in Fluid Flows

Abstract

Abundance of measurement and simulation data has led to the proliferation of machine learning tools for model-based analysis and prediction of fluid flows over the past few years. In this work we explore globally optimal multilayer convolution models such as feed forward neural-networks (FFNN) for learning and predicting dynamics from transient fluid flow data. While machine learning in general depends on data quality relative to the underlying dynamics of the system, it is important for a given data-driven learning architecture to make the most of this available information. To this end, we cast the suite of recently popular data-driven learning approaches that approximate Markovian dynamics through a linear model in a higher-dimensional feature space as a multilayer architecture similar to neural networks, but with layer-wise locally optimal convolution mappings. As a contrast, we also represent the traditional neural networks with some slight modifications as a multilayer architecture, with convolution maps optimized to minimize the global learning cost (i.e., not the cost of learning across two immediate layers). We show through examples of data-driven learning of canonical fluid flows that globally optimal FFNN-like methods owe their success to leveraging the extended learning parameter space available in multilayer models to achieve a common goal of minimizing the training cost function while incorporating nonlinear function maps between layers. On the other hand, locally optimal multilayer models also show improvement from the same factors, but behave like shallow neural networks requiring much larger hidden layers to achieve comparable learning and prediction accuracy. While locally optimal methods allow for forward-backward convolutions, the standard globally optimal FFNNs can only handle forward maps which prevent their use as Koopman approximation tools. To this end we developed novel deep learning neural network architecture, deep Koopman network which overcome this limitation of symmetry by addition of penalty network. Further, we explored the feasibility of deep autoencoder networks (DAENs) as data-driven mappings into the observable space where the dynamics of the system can be approximated as a linear time-invariant (LTI) system. The eigenmodes and the eigenvalues of the Koopman operator provide information about the structures in the data that are associated with their unique growth rate and frequency. Naturally, the relevance of these structures and eigenvalues to the real system represented by the data is tied to how closely the Markov Linear or Koopman operator-based model approximates the real dynamics, which, in turn depends on the choice of observable. Traditional approaches for non-local optimization such as those in neural networks and deep learning are gradient-based and hence, limited to convolution basis functions whose derivatives are either known or computed accurately using numerical means. To realize the full potential of this deep learning framework, these algorithms need to be extended to arbitrary choice of convolution basis. To this end, we explored the use of gradient free optimization techniques for learning using a wider choice of functions. we illustrate these ideas by learning the dynamics from snapshots of training data and predicting the temporal evolution of canonical nonlinear fluid flows including the transient limit-cycle attractor in a cylinder wake and the instability-driven dynamics of buoyant Boussinesq flow.