Networks

Each case in Cases folder is unique. But, they share the same ML architectures located in ./Cases/src/networks.py.

At this moment, AI4HPC includes:

1. Convolutional Neural Network (CNN)

2. Convolutional AutoEncoder (CAE)

3. Regression Neural Networks (RNN)

4. Convolutional Defiltering Model (CDM)

5. Vision Transformers (TR)

If new ML architectures is required, networks.py should be extended.

Convolutional Neural Network (CNN)

This network is used to train MNIST dataset. The input is hand-written digits, outputs are the predictions of this digits. This image classification case is a simple and powerful benchmarking tool.

Convolutional AutoEncoder (CAE)

The reconstruction of actuated TBL relies on CAEs. CAEs are unsupervised neural network models that summarize the general properties of the input dataset in fewer parameters while learning how to reconstruct the input again after compression, namely decompression [1]. Due to their simple implementation, CAEs are widely used for reducing the dimensionality of any dataset. The principle of CAE model is:

The network can be depicted as:

Two versions of this network exist:

1. CAE, to handle 2D input fields with 2D convolutions

2. TDCAE, to handle 3D input fields with 3D convolutions

Regression Neural Networks (REG)

This network combines the powerful features for CAEs with simple Feed-Forward Network (FFN) for a regression to predict the net power savings and drag reduction of an actuated TBL case. The network can be depicted as:

Convolutional Defilterin Model (CDM)

State-of-the-art Convolutional Defiltering Model (CDM) reconstructs turbulent wall-shear stresses, which can later be used to resolve near-wall regions in low-fidelity simulations. The CDM is a generative network which is based on the Diffusion Probabilistic Model (DPM) but modified to apply a Gaussian filter with gradually increasing filter width and Perlin noise to the input instead of Gaussian noise. The CDM learns to defilter any locally given filtered input. This way, the missing (near-wall) subgrid distribution of any quantity in a Large-Eddy Simulation (LES) is supposed to be reconstructed. The CDM trains U-NET [2].

Two versions of this network exist:

1. U_Net, to handle 2D input fields with 2D convolutions

2. TDU_Net, to handle 3D input fields with 3D convolutions

The workflow of this architecture is as given below

This network also employs Physics Informed Loss Function (PILF) [3] to combine Physics Informed Neural Network (PINN) with U-NET.

Vision Transformers (TR)

This network is used for time-series analysis of a given input for future forecast. Based on the work by Wu et al. [4] with the title of “Deep Transformer Models for Time Series Forecasting: The Influenza Prevalence Case”.

References
[1] https://nyuscholars.nyu.edu/en/publications/memoires-associatives-distribuees-une-comparaison-distributed-ass
[2] https://github.com/milesial/Pytorch-UNet/blob/master/unet/
[3] https://www.sciencedirect.com/science/article/pii/S1540748920300481
[4] https://arxiv.org/abs/2001.08317