SC17-015: A Short Course on Hierarchical Deep Learning Neural Network (HiDeNN) and Its Applications: Finite Elements, Isogeometric Analysis, Tensor Decomposition, and Beyond

In engineering and physical science, we routinely deal with discrete data. Depending on how much we know about the system (the system behavior may be completely, partially, or hardly understood), these data need to be curated and analyzed to obtain mechanistic insight. Numerical methods such as finite difference, finite elements, or similar tools have been widely used to deal with such discrete data[i]. However, dealing with high-resolution data (e.g., image or mesh data) for extreme-scale engineering problems is computationally challenging with traditional numerical methods. In recent times, researchers have resorted to data science techniques, especially deep learning to use the data for system characterization and analysis. However, little mechanistic insight can be obtained from pure data-driven approachesi. Hierarchical Deep Learning Neural Network (HiDeNN) and Deep-Learning Discrete Calculus (DLDC)[ii] research aim to bridge numerical techniques, mechanistic knowledge, and data science methods and propose computationally efficient tools designed for extreme-scale science and engineering problems. DLDC is also developed as a series of lectures for STEM education and frontier research, as it offers a new perspective on numerical methods and deep learning.

The term “deep learning” refers to a subset of artificial intelligence (AI) that uses universal approximation theory through a set of neurons, activation functions, and optimizable hyperparameters to approximate any linear or non-linear relationships. Calculus is a branch of mathematical science that deals with changes in a system. DLDC[iii] is a computational method that uses deep learning constructs to mimic numerical methods based on discrete calculus. The DLDC uses elements of data science (data generation and collection, feature extraction, dimension reduction, reduced order models, regression and classification, and system and design) to solve science and engineering problems[iv]. The DLDC method offers enhanced computational efficiency and accuracy compared to traditional numerical methods, especially when dealing with high-resolution experimental/computational datasets.

Let us consider designing a drone body frame with extremely high resolution (e.g., 10 billion degrees of freedom, DoFs). Most of the computing resources are devoted to solving the physical response of the system, which is excessively demanding. To solve such problems, the DLDC is extended to Hierarchical Deep Learning Neural Network (HiDeNN) and convolution hierarchical deep-learning neural network (C-HiDeNN)[v] and its reduced order method, i.e., C-HiDeNN tensor decomposition (C-HiDeNN-TD)[vi], that enables highly accurate solutions with less computational overhead. Convolution is a mathematical operation well-known in applications such as signal processing[vii] and convolutional neural networks (CNN)[viii]. C-HiDeNN builds adaptive (the convolution filter can have varying sizes and values like CNN) convolution filters that cover a domain outside of an element (a.k.a. patch domain) so that we can interpolate a larger domain with higher smoothness and completeness without increasing the global degrees of freedom like higher-order FEM. C-HiDeNN-TD[ix] is a reduced-order version of C-HiDeNN where we only use 1-dimensional convolutional filters. Based on the concept of separation of variables, the Tensor Decomposition (TD) method converts an “n”-dimensional problem into “n” 1-dimensional problems for some number of decomposition modes. Since solving a full-dimensional matrix equation takes most of the computation time for extremely large-scale problems, dividing the equation into multiple small equations can greatly reduce the computational burden. Orders of magnitude speedup and improvement in the accuracy of the solution can be obtained by C-HiDeNN-TD. Another extension of C-HiDeNN might be the convolution isogeometric analysis (C-IGA) that can recover exact geometry while maintaining higher order continuity and retaining the Kronecker delta property of the underlying shape functions.

This short course will introduce and demonstrate how to apply a) HiDeNN and C-HiDeNN, b) C-IGA, and c) C-HiDeNN-TD. In the later part of the course, Graphical Processing Unit (GPU) acceleration of DLDC[x] will be demonstrated via Google Colab with the JAX[xi] library in Python. Participants are welcome to bring their laptops, and there is no need for installation/registration during this demonstration session. The application examples will focus on using the DLDC technique for topology optimization and multiscale materials design. After finishing the course, the attendees will be able to understand and apply the DLDC methods for solving engineering problems that require very accurate solutions given high-resolution data.

Outline of the Course: