Predictive Maintenance Pipeline

This document summarizes the structure of the predictive maintenance pipeline for jet engines evaluating (simulated) thermodynamic engine data. The system processes different types of data and implements methods for denoising, classification, and evaluation. The studied problem space can be characterised as follows:

1. Multi Class: There data can be classified in several different sensor- and component faults (sFault & cFault).
2. Novel Class: The combinations of faults that are being predicted after training were not present in the training data and as such are to be considered novel.
3. Multi Label: One sample may represent several faults at a time. Hence several classes can be true at the same time.

• Fault Distinction:
  • cfault (component fault): Faults related to specific system components.
  • sfault (sensor fault): Faults arising from sensor inaccuracies or failures.

---

File Structure and Main Scripts

• denoiser.py
  Contains routines for denoising input data. It converts noisy data into a denoised format using an Autoencoder (AE) as well as a Variational Autoencoder (VAE). Both have the noisy data as input and model a relation to the clean data as output. For each class the MSE over all sampples between noisy and denoised data is calculated sample by sample. Then the pointwise residuals are plotted and compared the residuals to the nominal set to evaluate how well the noise are minimised without eliminating the fault characteristic.

• classifier.py
  Houses the classification algorithms, including the construction of neural network classifiers and evaluation methods for both known and novel data. This script also manages the distinction between component faults (cfault) and sensor faults (sfault). There are 4 algorithms / algorithm combinations: 1. Neural Network (NN) only 2. Independent Component Analysis (ICA) 3. VAE + NN 4. ICA + VAE + NN Regarding the VAE: It is training / mapping on it's own input (Input = Target). For the classification only the the VAE's Encoder's output - the Latent Space = (sigma, var) - is used as input for the NN.

• helper.py
  Provides common utilities such as data loading, key list creation, fault mapping, and plotting functionalities and more...

---

Key Concepts and Variables

User Configuration and Data Paths

• userInput[]
  A dict holding user-defined configurations and parameters necessary for running the pipeline. The values of the keys

  • 'denoiser' - denoiser algorithm (AE / VAE)
  • 'mode' - sFault or cFault
  • 'data_type' - clean, noisy and or or denoised data
  • 'model' - algorithm to be used (see above)

  are to be adjusted according to the DAE.

• directories
  Specifies the paths for data storage, logging, and saving results. This helps in organizing clean, noisy, and denoised data. This part requires to maintain the structure of the repo (e.g. in regards of the Data directory in this repo). Throughout the code paths and directories are mostly kept OS-neutral.

Data Types

• Clean Data:
  Original or pre-processed data without disturbances.

• Noisy Data:
  Data perturbed by measurement noise.

• Denoised Data:
  Data obtained after applying the denoising process.

• Nominal Data:
  Baseline data representing normal operating conditions.

• Nominal Distribution Data:
  Nominal Data that contain noise with different distributions:

  1. Normal
  2. Beta
  3. Laplace
  4. Uniform

• Mixed Data:
  Data containing different combinations of faults.

Data Loading and Key Management

• loadData()
  A function responsible for importing the dataset from .csv and organising them in a dict with the respective fault taken from the file name and used as key.

• createKeyList()
  Generates lists of keys (or identifiers) used to track, access and plot the different faults from the data dict created by loadData()

Fault Mapping and Distinction

• flag2Fault, fault2Flag, mapFault2ScenarioFlag() Every fault has a scenario flag as unique identifier. Functions that convert between fault indicators and flags:
  • mapFault2ScenarioFlag(): Associates faults with scenario flags for classification and evaluation (plotting)
  • flag2Fault: Maps a given flag to its corresponding fault.
  • fault2Flag: Converts a fault into a designated flag.

Hyperparameters and Training Dynamics

• hyperparams dict
  A dictionary storing key hyperparameters such as learning rate, number of epochs, batch size, etc. Devided in parameters specifically needed for different algorithms (NN only, ICA+NN...)

• kl_annealing
  A parameter controlling the annealing of the Kullback-Leibler divergence term during training (especially in Beta-VAE models) to balance reconstruction and regularization.

---

Classification Pipeline Processes

The pipeline is structured around a loop that performs preprocessing, training, and evaluation. The key functions involved include:

1. preProcessData_predictor()
   Prepares and cleans the data prior to training.

2. buildBetaVAE()
   Constructs a Beta-Variational Autoencoder (Beta-VAE) used for learning robust data representations.

3. buildNNClassifier()
   Builds a neural network classifier to classify the faults present in the data. The used loss funciton is binary crossentropy as this is common practise for these problems. The output is a probability for each class (confidence values != hard / final prediction)

4. trainModel()
   Orchestrates the training process, incorporating the hyperparams dictionary and applying kl_annealing for model regularization if model = VAE / Encoder

5. prepNovelData_predictor()
   Prepares novel data with combinations not seen in training for testing the predictive performance of the trained model.

6. evalNovelClass_predictor()
   Evaluates the classifier on novel classes to measure performance plotting differnt metrics also comparing across data types and algorithms.

Plotted Metrics

The following functions are used to visualize model performance:

• plotAccuracy(accuracy_dict, keys_novel_stripped, save_dir, unique)
  Plots accuracy metrics across different fault types and scenarios.

• plotAvgProbCorrect(avg_prob_correct_dict, keys_novel_stripped, save_dir, unique)
  Plots the average probability of correct classifications.

• plotPrecision(precision_dict, keys_novel_stripped, save_dir, unique)
  Visualizes precision metrics.

Data Generation

• genData()
  Generates synthetic or augmented data to simulate various operational scenarios and enrich the training datasets.

---

Bootstrapping and Robustness Enhancements

The pipeline also incorporates bootstrapping techniques to enhance model robustness:

• Monte Carlo (MC) Sampling:
  Utilized during inference, MC sampling involves running multiple forward passes through the model to generate a distribution of predictions. This approach provides a measure of uncertainty and stability in the model’s outputs.
• Test Time Augmentation (TTA):
  At inference, TTA applies various data augmentations to each input instance. Predictions are then averaged over these augmented samples, leading to more reliable and robust predictions.

---

Files / Repo / Results

cluster23/

• contains a script and version of the helper function that performs a hyperparameter grid search

experiments/

Contains the most recent results from the respective experiments of denoising and classification.

3_train_all_test_loop

• Results from denoiser.py: AE or VAE
• each contains 4 folders with data from denoising: cFault_target_set, cFault_novel_set, sFault_target_set, sFault_test_set
• each containing:
  • denoised data
  • loss
  • S_Fault - metric plots from denoising of respective data set.
  • mse_{timestamp}_{denoiser}.png - plot that summarises each denoising's mse
  • model
  • experimental data used during denoising

4_fail_prediction

• CONFIG_DS_FAULT: contains experimentation data from loop over different CONFIGurations, DataStructures and FAULTs
  • CONFIG: [NN_only, ICA+NN, VAE+NN, ICA+VAE+NN]
  • DS: [clean, noisy, denoised]
  • FAULT: sensor faults / component faults
• each experiment is contained in a {timestamp}-labeled directory
• each experiment contains:
  • metric comparisons between novel, predicted classes: accuracy, avg. positive (correct class) probability, precision
  • aggregated metrics: mean of each metric summarised over all classes for better comparisokn of each algorithm
  • directory for each algorithm_combination > datatype (clean, noisy, denpoised) > novel predictions & training loss curve
  • each novel prediction direcrtory contains:
    • predicted probability distribution for each true class
    • if provided: Monte Carlo (MC) hard predictions
    • prediction distribution (without MC)

Summary

The predictive maintenance pipeline is designed in a modular fashion with a clear separation between data preprocessing, model training, and evaluation. Leveraging dedicated scripts (denoiser.py, classifier.py, helper.py) and a suite of well-defined functions, the system processes, classifies, and evaluates data for predictive maintenance applications. Key concepts such as userInput configurations, data directories and dictionaries as well as key_lsits, fault mapping mechanisms, and visualization of key performance metrics facilitate debugging and future enhancements.