deep learning
good summary on normalization methods
link: https://towardsdatascience.com/different-normalization-layers-in-deep-learning-1a7214ff71d6 Batch Normalization Weight Normalization Layer Normalization Group Normalization Weight Standarization Recently, Siyun Qiao et al. introduced Weight Standardization in their paper “Micro-Batch Training with Batch-Channel Normalization and Weight Standardization” and found that group normalization when mixed with weight standardization, could outperform or perform equally well as BN even Read more…