deep learning Archives - Page 2 of 7

deep learning

paper summary: “Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization”

arxiv: https://arxiv.org/pdf/1703.06868.pdf key points arbitrary style transfer in real time use adaptive instance normalization(AdaIN) layers which aligns the mean and variance of content features allows to control content-style trade-off, style interpolation, color/spatial control previous works optimization approach using backprop of network to minimize style loss and content loss. This can be Read more…

By admin, 1 year1 year ago

deep learning

few shot learning good articles

https://towardsdatascience.com/advances-in-few-shot-learning-a-guided-tour-36bc10a68b77 great brief summary over matching networks, prototypical networks, model-agnostic meta-learning(MAML). well, the first two topics are well explained but MAML section needs a lot more thinking to understand. Also, I think MAML is closer to the topic of meta-learning rather than few shot learning. https://arxiv.org/pdf/2008.06365.pdf “An Overview of Deep Read more…

By chadrick_author, 1 year1 year ago

deep learning

paper summary: “Aggregated Residual Transformations for Deep Neural Networks” (ResNext Paper)

key point compared to resnet, the residual blocks are upgraded to have multiple “paths” or as the paper puts it “cardinality” which can be treated as another model architecture design hyper parameter. resnext architectures that have sufficient cardinality shows improved performance tldr: use improved residual blocks compared to resnet Different Read more…

By admin, 1 year1 year ago

deep learning

cosine annealing

good simple explanation and figure: https://www.kaggle.com/residentmario/cosine-annealed-warm-restart-learning-schedulers

By admin, 1 year ago

deep learning

paper review: “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks”

arxiv: https://arxiv.org/pdf/1905.11946.pdf key point propose ‘compound scaling method’ which scales all width/depth/resolution together which is an efficient scaling method that can be applied to any existing structure introduce a new family of baseline structure called ‘EfficientNets’. The very smallest baseline structure was found by authors through NAS, and then the Read more…

By chadrick_author, 1 year1 year ago

deep learning

paper review: “Semantic Segmentation using Adversarial Networks”

https://arxiv.org/pdf/1611.08408.pdf key point use adversarial net to distinguish if input masks are from segmentation net or ground truth. this allows to add auxiliary loss, like below: where mce: multi class cross-entropy bce: binary cross entropy s(): segmentation network a(): adversarial network y: ground truth x: input s(x): segmentation network output Read more…

By admin, 1 year1 year ago

deep learning

paper review: “Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Data”

https://arxiv.org/abs/1912.07768 This work suggests that surrogate data need not be drawn from the original data distribution.This paper investigates the question of whether we can train a data-generating network that can produce synthetic data that effectively and efficiently teaches a target task to a learner propose new method to create synthetic Read more…

By chadrick_author, 1 year1 year ago

deep learning

paper review: “High-Performance Large-Scale Image Recognition Without Normalization”

arxiv: https://arxiv.org/pdf/2102.06171v1.pdf key points introduce NF nets which combines multiple ideas to avoid using batch norm to get on-par performance but along with just using a bunch of non-BN techniques, this paper introduces adaptive gradient clipping(AGC) to make it actually train well to reach comparable results matching that of using Read more…

By chadrick_author, 1 year1 year ago

deep learning

paper review: “Path Aggregation Network for Instance Segmentation”

arxiv: https://arxiv.org/abs/1803.01534

key point

this paper propose a network structure(“PANet”) with bottom up path augmentation and a few other tweak methods which results in better instance segmentation performance.

(more…)

By admin, 1 year1 year ago

deep learning

paper review: “CBAM: Convolutional Block Attention Module”

arxiv: https://arxiv.org/pdf/1807.06521.pdf

key point

trying to attach attention module to CNN. but instead of blindly attaching it which would compute a 3D attention map which is computationally expensive, this work proposes to compute spatial attention and channel attention separately which achieves a similar effect with much less parameters.

(more…)

By admin, 1 year ago