Chadrick's Blog

  • Posts

pytorch

pytorch

is it okay to calculate loss in cpu and backpropagate in torch?

When using torch, while it is common to run the network in gpu, I wasn’t always so sure if it was mandatory for me to calculate the loss on the same gpu as the output at all times. If I can do loss calculation in cpu, then it would help Read more…

By chadrick_author, 20 hours20 hours ago
pytorch

torchvision fakedata example

torchvision supports various datasets and one of them is fakedata dataset. I was curious what this actually generated and here is an example the result is just an random RGB noised image.

By admin, 3 months3 months ago
deep learning

cross entropy loss / focal loss implmentation in pytorch

at the moment, the code is written for torch 1.4 binary cross entropy loss currently, torch 1.6 is out there and according to the pytorch docs, the torch.max function can receive two tensors and return element-wise max values. However, in 1.4 this feature is not yet supported and that is Read more…

By chadrick_author, 6 months ago
pytorch

check cuda version build to torch package and find cudnn version used in torch

use the following python snippet to check cuda version the torch package was built against use the following python snippet to check cudnn version used by torch

By chadrick_author, 7 months7 months ago
pytorch

calculating gradient for selected tensors in pytorch

the above is an example code of showing how to calculate gradients for a few wanted tensors. In this case, I only wanted to calculate the gradient of conv2.weight so that I can later on update only this weight with the amount calculated based on the the gradient produced by Read more…

By chadrick_author, 8 months8 months ago
pytorch

avoiding full gpu memory occupation during training in pytorch

Problem While training even a small model, I found that the gpu memory occupation neary reached 100%. This seemed odd and it made me to presume that my pytorch training code was not handling gpu memory management properly. Here is a pseudo code for my pytorch training script. With this Read more…

By chadrick_author, 11 months11 months ago
pytorch

pytorch ckpt loading error due to device mismatch

Problem I have trained and saved model checkpoint which was train in GPU 1. However, when I try to load the checkpoint in GPU 0, it fails. The following is my ckpt loading code: this gives me the following error: Solution when loading ckpt data, ensure that device is mapped Read more…

By chadrick_author, 11 months ago
deep learning

applying xavier normal initialization to conv/linear layer(module) in pytorch

in tensorflow, default initialization used is glorot normal initialization which is also known as xavier normal initialization. To use the same setting in pytorch, the following practice should be done. 2d convolution module example linear module example pytorch supports other initialization functions, and one can use those initialization functions in Read more…

By chadrick_author, 11 months3 months ago
Recent Posts
  • is it okay to calculate loss in cpu and backpropagate in torch?
  • paper review: “High-Performance Large-Scale Image Recognition Without Normalization”
  • gitignore ignore config files except sample config file
  • convert subquery columns into array of json objects
  • postgres database backup and restoring
Categories
  • algorithm
  • c/c++
  • cuda
  • data science
  • deep learning
  • error fix
  • kicad
  • kotlin
  • linux
  • postgres
  • python
  • pytorch
  • tensorflow
  • Uncategorized
  • web
Recent Comments
  • Anonymous on solving “‘grub-efi-amd64-signed’ package failed to install into /target/.” error while installing ubuntu 20.04
  • Anonymous on how to install cudnn(from tar file)
  • Vaishal on solving “‘grub-efi-amd64-signed’ package failed to install into /target/.” error while installing ubuntu 20.04
  • Anonymous on paper review: Intriguing properties of neural networks (getting adversarial samples using L-BGFS method)
  • Shraddha on cv2 resize interpolation methods
Archives
  • March 2021
  • February 2021
  • January 2021
  • December 2020
  • November 2020
  • October 2020
  • September 2020
  • August 2020
  • July 2020
  • June 2020
  • May 2020
  • April 2020
  • March 2020
  • February 2020
  • January 2020
  • December 2019
  • November 2019
  • October 2019
  • September 2019
  • August 2019
  • July 2019
  • June 2019
  • May 2019
  • April 2019
  • March 2019
  • February 2019
  • January 2019
  • December 2018
  • November 2018
Meta
  • Register
  • Log in
  • Entries feed
  • Comments feed
  • WordPress.org
  • Posts
Hestia | Developed by ThemeIsle