at the moment, the code is written for torch 1.4 binary cross entropy loss currently, torch 1.6 is out there and according to the pytorch docs, the torch.max function can receive two tensors and return element-wise max values. However, in 1.4 this feature is not yet supported and that is Read more…
use the following python snippet to check cuda version the torch package was built against use the following python snippet to check cudnn version used by torch
the above is an example code of showing how to calculate gradients for a few wanted tensors. In this case, I only wanted to calculate the gradient of conv2.weight so that I can later on update only this weight with the amount calculated based on the the gradient produced by Read more…
Problem While training even a small model, I found that the gpu memory occupation neary reached 100%. This seemed odd and it made me to presume that my pytorch training code was not handling gpu memory management properly. Here is a pseudo code for my pytorch training script. With this Read more…
Problem I have trained and saved model checkpoint which was train in GPU 1. However, when I try to load the checkpoint in GPU 0, it fails. The following is my ckpt loading code: this gives me the following error: Solution when loading ckpt data, ensure that device is mapped Read more…
in tensorflow, default initialization used is glorot normal initialization which is also known as xavier normal initialization. To use the same setting in pytorch, the following practice should be done. 2d convolution module example linear module example