deep learning Archives - Page 5 of 5

deep learning

How to implement ctc loss using tensorflow keras (feat. CRNN example)

Code: using tensorflow 1.14 The tk.keras.backend.ctc_batch_cost uses tensorflow.python.ops.ctc_ops.ctc_loss functions which has preprocess_collapse_repeated parameter. In some threads, it comments that this parameters should be set to True when the tf.keras.backend.ctc_batch_cost function does not seem to work, such as inconverging loss. However, my experience is that although setting this parameter to True Read more…

By chadrick_author, 2 years2 years ago

deep learning

tensorflow convolution + batchnorm + activation example

The “convolution + batchnorm + activation” layer combination is an extremely powerful set which stabilizes training a lot. Since it appears in many model architectures, here I keep a tensorflow implementation for future reference.

By chadrick_author, 2 years2 years ago

deep learning

how to manually write to tensorboard from tf.keras callback (useful trick when writing a handful of validation metrics at once)

tf.keras does support Metric classes which can evaludate metrics at each batch. However, it does have a limitation that it can only calculat on training data and it can only output only one value. This becomes a problem especially in cases such as when the user does validation evaluation and Read more…

By chadrick_author, 2 years ago

deep learning

strictly match cuda version when installing cudnn

I was naive to think that a cudnn version built for CUDA 10.1 will work with CUDA 10.0 installed in my machine. This mismatch will not produce any explicit error messages when building and training a tensorflow model. However, I notices that the loss does not reduce even after a Read more…

By admin, 2 years ago

deep learning

DenseNet paper review

paper url: https://arxiv.org/pdf/1608.06993.pdf Core Idea The core of DenseNet is using Dense blocks which is an essential of the idea behind it all. The core idea is that within a block, it contains multiple layers. All previous attempts before this paper only used the layers in sequential manner. An output Read more…

By admin, 2 years2 years ago

deep learning

tensorRT stuff

tensorRT support matrix: https://docs.nvidia.com/deeplearning/dgx/integrate-tf-trt/index.html#matrix to apply the tensorRT optimizations, it needs to call create_inference_graph function. Check here for more details on this function. the graph that is fed to create_inference_graph should be freezed. To know more on what exactly means by “freezing”, check here. for using bare tensorRT python module, Read more…

By chadrick_author, 2 years2 years ago

deep learning

implementing seq2seq RNN model in tensorflow

for tackling the many-to-many(with timedistributed) is easy to implement in keras code, it does not seem to be easy to do so in pure tensorflow. Of course, one can implement it when the sequence length is fixed for all input size. However when the sequence length among batches vary, then Read more…

By admin, 2 years2 years ago

deep learning

image dataset converted to tfrecord stats

I had 658240 image dataset. Each set has image and mask. Both are png files. Each image file was size of 224 x 224. To total size was 30G. After converting it to tfrecord, the size reduced to 27G.

By chadrick_author, 2 years2 years ago

deep learning

forcing memory release of gpu

identifying which pid is occupying the gpu can be found following this thread. However, the gpu number in the above link is not identical with the gpu number displayed with nvidia-smi. In my case, the processes that were occupying the gpu memory did not show up in nvidia-smi but it showed Read more…

By chadrick_author, 2 years ago

deep learning

Focal Loss (a.k.a RetinaNet) paper review

I have heard so much about Retinanet in the realm of object detection and recently I had the chance to read the paper. Surprisingly, the paper is titled with “focal loss” and not “RetinaNet”. The most important lesson that I learned here was that it was the “focal loss” that was Read more…

By admin, 2 years2 years ago