tensorflow

Loading multiple tensorflow models at once in tf1.x causes problems because the operations in each model may have same name, and thus cause collision. While this can be avoided by manually giving unique names to tensors when developing a model, this is a cumbersome strategy. Another approach would be to Read more…

tensorflow manual cross entropy loss calculation

In case you need a raw cross entropy loss calculation done not with logits but with the final value, and no reductions on the output whatsoever, here’s a code snippet for it.

keras.load_weight and keras.load_model gives different results

There can be several ways to load a model from ckpt file and run inference. Method1 Build model instance from source, just like in preparing for training from scratch. Method2 When the ckpt file is a bundle of model architecture and weights, then simply use load_model function. Method3 In case Read more…

How to implement ctc loss using tensorflow keras (feat. CRNN example)

Code: using tensorflow 1.14 The tk.keras.backend.ctc_batch_cost uses tensorflow.python.ops.ctc_ops.ctc_loss functions which has preprocess_collapse_repeated parameter. In some threads, it comments that this parameters should be set to True when the tf.keras.backend.ctc_batch_cost function does not seem to work, such as inconverging loss. However, my experience is that although setting this parameter to True Read more…

tensorflow convolution + batchnorm + activation example

The “convolution + batchnorm + activation” layer combination is an extremely powerful set which stabilizes training a lot. Since it appears in many model architectures, here I keep a tensorflow implementation for future reference.

how to manually write to tensorboard from tf.keras callback (useful trick when writing a handful of validation metrics at once)

tf.keras does support Metric classes which can evaludate metrics at each batch. However, it does have a limitation that it can only calculat on training data and it can only output only one value. This becomes a problem especially in cases such as when the user does validation evaluation and Read more…

tf.where examples and its behavior

Before going into tf.where example, a proper way to implement equal comparison in tensorflow will be demonstrated since equal comparison will be used as the condition of the tf.where. Equal Comparison in Tensorflow output: to fix this, do not use python native operands such as == in the above. instead Read more…

strictly match cuda version when installing cudnn

I was naive to think that a cudnn version built for CUDA 10.1 will work with CUDA 10.0 installed in my machine. This mismatch will not produce any explicit error messages when building and training a tensorflow model. However, I notices that the loss does not reduce even after a Read more…