Loading multiple tensorflow models at once in tf1.x causes problems because the operations in each model may have same name, and thus cause collision. While this can be avoided by manually giving unique names to tensors when developing a model, this is a cumbersome strategy. Another approach would be to Read more…
In case you need a raw cross entropy loss calculation done not with logits but with the final value, and no reductions on the output whatsoever, here’s a code snippet for it.
There can be several ways to load a model from ckpt file and run inference. Method1 Build model instance from source, just like in preparing for training from scratch. Method2 When the ckpt file is a bundle of model architecture and weights, then simply use load_model function. Method3 In case Read more…
put this code at the beginning before building/loading the model.
Code: using tensorflow 1.14 The tk.keras.backend.ctc_batch_cost uses tensorflow.python.ops.ctc_ops.ctc_loss functions which has preprocess_collapse_repeated parameter. In some threads, it comments that this parameters should be set to True when the tf.keras.backend.ctc_batch_cost function does not seem to work, such as inconverging loss. However, my experience is that although setting this parameter to True Read more…
The “convolution + batchnorm + activation” layer combination is an extremely powerful set which stabilizes training a lot. Since it appears in many model architectures, here I keep a tensorflow implementation for future reference.
tf.keras does support Metric classes which can evaludate metrics at each batch. However, it does have a limitation that it can only calculat on training data and it can only output only one value. This becomes a problem especially in cases such as when the user does validation evaluation and Read more…
Before going into tf.where example, a proper way to implement equal comparison in tensorflow will be demonstrated since equal comparison will be used as the condition of the tf.where. Equal Comparison in Tensorflow output: to fix this, do not use python native operands such as == in the above. instead Read more…
I was naive to think that a cudnn version built for CUDA 10.1 will work with CUDA 10.0 installed in my machine. This mismatch will not produce any explicit error messages when building and training a tensorflow model. However, I notices that the loss does not reduce even after a Read more…