pytorch
creating your own pytorch scheduler
here is an example of a scheduler that I subclassed in pytorch.
deep learning
paper review: “Donut : Document Understanding Transformer without OCR”
arxiv: https://arxiv.org/abs/2111.15664 Key Points visual document understanding model which does OCR + downstream task in one step with a single end-to-end model outputs are generative, and formatted to be convertible to JSON, which makes this architecture highly compatible to various downstream Read more…
deep learning
paper review: “BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension”
arxiv: https://arxiv.org/abs/1910.13461 key points propose autoregressive model named BART, which is architecturally similar to standard transformer encoder + decoder Check out 5 pretraining tasks, and experiment which pretraining task is most helpful test BART performance with large scale pretraining on downstream Read more…
deep learning
paper summary: “DocFormer: End-to-End Transformer for Document Understanding”
arxiv: https://arxiv.org/abs/2106.11539 this work proposes a backbone for visual document understanding domain. It uses text, visual, spatial features. Key points use text, visual, spatial features at each encoding layer, keep feeding in visual and spatial features on the input side. Read more…
deep learning
paper review: “LayoutLMV2: Multi-Modal Pre-training for Visually-Rich Document Understanding”
arxiv: https://arxiv.org/abs/2012.14740 LayoutLMv2 is a work improved from LayoutLM. This line of work focues on creating a model for document understanding. The main task of this genre is to extract useful information when a document is given. In this case, Read more…
error fix
error fix: pycuda “ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21′ not found”
Background I installed pycuda and was trying to run a file and error like this occured Solution After googling, I found this post saying that this was due to lower version of libstdc++.so.6 originally my /lib64/libstdc++.so.6 was pointing to /lib64/libstdc++.so.6.0.19. Read more…
error fix
error fix: pycuda “SystemError: initialization of _driver raised unreported exception”
Background I installed pycuda with pip install pycuda==2020.01 and it installed without any errors. However when running the code, I got the following error. appartenly importing pycuda itself was causing error. Fix I removed the install pycuda. downloaded pycuda source Read more…
error fix
how to handle “no disk space” while installing cuda toolkit
Background I downloaded another CUDA toolkit(11.4) installation .sh and executed it to install it. However, while installing it was stopped due to no disk space. The default install location is /usr/local/cuda-11.4 and unfortunately I only had about 5GB left under Read more…
pytorch
error fix: onnxruntime “Type Error: Type ‘tensor(int64)’ of input parameter of operator(Min) in node is invalid’
Background I was trying to convert pytorch bert-like model from torch to onnx and see if I can run it in onnxruntime. Here’s the environment I used python: 3.9 torch: 1.9.0 onnx: 1.10.2 onnxruntime-gpu: 1.9.0 I converted by pytorch model Read more…