pytorch
creating your own pytorch scheduler
here is an example of a scheduler that I subclassed in pytorch.
deep learning
paper review: “Donut : Document Understanding Transformer without OCR”
arxiv: https://arxiv.org/abs/2111.15664 Key Points visual document understanding model which does OCR + downstream task in one step with a single end-to-end model outputs are generative, and formatted to be convertible to JSON, which makes this architecture Read more…
deep learning
paper review: “BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension”
arxiv: https://arxiv.org/abs/1910.13461 key points propose autoregressive model named BART, which is architecturally similar to standard transformer encoder + decoder Check out 5 pretraining tasks, and experiment which pretraining task is most helpful test BART performance with Read more…