January 2022 - Chadrick's Blog

deep learning

paper review: “Donut : Document Understanding Transformer without OCR”

arxiv: https://arxiv.org/abs/2111.15664 Key Points visual document understanding model which does OCR + downstream task in one step with a single end-to-end model outputs are generative, and formatted to be convertible to JSON, which makes this architecture highly compatible to various downstream tasks. present SynthDoG, a synthetic document image generator used in Read more…

By admin, 2 years ago

deep learning

paper review: “BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension”

arxiv: https://arxiv.org/abs/1910.13461 key points propose autoregressive model named BART, which is architecturally similar to standard transformer encoder + decoder Check out 5 pretraining tasks, and experiment which pretraining task is most helpful test BART performance with large scale pretraining on downstream tasks Model Architecture This work introduces BART, which is fundamentally Read more…

By admin, 2 years2 years ago