paper review: “BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension”

arxiv: https://arxiv.org/abs/1910.13461 key points propose autoregressive model named BART, which is architecturally similar to standard transformer encoder + decoder Check out 5 pretraining tasks, and experiment which pretraining task is most helpful test BART performance with large scale pretraining on downstream Read more…

paper summary: “BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documents”

arxiv: https://arxiv.org/abs/2108.04539 key points use text and spatial information. doesn’t utilize image feature a better spatial information encoding method compared to LayoutLM propose new pretraining task: Area Masked Language Model spatial information encoding method For each text box, get four Read more…