arxiv: https://arxiv.org/abs/2111.15664 Key Points visual document understanding model which does OCR + downstream task in one step with a single end-to-end model outputs are generative, and formatted to be convertible to JSON, which makes this architecture highly compatible to various downstream tasks. present SynthDoG, a synthetic document image generator used in Read more…
arxiv: https://arxiv.org/abs/1910.13461 key points propose autoregressive model named BART, which is architecturally similar to standard transformer encoder + decoder Check out 5 pretraining tasks, and experiment which pretraining task is most helpful test BART performance with large scale pretraining on downstream tasks Model Architecture This work introduces BART, which is fundamentally Read more…
arxiv: https://arxiv.org/abs/2107.14795 Key points developing upon the Perceiver idea, Perceiver IO proposes a Perceiver like structure but where output size can be much larger and still keep overall complexity linear. (Checkout summary on Perceiver here) same with Perceiver, this work use latent array to save input information and run this through multiple self Read more…
arxiv: https://arxiv.org/abs/2103.14030 Key points multi scale feature extraction. Could think of as adoption of FPN idea. restrict transformer operation to within each window and not the entire feature map → allows to keep overall complexity linear instead of quadratic apply shifted window to allow inter-window interaction fuse relative position information in Read more…
arxiv: https://arxiv.org/abs/2103.03206 key points use learnable latent array with a fixed size as query in transformer module in perceiver architecture raw inputs can be long since it will be use as key and value in transformer module do cross attention to fusion input data into latent array. this allows perceiver to Read more…
from the official docs: https://code.visualstudio.com/docs/editor/variables-reference Predefined variables The following predefined variables are supported: ${workspaceFolder} – the path of the folder opened in VS Code ${workspaceFolderBasename} – the name of the folder opened in VS Code without any slashes (/) ${file} – the current opened file ${fileWorkspaceFolder} – the current opened file’s workspace folder ${relativeFile} – Read more…
I am running my wordpress in aws and recently my wordpress dashboard was giving my warnings to update php version from 7.2 to 7.4 for security. Updating php version I am currently using Ubuntu 18.04 version. This Ubuntu version doesn’t natively support php 7.4. So the following commands were required Read more…
arxiv: https://arxiv.org/pdf/1703.06868.pdf key points arbitrary style transfer in real time use adaptive instance normalization(AdaIN) layers which aligns the mean and variance of content features allows to control content-style trade-off, style interpolation, color/spatial control previous works optimization approach using backprop of network to minimize style loss and content loss. This can be Read more…
key point compared to resnet, the residual blocks are upgraded to have multiple “paths” or as the paper puts it “cardinality” which can be treated as another model architecture design hyper parameter. resnext architectures that have sufficient cardinality shows improved performance tldr: use improved residual blocks compared to resnet Different Read more…
good simple explanation and figure: https://www.kaggle.com/residentmario/cosine-annealed-warm-restart-learning-schedulers