Haiyang Wang

🚀 Introducing NSA: A Hardware-Aligned and Natively Trainable Sparse Attention mechanism for ultra-fast long-context training & inference! Core components of NSA: • Dynamic hierarchical sparse strategy • Coarse-grained token compression • Fine-grained token selection 💡 With…

TokenFormer, a new model architecture from @cvml_mpiinf and @PKU1898, scales from 124M to 1.4B parameters by treating parameters as tokens, maintaining Transformer performance with lower cost. Talk to the team @haiyang73756134 @ferjadnaeem @xyongqin @janericlenssen @fedassa here!

Our GiT got an 𝐨𝐫𝐚𝐥 𝐩𝐫𝐞𝐬𝐞𝐧𝐭𝐚𝐭𝐢𝐨𝐧 at #ECCV2024 . See you at #ECCV2024. Paper: GiT: Towards Generalist Vision Transformer through Universal Language Interface arxiv.org/pdf/2403.09394 Code: github.com/Haiyang-W/GiT (please star it if it's helpful😀)

#ICLR2024 Arrived Vienna! Happy to share our recent work 𝘁𝗼𝘄𝗮𝗿𝗱𝘀 𝗲𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝘁 𝗮𝗻𝗱 𝗲𝗳𝗳𝗲𝗰𝘁𝗶𝘃𝗲 𝗴𝗲𝗼𝗺𝗲𝘁𝗿𝗶𝗰 𝗱𝗲𝗲𝗽 𝗹𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗳𝗼𝗿 𝘀𝗰𝗶𝗲𝗻𝗰𝗲! With incredible CTL and @ask1729! May 9 10:45am-12:45am (Poster254, Halle B). Details⬇️ (1/n)

#ICLR2024 Just arrived in Vienna! Don't miss our oral presentation tomorrow afternoon in room Halle A3, focusing on 𝗚𝗡𝗡𝘀 and their 𝗲𝘅𝗽𝗿𝗲𝘀𝘀𝗶𝘃𝗲 𝗽𝗼𝘄𝗲𝗿! Also, swing by our poster session (Poster272, Halle B). See you there! 🌟

Visual Autoregressive Modeling Scalable Image Generation via Next-Scale Prediction We present Visual AutoRegressive modeling (VAR), a new generation paradigm that redefines the autoregressive learning on images as coarse-to-fine "next-scale prediction" or "next-resolution

To reduce human bias in model architecture, we propose a simple, yet effective LLM-like visual framework, called GiT, applicable for various vision tasks (e.g., VL tasks and segmentation) only with a vanilla ViT. :) Code: github.com/Haiyang-W/GiT arxiv.org/abs/2403.09394

GiT Towards Generalist Vision Transformer through Universal Language Interface This paper proposes a simple, yet effective framework, called GiT, simultaneously applicable for various vision tasks only with a vanilla ViT. Motivated by the universality of the Multi-layer