TransFusion: End-to-End Transformer Acceleration via Graph Fusion and Pipelining

Abstract

Transformer acceleration has increasingly emphasized local fusion within isolated submodules, such as multi-head attention (MHA) and softmax. However, as Transformer models continue to scale in both depth and context length, such fragmented optimizations fail to address end-to-end inefficiencies across the full encoder/decoder stack. This paper presents TransFusion, a comprehensive framework for end-to-end Transformer layers, including QKV projections, MHA, LayerNorm, and FFN, as structured Einsum Cascades, enabling precise modelling of data dependencies and execution order. TransFusion introduces DPipe, a unified graph-based scheduler that partitions the Einsum-centric directed acyclic graph (DAG) and applies latency-aware pipelining across hardware hierarchies using dynamic programming (DP). To enable scalable execution under strict memory budgets, TransFusion integrates TileSeek, a Monte Carlo Tree Search (MCTS)-based tiling search algorithm that balances buffer reuse and system constraints. Evaluated across both cloud and edge architecture, TransFusion achieves up to an average of 1.6 × speedup on cloud and 2.2 × on edge over the prior state-of-the-art, FuseMax, by jointly optimizing inter-layer data reuse, intra-layer pipelining, and operator scheduling.