Geometry-Preserving Manifold Flows for Zero-Sample Model Compression

Abstract

The rapid expansion of deep neural networks has led to unprecedented success across various domains, but their massive parameter counts present significant challenges for deployment on resource-constrained devices. Model compression techniques offer a viable solution, yet traditional approaches heavily rely on access to the original training data, which is often unavailable due to privacy regulations, proprietary restrictions, or data scale. Zero-sample model compression addresses this limitation by synthesizing proxy data to facilitate the compression process. However, existing data-free methods frequently fail to capture the intrinsic geometric structure of the original data manifold, leading to suboptimal student model performance. This paper proposes a novel framework utilizing geometry-preserving manifold flows for zero-sample model compression. By leveraging continuous normalizing flows constrained to lower-dimensional manifolds, our approach generates highly representative synthetic data that faithfully adheres to the topological properties of the original training distribution. The proposed framework employs a dual-phase optimization strategy, combining manifold adherence with knowledge distillation. Extensive theoretical analysis and empirical evaluations demonstrate that preserving the geometric structure during data synthesis significantly enhances the knowledge transfer efficiency. Our method achieves state-of-the-art performance across multiple benchmark architectures, delivering substantial compression ratios while maintaining high task accuracy without accessing a single original data sample.