Deep k-means:深度神经网络压缩新算法-Deep k-Means: Re-Training and Parameter Sharing with Harder Cluster Assignments for Compressing Deep Convolutions

PyTorch / Tensorflow实现ICML 2018年论文 “Deep k-Means: Re-Training and Parameter Sharing with Harder Cluster Assignments for Compressing Deep Convolutions”

在我们的论文中,我们提出了一种简单而有效的压缩卷积方案,即通过对权重应用k均值聚类,通过权重共享实现压缩,仅记录K个聚类中心和权重分配索引。

然后,我们引入了一种新的频谱松弛k-均值正则化,它倾向于在重新训练期间向K个已知聚类中心分配卷积层权重。

我们还提出了一套改进的度量标准来估计CNN硬件实现的能耗,其估计结果经过验证与先前提出的从实际硬件测量推断出的能量估算工具一致。

我们最终评估了几个CNN模型在压缩比和能耗降低方面的Deep k-Means,观察到非常有价值的结果改进而且不会导致网络精度损失。

PyTorch/Tensorflow Implementation of our ICML 2018 paper “Deep k-Means: Re-Training and Parameter Sharing with Harder Cluster Assignments for Compressing Deep Convolutions”.

In our paper, we proposed a simple yet effective scheme for compressing convolutions though applying k-means clustering on the weights, compression is achieved through weight-sharing, by only recording K cluster centers and weight assignment indexes.

We then introduced a novel spectrally relaxed k-means regularization, which tends to make hard assignments of convolutional layer weights to K learned cluster centers during re-training.

We additionally propose an improved set of metrics to estimate energy consumption of CNN hardware implementations, whose estimation results are verified to be consistent with previously proposed energy estimation tool extrapolated from actual hardware measurements.

We finally evaluated Deep k-Means across several CNN models in terms of both compression ratio and energy consumption reduction, observing promising results without incurring accuracy loss.

https://github.com/Sandbox3aster/Deep-K-Means

转载请注明:《Deep k-means:深度神经网络压缩新算法-Deep k-Means: Re-Training and Parameter Sharing with Harder Cluster Assignments for Compressing Deep Convolutions

发表评论