Efficient Quantization Strategies for Latent Diffusion Models
Latent Diffusion Models (LDMs) capture the dynamic evolution of latent
variables over time, blending patterns and multimodality in a generative
system. Despite the proficiency of LDM in various applications, such as
text-to-image generation, facilitated by robust text encoders and a variational
autoencoder, the critical need to deploy large generative models on edge
devices compels a search for more compact yet effective alternatives. Post
Training Quantization (PTQ), a method to compress the operational size of deep
learning models, encounters challenges when applied to LDM due to temporal and
structural complexities. This study proposes a quantization strategy that
efficiently quantize LDMs, leveraging Signal-to-Quantization-Noise Ratio (SQNR)
as a pivotal metric for evaluation. By treating the quantization discrepancy as
relative noise and identifying sensitive part(s) of a model, we propose an
efficient quantization approach encompassing both global and local strategies.
The global quantization process mitigates relative quantization noise by
initiating higher-precision quantization on sensitive blocks, while local
treatments address specific challenges in quantization-sensitive and
time-sensitive modules. The outcomes of our experiments reveal that the
implementation of both global and local treatments yields a highly efficient
and effective Post Training Quantization (PTQ) of LDMs.
6577f2903ceeb2f078f909a0
Yuewei Yang
admin_assigned
2023-12-12T08:36:27.713Z
64e61822204192b8d65f1c57
/avatars/7cc04dfc63c1c3f2012c5cb1334d064a.svg
Yuewei Yang
yy353
user
yy353
6577f2903ceeb2f078f909a1
Xiaoliang Dai
admin_assigned
2023-12-12T08:36:33.704Z
6549417b3ce45eb764faf993
/avatars/d310f475d0697f5f13b3d4141ea0ccaf.svg
Xiaoliang Dai
daixl1992
user
daixl1992
6577f2903ceeb2f078f909a2
Jialiang Wang
admin_assigned
2023-12-12T08:36:39.400Z
65300c8a5da070af779cf047
/avatars/8e24a2a1d7a8ced05a4c8e25b3d1be5a.svg
Jialiang Wang
jwhuggingf
user
jwhuggingf
6577f2903ceeb2f078f909a3
Peizhao Zhang
admin_assigned
2023-12-12T08:36:46.079Z
6433a635a5aed21dd110381e
/avatars/b1fa4fe1c8bd7989875233820fa425af.svg
Peizhao Zhang
stzpz
user
stzpz
6577f2903ceeb2f078f909a4
Hongbo Zhang
admin_assigned
2023-12-12T08:37:10.085Z
6182ac4df82814e44d033d51
/avatars/ff56e4a0654388be9b4c052237d8e3cb.svg
Hongbo Zhang
StevenZHB
user
StevenZHB
Efficient Quantization Strategies for Latent Diffusion Models
Latent Diffusion Models (LDMs) capture the dynamic evolution of latent
variables over time, blending patterns and multimodality in a generative
system. Despite the proficiency of LDM in various applications, such as
text-to-image generation, facilitated by robust text encoders and a variational
autoencoder, the critical need to deploy large generative models on edge
devices compels a search for more compact yet effective alternatives. Post
Training Quantization (PTQ), a method to compress the operational size of deep
learning models, encounters challenges when applied to LDM due to temporal and
structural complexities. This study proposes a quantization strategy that
efficiently quantize LDMs, leveraging Signal-to-Quantization-Noise Ratio (SQNR)
as a pivotal metric for evaluation. By treating the quantization discrepancy as
relative noise and identifying sensitive part(s) of a model, we propose an
efficient quantization approach encompassing both global and local strategies.
The global quantization process mitigates relative quantization noise by
initiating higher-precision quantization on sensitive blocks, while local
treatments address specific challenges in quantization-sensitive and
time-sensitive modules. The outcomes of our experiments reveal that the
implementation of both global and local treatments yields a highly efficient
and effective Post Training Quantization (PTQ) of LDMs.
A quantization strategy leveraging global and local treatments efficiently compresses Latent Diffusion Models (LDMs) for edge deployment, maintaining model performance.
Latent Diffusion Models LDMs Post Training Quantization PTQ Signal-to-Quantization-Noise Ratio SQNR quantization discrepancy quantization-sensitive time-sensitive modules