Rate-distortion optimization guided autoencoder for isometric embedding in Euclidean latent space

Keizo Kato, Jing Zhou, Tomotake Sasaki, Akira Nakagawa
Proceedings of the 37th International Conference on Machine Learning, PMLR 119:5166-5176, 2020.

Abstract

To analyze high-dimensional and complex data in the real world, deep generative models, such as variational autoencoder (VAE) embed data in a low-dimensional space (latent space) and learn a probabilistic model in the latent space. However, they struggle to accurately reproduce the probability distribution function (PDF) in the input space from that in the latent space. If the embedding were isometric, this issue can be solved, because the relation of PDFs can become tractable. To achieve isometric property, we propose Rate-Distortion Optimization guided autoencoder inspired by orthonormal transform coding. We show our method has the following properties: (i) the Jacobian matrix between the input space and a Euclidean latent space forms a constantly-scaled orthonormal system and enables isometric data embedding; (ii) the relation of PDFs in both spaces can become tractable one such as proportional relation. Furthermore, our method outperforms state-of-the-art methods in unsupervised anomaly detection with four public datasets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v119-kato20a, title = {Rate-distortion optimization guided autoencoder for isometric embedding in {E}uclidean latent space}, author = {Kato, Keizo and Zhou, Jing and Sasaki, Tomotake and Nakagawa, Akira}, booktitle = {Proceedings of the 37th International Conference on Machine Learning}, pages = {5166--5176}, year = {2020}, editor = {III, Hal Daumé and Singh, Aarti}, volume = {119}, series = {Proceedings of Machine Learning Research}, month = {13--18 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v119/kato20a/kato20a.pdf}, url = {https://proceedings.mlr.press/v119/kato20a.html}, abstract = {To analyze high-dimensional and complex data in the real world, deep generative models, such as variational autoencoder (VAE) embed data in a low-dimensional space (latent space) and learn a probabilistic model in the latent space. However, they struggle to accurately reproduce the probability distribution function (PDF) in the input space from that in the latent space. If the embedding were isometric, this issue can be solved, because the relation of PDFs can become tractable. To achieve isometric property, we propose Rate-Distortion Optimization guided autoencoder inspired by orthonormal transform coding. We show our method has the following properties: (i) the Jacobian matrix between the input space and a Euclidean latent space forms a constantly-scaled orthonormal system and enables isometric data embedding; (ii) the relation of PDFs in both spaces can become tractable one such as proportional relation. Furthermore, our method outperforms state-of-the-art methods in unsupervised anomaly detection with four public datasets.} }
Endnote
%0 Conference Paper %T Rate-distortion optimization guided autoencoder for isometric embedding in Euclidean latent space %A Keizo Kato %A Jing Zhou %A Tomotake Sasaki %A Akira Nakagawa %B Proceedings of the 37th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2020 %E Hal Daumé III %E Aarti Singh %F pmlr-v119-kato20a %I PMLR %P 5166--5176 %U https://proceedings.mlr.press/v119/kato20a.html %V 119 %X To analyze high-dimensional and complex data in the real world, deep generative models, such as variational autoencoder (VAE) embed data in a low-dimensional space (latent space) and learn a probabilistic model in the latent space. However, they struggle to accurately reproduce the probability distribution function (PDF) in the input space from that in the latent space. If the embedding were isometric, this issue can be solved, because the relation of PDFs can become tractable. To achieve isometric property, we propose Rate-Distortion Optimization guided autoencoder inspired by orthonormal transform coding. We show our method has the following properties: (i) the Jacobian matrix between the input space and a Euclidean latent space forms a constantly-scaled orthonormal system and enables isometric data embedding; (ii) the relation of PDFs in both spaces can become tractable one such as proportional relation. Furthermore, our method outperforms state-of-the-art methods in unsupervised anomaly detection with four public datasets.
APA
Kato, K., Zhou, J., Sasaki, T. & Nakagawa, A.. (2020). Rate-distortion optimization guided autoencoder for isometric embedding in Euclidean latent space. Proceedings of the 37th International Conference on Machine Learning, in Proceedings of Machine Learning Research 119:5166-5176 Available from https://proceedings.mlr.press/v119/kato20a.html.

Related Material