Iclr2020: Compression based bound for non-compressed network: unified generalization error analysis of large compressible deep neural network
By A Mystery Man Writer
Description
Iclr2020: Compression based bound for non-compressed network: unified generalization error analysis of large compressible deep neural network - Download as a PDF or view online for free
1) The document presents a new compression-based bound for analyzing the generalization error of large deep neural networks, even when the networks are not explicitly compressed. 2) It shows that if a trained network's weights and covariance matrices exhibit low-rank properties, then the network has a small intrinsic dimensionality and can be efficiently compressed. 3) This allows deriving a tighter generalization bound than existing approaches, providing insight into why overparameterized networks generalize well despite having more parameters than training examples.
1) The document presents a new compression-based bound for analyzing the generalization error of large deep neural networks, even when the networks are not explicitly compressed. 2) It shows that if a trained network's weights and covariance matrices exhibit low-rank properties, then the network has a small intrinsic dimensionality and can be efficiently compressed. 3) This allows deriving a tighter generalization bound than existing approaches, providing insight into why overparameterized networks generalize well despite having more parameters than training examples.
![Iclr2020: Compression based bound for non-compressed network: unified generalization error analysis of large compressible deep neural network](https://arxiv.org/html/2302.05825v3/extracted/5475051/cifar10_sing.png)
Koopman-based generalization bound: New aspect for full-rank weights
![Iclr2020: Compression based bound for non-compressed network: unified generalization error analysis of large compressible deep neural network](https://i0.wp.com/papercopilot.com/wp-content/uploads/2023/03/logo_v2.png?fit=512%2C512&ssl=1)
ICLR 2020 Statistics - Paper Copilot
![Iclr2020: Compression based bound for non-compressed network: unified generalization error analysis of large compressible deep neural network](https://cdn.slidesharecdn.com/ss_thumbnails/onlinecoresetselectionforrehearsal-basedcontinuallearning-220325050134-thumbnail.jpg?width=336&fit=bounds)
Iclr2020: Compression based bound for non-compressed network: unified generalization error analysis of large compressible deep neural network
![Iclr2020: Compression based bound for non-compressed network: unified generalization error analysis of large compressible deep neural network](https://pub.mdpi-res.com/entropy/entropy-25-01063/article_deploy/html/images/entropy-25-01063-g0A4.png?1689328166)
Entropy, Free Full-Text
![Iclr2020: Compression based bound for non-compressed network: unified generalization error analysis of large compressible deep neural network](https://www.researchgate.net/profile/Fan-Yang-41/publication/354235011/figure/fig2/AS:1062715703431168@1630382576007/A-recently-proposed-video-recognition-network-which-was-directly-trained-on-compressed_Q320.jpg)
PDF) Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions
![Iclr2020: Compression based bound for non-compressed network: unified generalization error analysis of large compressible deep neural network](http://www.endtoend.ai/assets/blog/misc/iclr2020-rl/iclr.png)
Papers Accepted to ICLR 2020
![Iclr2020: Compression based bound for non-compressed network: unified generalization error analysis of large compressible deep neural network](https://iclr.github.io/iclr-images/small/SklkDkSFPB.jpg)
ICLR 2020
![Iclr2020: Compression based bound for non-compressed network: unified generalization error analysis of large compressible deep neural network](https://cdn.slidesharecdn.com/ss_thumbnails/powerrotationalinterleaveronanidmasystem-130619222658-phpapp02-thumbnail.jpg?width=336&fit=bounds)
Iclr2020: Compression based bound for non-compressed network
![Iclr2020: Compression based bound for non-compressed network: unified generalization error analysis of large compressible deep neural network](https://image.slidesharecdn.com/iclr2020public-200422141800/85/iclr2020-compression-based-bound-for-noncompressed-network-unified-generalization-error-analysis-of-large-compressible-deep-neural-network-8-320.jpg?cb=1667961725)
Iclr2020: Compression based bound for non-compressed network: unified generalization error analysis of large compressible deep neural network
![Iclr2020: Compression based bound for non-compressed network: unified generalization error analysis of large compressible deep neural network](https://www.researchgate.net/profile/Fan-Yang-41/publication/354235011/figure/fig1/AS:1062715699253259@1630382575934/The-input-image-is-represented-as-a-series-of-compressed-feature-maps-and-subsequent_Q320.jpg)
PDF) Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions
from
per adult (price varies by group size)