[DL輪読会] Revealing the Dark Secrets of BERT (EMNLP-IJCNLP, 2019)Deep Learning JP
1. Recent studies have revealed the dark secrets of BERT by probing its internal representations, finding that BERT captures syntactic rules and hierarchical structure but struggles with semantic and world knowledge.
2. Additional research has shown how BERT can be influenced by context and training data distribution, and that its representations are focused on syntactic rules rather than real-world concepts.
3. Further work is needed to better understand BERT's capabilities and limitations at capturing linguistic meaning and world knowledge from language.
Version DRAFT d'une formation Data Scientist que j'ai conçue à partir de sources diverses (voir références bibliographiques à la fin de chaque diapositive).
La formation est destinée aux personnes possédant des bases (~BAC+2) en statistiques et programmation (j'utilise R).
Je reste ouvert à tout commentaire, critique et correction. Je continuerai à mettre à jour les diapositives et à en ajouter d'autres si j'ai le temps.
[DL輪読会] Revealing the Dark Secrets of BERT (EMNLP-IJCNLP, 2019)Deep Learning JP
1. Recent studies have revealed the dark secrets of BERT by probing its internal representations, finding that BERT captures syntactic rules and hierarchical structure but struggles with semantic and world knowledge.
2. Additional research has shown how BERT can be influenced by context and training data distribution, and that its representations are focused on syntactic rules rather than real-world concepts.
3. Further work is needed to better understand BERT's capabilities and limitations at capturing linguistic meaning and world knowledge from language.
Version DRAFT d'une formation Data Scientist que j'ai conçue à partir de sources diverses (voir références bibliographiques à la fin de chaque diapositive).
La formation est destinée aux personnes possédant des bases (~BAC+2) en statistiques et programmation (j'utilise R).
Je reste ouvert à tout commentaire, critique et correction. Je continuerai à mettre à jour les diapositives et à en ajouter d'autres si j'ai le temps.
You can learn Blender that is open source and free 3D CG software.
There are 800 pages in this slide.
It's so easy to learn basic usage of Blender .
This slide is japanese.