【文件属性】:
文件名称:PAYING MORE ATTENTION TO ATTENTION.pdf
文件大小:1.17MB
文件格式:PDF
更新时间:2021-04-29 08:46:08
attention
Attention plays a critical role in human visual experience. Furthermore, it has
recently been demonstrated that attention can also play an important role in the
context of applying artificial neural networks to a variety of tasks from fields such
as computer vision and NLP. In this work we show that, by properly defining
attention for convolutional neural networks, we can actually use this type of in-
formation in order to significantly improve the performance of a student CNN
network by forcing it to mimic the attention maps of a powerful teacher network.
To that end, we propose several novel methods of transferring attention, show-
ing consistent improvement across a variety of datasets and convolutional neural
network architectures.