为明确深度学习的一些概念，对一些优秀的论文进行选读。
Learning Deep Architectures for AI
Abstarct
theoretial results suggest that in order to learn the kind of complicated
functions that can represent high-level abstractions, one may need deep
architectures. Deep architectures are composed of multiple levels of non-linear
operators, such as in neural nets with many hidden layers or in complicated
propositional formulae re-using many sub-formulea. Searching the parameter
space of deep architectures is a difficult task, bust learning algorithms
such as those for Deep Belief Netwoks have recently been proposed to tackle
this problem with notable success, beating the state-of-art(最先进的) in certain areas.
This paper discusses the motivations and principles regarding learning algorithm
for deep architectures, in particular those exploiting as building blocks
unsupervised learning of single-layer models such as Restricted Boltzmann Machines,
used to construct deeper models such as Deep Belief Networks.
从摘要可以得知，本文主要讨论深度工程学习算法的的动机与原则。深度工程相关算法求
参数在技术上取得了一些成功。
Introduction
Allowing computers to model our word well enough to exhibit what we call intelligence
has been focus of more than half a century of research. To achieve this, it is clear
that a large quantity of information about our world should somehow be stored, explicitly
or implicitly, in the computer. Because it seems daunting(气馁) to formalize manually
all that information in a form that computers can use to answer questions and generalize
to new contexts, many researchers have turned to learning algorithms to capture a
large fraction of that information. Much progress has been made to understand and
improve learning algorithms, but the challenge of artifical intelligence(AI) remains.
Do we have algorithms that can understand scenes and describe them in natural language?
Not realy, except in very limited settings. Do we have algorithms that can infer enough
semantic(语义) concepts to be able to interact with most humans using these concepts?
No, If we consider image understandig ,one of the best specified of AI tasks, we realize
that we do not yet have learning algorithm that can discover the many visual and
semantic concepts that would seem to be necessary to interpret most images on the web.
The situation is similar for other AI tasks.
这部分指出无法用精确的数学公式刻画图片及语义。
Consider for example the task of interpreting an input image such as the one in Figure 1.
When humans try to solve a particular AI task(such as machine vision or natural langage
processing), they often exploit their intuition about how to decompose the problem into
sub-problems and multiple levels of representation, e.g. ,in object parts and constellation
(星群) models where models for parts can be re-used in different object instances. For
example, the current state-of-the-art in machine vision involves a sequence of modules
startig from pixels(像素) and ending in a linear or kernel classifier with intermediate modules
mixing engineered transformations and learning, e.g. first extracting low-level features
that are invariant to small geometric variations(such as edge detectors from Gabor filters),
transforming them gradually (e.g. to make them invariant to contrast changes and contrast
inversion(反向), sometimes by pooling and sub-sampling), and then detect the most frequent
patterns. A plausible and common way to extract useful information from a natural image
involves transfroming the raw pixel representation into gradually more abstract
representations, e.g. starting from the presence of edges, the detection of more complex
but local shapes, up to the identification of abstract categories associated with
sub-objects and objects which are parts of the image, and putting all these together
to capture enough understanding of the scene to answer questions about it.
此部分基本介绍了对图片深度学习的过程及意义，首先选取低水平的对于特征对于几何变化
不变的特征，（如利用Gabor filters提取边界的特征），再寻找对于validation不变的特征
（从图形角度可以理解为某种拓扑不变性，如伸缩、旋转），进一步地寻找最常用于区分的
特征部分，总的来说,就是采取非线性变换及卷积变换得到边界特征，再用于选择。