该系列仅在原课程基础上部分知识点添加个人学习笔记，或相关推导补充等。如有错误，还请批评指教。在学习了 Andrew Ng 课程的基础上，为了更方便的查阅复习，将其整理成文字。因本人一直在学习英语，所以该系列以英文为主，同时也建议读者以英文为主，中文辅助，以便后期进阶时，为学习相关领域的学术论文做铺垫。- ZJ
转载请注明作者和出处：微信公众号-「SelfImprovementLab」
知乎：https://zhuanlan.zhihu.com/c_147249273
CSDN：http://blog.csdn.net/JUNJUN_ZHAO/article/details/78773155
Supervised Learning with Neural Networks
Supervised Learning
There’s been a lot of hype about neural networks.And perhaps some of that type is justified,given how well they’re working.But it turns out that so far,almost all the economic value created by neural networks has been through one type of machine learning,called supervised learning. Let’s see what that means, and let’s go over some examples.In supervised learning, you have some input x,and you want to learn a function mapping to some output y.
神经网络有时媒体 炒作 得很厉害。考虑到它们的使用效果，有些说法还是靠谱的。事实上到目前为止，几乎所有由神经网络创造的经济价值都基于其中一种 机器学习 ，我们称之“监督学习”。那是什么意思呢？ 我们来看一些例子。在监督学习中，你有一些输入 X,然后你想学习到一个函数，可以映射到输出 y。
So for example, just now we saw the housing price prediction application where you input some features of a home and try to output or estimate the price y.Here are some other examples that neural networks have been applied to very effectively.Possibly the single most lucrative application of deep learning today is online advertising,maybe not the most inspiring, but certainly very lucrative, in which,by inputting an ad, the information of an ad to the website it’s thinking of showing you,and some information about the user,neural networks have gotten very good at predicting whether or not you click on an ad.
比如我们之前看到的，应用于房价预测的例子，输入房屋的一些特征就能输出或者预测价格 y。下面是一些其它例子，这些例子中神经网络效果拔群。很可能今天通过深度学习 获利最大 的，就是 在线广告，这也许不是最鼓舞人心的，但真的很赚钱，给网站输入广告信息，网站会考虑是否给你看这个广告，有时还需要输入一些用户信息，神经网络在预测你是否会点击这个广告方面已经表现得很好。
And by showing you and showing users the ads that you are most likely to click on,this has been an incredibly lucrative application of neural networks at multiple companies.Because the ability to show you ads that you’re more likely to click on has a direct impact on the bottom line of some of the very large online advertising companies.
通过向你展示，向用户展示，最有可能点开的广告，这就是神经网络在很多家公司 ，赚取无法想象的高额利润的应用方式。因为有了这种向你展示最有可能点击开的广告的能力，直接影响到了不少大型线上广告公司的收入。
Computer vision has also made huge strides in the last several years,mostly due to deep learning.So you might input an image and want to output an index,say from 1 to 1,000 trying to tell you if this picture,it might be any one of, say a 1000 different images.So, you might use that for photo tagging.I think the recent progress in speech recognition has also been very exciting,where you can now input an audio clip to a neural network,and have it output a text transcript.
过去的几年里，计算机视觉 也有很大进展，这要感谢深度学习。你输入一个图像，然后想输出一个指数，可以是从1到1000 来表明这张照片，是1000个不同的图像中的某一个，可以用来给照片打标签。深度学习最近在 语音识别 方面的进展，也是非常令人兴奋的。你可以将 音频片段 输入神经网络，它可以输出 文本。
Machine translation has also made huge strides thanks to deep learning where now you can have a neural network input an English sentence and directly output say,a Chinese sentence. And in autonomous driving, you might input an image,say a picture of what’s in front of your car as well as some information from a radar, and based on that, maybe a neural network can be trained to tell you the position of the other cars on the road.So this becomes a key component in autonomous driving systems.So a lot of the value creation through neural networks has been through cleverly selecting what should be x and what should be y for your particular problem, and then fitting this supervised learning component into often a bigger system such as an autonomous vehicle.
机器翻译 也进步很大，这得感谢深度学习，让你有一个神经网络能实现输入英语句子，它直接输出 一个中文句子。在无人驾驶技术中，你输入一幅图像，汽车前方的一个快照，还有一些雷达信息，基于这个训练过的神经网络，能告诉你路上其他汽车的位置，这是无人驾驶系统的关键组件。神经网络创造这么多价值的案例中，你要 机智地选择 x 和 y，才能解决特定问题，然后把这个监督学习过的组件，嵌入到更大型的系统中，比如无人驾驶。
It turns out that slightly different types of neural networks are useful for different applications. For example, in the real estate application that we saw in the previous video,we use a universally standard neural network architecture, right? Maybe for real estate and online advertising might be a relatively standard neural network,like the one that we saw.
可以看出稍微不同的神经网络应用到不同的地方，也都行之有效。比如说 应用到房地产上，我们上节课看过了，我们用了通用标准的神经网络架构 ，是吧？ 对于 房地产 和 在线广告，用的都是相对标准的神经网络，正如我们之前见到的。
For image applications we’ll often use convolutional neural networks,often abbreviated CNN.And for sequence data.So for example, audio has a temporal component, right?Audio is played out over time, so audio is most naturally represented as a one-dimensional time series or as a one-dimensional temporal sequence.And so for sequence data, you often use an RNN,a recurrent neural network.Language, English and Chinese, the alphabets or the words come one at a time.
图像领域里，我们经常应用的是 卷积神经网络，通常缩写为 CNN。对于序列数据，例如，音频中含有时间成分，对吧？音频是随着时间播放的 所以音频很自然地被表示为一维时间序列， 一维的时间序列 。对于序列数据，你经常使用 RNN 循环神经网络，语言、 英语和汉语 、字母或单词， 都是逐个出现的。
So language is also most naturally represented as**sequence data**.And so more complex versions of RNNs are often used for these applications.And then, for more complex applications, like autonomous driving,where you have an image, that might suggest more of**a CNN convolution neural network structur**e and radar info which is something quite different.You might end up with a more custom, or some more complex,hybrid neural network architecture.So, just to be a bit more concrete about what are the standard CNN and RNN architectures.So in the literature you might have seen pictures like this.So that’s a standard neural net.
所以语言最自然的表示方式也是序列数据。更复杂的 RNNs，经常会用于这些应用，对于更复杂的应用，比如无人驾驶。你有一张图片，可能需要 CNN “卷积神经网络结构” 架构去处理，雷达信息会更不一样，你需要一些更复杂的，混合的神经网络结构。所以，为了更具体地说明，标准的 CNN 和 RNN 结构是什么， 在文献中，你可能见过这样的图片，这是一个标准的神经网络。
Neural Network examples
You might have seen pictures like this.Well this is an example of a Convolutional Neural Network ,and we’ll see in a later course exactly what this picture means and how can you implement this.But convolutional networks are often use for image data.And you might also have seen pictures like this.And you’ll learn how to implement this in a later course.
你可能见过这样的图片，这是 一个卷积神经网络，在后续的课程，我们会去了解这幅图的含义和如何实现它。卷积网络通常用于图像数据，你可能也会看到这样的图片 后续的课程也会去实现它。
Structured Data and Unstructured Data
Recurrent neural networks are very good for this type of one-dimensional sequence data that has maybe a temporal component.You might also have heard about applications of machine learning to both Structured Data and Unstructured Data .Here’s what the terms mean.Structured Data means basically databases of data.So, for example, in housing price prediction,you might have a database or the column that tells you the size and the number of bedrooms.So, this is structured data,or in predicting whether or not a user will click on an ad,you might have information about the user, such as the age,some information about the ad, and then labels y that you’re trying to predict.
循环神经网络非常适合处理一维序列数据，其中包含时间成分，你可能也听说过机器学习被应用于结构化数据 和 非结构化数据，下面是这些术语的含义，结构化数据 是数据的数据库。例如，在房价预测中，你可能有一个数据库或者数据列，告诉你房间的大小和卧室数量，这就是结构化数据在预测用户是否会点击广告的例子中，你可能会有用户信息：比如年龄，还有广告信息，还有你要预测的标签 y。
So that’s structured data, meaning that each of the features,such as size of the house, the number of bedrooms,or the age of a user, has a very well defined meaning.In contrast, unstructured data refers to things like audio, raw audio, or images where you might want to recognize what’s in the image or text.Here the features might be the pixel values in an image or the individual words in a piece of text.
这就是结构化数据，意味着每个特征，比如说房屋大小，卧房数量，用户的年龄，都有着清晰的定义。相反，非结构化数据指的是，比如音频、原始音频、图像，你想要识别图像或文本中的内容，这里的特征可能是图像中的像素值，或者是文本中的单个单词。
Historically, it has been much harder for computers to make sense of unstructured data compared to structured data.And the fact the human race has evolved to be very good at understanding audio cues as well as images.And then text was a more recent invention,but people are just really good at interpreting unstructured data.And so one of the most exciting things about the rise of neural networks is that,thanks to deep learning, thanks to neural networks,computers are now much better at interpreting unstructured data as well compared to just a few years ago.
从历史角度看，非结构化数据与结构化数据比较 ，让计算机理解起来更难。但人类进化到现在，很擅长理解音频信号和图像， 文本是一个更近代的发明， 但人们真的很擅长解读非结构化数据，神经网络的兴起过程中，最令人兴奋的事情之一就是 多亏了深度学习，多亏了神经网络， 计算机现在能更好地解释非结构化数据 ，和几年前对比的话 。
And this creates opportunities for many new exciting applications that use
**speech recognition, image recognition,natural language processing on text,**much more than was possible even just two or three years ago.I think because people have a natural empathy to understanding unstructured data,you might hear about neural network successes on unstructured data more in the media because it’s just cool when the neural network recognizes a cat.
这给我们创造了很多令人兴奋的应用机会，语音识别、图像识别、自然语言文字处理 ，现在能做的事情比两三年前要丰富多了。我认为，因为人们生来就有能力 理解非结构化数据，你可能在媒体上听到了更多神经网络在非结构化数据上的成功，尤其是当神经网络识别了一只猫时。
We all like that, and we all know what that means.But it turns out that a lot of short term economic value that neural networks are creating has also been on structured data,such as much better advertising systems,much better profit recommendations,and just a much better ability to process the giant databases that many companies have to make accurate predictions from them.
那真的很酷，我们都知道那意味着什么。神经网络在很多短期经济价值的创造 是基于结构化数据的，比如更好的广告系统，更好的获利建议，有更好的能力去处理很多公司拥有的海量数据库，并用这些数据准确预测未来趋势。
So in this course, a lot of the techniques we’ll go over will apply to both structured data and to unstructured data.For the purposes of explaining the
algorithms,we will draw a little bit more on examples that use unstructured data.But as you think through applications of neural networks within your own team I hope you find both uses for them in both structured and unstructured data.
在这门课中，我们会学到很多技巧可以应用到结构化数据，也可以应用到非结构化数据 。为了更清楚地解释算法原理，我们会多用非结构化数据的例子，但当你自己的团队评估了各种神经网络的应用之后，希望你的算法能够同时学习结构化和非结构化数据。
So neural networks have transformed supervised learning and are creating tremendous economic value.It turns out though, that the basic technical ideas behind neural networks have mostly been around, sometimes for many decades.So why is it, then, that they’re only just now taking off and working so well?In the next video, we’ll talk about why it’s only quite recently that neural networks have become this incredibly powerful tool that you can use.
神经网络彻底改变了监督学习正创造着巨大的经济价值。其实呢 ，基本的神经网络背后的技术理念，大部分都不是新概念，有些甚至有几十年历史了。那么 为什么它们现在才流行，才行之有效呢？下一集视频中，我们将讨论为什么是最近神经网络才成为你可以使用的强大工具。
PS: 欢迎扫码关注公众号：「SelfImprovementLab」！专注「深度学习」，「机器学习」，「人工智能」。以及「早起」，「阅读」，「运动」，「英语」「其他」不定期建群 打卡互助活动。
