这里主要梳理一下作业的主要内容和思路，完整作业文件可参考:
https://github.com/pandenghuang/Andrew-Ng-Deep-Learning-notes/tree/master/assignments/C5W1
作业完整截图，参考本文结尾：作业完整截图。
Building your Recurrent Neural Network - Step by Step（搭建循环神经网络）
Welcome to Course 5's first assignment! In this assignment, you will implement your first Recurrent Neural Network in numpy.
Recurrent Neural Networks (RNN) are very effective for Natural Language Processing and other sequence tasks because they have "memory". They can read inputs 𝑥〈𝑡〉x〈t〉 (such as words) one at a time, and remember some information/context through the hidden layer activations that get passed from one time-step to the next. This allows a uni-directional RNN to take information from the past to process later inputs. A bidirection RNN can take context from both the past and the future.
...
1 - Forward propagation for the basic Recurrent Neural Network（基本循环神经网络的前向传播）
Later this week, you will generate music using an RNN. The basic RNN that you will implement has the structure below. In this example, 𝑇𝑥=𝑇𝑦.
Figure 1: Basic RNN model
1.1 - RNN cell（RNN单元）
A Recurrent neural network can be seen as the repetition of a single cell. You are first going to implement the computations for a single time-step. The following figure describes the operations for a single time-step of an RNN cell.
Figure 2: Basic RNN cell. Takes as input 𝑥〈𝑡〉(current input) and 𝑎〈𝑡−1〉 (previous hidden state containing information from the past), and outputs 𝑎〈𝑡〉 which is given to the next RNN cell and also used to predict 𝑦〈𝑡〉
...
1.2 - RNN forward pass（RNN前向传播）
You can see an RNN as the repetition of the cell you've just built. If your input sequence of data is carried over 10 time steps, then you will copy the RNN cell 10 times. Each cell takes as input the hidden state from the previous cell (𝑎〈𝑡−1〉) and the current time-step's input data (𝑥〈𝑡〉). It outputs a hidden state (𝑎〈𝑡〉) and a prediction (𝑦〈𝑡〉) for this time-step.
...
2 - Long Short-Term Memory (LSTM) network（长短期记忆网络）
This following figure shows the operations of an LSTM-cell.
Figure 4: LSTM-cell. This tracks and updates a "cell state" or memory variable 𝑐〈𝑡〉 at every time-step, which can be different from 𝑎〈𝑡〉.
...
2.1 - LSTM cell（LSTM单元）
Exercise: Implement the LSTM cell described in the Figure (3).
...
2.2 - Forward pass for LSTM（LSTM前向传播）
Now that you have implemented one step of an LSTM, you can now iterate this over this using a for-loop to process a sequence of 𝑇𝑥 inputs.
...
3 - Backpropagation in recurrent neural networks (OPTIONAL / UNGRADED)（循环神经网络中的反向传播）
In modern deep learning frameworks, you only have to implement the forward pass, and the framework takes care of the backward pass, so most deep learning engineers do not need to bother with the details of the backward pass. If however you are an expert in calculus and want to see the details of backprop in RNNs, you can work through this optional portion of the notebook.
When in an earlier course you implemented a simple (fully connected) neural network, you used backpropagation to compute the derivatives with respect to the cost to update the parameters. Similarly, in recurrent neural networks you can to calculate the derivatives with respect to the cost in order to update the parameters. The backprop equations are quite complicated and we did not derive them in lecture. However, we will briefly present them below.
3.1 - Basic RNN backward pass（基本RNN单元的反向传播）
We will start by computing the backward pass for the basic RNN-cell.
...
3.2 - LSTM backward pass（LSTM反向传播）
3.2.1 One Step backward（单步后向传播）
The LSTM backward pass is slighltly more complicated than the forward one. We have provided you with all the equations for the LSTM backward pass below. (If you enjoy calculus exercises feel free to try deriving these from scratch yourself.)
3.2.2 gate derivatives（门导数）
3.2.3 parameter derivatives（参数导数）
3.3 Backward pass through the LSTM RNN（LSTM RNN反向传播）
This part is very similar to the
rnn_backward
function you implemented above. You will first create variables of the same dimension as your return variables. You will then iterate over all the time steps starting from the end and call the one step function you implemented for LSTM at each iteration. You will then update the parameters by summing them individually. Finally return a dictionary with the new gradients.
Congratulations !
Congratulations on completing this assignment. You now understand how recurrent neural networks work!
Lets go on to the next exercise, where you'll use an RNN to build a character-level language model.