In a normal feedforward neural network, each input is independent of other input. But with a sequential dataset, we need to know about the past input to make a prediction. A sequence is an ordered set of items. For instance, a sentence is a sequence of words. Let's suppose that we want to predict the next word in a sentence; to do so, we need to remember the previous words. A normal feedforward neural network cannot predict the correct next word, as it will not remember the previous words of the sentence. Under such circumstances (in which we need to remember the previous input), to make predictions, we use recurrent neural networks (RNNs).

In this chapter, we will describe how an RNN is used to model sequential datasets and how it remembers the previous input. We will begin by investigating how an RNN differs from a feedforward neural network...