Practical Machine Learning with Tensorflow – Assignment 7

[Abhishek Tyagi]

1)- How many inputs are present in one LSTM unit at each time step?

Answer- 3

 

 

2)- Suppose you are given the task of building a named entity recognition model. Which of the following architectures should the model use?

Answer- Many-to-many.

 

 

3)- For the task in above, what is the relation between Tx and Ty ?

Answer- Tx = Ty

 

 

4)- For bidirectional layer output, we take the sum of outputs of both forward and backward direction layers. True or false?

Answer- False

 

 

5)- We will look at two methods to deal with the vanishing gradient problem – Long Short Term Memory (LSTM) and Gated Recurrent Unit (GRU).
We learnt that LSTM utilizes gates to decide the carry of information within the network. GRU, a more recent method, employs a
similar approach to tackle the vanishing gradient problem. What are the names of different gates in LSTM and GRU? (We acknowledge
that GRU was not covered in the lectures. However, we encourage you to harness the power of the internet and read more on the topic
to answer the question.)

Answer- LSTM: input gate, output gate, forget gate

GRU: update gate, reset gate

 

 

6)- Your task is to perform text generation using RNN. Please visit ​ this ​ notebook for answering the questions 6 to 9. Follow the instructions given
In the colab notebook, build the given model and train it for 10 epochs. What is the range of average loss of the trained model?
Answer- 1.2 – 2.0

 

 

7)- Now train the same model for 30 more epochs (without re-initializing the weights). What is the range of average loss of the trained model?</div>
Answer- 0.7-1.2

 

 

8)- If we change the<b> </b>embedding_dim ​ to 256 and train the model for 10 epochs from the start again, what is the range of average loss of the trained model?</div>
Answer- 1.2-2.0

 

 

9)- Train the same model for 10 more epochs (​ embedding_dim ​ = 256) (without re-initializing the weights). What is the range of average
loss of the trained model?
Answer- 1.2-2.0

 

 

10)-  looks like the model is not able to generate meaningful text. Possible improvements can be:

Answer-
level generation

Add more RNN layers

Increase embedding dimensions

• This topic was modified 2 years, 4 months ago by Abhishek Tyagi.

[Author]

Posts