Hi,
I'm training agents on a custom environment (just one env, not vectorized) where each episode lasts a few hundred time steps, and each training iteration is done after 10 episodes are collected. Up until now, I was using MLP layers for both actor and critic models, and everything was fine. However, when I switched to an RNN layer, such as LSTM, there was an extreme slowdown, the figure below for reference:
I know that RNNs are slower, but this seems a bit too much of a slowdown, especially in a simple environment. I suppose that, in part, this is because the LSTM takes as input the whole sequence. In the configuration file of the LSTM is not possible to customize the length of the sequence.
Do you have any suggestions on how to speed up the learning in the case of RNN?
Hi,
I'm training agents on a custom environment (just one env, not vectorized) where each episode lasts a few hundred time steps, and each training iteration is done after 10 episodes are collected. Up until now, I was using MLP layers for both actor and critic models, and everything was fine. However, when I switched to an RNN layer, such as LSTM, there was an extreme slowdown, the figure below for reference:
I know that RNNs are slower, but this seems a bit too much of a slowdown, especially in a simple environment. I suppose that, in part, this is because the LSTM takes as input the whole sequence. In the configuration file of the LSTM is not possible to customize the length of the sequence.
Do you have any suggestions on how to speed up the learning in the case of RNN?