Is per-agent advantage normalization intentionally left out?

[lmBored]

Hi, I noticed in TorchRL, there's normalize_advantage_exclude_dims which normalize advantage without mixing agent dimention; however, BenchMARL doesn't seem to have adopted it. I also notice an imbalance loss_entropy and loss_objective when training with MAPPO and IPPO.

Is this implementation detail deliberate?

[Xmaster6y]

Hey,
I think it's due to the option being added to torchrl after benchmarl being well established. But it should be an easy addition.