Clarification on evaluation metric

[gunshi]

Hi,
Thanks for open-sourcing this framework! I'm trying to reproduce the results of the baselines reported in the Robohive paper, and wanted to ask what is the exact metric that is averaged over 3 seeds in the Franka-expert data runs (here: https://github.com/facebookresearch/agenthive/tree/dev/scripts)?
Is it the maximum success rate over a run averaged over 3 seeds or the maximum of the average success rate over 3 seeds or something else?
The paper doesn't seem to mention exactly how the success rate of a run is decided (over many checkpoints).
Thanks!

[ShahRutav]

We report the average success rate over three seeds x three camera angles (except for Robel Suite where we use all the camera angles). We use the last checkpoint to measure the success rate.