Environment
- Hardware: NVIDIA H20 (Hopper / SM90 architecture)
- Container: Latest
nvcr.io/nvidia/merlin/merlin-tensorflow
- SOK Version: v2.x (pre-installed in the container)
Description
The tutorial code located in documents/tutorials/DLRM_Benchmark appears to be written for an older version of the Sparse Operation Kit (SOK v1.x) and is incompatible with the latest Merlin containers.
When trying to run the benchmark on Hopper architecture out-of-the-box, the scripts fail continuously due to deprecated APIs and structural changes in SOK v2.x.
Key Symptoms Observed
Instead of running successfully, the tutorial throws a cascade of errors related to outdated APIs, including but not limited to:
- Deprecated Initialization:
AttributeError: module 'sparse_operation_kit' has no attribute 'Init' (main.py).
- Removed Embedding Classes:
AttributeError: module 'sparse_operation_kit' has no attribute 'All2AllDenseEmbedding' (model.py).
- Invalid Arguments: Even if changed to
sok.all2all_dense_embedding, it throws TypeError for removed parameters like max_vocabulary_size_per_gpu and embedding_vec_size.
- Deprecated Variable Splitting:
AttributeError: module 'sparse_operation_kit' has no attribute 'split_embedding_variable_from_others' (trainer.py).
- Data Script Mismatches: The
preprocess/split_bin.py script generates files without extensions, but dataset.py looks for .bin files, leading to crashes.
Request
It seems the tutorial needs a complete overhaul to utilize the native sok.Variable approach and align with the current SOK v2.x standards.
Could the maintainers please update this benchmark tutorial so that users on the latest containers (and Hopper GPUs) can run it out-of-the-box?
Thank you!
Environment
nvcr.io/nvidia/merlin/merlin-tensorflowDescription
The tutorial code located in
documents/tutorials/DLRM_Benchmarkappears to be written for an older version of the Sparse Operation Kit (SOK v1.x) and is incompatible with the latest Merlin containers.When trying to run the benchmark on Hopper architecture out-of-the-box, the scripts fail continuously due to deprecated APIs and structural changes in SOK v2.x.
Key Symptoms Observed
Instead of running successfully, the tutorial throws a cascade of errors related to outdated APIs, including but not limited to:
AttributeError: module 'sparse_operation_kit' has no attribute 'Init'(main.py).AttributeError: module 'sparse_operation_kit' has no attribute 'All2AllDenseEmbedding'(model.py).sok.all2all_dense_embedding, it throwsTypeErrorfor removed parameters likemax_vocabulary_size_per_gpuandembedding_vec_size.AttributeError: module 'sparse_operation_kit' has no attribute 'split_embedding_variable_from_others'(trainer.py).preprocess/split_bin.pyscript generates files without extensions, butdataset.pylooks for.binfiles, leading to crashes.Request
It seems the tutorial needs a complete overhaul to utilize the native
sok.Variableapproach and align with the current SOK v2.x standards.Could the maintainers please update this benchmark tutorial so that users on the latest containers (and Hopper GPUs) can run it out-of-the-box?
Thank you!