[QUESTION] To convert a Llama 3.1 70B checkpoint in torch dcp format to the HuggingFace format,

[kaiyama12345679]

I've pre-trained Llama-3.1 70B using Megatron-LM (8 tensor parallel, 2 pipeline parallel, using distributed optimizer), and I have confirmed that the checkpoint is saved in the torch dcp format.
However, I am not sure how to convert this checkpoint into a format that can be uploaded to HuggingFace. If anyone knows how to do this, I would greatly appreciate your help.

[zhangyilalala]

@kaiyama12345679 Hi~ Did you find a solution later? I'm encountering the same issue.

[leondada]

@kaiyama12345679 @zhangyilalala Hi, I'm encountering the same issue. Have you solved it?

[gaojingwei]

try use nemo