Skip to content

The arugment order of Rouge score might be wrong. #307

@zhaowei-wang-nlp

Description

@zhaowei-wang-nlp

I am reading the code in generate_instruction.py.
If you see the doc of Google's rouge_scorer, the input order of def _score_lcs(target_tokens, prediction_tokens): is target_tokens in the first.
image at https://github.com/google-research/google-research/blob/master/rouge/rouge_scorer.py#L186

In our case, should we take new instructions as the target_tokens? It seems unnatural.
image
https://github.com/tatsu-lab/stanford_alpaca/blob/main/generate_instruction.py#L189

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions