Unable to reproduce the results of SV in the Wittyer benchmark

[Mistbornk]

Dear,
I have installed Wittyer v0.3.5.1 and followed the instructions in the Benchmarking.txt to download the Truth Set, BED region, and DRAGEN/Lumpy/Manta/Delly Callset. The SV INS/DEL F-score results for DRAGEN and Delly align with those mentioned in the paper, but the results for Lumpy/Manta do not. Specifically, the results for Lumpy/Manta are showing as "NAN". It appears that the issue may be related to missing values in the FILTER field of the call set VCF file ("." instead of "PASS").

In the wittyer with default configuration, any variants with a FILTER field other than "PASS" are filtered out. Do you have any additional processing? Or did I make a mistake in some step?

e.q. Manta
Command:

dotnet Wittyer.dll \
	     -t HG002_SVs_Tier1_v0.6.vcf.gz \
             -i HG002_manta_SV_hg19.vcf.gz \
	     -em SimpleCounting \
	     --configFile Config-default.json \
	     --includeBed HG002_SVs_Tier1_v0.6.bed \
	     -o Fig2D/Manta

Reaults:

    --------------------------------
    Deletion
    --------------------------------
    QuerySample     TruthSample     QueryTotal      QueryTp    QueryFp     Precision       TruthTotal      TruthTp     TruthFn     Recall      Fscore
    HG002           HG002           0               0          0           NaN             4203            0           4203        0.000%      NaN    
    --------------------------------
    Insertion
    --------------------------------
    QuerySample     TruthSample     QueryTotal      QueryTp    QueryFp     Precision       TruthTotal      TruthTp     TruthFn     Recall      Fscore
    HG002           HG002           0               0          0           NaN             5244            0           5244        0.000%      NaN   

Many thanks,
Max