Skip to content

limit keyword does not work with the hidden keyword #837

Open
@zhengtianbao

Description

@zhengtianbao

limit does not limit the hidden metrics.

Here is my test:

gen_log.sh

#!/bin/bash

for i in {1..100}
do
    echo id=$i >> test.log
done

Execute gen_log.sh will generate 100 lines with sequence IDs, and write to the file test.log.

test.mtail

counter test_total by id limit 10

/id=(?P<id>\d+)/ {
  test_total[$id] ++
}

test_hidden.mtail

hidden counter test_total by id limit 10

/id=(?P<id>\d+)/ {
  test_total[$id] ++
}

Both are the same except for the hidden prefix.

start mtail:

./mtail -expired_metrics_gc_interval=30s -logtostderr -mtailDebug -v 10 --progs test.mtail --logs test.log 

After 30 seconds, when the GC occurs, I see the following message:

I0422 15:58:35.501660   40454 filestream.go:182] &{0xc000181aa0}: current seek is 1184
I0422 15:58:35.501682   40454 filestream.go:183] &{0xc000181aa0}: new size is 1184
I0422 15:58:35.501600   40454 metric.go:192] removeOldest: removing oldest LV: &{[1] 0xc000113030 0s}
I0422 15:58:35.501695   40454 filestream.go:243] &{0xc000181aa0}: waiting
I0422 15:58:35.501709   40454 metric.go:192] removeOldest: removing oldest LV: &{[2] 0xc000113050 0s}
I0422 15:58:35.501725   40454 metric.go:192] removeOldest: removing oldest LV: &{[3] 0xc000113070 0s}
I0422 15:58:35.501737   40454 metric.go:192] removeOldest: removing oldest LV: &{[4] 0xc0001130a0 0s}
I0422 15:58:35.501748   40454 metric.go:192] removeOldest: removing oldest LV: &{[5] 0xc0001aa020 0s}
...
I0422 15:58:35.502735   40454 metric.go:192] removeOldest: removing oldest LV: &{[88] 0xc0001aa4c0 0s}
I0422 15:58:35.502746   40454 metric.go:192] removeOldest: removing oldest LV: &{[89] 0xc0001aa4e0 0s}
I0422 15:58:35.502756   40454 metric.go:192] removeOldest: removing oldest LV: &{[90] 0xc0001aa530 0s}

But when using test_hidden.mtail, the above message is not displayed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugThis is considered a bug and shall get fixedmtail-Language/VMIssues related to the mtail language, compiler, or VM

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions