Skip to content

Limit keyword not release memory after delete oldest metrics. #763

Open
@zhengtianbao

Description

@zhengtianbao

Hi @jaqx0r, I set the Limit keyword so mtail will remove the oldest metrics, but I noticed the memory usage of mtail process will increase and not decrease after GC.

https://github.com/google/mtail/blob/e35f4c5d7b53b9947336850b4e11d10a78428dfe/internal/metrics/metric.go#L215C20-L215C20

m.labelValuesMap type is map[string]*LabelValue, the delete will not shrink the map size. see golang/go#20135

If someone uses the Limit keyword, it means that the number of metrics he wants to limit may be very large. Do we need to consider the memory consumption of this part?

I changed the m.labelValuesMap type to map[[256]byte]*LabelValue, because golang will allocates data more than 128 bytes from heap and save pointers in the bucket of map, and after GC pointers at map will be replaced by nil pointer, so the memory pointed will be recollected. This solution may have some unconsidered issues, just for reference.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugThis is considered a bug and shall get fixedmtail-Language/VMIssues related to the mtail language, compiler, or VM

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions