Missing Hours in VisitorCount Panel

[nborrmann]

Hi,

I'm running the following command

zcat /var/log/nginx/myhost*.gz | /root/goaccess/goaccess -f /var/log/nginx/myhost.log.1 /var/log/nginx/myhost.log - --date-spec=hr

Recently I've run into an issue where lines from the Visitor Count Panel are dropped seemingly randomly. In the console window I even have items dis- and reappearing every second or so. The two screenshots show the line 18/Apr/2017:13 appearing in one screenshot but not in the other (I haven't pressed anything in between those shots). The static html output also has missing hours.

I suspect this might be because I'm over the limit of 366 items per panel (And indeed it lists only 366 items). But shouldn't this cut off at the beginning instead of randomly?

Here's the missing items in the (static) html output:

[allinurl]

Looks like a sorting issue. Let me try to reproduce this on my side and I'll post back.

BTW, does the appearing and disappearing occur with a smaller data set (<= 366 items)? Thanks.

[nborrmann]

Thanks. It worked fine when there were less than 366 items.

…

On 21 Apr 2017 23:39, "Gerardo O." ***@***.***> wrote: Looks like a sorting issue. Let me try to reproduce this on my side and I'll post back. BTW, does the appearing and disappearing occur with a smaller data set (<= 366 items)? Thanks. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#741 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABpLKDDvykW50K-66wkyIjtDgGJoTNz9ks5rySH-gaJpZM4NEWWP> .

[nborrmann]

I just recompiled goaccess without --enable-tcb=btree and the problem went away.

I also noticed that the parsing is noticeably faster when now, even though i wasn't using either of the --load-from-disk --keep-db-files options.

[allinurl]

Question for you: I think the answer is no, but does the log that you are parsing contain consecutive hours from the initial date until the current time? From the images posted, I'd assume it doesn't and it skips certain hours.

That's definitely odd, --enable-tcb=btree is noticeably slower since it has to commit date to disk while the default tables are in-memory. Are you sure you're using btree? You can run goaccess -s to check.

[nborrmann]

Are you sure you're using btree? You can run goaccess -s to check.

Well, I recompiled it now without the flag, so I'm currently not using btree. The issue with the missing dates only occured when I had it compiled with btree.

does the log that you are parsing contain consecutive hours from the initial date until the current time?

Yes. It's now working (without btree) and showing entries in every consecutive hour (the 366 latest hours).

That's definitely odd, --enable-tcb=btree is noticeably slower since it has to commit date to disk while the default tables are in-memory. Are you sure you're using btree? You can run goaccess -s to check.

My thought was that even though it is compiled with enable-tcb=btree it shouldn't write to disk when not using the flag --keep-db-files. But if that's not how it works, I guess it's fine.

[allinurl]

My bad, I thought the issue was occurring with the default storage. It's certainly faster without btree. Let me try to reproduce this with btree. Thanks.