Mobile app version of vmapp.org
Login or Join
Hamaas447

: Google Analytics "real time" numbers don't seem to come close to unique IP addresses in my access_log? I used this line to extract unique IPs from my access_log for the last minute: grep "2017:19:23"

@Hamaas447

Posted in: #GoogleAnalytics #RealTime

I used this line to extract unique IPs from my access_log for the last minute:

grep "2017:19:23" /var/log/nginx/access_log | awk '{print }' | sort -n | uniq -c | sort -nr | head -20000 | wc -l


Result: 185

I then tried again for 10 seconds:

grep "2017:19:23:0" /var/log/nginx/access_log | awk '{print }' | sort -n | uniq -c | sort -nr | head -20000 | wc -l


Result: 74

What does google analytics realtime tell me? 37. I've not seen it go over 50 all day and I'm confident that at time my concurrent users has been more like 200+.

Any idea why this would be?

10.02% popularity Vote Up Vote Down


Login to follow query

More posts by @Hamaas447

2 Comments

Sorted by latest first Latest Oldest Best

 

@Nickens628

...extract unique IPs from my access_log... What does google analytics realtime tell me? 37. I've not seen it go over 50 all day and I'm confident that at time my concurrent users has been more like 200+.


It's all about fine timing. Let's just assume the data you want to examime and the data google wants to examine is the same:

What you have demonstrated with your code is that the log is generated on the same server that you're querying accurate results from. The time to access a log locally is in the order of microseconds or less. It will even be faster if the log file was stored on a ram drive.

Now what google (or any other remote service) needs to do is connect to the internet in order to access a remote item (such as your log). Factor the time it takes for google to connect and factor your internet packet processing settings on your server (example: tcp/ip settings and packet priority settings and the like) then you can add that time to the time it takes for the local log file to be accessed. A normal amount for this time would be in the order of milliseconds which is 100x slower compared to accessing the file locally.

Now if google decided to download an entire log instead of requesting a specific entry, then the timing will be way off since time has to be spent downloading a bunch of entries only to get one.

10% popularity Vote Up Vote Down


 

@Speyer207

I've solved it.

I had cookieDomain set to www.my-domain-here.com. I've changed it to my-domain-here.com now and suddenly my real time users are up to 157.

Weird that that "bug" would affect some hits but not others - I'd understand if it had said "0"!

10% popularity Vote Up Vote Down


Back to top | Use Dark Theme