Rendered at 22:24:20 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
aleks_me2 2 hours ago [-]
I've revived AWFFull for static log analysis, just like in the good old webalizer days :-). Link to the report is in my bio, sorry, I can't post links here for now.
What I'm seeing matches the observations of the other commenters here exactly: most of the "users" are bots harvesting pages and PDFs. My solution for getting at the real users is GoatCounter.
vbernat 4 hours ago [-]
It is now mostly unusable for web analytics. So many scrapers are pretending to be browsers. On my site, there is a factor 10 with what I consider an honest view of human visitors.
janvdberg 4 hours ago [-]
Yes, this is unfortunately my experience with GoAccess
iJohnDoe 2 hours ago [-]
This always looks cool and the real-time aspect always gets attention. However, If I remember correctly, it was kind of a burden to get deployed.
Again, if I remember correctly, it was able to make reports for AWS Cloudfront logs, which was actually useful.
QuantumNomad_ 34 minutes ago [-]
> If I remember correctly, it was kind of a burden to get deployed
It probably depends a lot on your setup. I’ve been running GoAccess from a crontab on some of my sites for a couple of years. It was a bit fiddly to get the correct combination of Nginx logs, forwarding real IP from HAProxy, and getting it to show with GeoIP in generated GoAccess HTML reports, but once I got it working satisfactorily it’s been working well since. The only part that is missing in my GoAccess reports is the ASN of the clients. I don’t remember if that’s because ASN mappings are a paid GeoIP database feature or if I simply ignored that part or what, but I do have countries and municipalities correct (I can tell because for example on one of the sites the most frequent visitor is a client machine that I own, and it shows up accordingly prominently with the country and municipality where that client machine is located) and that’s the main thing I wanted to be able to see when setting up GoAccess to use a GeoIP database file.
I serve the generated GoAccess HTML reports from Nginx itself with a simple .htaccess file providing username and hashed password and Nginx using that file with HTTP Basic Auth. Since the only person that looks at those reports is myself, and the sites use TLS, I don’t need anything more advanced than HTTP Basic Auth to protect access to the GoAccess HTML reports.
And I suppose that in today’s world an LLM would probably be able to do most of the config writing for a setup such as mine for all of the above parts.
aleks_me2 1 hours ago [-]
The question I always come back to is: how long does it take you to get stats for something like 1 month or 1 quarter with the current "modern" tools?
For ad hoc analytics these tools are very nice, but for long-term stats they run into timeouts, because the backend needs a lot of time to collect the data and build the reports for ~1 month or a quarter — or am I wrong about that?
WesolyKubeczek 3 hours ago [-]
Unfortunately, each time a new log analyzer appears, it seems to only be analyzing web access logs in combined format.
tredre3 1 hours ago [-]
GoAccess supports more than combined format. I think they even support stubborn servers who output JSON logs only:
What I'm seeing matches the observations of the other commenters here exactly: most of the "users" are bots harvesting pages and PDFs. My solution for getting at the real users is GoatCounter.
Again, if I remember correctly, it was able to make reports for AWS Cloudfront logs, which was actually useful.
It probably depends a lot on your setup. I’ve been running GoAccess from a crontab on some of my sites for a couple of years. It was a bit fiddly to get the correct combination of Nginx logs, forwarding real IP from HAProxy, and getting it to show with GeoIP in generated GoAccess HTML reports, but once I got it working satisfactorily it’s been working well since. The only part that is missing in my GoAccess reports is the ASN of the clients. I don’t remember if that’s because ASN mappings are a paid GeoIP database feature or if I simply ignored that part or what, but I do have countries and municipalities correct (I can tell because for example on one of the sites the most frequent visitor is a client machine that I own, and it shows up accordingly prominently with the country and municipality where that client machine is located) and that’s the main thing I wanted to be able to see when setting up GoAccess to use a GeoIP database file.
I serve the generated GoAccess HTML reports from Nginx itself with a simple .htaccess file providing username and hashed password and Nginx using that file with HTTP Basic Auth. Since the only person that looks at those reports is myself, and the sites use TLS, I don’t need anything more advanced than HTTP Basic Auth to protect access to the GoAccess HTML reports.
And I suppose that in today’s world an LLM would probably be able to do most of the config writing for a setup such as mine for all of the above parts.
For ad hoc analytics these tools are very nice, but for long-term stats they run into timeouts, because the backend needs a lot of time to collect the data and build the reports for ~1 month or a quarter — or am I wrong about that?
https://goaccess.io/man#custom-log