Real-time Analysis of Caddy Logs Using goaccess

A few days ago, when I was doing log analysis using nginxpulse, I found some problems, such as the UI being distorted when accessing from mobile phones or tablets. The system was malfunctioning; the "Access Details" page was almost unreadable. So, I decided to try deploying goaccess.

There aren't many good examples online for implementing real-time access to multiple domains under Caddy. Below is a record of my troubleshooting process.

1. Installing GoAccess

I strongly recommend installing from source code, as the versions included in various Linux distribution repositories are lower than the current version of GoAccess, which can lead to limitations in some functions.

For example, in my case, the version on Debian was too low, resulting in the absence of the World Map and Azimuthal/Globe options in the Geo Location panel. The difference between this and the official demo is stark; the former displays a standard 2D world map, while the latter displays a 3D world map.

In addition, for source code installation, it's best to download the source package directly from the official website:

1$ wget https://tar.goaccess.io/goaccess-1.9.4.tar.gz
2$ tar -xzvf goaccess-1.9.4.tar.gz
3$ cd goaccess-1.9.4/
4$ ./configure --enable-utf8 --enable-geoip=mmdb --with-openssl
5$ make
6# make install

Downloading the package from GitHub may result in a missing configure program. Don't ask me how I know; I've encountered this problem before.

If compilation fails, it's most likely due to missing Ncursesw dependencies. Search for and install the corresponding packages for your distribution. The official website https://goaccess.io/download page provides instructions.

2. Configuring Caddy Log File Parsing

Log analysis involves parsing logs and then re-aggregating and displaying them.

Here's my configuration (comment lines, blank lines, and lines starting with static-file|no are filtered for easier viewing of key information):

 1root@tokyo:~# egrep -v "^#|^$|^static|^no" /etc/goaccess/goaccess.conf
 2date-format %s
 3log-format {ts:"%x.%^",request:{remote_ip:"%h",proto:"%H",method:"%m",host:"%v",uri:"%U",headers:{"User-Agent":["%u","%^"]},tls:{cipher_suite:"%k",proto:"%K"}},duration:"%T",size:"%b",status:"%s",resp_headers:{"Content-Type":["%M;%^"]}}
 4log-format CADDY
 5config-dialog false
 6hl-header true
 7json-pretty-print false
 8tz Asia/Shanghai
 9with-mouse false
10addr 127.0.0.1
11daemonize true
12real-time-html true
13ws-url wss://goaccess.xxx.com:443/ws
14log-file /var/log/caddy/xxx.com.cc.log
15agent-list false
16with-output-resolver false
17http-method yes
18http-protocol yes
19output /data/goaccess/xxx.com.html
20444-as-404 false
214xx-to-unique-count false
22all-static-files false
23double-decode false
24ignore-crawlers false
25crawlers-only false
26unknowns-as-crawlers false
27real-os true
28geoip-database /usr/share/GeoIP/GeoLite2-City.mmdb
29geoip-database /usr/share/GeoIP/GeoLite2-ASN.mmdb

Key configuration points:

• date-format %s Required for Caddy
• The two configurations starting with log-format: one for parsing CADDY JSON Structured; log-format CADDY tells GoAccess the current log type is Caddy.
• tz Asia/Shanghai Time zone setting, optional.
• addr 127.0.0.1 Critical, security-related configuration; recommended to listen locally and forward via a proxy.
• real-time-html true Enables real-time webpage refresh; GoAccess will refresh data via WebSocket.
• ws-url wss://goaccess.xxx.com:443/ws Real WebSocket response interface, very important. If there's a configuration error here, you can check it through the browser's debugging window.
• log-file /var/log/caddy/xxx.com.cc.log The target log file to analyze.
• output /data/goaccess/xxx.com.html The generated HTML file; the browser ultimately accesses this.
• geoip-database /usr/share/GeoIP/GeoLite2-City.mmdb IP addresses accurate to the city level; an additional IP address database is required.
• geoip-database /usr/share/GeoIP/GeoLite2-ASN.mmdb Seems to contain network operator information, such as AT&T Services, Inc., Chinanet, etc.

The listening port can also be modified as needed; the default is: port 7890

Download the GeoLite2 related database files here: https://github.com/P3TERX/GeoLite.mmdb

systemd file configuration:

 1root@tokyo:~# systemctl cat goaccess.service
 2# /etc/systemd/system/goaccess.service
 3[Unit]
 4Description=GoAccess Real-Time Log Analyzer
 5After=network.target caddy.service
 6
 7[Service]
 8User=root
 9ExecStart=/usr/local/bin/goaccess -p /etc/goaccess/goaccess.conf
10Type=forking
11Restart=on-failure
12RestartSec=5
13StartLimitBurst=5
14StartLimitInterval=30s
15
16[Install]
17WantedBy=multi-user.target

Note the Type=forking part; otherwise, it won't start correctly. Alternatively, if you don't configure daemonize true, this line can be removed. The main process will fork a child process at startup, then the parent process exits, and the child process continues running in the background, which is where Type=forking is needed.

3. Configure Caddy to Access the Output Page

See configuration below

 1logs.xxx.com {
 2	encode zstd br gzip
 3	root * /data/goaccess
 4	file_server
 5	basic_auth {
 6		your_username your_hash_password  # caddy hash-password
 7	}
 8	reverse_proxy /ws localhost:7890
 9	reverse_proxy /ws_another localhost:7891 # 多个域名日志分析，如果想分实例解析
10}

This allows the browser to access the real-time refreshed logs via https://logs.xxx.com/xxx.com.html.

If you encounter a real-time refresh error, press F12 to open a browser window, search for "ws", check if there is an error message, and whether the data in the response is refreshed normally. Follow the error message instructions to resolve the issue.

Here's a screenshot example:

4. Multi-Domain Log Configuration

If you have multiple domains and want to log to different paths, such as foo.xxx.com.log and bar.xxx.com.log, and want to view them independently on multiple pages.

You can consider using two goaccess instances to handle this. Separate the listening ports, and have Caddy proxy them to the different ports as shown above.

Alternatively, you can have AI write an index.html file to implement the routing logic to different pages, and then place the index.html file in the root directory specified by root * /data/goaccess. That's what I did, and it's very convenient.

5. Final Screenshot

Screenshot Example:

As you can see, the ASN and Geo Location are displayed correctly. Frankly, initial configuration will require some time to figure out.

Access the parsed log site using your mobile phone or computer, and the data will refresh in real time.

If you don't want to deploy and just want to see the effect, you can directly visit the live demo page provided on the goaccess official website.

Neither nginxpulse nor goaccess can currently directly view the raw logs. After all, they are lightweight tools and cannot be compared with enterprise-level large-scale logging systems such as OpenObserve and ELK. Their applicable scenarios are different, and their costs vary significantly.