Solving the stuck problem under Atuin ZFS

Recently, when using the fish + atuin combination, I found that lag occasionally occurs, but it is not inevitable. The symptom is that when you enter low-cost commands such as ip a and htop, the system feels stuck for a few seconds, which is obviously a problem.

As a Linux user for N years, I have never been so stuck in the terminal, which is intolerable. At first, I thought it was caused by the audit module. I checked the process carefully and found no abnormalities.

There were no special changes in the computer environment. Almost everything was customized by myself, and there was no possibility of any problems. I immediately realized that it might be a problem with the new use of atuin. After all, fish has been out for many years, so it would not be like this. Sample.

Search atuin lag in google, the first record hits the mark: https://github.com/atuinsh/atuin/issues/952

After a cursory reading of the above issues, the problem occurs when atuin uses SQLite to store history records in offline mode. SQLite has this problem in ZFS.

There are not many users who are dissatisfied. First you need to use ZFS, and then you need to use atuin + SQLite.

There are several solutions to issues, either modifying the source code of atuin, or waiting for ZFS or SQLite to be fixed.

None of the above methods are very easy. Personally, I think the solution with the most likes is the easiest (slightly modified).

1sudo zfs create -V 500MB rpool/atuin
2sudo zfs list -o name,encryption | grep atuin # ensure encryption for the zvol
3sudo mkfs.ext4 /dev/zvol/rpool/atuin
4mv ~/.local/share/atuin ~/.local/share/atuin-backup
5mkdir ~/.local/share/atuin
6sudo mount /dev/zvol/rpool/atuin ~/.local/share/atuin
7sudo chown -R ${USER}: ~/.local/share/atuin
8cp -rv ~/.local/share/atuin-backup/* ~/.local/share/atuin

The only bad thing is that the mount here is temporary and will be lost after restarting. You have to write it to /etc/fstab and load it automatically at boot.

The operation process is explained and recorded as follows:

1. Create Zvolumes datasets, etc.

1➜ ~ sudo zfs create -V 500MB rpool/atuin
2[sudo] password for mephisto:
3cannot create 'rpool/atuin': dataset already exists

Because I have already created it, it will prompt that it already exists. There will be no problem when creating it normally for the first time.

In order to take care of readers who are not familiar with ZFS, a picture must be taken

Focus on the zpool layer. Compared with the traditional file system, ZFS has an extra layer of abstraction. In the software industry, an extra layer of abstraction usually feels very powerful. If you are in trouble, add another layer 😅

Zpools contains various types of datasets, and Zvolumes datasets is one of them. It can be formatted into ext4, partitioned, and mounted, just like a block device, which is really powerful.

format

1sudo mkfs.ext4 /dev/zvol/rpool/atuin

View Results

 1➜ ~ ls -al /dev/zvol/rpool/atuin
 2lrwxrwxrwx 1 root root 9 Feb 26 18:42 /dev/zvol/rpool/atuin -> ../../zd0
 3➜ ~ sudo fdisk -l
 4[sudo] password for mephisto:
 5Disk /dev/sda: 476.94 GiB, 512110190592 bytes, 1000215216 sectors
 6Disk model: SanDisk SD8SN8U-
 7Units: sectors of 1 * 512 = 512 bytes
 8Sector size (logical/physical): 512 bytes / 4096 bytes
 9I/O size (minimum/optimal): 4096 bytes / 4096 bytes
10Disklabel type: gpt
11Disk identifier: 05A996E7-F044-4E59-9DED-F0C891E163CE
12
13Device Start End Sectors Size Type
14/dev/sda1 2048 1050623 1048576 512M EFI System
15/dev/sda2 1050624 5244927 4194304 2G Linux swap
16/dev/sda3 5244928 9439231 4194304 2G Solaris boot
17/dev/sda4 9439232 1000215182 990775951 472.4G Solaris root
18
19
20Disk /dev/zd0: 500 MiB, 524288000 bytes, 1024000 sectors
21Units: sectors of 1 * 512 = 512 bytes
22Sector size (logical/physical): 512 bytes / 16384 bytes
23I/O size (minimum/optimal): 16384 bytes / 16384 bytes

The subsequent steps are very simple and will not be demonstrated one by one.

 1➜ ~ mount -v |grep atuin
 2/dev/zd0 on /home/mephisto/.local/share/atuin type ext4 (rw,relatime,stripe=4)
 3➜ ~ df -h ~/.local/share/atuin
 4Filesystem Size Used Avail Use% Mounted on
 5/dev/zd0 452M 5.8M 412M 2% /home/mephisto/.local/share/atuin
 6➜ ~ ls -al ~/.local/share/atuin
 7total 5865
 8drwxr-xr-x 4 mephisto mephisto 4096 Feb 27 15:42 ./
 9drwx------ 83 mephisto mephisto 93 Feb 27 16:53 ../
10drwxrwxr-x 2 mephisto mephisto 4096 Feb 26 16:43 atuin/
11-rw-r--r-- 1 mephisto mephisto 4550656 Feb 27 17:12 history.db
12-rw-r--r-- 1 mephisto mephisto 32768 Feb 27 17:12 history.db-shm
13-rw-r--r-- 1 mephisto mephisto 1384352 Feb 27 17:12 history.db-wal
14-rw-rw-r-- 1 mephisto mephisto 48 Feb 26 16:43 key
15drwx------ 2 mephisto mephisto 16384 Feb 26 16:41 lost+found/

After switching to ext4, it is really smooth and has never been stuck again.

2. Automatically mount when booting

Query uuid

1➜ ~ sudo blkid /dev/zd0
2/dev/zd0: UUID="6a16a600-bd7c-4b6f-b371-d0fb2306263a" BLOCK_SIZE="4096" TYPE="ext4"

Back up /etc/fstab. If it is damaged without backing up, the file system will not be able to be loaded, which will be a bit troublesome to deal with.

1sudo cp /etc/fstab /etc/fstab_bak

The last line is added, "Don't touch other places" unless you know what they are used for. Everyone's system environment is different. Fill in the uuid queried above, separated by tabs.

1➜ ~ cat /etc/fstab | egrep -v '^#'
2UUID=742C-0CA0 /boot/efi vfat umask=0022,fmask=0022,dmask=0022 0 1
3/boot/efi/grub /boot/grub none defaults,bind 0 0
4UUID=fd8b7e16-9901-48c0-b8dc-ab7da1459126 none swap sw 0 0
5UUID=6a16a600-bd7c-4b6f-b371-d0fb2306263a /home/mephisto/.local/share/atuin ext4 defaults 0 0

In this way, the settings are set at startup, and it will automatically mount normally after restarting.

If you want to learn more about the ZFS file system, you can view Gentoo’s zfs documentation

Or check the help documentation

1man zfs
2man zpool

Finally, a friendly reminder: zfs operates datasets, and zpool operates zpolls (ZFS storage pools). Don't get confused and type the wrong command.