x

Resolving Root Partition Space Issues and Timeshift Snapshot Errors on Manjaro Linux

Incident Report

Incident ID: 001

Date and Time: June 17, 2024

Affected System: Lenovo Legion Slim 5 16AHP9, Manjaro Linux, Btrfs Filesystem

Summary:

The root partition on a Lenovo Legion Slim 5 running Manjaro Linux filled up due to large Timeshift snapshots. Attempts to delete specific Timeshift snapshots failed, resulting in persistent error messages.

Steps Taken:

  1. Initial Assessment:

    • Checked disk usage and identified that the root partition had less than 2GB free.

      df -h

      Output:

      Filesystem      Size  Used Avail Use% Mounted on
dev             7.5G     0  7.5G   0% /dev
run             7.5G  2.4M  7.5G   1% /run
efivarfs        148K   76K   68K  53% /sys/firmware/efi/efivars
/dev/nvme0n1p5  100G   98G  1.5G  99% /
tmpfs           7.5G  8.0K  7.5G   1% /dev/shm
/dev/nvme0n1p5  100G   98G  1.5G  99% /var/cache
/dev/nvme0n1p5  100G   98G  1.5G  99% /var/log
tmpfs           7.5G   13M  7.5G   1% /tmp
/dev/nvme0n1p7  836G  524G  310G  63% /home
/dev/nvme0n1p1  256M   46M  211M  18% /boot/efi
tmpfs           1.5G  224K  1.5G   1% /run/user/1000
/dev/nvme0n1p5  100G   98G  1.5G  99% /run/timeshift/218936/backup

    • Identified any snapshots or subvolumes that might be taking up space

      sudo btrfs subvolume list /

      Output:

      ID 256 gen 44711 top level 5 path timeshift-btrfs/snapshots/2024-06-16_22-56-43/@
ID 257 gen 45654 top level 5 path @cache
ID 258 gen 45673 top level 5 path @log
ID 276 gen 45673 top level 5 path timeshift-btrfs/snapshots/2024-06-08_15-21-11/@
ID 375 gen 44130 top level 256 path timeshift-btrfs/snapshots/2024-06-16_22-56-43/@/mnt/chroot/audacity
ID 376 gen 40225 top level 375 path timeshift-btrfs/snapshots/2024-06-16_22-56-43/@/mnt/chroot/audacity/var/lib/portables
ID 377 gen 40225 top level 375 path timeshift-btrfs/snapshots/2024-06-16_22-56-43/@/mnt/chroot/audacity/var/lib/machines
ID 384 gen 44727 top level 5 path @

    • Verified Timeshift snapshots and identified unnecessary snapshots occupying space.

      sudo timeshift --list

      Output:

      Mounted '/dev/nvme0n1p5' at '/run/timeshift/222221/backup'
btrfs: Quotas are not enabled
Device : /dev/nvme0n1p5
UUID   : a06ee1a0-a91c-4cca-98ac-8c8a10bf034a
Path   : /run/timeshift/222221/backup
Mode   : BTRFS
Status : OK
2 snapshots, 1.5 GB free

Num     Name                 Tags  Description                             
------------------------------------------------------------------------------
0    >  2024-06-08_15-21-11  O     Pre-installation of audacity snapshot   
1    >  2024-06-16_22-56-43  O     Before restoring '2024-06-08 15:21:11'

  2. Snapshot Deletion Attempts:

    • Attempted to delete snapshots using Timeshift CLI commands. 
      sudo timeshift --delete --snapshot '2024-06-16_22-56-43'
      Output: 
      Removing snapshot: 2024-06-16_22-56-43
Deleting subvolume: @ (Id:256)
E: ERROR: Could not destroy subvolume/snapshot: Directory not empty

E: Failed to delete snapshot subvolume: '/run/timeshift/222274/backup/timeshift-btrfs/snapshots/2024-06-16_22-56-43/@'
E: Failed to remove snapshot: 2024-06-16_22-56-43

  3. Manual Cleanup:

    • Listed Btrfs subvolumes. 
      sudo btrfs subvolume list /
      Output: 
      ID 256 gen 45691 top level 5 path timeshift-btrfs/snapshots/2024-06-16_22-56-43/@
ID 257 gen 45675 top level 5 path @cache
ID 258 gen 45699 top level 5 path @log
ID 276 gen 45698 top level 5 path timeshift-btrfs/snapshots/2024-06-08_15-21-11/@
ID 375 gen 45677 top level 256 path timeshift-btrfs/snapshots/2024-06-16_22-56-43/@/mnt/chroot/audacity
ID 376 gen 45677 top level 375 path timeshift-btrfs/snapshots/2024-06-16_22-56-43/@/mnt/chroot/audacity/var/lib/portables
ID 377 gen 45677 top level 375 path timeshift-btrfs/snapshots/2024-06-16_22-56-43/@/mnt/chroot/audacity/var/lib/machines
ID 384 gen 45683 top level 5 path @
    • Manually deleted nested subvolumes. 
      sudo btrfs subvolume delete /run/timeshift/223262/backup/timeshift-btrfs/snapshots/2024-06-16_22-56-43/@/mnt/chroot/audacity/var/lib/portables
sudo btrfs subvolume delete /run/timeshift/223262/backup/timeshift-btrfs/snapshots/2024-06-16_22-56-43/@/mnt/chroot/audacity/var/lib/machines
sudo btrfs subvolume delete /run/timeshift/223262/backup/timeshift-btrfs/snapshots/2024-06-16_22-56-43/@/mnt/chroot/audacity
sudo btrfs subvolume delete /run/timeshift/223262/backup/timeshift-btrfs/snapshots/2024-06-16_22-56-43/@
      Output: 
      ERROR: Could not statfs: No such file or directory
    • Cleared contents of snapshot directories to enable deletion. 
      sudo rm -rf /run/timeshift/223262/backup/timeshift-btrfs/snapshots/2024-06-16_22-56-43/@/*
      Output: 
      [no output, directory cleared successfully]
    • Successfully deleted the targeted snapshot subvolumes using Btrfs commands.

  4. Unmount and Verify:

    • Unmounted Timeshift backup directory to ensure no processes were using the path. 
      sudo umount /run/timeshift/223262/backup
      Output: 
      umount: /run/timeshift/223262/backup: target is busy.
    • Identified and killed processes using the directory. 
      sudo lsof | grep '/run/timeshift/223262/backup'
sudo killall nautilus
sudo umount /run/timeshift/223262/backup
      Output: 
      [no output, directory unmounted successfully]
    • Verified the removal of unnecessary snapshots and updated Timeshift state. 
      sudo timeshift --list
      Output: 
      Mounted '/dev/nvme0n1p5' at '/run/timeshift/224602/backup'
btrfs: Quotas are not enabled
Device : /dev/nvme0n1p5
UUID   : a06ee1a0-a91c-4cca-98ac-8c8a10bf034a
Path   : /run/timeshift/224602/backup
Mode   : BTRFS
Status : OK
1 snapshots, 38.1 GB free

Num     Name                 Tags  Description                            
------------------------------------------------------------------------------
0    >  2024-06-08_15-21-11  O     Pre-installation of audacity snapshot  

E: Failed to remove directory
Ret=256

  5. Resolution:

    • Deleted the problematic snapshot manually. 
      sudo rm -rf /run/timeshift/223262/backup/timeshift-btrfs/snapshots/2024-06-16_22-56-43
sudo btrfs subvolume delete /run/timeshift/223262/backup/timeshift-btrfs/snapshots/2024-06-16_22-56-43/@
      Output: 
      Delete subvolume 256 (no-commit): '/run/times
↑