Skip to main content

squashmount and aufs: Small footprint and high performance

By

Squashmount and aufs: Gentoo on tiny 4GB root!

I came across a post about "squashing" certain directories on a Gentoo install (portage tree, portage db, /usr/src/, etc) much like those minimal-install-cd's use!  AuFS is an improved "union" filesystem.  I don't understand it all, but as a hardware engineer it's kinda like having a dual-port memory (I think).  Given that many of the directories I want to squash also contain many, many tiny files, mashing them all together increases performance on disk reads (supposedly).

My Eee PC 901 still functioning...

I was intrigued as my little netbook has its SDcard slot occupied for quite some time with a 32GB card for the /usr directory (2GB swap space as well!).  I wanted to change that.  I wanted to see if I could actually use my netbook as an "ultra-portable internet-connected enduring-battery system..." and happen to run a really snappy source distro!  

The technique requires a very modern, (possibly) patched kernel and a well-documented script, squashmount.  I'm using aufs-sources-3.16.1, which includes the Gentoo patchset and aufs3 support.  To install, the ebuilds are in the 'mv' overlay.  Just emerge squashmount and aufs-sources.

Fitting onto 4GB?  On a normal quick install I got to almost installing Gnome3, then ran out of room.  I had the base systemd, xorg-server but there's at least 350 packages to go!  I'll try to outline the disk usage at that bloated point:

/usr/src/linux-3.14.14-gentoo ----------------- ~640MB
/usr/portage (excludes distfiles) ------------- ~800MB
/var/db ---------------------------------------- ~38MB

Here's where I'm at with the directories as squashfs's:

/usr/src.mount/src.sfs ---------------------- ~200MB
/usr/portage.mount/portage.sfs -------------- ~175MB
/var/db.mount/db.sfs ------------------------- ~13MB

Of course there will be extra 'changes' files that add a bit, but saving almost 1GB is amazing.  I'm down to 54% usage on my 4GB internal card.

Don't forget to execute these after modifying your /etc/squashmount.pl:
# squashmount create
# systemctl enable squashmount.service
# systemctl start squashmount.service

Here's my squashmount.pl Perl script:
# egrep -v "^[[:cntrl:] ]*[#;]|^$" /etc/squashmount.pl
^^^ I like egrep's ability to remove comments and empty lines!

#!/usr/bin/perl
use Sys::Hostname;
my $hostname = ($ENV{'HOSTNAME'} // hostname());
@order = qw(aufs overlayfs! unionfs-fuse! unionfs??# funionfs??#);
my $defaults = {
COMPRESSION => 'xz', # We could omit this line as xz is default.
COMPOPT_LZ4 => '-Xhc', # We could omit this line as -Xhc is default
COMPOPT_XZ => ['-Xbcj', 'x86'] # Use this in case COMPRESSION => 'xz'
};
my $non_binary = {
COMPOPT_XZ => undef # "-Xbcj x86" is slower for pure text archives
};
@mounts = (
added_hash($defaults, $non_binary,  {
TAG => 'db',
DIR => '/var/db',
FILE => '/var/db.mount/db.sfs',
BACKUP => '.bak', # keep a backup in /var/db.mount/db.sfs.bak
CHANGES => '/var/db.mount/changes',
WORKDIR => '/var/db.mount/workdir',
READONLY => '/var/db.mount/readonly',
THRESHOLD => '30m',
BLOCKSIZE => 65536
}),
standard_mount('kernel', '/usr/src', $defaults),
standard_mount('portage', '/usr/portage', $defaults, $non_binary, {
UMOUNT => ((@umount) ? undef : '-i'),
THRESHOLD => '80m',
FILL => qr{^local/(?!(?:\.git|profiles|metadata)(?:/|$))}
}),
);
my $custom = @ARGV;
my $file = undef;
if($custom) {
$file = pop(@ARGV);
fatal("argument '$file' of --args is not a file") unless(-f $file);
$locking = $storing = 1 # don't set $storing without $locking!
}
$custom ||= have_stored_data('custom');
$before = sub {
my ($mountpoint, $store, $config) = @_;
return 1 unless($mountpoint eq 'custom');
my $stored = $store->{FILE};
if(defined($stored)) {
if(defined($file) && ($stored ne $file)) {
error("stored path $stored",
"differs from --arg $file",
'Use "squashmount stop|forget custom"');
return ''
}
} else {
$store->{FILE} = $stored = $file
}
$config->{FILE} = $stored if(defined($stored));
1 # return a true value!
};
push(@mounts, # append the following to @mounts:
standard_mount('custom', '/var/custom', {
FILE => ($file // '/default.sfs')
})
) if($custom);
'EOF';# The last executed command in this file should be a true expression

The script I use is really just the default shipped example, and uncommented certain sections for 'portage', 'db' and 'src'.  After you stare at it and read it, (especially @mounts) the script is quite self-explanatory via the comments.

Now it's back to finishing the Gnome3 install on this baby...

~JWilly
Labels:

Comments

Post a Comment

Popular posts from this blog

Atmega328PB Dev on Linux x64

By
The Atmega328PB is the greatest thing to come to the Arduino platform since the UNO.  Who wouldn't want more 16-bit PWMs and double all 3 serial protocols? The new unofficial Arduino Mini PB Pro! To get the IDE to support it, follow this guide: https://github.com/watterott/ATmega328PB-Testing Since the JSON file is incomplete for Linux, do the manual toolchain replacement.  This was a little tricky, and there isn't much on the internet on where the toolchain resides inside the Arduino base folder.  Here's what I did to make it work. I downloaded the latest Arduino build, and I will refer to the extracted base directory as "arduino-1.6.9".  Also download the "hardware.zip" and the "Arduino Toolchain (4.9.2)" Linux x64 version from the github links. 1. Replace the toolchain The builtin toolchain resides in "arduino-1.6.9/hardware/tools/avr/".  Rename the avr folder to "avr.old".  Extract the "Arduino Toolc...
Post a Comment
Read more

PB Pro Mini

By
No, it isn't Lead, nor is it Peanut Butter.  It is the 'B' version of the prolific ATmega328P microcontroller which started all this maker craziness off! Top view of PB Pro Mini Description This is a minimal microcontroller board with only a voltage reg, crystal, I2C pull-ups, LEDs and the chip itself.  The goal here is to reduce costs while hosting the beefy 328PB! This is based on, and pin compatible with, the Arduino Pro Mini 5v/16MHz from  Sparkfun . Notable, not-so-minor improvements for the 'B' version: 2 UARTs 2 I2Cs (TWI) 2 SPI Qtouch 5 Timers (with OC, IC, and PWM) Everything else is about the same.  You should be aware the clock driver on the chip is a low-power version, but this board and crystal operate at 16MHz just fine at 5V.  The Arduino library I've been using for the PB has been working without a problem! Getting Started You'll need a FTDI Friend  or FTDI Cable  to program this as there is no seconda...
Post a Comment
Read more

Update to USBmicroISP: 6pin and 10pin headers

By
Actually using my USBmicroISP with microcontrollers other than Arduino Uno's led me to fix a udev rule issue with my Gentoo Linux laptop.  I was getting Permission Denied when I was trying to use avrdude as a normal user.  Yes, my user is part of the 'uucp' group... Background Some background on the Arduino ISP connector.  It is much different than the 10-pin AVR connector... 6-pin ICSP cable pinout.  NOTE: Numbers refer to ATtiny45/85 pins, not ICSP! Arduino ISP pinout. Notice the vertical line next to pin 1, which appears in silkscreen on the PCBs. Redboard ISP pinout.  Obviously the same as UNO.  Notice pin 1 location on upper-left, with horizontal white silk. My USBmicroISP has the cables coming off the IDC differently, but the key is in the correct orientation to the pinout in the top picture! (Please disregard numbering on first picture for ICSP purposes)  Usually the key and wires are facing inward (left side on 2nd and 3rd ...
Post a Comment
Read more