Skip to main content

squashmount and aufs: Small footprint and high performance

By

Squashmount and aufs: Gentoo on tiny 4GB root!

I came across a post about "squashing" certain directories on a Gentoo install (portage tree, portage db, /usr/src/, etc) much like those minimal-install-cd's use!  AuFS is an improved "union" filesystem.  I don't understand it all, but as a hardware engineer it's kinda like having a dual-port memory (I think).  Given that many of the directories I want to squash also contain many, many tiny files, mashing them all together increases performance on disk reads (supposedly).

My Eee PC 901 still functioning...

I was intrigued as my little netbook has its SDcard slot occupied for quite some time with a 32GB card for the /usr directory (2GB swap space as well!).  I wanted to change that.  I wanted to see if I could actually use my netbook as an "ultra-portable internet-connected enduring-battery system..." and happen to run a really snappy source distro!  

The technique requires a very modern, (possibly) patched kernel and a well-documented script, squashmount.  I'm using aufs-sources-3.16.1, which includes the Gentoo patchset and aufs3 support.  To install, the ebuilds are in the 'mv' overlay.  Just emerge squashmount and aufs-sources.

Fitting onto 4GB?  On a normal quick install I got to almost installing Gnome3, then ran out of room.  I had the base systemd, xorg-server but there's at least 350 packages to go!  I'll try to outline the disk usage at that bloated point:

/usr/src/linux-3.14.14-gentoo ----------------- ~640MB
/usr/portage (excludes distfiles) ------------- ~800MB
/var/db ---------------------------------------- ~38MB

Here's where I'm at with the directories as squashfs's:

/usr/src.mount/src.sfs ---------------------- ~200MB
/usr/portage.mount/portage.sfs -------------- ~175MB
/var/db.mount/db.sfs ------------------------- ~13MB

Of course there will be extra 'changes' files that add a bit, but saving almost 1GB is amazing.  I'm down to 54% usage on my 4GB internal card.

Don't forget to execute these after modifying your /etc/squashmount.pl:
# squashmount create
# systemctl enable squashmount.service
# systemctl start squashmount.service

Here's my squashmount.pl Perl script:
# egrep -v "^[[:cntrl:] ]*[#;]|^$" /etc/squashmount.pl
^^^ I like egrep's ability to remove comments and empty lines!

#!/usr/bin/perl
use Sys::Hostname;
my $hostname = ($ENV{'HOSTNAME'} // hostname());
@order = qw(aufs overlayfs! unionfs-fuse! unionfs??# funionfs??#);
my $defaults = {
COMPRESSION => 'xz', # We could omit this line as xz is default.
COMPOPT_LZ4 => '-Xhc', # We could omit this line as -Xhc is default
COMPOPT_XZ => ['-Xbcj', 'x86'] # Use this in case COMPRESSION => 'xz'
};
my $non_binary = {
COMPOPT_XZ => undef # "-Xbcj x86" is slower for pure text archives
};
@mounts = (
added_hash($defaults, $non_binary,  {
TAG => 'db',
DIR => '/var/db',
FILE => '/var/db.mount/db.sfs',
BACKUP => '.bak', # keep a backup in /var/db.mount/db.sfs.bak
CHANGES => '/var/db.mount/changes',
WORKDIR => '/var/db.mount/workdir',
READONLY => '/var/db.mount/readonly',
THRESHOLD => '30m',
BLOCKSIZE => 65536
}),
standard_mount('kernel', '/usr/src', $defaults),
standard_mount('portage', '/usr/portage', $defaults, $non_binary, {
UMOUNT => ((@umount) ? undef : '-i'),
THRESHOLD => '80m',
FILL => qr{^local/(?!(?:\.git|profiles|metadata)(?:/|$))}
}),
);
my $custom = @ARGV;
my $file = undef;
if($custom) {
$file = pop(@ARGV);
fatal("argument '$file' of --args is not a file") unless(-f $file);
$locking = $storing = 1 # don't set $storing without $locking!
}
$custom ||= have_stored_data('custom');
$before = sub {
my ($mountpoint, $store, $config) = @_;
return 1 unless($mountpoint eq 'custom');
my $stored = $store->{FILE};
if(defined($stored)) {
if(defined($file) && ($stored ne $file)) {
error("stored path $stored",
"differs from --arg $file",
'Use "squashmount stop|forget custom"');
return ''
}
} else {
$store->{FILE} = $stored = $file
}
$config->{FILE} = $stored if(defined($stored));
1 # return a true value!
};
push(@mounts, # append the following to @mounts:
standard_mount('custom', '/var/custom', {
FILE => ($file // '/default.sfs')
})
) if($custom);
'EOF';# The last executed command in this file should be a true expression

The script I use is really just the default shipped example, and uncommented certain sections for 'portage', 'db' and 'src'.  After you stare at it and read it, (especially @mounts) the script is quite self-explanatory via the comments.

Now it's back to finishing the Gnome3 install on this baby...

~JWilly
Labels:

Comments

Post a Comment

Popular posts from this blog

Atmega328PB Dev on Linux x64

By
The Atmega328PB is the greatest thing to come to the Arduino platform since the UNO.  Who wouldn't want more 16-bit PWMs and double all 3 serial protocols? The new unofficial Arduino Mini PB Pro! To get the IDE to support it, follow this guide: https://github.com/watterott/ATmega328PB-Testing Since the JSON file is incomplete for Linux, do the manual toolchain replacement.  This was a little tricky, and there isn't much on the internet on where the toolchain resides inside the Arduino base folder.  Here's what I did to make it work. I downloaded the latest Arduino build, and I will refer to the extracted base directory as "arduino-1.6.9".  Also download the "hardware.zip" and the "Arduino Toolchain (4.9.2)" Linux x64 version from the github links. 1. Replace the toolchain The builtin toolchain resides in "arduino-1.6.9/hardware/tools/avr/".  Rename the avr folder to "avr.old".  Extract the "Arduino Toolc...
Post a Comment
Read more

PB Pro Mini

By
No, it isn't Lead, nor is it Peanut Butter.  It is the 'B' version of the prolific ATmega328P microcontroller which started all this maker craziness off! Top view of PB Pro Mini Description This is a minimal microcontroller board with only a voltage reg, crystal, I2C pull-ups, LEDs and the chip itself.  The goal here is to reduce costs while hosting the beefy 328PB! This is based on, and pin compatible with, the Arduino Pro Mini 5v/16MHz from  Sparkfun . Notable, not-so-minor improvements for the 'B' version: 2 UARTs 2 I2Cs (TWI) 2 SPI Qtouch 5 Timers (with OC, IC, and PWM) Everything else is about the same.  You should be aware the clock driver on the chip is a low-power version, but this board and crystal operate at 16MHz just fine at 5V.  The Arduino library I've been using for the PB has been working without a problem! Getting Started You'll need a FTDI Friend  or FTDI Cable  to program this as there is no seconda...
Post a Comment
Read more

Geode LX800 cash-in

By
I discovered an AMD Geode processor lurking in a slim client SBC (single board computer) originally for a printer kiosk.  I wonder if I could fit a TinyGentoo  onto its 128MB compact flash card??? Introduction and overview I will utilize crossdev, and avoid any chrooting problems and mismatching CHOST/CFLAGS... Customize the target systems make.conf generated by crossdev. Compile and install packages into runtime tree. [picture of slim client and screen] >>>I'm writing this while my Gentoo system cross-compiles a i486-pc-linux-uclibc toolchain into /usr/i486-pc-linux-uclibc. There are many resources that I had to pull together to get this working: http://en.gentoo-wiki.com/wiki/Tiny_Gentoo  HOWTO guide to install a 5MB system on a USB flashdrive or similar http://www.gentoo.org/proj/en/base/embedded/handbook/index.xml?full=1  Gentoo Embedded Handbook details cross-compiling for various dev boards http://judepereira.com/blog/going-...
Post a Comment
Read more