Embedded Linux image size

Doing some work on a BeagleBone Black for a customer and had to load up the Debian reference distro to look at a few things. Size is 571MB for their Buster IoT (without graphical desktop) image. The yoe-qt5-image is 57MB – an order of magnitude smaller. For maker/tinkering, kitchen-sink images are great. But for products, a smaller image is much more manageable – especially when doing full image updates in the field, or over the air.

1 Like

You point at something important which in developer community maybe not as much appreciated but as a product management and support is quite well received

Yeah, it takes 15s or so to update a 57MB image, but with 500MB, you transition to a minute+ – a significant difference in the user experience.

and dependencies of packages are deeper and wider when you have such a large rootfs. Since lot of the buildsystems detect packages on platform so debian packages will enable everything in a package which means more surface for problems :slight_smile:

When you need to create images with graphical library and multimedia it’s a nice challenge.

@caiortp welcome to tmpdir community !!

You are absolutely spot on about complex images involving graphics and multimedia is a challange and one of the goal of yoe distro is to solve this issue, where we want to provide yoe-qt5-image which is able to display QT5 apps on drm/wayland and/or eglfs backends. So all BSP related challanges e.g. graphics driver details etc are hidden underneath.

Agreed, getting all the proprietary graphics drivers working can be a challenge! Khem just sorted through the AM335x drivers (BeagleBone Black), so that is working pretty well now with Qt5. We regularly test rPI graphics. I really need to do more testing on i.MX platforms here in the lab, as I don’t have any customers using graphics stacks on those right now.

If you have any specific setups/technologies you are using, we’re always interested in hearing what people are using.

One of the interesting choices for UIs is browser based technology vs something more conventional like Qt5. We’ve done both, and seems like people are still using both.

Is the 57MB image the compressed size or the uncompressed size?

At my previous job we ended up with about 70MB compressed images for one project, but when uncompressed they occupied about 350MB of eMMC using ext4. Firmware updates were pretty quick, usually about 1 minute. This image had a bunch of firmware for other devices in it, qt4 UI, and a handful of off the shelf and custom applications.

With today’s many GB sized eMMC, I really don’t feel that trimming down the size of the rootfs is worth it. And if you’re basing things on Debian, you’re never going to be as small as a custom built distro, but probably you have different values and ideals and that’s OK.

here is the Yoe distro update file and its contents:

[cbrake@ceres yoe-distro]$ ls -l deploy/beaglebone_0.0.1.upd 
-rw-r--r-- 1 cbrake cbrake 67553280 Dec  7 14:24 deploy/beaglebone_0.0.1.upd

[cbrake@ceres yoe-distro]$ cpio -ivt < deploy/beaglebone_0.0.1.upd
-rwxr-xr-x   1 cbrake   cbrake      93418 Dec  7 14:24 am335x-boneblack.dtb
-rwxr-xr-x   1 cbrake   cbrake      93867 Dec  7 14:24 am335x-boneblack-wireless.dtb
-rwxr-xr-x   1 cbrake   cbrake      90594 Dec  7 14:24 am335x-boneblue.dtb
-rwxr-xr-x   1 cbrake   cbrake      89299 Dec  7 14:24 am335x-bone.dtb
-rwxr-xr-x   1 cbrake   cbrake      89626 Dec  7 14:24 am335x-bonegreen.dtb
-rwxr-xr-x   1 cbrake   cbrake      91591 Dec  7 14:24 am335x-bonegreen-wireless.dtb
-rwxr-xr-x   1 cbrake   cbrake     108092 Dec  7 14:24 MLO
-rwxr-xr-x   1 cbrake   cbrake   58638096 Dec  7 14:24 rootfs.tar.xz
-rwxr-xr-x   1 cbrake   cbrake     894844 Dec  7 14:24 u-boot.img
-rw-r--r--   1 cbrake   cbrake        912 Dec  7 14:24 update.sha256
-rw-r--r--   1 cbrake   cbrake          6 Dec  7 14:24 version.txt
-rwxr-xr-x   1 cbrake   cbrake    7361024 Dec  7 14:24 zImage
131940 blocks

This expands to 177MB on disk:

root@beaglebone:~# more /etc/os-release 
NAME="Yoe Linux"
VERSION="2021.12-rc.1 (masai)"
PRETTY_NAME="Yoe Linux 2021.12-rc.1 (masai)"

root@beaglebone:~# df
Filesystem           1K-blocks      Used Available Use% Mounted on
none                    223088         0    223088   0% /dev
/dev/mmcblk1p2          996780    177456    750512  19% /
/dev/mmcblk1p1          100821      8707     92115   9% /boot
/dev/mmcblk1p3          692768        28    642208   0% /data
tmpfs                   251248         0    251248   0% /dev/shm
tmpfs                   100500     10180     90320  10% /run
tmpfs                     4096         0      4096   0% /sys/fs/cgroup
tmpfs                   251248         0    251248   0% /tmp
tmpfs                   251248        12    251236   0% /var/volatile
tmpfs                    50248         0     50248   0% /run/user/1000
tmpfs                    50248         0     50248   0% /run/user/0

This is compared to the BBB Debian image which is 498MB compressed:

[cbrake@ceres bbb]$ ls -l
total 584768
-rw-r--r-- 1 cbrake cbrake 598798048 Dec  6 13:12 bone-debian-10.3-iot-armhf-2020-04-06-4gb.img.xz

and uses 1.8GB on disk:

root@beaglebone:~# more /etc/os-release 
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION="10 (buster)"

root@beaglebone:~# df
Filesystem     1K-blocks    Used Available Use% Mounted on
udev              219204       0    219204   0% /dev
tmpfs              49504    2192     47312   5% /run
/dev/mmcblk0p1   3558936 1894576   1463864  57% /
tmpfs             247512       0    247512   0% /dev/shm
tmpfs               5120       0      5120   0% /run/lock
tmpfs             247512       0    247512   0% /sys/fs/cgroup
tmpfs              49500       0     49500   0% /run/user/0

This Debian image could no doubt be trimmed down quite a bit.

I have only done one Embedded Linux project with Debian, and sorely missed the OE tooling. Debian tooling for embedded has no doubt improved since, but it’s more of a mindset difference as well. With Debian, you typically build everything on the device, which is fine for limited development, but is more difficult to automate in a product development environment with CI pipelines, automated builds, etc. Yocto/OE is designed to cross-compile everything on a workstation/cloud – much more difficult to get set up, but long term is better for product dev teams where you maintain an application for 5+ years. I’ve had OE builds that have run for years with hundreds of releases as applications are updated – they just work – even over many host build system updates because we use a Docker wrapper.

Distributing sw updates also has a similar trade-off. A 500MB image is fine in some scenarios, but to update 100+ systems in the field, some of them OTA, the number of problems you have getting the update image intact to the target is directly proportional to the image size – especially with non-technical users involved.

That all said, I’m not super happy with the complexity of Yocto, so always thinking about something simpler – perhaps will be more of a reality when fewer of the application components are written in C/C++.

If you do any future Debian work, have a look at debos (GitHub - go-debos/debos: Debian OS builder) for creating disk images. It’s very powerful and has spoiled me.

And for upgrades, the previous 70MB compressed image I mentioned used A/B partition updates and it worked great for us. It was built using OE.

Yes, Debian based systems will likely consume more disk space than something built using OE or buildroot (or what ever other custom distro build systems are out there), but the advantage of using Debian is that if a human is going to have to interact with the system directly, there’s already lots of people who know how to do Debian. Even switching things to use the busybox shell instead of bash often throws “experienced” sysadmins for a loop. Debian surely has its faults, but it’s reasonably well known and there’s tons of info online about how to work with it.

It really all depends on what you’re building. The BeagleBoard Org choosing Debian was the right move in my eyes. I was a big proponent of this back around 2012-2013. From a “sell a board and people play with it” point of view, using Debian (or some other desktop-ish distro) is the right move and this is now seemingly the norm for Raspberry Pi and Beagle boards.

Very neat – thanks for the pointer to go-debos!