Flutter for Embedded Linux

A GUI for a touch screen embedded Linux device is on my todo-list, and Flutter keeps surfacing in my research of GUI tools.

As a test drive, I installed Flutter on a Windows desktop and followed the demo app instructions.

Android Studio has nice integration with Flutter when configured with the Flutter and Dart plugins:

From the dropdown I can deploy my app to Chrome, Edge, a connected Android tablet, Windows desktop, or an Android simulator.

Chrome deployment:

Flutter supports hot reload, which makes code testing and fixes efficient.

What about compiling Flutter for embedded Linux? I pushed the starter app to a Github repo and cloned it to a Linux desktop. After installing Flutter and Android Studio for Linux (both were available from the Snap store on Ubuntu 20.04) and configuring Android Studio, I was able to run the same app with no problems on Linux desktop, which was the only option in the drop down this time.

Desktop Linux is a breeze, but some searching indicates that Flutter does not have an official build engine for Arm processors. I did find a Flutter extension by Sony called flutter-elinux. I followed the installation guide - it depends on flutter-embedded-linux (the actual Flutter engine) but downloads it automatically when you run flutter-elinux build.

Create a Flutter elinux app:

  • flutter-elinux create <your_app_project_name>

First I tried running the app with Flutter elinux on the desktop:
Inside the created project folder, run the flutter app:

  • flutter-elinux run -d elinux-x11
    Or equivalently:
  • flutter-elinux build elinux --debug --target-backend-type=x11
  • <your_app_project_name>/build/elinux/x64/debug/bundle/displayui --bundle=./

That worked fine. Now to cross-compile for Arm

  • flutter-elinux build elinux --target-arch=arm64

This error results:

Failed to cmake build: 
Scanning dependencies of target flutter_assemble
[  0%] Built target flutter_assemble
Scanning dependencies of target flutter_wrapper_app
[  7%] Building CXX object flutter/CMakeFiles/flutter_wrapper_app.dir/ephemeral/cpp_client_wrapper/core_implementations.cc.o
[ 15%] Building CXX object flutter/CMakeFiles/flutter_wrapper_app.dir/ephemeral/cpp_client_wrapper/standard_codec.cc.o
[ 23%] Building CXX object flutter/CMakeFiles/flutter_wrapper_app.dir/ephemeral/cpp_client_wrapper/flutter_engine.cc.o
[ 30%] Building CXX object flutter/CMakeFiles/flutter_wrapper_app.dir/ephemeral/cpp_client_wrapper/flutter_view_controller.cc.o
[ 38%] Linking CXX static library libflutter_wrapper_app.a
[ 38%] Built target flutter_wrapper_app
Scanning dependencies of target flutter_wrapper_plugin
[ 46%] Building CXX object flutter/CMakeFiles/flutter_wrapper_plugin.dir/ephemeral/cpp_client_wrapper/core_implementations.cc.o
[ 53%] Building CXX object flutter/CMakeFiles/flutter_wrapper_plugin.dir/ephemeral/cpp_client_wrapper/standard_codec.cc.o
[ 61%] Building CXX object flutter/CMakeFiles/flutter_wrapper_plugin.dir/ephemeral/cpp_client_wrapper/plugin_registrar.cc.o
[ 69%] Linking CXX static library libflutter_wrapper_plugin.a
[ 69%] Built target flutter_wrapper_plugin
Scanning dependencies of target displayui
[ 76%] Building CXX object runner/CMakeFiles/displayui.dir/flutter_window.cc.o
[ 84%] Building CXX object runner/CMakeFiles/displayui.dir/main.cc.o
[ 92%] Building CXX object runner/CMakeFiles/displayui.dir/__/flutter/generated_plugin_registrant.cc.o
[100%] Linking CXX executable displayui

/usr/bin/ld: /home/....../displayui/elinux/flutter/ephemeral/libflutter_engine.so: error adding symbols: file in wrong format
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [runner/CMakeFiles/displayui.dir/build.make:117: runner/displayui] Error 1
make[1]: *** [CMakeFiles/Makefile2:200: runner/CMakeFiles/displayui.dir/all] Error 2
make: *** [Makefile:130: all] Error 2

One thing I need to look in to is setting sysroot with --target-sysroot build option (documentation states that this is required).

This is very useful information – thanks for posting! It looks like you have C++ code failing that is being built with CMAKE, so we should be able to figure this out.

Do you have a cross toolchain installed and configured in your build? How is that set up in flutter-elinux?

@cbrake

The flutter engine and toolchain are supposed to be downloaded automatically when building - is this the toolchain you are referencing?

Pre-built images

flutter-elinux download the Flutter engine artifacts such as libflutter_engine.so and libflutter_elinux_wayland.so to <flutter-elinux_install_path>/flutter/bin/cache/artifacts/engine directory automatically when you build your Flutter app. These .so files are download from sony/flutter-embedded-linux/releases built with a specific toolchain. Therefore, if you want to use your toolchain, you need to build it yourself.

For reference, the toolchain that the current .so files are built is below.

.so file toolchain sysroot glibc
libflutter_engine.so clang/llvm (Google Chromium) Google Chromium 2.29
libflutter_elinux_*.so clang/llvm (Ubuntu 18.04) Ubuntu 18.04 for arm64 2.27

When I look in the directory referenced above, this is the result:

~/displayui$ ls /opt/flutter-elinux/flutter/bin/cache/artifacts/engine/
android-arm64-profile  android-arm-profile  android-x64-profile  common              elinux-arm64-profile  elinux-common     elinux-x64-profile  linux-x64
android-arm64-release  android-arm-release  android-x64-release  elinux-arm64-debug  elinux-arm64-release  elinux-x64-debug  elinux-x64-release

However, there are explicit instructions to download a .so file and put it in the cmake build directory:

Install the Flutter Engine library

This embedder requres libflutter_engine.so (Pre-build library of the Flutter Engine). You need to install it in <path_to_cmake_build_directory> to build. See: Building Flutter Engine embedder

Quick start (Use pre-built image)

You can download a specific pre-built Flutter Engine (libflutter_engine.so) version from Release engine-artifacts-e85ea0e79c · sony/flutter-embedded-linux · GitHub without building it yourself. See also: Use pre-build image

Perhaps this is the issue, because the original error references this file: libflutter_engine.so
And I believe the .so file that the build should be using is: libflutter_elinux_wayland.so @cbrake, is my cmake build directory the top level build directory:

~/displayui$ ls build/
5469bbfab1b44a164799a6dc83acf3bb  c075001b96339384a97db4862b8ab8db.cache.dill.track.dill  elinux

or is it this directory:

~/displayui$ ls build/elinux/arm64/debug/
bundle/              CMakeCache.txt       CMakeFiles/          cmake_install.cmake  flutter/             Makefile             runner/

As a side note, I found these instructions that reference a toolchain file as well:

Cross-build

You need to create a toolchain file to cross compile using the Yocto SDK for aarch64 on x64 hosts. cross-toolchain-aarch64-template.cmake is the templete file for aarch64 toolchain. Also, you need to modify <path_to_user_target_sysroot> appropriately for your environment if you want to use the template file.

$ cmake -DUSER_PROJECT_PATH=<path_to_user_project> -DCMAKE_TOOLCHAIN_FILE= …

Wow, this is getting complex!

What are your thoughts on:

(interesting this is maintained by someone at Toyota: https://github.com/jwinarske)

and

Typically, when we need to cross built stuff that includes C++ code, we either do that in Yocto, or generate a SDK from Yocto with all included library dependencies.

@cbrake That looks really exciting - good to see very recent activity and a wide range of supported hardware:

CI Jobs

kirkstone-agl-renesas-m3.yml - Renesas M3 build. Time boxed GPU driver (30 minutes?). AGL canaray build.

kirkstone-agl-x86_64.yml - meta-flutter QEMU image used with tools/seup_flutter_workspace.py. Test build for AGL downstream work.

kirkstone-imx8mmevk.yml - NXP imx8mmevk baseline Wayland image.

kirkstone-linux-dummy.yml - Tests all recipes in the layer without a dummy Linux kernel (save build time).

kirkstone-qc-dragonboard.yml - DB410C and DB820C Wayland images.

kirkstone-rpi-zero2w-64.yml - RPI Zero2W Wayland image (flutter-auto). Farily full featured with Network Manager, BT, WiFi, etc.

kirkstone-stm32mp15.yml - eglfs (flutter-pi) st-core-image+SDK and wayland (flutter-auto) st-core-image+SDK.

Summary

Using the flutter-elinux repository worked for desktop (as did the regular flutter install) but I am having continued issues in cross-compiling it for arm64.

Last-ditch effort

This failed as well - I plan to move forward with flutter as a Yocto layer. Setup:

Side note: there are a lot of threads like this that discuss interest to make cross-compilation for arm a standard feature of flutter or of go-flutter.

Yeah, I like that meta-flutter/meta-flutter appears to have support for building everything from source. Looking at the flutter engine:

image

So this is still very much a C++ project, thus you have all the C++ build baggage to deal with. Yocto has good Clang integration thanks to @khem.

There are two ways to approach these projects:

  1. start at the application dev and worry about the OS build/integration later.

  2. start at the OS build, and then extract a way to do app dev after you get that working.

#1 tends to lead to lots of hacks and stuff like downloading toolchain binaries, pre-compiled libraries, etc.

#2 I feel is a better approach as the entire build is automated. The system part is the hard part so you want to make sure that is covered before you settle on a technology.

If you can’t build everything (including toolchains, etc) in an automated way, you do not have a sustainable way to maintain the platform over time, because too much manual hackery is required and things are always changing – pre-built libraries will soon be out of date, etc.

I’ve not looked into this extensively, but on the surface, it appears Sony has taken approach #1 and Toyota, approach #2. However, I could be wrong.

This is neat:

Also looked at go-flutter a bit – wow, what a rabbit hole of technology.

This is all neat, but still seems to be very complex. For systems that don’t need 3D UI, ironically I think Chromium in full-screen Kiosk mode and a web UI is probably the simplest way to add a UI to a i.MX8 class device. For all its faults, the web platform is at least stable and your app does not require linking directly to graphics libraries.

its a good idea to have cross platform UI but I have read that it does not do as good a job on iOS as it does on android

One interesting thing about Flutter Web is they don’t use HTML/CSS much – they do everything in a Canvas element. Below is about the extent of the entire HTML in the page.

Looking at the network assets, it does appear they are using WASM:

It is a little heavier load than compared to other frontend technologies.

In the examples I’ve tried, I also feel the Flutter web demos are still slow and don’t feel near as performant as native web technologies. This is probably expected as browsers have been optimized for years for HTML/CSS/JS.

So cross-platform is a nice idea, but in practice it is full of trade-offs.

People complain about the how bad mobile apps implemented with web technologies are – maybe I’m missing something, but they seem acceptable to me. As an example, running Discourse on a phone as an app works really well. However, there are limits in terms of APIs as to what the web platform supports.

The interesting question to me is will Flutter become a “platform,” like the web, or just another one-off UI library that will be gone in X years. The Web platform has stood the test of time, so I don’t think it will be going anywhere soon. Browsers are almost an OS in themselves and offer security models that users trust. When I run an application in my browser, I don’t worry too much about security. If I download a native app, I have to ask the question – do I trust this app? Browsers have standard models for accessibility if you use the platform. They have solved many of the hard problems in that they abstract the UI from the base system – this has a lot of value. There are times where you need native performance or APIs, but know what these reasons are.

This following is a good article that discusses some of the past cross platform efforts and provides an analysis of Flutter (TLDR – the author likes Flutter and thinks it will probably be around long term):

flutter-pi getting started documentation

Result on a Raspberry Pi 3 B+:

2 Likes

Nice! Is this on raspbian?

Yes, it’s rasbian bullseye.

I started a flutter app test build in my rPI build:

set up a yoe build
cd sources
git clone https://github.com/meta-flutter/meta-flutter
vi conf/bblayers.conf (add sources/meta-flutter)
yoe_setup
bitbake flutter-gallery

flutter-engine-runtimerelease took a really long time to download.

But then it compiled something, so hopefully it was not downloading *.so files …

The resulting package is 76MB:

[cbrake@ceres yoe-distro]$ ls -l build/tmp/deploy/ipk/cortexa72/flutter-gallery_git-r0.0_cortexa72.ipk 
-rw-r--r-- 2 cbrake cbrake 76452758 Aug 30 11:47 build/tmp/deploy/ipk/cortexa72/flutter-gallery_git-r0.0_cortexa72.ipk

The resulting control file in the package does not show any dependencies:

Package: flutter-gallery
Version: git-r0.0
Description: Flutter Gallery Application
Flutter Gallery Application
Section: graphics
Priority: optional
Maintainer: Yoe Distro Community <http://yoedistro.org>
License: BSD-3-Clause
Architecture: cortexa72
OE: flutter-gallery
Homepage: https://github.com/flutter/gallery
Source: flutter-gallery_git.bb

Sizes of some of the files:

[cbrake@ceres tmp]$ ls -l usr/share/flutter/gallery/lib/libapp.so 
-rw-r--r-- 1 cbrake cbrake 23790488 Nov  5  2021 usr/share/flutter/gallery/lib/libapp.so
[cbrake@ceres packages]$ pwd
/scratch/yoe/yoe-distro/tmp/usr/share/flutter/gallery/data/flutter_assets/packages
[cbrake@ceres packages]$ du --max-depth=1 -m
71      ./flutter_gallery_assets
1       ./rally_assets
1       ./cupertino_icons
6       ./shrine_images
17      ./flutter_localized_locales
93      .

There seems to be a large number of images in the app package.

Its odd there does not seem to be a runnable exe – apparently something else loads the libapp.so file or something?

The fact that this built out of the box is pretty amazing.

Next step is to get an image that includes the app …

In looking through the flutter recipes, I was puzzled by the gn:// prefix in the SRC_URI variables.

It turns out this is the Google GN (generate ninja) build tool.

https://gn.googlesource.com/gn/

GN is used as the build system for Chromium, Fuchsia, Flutter, etc.

Here is an example of a GN build file:

The Bitbake fetcher source is located in:

(thanks @khem for clarifying this)

@cbrake Which recipies were you looking through? recipies-graphics doesn’t seem to have any like that.

The following is the one I noticed: