[./Andrei_Gherzan]> \(
bash: (: command not found

64bit kernel on Raspberry Pi

by Andrei Gherzan 2019-07-07 Reading time: 3 minutes

TL;DR

About

Raspberry Pi LogoI should start this braindump by trying to address the question Why?. It is not an easy one. The performance impact is minimal. While maintaining the Yocto Raspberry Pi BSP layer, I have heard people saying that running video applications shows some noticeable performance gain - but that is with kernel and user-space on 64bit. This braindump doesn’t address the user-space part as it is based on Raspbian. If you want to go full 64, probably the easiest way forward remains Yocto/buildroot. Otherwise consider this an exercise for running your benchmarks or giving 64bit kernel a try.

A. Compile aarch64 toolchain

Let’s first create a workspace where we will build our toolchain:

mkdir -p toolchains/aarch64
cd toolchains/aarch64
export TOOLCHAIN=`pwd` # Used later to reference the toolchain location

First of all, we will need to compile a set of tools. The first bit is binutils which provides, in summary, a linker and an assembler. So we will download the current latest version (tested on 2.32) and compile it. Feel free to try a newer version. It shouldn’t have any impact (fingers crossed).

cd "$TOOLCHAIN"
wget https://ftp.gnu.org/gnu/binutils/binutils-2.32.tar.bz2
tar -xf binutils-2.32.tar.bz2
mkdir binutils-2.32-build
cd binutils-2.32-build
../binutils-2.32/configure --prefix="$TOOLCHAIN" --target=aarch64-linux-gnu --disable-nls
make -j4
make install

Now you have binutils in place. The next step is a C compiler, GCC. We will be compiling a very minimal compiler for C that is enough for our use-case. Feel free though to modify the configuration line as wanted or even download a newer version if available.

cd "$TOOLCHAIN"
wget https://ftp.gnu.org/gnu/gcc/gcc-9.1.0/gcc-9.1.0.tar.gz
tar -xf gcc-9.1.0.tar.gz
mkdir gcc-9.1.0-build
cd gcc-9.1.0-build

GCC will have some dependencies so make sure you have them installed on your host. For example, on Ubuntu you will need:

sudo apt-get install libgmp-dev libmpfr-dev libmpc-dev

As soon as you have all the needed dependencies installed we can proceed with GCC compilation. If there are more dependencies missing just rerun configure until it finishes successfully.

../gcc-9.1.0/configure --prefix="$TOOLCHAIN" --target=aarch64-linux-gnu --with-newlib --without-headers --disable-nls --disable-shared --disable-threads --disable-libssp --disable-decimal-float --disable-libquadmath --disable-libvtv --disable-libgomp --disable-libatomic --enable-languages=c
make all-gcc -j4
make install-gcc

B. Build the Raspberry Pi kernel

Before starting the compilation there are some packages we need on the host. Here is an example on Ubuntu:

sudo apt-get install bison flex

Same as above, if running the configure below fails asking for other dependencies, act accordingly: depending on your distro find the corresponding package and install it.

The current stable kernel branch (on the Raspberry Pi fork) is rpi-4-19-y so this is the one we will be using below. As development continues, modify the branch after checking the GitHub repository.

Raspberry Pi 3 and 4 are both 64bit capable and thus have a specific defconfig available in the kernel sources. We will need the name of the defconfig when configuring the kernel below. These are not the only defconfigs that can be used but are the official, tested ones for these boards:

That said, let’s compile a Raspberry Pi 64bit kernel:

git clone https://github.com/raspberrypi/linux.git rpi-linux
cd rpi-linux
git checkout origin/rpi-4.19.y # change the branch name for newer versions
mkdir kernel-build
PATH=$PATH:$TOOLCHAIN/bin make O=./kernel-build/ ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu-  thiswontwork_defconfig # change the name of the defconfig with the one for the targeted board
PATH=$PATH:$TOOLCHAIN/bin make -j4 O=./kernel-build/ ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu-
export KERNEL_VERSION=`cat ./kernel-build/include/generated/utsrelease.h | sed -e 's/.*"\(.*\)".*/\1/'` # we extract and export the kernel version to be able to correctly deploy the modules below when deploying them on the Raspbian image
make -j4 O=./kernel-build/ DEPMOD=echo MODLIB=./kernel-install/lib/modules/${KERNEL_VERSION} INSTALL_FW_PATH=./kernel-install/lib/firmware modules_install
depmod --basedir kernel-install "$KERNEL_VERSION"
export KERNEL_BUILD_DIR=`realpath kernel-build` # used if you want to deploy it to Raspbian, ignore otherwise

This will get us a kernel image, the associated kernel modules and a device tree.

C. Deploy the compiled 64bit kernel on a Raspbian image

You now have all the bits to proceed in modifying a Raspbian image to boot a Raspberry Pi board in 64bit mode. First thing first, download Raspbian and burn it with any tool you prefer. I’d use etcher. Mount the boot partition and copy kernel image and the kernel modules on it. The following commands assume that the boot partition mount point is /run/media/me/booti and the root filesystem mount point is /run/media/me/rootfs. Also, the paths rely on the exports set throughout this braindump.

cp $KERNEL_BUILD_DIR/arch/arm64/boot/Image /run/media/me/boot/kernel8.img
cp $KERNEL_BUILD_DIR/arch/arm64/boot/dts/broadcom/*.dtb /run/media/me/boot/
cp -r $KERNEL_BUILD_DIR/kernel-install/* /run/media/me/rootfs/

Be aware that Raspberry Pi 4 needs some additional quirks which are detailed in this braindump. This will be addressed soon but for now make sure you carry on those additional changes.

sync
umount /run/media/me/boot /run/media/me/rootfs

Pop the microSD card in your board, attach a serial console, boot, login and

uname -a
Linux raspberrypi 4.19.56-v8+ #1 SMP PREEMPT Thu Jul 4 12:43:20 BST 2019 aarch64 GNU/Linux

Everything should be up and running, that is a Raspbian 32bit user-space on a 64bit linux kernel.