Login   |   1.866.392.4897 |   sales@timesys.com English Japanese German French Korean Chinese (Simplified) Chinese (Traditional)
Securing U-Boot: A Guide to Mitigating Common Attack Vectors

Securing U-Boot: A Guide to Mitigating Common Attack Vectors

Overview

Many embedded systems implementing software authentication (secure boot and chain of trust) use U-Boot as their bootloader. Making sure this bootloader is properly secured so that someone cannot bypass your chain of trust and boot unauthenticated software is very important. We have commonly seen field-deployed embedded systems with secure boot setups which fail to mitigate against the direct execution of unauthenticated software from U-Boot. To help prevent these sorts of attacks, we suggest considering three main categories of attack surface reduction: environmental tampering protection, software authentication, and command line access limitation.

If you are new to investigating and understanding these issues, feel free to skim over some of the more verbose example blocks and patchset information. These are not strictly necessary to gain an initial understanding.

 

Software Authentication and Signing

First and foremost, you’ll want to make sure your first bootloader (i.e. U-Boot) is signed and being authenticated by your processor’s ROM through whatever mechanism is provided. On NXP processors, this is called High Assurance Boot (HAB). We’re going to jump ahead and assume this has been completed. Once completed, we can focus on creating a complete chain of trust for the following boot stages.

Now, once this signed U-Boot has started execution, we’ll need U-Boot to properly check that any following stages are signed correctly before proceeding to boot into them. This can be processor dependent, wherein some silicon vendors provide a ROM-based API for authenticating signed binaries. Those specific mechanisms (e.g. NXP AHAB containerization) are out of the scope of this article, so we’ll assume a common approach is being used. We tend to use a Flattened Image Tree (FIT) to bundle together the following boot stages. U-Boot then has built in mainline mechanisms which can be used to sign and authenticate the entire FIT bundle before booting.

A FIT image is defined by its Image Tree Source (ITS) file. This ITS file tends to look something like:

/dts-v1/;

/ {
        description = "U-Boot fitImage for Poky (Yocto Project Reference Distro)/1.0/imx8qxp-b0-mek";
        #address-cells = <1>;

        images {
                kernel-1 {
                        description = "Linux kernel";
                        data = /incbin/("Image");
                        type = "kernel";
                        arch = "arm64";
                        os = "linux";
                        compression = "none";
                        load = <0x80200000>;
                        entry = <0x80200000>;
                        hash-1 {
                                algo = "sha1";
                        };
                };
                fdt-1 {
                        description = "Flattened Device Tree blob";
                        data = /incbin/("imx8qxp-mek-ov5640-rpmsg.dtb");
                        type = "flat_dt";
                        arch = "arm64";
                        compression = "none";
                        load = <0x83000000>;
                        entry = <0x83000000>;
                        hash-1 {
                                algo = "sha1";
                        };
                };
                ramdisk-1 {
                        description = "timesys-initramfs-imx8qxp-b0-mek.cpio.gz";
                        data = /incbin/("timesys-initramfs-imx8qxp-b0-mek.cpio.gz");
                        type = "ramdisk";
                        arch = "arm64";
                        os = "linux";
                        compression = "gzip";
                        load = <0xd0000000>;
                        entry = <0xd0000000>;
                        hash-1 {
                                algo = "sha1";
                        };
                };
        };

        configurations {
                default = "conf-1";
                conf-1 {
                        description = "Linux kernel, FDT blob, ramdisk";
                        kernel = "kernel-1";
                        fdt = "fdt-1";
                        ramdisk = "ramdisk-1";
                        hash-1 {
                                algo = "sha1";
                        };
                };
        };
};

In here, we see there is a main configuration node which contains entries for the Linux kernel image, device tree blob, and initramfs.

This ITS is then compiled into the fitImage file via uboot-mkimage:

uboot-mkimage -D "-I dts -O dtb -p 2000" -f image.its fitImage

Then, we can sign this fitImage file with:

uboot-mkimage -D "-I dts -O dtb -p 2000" -F -k "/key_directory" -r fitImage

Where /key_directory is a directory which contains your RSA key pair for signing the fitImage. These can be generated using OpenSSL by:

cd /key_directory
openssl genpkey -algorithm RSA -out dev.key -pkeyopt rsa_keygen_bits:2048 -pkeyopt rsa_keygen_pubexp:65537
openssl req -batch -new -x509 -key dev.key -out dev.crt

You’ll also need U-Boot to be set up for FIT image booting and signing, otherwise uboot-mkimage will throw an error:

#ifdef CONFIG_FIT_SIGNATURE
	fprintf(stderr,
		"Signing / verified boot options: [-k keydir] [-K dtb] [ -c <comment>] [-p addr] [-r] ...\n"
		"          -k => set directory containing private keys\n"
		"          -K => write public keys to this .dtb file\n"
		"          -G => use this signing key (in lieu of -k)\n"
		"          -c => add comment in signature node\n"
		"          -F => re-sign existing FIT image\n"
		"          -p => place external data at a static position\n"
		"          -r => mark keys used as 'required' in dtb\n"
		"          -N => openssl engine to use for signing\n");
#else
	fprintf(stderr,
		"Signing / verified boot not supported (CONFIG_FIT_SIGNATURE undefined)\n");
#endif

To do this, you can set these config options in U-Boot:

CONFIG_SECURE_BOOT=y
CONFIG_FIT=y
CONFIG_FIT_SIGNATURE=y
CONFIG_FIT_VERBOSE=y
CONFIG_DEFAULT_FDT_FILE="u-boot-signed-devicetree.dtb"

Then, once U-Boot is compiled, we can add the public key into U-Boot’s compiled DTB. To do that, we’ll first need to create a dummy FIT image using this dummy ITS file:

/dts-v1/;

/ {
    description = "U-Boot Simple fitImage";
    #address-cells = <1>;

    images {
        dummy-1 {
            description = "dummy";
            data = /incbin/("empty_placeholder_file");
            type = "kernel";
            arch = "arm";
            os = "linux";
            compression = "none";
            load = <0x80008000>;
            entry = <0x80008000>;
            hash-1 {
                algo = "sha1";
            };
        };
    };

    configurations {
        default = "conf-1";
        conf-1 {
            description = "dummy";
            dummy = "dummy-1";
			hash-1 {
				algo = "sha1";
			};
			signature-1 {
				algo = "sha1,rsa2048";
				key-name-hint = "dev";
				sign-images = "dummy";
			};
        };
    };
};

Note: it is important to have the signature-1 node inside the configuration node here, instead of putting signature nodes on each image piece. With this layout, all of the associated image hashes are contained within the configuration signature, so someone cannot swap in different signed images and perform a version rollback attack by changing one image/binary in your FIT bundle separately.

We then package this dummy ITS into a file named simpleFitImage:

uboot-mkimage -D "-I dts -O dtb -p 2000" -f simple.its simpleFitImage

Then we can sign our actual U-Boot DTB, while using the simpleFitImage as a reference (for the signature-1 node):

uboot-mkimage -D "-I dts -O dtb -p 2000" -F -k "/key_directory" -K imx8.dtb -r simpleFitImage

If we decompile the resulting, signed DTB:

dtc -I dtb -O dts -o imx8_decompiled.dts imx8.dtb

Inside, we now see this node:

signature {
    key-dev {
        required = "conf";
        algo = "sha1,rsa2048";
        rsa,r-squared = <0x26b42979 0xf91dba64 0x11c5cab5 0x8273b76e 0xdc7562f3 0xcdd3742c ... etc>;
        rsa,modulus = <0xb4aac057 0xbddc7ce8 0x3c4d48b3 0x622d6e95 0xb09eb6c6 0xafc3c9d7 ... etc>;
        rsa,exponent = <0x00 0x10001>;
        rsa,n0-inverse = <0x5a7322b9>;
        rsa,num-bits = <0x800>;
        key-name-hint = "dev";
    };
};

Now, when booting via bootm, if the signature is bad/missing you should see an error similar to this:

## Loading kernel from FIT Image at ... ...
   Using 'conf@1' configuration
   Verifying Hash Integrity ... sha1,rsa2048:dev_bad- Failed to verify required signature 'key-dev'
Bad Data Hash

If there are no errors, each node should show a correct signature check similar to this:

## Loading kernel from FIT Image at ... ...
   Using 'conf-1' configuration
   Verifying Hash Integrity ... sha1,rsa2048:dev+ OK
   Trying 'kernel-1' kernel subimage

U-Boot’s Environment Pitfalls

This is one of the reasons U-Boot is so commonly liked. The entire boot flow is setup and controlled through environmental parameters. When U-Boot boots, it runs the specified set of commands listed inside the bootcmd parameter. So, by modifiying this, we can easily boot into alternate images or quickly make minor modifications to the boot process. When it comes to security, this becomes a double-edged sword. In a field-deployed embedded system, we do not want someone to be able to tamper with the environment and arbitrarily execute whatever U-Boot commands they would like.

So, how can we prevent this? Unfortunately, U-Boot does not offer an easy way to sign/authenticate or encrypt the environment. It’s usually easier to disable the other exploitable paths someone may use.

To start we can limit access to the U-Boot command line interface (CLI) via:

  • Disabling/Password Protecting Autoboot Interrupt
  • Disabling the serial console

Once those two are done, if your U-Boot environment never needs to change, you can make sure it is not stored in nonvolatile memory by setting this in your U-Boot configuration:

CONFIG_ENV_IS_NOWHERE=y

With these measures in place, there’s little left for an attacker to turn to when trying to modify your environment. They may still be able to perform a more cumbersome attack such as RAM modification/injection via JTAG or another means, but these attack vectors should also be limited (i.e. Make sure you disable JTAG in accordance if your processor’s reference manual).

If you do need your environment to remain modifiable due to something such as software update management, this becomes a bit trickier. With the environment stored in a nonvolatile device, you’re now subject to offline tampering of the storage device. This can still be mitigated against quite well by disabling any dangerous U-Boot commands. If all of the enabled U-Boot commands are benign (i.e. properly require signed software and cannot be used to modify memory), then you’re safe from environmental tampering too.

Now, with a cursory understanding of these environment-related pitfalls, let’s look into securing them.

 

Autoboot Interruption

I’m sure you’ve seen it before… When U-Boot starts, the serial console displays a 3 second countdown. If you enter a keystroke, you’re taken to U-Boot’s command line interface. So, if this is left enabled, anyone with access to your serial pins can easily stop U-Boot’s autoboot sequence and tamper with everything that’s left available to them (environment modification, unprotected boot commands, etc).

To disable autoboot interruption entirely, you’ll want to set this in your U-Boot configuration:

CONFIG_BOOTDELAY=-2

Note: This does not entirely prevent command prompt access. If a Linux/OS boot fails, U-boot may fall into the CLI. This is why it is still important to disable the serial console entirely. Or, at least patch U-Boot so that it will not enter the CLI after a failed autoboot sequence (appending the reset command to the end of your boot sequence can sometimes work as a fall through fail-safe).

 

Autoboot Password Protection

If disabling autoboot interruption is too extreme for your use case, you can add a sha256-backed interruption password. Be sure to make this string as long as possible, to avoid brute forcing (20+ characters!).

This can be performed by enabling the following in your U-Boot configuration:

 

CONFIG_AUTOBOOT_KEYED=y
CONFIG_AUTOBOOT_ENCRYPTION=y
CONFIG_AUTOBOOT_STOP_STR_SHA256="..."

As previously mentioned, if a Linux/OS boot fails, U-boot may still open up the CLI. So, this does not necessarily offer full protection.

 

Command Line Disablement

You can also consider disabling the U-Boot command line by turning this off:

# CONFIG_CMD_CMDLINE is not set

Most customers still want some form of the CLI left enabled for configuration and software update handling, so this is not an option we see commonly used. With this disabled, when a command is entered/run, U-Boot falls into this block:

__weak int board_run_command(const char *cmdline)
{
	printf("## Commands are disabled. Please enable CONFIG_CMDLINE.\n");

	return 1;
}

Console Disablement

To entirely disable the U-Boot console, append this to your defconfig:

CONFIG_DISABLE_CONSOLE=y

You’ll then need to set this in arch_cpu_init(or another corresponding function) to turn it on:

gd->flags |= GD_FLG_SILENT | GD_FLG_DISABLE_CONSOLE;

So for example,

From: Nathan Barrett-Morrison <nathan.morrison@timesys.com>
Date: Tue, 12 Apr 2022 19:04:34 -0400
Subject: [PATCH] Disable u-boot console

---
 arch/arm/cpu/armv7/sc58x/soc.c | 2 ++
 configs/sc589-ezkit_defconfig  | 1 +
 2 files changed, 3 insertions(+)

diff --git a/arch/arm/cpu/armv7/sc58x/soc.c b/arch/arm/cpu/armv7/sc58x/soc.c
index c4a4845114..87fa369de4 100644
--- a/arch/arm/cpu/armv7/sc58x/soc.c
+++ b/arch/arm/cpu/armv7/sc58x/soc.c
@@ -34,6 +34,8 @@ void v7_outer_cache_enable(void)
 
 int arch_cpu_init(void)
 {
+     gd->flags |= GD_FLG_SILENT | GD_FLG_DISABLE_CONSOLE;
+
 #ifdef CONFIG_DEBUG_EARLY_SERIAL
     return serial_early_init();
 #else
diff --git a/configs/sc589-ezkit_defconfig b/configs/sc589-ezkit_defconfig
index 7b978aeded..4da70a8f8f 100644
--- a/configs/sc589-ezkit_defconfig
+++ b/configs/sc589-ezkit_defconfig
@@ -27,3 +27,4 @@ CONFIG_SPI=y
 CONFIG_USB=y
 CONFIG_USB_MUSB_HCD=y
 CONFIG_OF_LIBFDT=y
+CONFIG_DISABLE_CONSOLE=y

Kernel Command Line Parameters

You should also make sure that the kernel command line parameters which U-Boot passes to the kernel (bootargs in U-Boot’s environment) cannot be modified to anything unexpected. If modification is allowed, someone can easily pass an unexpected argument to a driver or even set init= or rdinit= to /bin/sh to gain access to a shell.

I like to do this by checking that the bootargs match an expected string. So, if we have two sets of acceptable boot arguments, we might do:

From: Nathan Barrett-Morrison <nathan.morrison@timesys.com>
Date: Mon, 1 Nov 2021 16:39:01 -0400
Subject: [PATCH] Add in basic tamper detection for u-boot's bootargs variable,
 so that someone can not modify kernel boot arguments

---
 common/bootm.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/common/bootm.c b/common/bootm.c
index db4362a643..ca913ce945 100644
--- a/common/bootm.c
+++ b/common/bootm.c
@@ -524,6 +524,14 @@ int do_bootm_states(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[],
     boot_os_fn *boot_fn;
     ulong iflag = 0;
     int ret = 0, need_boot_fn;
+    static char* bootargs_a = "root=/dev/mmcblk2p2 console=ttymxc0,115200 rootwait rw";
+    static char* bootargs_b = "root=/dev/mmcblk2p3 console=ttymxc0,115200 rootwait rw";
+    char* bootargs = env_get("bootargs");
+
+    if( (strcmp(bootargs_a, bootargs) != 0) && (strcmp(bootargs_b, bootargs) != 0) ){
+        printf("\nDetected tampering of bootargs: blocking...\n");
+        while(1);
+    }
 
     images->state |= states;

Concerning Commands

Here’s a non-comprehensive list of U-Boot configuration settings which could be disabled to reduce your attack surface:

U-Boot Configuration Description
CONFIG_CMD_GO This is the equivalent of an assembly jump/branch operation. It allows an attacker to change execution to any arbitrary address.
CONFIG_CMD_BOOTI
CONFIG_CMD_BOOTZ
CONFIG_CMD_BOOTEFI
CONFIG_CMD_ELF
CONFIG_CMD_ABOOTIMG
CONFIG_CMD_ADTIMG
Assuming we’re using the signed FIT image strategy, these should be disabled (as FIT uses CMD_BOOTM only). These commands open alternate boot paths (booti, bootz, bootelf, bootvx, bootefi, android boot images)
CONFIG_CMD_MEMORY Enables memory dumping (md), memory writing (mw), and other memory operations
CONFIG_CMD_SMC
CONFIG_CMD_HVC
Enables injecting secure monitor calls. This could be concerning if you’re using ATF-A + OP-TEE
CONFIG_CMD_NET
CONFIG_CMD_USB
CONFIG_USB_STORAGE
CONFIG_CMD_BOOTP
CONFIG_CMD_TFTPBOOT
These can be used to externally load images from USB devices, network transfers, etc
CONFIG_CMD_REMOTEPROC
CONFIG_CMD_ICC
CONFIG_CMD_FPGA
Enables controlling secondary cores and FPGAs
CONFIG_CMD_IMI Enables dumping image info (iminfo)
CONFIG_CMD_I2C
CONFIG_CMD_SPI
Leaving this enabled may allow an attacker to modify your I2C/SPI/etc devices. This could give access to sorts of devices, including power management units.
CONFIG_CMD_DIAG
CONFIG_CMD_IRQ
CONFIG_CMD_BDI
and more
I would classify these as unnecessary information leakage. While not explicitly bad, they may give an attacker information you don’t want them to have (such as stack pointer locations, memory sizes, etc).

Again, this is not a complete list. In fact, it ultimately may be better to create a whitelist of known, acceptable commands and blacklist everything else. If you don’t need a command, disable it!

Also, given we’re booting via a signed FIT image, this uses the bootm command. I like to further secure this command by deleting any alternate boot paths from the code (in case someone mistakenly leaves the associated CONFIG options enabled).

To make sure bootm requires an authenticated FIT image, I do the following:

From: Nathan Barrett-Morrison <nathan.morrison@timesys.com>
Date: Wed, 25 Aug 2021 07:38:05 -0400
Subject: [PATCH] Make FIT the only bootm option

---
 cmd/bootm.c | 13 ++-----------
 1 file changed, 2 insertions(+), 11 deletions(-)

diff --git a/cmd/bootm.c b/cmd/bootm.c
index 03ea3b8998..d164f71572 100644
--- a/cmd/bootm.c
+++ b/cmd/bootm.c
@@ -163,17 +163,8 @@ int do_bootm(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[])
 #else
 
     switch (genimg_get_format((const void *)image_load_addr)) {
-#if defined(CONFIG_LEGACY_IMAGE_FORMAT)
-    case IMAGE_FORMAT_LEGACY:
-        if (authenticate_image(image_load_addr,
-            image_get_image_size((image_header_t *)image_load_addr)) != 0) {
-            printf("Authenticate uImage Fail, Please check\n");
-            return 1;
-        }
-        break;
-#endif
-#ifdef CONFIG_ANDROID_BOOT_IMAGE
-    case IMAGE_FORMAT_ANDROID:
+#ifdef CONFIG_FIT
+    case IMAGE_FORMAT_FIT:
         /* Do this authentication in boota command */
         break;
 #endif

In newer versions of U-Boot, most this logic has moved to common/bootm.c, under boot_get_kernel() and bootm_find_os().

 

Real World Example

Our customers commonly send us boards with field deployment issues. A lot of times, these boards have been permanently secured using some form of secure boot technology. In these cases, I try to avoid asking for possession of the associated private keys, as that opens up more attack vectors for the customer (if someone were to compromise one of my machines/etc). So, I’m sometimes left in a position where it would be nice to run a new software image, but I cannot do so because it’s unsigned. Sometimes we have to wait for the customer to sign the new software image for us… and sometimes, we find another way.

Here’s a recent board I received:

U-Boot dub-2017.03-r11.2+gf9055c2 (Mar 14 2022 - 12:42:45 +0000)

CPU:   Freescale i.MX6DL rev1.3 at 792MHz
CPU:   Industrial temperature grade (-40C to 105C) at 35C
Reset cause: POR
I2C:   ready
DRAM:  512 MiB
MMC:   FSL_SDHC: 0, FSL_SDHC: 1
In:    serial
Out:   serial
Err:   serial
Model: ...
Board: ...
Boot device: MMC4
PMIC:  DA9063, Device: 0x61, Variant: 0x60, Customer: 0x00, Config: 0x56
Net:   Board Net Initialization Failed
No ethernet found.
Hit any key to stop autoboot:  0 
=> 

It has an external SD card slot… so let’s see if we could easily change the MMC boot device from its internal eMMC to the external SD card.

Can we switch between devices?

=> mmc dev 0
switch to partitions #0, OK
mmc0(part 0) is current device
=> mmc dev 1
switch to partitions #0, OK
mmc1 is current device

Yep, that worked, device 1 is our SD card.

What about changing the boot device?

=> printenv bootcmd
bootcmd=if run loadscript; then setexpr bs_ivt_offset ${filesize} - 0x4020;if hab_auth_img ${loadaddr} ${bs_ivt_offset}; then source ${loadaddr};fi; fi;
=> printenv loadscript
loadscript=load mmc ${mmcbootdev}:${mmcpart} ${loadaddr} ${script}
=> printenv mmcbootdev
mmcbootdev=0
=> editenv mmcbootdev
edit: 1
## Error: Can't overwrite "mmcbootdev"
## Error inserting "mmcbootdev" variable, errno=1

We can see U-Boot appears to be hardened against changing the MMC boot device. Interesting.

What if we just directly run a modified boot command instead of modifying the environment? I happen to have a kernel image and device tree built for this board on my SD card already:

=> fatls mmc 1:1
  5710976   zImage-imx6.bin
    51503   zImage-imx6dl-imx6.dtb
     2430   boot.scr 
=> fatload mmc 1:1 ${loadaddr} zImage-imx6.bin
reading zImage-imx6.bin
5710976 bytes read in 289 ms (18.8 MiB/s)
=> printenv loadaddr
loadaddr=0x12000000
=> fatload mmc 1:1 0x18000000 zimage-imx6dl.dtb
reading zimage-imx6dl-ccimx6-iotest.dtb
51503 bytes read in 31 ms (1.6 MiB/s)
=> bootz 0x12000000 - 0x18000000                          
Kernel image @ 0x12000000 [ 0x000000 - 0x572480 ]
## Flattened Device Tree blob at 18000000
   Booting using the fdt blob at 0x18000000
   Authenticating image from DDR location 0x18000000... FAILED!
hab entry function fail

Secure boot enabled

We can see U-Boot also appears to be hardened against booting an unsigned kernel image. What else can we try?

=> go
go - start application at address 'addr'

Usage:
go addr [arg ...]
    - start application at address 'addr'
      passing 'arg' as arguments

go was left enabled on this board and most likely does not contain any signature checking.

Let’s try to jump into a custom built U-Boot version using go. First, let’s dump some memory info.

=> bdinfo
arch_number = 0x00001323
boot_params = 0x10000100
DRAM bank   = 0x00000000
-> start    = 0x10000000
-> size     = 0x20000000
current eth = unknown
ip_addr     = 192.168.42.30
baudrate    = 115200 bps
TLB addr    = 0x2FFF0000
relocaddr   = 0x2FF4E000
reloc off   = 0x1874E000
irq_sp      = 0x2EF3DBA0
sp start    = 0x2EF3DB90
Early malloc usage: ec / 400

Okay, so they have 512MB of RAM ranging from 0x10000000 to 0x30000000. We can see most of U-Boot has also been relocated to the upper region of memory. This is important to know, as booting another instance of U-Boot requires not trampling over the current stack/bss/etc.

Lets trick U-Boot into thinking it only has 256MB of RAM and rearrange some addresses so the new instance of U-Boot will not overlap any of these regions:

From: Nathan Barrett-Morrison <nathan.morrison@timesys.com>
Date: Thu, 14 Apr 2022 14:28:59 -0400
Subject: [PATCH] Allow U-Boot to boot from another currently running version
 of U-Boot via 'go'

---
 arch/arm/imx-common/hab.c       | 4 ++++
 board/vendor/imx6/imx6.c      | 2 +-
 common/board_f.c                | 7 ++++---
 common/board_r.c                | 4 ++++
 include/configs/imx6_common.h | 5 ++++-
 5 files changed, 17 insertions(+), 5 deletions(-)

diff --git a/arch/arm/imx-common/hab.c b/arch/arm/imx-common/hab.c
index fc970641d1..ec699967c8 100644
--- a/arch/arm/imx-common/hab.c
+++ b/arch/arm/imx-common/hab.c
@@ -629,6 +629,10 @@ static int validate_ivt(int ivt_offset, ulong start_addr)
 
 uint32_t authenticate_image(uint32_t ddr_start, uint32_t image_size)
 {
+    //Disable HAB security!    
+    //Always return true, even for unauthenticated images
+    return 1;
+
     ulong load_addr = 0;
     size_t bytes;
     ptrdiff_t ivt_offset = 0;
diff --git a/board/vendor/imx6/imx6.c b/board/vendor/imx6/imx6.c
index 3ec90f0369..58d580eb21 100644
--- a/board/vendor/imx6/imx6.c
+++ b/board/vendor/imx6/imx6.c
@@ -252,7 +252,7 @@ static struct imx6_variant imx6_variants[] = {
 /* 0x13 - 55001818-19 */
     {
         IMX6DL,
-        MEM_512MB,
+        MEM_256MB,




diff --git a/common/board_f.c b/common/board_f.c
index 7e40a35bb1..1e87cd1b89 100644
--- a/common/board_f.c
+++ b/common/board_f.c
@@ -359,13 +359,14 @@ static int setup_dest_addr(void)
      * thie mechanism. If memory is split into banks, addresses
      * need to be calculated.
      */
-    gd->ram_size = board_reserve_ram_top(gd->ram_size);
+    //Force the RAM size to 256MB
+    gd->ram_size = 0x10000000;
 
 #ifdef CONFIG_SYS_SDRAM_BASE
     gd->ram_top = CONFIG_SYS_SDRAM_BASE;
 #endif
-    gd->ram_top += get_effective_memsize();
-    gd->ram_top = board_get_usable_ram_top(gd->mon_len);
+    //Force the top of RAM to be at 0x20000000 instead of 0x30000000
+    gd->ram_top = 0x20000000;
     gd->relocaddr = gd->ram_top;
     debug("Ram top: %08lX\n", (ulong)gd->ram_top);
 #if defined(CONFIG_MP) && (defined(CONFIG_MPC86xx) || defined(CONFIG_E500))
diff --git a/common/board_r.c b/common/board_r.c
index 2b14e3d9f8..14747f1b4b 100644
--- a/common/board_r.c
+++ b/common/board_r.c
@@ -487,6 +487,10 @@ static int should_load_env(void)
 
 static int initr_env(void)
 {
+    //Always use the default environment -- don't read from nonvolatile storage
+    set_default_env(NULL);
+    return 0;
+
     /* initialize environment */
     if (should_load_env())
         env_relocate();
diff --git a/include/configs/imx6_common.h b/include/configs/imx6_common.h
index 7061a473d8..c21b0926ce 100644
--- a/include/configs/imx6_common.h
+++ b/include/configs/imx6_common.h
@@ -41,11 +41,14 @@
 /*
  * RAM
  */
+//Limit the amount of memory we're allowed to map to 256MB
+#define CONFIG_MAX_MEM_MAPPED       0x10000000
 #define CONFIG_LOADADDR             0x12000000
 #define CONFIG_SYS_LOAD_ADDR        CONFIG_LOADADDR
 #define CONFIG_DIGI_LZIPADDR        0x15000000
 #define CONFIG_DIGI_UPDATE_ADDR     CONFIG_LOADADDR
-#define CONFIG_SYS_TEXT_BASE        0x17800000
+//Move the starting text base to a lower region
+#define CONFIG_SYS_TEXT_BASE        0x12800000
 /* RAM memory reserved for U-Boot, stack, malloc pool... */
 #define CONFIG_UBOOT_RESERVED       (10 * 1024 * 1024)
 /* Size of malloc() pool */

Now, we build and store this U-Boot image on our SD card at an offset of 0x1000. Along with our custom Linux kernel, DTB, and file system.

U-Boot 2017.03-r11.2+gf9055c2 (Mar 14 2022 - 12:42:45 +0000)
...
=> mmc dev 1; mmc read 0x12800000 8 1000; go 0x12800000

U-Boot 2017.03-r2.3+g2002510765 (Mar 31 2022 - 22:52:18 +0000)
...
=> 

We’ve done it! We’re running an unsigned version of U-Boot!

Finishing the chain, we can boot all the way into Linux via:

U-Boot 2017.03-r2.3+g2002510765 (Mar 31 2022 - 22:52:18 +0000)
...
=> setenv mmc dev 1; setenv mmcroot /dev/mmcblk1p2; run mmcboot

Yocto 2.4-r3 imx6 /dev/ttymxc3
imx6 login: root

root@imx6:~# whoami
root

Command Whitelisting

So, what if you had a known subset of commands which were considered safe? How would you create a whitelist within U-Boot for this? A simple whitelist can be added to cmd_call() via something like this:

From: Nathan Barrett-Morrison <nathan.morrison@timesys.com>
Date: Mon, 20 Sep 2021 15:56:23 -0400
Subject: [PATCH] Lockdown U-boot to allow only a whitelist of commands to be
 used

---
 common/command.c                  | ...

diff --git a/common/command.c b/common/command.c
index 0d8bf244be..125dabd162 100644
--- a/common/command.c
+++ b/common/command.c
@@ -13,6 +13,7 @@
 #include <console.h>
 #include <env.h>
 #include <linux/ctype.h>
+#include <u-boot/sha256.h>
 
 /*
  * Use puts() instead of printf() to avoid printf buffer overflow
@@ -556,6 +557,42 @@ int cmd_discard_repeatable(cmd_tbl_t *cmdtp, int flag, int argc,
     return cmdtp->cmd_rep(cmdtp, flag, argc, argv, &repeatable);
 }
 
+//Create a whitelist of commands that can always be run inside U-Boot
+static char * customer_whitelist_table[] = {
+    "run",
+    "echo",
+    "bmode",
+    "fastboot",
+    "setenv",
+    "saveenv",
+    "mmc",
+    "bootm",
+    "ext4load",
+    "customer_authenticate",
+};
+
+#define CUSTOMER_WHITELIST_LENGTH ARRAY_SIZE(customer_whitelist_table)
+
+extern bool imx_hab_is_enabled(void);
+
+//Check if the current function name is within the whitelist
+static int customer_cmd_whitelist(char * name)
+{
+    if (imx_hab_is_enabled()){
+        for(int i = 0; i < CUSTOMER_WHITELIST_LENGTH; i++)
+        {
+            if(strcasecmp(customer_whitelist_table[i], name) == 0)
+            {
+                return 0;
+            }
+        }
+        printf("CUSTOMER Error: Attempted to run %s while unauthenticated\r\n", name);
+        return -1;
+    }else{
+        return 0;
+    }
+}
+
 /**
  * Call a command function. This should be the only route in U-Boot to call
  * a command, so that we can track whether we are waiting for input or
@@ -571,6 +608,13 @@ int cmd_discard_repeatable(cmd_tbl_t *cmdtp, int flag, int argc,
 static int cmd_call(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[],
             int *repeatable)
 {
+    //Only execute commands if they're in the whitelist or we're authenticated
+    if(customer_authentication != 1){
+        if(customer_cmd_whitelist(cmdtp->name) != 0){
+            return -1;
+        }
+    }
+
     int result;
 
     result = cmdtp->cmd_rep(cmdtp, flag, argc, argv, repeatable);

Note: imx_hab_is_enabled() is checking if secure boot is enabled on an NXP processor and will vary for you. Also, customer_authentication is another command that I will discuss more below. It’s used to give our customers the ability to disable the whitelist if a password is entered.

In cases where our customers do not want the entire CLI disabled, this will allow them to enter the CLI and then run ‘customer_authenticate password’ in order to bypass the whitelist and unlock all of U-Boot’s commands.

To do this, we’ll first enable autoboot password interruption again:

CONFIG_AUTOBOOT_KEYED=y
CONFIG_AUTOBOOT_ENCRYPTION=y
CONFIG_AUTOBOOT_STOP_STR_SHA256="..."

We then modify the passwd_abort_sha256 function to allow us to externally hook into it by passing in a password string. This string will be what is sent in from the password portion of ‘customer_authenticate password’.

From: Nathan Barrett-Morrison <nathan.morrison@timesys.com>
Date: Mon, 20 Sep 2021 15:56:23 -0400
Subject: [PATCH] Modify passwd_abort_sha256 so we can pass in an arbitrary password for verification

---
 common/autoboot.c                 | ...

diff --git a/common/autoboot.c b/common/autoboot.c
index 0a59b81ae2..13ea153531 100644
--- a/common/autoboot.c
+++ b/common/autoboot.c
@@ -75,7 +75,7 @@ static int slow_equals(u8 *a, u8 *b, int len)
  * @etime: Timeout value ticks (stop when get_ticks() reachs this)
  * @return 0 if autoboot should continue, 1 if it should stop
  */
-static int passwd_abort_sha256(uint64_t etime)
+static int passwd_abort_sha256(uint64_t etime, char * password)
 {
     const char *sha_env_str = env_get("bootstopkeysha256");
     u8 sha_env[SHA256_SUM_LEN];
@@ -109,32 +109,57 @@ static int passwd_abort_sha256(uint64_t etime)
      * generate the sha256 hash upon each input character and
      * compare the value with the one saved in the environment
      */
-    do {
-        if (tstc()) {
-            /* Check for input string overflow */
-            if (presskey_len >= MAX_DELAY_STOP_STR) {
-                free(presskey);
-                free(sha);
-                return 0;
-            }
 
-            presskey[presskey_len++] = getc();
+    if(password != NULL){
+        //This adds in ability to verify an arbitrary password string
 
-            /* Calculate sha256 upon each new char */
-            hash_block(algo_name, (const void *)presskey,
-                   presskey_len, sha, &size);
+        /* Calculate sha256 upon each new char */
+        hash_block(algo_name, (const void *)password,
+               strlen(password), sha, &size);
 
-            /* And check if sha matches saved value in env */
-            if (slow_equals(sha, sha_env, SHA256_SUM_LEN))
-                abort = 1;
-        }
-    } while (!abort && get_ticks() <= etime);
+        /* And check if sha matches saved value in env */
+        if (slow_equals(sha, sha_env, SHA256_SUM_LEN))
+            abort = 1;
+    }else{
+        do {
+            if (tstc()) {
+                /* Check for input string overflow */
+                if (presskey_len >= MAX_DELAY_STOP_STR) {
+                    free(presskey);
+                    free(sha);
+                    return 0;
+                }
+
+                presskey[presskey_len++] = getc();
+
+                /* Calculate sha256 upon each new char */
+                hash_block(algo_name, (const void *)presskey,
+                       presskey_len, sha, &size);
+
+                /* And check if sha matches saved value in env */
+                if (slow_equals(sha, sha_env, SHA256_SUM_LEN))
+                    abort = 1;
+            }
+        } while (!abort && get_ticks() <= etime);
+    }
 
     free(presskey);
     free(sha);
+
+    //1 = Authentication successful
+    //0 = Authentication failed
+    customer_authentication = abort;
+
     return abort;
 }
 
+//New function to allow us to hook pre-existing password
+//verification infrastructure with a passed string pointer
+int passwd_abort_sha256_string(char * password)
+{
+    passwd_abort_sha256(0, password);
+}
+

On this NXP processor, I also modified the autoboot interruption password to only be enabled while secure boot (HAB) is enabled. So, during development, you can still easily interrupt U-Boot with a single keystroke.

From: Nathan Barrett-Morrison <nathan.morrison@timesys.com>
Date: Mon, 20 Sep 2021 15:56:23 -0400
Subject: [PATCH] Only check the autoboot password if HAB is enabled

 common/autoboot.c                 | ...

...

 /**
  * passwd_abort_key() - check for a key sequence to aborted booting
  *
@@ -189,6 +214,7 @@ static int passwd_abort_key(uint64_t etime)
      */
     do {
         if (tstc()) {
+            return 1; //Abort if any key is pressed (until HAB fuses are burned)
             if (presskey_len < presskey_max) {
                 presskey[presskey_len++] = getc();
             } else {
@@ -220,6 +246,8 @@ static int passwd_abort_key(uint64_t etime)
     return abort;
 }
 
+extern bool imx_hab_is_enabled(void);
+
 /***************************************************************************
  * Watch for 'delay' seconds for autoboot stop or autoboot delay string.
  * returns: 0 -  no key string, allow autoboot 1 - got key string, abort
@@ -236,9 +264,8 @@ static int abortboot_key_sequence(int bootdelay)
      */
     printf(CONFIG_AUTOBOOT_PROMPT, bootdelay);
 #  endif
-
-    if (IS_ENABLED(CONFIG_AUTOBOOT_ENCRYPTION))
-        abort = passwd_abort_sha256(etime);
+    if (imx_hab_is_enabled() && IS_ENABLED(CONFIG_AUTOBOOT_ENCRYPTION))
+        abort = passwd_abort_sha256(etime, NULL);
     else
         abort = passwd_abort_key(etime);
     if (!abort)

And finally, we can add in the customer_authenticate command via this patch:

From: Nathan Barrett-Morrison <nathan.morrison@timesys.com>
Date: Mon, 20 Sep 2021 15:56:23 -0400
Subject: [PATCH] Add in customer_authenticate

---
 cmd/Makefile                      | ...
 cmd/customer.c                    | ...
 include/u-boot/sha256.h           | ...

diff --git a/cmd/Makefile b/cmd/Makefile
index 7c62e3becf..9a2836d8fb 100644
--- a/cmd/Makefile
+++ b/cmd/Makefile
@@ -155,6 +155,10 @@
 obj-$(CONFIG_CMD_FASTBOOT) += fastboot.o
 obj-$(CONFIG_CMD_FS_UUID) += fs_uuid.o
 
 obj-$(CONFIG_CMD_USB_MASS_STORAGE) += usb_mass_storage.o
+
+# Customer - Customer Custom Commands
+obj-y += customer.o
+
 obj-$(CONFIG_CMD_USB_SDP) += usb_gadget_sdp.o
 obj-$(CONFIG_CMD_THOR_DOWNLOAD) += thordown.o
 obj-$(CONFIG_CMD_XIMG) += ximg.o
diff --git a/cmd/customer.c b/cmd/customer.c
new file mode 100644
index 0000000000..d02e0bd4ae
--- /dev/null
+++ b/cmd/customer.c
@@ -0,0 +1,31 @@
+/*
+ * Copyright 2000-2009
+ * Wolfgang Denk, DENX Software Engineering, wd@denx.de.
+ *
+ * SPDX-License-Identifier:    GPL-2.0+
+ */
+
+#include <common.h>
+#include <command.h>
+#include <u-boot/sha256.h>
+
+uint8_t customer_authentication = 0;
+
+static int do_customer_authenticate(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[])
+{
+    if (argc > 1){
+        passwd_abort_sha256_string(argv[1]);
+        if(customer_authentication){
+            printf("Customer: Authentication Successful\r\n");
+        }else{
+            printf("Customer: Authentication Failed\r\n");
+        }
+    }
+    return 0;
+}
+
+U_BOOT_CMD(
+    customer_authenticate,    2,    1,    do_customer_authenticate,
+    "Command Authentication for Customer",
+    ""
+);

diff --git a/include/u-boot/sha256.h b/include/u-boot/sha256.h
index 6fbf542f67..6ae12193dc 100644
--- a/include/u-boot/sha256.h
+++ b/include/u-boot/sha256.h
@@ -5,6 +5,7 @@
 #define SHA256_DER_LEN    19
 
 extern const uint8_t sha256_der_prefix[];
+extern uint8_t customer_authentication;
 
 /* Reset watchdog each time we process this many bytes */
 #define CHUNKSZ_SHA256    (64 * 1024)
@@ -25,4 +26,7 @@ void sha256_csum_wd(const unsigned char *input, unsigned int ilen,
 void sha256_hmac(const unsigned char *key, int keylen,
         const unsigned char *input, unsigned int ilen,
         unsigned char *output);
+
+extern int passwd_abort_sha256_string(char * password);
+
 #endif /* _SHA256_H */

On this particular board, fastboot was left enabled as well. So, we’ll want to further lock down fastboot by incorporating our customer_authenticate mechanism:

From: Nathan Barrett-Morrison <nathan.morrison@timesys.com>
Date: Mon, 20 Sep 2021 16:27:05 -0400
Subject: [PATCH] Lockdown fastboot commands as well

---
 drivers/fastboot/fb_command.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/drivers/fastboot/fb_command.c b/drivers/fastboot/fb_command.c
index 3c4acfecf6..1fdb7544c0 100644
--- a/drivers/fastboot/fb_command.c
+++ b/drivers/fastboot/fb_command.c
@@ -108,6 +108,15 @@ int fastboot_handle_command(char *cmd_string, char *response)
 
     for (i = 0; i < FASTBOOT_COMMAND_COUNT; i++) {
         if (!strcmp(commands[i].command, cmd_string)) {
+
+            //If not authenticated, this disables all commands except UCmd.
+            //UCmd must remain available to allow for "ucmd customer_authenticate <password>" authentication
+            if(customer_authenticate != 1){
+                if(strcasecmp(commands[i].command, "UCmd:") != 0){
+                    break;
+                }
+            }
+
             if (commands[i].dispatch) {
                 commands[i].dispatch(cmd_parameter,
                             response);

Conclusion

I hope this post was a helpful tool for getting you started with securing your embedded U-Boot implementations. Security is a constant battle, and some of these suggestions and guidelines will most definitely morph and evolve as U-Boot development continues in the open source world. Nevertheless, this post should provide a good foundation for hardening many different versions of U-Boot. If you find a version of U-Boot in which some of these suggestions appear to be not applicable, feel free to ask us where they’ve gone.

 

Help!

As always, we offer professional engineering services to help you with your project. Whether you wish for us to integrate our security feature implementation, called VigiShield, into your project or just need another set of hands to help with general embedded software engineering and security, we’re available to help!

DM-Verity Without an Initramfs

DM-Verity Without an Initramfs

Summary

On Linux-based embedded systems implementing software authentication (secure boot and chain of trust), the file system verification is generally performed using an Initial RAM Filesystem (initramfs). Using an initramfs is more straight forward and flexible, as you can more easily adjust or calculate your verification arguments from the initramfs. For older kernels (< 5.4) which do not have in-kernel roothash signature verification, an initramfs is also essential if you wish to perform roothash signature verification from Linux during the boot sequence. However, as you might expect, using an initramfs may add undesirable boot time and storage requirements to the system. If your embedded system is operating with very strict boot timing or storage requirements, skipping the initramfs may be beneficial to you.  This can be done by using a technique called early device mapping.

 

Background Principles

The Linux kernel has a driver subsystem for device mapping, where a hardware storage device can be remapped through a target driver to produce a new, transparent, virtual storage device. This subsystem contains a few target drivers. One of these is called Verity, which is used for filesystem verification. Crypt is another one we commonly use, which provides filesystem encryption. We colloquially refer to these as DM-Verity and DM-Crypt. DM-Verity is what we will be using in this post. Let’s begin with a simple initramfs-based DM-Verity example.

Starting with an ext4 rootfs partition, we can generate the verity metadata from a build system via:

DATA_BLOCK_SIZE="4096"
HASH_BLOCK_SIZE="4096"
PART_SRC_EXT4=example.ext4
METADATA_HASH_DEV=example.ext4.metadata

veritysetup format --data-block-size=${DATA_BLOCK_SIZE} \
    --hash-block-size=${HASH_BLOCK_SIZE} \
    ${PART_SRC_EXT4} ${METADATA_HASH_DEV}

The output from this command looks like:

VERITY header information for example.ext4.metadata
UUID:               e17b33f3-ce02-4d9b-a0a8-90c85ebe3240
Hash type:          1
Data blocks:        16384
Data block size:    4096
Hash block size:    4096
Hash algorithm:     sha256
Salt:               2a4c7638f03b92bdb92d7284a742e0c4407c9ef65fdf2a7ea78ed02fde4a518b
Root hash:          b96a69664f9279857931dbf64f942caf909076e40fd5bd5ed8d30b53ff922941

Then when we mount and switch into the verified root filesystem from the initramfs during boot, we would do this from the target device:

MAPPER_NAME="verity_example"
PART_SRC_EXT4="/dev/mmcblk0p1"
METADATA_HASH_DEV="/dev/mmcblk0p2"
ROOTHASH="b96a69664f9279857931dbf64f942caf909076e40fd5bd5ed8d30b53ff922941"

veritysetup \
    --restart-on-corruption --ignore-zero-blocks \
    create ${MAPPER_NAME} ${PART_SRC_EXT4} \
    ${METADATA_HASH_DEV} ${ROOTHASH}

mkdir -p /newroot
mount /dev/mapper/${MAPPER_NAME} /newroot
switch_root /newroot /sbin/init

Now you’re done! Every block of data contained in /newroot will now verified by the kernel before it is used. However, part of the chain of trust is still missing from this example. To understand that, we’ll need to know what the DM-Verity root hash used above is. So, what is the root hash? It’s basically the last/top node of special a binary hash tree called a Merkle Tree. Each block of data is hashed and stored at the bottom level of this binary tree. Pairs of hashes are then subsequently hashed together until a final top hash is derived.

This Merkle Tree is also what’s contained inside the Verity metadata partition or file, along with some additional header information.

 

So, before running veritysetup to create the mountable Verity-backed /dev/mapper entry, the root hash needs to be verified too. Otherwise someone could break your chain of trust by replacing your entire root file system, metadata file system, and root hash input. You can verify this root hash from your bootloader via some form of trusted/secure boot like UEFI. However, an easy way to perform signature verification isn’t always available in many embedded bootloaders, so we opt to do it from within the kernel. Traditionally, this is done inside the initramfs using a library like OpenSSL.

So for example, from your build system, you could create an RSA-signed roothash via:

ROOTHASH="b96a69664f9279857931dbf64f942caf909076e40fd5bd5ed8d30b53ff922941"
PRIVATE_KEY="verity_private.pem"
PUBLIC_KEY="verity_public.pem"

#Generate RSA Key Pair
openssl genpkey -algorithm RSA -out ${PRIVATE_KEY} -pkeyopt rsa_keygen_bits:2048
openssl rsa -pubout -in ${PRIVATE_KEY} -out ${PUBLIC_KEY}

#Sign the root hash
echo ${ROOTHASH} | tr -d '\n' > roothash.txt
openssl rsautl -sign -inkey ${PRIVATE_KEY} -out ${ROOTHASH}.signed -in ${ROOTHASH}

Then on your embedded target, you would store the public key and signed root hash and check the authenticity via your initramfs:

#If this is correctly signed, it will return the roothash from the signed roothash file
ROOTHASH=$(openssl rsautl -verify -in "roothash.txt.signed" -inkey ${PUBLIC_KEY} -pubin)

#Result = b96a69664f9279857931dbf64f942caf909076e40fd5bd5ed8d30b53ff922941
echo ${ROOTHASH}

You can then use this resulting verified roothash to run “veritysetup create”, as shown above.

 

Linux Kernel-space Signature Verification

Signature verification in the Linux kernel is performed with CONFIG_DM_VERITY_VERIFY_ROOTHASH_SIG=y. This is a kernel configuration option which was added in 5.4. It adds the ability to do roothash signature verification in Linux’s kernel-space without any external tools or libraries.

This is utilized by creating an RSA certificate from the build system:

PRIVATE_KEY="verity_key.pem"
CERT="verity_cert.pem"
openssl req -x509 -newkey rsa:1024 -keyout ${PRIVATE_KEY} \
    -out ${CERT} -nodes -days 365 -set_serial 01 -subj /CN=example.com

Then you must add the certificate into your kernel build system with the appropriate config options:

CONFIG_DM_VERITY_VERIFY_ROOTHASH_SIG=y
CONFIG_SYSTEM_TRUSTED_KEYRING=y
CONFIG_SYSTEM_TRUSTED_KEYS="verity_cert.pem"

Then we create the signed root hash from the build system:

ROOTHASH="b96a69664f9279857931dbf64f942caf909076e40fd5bd5ed8d30b53ff922941"
PRIVATE_KEY="verity_key.pem"
CERT="verity_cert.pem"
echo ${ROOTHASH} | tr -d '\n' > roothash.txt
openssl smime -sign -nocerts -noattr -binary -in roothash.txt \
    -inkey ${PRIVATE_KEY} -signer ${CERT} -outform der -out roothash.txt.signed

Don’t forget to require signatures on the embedded target’s kernel command-line parameters by appending:

dm_verity.require_signatures=1

Then, finally, we can mount the file system from our initramfs:

MAPPER_NAME="verity_example"
PART_SRC_EXT4="/dev/mmcblk0p1"
METADATA_HASH_DEV="/dev/mmcblk0p2"
ROOTHASH="b96a69664f9279857931dbf64f942caf909076e40fd5bd5ed8d30b53ff922941"

veritysetup \
    --restart-on-corruption --ignore-zero-blocks --root-hash-signature=roothash.txt.signed \
    create "${MAPPER_NAME}" "${PART_SRC_EXT4}" \
    "${METADATA_HASH_DEV}" "${ROOTHASH}"

mkdir -p /newroot
mount /dev/mapper/${MAPPER_NAME} /newroot
switch_root /newroot /sbin/init

How can I do this without an initramfs? CONFIG_DM_INIT=y

CONFIG_DM_INIT is a kernel mechanism which allows you to pass a device mapper table to the kernel during boot, from the kernel command-line parameters.

So, from a command line perspective, the user-space command transforms from this:

veritysetup --ignore-zero-blocks \
    create "dm-0" "/dev/mmcblk0p1" "/dev/mmcblk0p2" \
    "b96a69664f9279857931dbf64f942caf909076e40fd5bd5ed8d30b53ff922941"

To this after we manually pass the DM table information into the kernel:

dm-mod.create="verity,,,ro,0 131072 verity 1 /dev/mmcblk0p1 /dev/mmcblk0p2 4096 4096 16384 1 sha256 \
               b96a69664f9279857931dbf64f942caf909076e40fd5bd5ed8d30b53ff922941 \
               2a4c7638f03b92bdb92d7284a742e0c4407c9ef65fdf2a7ea78ed02fde4a518b 1 ignore_zero_blocks"

To break this down a bit further:

dm-mod.create="<name>,<uuid>,<minor>,<flags>,[dm_table_params {dm_verity_params}]"

Where dm_table_params="<start_sector> <num_sectors> <target_type> <dm_verity_params>"

And dm_verity_params="<version> <dev> <hash_dev> <data_block_size> <hash_block_size> <num_data_blocks> \
                      <hash_start_block> <algorithm> <digest> <salt> [<#opt_params> <opt_params>]"

So, in our example:

name="verity"
uuid="unused/unset"
minor="unused/unset"
flags="ro"
dm_table_params=
    start_sector="0"
    num_sectors="131072"
    target_type="verity"
    target_args="dm_verity_params"
        version="1"
        dev="mmcblk0p1"
        hash_dev="mmcblk0p2"
        data_block_size="4096"
        hash_block_size="4096"
        num_data_blocks="16384"
        hash_start_block="1"
        algorithm="sha256"
        digest="b96a69664f9279857931dbf64f942caf909076e40fd5bd5ed8d30b53ff922941"
        salt="2a4c7638f03b92bdb92d7284a742e0c4407c9ef65fdf2a7ea78ed02fde4a518b"

From the “veritysetup format” command above, we can see how a lot of these parameters are derived:

VERITY header information for example.ext4.metadata
UUID:               e17b33f3-ce02-4d9b-a0a8-90c85ebe3240
Hash type:          1
Data blocks:        16384
Data block size:    4096
Hash block size:    4096
Hash algorithm:     sha256
Salt:               2a4c7638f03b92bdb92d7284a742e0c4407c9ef65fdf2a7ea78ed02fde4a518b
Root hash:          b96a69664f9279857931dbf64f942caf909076e40fd5bd5ed8d30b53ff922941

For the number of sectors, that is calculated via Verity Data Blocks * Verity Data Block Size / Sector Size. Assuming your sector size is 512 (most common), we then have 16384*4096/512 = 131072.

From U-Boot, I like to set all of this up similarly to:

setenv DATA_BLOCKS 16384
setenv DATA_BLOCK_SIZE 4096
setenv DATA_SECTORS 131072
setenv HASH_BLOCK_SIZE 4096
setenv HASH_ALG sha256
setenv SALT 2a4c7638f03b92bdb92d7284a742e0c4407c9ef65fdf2a7ea78ed02fde4a518b
setenv ROOT_HASH b96a69664f9279857931dbf64f942caf909076e40fd5bd5ed8d30b53ff922941
setenv DATA_DEV mmcblk0p1
setenv DATA_META_DEV mmcblk0p2

setenv bootargs ${bootargs} dm-mod.create="verity,,,ro,0 \${DATA_SECTORS} verity 1 /dev/\${DATA_DEV}
    /dev/\${DATA_META_DEV} \${HASH_BLOCK_SIZE} \${DATA_BLOCK_SIZE} \${DATA_BLOCKS} 1 \${HASH_ALG}
    \${ROOT_HASH} \${SALT} 1 ignore_zero_blocks" root=/dev/dm-0 dm_verity.require_signatures=1

Note: /dev/dm-0 is the first device mapper device that is created by this early device mapping procedure. If this is your rootfs, then you must also set root=/dev/dm-0, as we have above.

 

Wait, what about early root hash signature verification? Where did that go?

Good catch, we can’t use the OpenSSL+initramfs method anymore. Unfortunately, DM_VERITY_VERIFY_ROOTHASH_SIG also does not support early mapping, as it uses the kernel keyring system and does not provide a way to setup the keyring via the kernel command-line arguments.

Alright, fine, let’s fix it. We’ll add a new verity table parameter called “root_hash_sig_hex”, where we can set the incoming root hash signature, without the keyring.

From: Nathan Barrett-Morrison <nathan.morrison@timesys.com>
Date: Wed, 30 Mar 2022 10:57:25 -0400
Subject: [PATCH 1/1] DM-Verity: Add root_hash_sig_hex parameter to early
 verity device mapping, so that we can pass a roothash signature in via
 /proc/cmdline.  This enables the ability to use DM_VERITY_VERIFY_ROOTHASH_SIG
 alongside early device mapping

---

 drivers/md/dm-verity-target.c     |  8 +++-
 drivers/md/dm-verity-verify-sig.c | 63 +++++++++++++++++++++++++++++++
 drivers/md/dm-verity-verify-sig.h | 19 +++++++++-
 3 files changed, 87 insertions(+), 3 deletions(-)

diff --git a/drivers/md/dm-verity-target.c b/drivers/md/dm-verity-target.c
index 3fb02167a590..047dd9a10264 100644
--- a/drivers/md/dm-verity-target.c
+++ b/drivers/md/dm-verity-target.c
@@ -930,7 +930,13 @@ static int verity_parse_opt_args(struct dm_arg_set *as, struct dm_verity *v,
             if (r)
                 return r;
             continue;
-
+        } else if (verity_verify_is_hex_sig_opt_arg(arg_name)) {
+            r = verity_verify_hex_sig_parse_opt_args(as, v,
+                                 verify_args,
+                                 &argc, arg_name);
+            if (r)
+                return r;
+            continue;
         }
 
         ti->error = "Unrecognized verity feature request";
diff --git a/drivers/md/dm-verity-verify-sig.c b/drivers/md/dm-verity-verify-sig.c
index 919154ae4cae..906e5a2034b5 100644
--- a/drivers/md/dm-verity-verify-sig.c
+++ b/drivers/md/dm-verity-verify-sig.c
@@ -28,6 +28,12 @@ bool verity_verify_is_sig_opt_arg(const char *arg_name)
                 DM_VERITY_ROOT_HASH_VERIFICATION_OPT_SIG_KEY));
 }
 
+bool verity_verify_is_hex_sig_opt_arg(const char *arg_name)
+{
+    return (!strcasecmp(arg_name,
+                DM_VERITY_ROOT_HASH_VERIFICATION_OPT_HEX_SIG_KEY));
+}
+
 static int verity_verify_get_sig_from_key(const char *key_desc,
                     struct dm_verity_sig_opts *sig_opts)
 {
@@ -64,6 +70,33 @@ static int verity_verify_get_sig_from_key(const char *key_desc,
     return ret;
 }
 
+static int verity_verify_get_sig_from_hex(const char *key_desc,
+                    struct dm_verity_sig_opts *sig_opts)
+{
+    int ret = 0, i = 0, j = 0;
+    uint8_t byte[3] = {0x00, 0x00, 0x00};
+    long result;
+
+    sig_opts->sig = kmalloc(strlen(key_desc)/2, GFP_KERNEL);
+    if (!sig_opts->sig) {
+        ret = -ENOMEM;
+        goto end;
+    }
+
+    sig_opts->sig_size = strlen(key_desc)/2;
+
+    for(i = 0, j = 0; i < strlen(key_desc)-1; i+=2, j+=1){
+        byte[0] = key_desc[i];
+        byte[1] = key_desc[i+1];
+        kstrtol(byte, 16, &result);
+        sig_opts->sig[j] = result;
+    }
+
+end:
+
+    return ret;
+}
+
 int verity_verify_sig_parse_opt_args(struct dm_arg_set *as,
                      struct dm_verity *v,
                      struct dm_verity_sig_opts *sig_opts,
@@ -93,6 +126,36 @@ int verity_verify_sig_parse_opt_args(struct dm_arg_set *as,
     return ret;
 }
 
+int verity_verify_hex_sig_parse_opt_args(struct dm_arg_set *as,
+                     struct dm_verity *v,
+                     struct dm_verity_sig_opts *sig_opts,
+                     unsigned int *argc,
+                     const char *arg_name)
+{
+    struct dm_target *ti = v->ti;
+    int ret = 0;
+    const char *sig_key = NULL;
+
+    if (!*argc) {
+        ti->error = DM_VERITY_VERIFY_ERR("Signature key not specified");
+        return -EINVAL;
+    }
+
+    sig_key = dm_shift_arg(as);
+    (*argc)--;
+
+    ret = verity_verify_get_sig_from_hex(sig_key, sig_opts);
+    if (ret < 0)
+        ti->error = DM_VERITY_VERIFY_ERR("Invalid key specified");
+
+    v->signature_key_desc = kstrdup(sig_key, GFP_KERNEL);
+    if (!v->signature_key_desc)
+        return -ENOMEM;
+
+    return ret;
+}
+
+
 /*
  * verify_verify_roothash - Verify the root hash of the verity hash device
  *                 using builtin trusted keys.
diff --git a/drivers/md/dm-verity-verify-sig.h b/drivers/md/dm-verity-verify-sig.h
index 19b1547aa741..2be00114becc 100644
--- a/drivers/md/dm-verity-verify-sig.h
+++ b/drivers/md/dm-verity-verify-sig.h
@@ -10,6 +10,7 @@
 
 #define DM_VERITY_ROOT_HASH_VERIFICATION "DM Verity Sig Verification"
 #define DM_VERITY_ROOT_HASH_VERIFICATION_OPT_SIG_KEY "root_hash_sig_key_desc"
+#define DM_VERITY_ROOT_HASH_VERIFICATION_OPT_HEX_SIG_KEY "root_hash_sig_hex"
 
 struct dm_verity_sig_opts {
     unsigned int sig_size;
@@ -23,11 +24,13 @@ struct dm_verity_sig_opts {
 int verity_verify_root_hash(const void *data, size_t data_len,
                 const void *sig_data, size_t sig_len);
 bool verity_verify_is_sig_opt_arg(const char *arg_name);
-
+bool verity_verify_is_hex_sig_opt_arg(const char *arg_name);
 int verity_verify_sig_parse_opt_args(struct dm_arg_set *as, struct dm_verity *v,
                     struct dm_verity_sig_opts *sig_opts,
                     unsigned int *argc, const char *arg_name);
-
+int verity_verify_hex_sig_parse_opt_args(struct dm_arg_set *as, struct dm_verity *v,
+                    struct dm_verity_sig_opts *sig_opts,
+                    unsigned int *argc, const char *arg_name);
 void verity_verify_sig_opts_cleanup(struct dm_verity_sig_opts *sig_opts);
 
 #else
@@ -45,6 +48,11 @@ bool verity_verify_is_sig_opt_arg(const char *arg_name)
     return false;
 }
 
+bool verity_verify_is_hex_sig_opt_arg(const char *arg_name)
+{
+    return false;
+}
+
 int verity_verify_sig_parse_opt_args(struct dm_arg_set *as, struct dm_verity *v,
                     struct dm_verity_sig_opts *sig_opts,
                     unsigned int *argc, const char *arg_name)
@@ -52,6 +60,13 @@ int verity_verify_sig_parse_opt_args(struct dm_arg_set *as, struct dm_verity *v,
     return -EINVAL;
 }
 
+int verity_verify_hex_sig_parse_opt_args(struct dm_arg_set *as, struct dm_verity *v,
+                    struct dm_verity_sig_opts *sig_opts,
+                    unsigned int *argc, const char *arg_name)
+{
+    return -EINVAL;
+}
+
 void verity_verify_sig_opts_cleanup(struct dm_verity_sig_opts *sig_opts)
 {
 }

So, this patch allows the kernel to take the roothash signature in as a hex-formatted string from the command-line arguments and converts it back into raw data when it is copied into sig_opts->sig. This is done inside the verity_verify_get_sig_from_hex() section of the patch.

Now that our kernel is patched, we can do the following from our build system:

ROOTHASH="b96a69664f9279857931dbf64f942caf909076e40fd5bd5ed8d30b53ff922941"
PRIVATE_KEY="verity_key.pem"
CERT="verity_cert.pem"
echo ${ROOTHASH} | tr -d '\n' > roothash.txt
openssl smime -sign -nocerts -noattr -binary -in roothash.txt \
    -inkey ${PRIVATE_KEY} -signer ${CERT} -outform der -out roothash.txt.signed

xxd -p sign.txt | tr -d '\n'

#This will show a hexdump of a signature that looks like:
# 3081f906092a864886f70d010702a081eb3081e8020101310f300d06096086480165030402010500300
# b06092a864886f70d0107013181c43081c1020101301b30163114301206035504030c0b6578616d706c65
# 2e636f6d020101300d06096086480165030402010500300d06092a864886f70d01010105000481806f196
# a3081d941d22de98b34fc56f5c1b7ffc827ccd1307be9017bb6773da49026ef556668185c68b30562a60c
# ec635bbe0a52ad92b878dac9e4ad146f5d36101b48ec5f522d12772b8e915524586598c8659494fba427e
# e46c02043f30f45e096a1b9a987fc200b815f43bb48d42ad2d64b3f632f5332e6f74890b4541b467d

Then from our target system boot arguments, we can append the new “root_hash_sig_hex” parameters:

setenv DATA_BLOCKS 16384
setenv DATA_BLOCK_SIZE 4096
setenv DATA_SECTORS 131072
setenv HASH_BLOCK_SIZE 4096
setenv HASH_ALG sha256
setenv SALT 2a4c7638f03b92bdb92d7284a742e0c4407c9ef65fdf2a7ea78ed02fde4a518b
setenv ROOT_HASH b96a69664f9279857931dbf64f942caf909076e40fd5bd5ed8d30b53ff922941
setenv DATA_DEV mmcblk0p1
setenv DATA_META_DEV mmcblk0p2
setenv VERITY_SIGNATURE 3081f906092a864886f70d010702a081eb3081e8020101310f300d06096086480165030402010500300
    b06092a864886f70d0107013181c43081c1020101301b30163114301206035504030c0b6578616d706c652e636f6d020101300d
    06096086480165030402010500300d06092a864886f70d010101050004818071fcaaf1b252a56448438e0a9350b7380a407b1e9
    0ae869ec5062466b0eb6cc5358e253a9d57086c358220745bc60c2a6d8dbc30c02fb1714c9c98f10e0679b87deb0c19929675c8
    fcf89f37c684f043583fca52729ffb6e928eb29b7ee0c9eab3a3b0809a4463f3c8d6d458745c9116a7df1677c707df6352f2323
    13a62ce20

setenv bootargs ${bootargs} dm-mod.create="verity,,,ro,0 \${DATA_SECTORS} verity 1 /dev/\${DATA_DEV}
    /dev/\${DATA_META_DEV} \${HASH_BLOCK_SIZE} \${DATA_BLOCK_SIZE} \${DATA_BLOCKS} 1 \${HASH_ALG}
    \${ROOT_HASH} \${SALT} 3 ignore_zero_blocks root_hash_sig_hex \${VERITY_SIGNATURE}"
    root=/dev/dm-0 dm_verity.require_signatures=1

Now you’re really done! Your kernel will verify the DM-Verity file system’s roothash and boot without any requirement for an initramfs.

 

Other Considerations

1) Here are all of the kernel config options which are related to device mapping and verity:

CONFIG_BLK_DEV=y
CONFIG_BLK_DEV_LOOP=y
CONFIG_MD=y
CONFIG_BLK_DEV_DM=y
CONFIG_BLK_DEV_MD=y
CONFIG_DM_INIT=y
CONFIG_DM_VERITY=y
CONFIG_DM_VERITY_VERIFY_ROOTHASH_SIG=y
CONFIG_SYSTEM_TRUSTED_KEYRING=y
CONFIG_SYSTEM_TRUSTED_KEYS="verity_cert.pem"

2) Your verified file system is only as good as your chain of trust. Every stage in your boot process needs to be properly secured. Our secure boot article outlines this in more detail here.

3) If your U-Boot environment can be tampered with, such that dm_verity.require_signatures can be disabled, then your root file system verification can easily be defeated. You might consider forcefully enabling this with a kernel patch as such:

From: Nathan Barrett-Morrison <nathan.morrison@timesys.com>
Date: Wed, 30 Mar 2022 10:58:20 -0400
Subject: [PATCH 1/1] DM-Verity: Require dm-verity roothash signatures, with no
 ability to disable via /proc/cmdline

---
 drivers/md/dm-verity-verify-sig.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/md/dm-verity-verify-sig.c b/drivers/md/dm-verity-verify-sig.c
index 906e5a2034b5..b979e8e6b498 100644
--- a/drivers/md/dm-verity-verify-sig.c
+++ b/drivers/md/dm-verity-verify-sig.c
@@ -14,10 +14,12 @@
 
 #define DM_VERITY_VERIFY_ERR(s) DM_VERITY_ROOT_HASH_VERIFICATION " " s
 
-static bool require_signatures;
+static bool require_signatures = true;
+/*
 module_param(require_signatures, bool, 0444);
 MODULE_PARM_DESC(require_signatures,
        "Verify the roothash of dm-verity hash tree");
+*/
 
 #define DM_VERITY_IS_SIG_FORCE_ENABLED() \
    (require_signatures != false)

4) Small amounts of data corruption can be automatically corrected with CONFIG_DM_VERITY_FEC. If you’re interested in this, consider enabling this configuration option as well.

5) While booting from an MMC device, I observed that the MMC partition enumeration was out of sync with the early device mapping driver’s device lookup, resulting in a race condition. That is, it would attempt to find the enumerated MMC partitions (mmcblk0p1, mmcblk0p2) before they were instantiated and fail. I fixed this with another patch:

From: Nathan Barrett-Morrison <nathan.morrison@timesys.com>
Date: Wed, 30 Mar 2022 10:54:01 -0400
Subject: [PATCH 1/3] DM-Verity: Wait up to 10 seconds for eMMC/SD partitions
 to show up before failing dm_get_device()

---
 drivers/md/dm-verity-target.c | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/drivers/md/dm-verity-target.c b/drivers/md/dm-verity-target.c
index 711f101447e3..3fb02167a590 100644
--- a/drivers/md/dm-verity-target.c
+++ b/drivers/md/dm-verity-target.c
@@ -18,6 +18,7 @@
 #include "dm-verity-verify-sig.h"
 #include <linux/module.h>
 #include <linux/reboot.h>
+#include <linux/delay.h>
 
 #define DM_MSG_PREFIX            "verity"
 
@@ -998,10 +999,17 @@ static int verity_ctr(struct dm_target *ti, unsigned argc, char **argv)
     }
     v->version = num;
 
-    r = dm_get_device(ti, argv[1], FMODE_READ, &v->data_dev);
-    if (r) {
-        ti->error = "Data device lookup failed";
-        goto bad;
+    //Wait up to 10 seconds for devices to become available --
+    //wait_for_device_probe() sort of handles this, but the eMMC/SD probe finishes
+    //and dm_get_device() fails before the eMMC/SD partitions are found
+    for(i = 0; i <= 100; i++){
+        r = dm_get_device(ti, argv[1], FMODE_READ, &v->data_dev);
+        if (r && i < 100) {
+            msleep_interruptible(100);
+        }else if(r){
+            ti->error = "Data device lookup failed";
+            goto bad;
+        }
     }
 
     r = dm_get_device(ti, argv[2], FMODE_READ, &v->hash_dev);

 

Boot Time Comparisons

Testing on a relatively slower ARM processor (ADSP-SC589, Single core 500Mhz ARMv7) shows some major improvements. Keep in mind that this processor is favorable for showing the time difference here, as it has a slower initramfs loading and unpacking time. Newer processors with faster eMMC, DDR, and cores will narrow this margin.

On this processor, there is a total boot time difference of 29.19 seconds.

Using initramfs Not using initramfs Difference
Total boot time 53.33 s 24.14 s 29.19 s
U-Boot: Load initramfs from MMC 2.88 s 0.00 s 2.88 s
U-Boot: Verify initramfs 4.07 s 0.00 s 4.07 s
U-Boot: Relocate initramfs 3.81 s 0.00 s 3.81 s
Linux: Unpack initramfs 18.26 s 0.00 s 18.26 s
Linux: Run initramfs, mount partition, start systemd 1.41 s 1.09 s 0.32 s

On a faster, multi-core ARMv8+ processor, I believe you would still see a significant 1-2+ second difference in boot time. Early device mapping can be a very important tool if you are trying to hit a boot time goal of under 10 seconds.

 

Help!

As always, we offer professional engineering services to help you with your project. Whether you wish for us to integrate our security feature implementation, called VigiShield, into your project or just need another set of hands to help with general embedded software engineering and security, we’re available to help!

Securing your Linux Configuration (Kernel Hardening)

Securing your Linux Configuration (Kernel Hardening)

Subscribe to our RSS
Share this on Facebook
Share this on Twitter
Share this on LinkedIn

 

 

This article discusses the process by which your kernel’s configuration can be strengthened to protect against common security exploits. This is sometimes referred to as hardening, or specifically in this context, kernel configuration hardening.

 

Preface

A Linux kernel configuration is a file which defines all of the enabled (or disabled) options which are compiled in to your kernel. If you have not seen one before, they generally reside in the kernel’s build directory with a filename of “.config”. They are sometimes collapsed in to a defconfig (default config) file which only shows options which were not already selected by default.

This discussion will present many configuration tables in the form of:

Expected Conditional Architectures Kernel Versions Note
OPTION1=Y Architecture to which this option applys (X86 and/or ARM) Kernel versions in which OPTION1 exists A description of OPTION1
OPTION2=is not set Architecture to which this option applys (X86 and/or ARM) Kernel versions in which OPTION2 exists A description of OPTION2

In this case, it is recommended that the fictional option, OPTION1, is selected (CONFIG_OPTION1=y) in your kernel’s .config or defconfig file. It is also recommended that OPTION2 is disabled (CONFIG_OPTION2=is not set) in your kernel’s .config or defconfig file.

The option descriptions are sometimes difficult to interpret and may require further research. If you find an option that you’d like to know more information about, you may have to inspect the kernel source, search LWN, search Patchwork, or use your web search engine of choice.

 

Understanding Configuration Selections

When Linux kernel hardening, there are generally four categories of reason for which a configuration item may be enabled or disabled:

  • Adding an additional level of protection against a known exploit by enabling a configuration item. For example:
    • Mitigating Spectre attacks with CONFIG_RETPOLINE=y (which traps the processor’s speculative execution paths for indirect address calls by using a return trampoline).
  • Disabling a configuration item or subsystem which is known to be exploitable. For example:
    • Disabling the USB networking subsystem (USB_USBNET=is not set), so that applications using network-based IPC(Inter-Processor Communication) mechanisms may not be inadvertently exposed through a USB port.
    • Disabling /dev/mem access to physical memory (DEVMEM=is not set), so that physical memory cannot be easily modified.
  • Enabling general security strengthening features in the kernel (which may necessarily protect against a presently known attack). For example:
    • Reducing the risk of memory page leakage by enabling page poisoning (PAGE_POISONING=Y) to overwrite any potentially sensitive information upon freeing.
    • Enabling Security-Enhanced Linux
  • Disabling any unused kernel configuration options. If you don’t need it, disable it. As an added bonus, doing so may also improve your boot times.
Expected Conditional Architectures Kernel Versions Note
RETPOLINE=Y X86_32, X86_64 4.15-4.20, 5.0-5.17 Avoid speculative indirect branches in kernel (Spectre Mitigation)
USB_USBNET=is not set ARM, ARM64, X86_32, X86_64 2.6.22-2.6.39, 3.0-3.19, 4.0-4.20, 5.0-5.17 If enabled, this adds USB networking subsystem support to the kernel
DEVMEM=is not set ARM, ARM64, X86_32, X86_64 4.0-4.20, 5.0-5.17 Do not allow access to physical memory via /dev/mem
PAGE_POISONING=Y ARM, ARM64, X86_32, X86_64 4.6-4.20, 5.0-5.17 Fill the pages with poison patterns after free_pages() and verify the patterns before alloc_pages. The filling of the memory helps reduce the risk of information leaks from freed data. This must be enabled from the boot cmdline with page_poison=1

The latter two options can help to protect against exploits which have not yet been discovered or released into the public domain (e.g. zero-day exploits) by reducing your kernel’s exploitable attack surface.

Analyzing, understanding, and modifying the kernel configuration with these tasks in mind is not trivial. Furthermore, in a large project it may not be clear exactly who is using which kernel configuration option. This can result in iterative backstepping until you arrive at a final configuration which works for your entire team.

There’s also a maintenance burden once you have created your final configuration. When you upgrade your kernel, many of the configuration options will have been removed and renamed. This will require another assessment and configuration period. Keeping your kernel up to date is extremely important, as security features are continuously added and revised in newer kernels. Starting a project with a long term stable (LTS) kernel is recommended, as an LTS kernel provides smaller (and sometimes backported) version increments to resolve security flaws (e.g upgrading from 5.0.x to 5.0.y). Such patches are provided until the long term stable support period ends.

 

Cost-Benefit Analysis

Not all configuration items provide the same cost-benefit as others. To some extent, most options will have an impact in these key areas:

  • Compile Time
    • This is especially true for security features which are checked by the compiler during compile time. For example, GCC_PLUGIN_STACKLEAK=Y will block uninitialized stack variable attacks through the use of a compiler plugin which searches for and initializes such variables.
    • This should not be considered a concern. Added compilation time is well worth the added security.
  • Kernel Binary Size
    • This is usually not a concern. Adding and removing features may change the kernel size by a few megabytes, which is generally negligible on modern systems.
  • Boot time
    • For example, DM_CRYPT=y, which adds drive encryption capabilities to your kernel, will add additional non-neglibile boot time to your system. This is because it may require booting in to an Initial Ram Filesystem, retreiving your disk encryption key, and ultimately mounting the encrypted disk through the device mapper subsystem. All of these steps add time to the boot process.
  • Processor Load
    • This is perhaps the most concerning option and must be determined by trial and error. If every available security option is enabled on a slower ARM processor, there may be too much overhead to reliably run your processes (at a reasonable latency).
    • For example, erasing memory with PAGE_POISONING=Y will use CPU cycles. The amount of overhead added will be dependent upon factors such as your CPU and RAM speed.
Expected Conditional Architectures Kernel Versions Note
GCC_PLUGIN_STACKLEAK=Y ARM64, X86_32, X86_64 5.2-5.17 This blocks most uninitialized stack variable attacks, with the performance impact being driven by the depth of the stack usage, rather than the function calling complexity. The performance impact on a single CPU system kernel compilation sees a 1% slowdown.
DM_CRYPT=Y ARM, ARM64, X86_32, X86_64 2.6.4-2.6.39, 3.0-3.19, 4.0-4.20, 5.0-5.17 Filesystem Hardening (Block Level Encryption via dm-crypt) – Requires userspace support
PAGE_POISONING=Y ARM, ARM64, X86_32, X86_64 4.6-4.20, 5.0-5.17 Fill the pages with poison patterns after free_pages() and verify the patterns before alloc_pages. The filling of the memory helps reduce the risk of information leaks from freed data. This must be enabled from the boot cmdline with page_poison=1

Some configuration items will provide a better cost-benefit. Configuration items which have minimal impact on processor load are most valuable. For example, these add virtually no CPU overhead:

  • Disabling DEBUG_BUGVERBOSE, which will help ensure that sensitive backtrace information is not leaked upon a kernel BUG() condition.
  • Enabling ARCH_HAS_ELF_RANDOMIZE, which will make repeat exploits much more difficult by randomizing certain memory locations.

While these will add CPU overhead to some degree:

  • Enabling DEBUG_VIRTUAL will enable some sanity checking in virt_to_page translation at the cost of CPU cycles.
  • Enabling INIT_ON_ALLOC_DEFAULT_ON or INIT_ON_FREE_DEFAULT_ON will protect against heap memory leaks by erasing the regions after use, at the cost of erase time.
Expected Conditional Architectures Kernel Versions Note
DEBUG_BUGVERBOSE=is not set ARM, ARM64, X86_32, X86_64 2.6.9-2.6.39, 3.0-3.19, 4.0-4.20, 5.0-5.17 Make sure this is not enabled, as it could provide an attacker sensitive kernel backtrace information on BUG() conditions
ARCH_HAS_ELF_RANDOMIZE=Y ARM, ARM64, X86_32, X86_64 4.1-4.20, 5.0-5.17 Randomized locations for stack, mmap, brk, and ET_DYN
INIT_ON_FREE_DEFAULT_ON=Y ARM, ARM64, X86_32, X86_64 5.3-5.17 More expensive form of INIT_ON_ALLOC_DEFAULT_ON. The primary difference is that data lifetime in memory is reduced, as anything freed is wiped immediately, making live forensics or cold boot memory attacks unable to recover freed memory contents.
INIT_ON_ALLOC_DEFAULT_ON=Y ARM, ARM64, X86_32, X86_64 5.3-5.17 All page allocator and slab allocator memory will be zeroed when freed, eliminating many kinds of “uninitialized heap memory” flaws, especially heap content exposures.
DEBUG_VIRTUAL=Y ARM, ARM64, X86_32, X86_64 2.6.28-2.6.39, 3.0-3.19, 4.0-4.20, 5.0-5.17 Enable some costly sanity checks in virtual to page code. This can catch mistakes with virt_to_page() and friends.

Using multiple kernel configurations (development and production) can be an option. While it is wise to develop on a prototype system that most closely resembles a production system (as to not cause unforseen bugs in changing timing conditions and loads between configurations), some security options be too cumbersome to reasonably develop on. If you’re writing a kernel driver and you need to use a tracing tool or read a core dump, then certainly enable them while developing.

 

Categories of Linux Kernel Hardening

In the Timesys Kernel Hardening Analysis Tool, the kernel security options have been divided into various groups. This categorization is by no means a definitive separation; some options could be further categorized or applied to multiple categories.

Memory Protection

Memory exploits are classes of attack in which an entity is able to retrieve or modified privileged information about the system. These can be further categorized into:

  • Stack Overflow Protections: These are security features which seek to prevent access to and tampering of stack variables in memory. A stack canary (an arbitrary value sitting at the top of the stack, which, if modified, alerts the kernel of tampering) is sometimes mentioned in these protections.
Expected Conditional Architectures Kernel Versions Note
INIT_STACK_ALL_ZERO=Y ARM, ARM64, X86_32, X86_64 5.9-5.17 Initializes everything on the stack with a zero value. This is intended to eliminate all classes of uninitialized stack variable exploits and information exposures, even variables that were warned to have been left uninitialized. (Strongest, safest)
GCC_PLUGIN_ARM_SSP_PER_TASK=Y ARM 5.2-5.17 Generates a separate stack canary value for each task, so if one task’s canary value is leaked it does not cause all other tasks to become vulnerable.
STACKPROTECTOR=Y ARM, ARM64, X86_32, X86_64 4.18-4.20, 5.0-5.17 This option turns on the “stack-protector” GCC feature. This feature puts, at the beginning of functions, a canary value on the stack just before the return address, and validates the value just before actually returning. Stack based buffer overflows (that need to overwrite this return address) now also overwrite the canary, which gets detected and the attack is then neutralized via a kernel panic.
STACKPROTECTOR_STRONG=Y ARM, ARM64, X86_32, X86_64 4.18-4.20, 5.0-5.17 Adds the CONFIG_STACKPROTECTOR canary logic to additional conditions related to variable assignment.
STACKPROTECTOR_PER_TASK=Y ARM, ARM64 5.0-5.17 Use a different stack canary value for each task
VMAP_STACK=Y ARM64 4.9-4.20, 5.0-5.17 Enable this if you want the use virtually-mapped kernel stacks with guard pages. This causes kernel stack overflows to be caught immediately.
SCHED_STACK_END_CHECK=Y ARM, ARM64, X86_32, X86_64 3.18-3.19, 4.0-4.20, 5.0-5.17 Additional validation check on commonly targeted structure. Detect stack corruption on calls to schedule()
STACKLEAK_METRICS=is not set ARM64, X86_32, X86_64 5.2-5.17 If this is set, STACKLEAK metrics for every task are available in the /proc file system.
STACKLEAK_RUNTIME_DISABLE=is not set ARM64, X86_32, X86_64 5.2-5.17 If set, allows runtime disabling of kernel stack erasing
GCC_PLUGIN_STACKLEAK=Y ARM64, X86_32, X86_64 5.2-5.17 This blocks most uninitialized stack variable attacks, with the performance impact being driven by the depth of the stack usage, rather than the function calling complexity. The performance impact on a single CPU system kernel compilation sees a 1% slowdown.
  • Heap Overflow: These are security features which seek to prevent heap memory exposure and modification.
Expected Conditional Architectures Kernel Versions Note
STRICT_KERNEL_RWX=Y ARM, ARM64, X86_32, X86_64 4.11-4.20, 5.0-5.17 If this is set, kernel text and rodata memory will be made read-only, and non-text memory will be made non-executable. This provides protection against certain security exploits (e.g. executing the heap or modifying text)
SLAB_FREELIST_HARDENED=Y ARM, ARM64, X86_32, X86_64 4.14-4.20, 5.0-5.17 Harden slab freelist metadata: Many kernel heap attacks try to target slab cache metadata and other infrastructure. This options makes minor performance sacrifices to harden the kernel slab allocator against common freelist exploit methods. Some slab implementations have more sanity-checking than others. This option is most effective with CONFIG_SLUB.
SLAB_FREELIST_RANDOM=Y ARM, ARM64, X86_32, X86_64 4.7-4.20, 5.0-5.17 Randomizes the freelist order used on creating new pages. This security feature reduces the predictability of the kernel slab allocator against heap overflows.
COMPAT_BRK=is not set ARM, ARM64, X86_32, X86_64 2.6.25-2.6.39, 3.0-3.19, 4.0-4.20, 5.0-5.17 Do not disable heap randomization
INET_DIAG=is not set ARM, ARM64, X86_32, X86_64 2.6.14-2.6.39, 3.0-3.19, 4.0-4.20, 5.0-5.17 Do not allow INET socket monitoring interface. Assists heap memory attacks
  • User Copy Protection: These are security features which seek to prevent memory exploitation during kernel and userspace memory transfer transactions.
Expected Conditional Architectures Kernel Versions Note
HARDENED_USERCOPY=Y ARM, ARM64, X86_32, X86_64 4.8-4.20, 5.0-5.17 This option checks for obviously wrong memory regions when copying memory to/from the kernel (via copy_to_user() and copy_from_user() functions) by rejecting memory ranges that are larger than the specified heap object, span multiple separately allocated pages, are not on the process stack, or are part of the kernel text. This kills entire classes of heap overflow exploits and similar kernel memory exposures.
HARDENED_USERCOPY_FALLBACK=is not set ARM, ARM64, X86_32, X86_64 4.16-4.20, 5.0-5.15 This is a temporary option that allows missing usercopy whitelists to be discovered via a WARN() to the kernel log, instead of rejecting the copy, falling back to non-whitelisted hardened usercopy that checks the slab allocation size instead of the whitelist size.
HARDENED_USERCOPY_PAGESPAN=is not set ARM, ARM64, X86_32, X86_64 4.8-4.20, 5.0-5.17 When a multi-page allocation is done without __GFP_COMP, hardened usercopy will reject attempts to copy it. There are, however, several cases of this in the kernel that have not all been removed. This config is intended to be used only while trying to find such users.
HAVE_HARDENED_USERCOPY_ALLOCATOR=Y ARM, ARM64, X86_32, X86_64 4.8-4.20, 5.0-5.17 The heap allocator implements __check_heap_object() for validating memory ranges against heap object sizes.
  • Information exposure: Options which are selected to limit exposure to privileged information.
Expected Conditional Architectures Kernel Versions Note
X86_UMIP=Y X86_32, X86_64 5.5-5.17 If enabled, a general protection fault is issued if the SGDT, SLDT, SIDT, SMSW or STR instructions are executed in user mode. These instructions unnecessarily expose information about the hardware state.
PROC_PAGE_MONITOR=is not set ARM, ARM64, X86_32, X86_64 2.6.28-2.6.39, 3.0-3.19, 4.0-4.20, 5.0-5.17 Do not expose process memory utilization via /proc interfaces
PROC_VMCORE=is not set ARM, ARM64, X86_32, X86_64 2.6.37-2.6.39, 3.0-3.19, 4.0-4.20, 5.0-5.17 Do not export the dump image of crashed kernel
DEBUG_FS=is not set ARM, ARM64, X86_32, X86_64 2.6.11-2.6.39, 3.0-3.19, 4.0-4.20, 5.0-5.17 Do not enable debugfs, as it may expose vulnerabilities
  • Kernel Address Space Layout Randomization (KASLR): A security method by which kernel memory structures are randomized in order to prevent repeat or replay-style attacks.
Expected Conditional Architectures Kernel Versions Note
ARCH_HAS_ELF_RANDOMIZE=Y ARM, ARM64, X86_32, X86_64 4.1-4.20, 5.0-5.17 Randomized locations for stack, mmap, brk, and ET_DYN
RANDOMIZE_BASE=Y ARM64, X86_32, X86_64 4.7-4.20, 5.0-5.17 In support of Kernel Address Space Layout Randomization (KASLR), this randomizes the physical address at which the kernel image is decompressed and the virtual address where the kernel image is mapped, as a security feature that deters exploit attempts relying on knowledge of the location of kernel code internals.
RANDOMIZE_MEMORY=Y X86_64 4.8-4.20, 5.0-5.17 Randomizes the base virtual address of kernel memory sections (physical memory mapping, vmalloc & vmemmap). This security feature makes exploits relying on predictable memory locations less reliable.
SLAB_FREELIST_RANDOM=Y ARM, ARM64, X86_32, X86_64 4.7-4.20, 5.0-5.17 Randomizes the freelist order used on creating new pages. This security feature reduces the predictability of the kernel slab allocator against heap overflows.
GCC_PLUGIN_RANDSTRUCT=Y ARM, ARM64, X86_32, X86_64 4.13-4.20, 5.0-5.17 Randomizes layout of sensitive kernel structures

Reducing Attack Surface

These are configuration options which can be selected to reduce the potential for exposure to unknown zero-day attacks by limiting the attack surface as much as we can. These are options that reduce the amount of information exposure and compiled-firmware attack surface (Again: If you don’t need it, disable it).

  • Kernel Replacement Attacks: Methods in which a kernel binary could be replaced during runtime.
Expected Conditional Architectures Kernel Versions Note
HIBERNATION=is not set ARM, ARM64, X86_32, X86_64 2.6.23-2.6.39, 3.0-3.19, 4.0-4.20, 5.0-5.17 Do not support hibernation. Allows replacement of running kernel.
KEXEC=is not set ARM, ARM64, X86_32, X86_64 2.6.16-2.6.39, 3.0-3.19, 4.0-4.20, 5.0-5.17 Do not allow system to boot another Linux kernel
KEXEC_FILE=is not set ARM, ARM64, X86_32, X86_64 3.17-3.19, 4.0-4.20, 5.0-5.17 Do not allow system to boot another Linux kernel
  • Module Security Attacks: These are attacks which can be performed by loading a tainted, custom, module in to a system or maliciously modifying a pre-existing module’s memory. The mitigations for this mostly consist of restricting execution regions, making such regions read-only, and signature checking prior to loading modules.
Expected Conditional Architectures Kernel Versions Note
MODULES=is not set ARM, ARM64, X86_32, X86_64 You should not allow for modules to be loaded unless you have the proper signing and signature checks enabled. Allowing the kernel to load unsigned modules can be dangerous
STRICT_MODULE_RWX=Y ARM, ARM64, X86_32, X86_64 4.11-4.20, 5.0-5.17 Module text and rodata memory will be made read-only, and non-text memory will be made non-executable. This provides protection against certain security exploits (e.g. writing to text)
MODULE_SIG=Y ARM, ARM64, X86_32, X86_64 3.7-3.19, 4.0-4.20, 5.0-5.17 Enable module signature verification
MODULE_SIG_ALL=Y ARM, ARM64, X86_32, X86_64 3.9-3.19, 4.0-4.20, 5.0-5.17 Automatically sign all modules during modules_install (so we don’t have to do this manually)
MODULE_SIG_SHA512=Y ARM, ARM64, X86_32, X86_64 3.7-3.19, 4.0-4.20, 5.0-5.17 Sign modules with SHA-512 algorithm
MODULE_SIG_FORCE=Y ARM, ARM64, X86_32, X86_64 3.7-3.19, 4.0-4.20, 5.0-5.17 Require modules to be validly signed
DEBUG_SET_MODULE_RONX=Y ARM, ARM64, X86_32, X86_64 Varies depending on architecture Helps catch unintended modifications to loadable kernel module’s text and read-only data. It also prevents execution of module data.
  • Reducing Syscall Exposure: Syscalls are interfaces in which user-space and kernel-space can communicate and access each other. Some legacy syscalls may be exploitable and are generally not required on modern systems. Disabling syscalls when possible is a good way to reduce your attack surface.
Expected Conditional Architectures Kernel Versions Note
SECCOMP=Y ARM, ARM64, X86_32, X86_64 Varies depending on architecture This kernel feature is useful for number crunching applications that may need to compute untrusted bytecode during their execution. By using pipes or other transports made available to the process as file descriptors supporting the read/write syscalls, it’s possible to isolate those applications in their own address space using seccomp. Once seccomp is enabled via prctl(PR_SET_SECCOMP), it cannot be disabled and the task is only allowed to execute a few safe syscalls defined by each seccomp mode.
USELIB=is not set ARM, ARM64, X86_32, X86_64 4.5-4.20, 5.0-5.17 If enabled, this allows the libc5 and earlier dynamic linker usblib syscall. Should no longer be needed.
MODIFY_LDT_SYSCALL=is not set X86_32, X86_64 4.3-4.20, 5.0-5.17 Linux can allow user programs to install a per-process x86 Local Descriptor Table (LDT) using the modify_ldt(2) system call. This is required to run 16-bit or segmented code such as DOSEMU or some Wine programs. It is also used by some very old threading libraries. Enabling this feature adds a small amount of overhead to context switches and increases the low-level kernel attack surface. Disabling it removes the modify_ldt(2) system call.
LEGACY_VSYSCALL_NONE=Y X86_32, X86_64 4.4-4.20, 5.0-5.17 There will be no vsyscall mapping at all. This will eliminate any risk of ASLR bypass due to the vsyscall fixed address mapping. Attempts to use the vsyscalls will be reported to dmesg, so that either old or malicious userspace programs can be identified.
X86_VSYSCALL_EMULATION=is not set X86_32, X86_64 3.19, 4.0-4.20, 5.0-5.17 If set, this enables emulation of the legacy vsyscall page.
  • Security Policy Attacks: These are attacks which attempt to gain elevated (root) privileges within a system, generally through the use or execution of a misconfigured binary or file. Mitigations for this mostly rely on Linux Security Modules (LSMs) which extend discretionary access control (DAC) or implement mandatory access control (MAC, Security-Enhanced Linux).
Expected Conditional Architectures Kernel Versions Note
SECURITY=Y ARM, ARM64, X86_32, X86_64 2.5.50-2.5.75, 2.6.0-2.6.39, 3.0-3.19, 4.0-4.20, 5.0-5.17 This allows you to choose different security modules to be configured into your kernel.
SECURITY_YAMA=Y ARM, ARM64, X86_32, X86_64 3.4-3.19, 4.0-4.20, 5.0-5.17 This selects Yama, which extends DAC support with additional system-wide security settings beyond regular Linux discretionary access controls. Currently available is ptrace scope restriction. Like capabilities, this security module stacks with other LSMs. Further information can be found in Documentation/admin-guide/LSM/Yama.rst.
SECURITY_WRITABLE_HOOKS=is not set ARM, ARM64, X86_32, X86_64 4.12-4.20, 5.0-5.17 If SECURITY_SELINUX_DISABLE must be set, make sure this is not set. Subsequent patches will add RO hardening to LSM hooks, however, SELinux still needs to be able to perform runtime disablement after init to handle architectures where init-time disablement via boot parameters is not feasible. Introduce a new kernel configuration parameter CONFIG_SECURITY_WRITABLE_HOOKS, and a helper macro __lsm_ro_after_init, to handle this case.
SECURITY_SELINUX_DISABLE=is not set ARM, ARM64, X86_32, X86_64 2.6.6-2.6.39, 3.0-3.19, 4.0-4.20, 5.0-5.17 Do not allow NSA SELinux runtime disable
SECURITY_LOCKDOWN_LSM=Y ARM, ARM64, X86_32, X86_64 5.4-5.17 Enables the lockdown LSM, which enables you to set the lockdown=integrity or lockdown=confidentiality modes during boot. Integrity attempts to block userspace from modifying the running kernel, while confidentiality also restricts reading of confidential material.
SECURITY_LOCKDOWN_LSM_EARLY=Y ARM, ARM64, X86_32, X86_64 5.4-5.17 Enable lockdown LSM early in init
LOCK_DOWN_KERNEL_FORCE_CONFIDENTIALITY=Y ARM, ARM64, X86_32, X86_64 5.4-5.17 The kernel runs in confidentiality mode by default. Features that allow the kernel to be modified at runtime or that permit userland code to read confidential material held inside the kernel are disabled.
SECURITY_SAFESETID=Y ARM, ARM64, X86_32, X86_64 5.1-5.17 SafeSetID is an LSM module that gates the setid family of syscalls to restrict UID/GID transitions from a given UID/GID to only those approved by a system-wide whitelist. These restrictions also prohibit the given UIDs/GIDs from obtaining auxiliary privileges associated with CAP_SET{U/G}ID, such as allowing a user to set up user namespace UID mappings.
SECURITY_LOADPIN=Y ARM, ARM64, X86_32, X86_64 4.7-4.20, 5.0-5.17 Any files read through the kernel file reading interface (kernel modules, firmware, kexec images, security policy) can be pinned to the first filesystem used for loading. When enabled, any files that come from other filesystems will be rejected. This is best used on systems without an initrd that have a root filesystem backed by a read-only device such as dm-verity or a CDROM.
SECURITY_LOADPIN_ENFORCE=Y ARM, ARM64, X86_32, X86_64 4.20, 5.0-5.17 If selected, LoadPin will enforce pinning at boot. If not selected, it can be enabled at boot with the kernel parameter “loadpin.enforce=1”.

 

System Architecture

Many security features are architecture specific because of a specific hardware level reason (differing instruction set, caches, branch predictors, and more) or merely because they have not been implemented on a specific architecture. Looking at DEBUG_SET_MODULE_RONX, we find that it was a relatively recent addition for ARM and ARM64 architectures.

Expected Conditional Architectures Kernel Versions Note
DEBUG_SET_MODULE_RONX=Y X86_32, X86_64 2.6.38-2.6.39, 3.0-3.19, 4.0-4.10 Helps catch unintended modifications to loadable kernel module’s text and read-only data. It also prevents execution of module data.
DEBUG_SET_MODULE_RONX=Y ARM64 3.18-3.19, 4.0-4.10
DEBUG_SET_MODULE_RONX=Y ARM 3.14-3.19, 4.0-4.10

Looking at the Spectre and Meltdown variants, there are differing options depending on architecture as well:

Expected Conditional Architectures Kernel Versions Note
HARDEN_BRANCH_PREDICTOR=Y ARM, ARM64 4.16-4.20, 5.0-5.17 (Spectre related) Speculation attacks against some high-performance processors rely on being able to manipulate the branch predictor for a victim context by executing aliasing branches in the attacker context. Such attacks can be partially mitigated against by clearing internal branch predictor state and limiting the prediction logic in some situations.
RETPOLINE=Y X86_32, X86_64 4.15-4.20, 5.0-5.17 Avoid speculative indirect branches in kernel (Spectre Mitigation)

 

Timesys Kernel Hardening Analysis Tool

This tool is available as part of the meta-vigishield layer, learn more about VigiShield here.

This Yocto-based tool can perform some security-minded analysis of your kernel configuration. The tool generates a report that shows the status of many configuration items which we have assessed as being security related.

The output from the Timesys Kernel Hardening Analysis Tool is formatted as a Comma Separated List (CSV). As an example, here are the first few lines from a sample report are:

Detected Kernel Version: 5.0.19
Detected Architecture: ARM
Detected configuration at: /mnt/Projects/Yocto/build/tmp/work-shared/qemuarm-uboot/kernel
Report Generated At: 2022-02-01 12:22:07.088485
Expected Conditional Status    Priority Kernel Versions Category Note
GCC_PLUGIN_RANDSTRUCT=Y FAILED 3 (High) 4.13-4.20, 5.0-5.17 gcc_plugin Randomizes layout of sensitive kernel structures
GCC_PLUGIN_ARM_SSP_PER_TASK=Y SKIPPED (Version mismatch) 3 (High) 5.2-5.17 gcc_plugin Generates a separate stack canary value for each task, so if one task’s canary value is leaked it does not cause all other tasks to become vulnerable.
GCC_PLUGIN_STRUCTLEAK=Y SKIPPED (Version mismatch) 3 (High) 5.2-5.17 gcc_plugin This plugin is available to identify and zero-initialize stack variables that may have passed through uninitialized
STACKPROTECTOR=Y PASSED 3 (High) 4.18-4.20, 5.0-5.17 stack_canary This option turns on the “stack-protector” GCC feature. This feature puts, at the beginning of functions, a canary value on the stack just before the return address, and validates the value just before actually returning. Stack based buffer overflows (that need to overwrite this return address) now also overwrite the canary, which gets detected and the attack is then neutralized via a kernel panic.
STACKPROTECTOR_STRONG=Y PASSED 3 (High) 4.18-4.20, 5.0-5.17 stack_canary Adds the CONFIG_STACKPROTECTOR canary logic to additional conditions related to variable assignment.
INIT_ON_ALLOC_DEFAULT_ON=Y SKIPPED (Version mismatch) 3 (High) 5.3-5.17 memory_protection All page allocator and slab allocator memory will be zeroed when freed, eliminating many kinds of “uninitialized heap memory” flaws, especially heap content exposures.
INIT_ON_FREE_DEFAULT_ON=Y SKIPPED (Version mismatch) 3 (High) 5.3-5.17 memory_protection More expensive form of INIT_ON_ALLOC_DEFAULT_ON. The primary difference is that data lifetime in memory is reduced, as anything freed is wiped immediately, making live forensics or cold boot memory attacks unable to recover freed memory contents.
STRICT_KERNEL_RWX=Y PASSED 4.11-4.20, 5.0-5.17 3 (High) memory_protection If this is set, kernel text and rodata memory will be made read-only, and non-text memory will be made non-executable. This provides protection against certain security exploits (e.g. executing the heap or modifying text)
(MODULE_SIG_FORCE=Y) OR (MODULES=is not set) FAILED
DESCRIPTION: MODULES FAILED 1 (Low) 2.5.45-2.5.75, 2.6.0-2.6.39, 3.0-3.19, 4.0-4.20, 5.0-5.17 module_security You should not allow for modules to be loaded unless you have the proper signing and signature checks enabled. Allowing the kernel to load unsigned modules can be dangerous
DESCRIPTION: MODULE_SIG_FORCE FAILED 3 (High) 3.7-3.19, 4.0-4.20, 5.0-5.17 module_security Require modules to be validly signed

In this case, GCC_PLUGIN_RANDSTRUCT was not enabled in the kernel configuration file, so the return status is, “FAILED.

GCC_PLUGIN_ARM_SSP_PER_TASK, GCC_PLUGIN_STRUCTLEAK, INIT_ON_ALLOC_DEFAULT_ON, STRICT_KERNEL_RWX, and INIT_ON_FREE_DEFAULT_ON are all options which were available after the 5.0.19 kernel version, so those have been skipped with a, “SKIPPED (Version mismatch)” message.

The rest of the options were set appropriately and passed (STACKPROTECTOR and STACKPROTECTOR_STRONG).

Modules are also enabled without any forced signature checking, so the OR conditional [(MODULE_SIG_FORCE=Y) OR (MODULES=is not set)] for that has also failed.

Click here to get a free guide containing the entire list of recommended kernel security configurations to consider for hardening.

Subscribe to our RSS
Share this on Facebook
Share this on Twitter
Share this on LinkedIn

 

Nathan Barrett-Morrison, Senior Embedded Systems Engineer, has 8+ years of experience in designing and debugging embedded systems. In the past, he has been a software technical lead on ultrasonic-based digital processing systems and direct thermal printing solutions. He specializes in providing customers with lower level board support packages, drivers, and numerous other services. Nathan holds a bachelor’s degree in Electrical and Computer Engineering from OSU (Ohio State University).

About Timesys

Timesys has extensive experience with embedded system development and lifecycle management. Timesys has been instrumental in working with global leader semiconductor manufacturers with smart, quick and quality solutions for highly complex systems with accelerated product innovation and multiple product variants.

Discretionary Access Control (DAC) Hardening

Discretionary Access Control (DAC) Hardening

Subscribe to our RSS
Share this on Facebook
Share this on Twitter
Share this on LinkedIn

 

 

Discretionary Access Control hardening can further improve your embedded system’s security by limiting userspace access to proprietary intellectual property, exploitable binaries, and privileged information. The example permissions shown here are defaults produced during a demonstration Yocto build.

In Linux, a file has the following relevant parameters (when listing a file with the “ls” command):

File Info Owner Group Users Other Users
Name Owner Group Read Write Execute Read Write Execute Read Write Execute
/bin/tar.tar root root r w x r x r x

Where:

r Readable
w Writable
x Executable
No permissions
s Executable + Set User ID or Group ID

To fully secure your system and limit your vulnerable attack surface, these permissions should be restricted as much as possible.

 

User Accounts

You’ll want to ensure that your applications are well isolated from one another through the use of strict user accounting. Many embedded systems run everything under the root account. If security is a concern, this is not acceptable. If an application running as root becomes subject to an exploit (zero-day or otherwise), an attacker will be able to gain root access to your system. Compartmentalizing and restricting access between applications is critical.

This is why applications like NTPD, Apache2, Avahi, and DHCPD change their user after being initially launched as root. If someone were to successfully perform an injection-style attack on one of these network-based applications, root privileges are not exposed and, ideally, only the attacked application’s settings and files are modifiable.

Proprietary applications and intellectual property should also be executed under a separate, non-root, account. Alternatively, you can also promptly drop root permissions and change users after launching if desired. These applications are usually not as heavily tested for security vulnerabilities, as they’re not open source. This makes keeping them isolated from root access especially important, as they may be easier to attack.

 

SUID/SGID Protections

Also check for “Set User ID” and “Set Group ID” bits in your file system permissions. These have been commonly exploited in the past and can lead to unexpected root privilege escalation. Consider this CVE that is related to exploiting the commonly used Shadow utility (groupadd, groupdel, groupmod, useradd, userdel, and usermod).

  • In an embedded system, you may not need to add/remove/modify users after building the BSP. If so, consider deleting these tools from your system altogether.

When using modern versions of Busybox, the binary has been split in to two pieces. One with SUID permissions and one without.

In this demonstration build, the SUID binary is used (via symbolic links) for the following:

/usr/bin/passwd     -> /bin/busybox.suid
/usr/bin/traceroute -> /bin/busybox.suid
/usr/bin/vlock      -> /bin/busybox.suid
/bin/ping           -> /bin/busybox.suid
/bin/ping6          -> /bin/busybox.suid
/bin/login          -> /bin/busybox.suid
/bin/su             -> /bin/busybox.suid

You may not need some (or any) of these features in your field-ready systems. They should be disabled via the Busybox configuration whenever possible.

File Info Owner Group Users Other Users
Name Owner Group Read Write Execute Read Write Execute Read Write Execute
/bin/busybox.suid root root r w s r x r x
/bin/busybox.nosuid root root r w x r x r x

 

Read Permissions

Restricting group and other users from read access on any system files which may expose vulnerable information is another way to improve security. By default, Yocto sets most files to be user and group readable. This is largely unnecessary, as not all users should need read access on configuration files and applications which they do not have execution privileges for.

For example, consider fw_env.config. This file is used to control the address and size of the bootloader›s environment information (from within Linux userspace). While removing the read permissions on it may not prevent a motivated attacker from finding the addresses manually, why make it easier for them? There’s most likely no reason every user in your system needs read access to this file.

root@board:~# cat /etc/fw_env.config

# [Device]      [Offset]    [Size]   [Sector Size]   [Sector Count]
/dev/mmcblk1    0xC0000     0x2000      0x2000             1

 

File Info Owner Group Users Other Users
Name Owner Group Read Write Execute Read Write Execute Read Write Execute
/etc/fw_env.config root root r w r r

Properly adjusting read permissions may also be important for clone and binary analysis protections. Consider that a malicious entity is targeting your system in an attempt to copy out your proprietary binary files. The ARMv7 and ARMv8 architectures are so common now, it may be exceptionally easy to transplant these files on to a replica board. They may also disassemble said binary and attempt to find security flaws in it. If you can prevent read access to the application, it becomes more difficult for an attacker to obtain a binary copy.

Contemplate a system with three accounts named: root, user, and proprietary_app. Your system initialization manager will start up as root and launch your application under the separate proprietary_app account (or the application itself will quickly do so). At this point, if you have the permissions adjusted accordingly:

  • The application does not expose root privileges if an attacker finds an exploit on it, as it’s running under a separate account.
  • The application can only be read by two accounts (root and proprietary_app). If these two accounts are then passwordless, obtaining a binary dump is more difficult (bruteforce attacks via serial/SSH are not possible). It would also protect against offline cracking of the /etc/shadow file, although if an attacker is able to obtain this then they’ve already escalated to root privileges and that most likely becomes a moot point.
File Info Owner Group Users Other Users
Name Owner Group Read Write Execute Read Write Execute Read Write Execute
/usr/bin/some_proprietary_app proprietary_app proprietary_app r w x

 

Execution Permissions

This is another avenue through which you can help protect your system. When looking at the executable binaries in your system, you may find group and user executable bits which do not need to be set. Considering U-Boot’s environment tools (from Linux userspace) again, the tool binaries are also unnecessarily executable by group and other users. This may give an attacker an easy method of tampering with the bootloader (if it has not been adequately hardened).

File Info Owner Group Users Other Users
Name Owner Group Read Write Execute Read Write Execute Read Write Execute
./sbin/fw_printenv root root r w x r x r x
./sbin/fw_setenv root root r w x r x r x

 

Write Permissions

During the demonstration build with Yocto, only the owner appears to be given write access to files. There does not appear to be any improvement necessary here. Your build system’s results may vary. If you find a file with inappropriate write permissions then you will want to fix it.

 

Timesys DAC Reporting Tool

Timesys has a discretionary access control review tool which will generate a comma-separated value (CSV) file during your Yocto build process. This file contains a list of all your binaries, user and group ownerships, and their various read/write/execute permissions. This enables you to more quickly scan your file system for potential conflicts. The tool is part of Timesys VigiShield offering. To view a sample report generated by the tool, click here.

Subscribe to our RSS
Share this on Facebook
Share this on Twitter
Share this on LinkedIn

 

Nathan Barrett-Morrison, Senior Embedded Systems Engineer, has 8+ years of experience in designing and debugging embedded systems. In the past, he has been a software technical lead on ultrasonic-based digital processing systems and direct thermal printing solutions. He specializes in providing customers with lower level board support packages, drivers, and numerous other services. Nathan holds a bachelor’s degree in Electrical and Computer Engineering from OSU (Ohio State University).

About Timesys

Timesys has extensive experience with embedded system development and lifecycle management. Timesys has been instrumental in working with global leader semiconductor manufacturers with smart, quick and quality solutions for highly complex systems with accelerated product innovation and multiple product variants.