Provided by: swupdate_2021.11-1_amd64 
NAME
swupdate - Swupdate Documentation [image]
SWUpdate provides a reliable way to update the software on an embedded system. Sources are hosted at
https://github.com/sbabic/swupdate
SOFTWARE MANAGEMENT ON EMBEDDED SYSTEMS
Embedded Systems become more and more complex, and their software reflects the augmented complexity. New
features and fixes let much more as desirable that the software on an embedded system can be updated in
an absolutely reliable way.
On a Linux-based system, we can find in most cases the following elements:
• the boot loader.
• the kernel and the DT (Device Tree) file.
• the root file system
• other file systems, mounted at a later point
• customer data, in raw format or on a file system
• application specific software. For example, firmware to be downloaded on connected micro-controllers,
and so on.
Generally speaking, in most cases it is required to update kernel and root file system, preserving user
data - but cases vary.
In only a few cases it is required to update the boot loader, too. In fact, updating the boot loader is
quite always risky, because a failure in the update breaks the board. Restoring a broken board is
possible in some cases, but this is not left in most cases to the end user and the system must be sent
back to the manufacturer.
There are a lot of different concepts about updating the software. I like to expose some of them, and
then explain why I have implemented this project.
Updating through the boot loader
Boot loaders do much more as simply start the kernel. They have their own shell and can be managed using
a processor's peripheral, in most cases a serial line. They are often script-able, letting possible to
implement some kind of software update mechanism.
However, I found some drawbacks in this approach, that let me search for another solution, based on an
application running on Linux:
Boot loaders have limited access to peripherals
Not all peripherals supported by the kernel are available with the boot loader. When it makes sense to
add support to the kernel, because the peripheral is then available by the main application, it does not
always make sense to duplicate the effort to port the driver to the boot loader.
Boot loader's drivers are not updated
Boot loader's drivers are mostly ported from the Linux kernel, but due to adaptations they are not later
fixed or synchronized with the kernel, while bug fixes flow regularly in the Linux kernel. Some
peripherals can then work in a not reliable ways, and fixing the issues can be not easy. Drivers in boot
loaders are more or less a fork of the respective drivers in kernel.
As example, the UBI / UBIFS for NAND devices contains a lot of fixes in the kernel, that are not ported
back to the boot loaders. The same can be found for the USB stack. The effort to support new peripherals
or protocols is better to be used for the kernel as for the boot loaders.
Reduced file systems
The number of supported file systems is limited and porting a file system to the boot loader requires
high effort.
Network support is limited
Network stack is limited, generally an update is possible via UDP but not via TCP.
Interaction with the operator
It is difficult to expose an interface to the operator, such as a GUI with a browser or on a display.
A complex logic can be easier implemented inside an application else in the boot loader. Extending the
boot loader becomes complicated because the whole range of services and libraries are not available.
Boot loader's update advantages
However, this approach has some advantages, too:
• software for update is generally simpler. - smaller footprint: a stand-alone application only for
software management requires an own kernel and a root file system. Even if their size can be trimmed
dropping what is not required for updating the software, their size is not negligible.
Updating through a package manager
All Linux distributions are updating with a package manager. Why is it not suitable for embedded ?
I cannot say it cannot be used, but there is an important drawback using this approach. Embedded systems
are well tested with a specific software. Using a package manager can put weirdness because the software
itself is not anymore atomic, but split into a long list of packages. How can we be assured that an
application with library version x.y works, and also with different versions of the same library? How can
it be successfully tested?
For a manufacturer, it is generally better to say that a new release of software (well tested by its test
engineers) is released, and the new software (or firmware) is available for updating. Splitting in
packages can generate nightmare and high effort for the testers.
The ease of replacing single files can speed up the development, but it is a software-versions nightmare
at the customer site. If a customer report a bug, how can it is possible that software is "version 2.5"
when a patch for some files were sent previously to the customer ?
An atomic update is generally a must feature for an embedded system.
Strategies for an application doing software upgrade
Instead of using the boot loader, an application can take into charge to upgrade the system. The
application can use all services provided by the OS. The proposed solution is a stand-alone software,
that follow customer rules and performs checks to determine if a software is installable, and then
install the software on the desired storage.
The application can detect if the provided new software is suitable for the hardware, and it is can also
check if the software is released by a verified authority. The range of features can grow from small
system to a complex one, including the possibility to have pre- and post- install scripts, and so on.
Different strategies can be used, depending on the system's resources. I am listing some of them.
Double copy with fall-back
If there is enough space on the storage to save two copies of the whole software, it is possible to
guarantee that there is always a working copy even if the software update is interrupted or a power off
occurs.
Each copy must contain the kernel, the root file system, and each further component that can be updated.
It is required a mechanism to identify which version is running.
SWUpdate should be inserted in the application software, and the application software will trigger it
when an update is required. The duty of SWUpdate is to update the stand-by copy, leaving the running
copy of the software untouched.
A synergy with the boot loader is often necessary, because the boot loader must decide which copy should
be started. Again, it must be possible to switch between the two copies. After a reboot, the boot loader
decides which copy should run. [image]
Check the chapter about boot loader to see which mechanisms can be implemented to guarantee that the
target is not broken after an update.
The most evident drawback is the amount of required space. The available space for each copy is less than
half the size of the storage. However, an update is always safe even in case of power off.
This project supports this strategy. The application as part of this project should be installed in the
root file system and started or triggered as required. There is no need of an own kernel, because the two
copies guarantees that it is always possible to upgrade the not running copy.
SWUpdate will set bootloader's variable to signal the that a new image is successfully installed.
Single copy - running as standalone image
The software upgrade application consists of kernel (maybe reduced dropping not required drivers) and a
small root file system, with the application and its libraries. The whole size is much less than a single
copy of the system software. Depending on set up, I get sizes from 2.5 until 8 MB for the stand-alone
root file system. If the size is very important on small systems, it becomes negligible on systems with a
lot of storage or big NANDs.
The system can be put in "upgrade" mode, simply signaling to the boot loader that the upgrading software
must be started. The way can differ, for example setting a boot loader environment or using and external
GPIO.
The boot loader starts "SWUpdate", booting the SWUpdate kernel and the initrd image as root file system.
Because it runs in RAM, it is possible to upgrade the whole storage. Differently as in the double-copy
strategy, the systems must reboot to put itself in update mode.
This concept consumes less space in storage as having two copies, but it does not guarantee a fall-back
without updating again the software. However, it can be guaranteed that the system goes automatically in
upgrade mode when the productivity software is not found or corrupted, as well as when the upgrade
process is interrupted for some reason. [image]
In fact, it is possible to consider the upgrade procedure as a transaction, and only after the successful
upgrade the new software is set as "boot-able". With these considerations, an upgrade with this strategy
is safe: it is always guaranteed that the system boots and it is ready to get a new software, if the old
one is corrupted or cannot run. With U-Boot as boot loader, SWUpdate is able to manage U-Boot's
environment setting variables to indicate the start and the end of a transaction and that the storage
contains a valid software. A similar feature for GRUB environment block modification as well as for EFI
Boot Guard has been introduced.
SWUpdate is mainly used in this configuration. The recipes for Yocto generate an initrd image containing
the SWUpdate application, that is automatically started after mounting the root file system. [image]
Something went wrong ?
Many things can go wrong, and it must be guaranteed that the system is able to run again and maybe able
to reload a new software to fix a damaged image. SWUpdate works together with the boot loader to identify
the possible causes of failures. Currently U-Boot, GRUB, and EFI Boot Guard are supported.
We can at least group some of the common causes:
• damage / corrupted image during installing. SWUpdate is able to recognize it and the update process is
interrupted. The old software is preserved and nothing is really copied into the target's storage.
• corrupted image in the storage (flash)
• remote update interrupted due to communication problem.
• power-failure
SWUpdate works as transaction process. The boot loader environment variable "recovery_status" is set to
signal the update's status to the boot loader. Of course, further variables can be added to fine tuning
and report error causes. recovery_status can have the values "progress", "failed", or it can be unset.
When SWUpdate starts, it sets recovery_status to "progress". After an update is finished with success,
the variable is erased. If the update ends with an error, recovery_status has the value "failed".
When an update is interrupted, independently from the cause, the boot loader recognizes it because the
recovery_status variable is in "progress" or "failed". The boot loader can then start again SWUpdate to
load again the software (single-copy case) or run the old copy of the application (double-copy case).
Power Failure
If a power off occurs, it must be guaranteed that the system is able to work again - starting again
SWUpdate or restoring an old copy of the software.
Generally, the behavior can be split according to the chosen scenario:
• single copy: SWUpdate is interrupted and the update transaction did not end with a success. The boot
loader is able to start SWUpdate again, having the possibility to update the software again.
• double copy: SWUpdate did not switch between stand-by and current copy. The same version of software,
that was not touched by the update, is started again.
To be completely safe, SWUpdate and the bootloader need to exchange some information. The bootloader must
detect if an update was interrupted due to a power-off, and restart SWUpdate until an update is
successful. SWUpdate supports the U-Boot, GRUB, and EFI Boot Guard bootloaders. U-Boot and EFI Boot
Guard have a power-safe environment which SWUpdate is able to read and change in order to communicate
with them. In case of GRUB, a fixed 1024-byte environment block file is used instead. SWUpdate sets a
variable as flag when it starts to update the system and resets the same variable after completion. The
bootloader can read this flag to check if an update was running before a power-off. [image]
What about upgrading SWUpdate itself ?
SWUpdate is thought to be used in the whole development process, replacing customized process to update
the software during the development. Before going into production, SWUpdate is well tested for a project.
If SWUpdate itself should be updated, the update cannot be safe if there is only one copy of SWUpdate in
the storage. Safe update can be guaranteed only if SWUpdate is duplicated.
There are some ways to circumvent this issue if SWUpdate is part of the upgraded image:
• have two copies of SWUpdate
• take the risk, but have a rescue procedure using the boot loader.
What about upgrading the Boot loader ?
Updating the boot loader is in most cases a one-way process. On most SOCs, there is no possibility to
have multiple copies of the boot loader, and when boot loader is broken, the board does not simply boot.
Some SOCs allow one to have multiple copies of the boot loader. But again, there is no general solution
for this because it is very hardware specific.
In my experience, most targets do not allow one to update the boot loader. It is very uncommon that the
boot loader must be updated when the product is ready for production.
It is different if the U-Boot environment must be updated, that is a common practice. U-Boot provides a
double copy of the whole environment, and updating the environment from SWUpdate is power-off safe. Other
boot loaders can or cannot have this feature.
LICENSE
SWUpdate is Free Software. It is copyrighted by Stefano Babic and many others who contributed code (see
the actual source code and the git commit messages for details). You can redistribute SWUpdate and/or
modify it under the terms of version 2 of the GNU General Public License as published by the Free
Software Foundation. Some files can also be distributed, at your option, under any later version of the
GNU General Public License -- see individual files for exceptions.
To make this easier, license headers in the source files will be replaced with a single line reference to
Unique License Identifiers as defined by the Linux Foundation's SPDX project [1]. For example, in a
source file the full "GPL v2.0 only" header text will be replaced by a single line:
SPDX-License-Identifier: GPL-2.0-only
Ideally, the license terms of all files in the source tree should be defined by such License Identifiers;
in no case a file can contain more than one such License Identifier list.
If a "SPDX-License-Identifier:" line references more than one Unique License Identifier, then this means
that the respective file can be used under the terms of either of these licenses, i. e. with
SPDX-License-Identifier: GPL-2.0-only OR BSD-3-Clause
you can choose between GPL-2.0-only and BSD-3-Clause licensing.
We use the SPDX Unique License Identifiers (SPDX-Identifiers)
LICENSES
───────────────────────────────────────────────────────────────────
Full name SPDX Identifier OSI Approved
───────────────────────────────────────────────────────────────────
GNU General Public License GPL-2.0-only Y
v2.0 only
───────────────────────────────────────────────────────────────────
GNU Lesser General Public LGPL-2.1-or-later Y
License v2.1 or later
───────────────────────────────────────────────────────────────────
BSD 1-Clause License BSD-1-Clause Y
───────────────────────────────────────────────────────────────────
BSD 2-Clause License BSD-2-Clause Y
───────────────────────────────────────────────────────────────────
BSD 3-Clause "New" or BSD-3-Clause Y
"Revised" License
───────────────────────────────────────────────────────────────────
MIT License MIT Y
───────────────────────────────────────────────────────────────────
Creative Commons Zero 1.0 CC0-1.0 N
Universal (CC0)
───────────────────────────────────────────────────────────────────
Creative Commons Attribution CC-BY-SA-4.0 Y
Share Alike 4.0
───────────────────────────────────────────────────────────────────
ISC License (ISC) ISC Y
┌──────────────────────────────┬───────────────────┬──────────────┐
│ │ │ │
SWUPDATE: SOFTWARE UPDATE│FOR EMBEDDED SYSTEM │ │ │
Overview │ │ │ │
--
UPDATE STRATEGY EXAMPLES
SWUpdate is a building block and it allows one to design and implementing its own update strategy. Even
if many projects have common ways for updating, it is possible to high customize the update for each
project. The most common strategies (single-copy and dual-copy) were already described at the beginning
of this documentation and of course are well supported in SWUpdate.
Single copy - running as standalone image
See single_copy.
Double copy with fall-back
See double_copy.
Combine double-copy with rescue system
This provides a recovery procedure to cover update failure in severe cases when software is damaged. In
case none of the copy can be started, the bootloader will start the rescue system (possibly stored on
another storage as the main system) to try to rescue the board. [image]
The rescue system can be updated as well during a standard update.
Split system update with application update
Updating a whole image is quite straightforward, but this means to transfer bigger amount of data if just
a few files are updated. It is possible to split theupdate in several smaller parts to reduce the
transfer size. This requires a special care to take care of compatibility between system and application,
that can be solved with customized Lua scripts in the sw-description file. SWUpdate supports versioning
for each artefact, and anyone can add own rules to verify compatibility between components. [image]
Configuration update
Thought to update the software, SWUpdate can be used to install configuration data as well. Build system
can create configuration SWU with files / data for the configuration of the system. There is no
requirements what these SWU should contains - it is duty of the integrator to build them and make them
suitable for his own project. Again, configuration data can be updated as separate process using one of
the above scenarios.
SWUPDATE: SYNTAX AND TAGS WITH THE DEFAULT PARSER
Introduction
SWUpdate uses the library "libconfig" as default parser for the image description. However, it is
possible to extend SWUpdate and add an own parser, based on a different syntax and language as the one
supported by libconfig. In the examples directory there is the code for a parser written in Lua, with the
description in XML.
Using the default parser, sw-description follows the syntax rules described in the libconfig manual.
Please take a look at http://www.hyperrealm.com/libconfig/libconfig_manual.html for an explanation of
basic types. The whole description must be contained in the sw-description file itself: using of the
#include directive is not allowed by SWUpdate.
The following example explains better the implemented tags:
software =
{
version = "0.1.0";
description = "Firmware update for XXXXX Project";
hardware-compatibility: [ "1.0", "1.2", "1.3"];
/* partitions tag is used to resize UBI partitions */
partitions: ( /* UBI Volumes */
{
name = "rootfs";
device = "mtd4";
size = 104896512; /* in bytes */
},
{
name = "data";
device = "mtd5";
size = 50448384; /* in bytes */
}
);
images: (
{
filename = "rootfs.ubifs";
volume = "rootfs";
},
{
filename = "swupdate.ext3.gz.u-boot";
volume = "fs_recovery";
},
{
filename = "sdcard.ext3.gz";
device = "/dev/mmcblk0p1";
compressed = "zlib";
},
{
filename = "bootlogo.bmp";
volume = "splash";
},
{
filename = "uImage.bin";
volume = "kernel";
},
{
filename = "fpga.txt";
type = "fpga";
},
{
filename = "bootloader-env";
type = "bootloader";
}
);
files: (
{
filename = "README";
path = "/README";
device = "/dev/mmcblk0p1";
filesystem = "vfat"
}
);
scripts: (
{
filename = "erase_at_end";
type = "lua";
},
{
filename = "display_info";
type = "lua";
}
);
bootenv: (
{
name = "vram";
value = "4M";
},
{
name = "addfb";
value = "setenv bootargs ${bootargs} omapfb.vram=1:2M,2:2M,3:2M omapdss.def_disp=lcd"
}
);
}
The first tag is "software". The whole description is contained in this tag. It is possible to group
settings per device by using Board specific settings.
Handling configuration differences
The concept can be extended to deliver a single image containing the release for multiple devices. Each
device has its own kernel, dtb and root filesystem, or they can share some parts.
Currently this is managed (and already used in a real project) by writing an own parser, that checks
which images must be installed after recognizing which is the device where software is running.
Because the external parser can be written in Lua and it is completely customizable, everybody can set
his own rules. For this specific example, the sw-description is written in XML format, with tags
identifying the images for each device. To run it, the liblxp library is needed.
<?xml version="1.0" encoding="UTF-8"?>
<software version="1.0">
<name>Update Image</name>
<version>1.0.0</version>
<description>Firmware for XXXXX Project</description>
<images>
<image device="firstdevice" version="0.9">
<stream name="dev1-uImage" type="ubivol" volume="kernel" />
<stream name="dev1.dtb" type="ubivol" volume="dtb" />
<stream name="dev1-rootfs.ubifs" type="ubivol" volume="rootfs"/>
<stream name="dev1-uboot-env" type="uboot" />
<stream name="raw_vfat" type="raw" dest="/dev/mmcblk0p4" />
<stream name="sdcard.lua" type="lua" />
</image>
<image device="seconddevice" version="0.9">
<stream name="dev2-uImage" type="ubivol" volume="kernel" />
<stream name="dev2.dtb" rev="0.9" type="ubivol" volume="dtb" />
<stream name="dev2-rootfs.ubifs" type="ubivol" volume="rootfs"/>
</image>
</images>
</software>
The parser for this is in the /examples directory. By identifying which is the running device, the
parser return a table containing the images that must be installed and their associated handlers. By
reading the delivered image, SWUpdate will ignore all images that are not in the list processed by the
parser. In this way, it is possible to have a single delivered image for the update of multiple devices.
Multiple devices are supported by the default parser, too.
software =
{
version = "0.1.0";
target-1 = {
images: (
{
...
}
);
};
target-2 = {
images: (
{
...
}
);
};
}
In this way, it is possible to have a single image providing software for each device you have.
By default the hardware information is extracted from /etc/hwrevision file. The file should contain a
single line in the following format:
<boardname> <revision>
Where:
• <revision> will be used for matching with hardware compatibility list
• <boardname> can be used for grouping board specific settings
Software collections
Software collections and operation modes can be used to implement a dual copy strategy. The simplest case
is to define two installation locations for the firmware image and call SWUpdate selecting the
appropriate image.
software =
{
version = "0.1.0";
stable = {
copy-1: {
images: (
{
device = "/dev/mtd4"
...
}
);
}
copy-2: {
images: (
{
device = "/dev/mtd5"
...
}
);
}
};
}
In this way it is possible to specify that copy-1 gets installed to /dev/mtd4, while copy-2 to /dev/mtd5.
By properly selecting the installation locations, SWUpdate will update the firmware in the other slot.
The method of image selection is out of the scope of SWUpdate and user is responsible for calling
SWUpdate passing proper settings.
Priority finding the elements in the file
SWUpdate search for entries in the sw-description file according to the following priority:
1. Try <boardname>.<selection>.<mode>.<entry>
2. Try <selection>.<mode>.<entry>
3. Try <boardname>.<entry>
4. Try <entry>
Take an example. The following sw-description describes the release for a set of boards.
software =
{
version = "0.1.0";
myboard = {
stable = {
copy-1: {
images: (
{
device = "/dev/mtd4"
...
}
);
}
copy-2: {
images: (
{
device = "/dev/mtd5"
...
}
);
}
}
}
stable = {
copy-1: {
images: (
{
device = "/dev/mtd6"
...
}
);
}
copy-2: {
images: (
{
device = "/dev/mtd7"
...
}
);
}
}
}
On myboard, SWUpdate searches and find myboard.stable.copy1(2). When running on different boards,
SWUpdate does not find an entry corresponding to the boardname and it fallbacks to the version without
boardname. This lets realize the same release for different boards having a complete different hardware.
myboard could have a eMMC and an ext4 filesystem, while another device can have raw flash and install an
UBI filesystem. Nevertheless, they are both just a different format of the same release and they could be
described together in sw-description. It is important to understand the priorities how SWUpdate scans
for entries during the parsing.
Using links
sw-description can become very complex. Let's think to have just one board, but in multiple hw revision
and they differ in Hardware. Some of them can be grouped together, some of them require a dedicated
section. A way (but not the only one !) could be to add mode and selects the section with -e stable,<rev
number>.
software =
{
version = "0.1.0";
myboard = {
stable = {
hardware-compatibility: ["1.0", "1.2", "2.0", "1.§, "3.0", "3.1"];
rev-1.0: {
images: (
...
);
scripts: (
...
);
}
rev-1.2: {
hardware-compatibility: ["1.2"];
images: (
...
);
scripts: (
...
);
}
rev-2.0: {
hardware-compatibility: ["2.0"];
images: (
...
);
scripts: (
...
);
}
rev-1.3: {
hardware-compatibility: ["1.3"];
images: (
...
);
scripts: (
...
);
}
rev-3.0:
{
hardware-compatibility: ["3.0"];
images: (
...
);
scripts: (
...
);
}
rev-3.1:
{
hardware-compatibility: ["3.1"];
images: (
...
);
scripts: (
...
);
}
}
}
}
If each of them requires an own section, it is the way to do. Anyway, it is more probable than revisions
can be grouped together, for example board with the same major revision number could have the same
installation instructions. This leads in the example to 3 groups for rev1.X, rev2.X and rev3.X. Links
allow one to group section together. When a "ref" is found when SWUpdate searches for a group (images,
files, script, bootenv), it replaces the current path in the tree with the value of the string. In this
way, the example above can be written in this way:
software =
{
version = "0.1.0";
myboard = {
stable = {
hardware-compatibility: ["1.0", "1.2", "2.0", "1.3, "3.0", "3.1"];
rev-1x: {
images: (
...
);
scripts: (
...
);
}
rev1.0 = {
ref = "#./rev-1x";
}
rev1.2 = {
ref = "#./rev-1x";
}
rev1.3 = {
ref = "#./rev-1x";
}
rev-2x: {
images: (
...
);
scripts: (
...
);
}
rev2.0 = {
ref = "#./rev-2x";
}
rev-3x: {
images: (
...
);
scripts: (
...
);
}
rev3.0 = {
ref = "#./rev-3x";
}
rev3.1 = {
ref = "#./rev-3x";
}
}
}
}
The link can be absolute or relative. The keyword "ref" is used to indicate a link. If this is found,
SWUpdate will traverse the tree and replaces the current path with the values find in the string pointed
by "ref". There are simple rules for a link:
• it must start with the character '#'
• "." points to the current level in the tree, that means the parent of "ref"
• ".." points to the parent level in the tree
• "/" is used as filed separator in the link
A relative path has a number of leading "../" to move the current cursor to the parent leaf of the tree.
In the following example, rev40 sets a link to a "common" section, where images is found. This is sets
via a link, too, to a section in the parent node. The path software.myboard.stable.common.images is
then replaced by software.myboard.stable.trythis
software =
{
version = {
ref = "#./commonversion";
}
hardware-compatibility = ["rev10", "rev11", "rev20"];
commonversion = "0.7-linked";
pc:{
stable:{
common:{
images =
{
ref = "#./../trythis";
}
};
trythis:(
{
filename = "rootfs1.ext4";
device = "/dev/mmcblk0p8";
type = "raw";
} ,
{
filename = "rootfs5.ext4";
device = "/dev/mmcblk0p7";
type = "raw";
}
);
pdm3rev10:
{
images:(
{
filename = "rootfs.ext3"; device = "/dev/mmcblk0p2";}
);
uboot:(
{ name = "bootpart";
value = "0:2";}
);
};
pdm3rev11 =
{
ref = "#./pdm3rev10";
}
pdm3rev20 =
{
ref = "#./pdm3rev10";
}
pdm3rev40 =
{
ref = "#./common";
}
};
};
}
Each entry in sw-description can be redirect by a link as in the above example for the "version"
attribute.
hardware-compatibility
hardware-compatibility: [ "major.minor", "major.minor", ... ]
This entry lists the hardware revisions that are compatible with this software image.
Example:
hardware-compatibility: [ "1.0", "1.2", "1.3"];
This defines that the software is compatible with HW-Revisions 1.0, 1.2 and 1.3, but not with 1.1 or any
other version not explicitly listed here. In the above example, compatibility is checked by means of
string comparison. If the software is compatible with a large number of hardware revisions, it may get
cumbersome to enumerate all compatible versions. To allow more compact specifications, regular
expressions (POSIX extended) can be used by adding a prefix #RE: to the entry. Rewriting the above
example would yield:
hardware-compatibility: [ "#RE:^1\.[023]$" ];
It is in the responsibility of the respective project to find the revision of the board on which SWUpdate
is running. No assumptions are made about how the revision can be obtained (GPIOs, EEPROM,..) and each
project is free to select the most appropriate way. In the end the result must be written to the file
/etc/hwrevision (or in another file if specified as configuration option) before SWUpdate is started.
partitions : UBI layout
This tag allows one to change the layout of UBI volumes. Please take care that MTDs are not touched and
they are configured by the Device Tree or in another way directly in kernel.
partitions: (
{
name = <volume name>;
size = <size in bytes>;
device = <MTD device>;
}
);
All fields are mandatory. SWUpdate searches for a volume of the given name and if necessary adjusts size
or type (see below). If no volume with the given name is found, a new volume is created on the UBI device
attached to the MTD device given by device. device can be specified by number (e.g. "mtd4") or by name
(the name of the MTD device, e.g. "ubi_partition"). The UBI device is attached automatically.
The default behavior of swupdate is to create a dynamic UBI volume. To create a static volume, add a line
data = "static"; to the respective partition entry.
If a size of 0 is given, the volume will be deleted if it exists. This can be used to remove orphan
volumes possibly created by older software versions which are not required anymore.
images
The tag "images" collects the image that are installed to the system. The syntax is:
images: (
{
filename[mandatory] = <Name in CPIO Archive>;
volume[optional] = <destination volume>;
device[optional] = <destination volume>;
mtdname[optional] = <destination mtd name>;
type[optional] = <handler>;
/* optionally, the image can be copied at a specific offset */
offset[optional] = <offset>;
/* optionally, the image can be compressed if it is in raw mode */
compressed;
},
/* Next Image */
.....
);
volume is only used to install the image in a UBI volume. volume and device cannot be used at the same
time. If device is set, the raw handler is automatically selected.
The following example is to update a UBI volume:
{
filename = "core-image-base.ubifs";
volume = "rootfs";
}
To update an image in raw mode, the syntax is:
{
filename = "core-image-base.ext3";
device = "/dev/mmcblk0p1";
}
To flash an image at a specific offset, the syntax is:
{
filename = "u-boot.bin";
device = "/dev/mmcblk0p1";
offset = "16K";
}
The offset handles the following multiplicative suffixes: K=1024 and M=1024*1024.
However, writing to flash in raw mode must be managed in a special way. Flashes must be erased before
copying, and writing into NAND must take care of bad blocks and ECC errors. For this reasons, the handler
"flash" must be selected:
For example, to copy the kernel into the MTD7 of a NAND flash:
{
filename = "uImage";
device = "mtd7";
type = "flash";
}
The filename is mandatory. It is the Name of the file extracted by the stream. volume is only mandatory
in case of UBI volumes. It should be not used in other cases.
Alternatively, for the handler “flash”, the mtdname can be specified, instead of the device name:
{
filename = "uImage";
mtdname = "kernel";
type = "flash";
}
Files
It is possible to copy single files instead of images. This is not the preferred way, but it can be used
for debugging or special purposes.
files: (
{
filename = <Name in CPIO Archive>;
path = <path in filesystem>;
device[optional] = <device node >;
filesystem[optional] = <filesystem for mount>;
properties[optional] = {create-destination = "true";}
}
);
Entries in "files" section are managed as single files. The attributes "filename" and "path" are
mandatory. Attributes "device" and "filesystem" are optional; they tell SWUpdate to mount device (of the
given filesystem type, e.g. "ext4") before copying "filename" to "path". Without "device" and
"filesystem", the "filename" will be copied to "path" in the current rootfs.
As a general rule, swupdate doesn't copy out a file if the destination path doesn't exists. This behavior
could be changed using the special property "create-destination".
Scripts
Scripts runs in the order they are put into the sw-description file. The result of a script is valuated
by SWUpdate, that stops the update with an error if the result is <> 0.
They are copied into a temporary directory before execution and their name must be unique inside the same
cpio archive.
If no type is given, SWUpdate default to "lua".
Lua
scripts: (
{
filename = <Name in CPIO Archive>;
type = "lua";
}
);
Lua scripts are run using the internal interpreter.
They must have at least one of the following functions:
function preinst()
SWUpdate scans for all scripts and check for a preinst function. It is called before installing the
images.
function postinst()
SWUpdate scans for all scripts and check for a postinst function. It is called after installing the
images.
shellscript
scripts: (
{
filename = <Name in CPIO Archive>;
type = "shellscript";
}
);
Shell scripts are called via system command. SWUpdate scans for all scripts and calls them before and
after installing the images. SWUpdate passes 'preinst' or 'postinst' as first argument to the script. If
the data attribute is defined, its value is passed as the last argument(s) to the script.
preinstall
scripts: (
{
filename = <Name in CPIO Archive>;
type = "preinstall";
}
);
preinstall are shell scripts and called via system command. SWUpdate scans for all scripts and calls
them before installing the images. If the data attribute is defined, its value is passed as the last
argument(s) to the script.
postinstall
scripts: (
{
filename = <Name in CPIO Archive>;
type = "postinstall";
}
);
postinstall are shell scripts and called via system command. SWUpdate scans for all scripts and calls
them after installing the images. If the data attribute is defined, its value is passed as the last
argument(s) to the script.
Update Transaction and Status Marker
By default, SWUpdate sets the bootloader environment variable "recovery_status" to "in_progress" prior to
an update operation and either unsets it or sets it to "failed" after the update operation. This is an
interface for SWUpdate-external tooling: If there is no "recovery_status" variable in the bootloader's
environment, the update operation has been successful. Else, if there is a "recovery_status" variable
with the value "failed", the update operation has not been successful.
While this is in general essential behavior for firmware updates, it needn't be for less critical update
operations. Hence, whether or not the update transaction marker is set by SWUpdate can be controlled by
the boolean switch "bootloader_transaction_marker" which is global per sw-description file. It defaults
to true. The following example snippet disables the update transaction marker:
software =
{
version = "0.1.0";
bootloader_transaction_marker = false;
...
It is also possible to disable setting of the transaction marker entirely (and independently of the
setting in sw-description) by starting SWUpdate with the -M option.
The same applies to setting the update state in the bootloader via its environment variable "ustate"
(default) to STATE_INSTALLED=1 or STATE_FAILED=3 after an installation. This behavior can be turned off
globally via the -m option to SWUpdate or per sw-description via the boolean switch
"bootloader_state_marker".
bootloader
There are two ways to update the bootloader (currently U-Boot, GRUB, and EFI Boot Guard) environment.
First way is to add a file with the list of variables to be changed and setting "bootloader" as type of
the image. This informs SWUpdate to call the bootloader handler to manage the file (requires enabling
bootloader handler in configuration). There is one bootloader handler for all supported bootloaders. The
appropriate bootloader must be chosen from the bootloader selection menu in menuconfig.
images: (
{
filename = "bootloader-env";
type = "bootloader";
}
)
The format of the file is described in U-boot documentation. Each line is in the format
<name of variable>=<value>
if value is missing, the variable is unset.
The format is compatible with U-Boot "env import" command. It is possible to produce the file from target
as result of "env export".
Comments are allowed in the file to improve readability, see this example:
# Default variables
bootslot=0
board_name=myboard
baudrate=115200
## Board Revision dependent
board_revision=1.0
The second way is to define in a group setting the variables that must be changed:
bootenv: (
{
name = <Variable name>;
value = <Variable value>;
}
)
SWUpdate will internally generate a script that will be passed to the bootloader handler for adjusting
the environment.
For backward compatibility with previously built .swu images, the "uboot" group name is still supported
as an alias. However, its usage is deprecated.
Board specific settings
Each setting can be placed under a custom tag matching the board name. This mechanism can be used to
override particular setting in board specific fashion.
Assuming that the hardware information file /etc/hwrevision contains the following entry:
my-board 0.1.0
and the following description:
software =
{
version = "0.1.0";
my-board = {
bootenv: (
{
name = "bootpart";
value = "0:2";
}
);
};
bootenv: (
{
name = "bootpart";
value = "0:1";
}
);
}
SWUpdate will set bootpart to 0:2 in bootloader's environment for this board. For all other boards,
bootpart will be set to 0:1. Board specific settings take precedence over default scoped settings.
Software collections and operation modes
Software collections and operations modes extend the description file syntax to provide an overlay
grouping all previous configuration tags. The mechanism is similar to Board specific settings and can be
used for implementing a dual copy strategy or delivering both stable and unstable images within a single
update file.
The mechanism uses a custom user-defined tags placed within software scope. The tag names must not be any
of: version, hardware-compatibility, uboot, bootenv, files, scripts, partitions, images
An example description file:
software =
{
version = "0.1";
hardware-compatibility = [ "revA" ];
/* differentiate running image modes/sets */
stable:
{
main:
{
images: (
{
filename = "rootfs.ext3";
device = "/dev/mmcblk0p2";
}
);
bootenv: (
{
name = "bootpart";
value = "0:2";
}
);
};
alt:
{
images: (
{
filename = "rootfs.ext3";
device = "/dev/mmcblk0p1";
}
);
bootenv: (
{
name = "bootpart";
value = "0:1";
}
);
};
};
}
The configuration describes a single software collection named stable. Two distinct image locations are
specified for this collection: /dev/mmcblk0p1 and /dev/mmcblk0p2 for main mode and alt mode respectively.
This feature can be used to implement a dual copy strategy by specifying the collection and mode
explicitly.
Versioning schemas in SWUpdate
SWUpdate can perform version comparisons for the whole Software by checking the version attribute in the
common part of sw-description and / or for single artifacts. SWUpdate supports two different version
schemas, and they must be followed if version comparison is requested.
Numbering schema (default)
SWUpdate supports a version based on the schema:
<major>.<minor>.<revision>.<build>
where each field is a plain number (no alphanumeric) in the range 0..65535. User can add further fields
using the dot separator, but they are not considered for version comparison. SWUpdate will check if a
version number is set according to this rule, and fallback to semantic version if fails. The version is
converted in a 64 bit number (each field is 16 bit) and compare with the running version of the same
artifact.
Please consider that, because additional fields are descriptive only, for the comparison they are not
considered. This example contains version numbers that are interpreted as the same version number by
SWUpdate:
1.2.3.4
1.2.3.4.5
1.2.3.4.5.6
But the following is different:
1.2.3.4-alpha
And it is treated as semantic version
Semantic version
SWUpdate supports semantic version. See official documentation for more details.
Checking version of installed software
SWUpdate can optionally verify if a sub-image is already installed and, if the version to be installed is
exactly the same, it can skip to install it. This is very useful in case some high risky image should be
installed or to speed up the upgrade process. One case is if the bootloader needs to be updated. In most
time, there is no need to upgrade the bootloader, but practice showed that there are some cases where an
upgrade is strictly required - the project manager should take the risk. However, it is nicer to have
always the bootloader image as part of the .swu file, allowing to get the whole distro for the device in
a single file, but the device should install it just when needed.
SWUpdate searches for a file (/etc/sw-versions is the default location) containing all versions of the
installed images. This must be generated before running SWUpdate. The file must contains pairs with the
name of image and his version, as:
<name of component> <version>
In sw-description, the optional attributes "name", "version" and "install-if-different" provide the
connection. Name and version are then compared with the data in the versions file. install-if-different
is a boolean that enables the check for this image. It is then possible to check the version just for a
subset of the images to be installed.
If used with "install-if-different", then version can be any string. For example:
bootloader 2015.01-rc3-00456-gd4978d
kernel 3.17.0-00215-g2e876af
There is also an attribute "install-if-higher" that checks if the version of the new software is higher
than the version of the installed software. If it's false, the new software isn't installed. The goal is
to avoid installing an older version of software.
In this case, version can be any of 2 formats. Either the version consists of up to 4 numbers in the
range 0..65535 separated by a dot, e.g. <major>.<minor>.<rev>.<build>, or it is a semantic version.
bootloader 2018.03.01
kernel 3.17.0-pre1+g2e876af
rfs 0.17-foo3.bar5+2020.07.01
app 1.7
It is advised not to mix version formats! Semantic versions only support 3 numbers (major, minor, patch)
and the fourth number will be silently dropped if present.
Embedded Script
It is possible to embed a script inside sw-description. This is useful in a lot of conditions where some
parameters are known just by the target at runtime. The script is global to all sections, but it can
contain several functions that can be specific for each entry in the sw-description file.
These attributes are used for an embedded-script:
embedded-script = "<Lua code">
It must be taken into account that the parser has already run and usage of double quotes can interfere
with the parser. For this reason, each double quote in the script must be escaped.
That means a simple Lua code as:
print ("Test")
must be changed to:
print (\"Test\")
If not, the parser thinks to have the closure of the script and this generates an error. See the
examples directory for examples how to use it. Any entry in files or images can trigger one function in
the script. The "hook" attribute tells the parser to load the script and to search for the function
pointed to by the hook attribute. For example:
files: (
{
filename = "examples.tar";
type = "archive";
path = "/tmp/test";
hook = "set_version";
preserve-attributes = true;
}
);
After the entry is parsed, the parser runs the Lua function pointed to by hook. If Lua is not activated,
the parser raises an error because a sw-description with an embedded script must be parsed, but the
interpreter is not available.
Each Lua function receives as parameter a table with the setup for the current entry. A hook in Lua is in
the format:
function lua_hook(image)
image is a table where the keys are the list of available attributes. If an attribute contains a "-", it
is replaced with "_", because "-" cannot be used in Lua. This means, for example, that:
install-if-different ==> install_if_different
install-directly ==> install_directly
Attributes can be changed in the Lua script and values are taken over on return. The Lua function must
return 2 values:
• a boolean, to indicate whether the parsing was correct
• the image table or nil to indicate that the image should be skipped
Example:
function set_version(image)
print (\"RECOVERY_STATUS.RUN: \".. swupdate.RECOVERY_STATUS.RUN)
for k,l in pairs(image) do
swupdate.trace(\"image[\" .. tostring(k) .. \"] = \" .. tostring(l))
end
image.version = \"1.0\"
image.install_if_different = true
return true, image
end
The example sets a version for the installed image. Generally, this is detected at runtime reading from
the target.
Attribute reference
There are 4 main sections inside sw-description:
• images: entries are images and SWUpdate has no knowledge about them.
• files: entries are files, and SWUpdate needs a filesystem for them. This is generally used to expand
from a tar-ball or to update single files.
• scripts: all entries are treated as executables, and they will be run twice (as pre- and post- install
scripts).
• bootenv: entries are pair with bootloader environment variable name and its value.
Attributes in sw-description
─────────────────────────────────────────────────────────────────────────────────────────────
Name Type Applies to Description
─────────────────────────────────────────────────────────────────────────────────────────────
filename string images files scripts filename as found in
the cpio archive
─────────────────────────────────────────────────────────────────────────────────────────────
volume string images Just if type =
"ubivol". UBI volume
where image must be
installed.
─────────────────────────────────────────────────────────────────────────────────────────────
ubipartition string images Just if type =
"ubivol". Volume to be
created or adjusted
with a new size
─────────────────────────────────────────────────────────────────────────────────────────────
device string images files devicenode as found in
/dev or a symlink to
it. Can be specified
as absolute path or a
name in /dev folder
For example if
/dev/mtd-dtb is a link
to /dev/mtd3 "mtd3",
"mtd-dtb", "/dev/mtd3"
and "/dev/mtd-dtb" are
valid names. Usage
depends on handler.
For files, it
indicates on which
device the
"filesystem" must be
mounted. If not
specified, the current
rootfs will be used.
─────────────────────────────────────────────────────────────────────────────────────────────
filesystem string files indicates the
filesystem type where
the file must be
installed. Only used
if "device" attribute
is set.
─────────────────────────────────────────────────────────────────────────────────────────────
path string files For files: indicates
the path (absolute)
where the file must be
installed. If "device"
and "filesystem" are
set, SWUpdate will
install the file after
mounting "device" with
"filesystem" type.
(path is always
relative to the mount
point.)
─────────────────────────────────────────────────────────────────────────────────────────────
preserve-attributes bool files flag to control
whether the following
attributes will be
preserved when files
are unpacked from an
archive (assuming
destination filesystem
supports them, of
course): timestamp,
uid/gid (numeric),
perms, file
attributes, extended
attributes
─────────────────────────────────────────────────────────────────────────────────────────────
type string images files scripts string identifier for
the handler, as it is
set by the handler
when it regitsters
itself. Example:
"ubivol", "raw",
"rawfile",
─────────────────────────────────────────────────────────────────────────────────────────────
compressed string images files string to indicate the
"filename" is
compressed and must be
decompressed before
being installed. the
value denotes the
compression type.
currently supported
values are "zlib" and
"zstd".
─────────────────────────────────────────────────────────────────────────────────────────────
compressed bool (dep recated) images files Deprecated. Use the
string form. true is
equal to 'compression
= "zlib"'.
─────────────────────────────────────────────────────────────────────────────────────────────
installed-directly bool images flag to indicate that
image is streamed into
the target without any
temporary copy. Not
all handlers support
streaming.
─────────────────────────────────────────────────────────────────────────────────────────────
name string bootenv name of the bootloader
variable to be set.
─────────────────────────────────────────────────────────────────────────────────────────────
value string bootenv value to be assigned
to the bootloader
variable
─────────────────────────────────────────────────────────────────────────────────────────────
name string images files name that identifies
the sw-component it
can be any string and
it is compared with
the entries in
sw-versions
─────────────────────────────────────────────────────────────────────────────────────────────
version string images files version for the
sw-component it can be
any string and it is
compared with the
entries in sw-versions
─────────────────────────────────────────────────────────────────────────────────────────────
description string user-friendly
description of the
swupdate archive (any
string)
─────────────────────────────────────────────────────────────────────────────────────────────
install-if-different bool images files flag if set, name and
version are compared
with the entries in
─────────────────────────────────────────────────────────────────────────────────────────────
install-if-higher bool images files flag if set, name and
version are compared
with the entries in
─────────────────────────────────────────────────────────────────────────────────────────────
encrypted bool images files scripts flag if set, file is
encrypted and must be
decrypted before
installing.
─────────────────────────────────────────────────────────────────────────────────────────────
ivt string images files scripts IVT in case of
encrypted artefact It
has no value if
"encrypted" is not
set. Each artefact can
have an own IVT to
avoid attacker can
guess the the key. It
is a ASCII string of
32 chars
─────────────────────────────────────────────────────────────────────────────────────────────
data string images files scripts This is used to pass
arbitrary data to a
handler.
─────────────────────────────────────────────────────────────────────────────────────────────
sha256 string images files scripts sha256 hash of image,
file or script. Used
for verification of
signed images.
─────────────────────────────────────────────────────────────────────────────────────────────
embedded-script string Lua code that is
embedded in the
sw-description file.
─────────────────────────────────────────────────────────────────────────────────────────────
offset string images Optional destination
offset
─────────────────────────────────────────────────────────────────────────────────────────────
hook string images files The name of the
function (Lua) to be
called when the entry
is parsed.
─────────────────────────────────────────────────────────────────────────────────────────────
mtdname string images name of the MTD to
update. Used only by
the flash handler to
identify the the mtd
to update, instead of
specifying the
devicenode
┌──────────────────────┬────────────────────┬──────────────────────┬────────────────────────┐
│ │ │ │ │
--
SYMMETRICALLY ENCRYPTED UPDATE IMAGES
SWUpdate allows one to symmetrically encrypt update images using the AES block cipher in CBC mode. The
following shows encryption with 256 bit key length but you may use other key lengths as well.
Building an Encrypted SWU Image
First, create a key; for aes-256-cbc we need 32 bytes of key and 16 bytes for an initialisation vector
(IV). A complete documentation can be found at the OpenSSL Website.
openssl rand -hex 32
# key, for example 390ad54490a4a5f53722291023c19e08ffb5c4677a59e958c96ffa6e641df040
openssl rand -hex 16
# IV, for example d5d601bacfe13100b149177318ebc7a4
Then, encrypt an image using this information via
openssl enc -aes-256-cbc -in <INFILE> -out <OUTFILE> -K <KEY> -iv <IV>
where <INFILE> is the unencrypted source image file and <OUTFILE> is the encrypted output image file to
be referenced in sw-description. <KEY> is the hex value part of the 2nd line of output from the key
generation command above and <IV> is the hex value part of the 3rd line.
Then, create a key file to be supplied to SWUpdate via the -K switch by putting the key and
initialization vector hex values on one line separated by whitespace, e.g., for above example values
390ad54490a4a5f53722291023c19e08ffb5c4677a59e958c96ffa6e641df040 d5d601bacfe13100b149177318ebc7a4
Previous versions of SWUpdate allowed for a salt as third word in key file, that was never actually used
for aes and has been removed.
You should change the IV with every encryption, see CWE-329. The ivt sw-description attribute overrides
the key file's IV for one specific image.
Encryption of UBI volumes
Due to a limit in the Linux kernel API for UBI volumes, the size reserved to be written on disk should be
declared before actually writing anything.
See the property "decrypted-size" in UBI Volume Handler's documentation.
Example sw-description with Encrypted Image
The following example is a (minimal) sw-description for installing a Yocto image onto a Beaglebone. Pay
attention to the encrypted = true; setting.
software =
{
version = "0.0.1";
images: ( {
filename = "core-image-full-cmdline-beaglebone.ext3.enc";
device = "/dev/mmcblk0p3";
encrypted = true;
ivt = "65D793B87B6724BB27954C7664F15FF3";
}
);
}
Running SWUpdate with Encrypted Images
Symmetric encryption support is activated by setting the ENCRYPTED_IMAGES option in SWUpdate's
configuration. Use the -K parameter to provide the symmetric key file generated above to SWUpdate.
Decrypting with a PKCS#11 token
PKCS#11 support is activated by setting the PKCS11 option in SWUpdate's configuration. The key file has
to have a PKCS#11 URL instead of the key then, containing at least the elements of this example:
pkcs11:slot-id=42;id=%CA%FE%BA%BE?pin-value=1234&module-path=/usr/lib/libsofthsm2.so 65D793B87B6724BB27954C7664F15FF3
HANDLERS
Overview
It is quite difficult to foresee all possible installation cases. Instead of trying to find all use
cases, SWUpdate let the developer free to add his own installer (that is, a new handler), that must be
responsible to install an image of a certain type. An image is marked to be of a defined type to be
installed with a specific handler.
The parser make the connection between 'image type' and 'handler'. It fills a table containing the list
of images to be installed with the required handler to execute the installation. Each image can have a
different installer.
Supplied handlers
In mainline there are the handlers for the most common cases. They include:
• flash devices in raw mode (both NOR and NAND)
• UBI volumes
• UBI volumes partitioner
• raw flashes handler (NAND, NOR, SPI-NOR, CFI interface)
• disk partitioner
• raw devices, such as a SD Card partition
• bootloader (U-Boot, GRUB, EFI Boot Guard) environment
• Lua scripts handler
• shell scripts handler
• rdiff handler
• readback handler
• archive (zo, tarballs) handler
• remote handler
• microcontroller update handler
For example, if an image is marked to be updated into a UBI volume, the parser must fill a supplied table
setting "ubi" as required handler, and filling the other fields required for this handler: name of
volume, size, and so on.
Creating own handlers
SWUpdate can be extended with new handlers. The user needs to register his own handler with the core and
he must provide the callback that SWUpdate uses when an image required to be installed with the new
handler.
The prototype for the callback is:
int my_handler(struct img_type *img,
void __attribute__ ((__unused__)) *data)
The most important parameter is the pointer to a struct img_type. It describes a single image and inform
the handler where the image must be installed. The file descriptor of the incoming stream set to the
start of the image to be installed is also part of the structure.
The structure img_type contains the file descriptor of the stream pointing to the first byte of the image
to be installed. The handler must read the whole image, and when it returns back SWUpdate can go on with
the next image in the stream.
SWUpdate provides a general function to extract data from the stream and copy to somewhere else:
int copyfile(int fdin, int fdout, int nbytes, unsigned long *offs,
int skip_file, int compressed, uint32_t *checksum, unsigned char *hash);
fdin is the input stream, that is img->fdin from the callback. The hash, in case of signed images, is
simply passed to copyfile() to perform the check, exactly as the checksum parameter. copyfile() will
return an error if checksum or hash do not match. The handler does not need to bother with them. How the
handler manages the copied data, is specific to the handler itself. See supplied handlers code for a
better understanding.
The handler's developer registers his own handler with a call to:
__attribute__((constructor))
void my_handler_init(void)
{
register_handler("mytype", my_handler, my_mask, data);
}
SWUpdate uses the gcc constructors, and all supplied handlers are registered when SWUpdate is
initialized.
register_handler has the syntax:
register_handler(my_image_type, my_handler, my_mask, data);
Where:
• my_image_type : string identifying the own new image type.
• my_handler : pointer to the installer to be registered.
• my_mask : HANDLER_MASK enum value(s) specifying what input type(s) my_handler can process.
• data : an optional pointer to an own structure, that SWUpdate saves in the handlers' list and pass to
the handler when it will be executed.
UBI Volume Handler
The UBI volume handler will update UBI volumes without changing the layout on the storage. Therefore,
volumes must be created/adjusted beforehand. This can be done using the partitions tag (see
partitions-ubi-layout).
The UBI volume handler will search for volumes in all MTD devices (unless blacklisted, see UBIBLACKLIST)
to find the volume into which the image shall be installed. For this reason, volume names must be unique
within the system. Two volumes with the same name are not supported and will lead to unpredictable
results (SWUpdate will install the image to the first volume with that name it finds, which may not be
right one!).
When updating volumes, it is guaranteed that erase counters are preserved and not lost. The behavior of
updating is identical to that of the ubiupdatevol(1) tool from mtd-utils. In fact, the same library from
mtd-utils (libubi) is reused by SWUpdate.
atomic volume renaming
The UBI volume handler has basic support for carrying out atomic volume renames by defining the replaces
property, which must contain a valid UBI volume name. After successfully updating the image to volume, an
atomic swap of the names of volume and replaces is done. Consider the following example
{
filename ="u-boot.img";
volume ="u-boot_r";
properties: {
replaces = "u-boot";
}
}
After u-boot.img is successfully installed into the volume "u-boot_r", the volume "u-boot_r" is renamed
to "u-boot" and "u-boot" is renamed to "u-boot_r".
This mechanism allows one to implement a simple double copy update approach without the need of shared
state with the bootloader. For example, the U-Boot SPL can be configured to always load U-Boot from the
volume u-boot without the need to access the environment. The volume replace functionality will ensure
that this volume name always points to the currently valid volume.
However, please note the following limitations:
• Currently the rename takes place after each image was installed successfully. Hence, it only makes
sense to use this feature for images that are independent of the other installed images. A typical
example is the bootloader. This behavior may be modified in the future to only carry out one atomic
rename after all images were installed successfully.
• Atomic renames are only possible and permitted for volumes residing on the same UBI device.
There is a handler ubiswap that allow one to do an atomic swap for several ubi volume after all the
images were flashed. This handler is a script for the point of view of swudate, so the node that provide
it the data should be added in the section scripts.
scripts: (
{
type = "ubiswap";
properties: {
swap-0 = [ "boot" , " boot_r" ];
swap-1 = [ "kernel" , "kernel_r" ];
swap-2 = [ "rootfs" , "rootfs_r" ];
},
},
);
WARNING: if you use the property replaces on an ubi volume that is also used with the handler ubiswap,
this ubi volume will be swapped twice. It's probably not what you want ...
volume auto resize
The UBI volume handler has support to auto resize before flashing an image with the property auto-resize.
When this property is set on an image, the ubi volume is resized to fit exactly the image.
{
filename = "u-boot.img";
device = "mtd0";
volume = "u-boot_r";
properties: {
auto-resize = "true";
}
}
WARNING: when this property is used, the device must be defined.
volume always remove
The UBI volume handler has support to always remove ubi volume before flashing with the property
always-remove. When this property is set on an image, the ubi volume is always removed. This property
should be used with property auto-resize.
{
filename = "u-boot.img";
device = "mtd0";
volume = "u-boot_r";
properties: {
always-remove = "true";
auto-resize = "true";
}
}
size properties
Due to a limit in the Linux kernel API for UBI volumes, the size reserved to be written on disk should be
declared before actually writing anything. Unfortunately, the size of an encrypted or compressed image
is not known until the decryption or decompression finished. This prevents correct declaration of the
file size to be written on disk.
For this reason UBI images can declare the special properties "decrypted-size" or "decompressed-size"
like this:
images: ( {
filename = "rootfs.ubifs.enc";
volume = "rootfs";
encrypted = true;
properties: {
decrypted-size = "104857600";
}
},
{
filename = "homefs.ubifs.gz";
volume = "homefs";
compressed = "zlib";
properties: {
decompressed-size = "420000000";
}
}
);
The real size of the image should be calculated and written to the sw-description before assembling the
cpio archive. In this example, 104857600 is the size of the rootfs after the decryption: the encrypted
size is by the way larger. The decompressed size is of the homefs is 420000000.
The sizes are bytes in decimal notation.
Lua Handlers
In addition to the handlers written in C, it is possible to extend SWUpdate with handlers written in Lua
that get loaded at SWUpdate startup. The Lua handler source code file may either be embedded into the
SWUpdate binary via the CONFIG_EMBEDDED_LUA_HANDLER config option or has to be installed on the target
system in Lua's search path as swupdate_handlers.lua so that it can be loaded by the embedded Lua
interpreter at run-time.
In analogy to C handlers, the prototype for a Lua handler is
function lua_handler(image)
...
end
where image is a Lua table (with attributes according to sw-description's attribute reference) that
describes a single artifact to be processed by the handler.
Note that dashes in the attributes' names are replaced with underscores for the Lua domain to make them
idiomatic, e.g., installed-directly becomes installed_directly in the Lua domain.
To register a Lua handler, the swupdate module provides the swupdate.register_handler() method that takes
the handler's name, the Lua handler function to be registered under that name, and, optionally, the types
of artifacts for which the handler may be called. If the latter is not given, the Lua handler is
registered for all types of artifacts. The following call registers the above function lua_handler as
my_handler which may be called for images:
swupdate.register_handler("my_handler", lua_handler, swupdate.HANDLER_MASK.IMAGE_HANDLER)
A Lua handler may call C handlers ("chaining") via the swupdate.call_handler() method. The callable and
registered C handlers are available (as keys) in the table swupdate.handler. The following Lua code is an
example of a simple handler chain-calling the rawfile C handler:
function lua_handler(image)
if not swupdate.handler["rawfile"] then
swupdate.error("rawfile handler not available")
return 1
end
image.path = "/tmp/destination.path"
local err, msg = swupdate.call_handler("rawfile", image)
if err ~= 0 then
swupdate.error(string.format("Error chaining handlers: %s", msg))
return 1
end
return 0
end
Note that when chaining handlers and calling a C handler for a different type of artifact than the Lua
handler is registered for, the image table's values must satisfy the called C handler's expectations:
Consider the above Lua handler being registered for "images" (swupdate.HANDLER_MASK.IMAGE_HANDLER) via
the swupdate.register_handler() call shown above. As per the sw-description's attribute reference, the
"images" artifact type doesn't have the path attribute but the "file" artifact type does. So, for calling
the rawfile handler, image.path has to be set prior to chain-calling the rawfile handler, as done in the
example above. Usually, however, no such adaptation is necessary if the Lua handler is registered for
handling the type of artifact that image represents.
In addition to calling C handlers, the image table passed as parameter to a Lua handler has a
image:copy2file() method that implements the common use case of writing the input stream's data to a
file, which is passed as this method's argument. On success, image:copy2file() returns 0 or -1 plus an
error message on failure. The following Lua code is an example of a simple handler calling
image:copy2file():
function lua_handler(image)
local err, msg = image:copy2file("/tmp/destination.path")
if err ~= 0 then
swupdate.error(string.format("Error calling copy2file: %s", msg))
return 1
end
return 0
end
Beyond using image:copy2file() or chain-calling C handlers, the image table passed as parameter to a Lua
handler has a image:read(<callback()>) method that reads from the input stream and calls the Lua callback
function <callback()> for every chunk read, passing this chunk as parameter. On success, 0 is returned by
image:read(). On error, -1 plus an error message is returned. The following Lua code is an example of a
simple handler printing the artifact's content:
function lua_handler(image)
err, msg = image:read(function(data) print(data) end)
if err ~= 0 then
swupdate.error(string.format("Error reading image: %s", msg))
return 1
end
return 0
end
Using the image:read() method, an artifact's contents may be (post-)processed in and leveraging the power
of Lua without relying on preexisting C handlers for the purpose intended.
Just as C handlers, a Lua handler must consume the artifact described in its image parameter so that
SWUpdate can continue with the next artifact in the stream after the Lua handler returns. Chaining
handlers, calling image:copy2file(), or using image:read() satisfies this requirement.
Note that although the dynamic nature of Lua handlers would technically allow one to embed them into a to
be processed .swu image, this is not implemented as it carries some security implications since the
behavior of SWUpdate is changed dynamically.
Remote handler
Remote handlers are thought for binding legacy installers without having the necessity to rewrite them in
Lua. The remote handler forward the image to be installed to another process, waiting for an acknowledge
to be sure that the image is installed correctly. The remote handler makes use of the zeromq library -
this is to simplify the IPC with Unix Domain Socket. The remote handler is quite general, describing in
sw-description with the "data" attribute how to communicate with the external process. The remote
handler always acts as client, and try a connect() using the socket identified by the "data" attribute.
For example, a possible setup using a remote handler could be:
images: (
{
filename = "myimage"";
type = "remote";
data = "test_remote";
}
)
The connection is instantiated using the socket test_remote (according to the "data" field's value) in
the directory pointed to by the environment variable TMPDIR with /tmp as fall-back if TMPDIR is not set.
If connect() fails, the remote handler signals that the update is not successful. Each zeromq message
from SWUpdate is a multi-part message split into two frames:
• first frame contains a string with a command.
• second frame contains data and can be of 0 bytes.
There are currently just two possible commands: INIT and DATA. After a successful connect, SWUpdate sends
the initialization string in the format:
INIT:<size of image to be installed>
The external installer is informed about the size of the image to be installed, and it can assign
resources if it needs. It will answer with the string ACK or NACK. The first NACK received by SWUpdate
will interrupt the update. After sending the INIT command, the remote handler will send a sequence of
DATA commands, where the second frame in message will contain chunks of the image to be installed. It is
duty of the external process to take care of the amount of data transferred and to release resources when
the last chunk is received. For each DATA message, the external process answers with a ACK or NACK
message.
SWU forwarder
The SWU forwarder handler can be used to update other systems where SWUpdate is running. It can be used
in case of master / slaves systems, where the master is connected to the network and the "slaves" are
hidden to the external world. The master is then the only interface to the world. A general SWU can
contain embedded SWU images as single artifacts, and the SWU handler will forward it to the devices
listed in the description of the artifact. The handler can have a single "url" properties entry with an
array of urls. Each url is the address of a secondary board where SWUpdate is running with webserver
activated. The SWU handler expects to talk with SWUpdate's embedded webserver. This helps to update
systems where an old version of SWUpdate is running, because the embedded webserver is a common feature
present in all versions. The handler will send the embedded SWU to all URLs at the same time, and
setting installed-directly is supported by this handler. [image]
The following example shows how to set a SWU as artifact and enables the SWU forwarder:
images: (
{
filename = "image.swu";
type = "swuforward";
properties: {
url = ["http://192.168.178.41:8080", "http://192.168.178.42:8080"];
};
});
rdiff handler
The rdiff handler adds support for applying binary delta patches generated by librsync's rdiff tool.
Naturally, the smaller the difference between the diff's source and target, the more effective is using
this handler rather than shipping the full target, e.g., via the image handler. Hence, the most prominent
use case for the rdiff handler is when having a read-only root filesystem and applying a small update
like security fixes or feature additions. If the sweet spot is crossed, an rdiff patch may even exceed
the full target's size due to necessary patch metadata. Also note that in order to be most effective, an
image to be processed with rdiff should be built deterministic (see reproducible-builds.org).
The rdiff algorithm requires no resources whatsoever on the device as the patch is fully computed in the
backend. Consequently, the backend has to have knowledge of the current software running on the device in
order to compute a sensible patch. Alike, the patch has to be applied on the device to an unmodified
source as used in the backend for patch computation. This property is in particular useful for
resource-constrained devices as there's no need for the device to, e.g., aid in the difference
computation.
First, create the signature of the original (base) file via rdiff signature <basefile> <signaturefile>.
Then, create the delta file (i.e., patch) from the original base file to the target file via rdiff delta
<signaturefile> <targetfile> <deltafile>. The <deltafile> is the artifact to be applied via this handler
on the device. Essentially, it mimics running rdiff patch <basefile> <deltafile> <targetfile> on the
device. Naturally for patches, the very same <basefile> has to be used for creating as well as for
applying the patch to.
This handler registers itself for handling files and images. An exemplary sw-description fragment for
the files section is
files: (
{
type = "rdiff_file"
filename = "file.rdiff.delta";
path = "/usr/bin/file";
}
);
Note that the file referenced to by path serves as <basefile> and gets replaced by a temporary file
serving as <targetfile> while the rdiff patch processing.
An exemplary sw-description fragment for the images section is
images: (
{
type = "rdiff_image";
filename = "image.rdiff.delta";
device = "/dev/mmcblk0p2";
properties: {
rdiffbase = ["/dev/mmcblk0p1"];
};
}
);
Here, the property rdiffbase qualifies the <basefile> while the device attribute designates the
<targetfile>. Note that there's no support for the optional offset attribute in the rdiff_image handler
as there's currently no apparent use case for it and skipping over unchanged content is handled well by
the rdiff algorithm.
ucfw handler
This handler allows one to update the firmware on a microcontroller connected to the main controller via
UART. Parameters for setup are passed via sw-description file. Its behavior can be extended to be more
general. The protocol is ASCII based. There is a sequence to be done to put the microcontroller in
programming mode, after that the handler sends the data and waits for an ACK from the microcontroller.
The programming of the firmware shall be:
1. Enter firmware update mode (bootloader)
1. Set "reset line" to logical "low"
2. Set "update line" to logical "low"
3. Set "reset line" to logical "high"
2. Send programming message
$PROG;<<CS>><CR><LF>
to the microcontroller. (microcontroller will remain in programming state)
3. microcontroller confirms with
$READY;<<CS>><CR><LF>
4. Data transmissions package based from mainboard to microcontroller package definition:
• within a package the records are sent one after another without the end of line marker <CR><LF>
• the package is completed with <CR><LF>
5. The microcontroller requests the next package with $READY;<<CS>><CR><LF>
6. Repeat step 4 and 5 until the complete firmware is transmitted.
7. The keypad confirms the firmware completion with $COMPLETED;<<CS>><CR><LF>
8.
Leave firmware update mode
1. Set "Update line" to logical "high"
2. Perform a reset over the "reset line"
<<CS>> : checksum. The checksum is calculated as the two's complement of the modulo-256 sum over all
bytes of the message string except for the start marker "$". The handler expects to get in the
properties the setup for the reset and prog gpios. They should be in this format:
properties = {
reset = "<gpiodevice>:<gpionumber>:<activelow>";
prog = "<gpiodevice>:<gpionumber>:<activelow>";
}
Example:
images: (
{
filename = "microcontroller-image";
type = "ucfw";
device = "/dev/ttymxc5";
properties: {
reset = "/dev/gpiochip0:38:false";
prog = "/dev/gpiochip0:39:false";
};
}
);
SSBL Handler
This implements a way to switch two software sets using a duplicated structure saved on the flash
(currently, only NOR flash is supported). Each of the two structures contains address and size of the
image to be loaded by a first loader. A field contain the "age", and it is incremented after each switch
to show which is the active set.
Structure of SSBL Admin
┌────────────────────────────────┬─────────────┐
│ SSBL Magic Number (29 bit)Name │ Age (3 bit) │
├────────────────────────────────┼─────────────┤
│ Image Address Offset │ │
├────────────────────────────────┼─────────────┤
│ Image Size │ │
└────────────────────────────────┴─────────────┘
The handler implements a post install script. First, it checks for consistency the two structures and
find the active reading the first 32 bit value with a magic number and the age. It increments the age
and saves the new structure in the inactive copy. After a reboot, the loader will check it and switch the
software set.
scripts: (
{
type = "ssblswitch";
properties: {
device = ["mtdX", "mtdY"];
offset = ["0", "0"];
imageoffs = ["0x780000", "0xA40000"];
imagesize = ["0x800000", "0x800000"];
}
}
Properties in sw-description are all mandatory. They define where the SSBL Administration data are stored
for both sets. Each properties is an array of two entries, containing values for each of the two SSBL
administration.
Properties for SSBL handler
┌─────────────┬────────┬──────────────────────────────┐
│ Name │ Type │ Description │
├─────────────┼────────┼──────────────────────────────┤
│ device │ string │ MTD device where the SSBL │
│ │ │ Admin Header is stored │
├─────────────┼────────┼──────────────────────────────┤
│ offset │ hex │ Offset of SSBL header inside │
│ │ │ the MTD device │
├─────────────┼────────┼──────────────────────────────┤
│ imageoffset │ hex │ Offset of the image to be │
│ │ │ loaded by a bootloader when │
│ │ │ this SSBL is set. │
├─────────────┼────────┼──────────────────────────────┤
│ imagesize │ hex │ Size of the image to be │
│ │ │ loaded by a bootloader when │
│ │ │ this SSBL is set. │
└─────────────┴────────┴──────────────────────────────┘
Readback Handler
To verify that an image was written properly, this readback handler calculates the sha256 hash of a
partition (or part of it) and compares it against a given hash value.
The following example explains how to use this handler:
scripts: (
{
device = "/dev/mmcblk2p1";
type = "readback";
properties: {
sha256 = "e7afc9bd98afd4eb7d8325196d21f1ecc0c8864d6342bfc6b6b6c84eac86eb42";
size = "184728576";
offset = "0";
};
}
);
Properties size and offset are optional, all the other properties are mandatory.
Properties for readback handler
┌────────┬────────┬──────────────────────────────┐
│ Name │ Type │ Description │
├────────┼────────┼──────────────────────────────┤
│ device │ string │ The partition which shall be │
│ │ │ verified. │
├────────┼────────┼──────────────────────────────┤
│ type │ string │ Identifier for the handler. │
├────────┼────────┼──────────────────────────────┤
│ sha256 │ string │ Expected sha256 hash of the │
│ │ │ partition. │
├────────┼────────┼──────────────────────────────┤
│ size │ string │ Data size (in bytes) to be │
│ │ │ verified. If 0 or not set, │
│ │ │ the handler will get the │
│ │ │ partition size from the │
│ │ │ device. │
├────────┼────────┼──────────────────────────────┤
│ offset │ string │ Offset (in bytes) to the │
│ │ │ start of the partition. If │
│ │ │ not set, default value 0 │
│ │ │ will be used. │
└────────┴────────┴──────────────────────────────┘
Rawcopy handler
The rawcopy handler copies one source to a destination. It can be used to copy configuration data, or
parts that should be taken by the current installation. It requires just one property (copyfrom), while
device contains the destination path. The handler performs a byte copy, and it does not matter which is
the source - it can be a file or a partition.
scripts : (
{
device = "/dev/mmcblk2p1";
type = "rawcopy";
properties : {
copyfrom = "/dev/mmcblk2p2";
}
}
Disk partitioner
This handler creates or modifies partitions using the library libfdisk. Handler must be put into the
partitions section of sw-description. Setup for each partition is put into the properties field of
sw-description. After writing the partition table it may create a file system on selected partitions.
(Available only if CONFIG_DISKFORMAT is set.)
Properties for diskpart handler
┌─────────────┬────────┬──────────────────────────────┐
│ Name │ Type │ Description │
├─────────────┼────────┼──────────────────────────────┤
│ labeltype │ string │ "gpt" or "dos" │
├─────────────┼────────┼──────────────────────────────┤
│ nolock │ string │ "true" or "false" │
│ │ │ (default=false) This is like │
│ │ │ a force. If it is set, a │
│ │ │ lock failure will be │
│ │ │ ignored(lock will still be │
│ │ │ attempted). │
├─────────────┼────────┼──────────────────────────────┤
│ noinuse │ string │ "true" or "false" │
│ │ │ (default=false) If set, it │
│ │ │ does not require the device │
│ │ │ to be not in use (mounted, │
│ │ │ etc.) │
├─────────────┼────────┼──────────────────────────────┤
│ partition-X │ array │ Array of values belonging to │
│ │ │ the partition number X │
└─────────────┴────────┴──────────────────────────────┘
For each partition, an array of couples key=value must be given. The following keys are supported:
Setup for a disk partition
─────────────────────────────────────────────────────
Name Type Description
─────────────────────────────────────────────────────
size string Size of partition. K, M and
G can be used for Kilobytes,
Megabytes and Gigabytes.
─────────────────────────────────────────────────────
start integer First sector for the
partition
─────────────────────────────────────────────────────
name string Name of the partition
─────────────────────────────────────────────────────
type string Type of partition, it has
two different meanings. It
is the hex code for DOS
(MBR) partition table or it
is the string identifier in
case of GPT.
─────────────────────────────────────────────────────
dostype string Type of DOS (MBR) partition
entry when using a table
with a "gpt" labeltype.
Using this option will
create a hybrid MBR table.
It is the hex code for DOS
(MBR) partition table. This
would typically be used when
one wants to use a GPT
formatted disk with a board
that requires a dos table
entry for initial
bootstrapping. Note: A
maximum of 3 partitions can
have a dostype specified,
this limit only applies to
dos table entries and does
not affect partitions
without a dostype specified.
─────────────────────────────────────────────────────
fstype string Optional filesystem type to
be created on the partition.
If no fstype key is given,
no file will be created on
the corresponding partition.
vfat / ext2 / ext3 /ext4
file system is supported
─────────────────────────────────────────────────────
partuuid string The partition UUID (GPT
only). If omitted, a UUID
will be generated
automatically.
┌──────────┬─────────┬──────────────────────────────┐
│ │ │ │
GPT example: │ │ │ │
│ │ │ │
partitions: ( │ │ │ │
{ │ │ │ │
type = "diskpart"; │ │ │ │
--
MONGOOSE DAEMON MODE
Introduction
Mongoose is a daemon mode of SWUpdate that provides a web server, web interface and web application.
The web application in web-app uses the Node.js package manager and gulp as build tool. It depends on
Bootstrap 4, Font Awesome 5 and Dropzone.js.
Startup
After having configured and compiled SWUpdate with enabled mongoose web server:
./swupdate --help
lists the mandatory and optional arguments to be provided to mongoose. As an example,
./swupdate -l 5 -w '-r ./examples/www/v2 -p 8080' -p 'reboot'
runs SWUpdate in mongoose daemon mode with log-level TRACE and a web server at http://localhost:8080.
Example
The ready to use example of the web application in the examples/www/v2 directory uses a Public Domain
background.jpg image from pixabay with is released under the Creative Commons CC0 license. The used
favicon.png and logo.png images are made from the SWUpdate logo and therefore subject to the GNU General
Public License version 2. You must comply to this license or replace the images with your own files.
Customize
You could customize the web application inside the web-app directory. Beside the replace of the
favicon.png, logo.png and background.jpg images inside the images directory you could customize the
Bootstrap colors and settings inside the scss/bootstrap.scss style sheet. The style sheet changes need a
rebuild of the web application source code.
Develop
The development requires Node.js version 6 or greater and a prebuilt SWUpdate project with enabled
mongoose web server and web application interface version 2 support.
1. Enter the web application directory:
cd ./web-app
2. Install the dependencies:
npm install
3. Build the web application:
npm run build
4. Start the web application:
../swupdate -w '-r ./dist -p 8080' -p 'echo reboot'
5. Test the web application:
http://localhost:8080/
6. Pack the web application (optional):
npm run package -- --output swupdate-www.tar.gz
Contribute
Please run the linter before any commit
npm run lint
SURICATTA DAEMON MODE
Introduction
Suricatta is -- like mongoose -- a daemon mode of SWUpdate, hence the name suricatta (engl. meerkat) as
it belongs to the mongoose family.
Suricatta regularly polls a remote server for updates, downloads, and installs them. Thereafter, it
reboots the system and reports the update status to the server, based on an update state variable
currently stored in bootloader's environment ensuring persistent storage across reboots. Some U-Boot
script logics or U-Boot's bootcount feature may be utilized to alter this update state variable, e.g., by
setting it to reflect failure in case booting the newly flashed root file system has failed and a
switchback had to be performed.
Suricatta is designed to be extensible in terms of the servers supported as described in Section
Supporting different Servers. Currently, support for the hawkBit server is implemented via the hawkBit
Direct Device Integration API alongside a simple general purpose HTTP server.
Running suricatta
After having configured and compiled SWUpdate with enabled suricatta support,
./swupdate --help
lists the mandatory and optional arguments to be provided to suricatta when using hawkBit as server. As
an example,
./swupdate -l 5 -u '-t default -u http://10.0.0.2:8080 -i 25'
runs SWUpdate in suricatta daemon mode with log-level TRACE, polling a hawkBit instance at
http://10.0.0.2:8080 with tenant default and device ID 25.
Note that on startup when having installed an update, suricatta tries to report the update status to its
upstream server, e.g., hawkBit, prior to entering the main loop awaiting further updates. If this
initial report fails, e.g., because of a not (yet) configured network or a currently unavailable hawkBit
server, SWUpdate may exit with an according error code. This behavior allows one to, for example, try
several upstream servers sequentially. If suricatta should keep retrying until the update status is
reported to its upstream server irrespective of the error conditions, this has to be realized externally
in terms of restarting SWUpdate on exit.
After an update has been performed, an agent listening on the progress interface may execute post-update
actions, e.g., a reboot, on receiving DONE. Additionally, a post-update command specified in the
configuration file or given by the -p command line option can be executed.
Note that at least a restart of SWUpdate has to be performed as post-update action since only then
suricatta tries to report the update status to its upstream server. Otherwise, succinct update actions
announced by the upstream server are skipped with an according message until a restart of SWUpdate has
happened in order to not install the same update again.
Supporting different Servers
Support for servers other than hawkBit can be realized by implementing the "interfaces" described in
include/channel.h and include/suricatta/server.h. The former abstracts a particular connection to the
server, e.g., HTTP-based in case of hawkBit, while the latter implements the logics to poll and install
updates. See corelib/channel_curl.c/include/channel_curl.h and suricatta/server_hawkbit.{c,h} for an
example implementation targeted towards hawkBit.
include/channel.h describes the functionality a channel has to implement:
typedef struct channel channel_t;
struct channel {
...
};
channel_t *channel_new(void);
which sets up and returns a channel_t struct with pointers to functions for opening, closing, fetching,
and sending data over the channel.
include/suricatta/server.h describes the functionality a server has to implement:
server_op_res_t server_has_pending_action(int *action_id);
server_op_res_t server_install_update(void);
server_op_res_t server_send_target_data(void);
unsigned int server_get_polling_interval(void);
server_op_res_t server_start(const char *cfgfname, int argc, char *argv[]);
server_op_res_t server_stop(void);
server_op_res_t server_ipc(int fd);
The type server_op_res_t is defined in include/suricatta/suricatta.h. It represents the valid function
return codes for a server's implementation.
In addition to implementing the particular channel and server, the suricatta/Config.in file has to be
adapted to include a new option so that the new implementation becomes selectable in SWUpdate's
configuration. In the simplest case, adding an option like the following one for hawkBit into the menu
"Server" section is sufficient.
config SURICATTA_HAWKBIT
bool "hawkBit support"
depends on HAVE_LIBCURL
depends on HAVE_JSON_C
select JSON
select CURL
help
Support for hawkBit server.
https://projects.eclipse.org/projects/iot.hawkbit
Having included the new server implementation into the configuration, edit suricatta/Makefile to specify
the implementation's linkage into the SWUpdate binary, e.g., for the hawkBit example implementation, the
following lines add server_hawkbit.o to the resulting SWUpdate binary if SURICATTA_HAWKBIT was selected
while configuring SWUpdate.
ifneq ($(CONFIG_SURICATTA_HAWKBIT),)
lib-$(CONFIG_SURICATTA) += server_hawkbit.o
endif
Support for general purpose HTTP server
This is a very simple backend that uses standard HTTP response codes to signal if an update is available.
There are closed source backends implementing this interface, but because the interface is very simple
interface, this server type is also suitable for implementing an own backend server.
The API consists of a GET with Query parameters to inform the server about the installed version. The
query string has the format:
http(s)://<base URL>?param1=val1¶m2=value2...
As examples for parameters, the device can send its serial number, MAC address and the running version of
the software. It is duty of the backend to interpret this - SWUpdate just takes them from the "identify"
section of the configuration file and encodes the URL.
The server answers with the following return codes:
┌───────────┬─────────────┬──────────────────────────────┐
│ HTTP Code │ Text │ Description │
├───────────┼─────────────┼──────────────────────────────┤
│ 302 │ Found │ A new software is available │
│ │ │ at URL in the Location │
│ │ │ header │
├───────────┼─────────────┼──────────────────────────────┤
│ 400 │ Bad Request │ Some query parameters are │
│ │ │ missing or in wrong format │
├───────────┼─────────────┼──────────────────────────────┤
│ 403 │ Forbidden │ Client certificate not valid │
├───────────┼─────────────┼──────────────────────────────┤
│ 404 │ Not found │ No update is available for │
│ │ │ this device │
├───────────┼─────────────┼──────────────────────────────┤
│ 503 │ Unavailable │ An update is available but │
│ │ │ server can't handle another │
│ │ │ update process now. │
└───────────┴─────────────┴──────────────────────────────┘
Server's answer can contain the following headers:
┌───────────────┬───────┬──────────────────────────────┐
│ Header's name │ Codes │ Description │
├───────────────┼───────┼──────────────────────────────┤
│ Retry-after │ 503 │ Contains a number which │
│ │ │ tells the device how long to │
│ │ │ wait until ask the next time │
│ │ │ for updates. (Seconds) │
├───────────────┼───────┼──────────────────────────────┤
│ Content-MD5 │ 302 │ Contains the checksum of the │
│ │ │ update file which is │
│ │ │ available under the url of │
│ │ │ location header │
├───────────────┼───────┼──────────────────────────────┤
│ Location │ 302 │ URL where the update file │
│ │ │ can be downloaded. │
└───────────────┴───────┴──────────────────────────────┘
The device can send logging data to the server. Any information is transmitted in a HTTP PUT request with
the data as plain string in the message body. The Content-Type Header need to be set to text/plain.
The URL for the logging can be set as separate URL in the configuration file or via --logurl command line
parameter:
The device sends data in a CSV format (Comma Separated Values). The format is:
value1,value2,...
The format can be specified in the configuration file. A format For each event can be set. The supported
events are:
┌─────────┬───────────────────────────────────────┐
│ Event │ Description │
├─────────┼───────────────────────────────────────┤
│ check │ dummy. It could send an event each │
│ │ time the server is polled. │
├─────────┼───────────────────────────────────────┤
│ started │ A new software is found and SWUpdate │
│ │ starts to install it │
├─────────┼───────────────────────────────────────┤
│ success │ A new software was successfully │
│ │ installed │
├─────────┼───────────────────────────────────────┤
│ fail │ Failure by installing the new │
│ │ software │
└─────────┴───────────────────────────────────────┘
The general server has an own section inside the configuration file. As example:
gservice =
{
url = ....;
logurl = ;
logevent : (
{event = "check"; format="#2,date,fw,hw,sp"},
{event = "started"; format="#12,date,fw,hw,sp"},
{event = "success"; format="#13,date,fw,hw,sp"},
{event = "fail"; format="#14,date,fw,hw,sp"}
);
}
date is a special field and it is interpreted as localtime in RFC 2822 format. Each Comma Separated field
is looked up inside the identify section in the configuration file, and if a match is found the
substitution occurs. In case of no match, the field is sent as it is. For example, if the identify
section has the following values:
identify : (
{ name = "sp"; value = "333"; },
{ name = "hw"; value = "ipse"; },
{ name = "fw"; value = "1.0"; }
);
with the events set as above, the formatted text in case of "success" will be:
Formatted log: #13,Mon, 17 Sep 2018 10:55:18 CEST,1.0,ipse,333
CONFIG FOR HAWKBIT UNDER SSL/TLS USING PRIVATE CA / SUB CA
A user-contributed recipe based on hawkBit (0.2.0-SNAPSHOT) + swupdate (v2018.03)
Purpose
Use HTTPS on a hawkBit server to avoid server spoofing. Anonymous client connections are authorized.
Recipe
1. On the PKI:
• Create a pkcs#12 (.p12) file, rolling server key, server, private CA, sub CA certs into a single
file.
• Use a password on the server key you won't be ashamed of.
• Also create a single .pem file for the private CA + sub-CA
2. On the hawkBit host:
• hawkBit uses the Java KeyStore to access credentials, but a JKS is not designed apparently to hold
CA certs, which is a problem for private CAs. The workaround is to make it gulp an entire pkcs#12
file.
• It looks like a JKS like this cannot have a password different from the one protecting the .p12.
Keytool also seems to have a little tendency to destruct the .jks if you change your mind and want
to change the password... Basically do everything you need with openssl and use only keytool for
generating the .jks file.
The following command imports a .p12 into a "pkcs12 Java keystore", keeping the same password:
keytool -importkeystore -srckeystore hb-pass.p12 -srcstoretype pkcs12 \
-destkeystore hb-pass.jks -deststoretype pkcs12 \
-alias 1 -deststorepass <password_of_p12>
Then you need to adapt application.properties of the hawkBit server to make use of the keystore.
There are extra requirements to make hawkBit send artifacts via HTTPS.
This is the relevant part of <hawkBit
dir>/hawkbit-runtime/hawkbit-update-server/src/main/resources/application.properties:
# HTTPS mode working w/ swupdate
# See also https://docs.spring.io/spring-boot/docs/1.4.7.RELEASE/reference/html/howto-embedded-servlet-containers.html#howto-configure-ssl
# https://github.com/eclipse/hawkbit/issues/618
#
# Need to run as root to use port 443
server.hostname=hb.domain
server.port=8443
#
# Overriding some of hawkbit-artifactdl-defaults.properties is required
hawkbit.artifact.url.protocols.download-http.protocol=https
hawkbit.artifact.url.protocols.download-http.port=8443
#
# Upgrades http:8443 to https:8443
# Would redirect + upgrade http:80 to https:443
security.require-ssl=true
server.use-forward-headers=true
#
# Server cert+key w/ private CA + subCA
# See also https://stackoverflow.com/questions/906402/how-to-import-an-existing-x509-certificate-and-private-key-in-java-keystore-to-u
# http://cunning.sharp.fm/2008/06/importing_private_keys_into_a.html (2008, still relevant!?)
#
# File .jks is a .p12 imported via keytool. Only one password supported, set from openssl.
server.ssl.key-store=hb-pass.jks
server.ssl.key-password=password
server.ssl.key-store-password=password-yes_the_same_one
...
3. On the swupdate client host(s):
• The client needs the private CA certificate(s) to authenticate the server.
• There is a setting in swupdate to specify the path to a single CA cert, not a directory. Beyond
that, libcurl looks into /etc/ssl/certs. So we're using a compound "CA chain" .pem file to hold both
private CA and sub-CA in our preferred location.
This is the relevant part of /etc/swupdate/swupdate.conf:
...
suricatta :
{
tenant = "default";
id = "machineID";
url = "https://hb.domain:8443";
nocheckcert = false;
cafile = "/etc/swupdate/priv-cachain.pem"; /* CA + sub CA in one file */
/* sslkey = anon client: do not set; */
/* sslcert = anon client: do not set; */
...
SWUPDATE: API FOR EXTERNAL PROGRAMS
Overview
SWUpdate contains an integrated web-server to allow remote updating. However, which protocols are
involved during an update is project specific and differs significantly. Some projects can decide to use
FTP to load an image from an external server, or using even a proprietary protocol. The integrated
web-server uses this interface.
SWUpdate has a simple interface to let external programs to communicate with the installer. Clients can
start an upgrade and stream an image to the installer, querying then for the status and the final result.
The API is at the moment very simple, but it can easy be extended in the future if new use cases will
arise.
API Description
The communication runs via UDS (Unix Domain Socket). The socket is created at startup by SWUpdate in
/tmp/sockinstctrl as per default configuration. This socket should, however, not be used directly but
instead by the Client Library explained below.
The exchanged packets are described in network_ipc.h
typedef struct {
int magic;
int type;
msgdata data;
} ipc_message;
Where the fields have the meaning:
• magic : a magic number as simple proof of the packet
• type : one of REQ_INSTALL, ACK, NACK, GET_STATUS, POST_UPDATE, SWUPDATE_SUBPROCESS, SET_AES_KEY
• msgdata : a buffer used by the client to send the image or by SWUpdate to report back notifications and
status.
The client sends a REQ_INSTALL packet and waits for an answer. SWUpdate sends back ACK or NACK, if for
example an update is already in progress.
After the ACK, the client sends the whole image as a stream. SWUpdate expects that all bytes after the
ACK are part of the image to be installed. SWUpdate recognizes the size of the image from the CPIO
header. Any error lets SWUpdate to leave the update state, and further packets will be ignored until a
new REQ_INSTALL will be received. [image]
It is recommended to use the client library to communicate with SWUpdate. On the lower level with direct
socket communication, it cannot be guaranteed that the structures will remain compatible in the future.
The client library was affected by this issue, too, and it is changed to accept an opaque interface that
will survive API changes. Compatibility layers could be added on-demand in the future due to API changes.
Client Library
Functions to start an update
A library simplifies the usage of the IPC making available a way to start asynchronously an update.
The library consists of one function and several call-backs.
int swupdate_async_start(writedata wr_func, getstatus status_func,
terminated end_func, void *req, ssize_t size)
typedef int (*writedata)(char **buf, int *size);
typedef int (*getstatus)(ipc_message *msg);
typedef int (*terminated)(RECOVERY_STATUS status);
swupdate_async_start creates a new thread and start the communication with SWUpdate, triggering for a new
update. The wr_func is called to get the image to be installed. It is responsibility of the callback to
provide the buffer and the size of the chunk of data.
The getstatus call-back is called after the stream was downloaded to check how upgrade is going on. It
can be omitted if only the result is required.
The terminated call-back is called when SWUpdate has finished with the result of the upgrade.
Example about using this library is in the examples/client directory.
The req structure is casted to void to ensure API compatibility. Am user should instantiate it as struct
swupdate_request. This contains fields that can control the update process:
struct swupdate_request {
unsigned int apiversion;
sourcetype source;
int dry_run;
size_t len;
char info[512];
char software_set[256];
char running_mode[256];
};
A user should first call swupdate_prepare_req()
void swupdate_prepare_req(struct swupdate_request *req);
This fills the request structure with default values. After that, the user can fill the other fields as:
• sourcetype : one of SOURCE_UNKNOWN, SOURCE_WEBSERVER, SOURCE_SURICATTA, SOURCE_DOWNLOADER,
SOURCE_LOCAL
• dry_run : one of RUN_DEFAULT (set from command line), RUN_DRYRUN, RUN_INSTALL.
• info, len : a variable length data that can be forwarded to the progress interface. The installer in
SWUpdate does not evaluate it.
• software_set and running_mode : this allows one to set the selection for the update.
Functions to set AES keys
The key for decryption can be set with command line parameter (see -K), but it is possible to set it via
IPC. In this way, each update could have a different key.
int swupdate_set_aes(char *key, char *ivt)
The key is for AES-256. The length for key and ivt are then defined by the algorithm amd they are passed
as ASCII string, so the length must be 64 bytes for key and 32 bytes for IVT.
Functions to control SWUpdate
int ipc_send_cmd(ipc_message *msg);
ipc_send_cmd is used to send a command to a SWUpdate subprocess (as suricatta). The function is synchron,
that means it clocks until the subprocess has answered with ACK or NACK. This function sets type to
SWUPDATE_SUBPROCESS. The caller must then set the other fields in message according to the destination.
The msgdata field is a structure as:
struct {
sourcetype source; /* Who triggered the update */
int cmd; /* Optional encoded command */
int timeout; /* timeout in seconds if an aswer is expected */
unsigned int len; /* Len of data valid in buf */
char buf[2048]; /*
* Buffer that each source can fill
* with additional information
*/
}
The caller fills source with the subprocess that acceps the command. Values of cmd are in network_ipc.h.
Messages for suricatta
suricatta accepts messages in JSON format. The message must be formatted in the buf field of the message
data.
Setting the polling time
{ "polling" : <value in seconds, range 0..X>}
Setting it to 0 has the special meaning that the polling time is retrieved from the Backend (if this is
supported by the server).
Enable / disable Suricatta daemon
{ "enable" : true }
{ "enable" : false }
Set custom device attributes for Suricatta (for Hawkbit implementation)
{ "identify" : [
{
"name" : "customizableAttributeOne",
"value" : "valueOne"
},
{
"name" : "customizableAttributeTwo",
"value" : "valueTwo"
}
]}
New attributes can be added at runtime, and existing attributes can be modified in the same way. Changes
will be reflected on the server in the next poll iteration.
Trigger a check on the server
This is useful in case the device is mostly offline, and when it is online, it should check immediately
if an update exists and run it. In fact, after enabling the suricatta daemon, the update follows the
usual states, and the daemon waits for a polling time before loading the new software. This command
forces an update (if available) without changing the polling time.
{ "trigger" : true }
Activate an already installed Software
After a software was installed, the new software boots and if everything runs fine, an acknowledge should
be sent to the hawkBit server. If this feature is used, for example to let the end user decide if the new
software is accepted, the parameters used by the installation should be stored during the update process.
{ "id" : <action id>,
"finished" : "success", "failure", "none",
"execution" : ["closed", "proceeding", canceled", "rejected", "resumed"]
"details" : [ ]
}
Get hawkBit Server Status
To provide the hawkBit server status to other processes, it can be requested by sending an empty message
with message type CMD_GET_STATUS.
The response is a JSON object containing the hawkBit server status <status>. <status> is a number
representing the value of the channel_op_res_t enum from channel_op_res.h. As the hawkBit server is
polled, its status can only be updated when it has been polled. Therefore the response also contains the
time <time>, when the hawkBit server has been polled the last time. It is provided as ISO 8601 date and
time string. (2021-10-14T13:42:37.000+00)
{ "server" : {
"status" : <status>
"time" : <time>
}
}
An example application can be found under tools/swupdate-gethawkbitstatus.c
API to the integrated Webserver
The integrated Webserver provides REST resources to push a SWU package and to get inform about the update
process. This API is based on HTTP standards. There are to kind of interface:
• Install API to push a SWU and to restart the device after update.
• A WebSocket interface to send the status of the update process.
Install API
POST /upload
This initiates an update: the initiator sends the request and start to stream the SWU in the same way as
described in API Description.
Restart API
POST /restart
If configured (see post update command), this request will restart the device.
WebSocket API
The integrated Webserver exposes a WebSocket API. The WebSocket protocol specification defines ws
(WebSocket) and wss (WebSocket Secure) as two new uniform resource identifier (URI) schemes that are used
for unencrypted and encrypted con nections, respectively and both of them are supported by SWUpdate. A
WebSocket provides full-duplex communication but it is used in SWUpdate to send events to an external
host after each change in the update process. The Webserver sends JSON formatted responses as results of
internal events.
The response contains the field type, that defines which event is sent.
Event Type
┌─────────┬───────────────────────────────────────┐
│ type │ Description of event │
├─────────┼───────────────────────────────────────┤
│ status │ Event sent when SWUpdate's internal │
│ │ state changes │
├─────────┼───────────────────────────────────────┤
│ source │ Event to inform from which interface │
│ │ an update is received │
├─────────┼───────────────────────────────────────┤
│ info │ Event with custom message to be │
│ │ passed to an external process │
├─────────┼───────────────────────────────────────┤
│ message │ Event that contains the error message │
│ │ in case of error │
├─────────┼───────────────────────────────────────┤
│ step │ Event to inform about the running │
│ │ update │
└─────────┴───────────────────────────────────────┘
Status Change Event
This event is sent when the internal SWUpdate status change. Following status are supported:
IDLE
START
RUN
SUCCESS
Example:
{
"type": "status",
"status": "SUCCESS"
}
Source Event
This event informs from which interface a SWU is loaded.
{
"type": "source",
"source": "WEBSERVER"
}
The field source can have one of the following values:
UNKNOWN
WEBSERVER
SURICATTA
DOWNLOADER
LOCAL
Info Event
This event forwards all internal logs sent with level=INFO.
{
"type": "info",
"source": < text message >
}
Message Event
This event contains the error message in case of failure.
Fields for message event
┌────────┬───────────────────────────────────┐
│ name │ Description │
├────────┼───────────────────────────────────┤
│ status │ "message" │
├────────┼───────────────────────────────────┤
│ level │ "3" in case of error, "6" as info │
├────────┼───────────────────────────────────┤
│ text │ Message associated to the event │
└────────┴───────────────────────────────────┘
Example:
{
"type": "message",
"level": "3",
"text" : "[ERROR] : SWUPDATE failed [0] ERROR core/cpio_utils.c : ",
}
Step event
This event contains which is the current step running and which percentage of this step is currently
installed.
Fields for step event
┌─────────┬───────────────────────────────────────┐
│ name │ Description │
├─────────┼───────────────────────────────────────┤
│ number │ total number of steps N for this │
│ │ update │
├─────────┼───────────────────────────────────────┤
│ step │ running step in range [1..N] │
├─────────┼───────────────────────────────────────┤
│ name │ filename of artefact to be installed │
├─────────┼───────────────────────────────────────┤
│ percent │ percentage of the running step │
└─────────┴───────────────────────────────────────┘
Example:
{
"type": "step",
"number": "7",
"step": "2",
"name": "rootfs.ext4.gz",
"percent": "18"
}
GETTING INFORMATION ON RUNNING UPDATE
It is often required to inform the operator about the status of the running update and not just to return
if the update was successful or not. For example, if the target has a display or a remote interface, it
can be forwarded which is reached percentage of the update to let estimate how much the update will still
run. SWUpdate has an interface for this ("progress API"). An external process can register itself with
SWUpdate, and it will receive notifications when something in the update was changed. This is different
from the IPC API, because the last one is mainly used to transfer the SWU image, and it is only possible
to poll the interface to know if the update is still running.
API Description
An external process registers itself to SWUpdate with a connect() request to the domain socket
"/tmp/swupdateprog" as per default configuration of SWUpdate. There is no information to send, and
SWUpdate simply inserts the new connection into the list of processes to be informed. SWUpdate will send
a frame back after any change in the update process with the following data (see include/progress_ipc.h):
struct progress_msg {
unsigned int magic; /* Magic Number */
unsigned int status; /* Update Status (Running, Failure) */
unsigned int dwl_percent; /* % downloaded data */
unsigned int nsteps; /* No. total of steps */
unsigned int cur_step; /* Current step index */
unsigned int cur_percent; /* % in current step */
char cur_image[256]; /* Name of image to be installed */
char hnd_name[64]; /* Name of running handler */
sourcetype source; /* Interface that triggered the update */
unsigned int infolen; /* Len of data valid in info */
char info[2048]; /* additional information about install */
};
The single fields have the following meaning:
• magic is not yet used, it could be added for simply verification of the frame.
• status is one of the values in swupdate_status.h (START, RUN, SUCCESS, FAILURE, DOWNLOAD, DONE).
• dwl_percent is the percentage of downloaded data when status = DOWNLOAD.
• nsteps is the total number of installers (handlers) to be run.
• cur_step is the index of the running handler. cur_step is in the range 1..nsteps
• cur_percent is the percentage of work done inside the current handler. This is useful when updating
a slow interface, such as a slow flash, and signals which is the percentage of image already copied
into the destination.
• cur_image is the name of the image in sw-description that is currently being installed.
• hnd_name reports the name of the running handler.
• source is the interface that triggered the update.
• infolen length of data in the following info field.
• info additional information about installation.
As an example for a progress client, tools/swupdate-progress.c prints the status on the console and
drives "psplash" to draw a progress bar on a display.
LANGUAGE BINDINGS
Overview
In general, SWUpdate is agnostic to a particular language it is operated from, thanks to SWUpdate's
socket-based control and progress APIs for external programs. As long as the language of choice has
proper socket (library) support, SWUpdate can be operated with it.
However, for convenience, a Lua language binding in terms of a shared library, currently
lua_swupdate.so.0.1, is provided.
Lua Language Binding
The Lua language binding is realized in terms of the swupdate module that defines three bindings, namely
for the control interface, the progress interface, and a convenience function yielding a table holding
all local network interfaces including their IP addresses and submasks.
Note that, depending on the filesystem location of the Lua binding's shared library, Lua's package.cpath
may have to be adapted by setting the environment variable LUA_CPATH, modifying package.cpath prior to a
require('lua_swupdate'), or , as last resort, using package.loadlib() instead of require('lua_swupdate').
Control Interface
The swupdate module's control interface binding conveniently makes SWUpdate's socket-based control API
available to pure Lua.
The binding is captured in the swupdate_control object that is returned by a call to swupdate.control().
This object offers the three methods connect(), write(<chunkdata>), and close():
The connect() method initializes the connection to SWUpdate's control socket, sends REQ_INSTALL, and
waits for ACK or NACK, returning the socket connection file descriptor, mostly for information purposes,
or, in case of an error, nil plus an error message.
The artifact's data can then be sent to SWUpdate via the write(<chunkdata>) method, returning true, or,
in case of errors, nil plus an error message.
Finally, the close() method closes the connection to SWUpdate's control socket after which it waits for
SWUpdate to complete the update transaction and executes the post-install command, if given.
The following example snippet illustrates how to use the control interface binding:
local artifact = io.open("/some/path/to/artifact.swu", "rb" )
swupdate = require('lua_swupdate')
local ctrl = swupdate.control()
if not ctrl:connect() then
-- Deliberately neglecting error message.
io.stderr:write("Error connecting to SWUpdate control socket.\n")
return
end
while true do
local chunk = artifact:read(1024)
if not chunk then break end
if not ctrl:write(chunk) then
-- Deliberately neglecting error message.
io.stderr:write("Error writing to SWUpdate control socket.\n")
break
end
end
local res, msg = ctrl:close()
if not res then
io.stderr:write(string.format("Error finalizing update: %s\n", msg))
end
Progress Interface
The swupdate module's progress interface binding conveniently makes SWUpdate's socket-based progress API
available to pure Lua.
The binding is captured in the swupdate_progress object that is returned by a call to
swupdate.progress(). This object offers the three methods connect(), receive(), and close():
The connect() method connects to SWUpdate's progress socket, waiting until the connection has been
established. Note that it is only really required to explicitly call connect() to reestablish a broken
connection as the swupdate_progress object's instantiation already initiates the connection.
The receive() method returns a table representation of the struct progress_msg described in the progress
interface's API description.
The close() method deliberately closes the connection to SWUpdate's progress socket.
IPv4 Interface
For convenience, the swupdate module provides the ipv4() method returning a table holding the local
network interfaces as the table's keys and their space-separated IP addresses plus subnet masks as
respective values.
META-SWUPDATE: BUILDING WITH YOCTO
Overview
The Yocto-Project is a community project under the umbrella of the Linux Foundation that provides tools
and template to create the own custom Linux-based software for embedded systems.
Add-on features can be added using layers. meta-swupdate is the layer to cross-compile the SWUpdate
application and to generate the compound SWU images containing the release of the product. It contains
the required changes for mtd-utils and for generating Lua. Using meta-SWUpdate is a straightforward
process. As described in Yocto's documentation about layers, you should include it in your bblayers.conf
file to use it.
Add meta-SWUpdate as usual to your bblayers.conf. You have also to add meta-oe to the list.
In meta-SWUpdate there is a recipe to generate an initrd with a rescue system with SWUpdate. Use:
MACHINE=<your machine> bitbake swupdate-image
You will find the result in your tmp/deploy/<your machine> directory. How to install and start an initrd
is very target specific - please check in the documentation of your bootloader.
What about libubootenv ?
This is a common issue when SWUpdate is built. SWUpdate depends on this library, that is generated from
the U-Boot's sources. This library allows one to safe modify the U-Boot environment. It is not required
if U-Boot is not used as bootloader. If SWUpdate cannot be linked, you are using an old version of
U-Boot (you need at least 2016.05). If this is the case, you can add your own recipe for the package
u-boot-fw-utils, adding the code for the library.
It is important that the package u-boot-fw-utils is built with the same sources of the bootloader and for
the same machine. In fact, the target can have a default environment linked together with U-Boot's code,
and it is not (yet) stored into a storage. SWUpdate should be aware of it, because it cannot read it: the
default environment must be linked as well to SWUpdate's code. This is done inside the libubootenv.
If you build for a different machine, SWUpdate will destroy the environment when it tries to change it
the first time. In fact, a wrong default environment is taken, and your board won't boot again.
To avoid possible mismatch, a new library was developed to be hardware independent. A strict match with
the bootloader is not required anymore. The meta-swupdate layer contains recipes to build the new library
(libubootenv) and adjust SWUpdate to be linked against it. To use it as replacement for u-boot-fw-utils:
• set PREFERRED_PROVIDER_u-boot-fw-utils = "libubootenv"
• add to SWUpdate config:
CONFIG_UBOOT=y
CONFIG_UBOOT_NEWAPI=y
With this library, you can simply pass the default environment as file (u-boot-initial-env). It is
recommended for new project to switch to the new library to become independent from the bootloader.
The swupdate class
meta-swupdate contains a class specific for SWUpdate. It helps to generate the SWU image starting from
images built inside the Yocto. It requires that all components, that means the artifacts that are part of
the SWU image, are present in the Yocto's deploy directory. This class should be inherited by recipes
generating the SWU. The class defines new variables, all of them have the prefix SWUPDATE_ in the name.
• SWUPDATE_IMAGES : this is a list of the artifacts to be packaged together. The list contains the name
of images without any extension for MACHINE or filetype, that are added automatically. Example :
SWUPDATE_IMAGES = "core-image-full-cmdline uImage"
• SWUPDATE_IMAGES_FSTYPES : extension of the artifact. Each artifact can have multiple extension
according to the IMAGE_FSTYPES variable. For example, an image can be generated as tarball and as
UBIFS for target. Setting the variable for each artifact tells the class which file must be packed
into the SWU image.
SWUPDATE_IMAGES_FSTYPES[core-image-full-cmdline] = ".ubifs"
• SWUPDATE_IMAGES_NOAPPEND_MACHINE : flag to use drop the machine name from the artifact file. Most
images in deploy have the name of the Yocto's machine in the filename. The class adds automatically the
name of the MACHINE to the file, but some artifacts can be deployed without it.
SWUPDATE_IMAGES_NOAPPEND_MACHINE[my-image] = "1"
• SWUPDATE_SIGNING : if set, the SWU is signed. There are 3 allowed values: RSA, CMS, CUSTOM. This value
determines used signing mechanism.
• SWUPDATE_SIGN_TOOL : instead of using openssl, use SWUPDATE_SIGN_TOOL to sign the image. A typical use
case is together with a hardware key. It is available if SWUPDATE_SIGNING is set to CUSTOM
• SWUPDATE_PRIVATE_KEY : this is the file with the private key used to sign the image using RSA
mechanism. Is available if SWUPDATE_SIGNING is set to RSA.
• SWUPDATE_PASSWORD_FILE : an optional file containing the password for the private key. It is available
if SWUPDATE_SIGNING is set to RSA.
• SWUPDATE_CMS_KEY : this is the file with the private key used in signing process using CMS mechanism.
It is available if SWUPDATE_SIGNING is set to CMS.
• SWUPDATE_CMS_CERT : this is the file with the certificate used in signing process using using CMS
method. It is available if SWUPDATE_SIGNING is set to CMS.
• SWUPDATE_AES_FILE : this is the file with the AES password to encrypt artifact. A new fstype is
supported by the class (type: enc). SWUPDATE_AES_FILE is generated as output from openssl to create a
new key with
openssl enc -aes-256-cbc -k <PASSPHRASE> -P -md sha1 > $SWUPDATE_AES_FILE
To use it, it is enough to add IMAGE_FSTYPES += "enc" to the artifact. SWUpdate supports decryption of
compressed artifact, such as
IMAGE_FSTYPES += ".ext4.gz.enc"
Automatic sha256 in sw-description
The swupdate class takes care of computing and inserting sha256 hashes in the sw-description file. The
attribute sha256 must be set in case the image is signed. Each artifact must have the attribute:
sha256 = "@artifact-file-name"
For example, to add sha256 to the standard Yocto core-image-full-cmdline:
sha256 = "@core-image-full-cmdline-machine.ubifs";
The name of the file must be the same as in deploy directory.
BitBake variable expansion in sw-description
To insert the value of a BitBake variable into the update file, pre- and postfix the variable name with
"@@". For example, to automatically set the version tag:
version = "@@DISTRO_VERSION@@";
Automatic versions in sw-description
By setting the version tag in the update file to @SWU_AUTO_VERSION it is automatically replaced with PV
from BitBake's package-data-file for the package matching the name of the provided filename tag. For
example, to set the version tag to PV of package u-boot:
filename = "u-boot";
...
version = "@SWU_AUTO_VERSION";
Since the filename can differ from package name (deployed with another name or the file is a container
for the real package) you can append the correct package name to the tag:
@SWU_AUTO_VERSION:<package-name>. For example, to set the version tag of the file packed-bootloader to
PV of package u-boot:
filename = "packed-bootloader";
...
version = "@SWU_AUTO_VERSION:u-boot";
To automatically insert the value of a variable from BitBake's package-data-file different to PV (e.g.
PKGV) you can append the variable name to the tag: @SWU_AUTO_VERSION@<package-data-variable>. For
example, to set the version tag to PKGV of package u-boot:
filename = "u-boot";
...
version = "@SWU_AUTO_VERSION@PKGV";
Or combined with a different package name:
filename = "packed-bootloader";
...
version = "@SWU_AUTO_VERSION:u-boot@PKGV";
Using checksum for version
It is possible to use the hash of an artifact as the version in order to use "install-if-different".
This allows versionless artifacts to be skipped if the artifact in the update matches the currently
installed artifact.
In order to use the hash as the version, the sha256 hash file placeholder described above in Automatic
sha256 in sw-description must be used for version.
Each artifact must have the attribute:
version = "@artifact-file-name"
The name of the file must be the same as in deploy directory.
Template for recipe using the class
DESCRIPTION = "Example recipe generating SWU image"
SECTION = ""
LICENSE = ""
# Add all local files to be added to the SWU
# sw-description must always be in the list.
# You can extend with scripts or wahtever you need
SRC_URI = " \
file://sw-description \
"
# images to build before building swupdate image
IMAGE_DEPENDS = "core-image-full-cmdline virtual/kernel"
# images and files that will be included in the .swu image
SWUPDATE_IMAGES = "core-image-full-cmdline uImage"
# a deployable image can have multiple format, choose one
SWUPDATE_IMAGES_FSTYPES[core-image-full-cmdline] = ".ubifs"
SWUPDATE_IMAGES_FSTYPES[uImage] = ".bin"
inherit swupdate
Simplified version for just image
In many cases there is a single image in the SWU. This is for example when just rootfs is updated. The
generic case described above required an additional recipe that must be written and maintained. For this
reason, a simplified version of the class is introduced that allowed to build the SWU from the image
recipe.
Users just need to import the swupdate-image class. This already sets some variables. A sw-description
must still be added into a files directory, that is automatically searched by the class. User still
needs to set SWUPDATE_IMAGE_FSTYPES[your image] to the fstype that should be packed into the SWU - an
error is raised if the flag is not set.
In the simple way, your recipe looks like
:: <your original recipe code>
SWUPDATE_IMAGES_FSTYPES[<name of your image>] = <fstype to be put into SWU> inherit swupdate-image
SWUPDATE BEST PRACTICE
This is intended as general rule to integrate SWUpdate into a custom project.
SWUpdate is an update agent and it is thought to be a framework. This means it is highly configurable and
SWUpdate should be configured to fit into a project, not vice versa. SWUpdate makes just a few
requirements on the system and it has no fixed update schema. There is no restriction on how many
partitions or which storage you are using. In some more complex cases, the update depends on a lot of
conditions, and SWUpdate can run differently according to the mode a device is started in. Think about
SWUpdate not being a ready-to-use updater but a framework, and hence you should first write a meaningful:
Update Concept
Take your time and write first an update concept for your device. It is not wasted time. You have to
imagine conditions when an update is not working, and try to write down the use cases when an update can
fail and how the device can be restored. SWUpdate installs new software, but a successful update means
that the new software is started and runs flawlessly. The interface with the bootloader (or the one that
starts the software) must be checked in details. A successful update means:
• SWUpdate runs successfully,
• the device reboots,
• the bootloader can start the new software, and
• the new software runs, makes some consistency checks, and declares that the transaction (that is from
old to new software) is terminated.
This means that some coordination between the bootloader and the update agent is necessary. In most
cases, this is done via persistent variables that are available to both SWUpdate and the bootloader.
SWUpdate has two built-in variables:
• recovery_status: this is set when SWUpdate starts to write to the device, and it is unset after the
installation completed with success or it is set to failed in case of error. A bootloader can use it to
check if an update was interrupted. This is a must in case a single copy of the software is on the
device.
• ustate: this triggers a state machine. SWUpdate sets it to 1 (INSTALL) after an update. The bootloader
can use it to check whether a new software must be tested. The bootloader should implement a counter
mechanism to check how many times it tried to start a new software. When a threshold is reached, the
bootloader should declare the new software as buggy, and proceed with a fallback in case the old
software is available.
A fallback is always initiated by the bootloader, because it knows if the new software is running. It
should toggle the copies and start the old software set. To communicate this to user space and to
SWUpdate, the bootloader sets the ustate variable to 3 (FAILED). SWUpdate uses this information in case
the result must be forwarded to an external server (like a backend). There is a time window when a
fallback can take place. In fact, after a reboot and some attempts, the update transaction is declared
successful or failed, and later a new update can be executed. When a new update runs, the status of the
stand-by copy is unknown, because it could be the result of an interrupted update. Running an incomplete
software can lead to unpredictable results and must be strongly avoided. A common pattern for a toggling
in the bootloader is:
• ustate is not set or set to 0 (no update pending). The bootloader runs the configured copy and won't
ever toggle. In case of failure, a rescue system can be started if available.
• ustate is set to 1 (INSTALLED): the new software is under test. The bootloader initiates a fallback if
the new software is not running and sets ustate to 3 (FAILED). If the new software runs, it is the duty
of user space (often SWUpdate or the init script for SWUpdate) to reset ustate to 0. Note that
resetting the variable is project specific, and it could be set as last action after having
sufficiently checked that the new software is running. This includes performing in the application a
database migration, starting communicating with peers, whatever.
Check in advance which security topics are relevant for your project. This includes:
• signed images (SWU is verified before installing), and then which crypto mechanism is used (RSA keys,
certificates, PKI)
• encryption for artifacts
• under which user and group SWUpdate and other components are allowed to run. Set user id and group id
if not root in swupdate.cfg.
• if any version can be installed or if you forbid a downgrade, and then be sure to pass the range of
versions you allow via --M, -R and --max-version.
• hardware-software compatibility check and how your device knows which hardware revision is running.
Reduce dependencies to minimum
An update should be possible in any condition. Even if the system is degraded or in a bad shape, if an
update can work, the device can be functional again without returning it back to the factory. SWUpdate
is thought to be self contained: that means it does not make use of external tools. If your system is
degraded and filesystems get corrupted, there are less chances to restore it if the update calls external
tools. SWUpdate is started at boot time and there are good chances it succeeds even if your system has
some (software) flaws. Be careful to make an update depending on your application or try to reduce the
dependencies. In fact, the application is updated often and an introduction of new bugs can make the
device no longer updatable. Take the dependencies under control, and if you have any, be sure that the
update is still working. You can fix any bugs if the update works, but not anymore if the device cannot
be updated.
Make a risk analysis
A more accurate analysis brings less surprises in the field. Think twice about what you want to update,
which components should be updated, and the risks of updating a single point of failure. Very often,
this means the bootloader. Compare risks and benefits: it happens in many projects that having the
possibility (with some risk) to update the bootloader is better that returning the devices back to
service. A cost / benefits analysis should be part of the integration of the update agent.
SWUpdate builtin configuration
SWUpdate has a compile time configuration. The default configuration delivered with meta-swupdate is not
suitable for most projects. The easy way to check configuration in Yocto is to run:
bitbake -c menuconfig swupdate
Outside Yocto, just run in SWUpdate's sources:
make menuconfig
Check security, bootloader, and which handlers should be installed. They depend strongly on your project.
If you build with OE, add a swupdate_%.bbappend to one of your layers, and put the resulting
configuration file as defconfig that can be fetched. Please review the following configuration:
• Security settings
• Interfaces required (where the software is coming from). Disable the interface you do not need.
• Handlers required for your project. Disable what you do not need, but consider if you could need some
of them in future. As example, you can safely disable ubivol if you do not use raw NAND, but you can
let archive enabled if you plan to install artifacts from tarballs in future.
• It is highly recommended to enable Lua to extend runtime behavior.
SWUpdate startup
An easy way to start SWUpdate is provided only with meta-swupdate and Yocto. A generic SystemV init
script or a systemd unit for SWUpdate are executing a script swupdate.sh, that is delivered together with
the SWUpdate binaries. The script goes through /etc/swupdate/conf.d/ and sources all found files. The
integrator can use a set of predefined variables to configure SWUpdate's command line parameters.
• SWUPDATE_WEBSERVER_ARGS : This string is passed if the webserver must be started. It consists of the
webserver specific parameters. If this variable is set, the script will add -w to the list of
parameters. Note: meta-swupdate contains a default configuration for SWUPDATE_WEBSERVER_ARGS, that
uses /www as document root for the Website and default port 8080.
• SWUPDATE_SURICATTA_ARGS : Suricatta (backend) specific parameters. There is no default.
• SWUPDATE_ARGS : Parameters not belonging to Webserver or Suricatta.
Note that swupdate.sh sources the files in sorted order, so it is possible to override the variables with
a configuration file whose filename is loaded at the end. Preferred style is to use SystemV like names,
for example 10-webserver, 11-suricatta, and so on.
Write sw-description
sw-description is the central file that describes a new software release and how a release must be
installed. It should be a consequence of the update concept. There is not a single right way. SWUpdate
heavily uses 'selections' and links to extract just one part of the whole sw-description, that can be
used for different situations and different ways to run the device. One use case for selections is to
implement the dual-copy (often referred to as A/B) mode: one selection contains instructions for one
copy, the other for the second copy. Which copy is the stand-by must be detected before running SWUpdate
and passed via the -e <selection,mode> switch. Other methods set up a link to the standby storage (like
/dev/standby) during boot. Or the standby device can be detected at runtime with an embedded-script, as
part of sw-description, with Lua code. Please note that for the last case, SWUpdate is extended with
functions exported to the Lua context that simplify the detection. SWUpdate exports a getroot() function
that returns type and value for the device used as rootfs. See SWUpdate documentation for a complete list
of functions exported by SWUpdate that can be used in Lua. An embedded Lua script must just start with
require ('swupdate')
to make use of them.
Use OE variables as much as possible
meta-swupdate replaces a special construct in sw-description with the values of build variables. The
recognized construct in sw-description is delimited by @@, that is @@VARIABLE-NAME@@. The exception (for
compatibility reasons) is the automatic generation of sha256. The syntax in that case is :
sha256 = "@<name of artifact>"
You can again use variable substitution for artifact names. Example:
sha256 = "@@@SYSTEM_IMAGE@@-@@MACHINE@@@@SWUPDATE_IMAGES_FSTYPES[@@SYSTEM_IMAGE@@]@@";
Please note that each variable is double delimited (at the beginning and at the end) by @@.
Deliver your scripts instead of relying on them being installed
You have the freedom to call any tools during an update. However, take care if you are using some tools
from the running rootfs / current software. This implies that the current software is running flawlessly,
as well as the tools you are calling. And this may not always be the case.
Prefer Lua to shell scripts
Shell scripts are very popular, and they are often used even when they are not strictly required. They
can raise security issues. In fact, take as example a simple shell script. Goal of rootkits is often the
shell, because taking control of the shell means to control the whole device. If the shell is
compromised, the whole system is compromised. Running a shell script means that SWUpdate should call
"fork" followed by an "exec". This means also that many resources are duplicated in the child process,
and it could cause a further problem if system is getting rid of resources. A better approach is to use
Lua and to deliver the scripts inside the SWU. In fact, the Lua interpreter is linked to SWUpdate and
runs in context of the SWUpdate process without forking a child process. Shell is not involved at all. Of
course, Lua scripts should be written to be self-contained, too, and executing external tools should be
done only if unavoidable.
Use installed-directly when possible
SWUpdate can be enabled for zero-copy (or streaming mode), that is the incoming SWU is analyzed on the
fly and it is installed by the associated handler without any temporary copy. If this is not set,
SWUpdate creates a temporary copy in $TMPDIR before passing it to the handlers. Note that $TMPDIR
generally points to a RAMDISK and storing files there reduces the amount of memory available for the
application. It makes sense to disable the flag in case the artifact is a single point of failure. A
typical example could be the bootloader (not duplicated on the devices), and if the SWU is corrupted or
the connection gets broken, the board is left in a bricked state. It makes sense then to download the
whole artifact before installing.
Always enable sha256 verification
The SWU image is a CPIO archive with CRC (new ASCII format), but the check in CPIO is very weak. Do not
trust it, but enable sha256 for each artifact.
Always set the type attribute
SWUpdate sets some default handler if the type is not set. Do not use it, but set explicitly the type
(that is, which handler should install the artifact) in sw-description.
Do not rely on install order
SWUpdate does not require that artifacts are put into the CPIO in a specific order. The exception is
sw-description, that must be the first file in a SWU. Avoid dependencies inside the SWU, that is an
artifact that can be installed only after another one was installed before. If you really need it, for
example if you want to install a file into a filesystem provided as image, disable installed-directy for
the file and enable it for the filesystem image.
Do not drop atomicity !
SWUpdate guarantees atomicity as long as you don't do something that simply breaks it. As example, think
about the bootloader's environment. In an sw-description, there is a specific section where the
environment can be set, adding / modifying / deleting variables. SWUpdate does not change single
variables, but generates the resulting new environment for the supported bootloader and this is written
in one shot in a way (for U-Boot / EFIBootguard, not for GRUB) that is power-cut safe. You can of course
change the environment in a postinstall script, like in the following way (for U-Boot):
fw_setenv var1 val1
fw_setenv var2 val2
fw_setenv var3 val3
fw_setenv var4 val4
fw_setenv var5 val5
If a power cut happens during two calls of fw_setenv, the environment is in an intermediate state and
this can brick the device.
Plan to have a rescue system
Even if you have a double-copy setup, something can go wrong. Plan to have a rescue system
(swupdate-image in meta-swupdate) and to install it on a separate storage than the main system, if it is
possible. This helps when the main storage is corrupted, and the device can be restored in the field
without returning it back to the factory. Plan to update the rescue system as well: it is software, too,
and its bugs should be fixed, too.
DELTA UPDATE WITH SWUPDATE
Overview
The size of update packages is steadily increasing. While once the whole software was just a bunch of
megabytes, it is not unusual now that OS and application on devices running Linux as OS reach huge size
of Gigabytes.
Several mechanisms can be used to reduce the size of downloaded data. The resulting images can be
compressed. However, this is not enough when bandwidth is important and not cheap. It is very common
that a device will be upgraded to a version that is similar to the running one but add new features and
solves some bugs. Specially in case of just fixes, the new version is pretty much equal as the original
one. This asks to find methods to download just the differences with the current software without
downloading a full image. In case an update is performed from a known base, we talk about delta updates.
In the following chapter some well known algorithms are considered and verified if they can be integrated
into SWUpdate. The following criteria are important to find a suitable algorithm:
• license must be compatible with GPLv2
• good performance for smaller downloads, but not necessarily the best one.
• SWUpdate remains with the concept to deliver one package (SWU), the same independently from the
source where the SWU is stored (USB, OTA, etc.)
• It must comply to SWUpdate's security requirements (signed images, privilege separation, etc.)
Specific ad-hoc delta updates mechanisms can be realized when the nature of the updated files is the
same. It is always possible with SWUpdate to install single files, but coherency and compatibility with
the runningsoftware must be guaranteed by the integratot / manufacturer. This is not covered here: the
scope is to get an efficient and content unaware delta mechanism, that can upgrade in differential mode
two arbitrary images, without any previous knowledge about what they content.
FOSS projects for delta encoding
There are several algorithms for delta encoding, that is to find the difference between files, generally
in binary format. Only algorithms available under a compatible FOSS license (GPLv2) are considered for
SWUpdate. One of the goals in SWUpdate is that it should work independently which is the format of the
artifacts. Very specialized algorithm and libraries like Google's Courgette used in Chromium will give
much better results, but it works on programs (ELF files) and take advantages of the structure of
compiled code. In case of OTA update, not only software, but any kind of artifact can be delivered, and
this includes configuration data, databases, videos, docs, etc.
librsync
librsync is an independent implementation for rsync and does not use the rsync protocol. It is well
suited to generate offline differential update and it is already integrated into SWUpdate. However,
librsync takes the whole artifact and generates a differential image that is applied on the whole image.
It gives the best results in terms of reduced size when differences are very small, but the differential
output tends to be very large as soon as the differences are meaningful. Differential images created for
SWUpdate show that, as soon as the difference larger is, the resulting delta image can even become larger
as the original one.
SWUpdate supports librsync as delta encoder via the rdiff handler.
xdelta
xdelta uses the VCDIFF algorithm to compute differences between binaries. It is often used to deliver
smaller images for CD and DVD. The resulting images are created from an installed image that should be
loaded entirely in main memory. For this reason, it does not scale well when the images are becoming
larger and it is unsuitable for embedded systems and SWUpdate.
casync
casync is, according to his author. a tool for distributing images. It has several interesting aspects
that can be helpful with OTA update. Files itself are grouped together in chunks and casync creates a
"Chunk storage" where each chunk is stored on a separate file. The chunk storage is part of the delivery,
and it must be stored on a server. casync checks if the chunk is already present on the target, and if
not download it. If this seems to be what is required, there are some drawbacks if casync should be
integrated in SWUpdate:
• because of the nature of casync, each chunk is a separate file. This cause a huge number of new
connections, because each file is a separate GET on the server. The overhead caused to
re-instantiate connection is high on small devices, where SSL connections are also increasing CPU
load. There are downloads of hundreds or thousands of small files just to recreate the original
metadata file.
• casync has no authentication and verification and the index (.caidx or .caibx) are not signed. This
is known, but casync goals and scopes are outside the ones on embedded devices.
• it is difficult to deliver a whole chunk storage. The common usage for OTA is to deliver artifacts,
and they should be just a few. Thousands of files to be delivered to let casync to compute the new
image is not practical for companies: they have a new "firmware" or "software" and they need an easy
way to deliver this file (the output from their build system) to the devices. In some cases, they
are even not responsible for that, and the firmware is given to another authority that groups all
packages from vendors and realizes a sort of OTA service.
• casync is quite a huge project - even if it was stated that it will be converted into a library,
this never happened. This makes difficult to interface to SWUpdate, and using it as external process
is a no way in SWUpdate for security reason. It breaks privilege separation, and adds a lot of code
that is difficult to maintain.
For all these reasons, even if the idea of a chunk storage is good for an OTA updater, casync is not a
candidate for SWUpdate. A out-of-the-box solution cannot be found, and it is required to implement an own
solution that better suits for SWUpdate.
Zchunk - compression format
zchunk seems to combine the usage of a chunk storage without having to deliver it on a server. zchunk is
a FOSS project released under BSD by its author. The goal of this project is something else: zchunk
creates a new compression format that adds the ability to download the differences between new and old
file. This matches very well with SWUpdate. A zchunk file contains a header that has metadata for all
chunks, and according to the header, it is known which chunks must be downloaded and which ones can be
reused. zchunk has utilities to download itself the missing chunks, but it could be just used to find
which part of an artifact must be downloading, and SWUpdate can go on with its own way to do this.
One big advantage on this approach is that metadata and compressed chunks are still bound into a single
file, that can be built by the buildsystem and delivered as it is used to. The updater needs first the
metadata, that is the header in zchunk file, and processes it to detect which chunks need to be
downloaded. Each chunk has its own hash, and the chunks already available on the device are verified
against the hash to be sure they are not corrupted.
Zchunk supports multiple sha algorithms - to be compatible with SWUpdate, zchunk should be informed to
generate sha256 hashes.
Design Delta Update in SWUpdate
For all reasons stated before, zchunk is chosen as format to deliver delta update in SWUpdate. An
artifact can be generated in ZCK format and then the ZCK's header (as described in format) can be
extracted and added to the SWU. In this way, a ZCK file is signed (and if requested compressed and/or
encrypted) as part of the SWU, and loading chunks from an external URL can be verified as well because
the corresponding hashes are already verified as part of the header.
Changes in ZCHUNK project
Zchunk has an API that hides most of its internal, and provides a set of tools for creating and
downloading itself a file in ZCK format. Nevertheless, Zchunk relies on hashes for the compressed (ZST)
chunks, and it was missing for support for uncompressed data. To combine SWUpdate and zchunk, it is
required that a comparison can be done between uncompressed data, because it is unwanted that a device is
obliged to compress big amount of data just to perform a comparisons. A short list of changes in the
Zchunk project is:
• create hashes for uncompressed data and extend format to support it. The header must be extended to
include both size and hash of uncompressed data.
• make the library embedded friendly, that means reports errors in case of failure instead of exiting
and find a suitable way to integrate the log output for the caller.
• allow to use sha256 (already foreseen in zchunk) as this is the only hash type used in SWUpdate.
• add API to allow an external caller to take itself the decision if a chunk must be downloaded or
reused.
These changes were merged into Zchunk project - be sure to get a recent version of Zchunk, at least with
commit 1b36f8b5e0ecb, that means newer as 1.1.16.
Most of missing features in Zchunk listed in TODO for the project have no relevance here: SWUpdate
already verifies the downloaded data, and there is no need to add signatures to Zchunk itself.
Integration in sw-description
The most important part in a Zchunk file is the header: this contains all metadata and hashes to perform
comparisons. The zck tool splits a file in chunks and creates the header. Size of the header are know,
and the header itself can be extracted from the ZCK file. The header will be part of sw-description:
this is the header for the file that must be installed. Because the header is very small compared to the
size of the whole file (quite 1 %), this header can be delivered into the SWU.
Integration in SWUpdate: the delta handler
The delta handler is responsible to compute the differences and to download the missing parts. It is not
responsible to install the artifact, because this breaks the module design in SWUpdate and will constrain
to have just one artifact type, for example installing as raw or rawfile. But what about if the artifact
should be installed by a different handler, for example UBI, or a custom handler ? The best way is that
the delta handler does not install, but it creates the stream itself so that this stream can be passed to
another (chained) handler, that is responsible for installing. All current SWUpdate's handlers can be
reused: each handler does not know that the artifact is coming with separate chunks and it sees just a
stream as before. The delta handler has in short the following duties:
• parse and understanf the ZCK header
• create a ZCK header from the file / partition used as source for the comparison
• detect which chunks are missing and which one must be copied.
• build a mixer that copies and downloads all chunks and generates a stream for the following
handler.
• detect any error coming form the chained handler.
Because the delta handler requires to download more data, it must start a connection to the storage where
the original ZCK is stored. This can lead to security issues, because handlers run with high priviliges
because they write into the hardware. In fact, this breaks privilege separation that is part of SWUpdate
design. To avoid this, the delta handler does not download itself. A separate process, that can runs
with different userid and groupid, is responsible for this. The handler sends a request to this process
with a list of ranges that should be downloaded (see HTTP Range request). The delta handler does not know
how the chunks are downlaoded, and even if using HTTP Range Request is the most frequent choice, it is
open to further implementations. The downloader process prepares the connection and asks the server for
ranges. If the server is not able to provide ranges, the update aborts. It is in fact a requirement for
delta update that the server storing the ZCK file is able to answer to HTTP Range Request, and there is
no fallback to download the full file. An easy IPC is implemented between the delta handler and the
downloader process. This allows to exchange messages, and the downloader can inform the handler if any
error occurs so that the update can be stopped. The downloader will send a termination message when all
chunks will be downloaded. Because the number of missing chunks can be very high, the delta handler must
sends and organize several requests to the downloader, and tracking each of them. The downloader is
thought as dummy servant: it starts the connection, retrieves HTTP headers and data, and sends them back
to the caller. The delta handler is then responsible to parse the answer, and to retrieve the missing
chunks from the multipart HTTP body.
Creation of ZCK Header and ZCK file for SWUpdate
Zchunk supports more SHA algorithms and it sets as default SHA512/128. This is not compatible with
SWUpdate that just support SHA256. Be sure to generate header and chunks with SHA256 support. You have
to enable the generation of hashes for uncompressed chunk, too. A possible usage of the tool is:
zck --output <output file> -u --chunk-hash-type sha256 <artifact, like rootfs>
The output is the ZCK file with all chunks. This file should be put on a Webserver accessible to the
target, and that supports Range Request (RFC 7233). All modern Webserver support it.
The SWU must just contain the header. This can be easy extracted from the ZCK file with:
HSIZE=`zck_read_header -v <ZCK file> | grep "Header size" | cut -d':' -f2`
dd if=<ZCK FILE> of=<ZCK HEADER file> bs=1 count=$((HSIZE))
Using ZCK tools to foresee download size
There are tools that can be used at build time to know how many chunks should be downloaded when a device
is upgrading from a known version. You can use zck_cmp_uncomp from the test directory:
../build/test/zck_cmp_uncomp --verbose <uncompressed old version file> <ZCK file>
This prints a list with all chunks, marking them with SRC if they are the same in the old version and
they should not retrieved and with DST if they are new and must be downloaded. The tool show at the end
a summary with the total number of bytes of the new release (uncompressed) and how many bytes must be
downloaded for the upgrade. Please remmeber that these value are just payload. SWUpdate reports a
summary, too, but it takes into account also the HTTP overhead (headers, etc.), so that values are not
the same and the ones from SWUpdate are slightly bigger.
SWUPDATE-CLIENT
swupdate-client is a small tool that sends a SWU image to a running instance of SWUpdate. It can be used
if the update package (SWU) is downloaded by another application external to SWUpdate. It is an example
how to use the IPC to forward an image to SWUpdate.
SYNOPSIS
swupdate-client [OPTIONS] <image.swu to be installed>...
DESCRIPTION
-h print help and exit
-d ask the server to only perform a dry-run
-q go quiet, resets verbosity
-v go verbose, essentially print upgrade status messages from server
-p ask the server to run post-update commands if upgrade succeeds
SWUPDATE-PROGRESS
swupdate-progress tries to connect to a running instance of SWUpdate to get the status of a running
update.
SYNOPSIS
swupdate-progress [option]
DESCRIPTION
swupdate-progress is an example how to connect to SWUpdate via the progress interface. It shows on
stdout a simple bar with the percent indication of the current update and reports the result of the
update. It can optionally drive "psplash" or execute a script after an update.
-c Use colors to show results on stdout
-e Command to be execute after an update
-p send percentage to psplash
-r optionally reboot the target after a successful update
-s path to progress IPC socket in case the default is not taken
-w waits for a SWUpdate connection instead of exit with error
-h print a help
SWUPDATE-IPC
swupdate-ipc sends a specific command to SWUpdate. The following commands are enabled:
aes
send a new aes key to SWUpdate
setversion
sends a range of versions that can be accepted.
gethawkbit
return status of the connection to Hawkbit.
sendtohawkbit
send data to the Hawkbit Server. A typical use case is acknowledgement for activation by an application
or operator after a new software has been installed. The tool can forward the result for the activation
to the hawkBit server.
sysrestart
send a restart command after a network update
SYNOPSIS
swupdate-ipc <cmd> [option]
Where cmd is one of the listed above.
DESCRIPTION
aes <key> <ivt>
AES key to be used for decryption
setversion <min> <max> <current>
configure the accepted range of versions
hawkbitcfg
configuration for Hawkbit Module
-h help
-p allows one to set the polling time (in seconds)
-e enable suricatta mode
-d disable suricatta mode
swupdate-sendtohawkbit <action id> <status> <finished> <execution> <detail 1> <detail 2> ..
Send Acknolwedge to Hawkbit server after an update if SWUpdate is set to wait for.
sysrestart
Used with SWU handler, allow to perform a network restart
-r optionally reboot the target after a successful update
-w waits for a SWUpdate connection instead of exit with error
-s <path>
path to progress IPC socket
HELP AND SUPPORT
Mailing List
There is a mailing list for this project:
swupdate@googlegroups.com
Issue related to the project or to the documentation are discussed here.
SWUpdate Flyer
A short description about the project and the features (in English and German) can be found in the flyer
Workshop and SWUpdate integration in project
For quick integration of SWUpdate in your project, you could be interested in the Training
Commercial support and board integration
Please check for services <https://swupdate.org/services> if you need professional support or you need
help to get SWUpdate on your device.
Talks about SWUpdate
• Software Update in Embedded Systems by Stefano Babic
• Updating Embedded Linux devices in field by Chris Simmonds
• OpenEmbedded in the Real World by Scott Murray
• [RFC] Device-side support for software update in AGL by Matt Porter
• Open Source secure software updates for Linux-based IVI systems by Arthur Taylor
• How do you update your embedded Linux devices? by Daniel Sangorrin / Keijiro Yano
• Comparison of Linux Software Update Technologies by Matt Porter
• Software update for IoT: the current state of play by Chris Simmonds, ELCE 2016, Slides, Video
• OSS Remote Firmware Updates for IoT-like Projects by Silvano Cirujano Cuesta, ELCE 2016, Slides ELCE
2016, Video ELCE 2016
• System Upgrade with SWUpdate by Gabriel Huau, ELC 2017, Slides ELC 2017, Video ELC 2017
• BoF: Secure OTA Collaboration, by Ricardo Salveti and Alan Bennett, ELCE 2017
• Orchestrated Android-Style System Upgrades for Embedded Linux by Diego Rondini, ELCE 2017, Slides
Android-Style, Video Android-Style
• Updating an Embedded System with SWUpdate Framework by Stefano Babic, ELCE 2017, Slides ELCE 2017,
Video ELCE 2017
• Upgrading buildroot based devices with SWUpdate by Angelo Compagnucci, LinuxLab 2018, Slides
LinuxLab 2018, Video LinuxLab 2018,
• Evolution of (OTA) Update in the IoT world by Stefano Babic, ELC 2019, Slides ELC 2019, Video ELC
2019,
• Introduction of CIP Software Updates Working Group by Akihiro Suzuki, CIP Mini Summit 2019, Slides
CIP 2019,
• There is No Store For Self-Driving Car Parts by Stephen Segal and Matt Fornero (Cruise LLC), ELC
2020, Slides Cruise ELC 2020, Video Cruise
• Secure Boot and Over-the-Air Updates - That's simple, no ? by Jan Kiszka (Siemens AG), ELC 2020,
Slides Secure OTA ELC 2020, Video Secure OTA
• Diving into SWUpdate: adding new platform support with Yocto/OE! by Pierre-Jean Texier, LiveEmbedded
2020, Slides Diving into SWUpdate, Video Diving into SWUpdate
Useful references
• Boundary Devices, Using SWUpdate to upgrade your system
• Présentation de Software Update (French)
• Easy OS upgrades with SWUpdate
• SWUpdate for feature-rich IoT applications
• Implement swupdate - replacing opkg based updating, VictronEnergy
• Variscite, SWUpdate
• Updating Embedded Linux Devices: SWUpdate
• Approach to Software Update Management, Pelux
• SOTA System, Pelux
• Building a Linux system for the STM32MP1: remote firmware updates, Bootlin
CONTRIBUTING TO SWUPDATE
Contributions are welcome ! Please follow the following guideline for contributions.
Contribution Checklist
These are mostly general recommendations and are common practice in a lot of FOSS projects.
• use git to manage your changes [recomended]
• follow as much as possible kernel codestyle [recomended] Nevertheless, some rules are not so strict as
in kernel. The maximum line length can be extended over 80 chars if this increase code readability.
• add the required copyright header to each new file introduced [required]
•
add signed-off to all patches [required]
• to certify the "Developer's Certificate of Origin", see below
• check with your employer when not working on your own!
•
add version number for your patches if follow-up versions are requested [recomended]
• Add a "Change from Vx" description under the commit message to take track of the history of
the patch.
• It is suggested to use excellent "patman" tool to manage patches series. This is part of
U-Boot's project (tools/patman), but it can be used in other projects, too.
• check that your patches do not break build [required]
• There is a set of configuration files in the configs/ directory. Please run a build for all files in
the directory to ensure that SWUpdate is still buildable from configurations different as yours.
•
post patches to mailing list [required]
• use git format-patch to generate your patches.
• use git send-email if possible. This avoid corruptions due to the mailers
• add a prefix [meta-swupdate] if patches are intended to the Yocto's meta layer.
• send patches inline, do not append them
• no HTML emails!
• do not use github Pull Request. github facilities are not used for this project. The review is done in
a single place : the Mailing List. PR from github are ignored.
Patches are tracked by patchwork (see http://jk.ozlabs.org/projects/patchwork/). You can see the status
of your patches at http://patchwork.ozlabs.org/project/swupdate/list.
Developer's Certificate of Origin 1.1
When signing-off a patch for this project like this
Signed-off-by: Random J Developer <random@developer.example.org>
using your real name (no pseudonyms or anonymous contributions), you declare the following:
By making a contribution to this project, I certify that:
a. The contribution was created in whole or in part by me and I have the right to submit it under
the open source license indicated in the file; or
b. The contribution is based upon previous work that, to the best of my knowledge, is covered
under an appropriate open source license and I have the right under that license to submit that
work with modifications, whether created in whole or in part by me, under the same open source
license (unless I am permitted to submit under a different license), as indicated in the file;
or
c. The contribution was provided directly to me by some other person who certified (a), (b) or (c)
and I have not modified it.
d. I understand and agree that this project and the contribution are public and that a record of
the contribution (including all personal information I submit with it, including my sign-off)
is maintained indefinitely and may be redistributed consistent with this project or the open
source license(s) involved.
PROJECT'S ROAD-MAP
Please take into account that most of the items here are proposals. I get some ideas talking with
customers, some ideas are my own thoughts. There is no plan when these features will be implemented -
this depends if there will be contribution to the project in terms of patches or financial contributions
to develop a feature.
Thanks again to all companies that have supported my work up now and to everybody who has contributed to
the project, let me bring SWUpdate to the current status !
Main goal
First goal is to reach a quite big audience, making SWUpdate suitable for a large number of products.
This will help to build a community around the project itself.
Core features
Support for further compressors
SWUpdate supports image compressed with following formats: zlib, zstd. This is a compromise between
compression rate and speed to decompress the single artifact. To reduce bandwidth or for big images, a
stronger compressor could help. Adding a new compressor must be careful done because it changes the core
of handling an image.
Support for OpenWRT
OpenWRT is used on many routers and has its own way for updating that is not power-cut safe.
Selective downloading
Bandwidth can be saved not only via delta, but identifying which part of the SWu must be loaded and
skipping the rest. For example, SWUpdate can detect the versions for artifact before downloading them and
ask the servers to send just the relevant artifacts.
Software-Software compatibility
SWUpdate has from the early stage a hardware to software compatibility check. In case software is split
in several components (like OS and application), it is desirable to have a sort of software compatibility
check. For example, SWUpdate verifies if a component (like an application) is compatible with a runningOS
and reject the update in case of mismatch.
Parser
SWUpdate supports two parsers : libconfig and JSON. It would be nice if tools can be used to convert from
one format to the other one. Currently, due to some specialties in libconfig, a manual conversion is
still required.
Fetcher and interfaces
Downloader
The downloader is a one-shot command: when -d is set, SWUpdate loads the SWU from the provided URL. This
behavior is high requested and must be even supported in future, but another use case is to run the
downloader as daemon (like suricatta) and checks if a new SWU is available at the specified URL. It
should be as an alternative server for suricatta and this allows to control it via IPC
(enable/disable/trigger).
Tools and utilities
Self contained tool to generate Update Packages (SWU)
Generation of SWUs is fully supported inside OE via meta-swupdate, but there is no support at all with
other buildsystems (Buildroot, Debian). The user have a not preordered bunch of programs and scripts to
generate the SWU, and mostly they are not generic enough. It will be interesting to create a buildswu
tool, running on host system, that can create form a configuration a SWU. The tool must support all
features, that means it should be able to pack artfact, generate sw-description from templates, sign the
SWU, encrypt the artifact, etc.
Lua
• API between SWUpdate and Lua is poorly documented.
• Extend Lua to load modules at startup with functions that are globally visible and can be used by own
Lua scripts or by the embedded-script in sw-description.
• Store in SWUpdate's repo Lua libraries and common functions to be reused by projects.
Handlers:
New Handlers
Users develop own custom handlers - I just enforce and encourage everyone to send them and discuss how to
integrate custom handler in mainline.
Some ideas for new handlers:
• FPGA updater for FPGA with Flash
• Package handler to install packages (ipk, deb) Packages can be inserted into the SWU and the
atomicity is guaranteed by SWUpdate.
• Lua handlers should be added if possible to the project to show how to solve custom install.
Handlers installable as plugin at runtime
The project supports Lua as script language for pre- and postinstall script. It will be easy to add a way
for installing a handler at run-time written in Lua, allowing to expand SWUpdate to the cases not covered
in the design phase of a product.
Of course, this issue is related to the security features: it must be ensured that only verified handlers
can be added to the system to avoid that malware can get the control of the target.
Current release supports verified images. That means that a handler written in Lua could be now be part
of the compound image, because a unauthenticated handler cannot run.
Support for BTRFS snapshot
BTRFS supports subvolume and delta backup for volumes - supporting subvolumes is a way to move the delta
approach to filesystems, while SWUpdate should apply the deltas generated by BTRFS utilities.
Security
• add support for asymmetryc decryption
Support for evaluation boards
meta-swupdate-boards contains examples with evaluation boards. Currently, there are examples using
Beaglebone Black, Raspberri PI 3 and Wandboard. The repo is a community driven project: patches welcome.
Back-end support (suricatta mode)
Back-end: responsiveness for IPC
Suricatta is implemented as process that launches functions for the selected module. This means that the
IPC does not answer if Suricatta is doing something, specially if it is downloading and upgrading the
system. This can be enhanced adding a separate thread for IPC and of course all required synchronization
with the main modules.
Back-end: check before installing
In some cases (for example, where bandwidth is important), it is better to check if an update must be
installed instead of installing and performs checks later. If SWUpdate provides a way to inform a
checker if an update can be accepted before downloading, a download is only done when it is really
necessary.
Back-end: hawkBit Offline support
There are several discussions on hawkBit's ML about how to synchronize an offline update (done locally or
via the internal Web-server) with the hawkBit's server. Currently, hawkBit thinks to be the only one
deploying software. hawkBit DDI API should be extended, and afterwards changes must be implemented in
SWUpdate.
Back-end: support for generic down-loader
SWUpdate in down-loader mode works as one-shot: it simply try to download a SWU from a URL. For simple
applications, it could be moved into suricatta to detect if a new version is available before downloading
and installing.
Back-end: support for Mender
There was several discussion how to make a stronger collaboration between different update solution and a
proposal discussed previously is to use SWUpdate as client to upgrade from a Mender server, see BOF at
ELCE 2017
Support for multiple Servers simultaneously
Currently, suricatta's server backends are a mutually exclusive compile-time choice. There is no interest
to have multiple OTA at the same time. This feature won't be implemented and I will remove this from
roadmap if no interest will be waked up.
Test and Continuous Integration
The number of configurations and features in SWUpdate is steadily increasing and it becomes urgent to
find a way to test all incoming patch to fix regression issues. One step in this direction is the
support for Travis build - a set of configuration files is stored with the project and should help to
find fast breakages in the build. More in this direction must be done to perform test on targets. A
suitable test framework should be found. Scope is to have a "SWUpdate factory" where patches are fast
integrated and tested on real hardware.
Documentation
Documentation is a central point in SWUpdate - maintaining it up to date is a must in this project. Help
from any user fixing wrong sentence, bad english, adding missing topics is high appreciated.
DOCUMENTATION FOR PREVIOUS RELEASES
• 2021.04
• 2020.11
• 2020.04
• 2019.11
• 2019.04
• 2018.11
INDICES AND TABLES
• genindex
• modindex
• search
AUTHOR
Stefano Babic
COPYRIGHT
2013-2021, Stefano Babic
2021.11 Dec 10, 2021 SWUPDATE(1)