code:
One Platform driver to rule them all
====================================

In the previous section, we
exercised direct-device access from user-level components. In Genode systems
beyond such toy scenarios, however, it would be irresponsible to follow the
path of allowing arbitrary drivers to access any device willy-nilly. Our call
for discipline and rigidity is answered by the _(rising drum roll)_ platform
driver.

Let's recap the scenario of the previous article.

[tikz img/pio_device_core_access]

Our user-level test program created connections to core's services for
accessing memory-mapped I/O registers and receiving notifications for device
interrupts. The choice of physical register addresses and the GIC interrupt
number was up to the test program. So in principle, our program could access
any part of the platform by just requesting it. Hence, the mere fact that the
driver code has the form of a regular user-level component does not buy us a
security gain per se. To benefit from Genode's architecture, we need to
rigidly limit the reach of each individual driver to the smallest-possible set
of device resources.

Remember, even though we want to use drivers, we distrust them. Consequently,
besides enforcing access control, we generally don't want to expose
system-global information to such untrusted components, asking questions
like: Does a driver even need to know the physical address of a memory-mapped
I/O register? Does it need to know the GIC interrupt number of the device
it is driving? The perhaps surprising answer is that - no! - many drivers can
happily do their job without any knowledge about these technicalities. All a
driver needs to know is _how to speak_ to a device of a certain type, not
where a particular instance of a device is located and how it is wired up.
This principled approach leads to a clear-cut separation of driver logic from
parametrization.

Platform driver
~~~~~~~~~~~~~~~

To separate the concerns of parametrization and access control from the
device drivers, Genode employs the so-called _platform driver_ as a level
of indirection between core's services and the individual drivers.
The platform driver has a global view over all device resources and follows
a configured policy to partition those resources between its clients.
Each session to the platform service can comprise potentially multiple devices,
depending on the configured policy.

[tikz img/pio_device_platform_access]

To integrate the notion the platform driver into our existing scenario of
accessing general-purpose I/O pins via a to-be-developed PIO device driver,
it is best to first sketch a run script that mirrors the picture above.
We have to find a suitable name and location within our source tree for our
designated driver component.

The naming of driver components within Genode follows the pattern
! <device-or-platform>_<driver-type>_drv
For example, the 'imx8_fb_drv' is a
framebuffer (fb) driver for the 'i.MX8' SoC. In our case of a PIO driver for
the Allwinner A64 SoC, the name 'a64_pio_drv' is a sensible choice.

Even though there is no strict convention of the directory where a driver is
located, drivers usually reside in a subdirectory of _src/drivers/_ that
corresponds to the primary purpose of the driver. E.g., framebuffer drivers
are located at _src/drivers/framebuffer/_. Our designated driver drives
GPIO pins. So I settled on placing it at _src/driver/pin/a64/_ within the
[https://github.com/genodelabs/genode-allwinner - genode-allwinner repository].
With the complicated naming-things-topics behind us, let's turn our attention
to the run script, appropriately named _a64_pio_drv.run_.

# Building the components including the platform driver along with our new
  custom driver.

  ! build { core init drivers/platform drivers/pin/a64 }

# Creating a boot directory with the configuration of the init component.

  ! create_boot_directory
  !
  ! install_config {
  !   <config>
  !     <parent-provides>
  !       <service name="LOG"/>
  !       <service name="PD"/>
  !       <service name="CPU"/>
  !       <service name="ROM"/>
  !       <service name="IO_MEM"/>
  !       <service name="IRQ"/>
  !     </parent-provides>
  !
  !     <default caps="100"/>
  !     ...
  !
  !   </config>
  ! }

  At this time the scenario consists of only two components, namely the
  platform driver and our PIO driver. The <start> node for the platform
  driver is particularly interesting.

  ! <start name="platform_drv">
  !   <resource name="RAM" quantum="1M"/>
  !   <provides> <service name="Platform"/> </provides>
  !   <route> <any-service> <parent/> </any-service> </route>
  !   <config devices_rom="config">
  !     <device name="pio">
  !       <io_mem address="0x1c20800" size="0x400"/>
  !       <irq number="43"/>
  !     </device>
  !     <policy label="a64_pio_drv -> ">
  !       <device name="pio"/>
  !     </policy>
  !   </config>
  ! </start>

  The routing rule states that the platform driver is permitted to
  open arbitrary sessions to core, including IRQ and IO_MEM. There
  are no restrictions.

  The <provides> declaration states that this component offers a "Platform"
  service.

  The <config> node tells the platform driver to determine the information
  about the present devices from the ROM module labeled "config", which allows
  us to specify both the device information and the policy inside the same
  <config> node.
  Without the 'devices_rom="config"' attribute, the platform driver would
  request a "devices" ROM session instead.
  We declare the existence of a single "pio" device that features one
  memory-mapped I/O range and the GIC interrupt 43.
  You may recall those values from
  Section [Device access from the user level].

  The <config> node also tells the platform driver about the access-control
  policy applied to clients that connect to the platform service.
  In the case at hand, we dictate that a client labeled as "a64_pio_drv -> "
  gets access to the "pio" device. You may wonder about the trailing
  '->' part of the label. The part before the arrow is hard-wired by the
  parent of the 'a64_pio_drv' and thereby reflects the identity of the
  client in a way that cannot be forged by the client. The part after the
  arrow is controlled by the client. The client can use this part to provide
  hints about the purpose of the session. So a client that creates multiple
  sessions to the same server can express the intention behind those sessions.
  In our case, this client-controlled part remains unused.

  The <start> node for our designated PIO driver looks as follows.

  ! <start name="a64_pio_drv">
  !   <resource name="RAM" quantum="1M"/>
  !   <route>
  !     <service name="ROM"> <parent/> </service>
  !     <service name="CPU"> <parent/> </service>
  !     <service name="PD">  <parent/> </service>
  !     <service name="LOG"> <parent/> </service>
  !     <service name="Platform"> <child name="platform_drv"/> </service>
  !   </route>
  !   <config/>
  ! </start>

  Let me bring the <route> node to your attention. In contrast to the
  wildcard rule <any-service> used for the platform driver, the
  rules for the PIO driver state explicit permissions. From these rules,
  we can immediately infer the potential reach of the component.

  The driver is permitted to connect to the platform driver. That's
  what we want.
  It is also able to use core's ROM, CPU, PD, and LOG services, which provide
  the fundamental ingredients for executing the program.

  Most importantly, no other service is reachable. In particular, the
  direct use of core's IRQ and IO_MEM is out of question. The only way
  to access a device is the platform driver that imposes its policy.

# Building the boot image containing the ELF binaries for the components and
  executing the scenario.

  ! build_boot_image { core ld.lib.so init platform_drv a64_pio_drv }
  !
  ! run_genode_until forever

For reference you can find a commit for this step
[https://github.com/genodelabs/genode-allwinner/commit/febf53b8ad6819757eeeaf20eeb1b634ade0b668 - here].

To exercise the interplay between the designated PIO driver with the platform
driver, it is a good idea to transplant the
[https://github.com/genodelabs/genode-allwinner/blob/pio/src/test/pin_state/main.cc - test/pin_state]
program of the previous section from the use of core's services to the use of
the platform driver. The following snippet highlights the important changes.

! #include <platform_session/device.h>
! ...
!
! struct Pio_driver::Main
! {
!    Env &_env;
!
!    Platform::Connection _platform { _env };
!
!    Platform::Device _device { _platform };
!
!    struct Pio : Platform::Device::Mmio
!    {
!       struct Pb_cfg0 : Register<0x24, 32>
!       {
!          ...
!       };
!
!       ...
!
!       Pio(Platform::Device &device) : Mmio(device)
!       {
!          ...
!       }
!    };
!    ...
!    Pio _pio { _device };
!    ...
! };

* The API for using the platform driver becomes available via
  ! #include <platform_session/device.h>

* A session to the platform service is established by creating an instance of
  a 'Platform::Connection' passing the Genode environment as argument.
  ! Platform::Connection _platform { _env };
  By passing the '_env', we explicitly give our consent that
  the 'Platform::Connection' can have global side effects such as the
  communication with the outside world.

* Access to one particular device of the platform session can be obtained by
  creating an instance of a 'Platform::Device'.
  ! Platform::Device _device { _platform };
  When called with only the 'Platform::Connection' as argument, the device
  refers to the first - and in our case only - device of the platform
  session. In cases where multiple devices appear grouped in one platform
  session, a second argument allows for the selection of the device.

* The memory-mapped registers of the PIO device are represented by a
  custom 'Pio' type that inherits the 'Platform::Device::Mmio' type.
  ! struct Pio : Platform::Device::Mmio
  The constructor takes a 'Platform::Device' and an optional
  index as arguments.
  ! Pio _pio { _device };
  If no index is provided, it refers to the first <io_mem> resources as
  declared in the platform-driver's configuration.

* Thanks to the inherited 'Platform::Device::Mmio' type, the individual
  registers can be accessed in the same way as we did in the previous
  article.

Note that in contrast to the previous examples, the code is void of physical
addresses. Now, those addresses are the business of the platform driver only.

Session interfaces for accessing pins
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

We ultimately want to allow multiple programs to interact with different GPIO
pins. So our PIO driver must evolve into a server component that allows
clients to interact with pins. Analogously to how the platform driver
safeguards the access to device resources by different - mutually
distrusting - device drivers, the PIO driver's job will be the safeguarding of
GPIO pins.

Traditionally, Genode features the "Gpio" session interface
for this purpose. This interface allows a client to access an individual pin.
Once assigned to a pin, the session grants the client the full responsibility
for the pin. In particular the direction of the I/O pin is laid into the hands
of the client.
We later realized that the wiring and thereby the direction of a pin is
ultimately a board-level decision. Wrongly operating an input pin in output
mode can easily result in a short-circuit. Therefore, the client of an
individual pin should better not be burdened with the responsibility to
control the pin direction or pull resistors. To address this concern, it is
best to split the roles of GPIO pins into clear-cut session interfaces. Those
roles are:

# The sensing of the state of a GPIO pin, e.g., detecting whether a button
  is pressed or not: operating a pin as an input signal. This role is now
  covered by the "Pin_state" session interface with the single RPC function

  ! bool state() const;

  By calling this function, the client can request the state of the pin.
  That's it.

# Controlling the signal level of a pin: operating a pin as an output signal.
  This role is now addressed by the "Pin_control" session interface that
  provides an interface of only one rather unsurprising RPC function

  ! void state(bool);

# Receiving a notification of a change of the signal level of a GPIO pin:
  operating a pin as an interrupt source. This role can be represented by
  Genode's existing IRQ session interface - the same interface as provided
  by Genode's core for GIC interrupts.

PIO device driver
~~~~~~~~~~~~~~~~~

The A64 PIO driver implements the three session interfaces outlined above.
It resides at
[https://github.com/genodelabs/genode-allwinner/tree/master/src/drivers/pin/a64 - src/drivers/pin/a64]
within the genode-allwinner repository. The accompanied README covers the
details about its use and configuration.

Similar to how the platform-driver configuration declares device resources
like IRQs and memory-mapped I/O regions, the PIO driver's configuration
declares pins.

! <config>
!   <out name="led"    bank="B" index="2" default="on"/>
!   <in  name="button" bank="H" index="8" pull="up" irq="edges"/>
!   ...
! </config>

Here we see the declaration of an output pin named "led" and an input pin
"button". The 'bank' and 'index' denote the physical location of the pin at
the SoC. Further pin parameters are expressed as attributes. For example,
in the absence of a "Pin_control" client for the "led", the led is set to
state "on" according to the 'default' attribute.

Since the A64 PIO device subsumes GPIO functionality as well as I/O MUX
functionality, the driver also offers the selection of pin functions
beyond <in> and <out>.

For reference, the commit for the driver implementation can be found
[https://github.com/genodelabs/genode-allwinner/commit/11ec1ae2b8fe39de4f8059eb1e28c9a1bcb263a5 - here].
A few technical tidbits and caveats I encountered during its development are
worth sharing:

:Device-register interaction:

  The actual interplay of the driver with the hardware registers is completely
  covered by the code found in
  [https://github.com/genodelabs/genode-allwinner/blob/master/src/drivers/pin/a64/pio.h - pio.h].
  Genode's 'Mmio' framework API makes this code strikingly simple, almost
  self-describing. There is no manual bit fiddling to be found, thanks to the
  wonderful 'Register_array'.

:Code organization:

  I deliberately split the code into a boring and an interesting part.

  The boring part models the SoC-specific terminology as a bunch of
  corresponding C++ types. In
  [https://github.com/genodelabs/genode-allwinner/blob/master/src/drivers/pin/a64/types.h - types.h]
  one can find types for any term we deal with - however boring it is. Most of
  these types have a local 'Value' type that is as rigid as possible. E.g.,
  the 'Pull' type contains an enum with the values DISABLE, UP, and DOWN as
  the 'Value' type. The degrees of freedom mirror the information found in the
  SoC manual. Each type is equipped with a class function 'from_xml' that
  encodes the knowledge of how values of the type relate to XML
  representation. Some of the types go as far as deliberately disabling any
  means to construct instances of the type without using 'from_xml' by
  deleting the default constructor. This way, program-global invariants of the
  type can be enforced at a single place. The boring code makes up the
  biggest part of the driver. This is good because with "boring" I mean
  simple and easy to assess for correctness.

  The interesting part lives in the
  [https://github.com/genodelabs/genode-allwinner/blob/master/src/drivers/pin/a64/main.cc - main.cc]
  file where all the strings are coming together.

:Stumbling blocks:

  Quite a bit of time went wasted because of silly mistakes of mine.

  Sometimes I went too hastily over the SoC documentation without double
  checking.
  In particular, I allowed myself be become misled by a table in the SoC
  [https://linux-sunxi.org/images/b/b4/Allwinner_A64_User_Manual_V1.1.pdf - documentation]
  at page 376 where I wrongly identified patterns that do not exist.
  In one part of the table, the symbol 'n' _seemingly_ refers to a zero-indexed
  value corresponding to GPIO banks in alphabetic order. Some lines below
  (at the Pn_INT_*) definitions, the 'n' refers only to a few banks, namely
  'B', 'G', 'H'. I wrongly assumed the same linearity of register layouts
  to apply for both parts of the table. In reality, 'n' must just be read
  as a shorthand of "some value". Note to myself: Double check my assumptions
  each time I'm overconfident that _I got it_.

  Because of my prolonged intimacy with pin 2 at bank B, I lost sight
  of the other banks, in particular the fact that each bank is wired up
  with a distinct GIC interrupt. Once I tried to receive interrupts for
  pin 8 at bank H, I first struggled to get the interrupt mechanism to work,
  until I realized that bank H interrupts end up at GIC IRQ 53, not 43.
  In fact, the "pio" device in the platform driver configuration now looks
  like this:

  ! <device name="pio">
  !   <io_mem address="0x1c20800" size="0x400"/>
  !   <irq number="43"/> <!-- Port B -->
  !   <irq number="49"/> <!-- Port G -->
  !   <irq number="53"/> <!-- Port H -->
  ! </device>

:Implementation of dynamic re-configurability:

  For maintaining the internal data model of the pin-state configuration,
  the driver employs Genode's 'List_model' utility. By using this utility,
  the creation and updating of such a data model from XML data becomes very
  simple. It comes down to providing hook functions for creating, destroying,
  matching, and updating model items.

  It is worth noting that the driver configuration is not static but it can be
  dynamically adjusted during runtime. So in principle, we can attain a
  blinking LED by the sole means of re-configuring the driver.

Dynamic configuration testing
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

_Wait what!?_

If blinking an LED by reconfiguring the PIO driver sounds as irresistible to
you as to me, follow me for a moment.

For test-driving the dynamic configuration handling of components like the
A64 PIO driver, there exists a handy utility component called _dynamic_rom_,
which provides a ROM service that feeds the client with different version
of ROM content over time. Here is how a <start> node of a dynamic_rom server
looks like.

! <start name="dynamic_rom">
!   <resource name="RAM" quantum="1M"/>
!   <provides> <service name="ROM"/> </provides>
!   <route>
!     <service name="Timer"> <child name="timer"/> </service>
!     <any-service> <parent/> </any-service>
!   </route>
!   <config>
!     <rom name="config">
!       <inline description="LED off">
!         <config>
!           <out name="led" bank="B" index="2" default="off"/>
!         </config>
!       </inline>
!       <sleep milliseconds="1000"/>
!       <inline description="LED on">
!         <config>
!           <out name="led" bank="B" index="2" default="on"/>
!         </config>
!       </inline>
!       <sleep milliseconds="1000"/>
!     </rom>
!   </config>
! </start>

The <rom> node within its configuration defines a PIO <config>. After 1
second, the <config> is replaced with a new version where the 'default'
attribute of the <out> pin is toggled. After one more second, the
first <config> becomes active again.

The remaining piece of the puzzle is feeding the ROM provided by the
dynamic_rom server as config ROM to the a64_pio_drv driver. This can
be achieved by the following routing rule in the <start> node of the
a64_pio_drv component.

! <start name="a64_pio_drv">
!   ...
!   <route>
!     <service name="ROM" label="config">
!       <child name="dynamic_rom"/> </service>
!     ...
!   </route>
! </start>

By wiring up the driver configuration to the dynamic_rom we can see the
LED happily blinking even without any "Pio_control" client present.

The dynamic_rom server is handy utility in many testing situations. Besides
issuing time-triggered configuration updates, it can be used to mock
system-state changes that are normally driven by real components or sensory
input that is difficult to fabricate manually.

Cascaded authorities
~~~~~~~~~~~~~~~~~~~~

Similarly to the configuration concept of the platform driver, the
pin-declarations of the PIO driver configuration are followed by a policy part
of the configuration that associates clients with pins.

! <config>
!   ...
!   <policy label_prefix="pin_event ->" pin="button"/>
!   <policy label_prefix="pin_pulse ->" pin="led"/>
! </config>

This configuration assigns the "led" pin to the program "pin_pulse", and the
"button" to the program "pin_event". Note that - like the pin declarations -
these assignments can be dynamically changed by the means of configuration
updates.

The "pin_pulse" component uses the "Pin_control" session to drive the digital
signal of an LED with a pulse-width-modulated pattern.
Effectively, the program toggles the LED 200 times per second
while adjusting the relation of the durations of the low and high signal
levels over time. The result is a nice breathing effect.

The "pin_event" component watches the state of a pin using a combination of an
IRQ session and a "Pin_state" session. Each time when the signal changes, an
IRQ is triggered, which prompts the component to obtain the pin state by
calling 'Pin_state::state'.

The component composition of the scenario looks as follows.

[tikz img/pio_device_driver]

The higher up we get, the less influential the components become. Whereas the
kernel has ultimate authority over everything, the reach of the pin-pulse
component is limited to the control of the output signal of a single GPIO pin
only.

As indicated by the coloring of the components, policy and mechanisms are
nicely separated. The pin-pulse component does not even know which pin it is
driving. It merely contains the logic needed to modulate the PWM pattern on a
digital output signal. At the bottom end of the picture, the core / kernel
component does have no interpretation of physical device addresses or IRQ
numbers. It is indifferent regarding GIC IRQ number 43 and free from policy.
The policy is encapsulated in the forms of the platform and PIO driver
components, each respectively applying a policy at a _useful_ level of
abstraction.

Integrated test scenario
~~~~~~~~~~~~~~~~~~~~~~~~

The final version of the
[https://github.com/genodelabs/genode-allwinner/blob/master/run/a64_pio_drv.run - a64_pio_drv.run]
script contains the combinations of the various fragments discussed above.
It test-drives the dynamic re-configurability of the PIO driver along with
the "Pin_state", "Pin_control", and IRQ session interfaces.

For the test of the GPIO input, I selected pin 8 of bank H. This pin is
accessible at the Euler connector at pin 10 of the Pine-A64-LTS board.
The board has a button labeled "power" just besides the reset button.
Although this "power" button is connected to the AXP803 power management
chip, it doesn't appear to have any effect when pressed while the board is
on. According to the board
[https://files.pine64.org/doc/SOPINE-A64/PINE%20A64-TLS-20180130.pdf - schematics],
the button happens to be also wired to pin 5 of the smaller 10-pin Euler
header. I figured that I can thereby feed the button state to the GPIO pin
H8 be connecting pin 5 of the small Euler header with pin 10 of the large
Euler header. The signal is active-low, which can be explained by the
schematics that shows that the button pulls the PWR_ON signal to ground when
pressed. Long story short, with this wiring in place, the power button can
be observed via GPIO H8. The GPIO pin B2 can be connected to an LED as
we did for the test/pin_control example described in the previous article.


code:
Porting Genode to a new SoC
###########################

We get repeatedly asked about the principle steps and costs needed to enable
Genode - and in particular Sculpt OS [1] - for various ARM-based hardware
platforms. The variety of SoCs is too great to give a general answer. However,
drawing from our experience with the porting Genode to several ARM-based
platforms such as NXP's i.MX8, this chapter provides a practical guide for the
steps of such a porting endeavour.

[1] [https://genode.org/download/sculpt]

The guide is based on an article series at [https://genodians.org]. It is
written in an informal style from the perspective of a developer carrying
out the work, taking a specific board - namely the Pine-A64-LTS single
board computer - as a playground. The code discussed throughout this
chapter is available at the following public Git repository.

:Git repository of the Allwinner board support:

  [https://github.com/nfeske/genode-allwinner]

The guide is not carved in stone. It will be progressively enhanced with
further information - e.g., details about various classes of drivers - over
time. Should you happen find important topics missing or spot mistakes or have
suggestions for improving the material, please don't hesitate to send your
feedback to _norman.feske@genode-labs.com_.

Goals
-----

Our goal would be to get the bare-bones Sculpt system up and running on an ARM
SoC not yet supported by Genode. This bare-bones Sculpt system entails

* The principal ability for the user to interact with the system via
  a graphical user interface,

* Support for installing and deploying the existing arsenal of Genode
  components from regular packages,

* The ability to store information persistently on the device, and

* Network connectivity.

Thanks to Sculpt's built-in ability to integrate 3rd-party components -
including functionality that is traditionally attributed to the core of the
operating system - into the system in the form of packages, this bare-bones
system enables a great variety of usage scenarios.

Non-goals
---------

That said, the following features remain beyond the scope of this document
because they are either too vendor-specific to be described in a general
fashion or can be realized in the form of supplemental components.

* Hardware-accelerated graphics,

* Audio,

* Power management,

* Mobile data communication,

* Secure boot.

Working steps
-------------

The work of enabling Genode for a new SoC requires the following steps in the
described order. To give an indicator of the effort to be expected, each step
is accompanied with a rough estimation.

# Preparing the development testbed

  Before the actual development work can start, a few preparations are needed
  or at least recommended.

  One of our team members typically spends up to *one month* for this step.

  * Building and running a working Linux-based OS on the target board as
    reference, following the instructions of the vendor

  * Exploration and configuration of the target's boot mechanism

  * Creation of a test-control loop for triggering the booting the target board
    via the run tool, serving the boot image over the local network, and
    obtaining the log output.

  * Familiarization with the available board and SoC-vendor documentation
    and the Vendor-specific subsystems in the vendor's Linux kernel

  * Studying the device tree, correlating it with information gathered
    from the documentation.

# Code skeleton for a new SoC

  Given the impressions gathered during the preparatory step, we take one
  of the SoCs that are already supported by Genode as reference. One should
  select the SoC with the most similarities such as the same ARM core revision
  or the same interrupt controller. The goal of this step is an almost
  empty skeleton code of Genode that gives us a little life sign when booted
  on the real hardware.

  It does not take a seasoned Genode developer longer than *two weeks* to
  complete this step. However, for a developer with no prior experience with
  Genode's code base, an *additional* effort of *two weeks* for the required
  familiarization should be planned for.

  * Mirroring the files of another SoC but with empty bodies,
    (describing roles of the individual files)
  * Creating a bare-bone base-hw kernel ELF image
  * Booting the custom image on the target hardware
  * Serial output driver

# Basic kernel functionality

  The goal of this step is getting the most basic Genode system scenario
  to run on the new SoC. This scenario comprises three components, namely
  the Genode core component (including the kernel), the init component,
  and a test program that produces some log output.

  On this way, one has to overcome the challenges of initializing the kernel,
  enabling the MMU, and exercising the kernel's IPC and context-switching
  mechanism. Assuming that the new SoC has the same architecture revision
  as the ones already supported by Genode, this step should take no longer
  than *two weeks*.

  * Enabling the MMU
  * Enabling caches
  * Memory layout parameters
  * Entering and returning from the user land (IPC, context switches)
  * Running Genode's log scenario

# Support for user-level device drivers

  With the principal ability of running multiple user-level components,
  it is time to enable preemptive scheduling and the kernel mechanisms
  needed by user-level device drivers. Assuming the new SoC uses standard
  ARM building blocks like the core-local timer and the GIC interrupt
  controller as readily supported by Genode, this step does not entail
  much risks and should be completed within *a week*.

  However, should the SoC deviate from the beaten track of standard ARM
  building blocks, e.g., using a custom interrupt controller, the step may
  additionally require the development of an in-kernel driver for such a
  device. Genode provides several existing drivers that can be taken as a blue
  print. Depending of the quirkiness of the device, the development can take
  one or two weeks. Fortunately, vendor-specific timers and interrupt
  controllers are largely a problem of the past.

  * Enabling the in-kernel interrupt controller driver
  * Enabling in-kernel timer driver
  * Definition of I/O resources
  * IOMUX configuration (board-specific)

  Once the principal support for user-level device drivers is in place,
  the development work can be tackled by multiple developers in parallel.

# Network driver

  We usually plan to spend about *one month* for enabling a network driver
  for Genode. Depending on the complexity of the network controller,
  the driver may be ported from the Linux kernel, from the U-Boot boot
  loader, or written from scratch.

# SD-card driver

  For driving SD-cards, we usually extend Genode's custom SD-card driver
  with SoC-specific support, which takes usually *two weeks*. One should
  be prepared for device-specific peculiarities though. In some cases,
  in the presence of flaky hardware, it took us up to 3 weeks more to reach
  a stable and performant state.

# Framebuffer driver

  In the past, we used to develop framebuffer drivers from scratch. But
  nowadays, we prefer to reuse the vendor-provided driver code from the Linux
  kernel to attain feature parity with Linux. That said, depending on the
  driver, such porting work still requires substantial manual labour because
  the driver often does not only drive one device but multiple (such as
  power-gating via additional I2C-connected controllers, or a dedicated HDMI
  chip). As an indicator for the expected effort, the i.MX framebuffer driver
  took us *two months* to bring to life.

# USB host-controller driver

  Genode's USB host-controller driver is based on the Linux USB driver.
  Adding supplemental support for a new SoC should generally possible within
  *one month*. With the USB host-controller driver in place, the actual USB
  device drivers (e.g., for HID and storage) should work out of the box.

  As a note of caution, in rare cases, in particular for the Raspberry Pi, the
  USB host controller driver can become an almost infinite time sink though.

# Multi-processor support

  Real-world workloads demand multi-processor support. In theory, this should
  generally be covered well by Genode's ARM support as long as the SoC stays
  close to ARM's reference design. However, the bring-up of secondary CPUs,
  inter-processor interrupts, and the maintenance of TLB/cache coherence still
  poses risks because those topics may involve upcalls to vendor-specific
  firmware or may depend on the unexpected vendor-specific boot-time
  configuration (like the surprise of one CPU core left configured with a
  different byte order). To stay on the safe side, one should plan *one month*
  for the potential troubleshooting around these areas.

# Sculpt OS integration

  With the four peripheral drivers in place, Sculpt's demands on the platform's
  feature set is satisfied. The remaining task is the integration of those
  drivers into Sculpt, which should be doable in no more than *two weeks*.

  * Drivers subsystem definition
  * Sculpt-manager tweaks
  * Configuration

Summary
-------

Based on the steps outlined above, the effort seems to be modest but - given a
healthy dose of enthusiasm - quite doable for an individual or a small team.
The biggest risk is the incomplete or lacking documentation for most ARM SoCs.

Granted, such a bare-bone system is still a far cry from a sophisticated
product like a smart phone, which features plenty of additional peripheral
devices, an aggressive power-management regime, GPU-accelerated rendering, or
Bluetooth. But once a bare-bones Sculpt system is ready to run, further device
drivers can be developed as regular components independent from each other,
which is the beauty of a component-based operating system like Sculpt OS.

repo_name: Fhiss/genode


code:
Preparatory steps
=================

After getting a rough overview of undertaking the port of Sculpt OS to another
SoC in the previous section, let us take a closer look at the first step -
taking technical and non-technical preparations.

For the preparatory work, I recommend taking one month of time. This may sound
excessive but there are good reasons. First, Genode's tooling deviates from
the beaten tracks known from commodity operating systems. In particular
Genode's run tool is quite unique and powerful. But it comes at the price of a
learning curve. The learning should not be done as a side activity but
requires the focus of the developer. Second, the initial steps of enabling a
new hardware tend to be fiddly. Especially when it comes to compiling and
testing out a vendor-customized boot loader and Linux kernel from source,
this can become a walk on muddy ground. Without patience or with time
pressure, it can get messy and exhausting. Third, contemplating about
non-technical preparatory aspects like licensing deserves some nights to sleep
over it.

Licensing considerations
~~~~~~~~~~~~~~~~~~~~~~~~

I see your raised eyebrows. Why bother with software licensing at this point?
To pursue the upcoming steps with as little friction as
possible, make up your mind about *your objectives* behind pursuing the porting work.
The licensing of your code should follow from that. From the chosen license,
in turn, follows the way of how to interact with the community. Let me
illustrate this point with three example scenarios:

:No strings attached:

  Open-source driver code authored by hardware vendors is often published under
  a permissive license to make the code broadly usable across projects with
  different open-source and proprietary licenses. Even for code contributed to
  the GPL-licensed Linux kernel, some vendors like Intel provide
  their contributions under the terms of the permissive MIT or BSD licenses,
  and thereby allow anyone to incorporate such code into other operating systems
  without licensing constraints. Usually such code is a clean-room
  implementation developed in-house at the vendor without incorporating
  3rd-party code. This approach is preferable whenever the objective is
  the *highest possible adoption* of the code.

:Submitting code upstream to the Genode project:

  A second possible objective may be the integration of your work upstream
  into the official Genode project to make the new SoC platform
  straight-forward to use for the Genode community and to benefit from
  the *ongoing maintenance* of the code *by Genode Labs*. However, with this ambition
  in mind, you need to ensure that you and your employer agree with the
  process of [https://genode.org/community/contributions - contributing] and
  in particular with the terms of the Genode contributor's agreement
  ([https://genode.org/community/gca.pdf - PDF]), which grants Genode Labs
  the right to offer Genode - including your code - under both open-source
  and commercial licensing terms.

:Pursuing a dual-licensing business:

  At the other extreme, your objective may be offering the results of your
  work as a *commercial product*, following a dual-licensing business model.
  In this case, you may consider publishing the code under the most
  restrictive copyleft license possible, along with the option for a
  commercial license.
  Or you may even go as far as considering the
  [https://genode.org/documentation/articles/component_public_license - Genode Component Public License] (CPL).
  This route should be considered only when planning a *long-term commitment* in
  actively productising and supporting your code. Note that the GCPL is no
  win for the open-source community beyond Genode.

The path taken has far-reaching ramifications. The ability to incorporate
3rd-party code into your work. The visibility of your work within the Genode
community. The selection of a suitable place for hosting your code.
Community spirit. Or the viability of contributions by others to your code.

The decision may be taken for different components individually. For example,
when taking the Linux USB stack as the basis for a USB host-controller
driver component, this component naturally inherits Linux' GPLv2 license.
At the same time, your custom in-kernel timer driver might fit best
into the upstream Genode project.

In our experience, taking and openly communicating licensing decisions up
front before starting actual development work reduces possible friction -
especially if a legal department is involved - and avoids wrong expectations.

Selecting a suitable SoC
~~~~~~~~~~~~~~~~~~~~~~~~

The question of which particular SoC to select as the basis for your work is
of course closely related with the same objectives as discussed
above. You may consider the following points:

* Costs of the chip and the devices featuring the chip. E.g., if you primarily
  intend to accommodate hobbyists, a low-end device might be preferable.
  But there are other arguments:

* Availability of accessible hardware featuring the SoC. Many SoCs are available
  only in large volumes and thereby end up in consumer devices only. More
  often than not, such consumer devices are completely locked down, rendering
  the attempt to install a custom operating system moot.

  With _accessible_ hardware, I'm also referring to the availability of
  development boards that mirror the architecture of a consumer device but
  with additional connectors for obtaining serial output, network
  connectivity, and possibly JTAG.

* Availability and quality of technical documentation. Even for many
  SoCs popular in the Linux community - think of the Raspberry Pi
  devices - public documentation is sparse or of questionable quality.
  If you find a "reference manual" of only a few hundred pages online,
  possibly imprinted with the term "CONFIDENTIAL", it's probably better to
  stay away from this chip. A modern SoC has usually more than 4000 pages of
  documentation. When browsing through it, look out for prose and
  architectural diagrams. Some "reference manuals" are merely disguised
  register listings, which are not very insightful.

* Support by the official Linux kernel. Even though most ARM devices
  run Linux, many vendors do not even attempt to contribute vendor-specific
  code upstream to the Linux project. Should the official Linux kernel features
  support for a particular SoC, this is a good sign for the maturity of the
  open-source drivers. In contrary, if only a certain whacky vendor kernel
  is known to work well with the SoC, it's probably best to shy away.

* Presence of hardware-based I/O protection (System-MMU). To fully
  leverage the advantages of Genode's architecture, the sandboxing of
  device drivers is important. Otherwise, all device drivers must be
  considered trusted.

  When we originally embraced the i.MX8M SoC, we silently assumed that every
  modern 64-bit SoC should feature a System-MMU in our modern times. We
  eventually learned that this is actually not the case for the i.MX8M.

  If different variants of one SoC with and without System-MMU are
  available, make sure to pick the variant that includes this feature.

Start by taking the known-good path
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Even though you may be eager with bringing Genode to the new device, let us
first exercise the device with its known-to work software stack.

# Usually, development boards come with a Linux-based system pre-installed.
  Try it out. Test the functioning of all hardware connectors that are
  important to you.

# Chase down the source code of the exact Linux kernel that is pre-installed
  on your board. In most cases, this so-called _vendor kernel_ is a customized
  version of Linux, with the source code provided at a vendor-specific
  place. Download it. Follow the vendor-provided instructions to build it
  from source. Boot your custom built Linux kernel on your device.

  This kernel will serve us as a working reference later. It allows us
  to cross-correlate problems between Genode and Linux, obtain traces of
  Linux device drivers, or to get hold of system-register states initialized
  by the Linux kernel to a working state.

# Study the device tree of the working Linux kernel and correlate this
  information with the documentation. This helps to form a mental picture
  of the hardware and to identify possible risks (indicated by your level
  of confusion) early on.

  ...slowly leaving the known-good path...

# Now that you are familiar with the vendor kernel, let's cross fingers
  and hope that the vanilla Linux kernel works just as well. Download the
  vanilla Linux kernel and look out for the support for your SoC. In the
  worst case, you won't find any. In the best case, the vanilla kernel
  works out of the box. In case the vanilla kernel works well, better use
  this one a reference for your further work.

Setting up an efficient development workflow
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

For the few test drives taken until this point, juggling SD-cards is probably fine.
But down the road, you will need to boot your device with custom system image
hundreds of times. Take the time for setting up a convenient test-control loop
for your device to make this work enjoyable.

Explore Genode's run tool
-------------------------

Read Section 5.4 "System integration and automated testing"
of the Genode Foundations book as found at [https://genode.org].

Try out the various options with an already supported
platform.
Browse the files at
[https://github.com/genodelabs/genode/tree/master/tool/run - tool/run/]
to learn about the
various backend modules and options. E.g., look at
[https://github.com/genodelabs/genode/tree/master/tool/run/image/uboot - tool/run/image/uboot]
to demystify the creation of uImage files by Genode.

Run and test the U-Boot loader on your device
---------------------------------------------

U-Boot is the de-facto standard of booting embedded ARM boards today.
We primarily use U-Boot for its ability to fetch a system image over the network.
There is a good chance that your board comes equipped with U-Boot already.
If not, investigate the option to chain-load U-Boot from your board's boot
loader.

Once you got U-Boot to work, continue with reproducing the U-Boot binary from
source. This may become handy for investigating device-driver issues later on
(e.g., taking U-Boot's IOMUX or power or clock configuration as reference,
peeking device states at boot time). Consider extending Genode's
[https://github.com/genodelabs/genode/blob/master/tool/create_uboot - tool/create_uboot]
utility, thereby documenting the steps for reproducing the U-Boot version
for your particular board from source.

Create a working test-control loop
----------------------------------

The goal of this step is to reach a state where you can type only one command
like following from the Genode build directory to trigger a complete
build-test cycle.

! make run/log KERNEL=hw BOARD=<your-board>

The build-test cycle entails:

# Compiling the source code of Genode components,
# Applying a system configuration,
# Assembling a system image,
# Making the system image available over TFTP,
# Power-cycling the board,
# Letting the board fetch the system image and start it, and
# Getting the serial output of the board right in your terminal.

To reach this level of convenience, the following topics must be addressed:

:Network boot:

  * Set up TFTP server on you development machine
  * Test your TFTP server locally from your development machine
  * Configure DHCP server in your network to direct the boot loader of your
    development board to the TFTP server on your development machine

:Let the run tool obtain the serial output from your board:

  Take a look at the various options of run tool at
  [https://github.com/genodelabs/genode/blob/master/tool/run/log - tool/run/log].

:Network-controlled reset / power switch:

  As the icing on the cake, consider powering your board via a
  network-controlled power socket as described in [1].

  More options can be found at at _tool/run/power_off_ [2] and
  _tool/run/power_on_ [3].

  For further inspiration, you may also enjoy the article [4].

[1] [https://genodians.org/chelmuth/2019-03-13-powerplug]

[2] [https://github.com/genodelabs/genode/blob/master/tool/run/power_off]

[3] [https://github.com/genodelabs/genode/blob/master/tool/run/power_on]

[4] [https://genodians.org/tomga/2019-08-13-rpi-automation]


code:
Bare-metal serial output
========================

In the previous section,
we started getting acquainted with the Pine64 hardware, established a serial
connection using Linux, and explored the use of the U-Boot boot loader.
Now we can move towards running Genode's kernel on the device. Before
touching Genode, however, we need to take two precautions.

# We need to understand the hand-over of execution from the boot loader to the
  loaded kernel code.

# In order to know that the right things are happening within our custom
  code, we need a way to get information out.

To address both questions, we are going to build a custom code blob that can
be copied to a predefined physical-memory address and, when executed, prints
characters over the serial line.
For the latter, we need a primitive way to print debug messages over a serial
connection. This section goes through the steps of executing custom code on
bare-metal hardware with no kernel underneath, and attaining serial output by
poking UART device registers directly.

Information gathering
---------------------

During our initial exploration in Section
[Getting acquainted with the target platform], we stumbled over a serial device
of type "16550A" at address 0x1c28000 that is apparently used by the Linux kernel by
default. We have already seen it in action when we interacted with U-Boot and
the Armbian system over the serial connection. Just for reference, here is
the corresponding 'dmesg' output again:

! [    2.250163] 1c28000.serial: ttyS0 at MMIO 0x1c28000
!                (irq = 31, base_baud = 1500000) is a 16550A

There are several ways to find out more about this particular device.
For example, one might be inclined to consult chip-vendor documentation. This,
however, can be a muddy approach. More often than not, ARM-based SoCs are
poorly covered by public documentation, or the available documentation
contains uncertainties or even errors. Whenever feasible, I like to follow the
path of ground truth, looking at known-to-work code as reference.
Let's examine the build configuration of our build of U-Boot, which can
be readily found in the _u-boot/.config_ file. When searching it for the string
"Serial", we quickly end up at the following line:

! CONFIG_SYS_NS16550=y

The driver has to have something like "NS16550" in its name. So let's grep
the source tree for files named after this string:

! $ cd u-boot
! $ find | grep -i NS16550
! ./drivers/serial/ns16550.c
! ./drivers/serial/ns16550.su
! ./drivers/serial/.ns16550.o.cmd
! ./drivers/serial/ns16550.o
! ./drivers/serial/serial_ns16550.c
! ./include/ns16550.h
! ./include/config/sys/ns16550.h
! ./spl/drivers/serial/ns16550.su
! ./spl/drivers/serial/serial_ns16550.o
! ./spl/drivers/serial/.ns16550.o.cmd
! ./spl/drivers/serial/ns16550.o
! ./spl/drivers/serial/serial_ns16550.su
! ./spl/drivers/serial/.serial_ns16550.o.cmd

That looks promising. At this point, we are especially interested in drawing
the connection to the UART device address 0x1c28000. Remember how we specified
'PLAT=sun50i_a64' to the build system of U-Boot? The "sun50i_a64" has to refer
to our SoC. So let's grep the source tree for any connection between "sun" and
"NS16550".

! grep -r NS16550 | grep -i sun
! ...
! include/configs/sunxi-common.h:# define CONFIG_SYS_NS16550_COM1  SUNXI_UART0_BASE
! include/configs/sunxi-common.h:# define CONFIG_SYS_NS16550_COM2  SUNXI_UART1_BASE
! ...

Next stop, SUNXI_UART0_BASE:

! grep -r SUNXI_UART0_BASE
! ...
! arch/arm/include/asm/arch-sunxi/cpu_sun9i.h:#define SUNXI_UART0_BASE (REGS_APB1_BASE + 0x0000)
! arch/arm/include/asm/arch-sunxi/cpu_sun4i.h:#define SUNXI_UART0_BASE 0x01c28000
! ...

Now seeing the address 0x01c28000, we know for certain that we are looking at
the right device and the corresponding driver code.

The next step consists of cross-checking several pieces of information.
Searching the web for "NS16550 pdf" brings up the data sheet for the
device ([http://caro.su/msx/ocm_de1/16550.pdf]). In contrast to SoC
chip vendor documentation, data sheets of individual IP cores like this are -
if publicly available - usually of good quality. So we are lucky.
Glimpsing over the data sheet, we learn that the register at offset 0 is
the so-called transmitter holding register (THR). We must write to this
register to print a character.
It is interesting to see that all device registers are 8 bits wide.
This raises the question how those registers are mapped to system-bus
addresses of the ARM SoC. The answer can be found at the
[https://files.pine64.org/doc/datasheet/pine64/Allwinner_A64_User_Manual_V1.0.pdf - Allwinner A64 manual]
as linked by the Pine64 wiki.
Here, we learn that the individual registers are mapped to 32-bit aligned
memory-mapped I/O registers.
Thus, the register offsets found in the NS16550 data sheet have to be
multiplied with 4.
The THR register is of course mapped to offset 0.
For cross-checking this information, the U-Boot driver code at
_drivers/serial/ns16550.c_ becomes handy.

What has the NS16550 data sheet has to say about the THR register?

! Before writing this register the user must ensure that the UART is
! ready to accept data for transmission, for example checking that THR
! Empty flag is set in the LSR

LSR stands for line status register. According to the data sheet, it is
the 5th register. Hence, it should be accessible at the ARM system bus at
offset 5*4 = 0x14. We also learn that the mentioned "Empty" flag hides behind
bit 5 of the LSR.

20 bytes yelling "U"
-------------------

As a preliminary test, let's try to unconditionally write the character 'U'
(ASCII value 0x55) to the THR register in an infinite loop.
The corresponding C program (saving the file as _main.c_) looks as follows:

! int _start()
! {
!   for (;;)
!     *(unsigned long *)0x1c28000 = 'U';
! }

Since we will ultimately have to use Genode's tool chain very soon, now
would be a good time to [https://genode.org/download/tool-chain - install it].
The tool chain comes with AARCH64 support. All the utilities can be found at
_/usr/local/genode/tool/current/bin/_. One may consider adding this directory
to the shell's PATH variable to avoid the need for typing out this rather
long path. But that is just a matter of convenience.

The following invocation of GCC compiles our little C program into an ELF binary:

! $ genode-aarch64-gcc -nostdlib main.c -o serial_test

The '-nostdlib' flag tells the compiler that we don't want to link any
C runtime or default startup code. Let's inspect the result by disassembling
the binary using 'objdump'.

! $ genode-aarch64-objdump -ld serial_test
!
! serial_test:     file format elf64-littleaarch64
!
!
! Disassembly of section .text:
!
! 0000000000400000 <_start>:
! _start():
!   400000:	d2900000 	mov	x0, #0x8000                	// #32768
!   400004:	f2a03840 	movk	x0, #0x1c2, lsl #16
!   400008:	d2800aa1 	mov	x1, #0x55                  	// #85
!   40000c:	f9000001 	str	x1, [x0]
!   400010:	17fffffc 	b	400000 <_start>

Even though the instructions look quite alien to me (not being too familiar
with the AARCH64 ISA at this point), this looks very reasonable. It's good
that the generated code does not rely on a stack pointer because we cannot
assume to have a valid stack. However, the link address 0x400000 is concerning
because the RAM base address of the A64 SoC is not lower than 0x40000000.
Remember, when we looked at Linux' _/proc/iomem_, we spotted the following
line:

! 40000000-bdffffff : System RAM

So we will have to tweak the linker arguments a bit.
From our experiments with U-Boot, we learned that U-Boot's default
load address 0x42000000 lies within this range. We can use the linker argument
'-Ttext' to explicitly specify our desired link address for the text (code)
segment:

! genode-aarch64-gcc -Wl,-Ttext=0x42000000 -nostdlib main.c -o serial_test

The '-Wl,' prefix is merely needed to tell the GCC frontend to pass the
following argument to the linker. With this tweak, the disassembled binary
looks even better:

! Disassembly of section .text:
!
! 0000000042000000 <_start>:
! _start():
!     42000000:	d2900000 	mov	x0, #0x8000                	// #32768
!     42000004:	f2a03840 	movk	x0, #0x1c2, lsl #16
!     42000008:	d2800aa1 	mov	x1, #0x55                  	// #85
!     4200000c:	f9000001 	str	x1, [x0]
!     42000010:	17fffffc 	b	42000000 <_start>

The 'serial_test' is a complete ELF binary with all kinds of meta data.
For running the instructions on the target, we either need an ELF loader
(U-Boot can of course do that for us) or we climb the hill barefoot.
The latter gives us more control. So let's convert the ELF binary into
a raw binary using 'objcopy'.

! genode-aarch64-objcopy -Obinary serial_test serial_test.img

We named the raw binary _serial_test.img_. Checking its size, it is quite
thrilling to see that it is just 20 bytes of pure usefulness! No overhead.

The next step would be fetching the image via U-Boot's TFTP support. The
TFTP server running my development machine serves the directory
_/var/lib/tftpboot/_. So we have to copy our _serial_test.img_ to this
directory before turning to U-Boot's console:

! => bootp 10.0.0.32:/var/lib/tftpboot/serial_test.img
! BOOTP broadcast 1
! BOOTP broadcast 2
! BOOTP broadcast 3
! DHCP client bound to address 10.0.0.178 (1100 ms)
! Using ethernet@1c30000 device
! TFTP from server 10.0.0.32; our IP address is 10.0.0.178
! Filename '/var/lib/tftpboot/serial_test.img'.
! Load address: 0x42000000
! Loading: #
! 	 4.9 KiB/s
! done
! Bytes transferred = 20 (14 hex)

It seems our program in its entirety reached its designated place.
Now it's time to take a jump!

! => go 0x42000000
! ## Starting application at 0x42000000 ...
! UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU...

The serial console gets flooded with U characters. What a joyful moment!

Let's reiterate what we gained by this experiment:

* We know how to compile a custom C program into binary code that works
  on the target.

* We successfully loaded our binary onto the target and passed control
  from the boot loader to our code.

* We got a positive lifesign back from our code.

Evolving from primordial vocals to words
----------------------------------------

Until now, we just violently poked the THR register without listening for the
status of the UART device. To make the program utter words instead of merely
vocals, this ignorance has to stop.

While modifying our program, we have to be careful to not using the stack.
While doing these iterative experiments, a little Makefile becomes handy, which
prints the disassembled program after each compilation:

! CROSS_DEV_PREFIX := /usr/local/genode/tool/current/bin/genode-aarch64-
!
! serial_test: main.c
! 	$(CROSS_DEV_PREFIX)gcc -Wl,-Ttext=0x42000000 -nostdlib $< -o $@
! 	$(CROSS_DEV_PREFIX)objdump -ld $@
!
! serial_test.img: serial_test
! 	$(CROSS_DEV_PREFIX)objcopy -Obinary $< $@
!
! test: serial_test.img
! 	cp $< /var/lib/tftpboot/

This little workflow tool not only makes life so much more convenient but it
also documents the use of the various commands for the future me. Since I
regard it as a mere personal tool of mine, I even don't hesitate place
commands like the copying of the image to my TFTP directory in there. Now, by
issuing 'make test', the command takes all the steps of compiling, showing the
assembly code, creating the raw binary, and copying to the TFTP directory all
at once.

Turning back to our actual program, the next baby step would be the output
of a string of characters instead of just one character, like so:

! static char const *text = "Aye aye.\n\r";
! static char const *s;
!
! for (;;)
!   for (s = text; *s; s++)
!     *(unsigned int volatile *)0x1c28000 = *s;

You may wonder why the variables 'text' and 's' are marked as static? If I
made them local variables, which would normally be the better practice, the
compiler would generate a stack frame. For example, by merely changing the
'for' loop to the innocent looking line
! for (char const *s = text; *s; s++)
the corresponding assembly program will generate instructions changing and
de-referencing the stack-pointer register:

!    42000000:	d10043ff 	sub	sp, sp, #0x10
!    42000004:	90000080 	adrp	x0, 42010000 <_start+0x10000>
!    42000008:	91018000 	add	x0, x0, #0x60
!    4200000c:	f9400000 	ldr	x0, [x0]
!    42000010:	f90007e0 	str	x0, [sp, #8]
!    ...

Since we don't have a stack, this is a big no-no!
The 'static' keyword tells the compiler to statically allocate the variable
at the data (or bss) segment of the binary. Speaking of binary segments,
for a bit of a shock, have a look at the binary size now:

! $ ls -la serial_test.img
! -rwxrwxr-x 1 no no 65640 Dez 17 15:30 serial_test.img

Isn't that embarrassing? With our change, we inflated the binary size from
20 bytes to more than 64 KiB. This effect is caused by our use of variables,
which were completely absent in the initial version. The use of at least one
variable prompts the compiler/linker to generate a data segment in addition to
the text (code) segment. By default, the linker places each segment at an
aligned address using a default alignment. On AARCH64, this default alignment
is 64 KiB so that the segment always starts at the beginning of a MMU page
when using virtual memory. Because of this default behavior, our few
instructions are followed by almost 64 KiB of zeros before the variables
start at the next 64 KiB boundary. As of now, we don't use any MMU. So we
could in principle weaken the default alignment. Just for reference, the GCC
argument for defining a segment alignment of 16 bytes would
be '-Wl,-z -Wl,max-page-size=0x10'. Voila! The image shrunk from 64 KiB to
less that 200 bytes. Well, I'll stop the bean counting for now and run this
version of the program:

! ## Starting application at 0x42000000 ...
! Aye aye.
! Aye aye.
! Aye aye.
! Aye aye.
! Aye aye.
! Aye aye.
! Ayeaaaaaaaaaaaaaaaaaaaa...

[image serial_aaaa]

Even though we can see strings of characters, at one point, the output
regresses to primordial vocals again.
This had to be anticipated since we don't yet check the TX status bit before
writing a new character to the THR register. Interestingly, it worked for
a while, presumably as long as the capacity of the UART's TX FIFO buffer
could swallow the characters.

By the way, while tinkering with devices at such a bare-bones level with
almost no infrastructure, an artificial delay can be accomplished as follows:

! for (i = 0; i < 1000000; i++)
!   asm volatile("nop");

By adding these lines to the body of the outer for loop, we can indeed
observe stable output. But that is of course just a hack. Let's us better
change the code to actually evaluate the status bit.

! int _start()
! {
!   enum {
!     UART_BASE = 0x1c28000,
!
!     THR = UART_BASE,
!     LSR = UART_BASE + 0x14,
!
!     LSR_THRE = (1 << 5)
!   };
!
!   /* static is needed to prevent the compiler from creating a stack frame */
!   static char const *text = "Aye aye.";
!
!   for (;;) {
!
!     static char const *s;
!
!     for (s = text; *s; s++) {
!
!       /* poll 'TX Holding Register Empty' bit */
!       while (((*(unsigned int volatile *)LSR) & LSR_THRE) == 0);
!
!       *(unsigned int volatile *)THR = *s;
!     }
!   }
! }

Note the amount of lipstick I applied to the code.

* Adding a comment here and there.
* Grouping things with vertical whitespace.
* Using enum values to give magic values tangible names.

I agree that this may be a little excessive for such a temporary test program.
But keep in mind that I wrote it not for my present me, but for you, and my
future me. Also note that I removed the line break from the 'text', which
has no reason other than making the following picture more pretty.

[image serial_aye]
  Infinite obedience

Thanks to listening to the UART's TX status bit, the output has become
reliable. So now, we have a minimal and known-to-work blueprint for
our upcoming kernel's UART driver. With this primitive way to get information
out of the board, we can turn our attention to the kernel-porting work, which
is the topic of the next section.


code:
Telephony
=========

_This section is based on Sebastian Sumpf's_
_[https://genodians.org/ssumpf/2022-05-09-telephony - Telephony article]_
_at [https://genodians.org]_.

It describes the steps for enabling basic telephony support on the PinePhone.

LTE Modem
---------

The PinePhone ships with a stock Quectel
[https://wiki.pine64.org/images/8/82/Quectel_EG25-G_LTE_Standard_Specification_V1.3.pdf - EG-25-G]
LTE capable modem designed
for machine to machine communication. Inherently, the
[https://wiki.pine64.org/wiki/PineModems - modem]
is a system on a chip (SoC) in itself with its own CPU core (Qualcomm MDM9207),
RAM (256MB), and flash memory (256MB). It features next to LTE/UMTS/GSM, audio
codecs, audio playback, GPS, and Bluetooth support.

Firmware
--------

The firmware is essentially a custom Linux system that PinePhone's Allwinner A64
[https://linux-sunxi.org/images/b/b4/Allwinner_A64_User_Manual_V1.1.pdf - SoC]
can interact with through UART, USB, and PCM interfaces. A speciality
of the Quectel modem is the support to flash custom firmware into the modem.
With this feature, it is possible to execute well-known trusted firmware instead
of opaque binary blobs on the modem. An example of such a firmware is the
_PinePhone Modem SDK_ as provided by
[https://github.com/Biktorgj/pinephone_modem_sdk - Biktorgj].
The firmware offers better power-management support because it can scale the CPU
frequency down to 100MHz (stock firmware is 400MHz) during sleep. Additionally
it offers some features like non-persistent storage, time synchronization from
carrier to user space, as well as zero binary blobs unlike the stock firmware.
The non-persistent storage is especially interesting because the stock firmware
may corrupt the flash memory - and thereby the whole SoC - if not shutdown
correctly. This could happen on a crash or in case of battery failure. A
downside of the custom firmware is that it is not feature complete, might be
unstable, and in the worst case can even brick the modem. For these reasons, we
decided to stay with the stock firmware for our telephony experiments.

UART and USB interfaces
-----------------------

Traditionally, modems are accessed through serial UART interfaces using the *AT*
[https://en.wikipedia.org/wiki/Hayes_command_seti - command set]
This feature is still provided by all modern modems. It can be
used for placing and receiving calls, sending and receiving SMS, managing the
SIM card, phone book, selecting carriers, managing audio, or GPS. For a complete
list of the AT commands supported by the Quectel EG25-G, please refer to the
[https://wiki.pine64.org/images/1/1b/Quectel_EC2x&EG9x&EG2x-G&EM05_Series_AT_Commands_Manual_V2.0.pdf - official manual].

The UART of the Quectel modem is connected to UART3 on the Allwinner SoC, which
is a standard
[https://en.wikipedia.org/wiki/16550_UART - 16550 UART]
as used by normal PCs since 1987. A driver for this
UART was already present in Genode but only for the output (TX) case, since
it was solely used for logging to the serial port. Because for each AT command
sent via the UART to the modem a response is generated, we had to extend Genode's UART
driver with receive (RX) support. This way, any terminal emulator (e.g.,
minicom, picocom, screen) can connect to the UART, send AT commands, and receive
responses. Unfortunately this will only work if the modem is powered on.

The USB interfaces are used for data connections (2G/3G/4G) because UARTs are
not fast enough (i.e., 115200 baud) to handle the data rates - up to 300 MBit/s -
supported by modern standards. In this case, the modem exposes a USB network interface
in order to send and receive data plus an additional control
channel to configure the connection. For a basic description on the workings of
data connections, please refer to our
[https://genodians.org/ssumpf/2020-12-04-mbim - LTE modem support for Genode article].
Because we were primarily interested in telephony at this point, we did not enable the USB
part nor the Allwinner SoC connections of the modem - which in turn conserves power - and
handled telephony through the UART interface using AT commands.

Modem startup
~~~~~~~~~~~~~

As with most SoCs, the modem is not powered during PinePhone's boot process. This
requires manual power-on/off handling by Genode. Important to know is that the
modem is powered by the battery directly and cannot be started with just the USB
charger plugged in. The reason for this is that the power consumption of the
modem can be anywhere from a 2 mA (sleep) up to 700 mA
([https://wiki.pine64.org/images/2/20/Quectel_EG25-G_Hardware_Design_V1.4.pdf - LTE data])
and may vary abruptly, which does not go well with power supplies but can be
handled by a battery.

The modem is connected through various input/output pins (GPIO) to the Allwinner
SoC through the Port
[https://linux-sunxi.org/images/b/b4/Allwinner_A64_User_Manual_V1.1.pdf - controller].
Without going into too much detail, there is one pin that enables the power
supply from the battery to the modem. Once the power supply is ready, the modem
can be powered on through a PWRKEY pin. This pin must be pulled down for at
least 500 ms, which will initiate the modem's boot process that can take up to
30 seconds. In order to determine if the modem is powered on, another pin
(status) can be observed. When the pin goes low, the modem has finished booting.
For Genode, we have implemented this behavior in a small component called
[https://github.com/genodelabs/genode-allwinner/blob/master/src/drivers/modem/pinephone/main.cc - modem manager].

[tikz img/pinephone_telephony]
  Experimental modem setup on Genode's PinePhone.

Basic telephony scenario
------------------------

With the modem powered, we can connect to the UART of the modem using a terminal
emulator and should see:
! > RDY
This means we can now enter AT commands, the simplest would be:
! AT
! > OK
Enter the PIN for the SIM card:
! AT+CPIN=1234
! > OK
Check the connection status:
! AT+CREG?
! > CGREG: 0,1
! > OK
This means the modem is connected to the home network.
Issue a call:
! ATD03513231421424;
! > OK
Hang up:
! ATH
! > OK
If one sees a
! > RING
one can accept the call by
! > ATA
The recommended way to shutdown the modem before the PinePhone is powered off is
! AT+QPOWD

With these simple commands, we achieved basic telephony support at an extremely
low complexity on the Genode side. The only noteworthy drawback is that no one
can hear you and you can hear no one because audio has not been setup yet. We
will look into this next.

Audio codec
~~~~~~~~~~~

The Allwinner SoC audio codec is split into two parts. First, an analog part for
microphones, speakers, and headsets. And second, a digital part where the SoC
itself, the modem, and the Bluetooth interfaces are connected to. As expected,
analog-to-digital converters (ADC) and digital-to-analog converters (DAC)
bridge both the analog and digital worlds for recording and playback.

Fortunately, we are not the first ones interested in the topology of all the
components involved.
For example, [https://xnux.eu] gives valuable insights in the dedicated article
[https://xnux.eu/devices/feature/audio-pp.html - Audio on PinePhone], in
particular an [https://xnux.eu/devices/feature/audio-controls.svg - annotated version]
of the audio diagram depicted in the Allwinner A64 manual.
At the top, three different AIFs (audio interface) are displayed. AIF1
leads to the APB bus, which in turn connects to the Allwinner SoC. AIF2 connects
to the BB (base band), which is the Quectel modem whereas AIF3 connects to
Bluetooth that is not of interest for this line of work. The green boxes ADCL/R
and DACL/R are the connection to the analog part of the codec, where the
recording and playback devices are connected to.

The whole scenario depicted in the diagram leads to
the assumption that the microphone of the PinePhone and the earpiece/speaker can
be routed through simple configuration to and from the modem without the actual
involvement of the SoC (AIF1). AIF1 exists for playback sounds/music, record
audio, and handle data connections like video conferencing where data packets
need to be sent to the speaker while microphone inputs need to be transferred to
the modem. AIF1 would also require an actual sound driver within Genode. Note
that the modem implements this driver locally for itself because it has its own
operating system.

Following this idea, Figure [telephony_audio_controls_modem] depicts the
setup we want to achieve for telephony for the left channel. The red arrow marks
the path from the microphone to the modem whereas the green arrow shows the path
from the modem to the earpiece/speaker.

[image telephony_audio_controls_modem 100%]
  Telephony routes for the left channel

Low-level setup
---------------

Without going into too much detail, the enablement of any device on ARM platforms
is what the BIOS usually does for x86. Meaning - everything is turned off. This
step required configuring and activating the Audio PLL, setting up and (un)gateing
necessary clocks, bringing the audio codec out of reset state, and configuring the
GPIO pins to take and send signals to/from the AIF2 (modem) bus.

Digital audio codec
-------------------

For the digital part, we want to route the ADC where the microphone is connected to
the modem (AIF2 mixer), and the playback from the modem to the DAC where the
speaker and the earplugs are connected. For each stage, there exist different
mixers that must be configured as well as additional registers to set the data
rate, volume, bit width, and audio format. The essential step is to enable ADC (left
and right channel) as one source of the AIF2 mixer. There can be more than one
source, for example, the AIF2 mixer can also use AIF1 (the Allwinner SoC) as
input. For the playback route, we set the source of the DAC mixer to the AIF2
DAC output of the modem. Of course, all parts need to be enabled separately.

Analog audio
------------

The analog part of the codec is accessed through a single memory-mapped I/O
register called AC_PR (Figure [telephony_ac_pr]). Through this register, 32
8-bit data registers can be addressed through the _Addr_ field. In and out data
is read/placed by the codec from/into the _Data_in_ and _Data_out_ fields.

[image telephony_ac_pr]
  Analog configuration register
  (Allwinner A64 Manual v1.1, page 362)

With the digital part ready, it was time to activate the actual microphone and
the earpiece speaker. In Figure [telephony_audio_controls_modem] all gray
boxes marked with an "m" - for mute - can be configured through the ADC mixer.
For the microphone, this means that we have to set the microphone as a source of
the ADC mixer. Additionally, every device has a configurable amplifier that
needs to be enabled. For the microphone, that is the boost amplifier. Since
microphones also require a voltage (bias voltage) this must also be configured.
And voila, the microphone was working on a test phone call.

For the earpiece, in turn, we needed to configure the DAC mixer as an input source,
enable the amp and unmute the device. This worked out of the box and we were
able to perform an actual phone call with the PinePhone running Genode.

As a final test, we enabled the speaker for hand-free communication that is
connected to the line out. This works analogously to the earpiece, with the exception
that the speaker amplifier is external on the PinePhone and needs to be enabled
via a dedicated GPIO pin.

So with everything working it's time for a test call using
[https://github.com/genodelabs/genode-allwinner/blob/master/run/modem_pinephone.run - this run script]:

_Roger Murdock: We have clearance, Clarence._

_Captain Oveur: Roger, Roger. What's our vector, Victor?_

_Tower voice: Tower's radio clearance, over!_

_Captain Oveur: That's Clarence Oveur. Over._


code:
Kernel packaging and testing
============================

With our toes still a bit frozen from testing the waters of the user land, we
now take the remaining steps towards a cultivated Genode life, largely
automating our work flow, packaging the kernel, and testing the platform like
there is no tomorrow.

During the initial user-land bring-up described in the
previous Section [Excursion to the user land], the process of
building a system image, loading the image onto the board, and obtaining log
output required quite a few manual steps: Starting picocom, issuing
the 'make run/log' command, copying the system image to the TFTP directory,
resetting the board, scanning the log output with our eyes. Some parts of this
process can be streamlined.

Accelerating our run-script workflow
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

First, instead of manually instructing the run tool to produce a uImage
instead of an ELF image at the command line, we can place the following
line into our _etc/built.conf_ file.

! RUN_OPT += --include image/uboot

Second, we can let the run tool manage the execution of the 'picocom' command
instead of manually starting the it by adding the following line:

! RUN_OPT += --include log/serial

This way, we can skip the step of spawning of picocom. But more importantly,
the run tool becomes able to detect the success of run scripts automatically!
So the part about "scanning the log output with our eyes" becomes much more
relaxing.

Third, the copying of the uImage file into the TFTP directory can be
automated by adding the following lines.

! RUN_OPT += --include load/tftp
! RUN_OPT +=   --load-tftp-base-dir /var/lib/tftpboot
! RUN_OPT +=   --load-tftp-offset-dir /$(BOARD)

Upon the next execution of the 'make run/log KERNEL=hw BOARD=pine_a64lts'
command, a new symbolic link appears at _/var/lib/tftpboot_.

! $ ls -la /var/lib/tftpboot/pine_a64lts
! lrwxrwxrwx ... /var/lib/tftpboot/pine_a64lts -> /.../build/arm_v8a/var/run/log/uImage

The symlink is updated each time a run script is executed. It always points
to the most recently built system image.
By setting the U-Boot bootcmd to load _/var/lib/tftpboot/pine_a64lts_, the board
will automatically fetch the most recently built system image.

! => env edit bootcmd
! edit: bootp 10.0.0.32:/var/lib/tftpboot/pine_a64lts ; bootm
! => env save
! Saving Environment to FAT... OK

With these little tweaks, the work flow with run scripts becomes almost
fully automated. The only remaining manual steps are:

# Issuing the 'make run/...' command at the build directory.
# Once the message 'Terminal ready' appears, pressing the reset button
  on the board.

This is a perfectly acceptable level of convenience.
If you want to go even further, you may find the following two articles
inspiring.

:Remote-control your test target via power scripts:

  [https://genodians.org/chelmuth/2019-03-13-powerplug]

:Exploring Genode Base HW with Raspberry Pi - further workflow automation:

  [https://genodians.org/tomga/2019-08-13-rpi-automation]

There is no better way to celebrate the new level of efficiency than to
test-drive a few hand-picked run scripts.

Stress-testing the init component
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The log scenario that we executed so far already employed Genode's init
component, which is the first (and only) component immediately started by
core. Init constructs a subsystem of components according to a
configuration in XML form. The init configuration for the log scenario
was rather primitive. There exists comprehensive test that exercises
the entire feature set of init by running a dynamically configured
instance of init as a child of init.
The test is hosted at the _repos/os/_ repository and has the form of a
deployable package (more on that later).
You can find the test's ingredients at _recipes/pkg/test-init/_ (package with
the runtime description) _src/test/init/_ (driver for executing a test
sequence) _recipes/raw/test-init/test-init.config_ (sequence executed by the
test driver).

That's probably be a bit overwhelming. Let's better just try it out. To run a
test package, the _os/run/test.run_ script becomes handy. We can use it as
follows, passing the name of the test package as 'PKG' argument.

! build/arm_v8a$ make run/test PKG=test-init KERNEL=hw BOARD=pine_a64lts
! ...
! ...
! genode build completed
! ...
! Terminal ready

At this point, we have to press the reset button of the board.

! ...
! ... log output of more than 200 test steps
! ...
! [init -> test -> test-init] --- test complete ---
! [init -> test] child "test-init" exited with exit value 0
!
! Run script execution successful.

What else could we ask for! When examining the log output, you can get a
glimpse of the feature set at work: Addition and deletion of subsystems,
changing access-control policies in the fly, reconfiguring child components,
chaining services, balancing resources among the components, heartbeat
monitoring, and exit handling.

As another noteworthy detail, in contrast to the simple log test, the init
test employs a timer at the user level. Since the test passed, we have the
confirmation that the in-kernel timer driver and interrupt-controller driver
work in principle.

Timer accuracy test
~~~~~~~~~~~~~~~~~~~

Speaking of the timer, it is generally not enough to know that the timer
works in principle but also that it is precise, which comes down to its
correct calibration. Genode provides a ready-to-use test that compares
the notion of time as observed by the Genode system with the wall-clock time
as known on your host system. The run script for this low-level test
is located at _repos/base/run/timer_accuracy.run_.

! build/arm_v8a$ make run/timer_accuracy KERNEL=hw BOARD=pine_a64lts
! ...
! ...
! Genode 20.11-197-g635985f542 <local changes>
! 2010 MiB RAM and 64533 caps assigned to init
! [init -> test-timer_accuracy]
! [init -> test-timer_accuracy]
! [init -> test-timer_accuracy]
! [init -> test-timer_accuracy]
! [init -> test-timer_accuracy]
! [init -> test-timer_accuracy]
! [init -> test-timer_accuracy]
! [init -> test-timer_accuracy]
! [init -> test-timer_accuracy]
! [init -> test-timer_accuracy]
! Good: round 1, host measured 1000 ms, test measured 1008 ms
! Good: round 2, host measured 2000 ms, test measured 2000 ms
! Good: round 3, host measured 3000 ms, test measured 3006 ms
! Good: round 4, host measured 4000 ms, test measured 3997 ms
! Good: round 5, host measured 5000 ms, test measured 5003 ms
! Good: round 6, host measured 6000 ms, test measured 6007 ms
! Good: round 7, host measured 7000 ms, test measured 6995 ms
! Good: round 8, host measured 8000 ms, test measured 7984 ms
! Good: round 9, host measured 9000 ms, test measured 9005 ms
!
! Run script execution successful.

Be patient, the test can take up to 40 seconds.
The output looks just perfect.

Testing the dynamic linker
~~~~~~~~~~~~~~~~~~~~~~~~~~

The ldso test exercises the functionality of the dynamic linker, including
the execution of global constructors, transitive library dependencies,
exception handling across libraries, and cross-library symbol resolution.

! build/arm_v8a$ make run/test PKG=test-ldso KERNEL=hw BOARD=pine_a64lts
! ...
! ... build libc
! ...
! [init -> test] child "test-ldso" exited with exit value 123
!
! Run script execution successful.

Given that the init test succeeded, which already employed the dynamic linker,
the result is not surprising but reassuring.

Packaging the kernel
~~~~~~~~~~~~~~~~~~~~

All but the most basic run scripts leverage Genode's
[https://genode.org/documentation/genode-foundations/22.05/development/Package_management.html - package management],
often referred to as _depot_. A run script can conveniently incorporate
packaged components into a system scenario via the 'import_from_depot'
function. When reviewing the various existing run scripts in the Genode source
tree for this function, one can spot the following pattern.

! import_from_depot [depot_user]/src/[base_src] \
!                   [depot_user]/...

Each argument denotes a path of a depot archive. The 'depot_user' and
'base_src' are function calls. The 'depot_user' function returns the name of
the originator/creator of the given depot archive. It returns 'genodelabs' by
default and can be customized in the _etc/build.conf_ file.

The 'base_src' function returns the archive name of the so-called "base"
source archive for a given combination of board and kernel.
It contains the lowest-level and kernel-specific fundamentals any
system scenario relies on, namely the kernel/core, the dynamic linker, and a
timer driver. In order to execute any of the run scripts that follow this
pattern, we need to create such a depot archive for our version
of the kernel. When using the "hw" kernel, the 'base_src' function can be
found at _tool/run/boot_dir/hw_.

! $ grep -r base_src tool/run/
! ...
! run/boot_dir/hw:proc base_src { } { return "base-hw-[board]" }
! ...

In the case of our 'pine_a64lts' board, the source archive would hence be
named 'base-hw-pine_a64lts'. The _tool/depot/create_ tool can be used
to populate the depot. Even though we have not yet provided any declaration
for our base archive, let's call the tool and see how it breaks:

! $ ./tool/depot/create x/src/base-hw-pine_a64lts UPDATE_VERSIONS=1 FORCE=1
! Error: incomplete or missing recipe (x/src/base-hw-pine_a64lts)

The following things are worth noting about the command-line arguments.

* I supply 'x' as depot user, which is just a dummy name that is good
  enough while pursuing the packaging work. Once the work is finished, it
  allows me to just remove the _depot/x/_ directory and all testing artifacts
  are gone.

* The UPDATE_VERSIONS=1 argument tells the tool to automatically increase
  the version of the depot archive whenever the content differs from the
  previously packaged version. During the packaging work, I always set it
  to 1.

* The FORCE=1 argument tells the tool to perform all packaging steps from
  scratch instead of reusing artifacts from previous runs.

Now, let's address the error message. It tell us that the tool expected a
so-called recipe for the given depot archive, which does not exist yet. The
so-called recipes describe how a depot archive can be extracted from the
source tree. They are searched in the _<repo>/recipes/_ directories of all
repositories. E.g., the recipe for the base source archive for the i.MX8 EVK
board resides at _repos/base-hw/recipes/src/base-hw-imx8q_evk/_. This
is a suitable template for out pine_a64lts recipe.

! $ mkdir -p repos/allwinner/recipes/src/base-hw-pine_a64lts
! $ cp -r repos/base-hw/recipes/src/base-hw-imx8q_evk/* \
!         repos/allwinner/recipes/src/base-hw-pine_a64lts/

The directory hosts three files:

:content.mk:

  This is Makefile snippet with rules for gathering the content of the
  archive from the source tree. The copied file, however, merely includes
  rules from a file called _base-hw_content.inc_. We can keep this line.

  ! include $(GENODE_DIR)/repos/base-hw/recipes/src/base-hw_content.inc

:used_apis:

  This file contains a list of APIs required to build a binary archive
  from the source archive. The copied template contains merely two lines,
  which we can keep that way. Naturally, the base-hw kernel requires the
  definitions of the generic Genode API (base API) and the supplements
  that are specific for the base-hw kernel (base-hw API).

  ! base-hw
  ! base

:hash:

  The hash file tells the depot tools about the current version of the
  archive and draws the connection to the corresponding archive content
  by specifying a hash value.

  ! 2021-02-24 d122ddee70f0b075de8cec50a41c9f4783702e05

  The hash value is computed over the entire content of the archive. Should
  the hash of a freshly created archive deviate from the hash stored at the
  recipe, we know that the version should better be updated. Of course, it
  would be tiresome to calculate such hash values manually. Thankfully, the
  depot tools do this job for us. While keeping in mind that the hash is most
  certainly wrong for our the pine_a64lts source archive, we leave it as is
  because we don't know any better value anyway at this point.

With the new source recipe in place, let's give the package creation
another try.

! $ ./tool/depot/create x/src/base-hw-pine_a64lts UPDATE_VERSIONS=1 FORCE=1

This time, the output looks different:

! $ ./tool/depot/create x/src/base-hw-pine_a64lts UPDATE_VERSIONS=1 FORCE=1
! created x/api/base/2021-02-22
! created x/api/base-hw/2021-02-22
!
! Error: CPU architecure for board pine_a64lts undefined
!
!   missing file /.../repos/allwinner/board/pine_a64lts/arch

The tool has successfully created the API archives for the dependencies
we stated in the _used_apis_ file. However, the source-archive creation
still backs out, missing the information of the board's CPU architecture.
The CPU architecture dictates the subset of files of the base-hw repository
that are relevant for the given board. This information is expected at
the path printed by the error message. That's our call!

! $ mkdir -p repos/allwinner/board/pine_a64lts
! $ echo arm_v8a > repos/allwinner/board/pine_a64lts/arch

Upon the next attempt to create the source archive, the creation-process
succeeds.

! $ ./tool/depot/create x/src/base-hw-pine_a64lts UPDATE_VERSIONS=1 FORCE=1
! created x/api/base/2021-02-22
! created x/api/base-hw/2021-02-22
! created x/src/base-hw-pine_a64lts/2021-03-04 (new version)

The last line tells us that the tool has detected the inconsistency
of our recipe's hash file with the assembled archive content and has
automatically adjusted the version (taking the current date) and hash in the
hash file. Now it looks as follows:

! $ cat repos/allwinner/recipes/src/base-hw-pine_a64lts/hash
! 2021-03-04 4589eb3b4d816d17a2f2a539031a54eff9dd3712

You may like to have a look at the resulting archive.

! $ ls depot/x/src/base-hw-pine_a64lts/2021-03-04/
! etc  include  lib  LICENSE  src  used_apis

Finally, let us check that it is possible to create a binary archive from our
source archive by specifying 'x/bin/arm_v8a/base-hw-pine_a64lts' as
depot-archive path to the depot-create tool. To accelerate the build, we can
append '-j8' as argument to enable the use of multiple CPUs.

! $ ./tool/depot/create x/bin/arm_v8a/base-hw-pine_a64lts \
!                       UPDATE_VERSIONS=1 FORCE=1 -j8
! created x/api/base-hw/2021-02-22
! created x/api/base/2021-02-22
! created x/src/base-hw-pine_a64lts/2021-03-04
! checking library dependencies...
! ...
! ... many build steps
! ...
!     LINK     timer
! created x/bin/arm_v8a/base-hw-pine_a64lts/2021-03-04

This time, the tool was satisfied with the current hash of our source recipe,
the build process ran to completion, and we can inspect the results at
the printed location within the depot.

! $ ls -1 depot/x/bin/arm_v8a/base-hw-pine_a64lts/2021-03-04/
! bootstrap-hw-pine_a64lts.o
! core-hw-pine_a64lts.a
! ld.lib.so
! timer

Combined test suite of over 80 system scenarios
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

With the kernel packaged, a whole new world of run scripts opens up for us.
The most intriguing one is _repos/gems/run/depot_autopilot.run_. It is a
system that does not only execute a single test scenario but orchestrates the
execution of more than 80 system scenarios one after the other. The init test
we executed earlier is just one of these scenarios. Combined, the
scenarios form the comprehensive test suite for Genode's base framework
covering the following topics.

* Low-level data structures and allocators
* Parsing and generating XML, UTF-8
* Integration of Ada/SPARK with Genode's C++ API
* Publisher-subscriber mechanism
* Management of dynamic subsystems
* Fault detection mechanism
* Synthetic tests for low-level components and interfaces
  such as init, timer, VFS, block access, terminal
* VFS infrastructure
* C runtime (I/O, execve, fork, pthreads)
* Standard C++ library
* TCP/IP
* Network routing
* Tracing
* On-target deployment of depot packages

[tikz img/depot_autopilot_arch 100%]

The diagram gives an overview of the architecture of the system scenario. It
is described in great detail in the release documentation of
[https://genode.org/documentation/release-notes/18.11#Automated_test_infrastructure_hosted_on_top_of_Genode - Genode 18.11].

As a prerequisite for executing the depot-autopilot scenario, the depot
packages for the whole arsenal of tests must be made available. We can
instruct the build system to automatically create the depot content as needed,
by enabling the following option at the _etc/build.conf_ file:

! RUN_OPT += --depot-auto-update

Furthermore, we need to make sure to have the following repositories
enabled in the _etc/build.conf_ file.

! REPOSITORIES += $(GENODE_DIR)/repos/libports
! REPOSITORIES += $(GENODE_DIR)/repos/dde_linux
! REPOSITORIES += $(GENODE_DIR)/repos/gems

The dde_linux repository is solely needed for the TCP/IP stack ported from the
Linux kernel (lxip). The gems repository hosts the depot-autopilot. With these
precautions taken, we can kick off the _depot_autopilot.run_ script as usual.

! build/arm_v8a$ make run/depot_autopilot KERNEL=hw BOARD=pine_a64lts

You will most likely encounter an error like the following.

! Error: Ports not prepared or outdated:
!   ada-runtime dde_linux expat gcov gmp libc lwip sanitizer stdcxx
!
! You can prepare respectively update them as follows:
!   /.../tool/ports/prepare_port ada-runtime dde_linux expat gcov \
!                                gmp libc lwip sanitizer stdcxx

The printed ports of 3rd-party software are required. They can be imported
into Genode's _contrib/_ directory by executing the command as suggested
by the error message.

Once the prepare_port command has completed, we can give the
depot_autopilot.run script another try. This time, we can lay back and enjoy
tons of build output scroll by, take a nip at a cup of coffee, maybe stretch
our back a little, continue watching the build output, relax, not to forget to
keep breathing. Have I mentioned looking at the build output?

When finally loading the resulting uImage on the board, we are greeted
with a shocking message:

! TFTP from server 10.0.0.32; our IP address is 10.0.0.178
! Filename '/var/lib/tftpboot/uImage'.
! Load address: 0x42000000
! Loading: #################################################################
!          #################################################################
!          #################################################################
!          #################################################################
!          #################################################################
!          #################################################################
!          #################################################################
!          #################################################################
!          #################################################################
!          ##
!          3.5 MiB/s
! done
! Bytes transferred = 8608974 (835cce hex)
! ## Booting kernel from Legacy Image at 42000000 ...
!    Image Name:
!    Image Type:   AArch64 Linux Kernel Image (gzip compressed)
!    Data Size:    8608910 Bytes = 8.2 MiB
!    Load Address: 40010000
!    Entry Point:  40010000
!    Verifying Checksum ... OK
!    Uncompressing Kernel Image
! Error: inflate() returned -5
! Image too large: increase CONFIG_SYS_BOOTM_LEN
! Must RESET board to recover
! resetting ...

Just at the climax of our expectations, U-Boot's ELF loader went on strike.
The ELF image is apparently too large. We take the mental note to adjust
'CONFIG_SYS_BOOTM_LEN' and re-build and re-install U-Boot taking the steps of
the [https:2020-12-10-pine-fun-warmup - earlier article].

To satisfy our urgent need of a reward for our patience during the build of
the depot-autopilot system image, we can side step U-Boot's image-size
constraint by loading a raw binary image. Since we need an ELF image instead
of a uImage, we have to temporarily disable the '--include image/uboot'
RUN_OPT in the _etc/build.conf_ file and rebuild the image.

! build/arm_v8a$ make run/depot_autopilot KERNEL=hw BOARD=pine_a64lts
! ....

A quick look at the result reveals that the uncompressed ELF image is
quite large compared to the uImage file of 8 MiB. Here we can see the benefit
of uImage files.

! build/arm_v8a$ ls -lh var/run/depot_autopilot/boot/image.elf
! -rwxrwxr-x 1 ... 49M ... var/run/depot_autopilot/boot/image.elf

The ELF image is swiftly converted to a raw binary placed in our TFTP directory.

! build/arm_v8a$ /usr/local/genode/tool/current/bin/genode-aarch64-objcopy \
!                    -Obinary \
!                    var/run/depot_autopilot/boot/image.elf \
!                    /var/lib/tftpboot/depot_autopilot.img
! build/arm_v8a$ ls -lh /var/lib/tftpboot/depot_autopilot.img
! -rwxrwxr-x 1 ... 49M ... /var/lib/tftpboot/depot_autopilot.img

Now, we can use the following U-Boot command to load it on the board.

! => bootp 0x40010000 10.0.0.32:/var/lib/tftpboot/depot_autopilot.img
! BOOTP broadcast 1
! DHCP client bound to address 10.0.0.178 (109 ms)
! Using ethernet@1c30000 device
! TFTP from server 10.0.0.32; our IP address is 10.0.0.178
! Filename '/var/lib/tftpboot/depot_autopilot'.
! Load address: 0x40010000
! Loading: #################################################################
! ...
!          ######################################################
!          3.2 MiB/s
! done
! Bytes transferred = 51351552 (30f9000 hex)

... and run it!

! => go 0x40010000
! ## Starting application at 0x40010000 ...
!
! kernel initialized
! ...

... massive amount of log output scrolls by for about 9 minutes ...

! ...
! [init -> depot_autopilot] --- Finished after 519.179 sec ---
! [init -> depot_autopilot]
! [init -> depot_autopilot]  test-spark                 ok   0.239
! [init -> depot_autopilot]  test-spark_exception       ok   0.161
! [init -> depot_autopilot]  test-spark_secondary_stack ok   0.518
! [init -> depot_autopilot]  test-block                 ok   1.124
! [init -> depot_autopilot]  test-block_cache           ok   0.692
! [init -> depot_autopilot]  test-clipboard             ok   3.676
! [init -> depot_autopilot]  test-depot_query_index     ok   0.289
! [init -> depot_autopilot]  test-ds_ownership          ok   0.227
! [init -> depot_autopilot]  test-dynamic_config        ok   3.154
! [init -> depot_autopilot]  test-dynamic_config_loader ok   3.211
! [init -> depot_autopilot]  test-dynamic_config_slave  ok   2.640
! [init -> depot_autopilot]  test-entrypoint            ok  40.117
! [init -> depot_autopilot]  test-weak_ptr              ok   2.900
! ...
! ...
! ...
! [init -> depot_autopilot]  test-xml_generator         ok   0.570
! [init -> depot_autopilot]  test-xml_node              ok   1.028
! [init -> depot_autopilot]  gcov                       ok  39.064
! [init -> depot_autopilot]
! [init -> depot_autopilot] succeeded: 82 failed: 0 skipped: 4
! [init -> depot_autopilot]
! [init] child "depot_autopilot" exited with exit value 0

The entire test suite succeeded with no errors!

It took 519 seconds. To cross-correlate this duration with the depot-autopilot
test on the i.MX8q EVK board: The i.MX board takes 465 seconds, which makes
the Pine-A64-LTS around 10% slower than the i.MX8 EVK. This is of course no
benchmark to draw meaningful conclusions from. But the fact that both values
are in the same ballpark reassures us that nothing fundamental (like the
low-level CPU or memory configuration) went wrong with our port.


code:
                    =================================
                    Genode Operating System Framework
                               Platforms
                    =================================

                              Norman Feske

: \vfill \epsfig{file=img/by-sa,angle=0,width=0.2\textwidth} \hspace{0.3cm}
: \begin{minipage}[b]{0.8\textwidth} \raggedright \small
: This work is licensed under the Creative Commons Attribution + ShareAlike
: License (CC-BY-SA). To view a copy of the license, visit
: http://creativecommons.org/licenses/by-sa/4.0/legalcode
: \end{minipage}