sirian/comp.ce.460-embedded-linux-drivers
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[EX01] Introduction to Yocto and course development platform

Browse Source
This commit is contained in:
Panu Sjövall
2022-08-08 11:57:11 +03:00
parent a1e5a3b0e9
commit 6b0ce26dbd
9 changed files with 338 additions and 24 deletions
Download Patch File Download Diff File Expand all files Collapse all files

0
.gitattributes vendored Normal file
View File

24
.gitignore vendored Normal file
View File

@@ -0,0 +1,24 @@
/build/bitbake-cookerdaemon.log
/build/cache
!/build/conf
/build/conf/sanity_info
/build/downloads
/build/sstate-cache
/build/tmp/abi_version
/build/tmp/buildstats
/build/tmp/cache
/build/tmp/deploy/*
!/build/tmp/deploy/images
!/build/tmp/deploy/sdk
/build/tmp/hosttools
/build/tmp/log
/build/tmp/pkgdata
/build/tmp/saved_tmpdir
/build/tmp/sstate-control
/build/tmp/stamps
/build/tmp/sysroots
/build/tmp/sysroots-components
/build/tmp/sysroots-uninative
/build/tmp/work
/build/tmp/work-shared
*.testdata.json

29
README.md
View File

@@ -1,22 +1,6 @@
# COMP.CE.460 Real-time systems
# COMP.CE.460 Embedded Linux Drivers
--------------------------------------------------------------------------------
## Important Notice about LFS
This project uses a Git extension for versioning large binary files called [Git Large File Storage][git-lfs].
Every clone of this repository **MUST** initialize the Git LFS extension. Immediately after a clone operation, inside the repository, the following command **MUST** be issued:
~~~bash
git lfs install
~~~
The above command will setup the correct *hooks* in the <u>local copy</u> of the repository: these *hooks* will automate the use of the LFS extension, so that, transparently, using the usual workflow of `git add`/`git commit`/`git push`, a selection of the files will be stored differently.
The course VM already includes all the required dependencies to use this extension.
--------------------------------------------------------------------------------
## Quick guide to resume your work after rebooting the VM
If you are using HTTPS remotes, you might want to run `git config --global credential.helper 'cache --timeout=3600'` before cloning: this way you will have to input your username and password only after 1 hour without interacting with your remote.
@@ -24,8 +8,6 @@ If you are using HTTPS remotes, you might want to run `git config --global crede
1. `cd ~`
2. `git clone --recurse-submodules <STUDENT_REPO_URL> [<LOCAL_CLONE_DIR>]`
- by default, if you don't specify `<LOCAL_CLONE_DIR>` the local clone will be in `~/NN/`, if for any reason, during the first exercise you used a different name or path, make sure to replicate the one you used previously, or the layers configuration you committed previously will be incorrect;
3. `cd <LOCAL_CLONE_DIR>; git lfs install && git lfs pull`
## Quick guide to pull new exercise instructions
@@ -66,9 +48,8 @@ If you decide to use HTTPS remotes, you might want to run `git config --global c
If you need training to use Git, visit [Git Course in Plussa][git-plussa].
[moodle.COMP.CE.460]: https://moodle.tuni.fi/course/view.php?id=29722
[git-plussa]: https://plus.tuni.fi/tie-git/summer-2022/
[course_upstream project]: https://course-gitlab.tuni.fi/comp.ce.460-real-time-systems_2022-2023/course_upstream
[git-lfs]: https://git-lfs.github.com/
[moodle.COMP.CE.460]: https://moodle.tuni.fi/course/view.php?id=50067
[git-plussa]: https://plus.tuni.fi/tie-git/
[course_upstream project]: https://course-gitlab.tuni.fi/comp-ce-460-embedded-linux-drivers_2025-2026/course_upstream
[semipermanent-ssh-keys]: semipermanent-ssh-keys.md
[update_from_course_upstream]: https://course-gitlab.tuni.fi/comp.ce.460-real-time-systems_2022-2023/course_upstream/raw/master/scripts/update_from_course_upstream.sh
[update_from_course_upstream]: https://course-gitlab.tuni.fi/comp-ce-460-embedded-linux-drivers_2025-2026/course_upstream/raw/main/scripts/update_from_course_upstream.sh

1
exercises/01/README.md Normal file
View File

@@ -0,0 +1 @@
instructions.md

71
exercises/01/instructions.md Normal file
View File

@@ -0,0 +1,71 @@
# Introduction
The goal of this first exercise session is to familiarize with the tools used throughout the exercises for COMP.CE.460.
Throughout the exercise sessions of COMP.CE.460 *Embedded Linux Drivers*, we designed, in collaboration with Wapice, a simulation of a typical industry workflow as a System Engineer. We are initially tasked with getting the board to boot a custom GNU Linux distribution built with Yocto.
The starting point of the simulation will be set at receiving a new sample board for a new embedded project.
Rather than providing step-by-step instructions, we provide a list of reading materials and resources you should familiarize with in order to solve the task at hand.
We will use a Ubuntu virtual machine (which already includes all the required prerequisites to successfully build the Yocto Project).
The username and (very secure) password for the VM account are: `student`/`student`.
**WARNING**: be aware that **files on the VM will be erased at each reboot**, commit and push your work to your remote repository (or save your important files to an external drive) before rebooting the VM.
# Resources and reading material
- [Lecture #1 and #2 on Moodle][moodle.COMP.CE.460]
- [Yocto Project Quick Start][YoctoQS]
- [Yocto Reference Manual (C#7: *Source Directory Structure*)][YoctoREFMAN:sec7.1]
- [PYNQ-Z1 Board Reference Manual][PYNQ-Z1-REFMAN]
- [Git Submodules][Git Submodules]
- [meta-pynq layer repository][meta-pynq]
# Checklist
- [ ] get a microSD kit for your group from the TA
- [ ] clone Yocto `poky` and add support for the PYNQ board <u>**as submodules**</u>.
- **Hint**: check documentation of the PYNQ support layer to determine the proper version of `poky` and other requirements
- add each of these repositories as [Git Submodules] in the root of the student group repository
- [ ] initialize the Build Environment, **making sure the *Build Directory* is created as `<STUDENT_REPO_ROOT>/build`**
- notice that we require the *Build Directory* to be in a non-default location, requiring knowledge from the [linked section of the *Yocto Reference Manual*][YoctoREFMAN:sec7.1]
- <u>DO NOT COMMIT THE WHOLE BUILD DIRECTORY</u>: only the configuration files should be committed.
- [ ] setup your local configuration file
- [ ] build `core-image-minimal`
- **Hint**: building from scratch could require hours. Read on the *Yocto Reference Manual* about the `SSTATE_MIRRORS` variable and check the contents of `/opt/poky_cache` to speed up this step
- [ ] verify that the build was successful and copy the *required* files to the microSD
- required files:
- uImage
- u-boot.img
- boot.bin
- core-image-minimal-pynq.cpio.gz.u-boot
- uEnv.txt
- devicetree.dtb
- [ ] turn off the PYNQ board and set it to boot from the microSD
- [ ] turn on and connect via serial to verify successful boot
- the USB connection provides power to the board and provides a USB serial port on the host workstation: by default the bootloader and the kernel expose a virtual console on this serial interface.
- use PuTTY on the host to connect to the console
- [ ] commit Yocto build configuration files to the student repository
- [ ] commit the changes to the question file for this exercise to include your answers
- [ ] <u>**push to your remote repository**</u>
- [ ] demonstrate to the TA
- [ ] <u>remember to recover your microSD card before leaving</u>
[Git Submodules]: https://git-scm.com/book/en/v2/Git-Tools-Submodules
[YoctoQS]: https://docs.yoctoproject.org/2.4.3/yocto-project-qs/yocto-project-qs.html
[moodle.COMP.CE.460]: https://moodle.tuni.fi/course/view.php?id=50067
[YoctoREFMAN:sec7.1]: https://docs.yoctoproject.org/2.4.3/ref-manual/ref-manual.html#structure-core
[PYNQ-Z1-REFMAN]: https://reference.digilentinc.com/_media/reference/programmable-logic/pynq-z1/pynq-rm.pdf
[meta-pynq]: https://course-gitlab.tuni.fi/comp-ce-460-embedded-linux-drivers_2025-2026/meta-pynq

15
exercises/01/questions.md Normal file
View File

@@ -0,0 +1,15 @@
# Exercise 01 - Q&A
## 1. What did you accomplish in this exercise?
## 2. How does Yocto recognize that the target machine "pynq" is supported?
## 3. Where do you find the device-tree source code for the PYNQ board?
## 4. What is the priority of the PYNQ support layer?
## 5. What additional applications are installed to the `core-image-minimal` target in the PYNQ support layer?

32
exercises/README.md Normal file
View File

@@ -0,0 +1,32 @@
# Introduction
Throughout the exercise sessions of COMP.CE.460 *Embedded Linux Drivers*, we designed, in collaboration with Wapice, a simulation of a typical industry workflow as a System Engineer.
The starting point of the simulation will be set at receiving a new sample board for a new embedded project.
We are initially tasked with getting the board to boot a custom GNU Linux distribution built with Yocto, and will then proceed to develop a kernel driver to support a custom IP required for the project, eventually improving and tweaking it for real-time applications.
We will also use Yocto to generate a dedicated SDK to develop and cross-compile userland applications for the project, and in the final stages we will go back to work on the bootloader to adapt to different usage requirements.
Rather than providing a step-by-step guide, in general the instructions for each exercise session will include an introduction detailing the specific goals of the exercise, links to related reading materials and resources, and a checklist to self-validate your progression towards the solution of the exercise.
Generally each exercise will include an instructions file and a questions file: completion of the exercise requires completing the tasks listed in the instructions and adding answers to the questions file.
In general, the completion of each exercise will be marked after submitting a properly tagged commit (`ex??`, where `??` is replaced by the exercise number) to the group repository and presenting a working demonstration to the TA during the exercise sessions.
The evaluation of the submitted commits is done manually and might require some time; unless otherwise stated, the deadline for submitting the solution to an exercise and to demonstrate it to the course staff is set to one week after the scheduled exercise session (i.e., the first exercise should be demonstrated either during the first or the second exercise session).
The exercise sessions are conducted in groups (exercises and seminar presentations will be organized using the same groups), using any of the PYNQ-Z1 boards installed in lab TC217 and a microSD card: each member of the group should make sure to have valid access rights to TC217 (or compile the required forms).
The course staff will provide a microSD card kit per group (which includes a microSD card and a USB microSD reader): **it is important that you take good care of the card and reader, as it is required that you return them after the last exercise. You will not be getting your final grade before you have returned the borrowed equipment!**.
# Tools
Throughout the course we will use the following tools:
- Xilinx PYNQ-Z1 board (installed in TC217, must be set to boot from the microSD card)
- microSD card kit (which includes a microSD card and a USB microSD reader)
- Xilinx Vivado (installed in TC217 workstations)
- vmware Workstation/Player (installed in TC217 workstations)
- an Ubuntu virtual machine image:
- the image comes preinstalled with all the requirements for Yocto development
- located in `C:\Work\COMP-CE-460-vm` in TC217 workstations
- it will be used for most of the development and compilation tasks
- the image comes with vmware tools support: we will use `Shared Folders` to export the compiled artifacts to the host system and to write them to the microSD card
- usr/pwd: student/student
- **WARNING**: be aware that **files on the VM will be erased at each reboot**, commit and push your work to your remote repository (or save your important files to an external drive) before rebooting the VM.

96
scripts/update_from_course_upstream.sh Normal file
View File

@@ -0,0 +1,96 @@
#!/bin/bash
GIT=`which git`
function msg_ex() {
echo -e "$@" >&2
}
function msg() {
msg_ex "#### $@"
}
set -e
. "$(${GIT} --exec-path)/git-sh-setup"
function require_user_name_and_email() {
if ${GIT} config user.name >/dev/null && ${GIT} config user.email >/dev/null; then
true
else
msg "You need to configure your git user name and email"
msg_ex "\tgit config user.name \"<Your Name>\""
msg_ex "\tgit config user.email your@email.com"
die "Configure Git user name and email"
fi
}
function require_clean_work_tree_and_no_untracked() {
local action=$1
local hint="$2"
require_clean_work_tree ${action} "${hint}"
if [[ ! -z "$(${GIT} status --porcelain)" ]]; then
msg "Cannot ${action}: You have untracked files."
die "${hint}"
fi
}
function local_branch_exists() {
msg "Checking if local branch \"$1\" already exists"
${GIT} rev-parse --verify $1 >/dev/null 2>&1
}
function remote_is_defined() {
msg "Checking if remote \"$1\" is already defined"
${GIT} config "remote.$1.url" >/dev/null
}
function git_remote_add() {
msg "Adding remote \"$1\" (tracking $2)"
${GIT} remote add $1 $2
}
: ${GITURL_BASE_SSH:="git@course-gitlab.tuni.fi:comp-ce-460-embedded-linux-drivers_2025-2026"}
: ${GITURL_BASE_HTTPS:="https://course-gitlab.tuni.fi/comp-ce-460-embedded-linux-drivers_2025-2026"}
function ssh_or_https_remote() {
if [[ -z $(${GIT} remote -v | grep -m1 '^origin' | sed -Ene's#.*(https://[^[:space:]]*).*#\1#p') ]]; then
msg "Selecting SSH based remotes"
echo ${GITURL_BASE_SSH}
else
msg "Selecting HTTPS based remotes"
echo ${GITURL_BASE_HTTPS}
fi
}
cd_to_toplevel
require_user_name_and_email
require_clean_work_tree_and_no_untracked pull "Please commit or stash them."
GITURL_BASE="$(ssh_or_https_remote)"
GITURL_COURSE_UPSTREAM="${GITURL_BASE}/course_upstream.git"
START_BRANCH=$(${GIT} rev-parse --abbrev-ref HEAD)
REMOTE_NAME="course_upstream"
RBRANCH_NAME="main"
LBRANCH_NAME="${REMOTE_NAME}_updates"
remote_is_defined ${REMOTE_NAME} || git_remote_add ${REMOTE_NAME} ${GITURL_COURSE_UPSTREAM}
${GIT} fetch ${REMOTE_NAME}
if local_branch_exists ${LBRANCH_NAME}; then
msg "Checking out and updating branch ${LBRANCH_NAME}"
${GIT} checkout ${LBRANCH_NAME}
${GIT} pull
else
msg "Checking out branch ${LBRANCH_NAME} to track ${REMOTE_NAME}/${RBRANCH_NAME}"
${GIT} checkout -b ${LBRANCH_NAME} ${REMOTE_NAME}/${RBRANCH_NAME}
fi
${GIT} fetch --all --tags
msg "LFS fetch and checkout from ${REMOTE_NAME}"
${GIT} lfs fetch ${REMOTE_NAME}
${GIT} lfs checkout
msg "Checking out ${START_BRANCH} again"
${GIT} checkout ${START_BRANCH}
msg "Merging updates from ${REMOTE_NAME}/${RBRANCH_NAME} into ${START_BRANCH}..."; sleep 3
${GIT} merge --edit -m "Merge updates from ${REMOTE_NAME}/${RBRANCH_NAME}" ${LBRANCH_NAME}

94
semipermanent-ssh-keys.md Normal file
View File

@@ -0,0 +1,94 @@
# Introduction
Accessing remote git repositories using HTTPS quickly becomes inconvenient: depending on the configuration of a git repository, a single `git pull` might require to input the username/password three or more times.
Usually, generating a SSH keypair and adding the public half to the remote user profile would be easy and preferred, but working on a shared VM which erases its contents at every reboot creates some challenges: the goal of this document is to illustrate a potential workflow to securely use SSH git remotes in this environment.
## Table of Contents
1. [Workflow to generate a semipermanent git profile](#workflow-to-generate-a-semipermanent-git-profile)
2. [Workflow to restore your profile after VM reboot](#workflow-to-restore-your-profile-after-vm-reboot)
3. [Update existing `git` remotes to use an SSH URL](#update-existing-git-remotes-to-use-an-ssh-url)
4. [Alternatives](#alternatives)
# Workflow to generate a semipermanent git profile
These instructions should be followed only once, when generating for the first time your profile and keys.
The newly generated private key and the profile archive will be encrypted using (different) passphrases to minimize security risks associated with the shared environment.
## Generate SSH public keypair
- `ssh-keygen -t ed25519`
- default location is recommended
- **NOTICE**: Set a (strong) passphrase on the SSH private key (the `ssh-agent` will automatically ensure you will not have to enter this passphrase several times in a session)
- add the new public key to your Gitlab profile
- `cat $HOME/.ssh/id_ed25519.pub` to show the public key
- on Gitlab web UI: `User > Settings > SSH Keys`
## Add the new public key to your Gitlab profile
## Create a global `git` configuration
~~~bash
git config --global user.name "Mario Super"
git config --global user.email "the.mario@student.tut.fi"
~~~
## Create an encrypted tarball containing the profile
~~~bash
cd $HOME
tar cpvz .ssh .gitconfig | gpg --symmetric --output gitprofile.tar.gz.gpg
~~~
- Use a strong passphrase
- Avoid reusing the same passphrase protecting the SSH private key
- <u>Don't forget your passphrases!!</u>
- Save the encrypted archive either to an external USB drive (connecting it to the VM) or to your TUT profile folder (enabling and setting a Shared Folder from `VM > Settings > Options`)
# Workflow to restore your profile after VM reboot
- recover the encrypted tarball from the storage you selected (external USB drive or TUT profile folder)
- decrypt and restore the archive:
~~~bash
gpg --decrypt /path/to/gitprofile.tar.gz.gpg | tar xpvz -C $HOME
~~~
# Update existing `git` remotes to use an SSH URL
If you used HTTPS URLs to define the remotes in your local `git` repository and wish to switch to SSH to avoid repeated user/password accesses, you can use the following commands:
~~~bash
cd /path/to/your/local/repository
git remote set-url <remote_name> <remote_SSH_URL>
~~~
Examples:
- `course_upstream`:
~~~bash
git remote set-url course_upstream git@course-gitlab.tuni.fi:comp.ce.460-real-time-systems_2023-2024/course_upstream.git
~~~
- `origin` (student project, where `${GROUP_ID}` is your group number):
~~~bash
GROUP_ID="<NN>"
git remote set-url origin git@course-gitlab.tuni.fi:comp.ce.460-real-time-systems_2023-2024/${GROUP_ID}.git
unset GROUP_ID
~~~
# Alternatives
The described workflow is by no mean the only one.
It would be perfectly fine, for example, to generate a new keypair for every session and add the ephemeral public key to Gitlab every time (and remove old ephemeral keys). In this case you would also need to set the `user.name` and `user.email` properties in the `git` configuration at every reboot.