When building a CI/CD IT infrastructure for embedded SW we decided to use more flexible tools that can work on-premise and as hosted cloud services (there are companies that have strict privacy and security policies which prevent them from using hosted cloud solutions). Additionally, toolchains such as IAR that have a node lock license, and when doing on target tests, you must have an on-prem server to serve as your test harness. All of these components run on VMs (either windows or Linux ones). We decided to use different VMs for different types of test for modularity and support for future infrastructure updates.
CI/CD Master Server
We decided to use Jenkins as our CI/CD manager. Jenkins can run on-prem and on the cloud, on Windows/Linux machined, supports HiL tests and everything you need for embedded SW development. We are also very familiar with it, so it was a straightforward choice for us to make.
Source Control Server
We decided to use GitLab as it supports both on-prem and hosted cloud service. GitLab has a very friendly user interface and reliable plugin to Jenkins. In an on-prem scenario, we also connected the VM to a central backup service. If you don’t have an on-prem constraint, GitHub is probably the way to go.
CI Build Server
Our build server will also perform static analysis tests. For a build server, we used one of the two options:
- IAR build server — an on-prem Windows VM with Node lock license. Note that static analysis tools require a more expensive IAR license.
- GCC ARM build server — can be on-prem or cloud-based Linux VM (or even Docker-based).
Non-Host Testing Server
A Linux based VM, where you can run all non-target tests — unit tests and module testing with mocking. Our favorite tools for unit tests for embedded C are the tools provided by ThrowTheSwich.org — Unity, CMock, and Ceedling.
Simulated Testing Server
A Linux based VM, where you can run your binaries on a simulator of your devices and your entire system. There are few options for that like Synopsys’ VDK, WindRiver’s Simics, QEMU, Mentor’s Vista, etc. We recommend our own solution — Jumper’s Virtual Lab (we’ll have a different post covering this as well). A proper simulation/emulation environment should allow you to run-
- Unit tests
- Simulating sensors and peripherals I/Os
- Smoke tests — device and cluster of devices
- Black box (end to end) tests
- System level tests — load/stress, performance, security, etc. tests
- Cluster (scalability) testing with communication simulation
- Acceptance tests
- Performance tests
Target HW Testing Server
A Linux/Windows VM (the physical server should be located in the physical lab), where you can run tests on the target HW. This server will act as the testing harness or will be connected to an external HW testing harness. This server will control lab testing equipment such as scope, network analyzer, voltmeter, logic, packet generator, etc. We haven’t chosen specific testing framework here, as it depends on the specific device and system setup. On the target device and physical lab you should run -
- Smoke tests
- Black box tests
- System level tests
- HW validation and timing related tests
- Cluster testing with communication testing (note that the scalability here is limited)
- Certification tests
- Acceptance tests
Streamline Your CI/CD Flow and Automated Testing
The fundamental requirement for CI/CD flow is to have a fast feedback loop from code commit through integration to test results. The way to get there is through automating your process and mainly having automated tests. At Jumper, we’ve built a Virtual Lab platform to streamline CI/CD and automated testing. Jumper’s Virtual Lab allows R&D and QA teams to have a virtual production clone that runs the same physical device’s software with complete environment and communication simulation. If you work with embedded systems and want to have an easy way for testing your physical device’s embedded software, you’re invited to try Jumper and join our Beta Program.