|
|
The main gitlab server to develop CORSIKA 8 is gitlab.ikp.kit.edu
|
|
|
|
|
|
It is configured to run CI jobs for all pushes and merge requests. The configuration can be found in the repository's **.gitlab-ci.yml**
|
|
|
|
|
|
The jobs are based on docker containers published here ```https://hub.docker.com/r/corsika/devel/tags``` and developed in the ```corsika-docker``` git project in parallel to the CORSIKA source code.
|
|
|
|
|
|
The CI jobs are distributed on Runners via the ```corsika``` **tag**. Runners need to support **docker**.
|
|
|
|
|
|
Note, we setup our Runners to be **assigned** to the corsika and related projects (group runners for https://gitlab.ikp.kit.edu/AirShowerPhysics). This also means: from your personal fork you might not be able to run CI jobs on the Runners. Thus, it is an advantage to keep main development on the main corsika repository.
|
|
|
|
|
|
## General setup of Runners
|
|
|
|
|
|
Runners should be group-runners and have a name indicating their physical location, e.g. corsika-hawaii-1, corsika-unam-1, etc.
|
|
|
|
|
|
The following tags should be assigned:
|
|
|
- **corsika**
|
|
|
- for later use: **gpu** (if GPU is installed)
|
|
|
- number of cores to be available for parallel execution: e.g. **cores_4**
|
|
|
|
|
|
The following configurations are needed (config.toml):
|
|
|
|
|
|
- we used **docker** executors
|
|
|
- enable concurrent builds (if number of cores is larger than number of cores assigned to one job): set **concurrent** to 1 in global section
|
|
|
- We also want to enable custom build dir:
|
|
|
```
|
|
|
[runners.custom_build_dir]
|
|
|
enabled = true
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Here are some commands used to setup Runners on ```augerbot```:
|
|
|
|
|
|
Note: This is not the best way to setup runners, we prefer that it is better to use containers to start runners!
|
|
|
|
|
|
- setup service
|
|
|
```sudo gitlab-runner install --working-directory ~ --config /etc/gitlab-runner/config.toml --service gitlab-runner --syslog --user gitlab-runner```
|
|
|
- start service
|
|
|
``` sudo gitlab-runner run --working-directory ~ --config /etc/gitlab-runner/config.toml --service gitlab-runner --syslog --user gitlab-runner```
|
|
|
|
|
|
## Observed problems
|
|
|
|
|
|
If the Runners using docker get stuck periodically (this seems not to happen any more with recent runner versions!). It seems this is most likely related to (stale?) docker images and missing cleanup. We developed the following script which so far works to get Runners back online:
|
|
|
|
|
|
```
|
|
|
df -h
|
|
|
echo "start"
|
|
|
sudo service gitlab-runner stop
|
|
|
sudo service docker stop
|
|
|
sudo service docker start
|
|
|
sudo docker system prune -af
|
|
|
sudo docker volume prune
|
|
|
sudo service gitlab-runner start
|
|
|
echo "done"
|
|
|
```
|
|
|
|
|
|
If needed, run this in a crontab. There might be further, and more intelligent, options to optimize our use of docker containers. Currently (Aug 2020) pulling docker containers is responsible for a sizeable fraction of runtime on the runners.
|
|
|
|