IAP GITLAB

Cumbersome to get the test logs

Normally, unit tests are configured such that you do not see the output of tests, only which tests fail.

This project uses a different approach. The tests are generated with full output and written to test.log. Then test.log is compressed and published as an artifact when the tests fail.

I have some issue with this approach. I cannot see quickly at a glance which tests failed by looking into the build output. I have to first download the log. Then I have to unpack the download, then I have to unpack the log. Then I have trouble finding the failing test, because the log is full of stdout from the tests.

Questions/suggestions to make investigating test failures less cumbersome:

  • Is it really necessary to compress test.log? Gitlab automatically compresses the artifact when I download it. The logs are only kept for a week, so the disk load should be ok even if the logs are stored uncompressed? When I download the logs, I have to unpack twice, which I would like to avoid.
  • How about running the tests without -V and show the output directly in the pipeline output. Only if the tests fail, run them again with -V, produce the test.log, etc.

Ideally, we want to run the tests once and capture stdout of the tests in a file, but still see the own output of ctest in the pipeline output. Only if the tests fail, we want to see the stdout. Perhaps that's possible with the -O flag of ctest, I have to test.

Edited by Hans Dembinski
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information