Jacamar CI v0.6.0

• Release: v0.6.0

• Date: 04/29/2021

Admin Changes

• Improved Slurm executor by utilizing sbatch --wait as opposed to relying on sacct (!143, !164).

  • Prior interactions leveraged the sacct command, parsing any stdout generated. By moving away from this we aim to support a wider range of instances and avoid some complicated edge-case scenarios we’ve noticed with the previous iteration.

  • This change should not affect user’s experience with the Slurm executor as it still fundamentally relies upon sbatch

  • 

• Extensive improvement to the Git command modification process (!154).

  • We hope this addresses issues experienced with the required modifications to the get_sources script to account for the range of user/system configurations that are not experienced in traditional CI workflows.

• Cobalt corrections to address issues reported with version 0.5.0 testing (!135, !150).

  • Please extensively test the improvement to Cobalt and report any unexpected behaviors or errors encountered.

  • Similar to Slurm we will now override the job script to provide a /bin/bash --login operator, this should ensure a proper environment for any job script submitted.

• Remove support for source_script in the [auth] configuration (!143).

  • Incorrect configurations found within the script could lead to very difficult to manage/troubleshoot errors and we found that other improvement can account for the issues this configuration tried to address.

• Correctly build JWKS URL when server found in path (!157).

• In order to use jacamar directly without jacamar-auth, the application will only function as expected with the --no-auth command, signifying no authorization is expected (!160).

  • If using the jacamar-auth you will not notice this change, it is only designed to add an additional level of complexity to prevent miss-configuration in aspects of the workflow we cannot change.

Bug & Development Fixes

• Added support for Runner release 13.11 and 13.10 (!161, !138).

• The authorization workflow will now validate the supplied JSON Web Token at each stage of the job (!147).

• Added support for make rpm-docker and make runner-docker commands to build related RPMs locally via Docker (!138 !137).

  • Resulting RPMs can be found in the root of the project folder and properly accounted for by a make clean command.

• The internal/command packages structure has been improved to better differentiate between dowscoping and standard execution methods (!143).

• The first steps have been taken to improve the internal/authuser packages structure, more work will follow (!160).

• Added ModifyCmd(string, string...) to Runner and Commander interface (!150).

  • This allows an executor to override the default /usr/bin/evn ... bash --login behavior if required.

• Added the project path as a stateful variable (!153).

• Added support for downscope = "su" as it nearly follow the same command structure as sudo (!143).

  • Important, as with all downscoping methods we highly encourage extensive testing and acceptance prior to any deployments. Please don’t hesitate to report any issues you experience.

• Added optional downscope_env to the [auth] configuration (!149)

  • Any key=value string supplied will be used in the construction of the downscoped application and override any defaults established by Jacamar.

  • [auth]
      downscope_env = [
          "SYSTEM_VAR=EXAMPLE",
      ]
    

  • The goal of this addition is to support troubleshooting and short-term corrections until more comprehensive changes can be made to the “safe environment” creation process if required.

• Migrated Pavilion testing from sudo to su due to oddities in testing sudo -E su ... commands in CI (!143).

• Updated command make test-docker to align with other corrections (!140).

• Updates to directory identification (in package authuser) to align across builds and script directories (!148).

• Corrected command handling of goroutines causing blocking actions (!165).

• Configurable command directory (!162).