Welcome to the Jacamar CI Documentation

The Exascale Compute Project CI effort focuses on empowering software projects with new testing models by enabling access to development HPC resources within their CI/CD workflows. This effort preserves teams’ existing development processes while collaborating with facilities to ensure the security and reliability of all testing capabilities.

Contents

• Project Developer
  • Continuous Integration
  • Guides
  • Tutorials
• Administration
  • Jacamar CI
  • Server
  • Guides
  • Latest Releases

If you are wondering “What is Continuous Integration (CI) testing”, then we encourage you to explore the related articles from Better Scientific Software.

Jacamar CI

Jacamar CI is the HPC-focused CI/CD driver using GitLab’s custom executor model. The core goal of this project is to establish a maintainable, yet extensively configurable tool that will allow for the use of GitLab’s robust testing on unique resources, allowing code teams to integrate potentially existing pipelines on powerful scientific development environments.

Facilities and users can leverage the Jacamar CI enhancements realized in the runner via the custom executor in a manner that supports the evolving needs of projects. This is possible due to the flexibility afforded by GitLab in both source code management and CI, via a range of supported executors and project-level customization.

This project is not intended to completely replace any existing CI workflows. Instead, we want these tools to benefit from the flexibility offered to CI/CD pipelines in both configuration and runner deployment. Ideally, these CI models will create added value when leveraged as part of a comprehensive testing strategy.

Key Enhancements

Supporting continuous integration on HPC resources necessitated a number of targeted software enhancements, new services, and the establishment of improved workflow capabilities:

• Advanced Workflows: An ongoing collaborative effort between facility administrators and project teams to establish documented workflows and best practices.

• Batch Executors: Run user-defined scripts at the job level by submitting to the underlying scheduling system.

• Custom Executor Compatibility: Upstream GitLab has added support for the custom executor, which allows for the implementation of administratively defined applications to realize runner functionality. All future runner-side enhancements will be completed within this model.

• Downscoping Permissions: Target the validated user who triggered the CI pipeline and drop permissions prior to job execution to that of their system local account.

The documentation on this site elaborates on these enhancements for both individuals who wish to leverage CI resources and administrators planning to deploy these tools.