Manual Directory Cleanup

The goal of this example is to demonstrate a potential workflow that will provide teams with the ability to manually run cleanup tasks. This may be a desirable workflow for teams that generate substantial data and use underlying file systems.

Important

Removing files from a system as part of your CI job can be dangerous. Take care to verify the target directories are properly defined to avoid automatically removing required directories.

Example `.gitlab-ci.yml`

variables:
  # We are using STORE_DIR as an arbitrary storage location.
  STORE_DIR: "/ecp/scratch/tutorial/examples/ci_storage"

#
# Placeholder for other jobs/stages.
#

# 'file-removal' allows manual initiation and avoid impacting the other stages.
file-removal:
  # Source: https://docs.gitlab.com/ee/ci/yaml/#stage-post
  stage: .post
  # We should always attempt to leverage a shell executor for this job.
  tags: [generic, shell]
  # Source: https://docs.gitlab.com/ee/ci/yaml/#rules
  rules:
    - when: manual
  allow_failure: true
  script:
    - rm -rf ${STORE_DIR}

Pipeline Results

../../_images/manual-remove-generic.png

Notes

There are other GitLab supported tools you can leverage in such workflows. We recommend reading: protecting manual jobs.
Make cleanup a job-level action by creating an after_script; however, it should be noted that canceled jobs end abruptly preventing the after_script from running. Please refer to official GitLab efforts to ensure after_script is called for cancelled and timed out pipelines for the status of a fix.