Skip to content

Improve usability of example job scripts to allow passing variables at runtime instead of needing direct script modification

Matthew K Defenderfer requested to merge feat-improve-example-jobs into patch-0.5.5

Added options to control which policies are processed based on date and source. These are passed via variables directly when executing the script. For example:

# To split all logs collected in the month of November 2025
DATE_GLOB=2025-11 ./00-split-logs.sh

# To convert split logs to parquet for policies run on either /scratch or /data/user over a period from June 21, 2025 to June 28, 2025
DATE_GLOB=2025-06-2[12345678] SOURCE='data-user,scratch' ./10-convert-to-parquet.sh

Also changed the default directory storing these data to /gpfs/project/rc/gpfs-policy/data to account for move to GPFS 5

Merge request reports