Skip to content

fix post_easyblock_hook_copy_easybuild_subdir: check that easybuild subdirectory exists before trying to copy it to reprod directory - #276

Merged
bedroge merged 2 commits into
EESSI:mainfrom
boegel:fix_post_easyblock_hook_copy_easybuild_subdir
Jul 30, 2026
Merged

fix post_easyblock_hook_copy_easybuild_subdir: check that easybuild subdirectory exists before trying to copy it to reprod directory#276
bedroge merged 2 commits into
EESSI:mainfrom
boegel:fix_post_easyblock_hook_copy_easybuild_subdir

Conversation

@boegel

@boegel boegel commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

This avoids a confusing "Failed to copy directory" error message when an installation failed because downloading of source files files, for example:

== 2026-07-30 13:31:38,659 build_log.py:233 ERROR EasyBuild encountered an error: Failed to copy directory /cvmfs/software.eessi.io/versions/2025.06/software/linux/x86_64/amd/zen5/software/nettle/3.10.1-GCCcore-14.2.0/easybuild to /cvmfs/s
oftware.eessi.io/versions/2025.06/software/linux/x86_64/amd/zen5/reprod/nettle/3.10.1-GCCcore-14.2.0/20260730_133138UTC/easybuild: [Errno 2] No such file or directory: '/cvmfs/software.eessi.io/versions/2025.06/software/linux/x86_64/amd/ze
n5/software/nettle/3.10.1-GCCcore-14.2.0/easybuild' (at easybuild/tools/filetools.py:2738 in copy_dir)
Callstack:
        easybuild/tools/filetools.py:2738 in copy_dir
        easybuild/eb_hooks.py:964 in post_easyblock_hook_copy_easybuild_subdir
        easybuild/eb_hooks.py:2199 in post_easyblock_hook
        easybuild/tools/hooks.py:252 in run_hook
        easybuild/framework/easyblock.py:5365 in build_and_install_one
        easybuild/main.py:184 in build_and_install_software
        easybuild/main.py:630 in process_eb_args
        easybuild/main.py:337 in process_easystack
        easybuild/main.py:812 in main
        easybuild/main.py:872 in main_with_hooks
        easybuild/main.py:891 in <module>

While the actual problem (higher up in the log) is a download failure:

== 2026-07-30 14:13:03,592 build_log.py:233 ERROR EasyBuild encountered an error: Couldn't find file nettle-3.10.1.tar.gz anywhere, and downloading it didn't work either...

… subdirectory exists before trying to copy it to reprod directory
@bedroge

bedroge commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

bot: build repo:eessi.io-2023.06-software instance:eessi-bot-mc-aws for:arch=x86_64/amd/zen2
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-mc-aws for:arch=x86_64/amd/zen2

@eessi-bot-aws

eessi-bot-aws Bot commented Jul 30, 2026

Copy link
Copy Markdown

New job on instance eessi-bot-mc-aws for repository eessi.io-2023.06-software
Building on: amd-zen2
Building for: x86_64/amd/zen2
Job dir: /project/def-users/SHARED/jobs/2026.07/pr_276/181662

date job status comment
Jul 30 15:32:04 UTC 2026 submitted job id 181662 awaits release by job manager
Jul 30 15:32:28 UTC 2026 released job awaits launch by Slurm scheduler
Jul 30 15:38:01 UTC 2026 running job 181662 is running
Jul 30 15:48:00 UTC 2026 finished
😁 SUCCESS (click triangle for details)
Details
✅ job output file slurm-181662.out
✅ no message matching FATAL:
✅ no message matching ERROR:
✅ no message matching FAILED:
✅ no message matching required modules missing:
✅ found message(s) matching No missing installations
✅ found message matching .tar.* created!
Artefacts
eessi-2023.06-software-linux-x86_64-amd-zen2-17854260650.tar.zstsize: 0 MiB (31411 bytes)
entries: 1
modules under 2023.06/software/linux/x86_64/amd/zen2/modules/all
no module files in tarball
software under 2023.06/software/linux/x86_64/amd/zen2/software
no software packages in tarball
reprod directories under 2023.06/software/linux/x86_64/amd/zen2/reprod
no reprod directories in tarball
other under 2023.06/software/linux/x86_64/amd/zen2
no other files in tarball
Jul 30 15:48:00 UTC 2026 test result
😁 SUCCESS (click triangle for details)
ReFrame Summary
[ OK ] ( 1/10) EESSI_LAMMPS_lj %device_type=cpu %module_name=LAMMPS/29Aug2024-foss-2023b-kokkos %scale=1_node /aeb2d9df @BotBuildTests:x86-64-zen2+default
P: perf: 440.762 timesteps/s (r:0, l:None, u:None)
[ OK ] ( 2/10) EESSI_LAMMPS_lj %device_type=cpu %module_name=LAMMPS/2Aug2023_update2-foss-2023a-kokkos %scale=1_node /04ff9ece @BotBuildTests:x86-64-zen2+default
P: perf: 455.857 timesteps/s (r:0, l:None, u:None)
[ OK ] ( 3/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node %device_type=cpu /775175bf @BotBuildTests:x86-64-zen2+default
P: latency: 3.29 us (r:0, l:None, u:None)
[ OK ] ( 4/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node %device_type=cpu /52707c40 @BotBuildTests:x86-64-zen2+default
P: latency: 2.86 us (r:0, l:None, u:None)
[ OK ] ( 5/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node %device_type=cpu /b1aacda9 @BotBuildTests:x86-64-zen2+default
P: latency: 5.69 us (r:0, l:None, u:None)
[ OK ] ( 6/10) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node %device_type=cpu /c6bad193 @BotBuildTests:x86-64-zen2+default
P: latency: 5.83 us (r:0, l:None, u:None)
[ OK ] ( 7/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node /15cad6c4 @BotBuildTests:x86-64-zen2+default
P: latency: 0.73 us (r:0, l:None, u:None)
[ OK ] ( 8/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node /6672deda @BotBuildTests:x86-64-zen2+default
P: latency: 0.9 us (r:0, l:None, u:None)
[ OK ] ( 9/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.2-gompi-2023b %scale=1_node /2a9a47b1 @BotBuildTests:x86-64-zen2+default
P: bandwidth: 6454.22 MB/s (r:0, l:None, u:None)
[ OK ] (10/10) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.1-1-gompi-2023a %scale=1_node /1b24ab8e @BotBuildTests:x86-64-zen2+default
P: bandwidth: 6475.86 MB/s (r:0, l:None, u:None)
[ PASSED ] Ran 10/10 test case(s) from 10 check(s) (0 failure(s), 0 skipped, 0 aborted)
Details
✅ job output file slurm-181662.out
✅ no message matching ERROR:
✅ no message matching [\s*FAILED\s*].*Ran .* test case
Jul 30 16:09:42 UTC 2026 uploaded transfer of eessi-2023.06-software-linux-x86_64-amd-zen2-17854260650.tar.zst to S3 bucket succeeded

@eessi-bot-aws

eessi-bot-aws Bot commented Jul 30, 2026

Copy link
Copy Markdown

New job on instance eessi-bot-mc-aws for repository eessi.io-2025.06-software
Building on: amd-zen2
Building for: x86_64/amd/zen2
Job dir: /project/def-users/SHARED/jobs/2026.07/pr_276/181663

date job status comment
Jul 30 15:32:10 UTC 2026 submitted job id 181663 awaits release by job manager
Jul 30 15:32:25 UTC 2026 released job awaits launch by Slurm scheduler
Jul 30 15:33:36 UTC 2026 running job 181663 is running
Jul 30 15:35:51 UTC 2026 finished
😁 SUCCESS (click triangle for details)
Details
✅ job output file slurm-181663.out
✅ no message matching FATAL:
✅ no message matching ERROR:
✅ no message matching FAILED:
✅ no message matching required modules missing:
✅ found message(s) matching No missing installations
✅ found message matching .tar.* created!
Artefacts
eessi-2025.06-software-linux-x86_64-amd-zen2-17854255800.tar.zstsize: 0 MiB (31414 bytes)
entries: 1
modules under 2025.06/software/linux/x86_64/amd/zen2/modules/all
no module files in tarball
software under 2025.06/software/linux/x86_64/amd/zen2/software
no software packages in tarball
reprod directories under 2025.06/software/linux/x86_64/amd/zen2/reprod
no reprod directories in tarball
other under 2025.06/software/linux/x86_64/amd/zen2
no other files in tarball
Jul 30 15:35:51 UTC 2026 test result
😁 SUCCESS (click triangle for details)
ReFrame Summary
[ OK ] (1/6) EESSI_LAMMPS_lj %device_type=cpu %module_name=LAMMPS/22Jul2025-foss-2024a-kokkos %scale=1_node /ade8cad7 @BotBuildTests:x86-64-zen2+default
P: perf: 435.496 timesteps/s (r:0, l:None, u:None)
[ OK ] (2/6) EESSI_LAMMPS_lj %device_type=cpu %module_name=LAMMPS/22Jul2025_update4-foss-2025b-kokkos %scale=1_node /e121eb9c @BotBuildTests:x86-64-zen2+default
P: perf: 443.576 timesteps/s (r:0, l:None, u:None)
[ OK ] (3/6) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a %scale=1_node %device_type=cpu /e4bf9965 @BotBuildTests:x86-64-zen2+default
P: latency: 1.36 us (r:0, l:None, u:None)
[ OK ] (4/6) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a %scale=1_node %device_type=cpu /3da4890b @BotBuildTests:x86-64-zen2+default
P: latency: 2.06 us (r:0, l:None, u:None)
[ OK ] (5/6) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a %scale=1_node /3255009a @BotBuildTests:x86-64-zen2+default
P: latency: 0.17 us (r:0, l:None, u:None)
[ OK ] (6/6) EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a %scale=1_node /59f4b331 @BotBuildTests:x86-64-zen2+default
P: bandwidth: 7796.25 MB/s (r:0, l:None, u:None)
[ PASSED ] Ran 6/6 test case(s) from 6 check(s) (0 failure(s), 0 skipped, 0 aborted)
Details
✅ job output file slurm-181663.out
✅ no message matching ERROR:
✅ no message matching [\s*FAILED\s*].*Ran .* test case
Jul 30 16:09:51 UTC 2026 uploaded transfer of eessi-2025.06-software-linux-x86_64-amd-zen2-17854255800.tar.zst to S3 bucket succeeded

@bedroge
bedroge merged commit 73788f4 into EESSI:main Jul 30, 2026
94 of 96 checks passed
@boegel
boegel deleted the fix_post_easyblock_hook_copy_easybuild_subdir branch July 30, 2026 20:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants