==== wrapper started @ Fri Apr 16 00:13:13 EDT 2021 ==== == startup: hostname: d8civy008.crc.nd.edu == startup: kernel: Linux d8civy008.crc.nd.edu 3.10.0-1160.el7.x86_64 #1 SMP Tue Aug 18 14:50:17 EDT 2020 x86_64 x86_64 x86_64 GNU/Linux ==== tracing google @ Fri Apr 16 00:13:18 EDT 2021 ==== wrapper.sh: line 20: traceroute: command not found ==== environment at startup @ Fri Apr 16 00:13:23 EDT 2021 ==== == env: VC3_VERSION_SED=4.2 == env: CCTOOLS_RESOURCE_PROCESS_START=1618546403185315 == env: VC3_INSIDE_WRAPPER_SCRIPT=yes == env: VC3_ROOT_GZIP=/usr/ == env: SHELL=/bin/sh == env: TERM=xterm-256color == env: MAKEFLAGS=-j4 == env: TMPDIR=/disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL == env: VC3_ROOT_WHICH=/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0 == env: LOBSTER_CVMFS_PROXY=http://eddie.crc.nd.edu:3128 == env: LD_PRELOAD=/disk/vc3-root/tmp/worker-230547-12913/t.167220/librmonitor_helper.so.zzOzh6 == env: LOBSTER_FRONTIER_PROXY=http://eddie.crc.nd.edu:3128 == env: LOBSTER_CMSSW_VERSION=CMSSW_10_6_20 == env: VC3_VERSION_FULL_TAR=1.26.0 == env: VC3_MACHINE_OS=Linux/redhat7.4.1708 == env: VC3_VERSION_FULL_CCTOOLS_WQ_WORKER=999.999.999 == env: USER=kmohrman == env: TEMP=/disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL == env: VC3_BUILDER_PATH=/disk/vc3-builder == env: LD_LIBRARY_PATH=/.singularity.d/libs == env: VC3_VERSION_VC3_BUILDER_HANDS=1.0 == env: VC3_ROOT_MAKE=/usr/ == env: VC3_DISTFILES=/disk/vc3-distfiles == env: WORKER_TMPDIR=/disk/vc3-root/tmp/worker-230547-12913/cache/tmp == env: VC3_VERSION_TAR=1.26 == env: ENV=/disk/vc3-root/vc3-home/.vc3_sh-DnPqmi == env: VC3_VERSION_CCTOOLS_WQ_WORKER=999.999 == env: VC3_VERSION_FULL_PKG_CONFIG=0.27.1 == env: VC3_VERSION_GZIP=1.5 == env: VC3_ENV_TAG_GLOBAL=12661 == env: PATH=/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0/bin:/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin == env: VC3_ROOT=/disk/vc3-root == env: VC3_SH_PROFILE_ENV=/disk/vc3-root/vc3-home/.vc3_sh-DnPqmi == env: VC3_ROOT_PKG_CONFIG=/usr/ == env: PWD=/disk/vc3-root/tmp/worker-230547-12913/t.167220 == env: LOBSTER_OSG_VERSION=3.4 == env: VC3_VERSION_FULL_SED=4.2.2 == env: VC3_VERSION_WHICH=2.20 == env: VC3_MACHINE_GLIBC_VERSION=2.17 == env: MODULEPATH=/afs/crc.nd.edu/x86_64_linux/Modules/modules/development_tools_and_libraries:/afs/crc.nd.edu/x86_64_linux/Modules/modules/general_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/restricted_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/system_modules:/afs/crc.nd.edu/x86_64_linux/Modules/modules/deprecated_software:/afs/crc.nd.edu/x86_64_linux/Modules/current/modulefiles == env: WORK_QUEUE_SANDBOX=/disk/vc3-root/tmp/worker-230547-12913 == env: VC3_ROOT_CCTOOLS_WQ_WORKER=/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999 == env: VC3_ROOT_SED=/usr/ == env: VC3_MACHINE_TARGET=x86_64/redhat7.4.1708 == env: CXX=g++ == env: SHLVL=3 == env: HOME=/disk/vc3-root/vc3-home == env: VC3_VERSION_MAKE=3.82 == env: VC3_VERSION_FULL_GZIP=1.5.0 == env: DISK=361 == env: MEMORY=1044 == env: VC3_VERSION_FULL_WHICH=2.20.0 == env: TMP=/disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL == env: VC3_INSTALL_USER_HOME=/disk/vc3-root/vc3-home == env: VC3_VERSION_FULL_VC3_BUILDER_HANDS=1.0.0 == env: VC3_VERSION_PKG_CONFIG=0.27 == env: VC3_ROOT_TAR=/usr/ == env: VC3_SH_PROFILE_WRAPPER=/disk/vc3-root/vc3-home/.vc3_sh-DnPqmi.wrapper == env: CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 == env: CC=gcc == env: CORES=1 == env: VC3_VERSION_FULL_MAKE=3.82.0 == env: CCTOOLS_RESOURCE_MONITOR_INFO=1029 == env: VC3_ROOT_VC3_BUILDER_HANDS=/disk/vc3-root/x86_64/redhat7.4.1708/vc3-builder-hands/v1.0.0 == env: VC3_MACHINE_ARCH=x86_64 == env: VC3_COMMAND_LINE=/disk/vc3-builder --var CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 --require cctools-wq-worker -- work_queue_worker -M lobster_kmohrman_EFT_CRC_postLHE_crc.* -t 300 -C catalog.cse.nd.edu,backup-catalog.cse.nd.edu -d all -o worker.log --cores=12 --memory=16000 --disk=17076 == env: _=/usr/bin/env ==== cpu info @ Fri Apr 16 00:13:28 EDT 2021 ==== == cpu: processor : 0 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 0 == cpu: cpu cores : 8 == cpu: apicid : 0 == cpu: initial apicid : 0 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.19 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 1 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 1 == cpu: cpu cores : 8 == cpu: apicid : 2 == cpu: initial apicid : 2 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.19 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 2 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 2 == cpu: cpu cores : 8 == cpu: apicid : 4 == cpu: initial apicid : 4 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.19 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 3 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 3 == cpu: cpu cores : 8 == cpu: apicid : 6 == cpu: initial apicid : 6 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.19 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 4 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 4 == cpu: cpu cores : 8 == cpu: apicid : 8 == cpu: initial apicid : 8 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.19 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 5 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3000.219 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 5 == cpu: cpu cores : 8 == cpu: apicid : 10 == cpu: initial apicid : 10 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.19 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 6 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3000.061 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 6 == cpu: cpu cores : 8 == cpu: apicid : 12 == cpu: initial apicid : 12 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.19 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 7 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 7 == cpu: cpu cores : 8 == cpu: apicid : 14 == cpu: initial apicid : 14 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.19 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 8 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 0 == cpu: cpu cores : 8 == cpu: apicid : 32 == cpu: initial apicid : 32 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.35 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 9 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 1 == cpu: cpu cores : 8 == cpu: apicid : 34 == cpu: initial apicid : 34 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.35 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 10 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 2 == cpu: cpu cores : 8 == cpu: apicid : 36 == cpu: initial apicid : 36 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.35 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 11 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3000.061 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 3 == cpu: cpu cores : 8 == cpu: apicid : 38 == cpu: initial apicid : 38 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.35 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 12 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 4 == cpu: cpu cores : 8 == cpu: apicid : 40 == cpu: initial apicid : 40 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.35 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 13 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3001.013 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 5 == cpu: cpu cores : 8 == cpu: apicid : 42 == cpu: initial apicid : 42 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.35 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 14 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 6 == cpu: cpu cores : 8 == cpu: apicid : 44 == cpu: initial apicid : 44 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.35 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 15 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 7 == cpu: cpu cores : 8 == cpu: apicid : 46 == cpu: initial apicid : 46 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.35 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: === using CVMFS proxy: http://eddie.crc.nd.edu:3128 @ Fri Apr 16 00:13:53 EDT 2021 === using Frontier proxy: http://eddie.crc.nd.edu:3128 @ Fri Apr 16 00:13:53 EDT 2021 === parrot helper: /disk/vc3-root/tmp/worker-230547-12913/t.167220/lib/libparrot_helper.so @ Fri Apr 16 00:13:58 EDT 2021 ==== content of /disk/vc3-root/tmp/worker-230547-12913/cache/tmp @ Fri Apr 16 00:13:58 EDT 2021 ==== == cache: total 28 == cache: drwx------ 265 kmohrman campus 8192 Apr 16 00:02 ex_parrot_kmohrman == cache: -rw-r--r-- 1 kmohrman campus 10 Apr 15 16:11 hot_cache == cache: drwxr-x--- 260 kmohrman campus 8192 Apr 15 14:29 cvmfs === OSG certificate location: /cvmfs/oasis.opensciencegrid.org/mis/certificates @ Fri Apr 16 00:14:03 EDT 2021 === testing parrot usage @ Fri Apr 16 00:14:03 EDT 2021 === parrot OK @ Fri Apr 16 00:14:08 EDT 2021 === starting parrot to access CMSSW... @ Fri Apr 16 00:14:08 EDT 2021 ==== wrapper started @ Fri Apr 16 00:14:08 EDT 2021 ==== == startup: hostname: d8civy008.crc.nd.edu == startup: kernel: Linux d8civy008.crc.nd.edu 3.10.0-1160.el7.x86_64 #1 SMP Tue Aug 18 14:50:17 EDT 2020 x86_64 x86_64 x86_64 GNU/Linux ==== tracing google @ Fri Apr 16 00:14:08 EDT 2021 ==== wrapper.sh: line 20: traceroute: command not found ==== environment at startup @ Fri Apr 16 00:14:08 EDT 2021 ==== == env: VC3_VERSION_SED=4.2 == env: CCTOOLS_RESOURCE_PROCESS_START=1618546448562212 == env: VC3_INSIDE_WRAPPER_SCRIPT=yes == env: PARROT_HTTP_PROXY=http://eddie.crc.nd.edu:3128 == env: TERM=xterm-256color == env: SHELL=/bin/sh == env: VC3_ROOT_GZIP=/usr/ == env: LOBSTER_GFAL_COPY= == env: MAKEFLAGS=-j4 == env: PARROT_HELPER=/disk/vc3-root/tmp/worker-230547-12913/t.167220/lib/libparrot_helper.so == env: TMPDIR=/disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL == env: VC3_ROOT_WHICH=/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0 == env: LD_PRELOAD=/disk/vc3-root/tmp/worker-230547-12913/t.167220/librmonitor_helper.so.zzOzh6 == env: LOBSTER_CVMFS_PROXY=http://eddie.crc.nd.edu:3128 == env: PARROT_PATH=./bin == env: LOBSTER_FRONTIER_PROXY=http://eddie.crc.nd.edu:3128 == env: LOBSTER_CMSSW_VERSION=CMSSW_10_6_20 == env: VC3_MACHINE_OS=Linux/redhat7.4.1708 == env: VC3_VERSION_FULL_TAR=1.26.0 == env: USER=kmohrman == env: VC3_VERSION_FULL_CCTOOLS_WQ_WORKER=999.999.999 == env: LD_LIBRARY_PATH=/.singularity.d/libs:lib == env: VC3_BUILDER_PATH=/disk/vc3-builder == env: TEMP=/disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL == env: VC3_VERSION_VC3_BUILDER_HANDS=1.0 == env: VC3_DISTFILES=/disk/vc3-distfiles == env: VC3_ROOT_MAKE=/usr/ == env: ENV=/disk/vc3-root/vc3-home/.vc3_sh-DnPqmi == env: VC3_VERSION_TAR=1.26 == env: WORKER_TMPDIR=/disk/vc3-root/tmp/worker-230547-12913/cache/tmp == env: FRONTIER_PROXY=http://eddie.crc.nd.edu:3128 == env: PARROT_ALLOW_SWITCHING_CVMFS_REPOSITORIES=TRUE == env: VC3_VERSION_CCTOOLS_WQ_WORKER=999.999 == env: VC3_VERSION_FULL_PKG_CONFIG=0.27.1 == env: PARROT_CVMFS_REPO= == env: VC3_VERSION_GZIP=1.5 == env: PATH=/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0/bin:/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin == env: VC3_ENV_TAG_GLOBAL=12661 == env: VC3_ROOT=/disk/vc3-root == env: VC3_SH_PROFILE_ENV=/disk/vc3-root/vc3-home/.vc3_sh-DnPqmi == env: PWD=/disk/vc3-root/tmp/worker-230547-12913/t.167220 == env: VC3_ROOT_PKG_CONFIG=/usr/ == env: PARROT_ENABLED=TRUE == env: VC3_VERSION_FULL_SED=4.2.2 == env: LOBSTER_OSG_VERSION=3.4 == env: VC3_VERSION_WHICH=2.20 == env: MODULEPATH=/afs/crc.nd.edu/x86_64_linux/Modules/modules/development_tools_and_libraries:/afs/crc.nd.edu/x86_64_linux/Modules/modules/general_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/restricted_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/system_modules:/afs/crc.nd.edu/x86_64_linux/Modules/modules/deprecated_software:/afs/crc.nd.edu/x86_64_linux/Modules/current/modulefiles == env: VC3_MACHINE_GLIBC_VERSION=2.17 == env: LOBSTER_LCG_CP= == env: WORK_QUEUE_SANDBOX=/disk/vc3-root/tmp/worker-230547-12913 == env: VC3_ROOT_CCTOOLS_WQ_WORKER=/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999 == env: VC3_MACHINE_TARGET=x86_64/redhat7.4.1708 == env: VC3_ROOT_SED=/usr/ == env: OASIS_CERTIFICATES=/cvmfs/oasis.opensciencegrid.org/mis/certificates == env: CXX=g++ == env: HOME=/disk/vc3-root/vc3-home == env: SHLVL=3 == env: VC3_VERSION_FULL_GZIP=1.5.0 == env: VC3_VERSION_MAKE=3.82 == env: DISK=361 == env: PARROT_CACHE=/disk/vc3-root/tmp/worker-230547-12913/cache/tmp == env: PARROT_CVMFS_ALIEN_CACHE=/disk/vc3-root/tmp/worker-230547-12913/cache/tmp/cvmfs == env: VC3_VERSION_FULL_WHICH=2.20.0 == env: MEMORY=1044 == env: HTTP_PROXY=http://eddie.crc.nd.edu:3128 == env: TMP=/disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL == env: PYTHONPATH=python: == env: VC3_INSTALL_USER_HOME=/disk/vc3-root/vc3-home == env: VC3_VERSION_FULL_VC3_BUILDER_HANDS=1.0.0 == env: VC3_ROOT_TAR=/usr/ == env: VC3_VERSION_PKG_CONFIG=0.27 == env: CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 == env: VC3_SH_PROFILE_WRAPPER=/disk/vc3-root/vc3-home/.vc3_sh-DnPqmi.wrapper == env: CC=gcc == env: CCTOOLS_RESOURCE_MONITOR_INFO=1029 == env: VC3_VERSION_FULL_MAKE=3.82.0 == env: CORES=1 == env: PARROT_DEBUG_FLAGS= == env: VC3_MACHINE_ARCH=x86_64 == env: VC3_ROOT_VC3_BUILDER_HANDS=/disk/vc3-root/x86_64/redhat7.4.1708/vc3-builder-hands/v1.0.0 == env: VC3_COMMAND_LINE=/disk/vc3-builder --var CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 --require cctools-wq-worker -- work_queue_worker -M lobster_kmohrman_EFT_CRC_postLHE_crc.* -t 300 -C catalog.cse.nd.edu,backup-catalog.cse.nd.edu -d all -o worker.log --cores=12 --memory=16000 --disk=17076 == env: _=/usr/bin/env ==== cpu info @ Fri Apr 16 00:14:08 EDT 2021 ==== == cpu: processor : 0 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 0 == cpu: cpu cores : 8 == cpu: apicid : 0 == cpu: initial apicid : 0 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.19 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 1 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 1 == cpu: cpu cores : 8 == cpu: apicid : 2 == cpu: initial apicid : 2 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.19 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 2 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3000.061 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 2 == cpu: cpu cores : 8 == cpu: apicid : 4 == cpu: initial apicid : 4 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.19 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 3 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 3 == cpu: cpu cores : 8 == cpu: apicid : 6 == cpu: initial apicid : 6 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.19 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 4 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 4 == cpu: cpu cores : 8 == cpu: apicid : 8 == cpu: initial apicid : 8 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.19 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 5 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 5 == cpu: cpu cores : 8 == cpu: apicid : 10 == cpu: initial apicid : 10 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.19 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 6 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 6 == cpu: cpu cores : 8 == cpu: apicid : 12 == cpu: initial apicid : 12 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.19 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 7 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 7 == cpu: cpu cores : 8 == cpu: apicid : 14 == cpu: initial apicid : 14 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.19 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 8 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 0 == cpu: cpu cores : 8 == cpu: apicid : 32 == cpu: initial apicid : 32 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.35 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 9 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 1 == cpu: cpu cores : 8 == cpu: apicid : 34 == cpu: initial apicid : 34 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.35 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 10 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2998.791 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 2 == cpu: cpu cores : 8 == cpu: apicid : 36 == cpu: initial apicid : 36 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.35 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 11 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 3 == cpu: cpu cores : 8 == cpu: apicid : 38 == cpu: initial apicid : 38 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.35 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 12 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 4 == cpu: cpu cores : 8 == cpu: apicid : 40 == cpu: initial apicid : 40 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.35 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 13 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 5 == cpu: cpu cores : 8 == cpu: apicid : 42 == cpu: initial apicid : 42 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.35 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 14 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 6 == cpu: cpu cores : 8 == cpu: apicid : 44 == cpu: initial apicid : 44 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.35 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 15 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 7 == cpu: cpu cores : 8 == cpu: apicid : 46 == cpu: initial apicid : 46 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.35 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: === using parrot @ Fri Apr 16 00:14:09 EDT 2021 === sourcing CMS setup @ Fri Apr 16 00:14:09 EDT 2021 LibCvmfs version 2.4, revision 25 === sourcing OSG setup @ Fri Apr 16 00:14:11 EDT 2021 ==== environment after sourcing startup scripts @ Fri Apr 16 00:14:11 EDT 2021 ==== == env: VC3_VERSION_SED=4.2 == env: MANPATH=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/share/man:/cvmfs/cms.cern.ch/share/man: == env: CCTOOLS_RESOURCE_PROCESS_START=1618546451088026 == env: VC3_INSIDE_WRAPPER_SCRIPT=yes == env: GFAL_PLUGIN_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/gfal2-plugins/ == env: PARROT_HTTP_PROXY=http://eddie.crc.nd.edu:3128 == env: TERM=xterm-256color == env: SHELL=/bin/sh == env: VC3_ROOT_GZIP=/usr/ == env: LOBSTER_GFAL_COPY=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/bin/gfal-copy == env: VOMS_USERCONF=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/etc/vomses == env: MAKEFLAGS=-j4 == env: GLOBUS_LOCATION=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr == env: TMPDIR=/disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL == env: PERL5LIB=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/share/perl5/vendor_perl:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/share/perl5:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/perl5/vendor_perl:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/perl5 == env: VC3_ROOT_WHICH=/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0 == env: CVSROOT=:gserver:cmssw.cvs.cern.ch:/local/reps/CMSSW == env: LD_PRELOAD=/disk/vc3-root/tmp/worker-230547-12913/t.167220/librmonitor_helper.so.zzOzh6 == env: LOBSTER_CVMFS_PROXY=http://eddie.crc.nd.edu:3128 == env: PARROT_PATH=./bin == env: LOBSTER_FRONTIER_PROXY=http://eddie.crc.nd.edu:3128 == env: X509_CERT_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/etc/grid-security/certificates == env: SCRAM_ARCH=slc7_amd64_gcc820 == env: LOBSTER_CMSSW_VERSION=CMSSW_10_6_20 == env: VC3_MACHINE_OS=Linux/redhat7.4.1708 == env: VC3_VERSION_FULL_TAR=1.26.0 == env: USER=kmohrman == env: VC3_VERSION_FULL_CCTOOLS_WQ_WORKER=999.999.999 == env: LD_LIBRARY_PATH=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/dcap:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/lcgdm:/.singularity.d/libs:lib:lib == env: VC3_BUILDER_PATH=/disk/vc3-builder == env: TEMP=/disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL == env: VC3_VERSION_VC3_BUILDER_HANDS=1.0 == env: VC3_DISTFILES=/disk/vc3-distfiles == env: VC3_ROOT_MAKE=/usr/ == env: OSG_LOCATION=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64 == env: ENV=/disk/vc3-root/vc3-home/.vc3_sh-DnPqmi == env: VC3_VERSION_TAR=1.26 == env: WORKER_TMPDIR=/disk/vc3-root/tmp/worker-230547-12913/cache/tmp == env: FRONTIER_PROXY=http://eddie.crc.nd.edu:3128 == env: PARROT_ALLOW_SWITCHING_CVMFS_REPOSITORIES=TRUE == env: VC3_VERSION_CCTOOLS_WQ_WORKER=999.999 == env: VC3_VERSION_FULL_PKG_CONFIG=0.27.1 == env: SITECONFIG_PATH=/cvmfs/cms.cern.ch/SITECONF/local == env: PARROT_CVMFS_REPO= == env: VC3_VERSION_GZIP=1.5 == env: PATH=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/bin:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/sbin:/cvmfs/cms.cern.ch/common:/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0/bin:/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin == env: VC3_ENV_TAG_GLOBAL=12661 == env: CMS_PATH=/cvmfs/cms.cern.ch == env: VC3_ROOT=/disk/vc3-root == env: VC3_SH_PROFILE_ENV=/disk/vc3-root/vc3-home/.vc3_sh-DnPqmi == env: PWD=/disk/vc3-root/tmp/worker-230547-12913/t.167220 == env: VC3_ROOT_PKG_CONFIG=/usr/ == env: PARROT_ENABLED=TRUE == env: VC3_VERSION_FULL_SED=4.2.2 == env: LOBSTER_OSG_VERSION=3.4 == env: VC3_VERSION_WHICH=2.20 == env: MODULEPATH=/afs/crc.nd.edu/x86_64_linux/Modules/modules/development_tools_and_libraries:/afs/crc.nd.edu/x86_64_linux/Modules/modules/general_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/restricted_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/system_modules:/afs/crc.nd.edu/x86_64_linux/Modules/modules/deprecated_software:/afs/crc.nd.edu/x86_64_linux/Modules/current/modulefiles == env: VC3_MACHINE_GLIBC_VERSION=2.17 == env: X509_VOMS_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/etc/grid-security/vomsdir == env: LOBSTER_LCG_CP= == env: WORK_QUEUE_SANDBOX=/disk/vc3-root/tmp/worker-230547-12913 == env: VC3_ROOT_CCTOOLS_WQ_WORKER=/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999 == env: VC3_MACHINE_TARGET=x86_64/redhat7.4.1708 == env: VC3_ROOT_SED=/usr/ == env: OASIS_CERTIFICATES=/cvmfs/oasis.opensciencegrid.org/mis/certificates == env: CXX=g++ == env: HOME=/disk/vc3-root/vc3-home == env: SHLVL=3 == env: VC3_VERSION_FULL_GZIP=1.5.0 == env: VC3_VERSION_MAKE=3.82 == env: DISK=361 == env: PARROT_CACHE=/disk/vc3-root/tmp/worker-230547-12913/cache/tmp == env: PARROT_CVMFS_ALIEN_CACHE=/disk/vc3-root/tmp/worker-230547-12913/cache/tmp/cvmfs == env: GFAL_CONFIG_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/etc/gfal2.d/ == env: VC3_VERSION_FULL_WHICH=2.20.0 == env: MEMORY=1044 == env: HTTP_PROXY=http://eddie.crc.nd.edu:3128 == env: TMP=/disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL == env: PYTHONPATH=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib/python2.7/site-packages:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/python2.7/site-packages:python:python: == env: VC3_INSTALL_USER_HOME=/disk/vc3-root/vc3-home == env: VC3_VERSION_FULL_VC3_BUILDER_HANDS=1.0.0 == env: VC3_ROOT_TAR=/usr/ == env: VC3_VERSION_PKG_CONFIG=0.27 == env: CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 == env: VC3_SH_PROFILE_WRAPPER=/disk/vc3-root/vc3-home/.vc3_sh-DnPqmi.wrapper == env: CC=gcc == env: CCTOOLS_RESOURCE_MONITOR_INFO=1029 == env: VC3_VERSION_FULL_MAKE=3.82.0 == env: CORES=1 == env: PARROT_DEBUG_FLAGS= == env: VC3_MACHINE_ARCH=x86_64 == env: VC3_ROOT_VC3_BUILDER_HANDS=/disk/vc3-root/x86_64/redhat7.4.1708/vc3-builder-hands/v1.0.0 == env: VC3_COMMAND_LINE=/disk/vc3-builder --var CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 --require cctools-wq-worker -- work_queue_worker -M lobster_kmohrman_EFT_CRC_postLHE_crc.* -t 300 -C catalog.cse.nd.edu,backup-catalog.cse.nd.edu -d all -o worker.log --cores=12 --memory=16000 --disk=17076 == env: _=/usr/bin/env ==== proxy information @ Fri Apr 16 00:14:11 EDT 2021 ==== == proxy: subject : /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=kmohrman/CN=825714/CN=Kelci Ann Mohrman/CN=3380169703 == proxy: issuer : /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=kmohrman/CN=825714/CN=Kelci Ann Mohrman == proxy: identity : /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=kmohrman/CN=825714/CN=Kelci Ann Mohrman == proxy: type : RFC compliant proxy == proxy: strength : 1024 bits == proxy: path : proxy == proxy: timeleft : 163:28:08 ==== working directory at startup @ Fri Apr 16 00:14:11 EDT 2021 ==== == dir: total 872 == dir: -rwxr-xr-x 2 kmohrman campus 1575 Apr 16 00:13 0008_7320_report.json == dir: -rwxr-xr-x 2 kmohrman campus 1630 Apr 16 00:13 0013_6164_report.json == dir: -rwxr-xr-x 2 kmohrman campus 1462 Apr 16 00:13 0016_0941_report.json == dir: drwxr-xr-x 2 kmohrman campus 71 Apr 16 00:13 bin == dir: -rwxr-xr-x 7 kmohrman campus 627272 Apr 15 14:28 cctools-monitor == dir: -rw-r--r-- 1 kmohrman campus 0 Apr 16 00:13 cctools-monitor.summary == dir: drwxrwxrwx 2 kmohrman campus 6 Apr 16 00:13 cctools-temp-t.167220.2PbMPL == dir: -rw------- 1 kmohrman campus 51 Apr 16 00:14 cvmfschecksum.cms.cern.ch == dir: -rw------- 1 kmohrman campus 51 Apr 16 00:14 cvmfschecksum.oasis.opensciencegrid.org == dir: drwxr-xr-x 2 kmohrman campus 33 Apr 16 00:13 lib == dir: -rwxrwxrwx 1 kmohrman campus 146912 Apr 16 00:13 librmonitor_helper.so.zzOzh6 == dir: -rw-r--r-- 2 kmohrman campus 868 Apr 15 23:26 merge_cfg.py == dir: -rw-r--r-- 2 kmohrman campus 808 Apr 15 23:26 merge_reports.py == dir: -rw-r--r-- 2 kmohrman campus 2354 Apr 16 00:13 parameters.json == dir: -rw------- 2 kmohrman campus 9917 Apr 16 00:13 proxy == dir: drwxr-xr-x 3 kmohrman campus 20 Apr 16 00:13 python == dir: -rw-r--r-- 7 kmohrman campus 570 Apr 15 14:28 report.json.in == dir: -rw-r--r-- 2 kmohrman campus 549 Apr 15 23:26 sandbox-CMSSW_10_6_20-slc7_amd64_gcc700.tar.bz2 == dir: drwxr-xr-x 4 kmohrman campus 37 Apr 16 00:13 siteconf == dir: -rw-r--r-- 1 kmohrman campus 11 Apr 16 00:14 t_wrapper_start == dir: -rwxr-xr-x 7 kmohrman campus 47217 Apr 15 14:28 task.py == dir: -rwxr-xr-x 7 kmohrman campus 5986 Apr 15 14:28 wrapper.sh === creating new release CMSSW_10_6_20 for scram arch slc7_amd64_gcc700 @ Fri Apr 16 00:14:11 EDT 2021 WARNING: In non-interactive mode release checks e.g. deprecated releases, production architectures are disabled. === unpacking sandbox-CMSSW_10_6_20-slc7_amd64_gcc700.tar.bz2 @ Fri Apr 16 00:14:15 EDT 2021 ==== machine load @ Fri Apr 16 00:14:17 EDT 2021 ==== top: unknown option 'M' Usage: top -hv | -bcHiOSs -d secs -n max -u|U user -p pid(s) -o field -w [cols] ==== environment before execution @ Fri Apr 16 00:14:17 EDT 2021 ==== == env: G4LEVELGAMMADATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4PhotonEvaporation/5.2/data/PhotonEvaporation5.2 == env: UTM_XSD_DIR=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/utm/utm_0.7.1-pafccj == env: SRT_G4LEVELGAMMADATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4PhotonEvaporation/5.2/data/PhotonEvaporation5.2 == env: VC3_VERSION_SED=4.2 == env: CMSSW_RELEASE_BASE=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20 == env: SCRAMRT_SET=CMSSW:CMSSW_10_6_20:slc7_amd64_gcc700:V2_2_9_pre13:SRT_ == env: MANPATH=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/share/man:/cvmfs/cms.cern.ch/share/man: == env: CCTOOLS_RESOURCE_PROCESS_START=1618546457520146 == env: CMSSW_FWLITE_INCLUDE_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sherpa/2.2.10-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/hepmc/2.06.07-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/clhep/2.4.0.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/boost/1.67.0-pafccj/include == env: SRT_GCC_RUNTIME_ASAN_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib64/libasan.so == env: VC3_INSIDE_WRAPPER_SCRIPT=yes == env: ROOT_INCLUDE_PATH=/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20/src:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/src:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/coral/CORAL_2_3_21-pafccj7/include/LCG:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/mctester/1.25.0a-pafccj9/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/lcg/root/6.14.09-pafccj6/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/dd4hep/v01-10x-pafccj9/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/QtDesigner:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/herwigpp/7.2.0-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/tauolapp/1.1.5-pafccj5/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/charybdis/1.003-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sherpa/2.2.10-pafccj2/include/SHERPA-MC:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/QtOpenGL:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/QtGui:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/thepeg/2.2.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/pythia8/240-pafccj5/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/herwig/6.521-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/rivet/2.7.0-pafccj8/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/Qt3Support:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/lwtnn/2.4-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4/10.4.3-pafccj2/include/Geant4:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/classlib/3.1.3-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/lhapdf/6.2.1-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/cgal/4.2-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/tkonlinesw/4.2.0-1_gcc7-pafccj9/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/starlight/r193-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/Qt:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/QtCore:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/QtXml:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/mcdb/1.0.3-pafccj/interface:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libungif/4.1.4-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libtiff/4.0.3-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/highfive/2.0/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libpng/1.6.16-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/frontier_client/2.9.1/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/pcre/8.37-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/boost/1.67.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/xrootd/4.8.5-pafccj2/include/xrootd/private:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/vdt/0.4.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/valgrind/3.13.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/utm/utm_0.7.1-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/toprex/4.23-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/tbb/2019_U3-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/tauola/27.121.5-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sigcpp/2.6.2-pafccj/include/sigc++-2.0:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sqlite/3.22.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/protobuf/3.5.2-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/pacparser/1.3.5-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/oracle/12.1.0.2.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/py2-numpy/1.16.2-nmpfii2/c-api/core/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/meschach/1.2.pCMS1-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/lz4/1.9.2-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libuuid/2.22.2-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libunwind/1.2.1-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libhepml/0.2.1-pafccj/interface:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/ktjet/1.06-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/jimmy/4.2-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/jemalloc/5.1.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/hls/2019.08/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/madgraph5amcatnlo/2.6.0-pafccj12:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/heppdt/3.03.00-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/hector/1.3.4_patch1-pafccj9/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libjpeg-turbo/2.0.1-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/giflib/4.2.3-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gsl/2.2.1-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gdbm/1.10-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/freetype/2.5.3-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/fftw3/3.3.2-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/fftjet/1.5.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/fastjet/3.3.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/expat/2.1.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/dpm/1.8.0.1-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/hepmc/2.06.07-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/vecgeom/v00.05.00-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/xerces-c/3.1.3-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/xz/5.2.2-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/dcap/2.47.8-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libxml2/2.9.1-pafccj/include/libxml2:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/curl/7.59.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/cppunit/1.40.1-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/clhep/2.4.0.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/openssl/1.0.2d-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/pythia6/426-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/photos/215.5-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/zlib-x86_64/1.2.11-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/cascade/2.2.04-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/bz2lib/1.0.6-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/python/2.7.14-pafccj/include/python2.7:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/tinyxml2/6.2.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/ittnotify/16.06.18-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gosamcontrib/2.0-20150803-pafccj/include:/usr/local/include:/usr/include == env: INTEL_LICENSE_FILE=28518@lxlicen01.cern.ch,28518@lxlicen02.cern.ch,28518@lxlicen03.cern.ch == env: SRT_TEXMFHOME_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/rivet/2.7.0-pafccj8/share/Rivet/texmf == env: SRT_GEANT4_PARFULLCMS_CONFIG_DIR_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-parfullcms/2014.01.27-pafccj2/share/ParFullCMS == env: GFAL_PLUGIN_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/gfal2-plugins/ == env: PARROT_HTTP_PROXY=http://eddie.crc.nd.edu:3128 == env: TERM=xterm-256color == env: SHELL=/bin/sh == env: VC3_ROOT_GZIP=/usr/ == env: LOBSTER_GFAL_COPY=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/bin/gfal-copy == env: PDFPATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/rivet/2.7.0-pafccj8/share == env: SRT_TNS_ADMIN_SCRAMRTDEL=/etc == env: VOMS_USERCONF=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/etc/vomses == env: MAKEFLAGS=-j4 == env: ROOT_GCC_TOOLCHAIN=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj == env: GLOBUS_LOCATION=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr == env: TMPDIR=/disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL == env: G4FORCENUMBEROFTHREADS=max == env: OPENBLAS_NUM_THREADS=1 == env: SRT_PDFPATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/rivet/2.7.0-pafccj8/share == env: PERL5LIB=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/git/2.19.0-pafccj/share/perl5:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/share/perl5/vendor_perl:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/share/perl5:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/perl5/vendor_perl:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/perl5 == env: VC3_ROOT_WHICH=/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0 == env: CMSSW_DATA_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms == env: LHAPDF_DATA_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/lhapdf/6.2.1-pafccj3/share/LHAPDF == env: CVSROOT=:gserver:cmssw.cvs.cern.ch:/local/reps/CMSSW == env: LD_PRELOAD=/disk/vc3-root/tmp/worker-230547-12913/t.167220/librmonitor_helper.so.zzOzh6 == env: LOBSTER_CVMFS_PROXY=http://eddie.crc.nd.edu:3128 == env: PARROT_PATH=./bin == env: CMSSW_GIT_HASH=CMSSW_10_6_20 == env: SRT_G4ABLADATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4ABLA/3.0/data/G4ABLA3.0 == env: QTDIR=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2 == env: SRT_SCONS_LIB_DIR_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/scons/3.0.1-pafccj/lib == env: OLDPWD=/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20 == env: LOBSTER_FRONTIER_PROXY=http://eddie.crc.nd.edu:3128 == env: G4NEUTRONXSDATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4NEUTRONXS/1.4/data/G4NEUTRONXS1.4 == env: QTINC=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include == env: X509_CERT_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/etc/grid-security/certificates == env: SCRAM_ARCH=slc7_amd64_gcc700 == env: GIT_SSL_CAINFO=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/git/2.19.0-pafccj/share/ssl/certs/ca-bundle.crt == env: SRT_CMS_OPENLOOPS_PREFIX_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/openloops/2.1.0-pafccj2 == env: SRT_GIT_EXEC_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/git/2.19.0-pafccj/libexec/git-core == env: SRT_THEPEGPATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/thepeg/2.2.0-pafccj2/share/ThePEG == env: LOBSTER_CMSSW_VERSION=CMSSW_10_6_20 == env: ROOT_TTREECACHE_SIZE=0 == env: EVTGENDATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/evtgen/1.6.0-pafccj5/share == env: SRT_INTEL_LICENSE_FILE_SCRAMRTDEL=28518@lxlicen01.cern.ch,28518@lxlicen02.cern.ch,28518@lxlicen03.cern.ch == env: SRT_ROOT_INCLUDE_PATH_SCRAMRTDEL=/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20/src:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/src:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/coral/CORAL_2_3_21-pafccj7/include/LCG:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/mctester/1.25.0a-pafccj9/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/lcg/root/6.14.09-pafccj6/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/dd4hep/v01-10x-pafccj9/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/QtDesigner:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/herwigpp/7.2.0-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/tauolapp/1.1.5-pafccj5/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/charybdis/1.003-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sherpa/2.2.10-pafccj2/include/SHERPA-MC:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/QtOpenGL:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/QtGui:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/thepeg/2.2.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/pythia8/240-pafccj5/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/herwig/6.521-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/rivet/2.7.0-pafccj8/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/Qt3Support:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/lwtnn/2.4-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4/10.4.3-pafccj2/include/Geant4:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/classlib/3.1.3-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/lhapdf/6.2.1-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/cgal/4.2-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/tkonlinesw/4.2.0-1_gcc7-pafccj9/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/starlight/r193-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/Qt:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/QtCore:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/QtXml:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/mcdb/1.0.3-pafccj/interface:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libungif/4.1.4-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libtiff/4.0.3-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/highfive/2.0/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libpng/1.6.16-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/frontier_client/2.9.1/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/pcre/8.37-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/boost/1.67.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/xrootd/4.8.5-pafccj2/include/xrootd/private:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/vdt/0.4.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/valgrind/3.13.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/utm/utm_0.7.1-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/toprex/4.23-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/tbb/2019_U3-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/tauola/27.121.5-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sigcpp/2.6.2-pafccj/include/sigc++-2.0:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sqlite/3.22.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/protobuf/3.5.2-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/pacparser/1.3.5-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/oracle/12.1.0.2.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/py2-numpy/1.16.2-nmpfii2/c-api/core/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/meschach/1.2.pCMS1-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/lz4/1.9.2-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libuuid/2.22.2-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libunwind/1.2.1-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libhepml/0.2.1-pafccj/interface:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/ktjet/1.06-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/jimmy/4.2-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/jemalloc/5.1.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/hls/2019.08/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/madgraph5amcatnlo/2.6.0-pafccj12:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/heppdt/3.03.00-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/hector/1.3.4_patch1-pafccj9/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libjpeg-turbo/2.0.1-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/giflib/4.2.3-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gsl/2.2.1-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gdbm/1.10-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/freetype/2.5.3-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/fftw3/3.3.2-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/fftjet/1.5.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/fastjet/3.3.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/expat/2.1.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/dpm/1.8.0.1-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/hepmc/2.06.07-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/vecgeom/v00.05.00-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/xerces-c/3.1.3-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/xz/5.2.2-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/dcap/2.47.8-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libxml2/2.9.1-pafccj/include/libxml2:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/curl/7.59.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/cppunit/1.40.1-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/clhep/2.4.0.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/openssl/1.0.2d-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/pythia6/426-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/photos/215.5-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/zlib-x86_64/1.2.11-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/cascade/2.2.04-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/bz2lib/1.0.6-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/python/2.7.14-pafccj/include/python2.7:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/tinyxml2/6.2.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/ittnotify/16.06.18-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gosamcontrib/2.0-20150803-pafccj/include:/usr/local/include:/usr/include == env: VC3_MACHINE_OS=Linux/redhat7.4.1708 == env: VC3_VERSION_FULL_TAR=1.26.0 == env: G4LEDATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4EMLOW/7.3/data/G4EMLOW7.3 == env: SRT_VINCIADATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/vincia/2.2.04-pafccj5/share/Vincia/xmldoc == env: SRT_ROOT_TTREECACHE_PREFILL_SCRAMRTDEL=0 == env: OPAL_PREFIX=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/openmpi/2.1.5-pafccj == env: USER=kmohrman == env: VC3_VERSION_FULL_CCTOOLS_WQ_WORKER=999.999.999 == env: ROOT_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/madgraph5amcatnlo/2.6.0-pafccj12:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gosamcontrib/2.0-20150803-pafccj == env: SHERPA_INCLUDE_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sherpa/2.2.10-pafccj2/include/SHERPA-MC == env: SRT_LHAPDF_DATA_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/lhapdf/6.2.1-pafccj3/share/LHAPDF == env: SRT_PYTHON27PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/share/overrides/python:/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20/python:/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/python:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/coral/CORAL_2_3_21-pafccj7/slc7_amd64_gcc700/python:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/coral/CORAL_2_3_21-pafccj7/slc7_amd64_gcc700/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/xrootd/4.8.5-pafccj2/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/fastjet/3.3.0-pafccj/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw-tool-conf/45.0-pafccj26/lib/python2.7/site-packages == env: LD_LIBRARY_PATH=/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20/biglib/slc7_amd64_gcc700:/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20/external/slc7_amd64_gcc700/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/biglib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/external/slc7_amd64_gcc700/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/llvm/7.1.0-pafccj/lib64:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib64:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/cuda/10.1.105-pafccj2/drivers:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/dcap:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/lcgdm:/.singularity.d/libs:lib:lib == env: VC3_BUILDER_PATH=/disk/vc3-builder == env: TEMP=/disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL == env: SRT_CMSSW_RELEASE_BASE_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20 == env: SRT_RIVET_ANALYSIS_PATH_SCRAMRTDEL=/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/rivet/2.7.0-pafccj8/lib == env: VC3_VERSION_VC3_BUILDER_HANDS=1.0 == env: SRT_PERL5LIB_SCRAMRT=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/git/2.19.0-pafccj/share/perl5 == env: SRT_CMSSW_VERSION_SCRAMRTDEL=CMSSW_10_6_20 == env: VC3_DISTFILES=/disk/vc3-distfiles == env: VC3_ROOT_MAKE=/usr/ == env: GCC_RUNTIME_LSAN=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib64/libasan.so == env: SCRAM_CXX11_ABI=1 == env: VALGRIND_LIB=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/valgrind/3.13.0-pafccj/lib/valgrind == env: SRT_CMSSW_GIT_HASH_SCRAMRTDEL=CMSSW_10_6_20 == env: SRT_ROOTSYS_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/lcg/root/6.14.09-pafccj6/ == env: OSG_LOCATION=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64 == env: ENV=/disk/vc3-root/vc3-home/.vc3_sh-DnPqmi == env: VC3_VERSION_TAR=1.26 == env: WORKER_TMPDIR=/disk/vc3-root/tmp/worker-230547-12913/cache/tmp == env: G4NEUTRONHPDATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4NDL/4.5/data/G4NDL4.5 == env: GEANT4_PARFULLCMS_CONFIG_DIR=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-parfullcms/2014.01.27-pafccj2/share/ParFullCMS == env: FRONTIER_PROXY=http://eddie.crc.nd.edu:3128 == env: PARROT_ALLOW_SWITCHING_CVMFS_REPOSITORIES=TRUE == env: CSC_TRACK_FINDER_DATA_DIR=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/CSCTrackFinderEmulation/1.2-pafccj/data/ == env: SRT_QTINC_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include == env: VC3_VERSION_CCTOOLS_WQ_WORKER=999.999 == env: G4ENSDFSTATEDATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4ENSDFSTATE/2.2/data/G4ENSDFSTATE2.2 == env: G4RADIOACTIVEDATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4RadioactiveDecay/5.2/data/RadioactiveDecay5.2 == env: ROOFITSYS=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/lcg/root/6.14.09-pafccj6/ == env: SRT_ROOFITSYS_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/lcg/root/6.14.09-pafccj6/ == env: SRT_G4RADIOACTIVEDATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4RadioactiveDecay/5.2/data/RadioactiveDecay5.2 == env: SRT_G4ENSDFSTATEDATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4ENSDFSTATE/2.2/data/G4ENSDFSTATE2.2 == env: SRT_CLHEP_PARAM_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/clhep/2.4.0.0-pafccj2 == env: G4ABLADATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4ABLA/3.0/data/G4ABLA3.0 == env: VC3_VERSION_FULL_PKG_CONFIG=0.27.1 == env: SRT_SHERPA_INCLUDE_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sherpa/2.2.10-pafccj2/include/SHERPA-MC == env: SRT_G4FORCENUMBEROFTHREADS_SCRAMRTDEL=max == env: SRT_WORKER_DATA_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/blackhat/0.9.9-pafccj/share/blackhat/datafiles/ == env: SITECONFIG_PATH=/cvmfs/cms.cern.ch/SITECONF/local == env: PARROT_CVMFS_REPO= == env: VC3_VERSION_GZIP=1.5 == env: GCC_RUNTIME_TSAN=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib64/libtsan.so == env: TNS_ADMIN=/etc == env: SRT_HEPPDT_PARAM_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/heppdt/3.03.00-pafccj == env: PATH=/cvmfs/cms.cern.ch/share/overrides/bin:/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20/bin/slc7_amd64_gcc700:/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20/external/slc7_amd64_gcc700/bin:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/bin/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/external/slc7_amd64_gcc700/bin:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/llvm/7.1.0-pafccj/bin:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/bin:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/bin:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/sbin:/cvmfs/cms.cern.ch/common:/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0/bin:/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin == env: VC3_ENV_TAG_GLOBAL=12661 == env: THEPEGPATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/thepeg/2.2.0-pafccj2/share/ThePEG == env: SRT_CMSSW_SEARCH_PATH_SCRAMRTDEL=/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20/poison:/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20/src:/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20/external/slc7_amd64_gcc700/data:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/src:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/external/slc7_amd64_gcc700/data == env: SRT_GIT_SSL_CAINFO_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/git/2.19.0-pafccj/share/ssl/certs/ca-bundle.crt == env: SRT_GCC_RUNTIME_LSAN_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib64/libasan.so == env: CMS_PATH=/cvmfs/cms.cern.ch == env: VC3_ROOT=/disk/vc3-root == env: SRT_UTM_XSD_DIR_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/utm/utm_0.7.1-pafccj == env: SRT_PATH_SCRAMRT=/cvmfs/cms.cern.ch/share/overrides/bin:/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20/bin/slc7_amd64_gcc700:/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20/external/slc7_amd64_gcc700/bin:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/bin/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/external/slc7_amd64_gcc700/bin:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/llvm/7.1.0-pafccj/bin:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/bin == env: VC3_SH_PROFILE_ENV=/disk/vc3-root/vc3-home/.vc3_sh-DnPqmi == env: RIVET_ANALYSIS_PATH=/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/rivet/2.7.0-pafccj8/lib == env: SRT_LOCALRT_SCRAMRTDEL=/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20 == env: SRT_CMSSW_BASE_SCRAMRTDEL=/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20 == env: PWD=/disk/vc3-root/tmp/worker-230547-12913/t.167220 == env: VC3_ROOT_PKG_CONFIG=/usr/ == env: PARROT_ENABLED=TRUE == env: VC3_VERSION_FULL_SED=4.2.2 == env: LOBSTER_OSG_VERSION=3.4 == env: PYTHON_VALGRIND_SUPP=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/python/2.7.14-pafccj/share/valgrind/valgrind-python.supp == env: SRT_OPAL_PREFIX_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/openmpi/2.1.5-pafccj == env: VC3_VERSION_WHICH=2.20 == env: GCC_RUNTIME_UBSAN=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib64/libubsan.so == env: LANG=C == env: SRT_QTDIR_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2 == env: MODULEPATH=/afs/crc.nd.edu/x86_64_linux/Modules/modules/development_tools_and_libraries:/afs/crc.nd.edu/x86_64_linux/Modules/modules/general_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/restricted_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/system_modules:/afs/crc.nd.edu/x86_64_linux/Modules/modules/deprecated_software:/afs/crc.nd.edu/x86_64_linux/Modules/current/modulefiles == env: VC3_MACHINE_GLIBC_VERSION=2.17 == env: SRT_G4NEUTRONHPDATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4NDL/4.5/data/G4NDL4.5 == env: X509_VOMS_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/etc/grid-security/vomsdir == env: LOBSTER_LCG_CP= == env: SRT_SHERPA_SHARE_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sherpa/2.2.10-pafccj2/share/SHERPA-MC == env: SRT_GCC_RUNTIME_TSAN_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib64/libtsan.so == env: WORK_QUEUE_SANDBOX=/disk/vc3-root/tmp/worker-230547-12913 == env: CASCADE_PDFPATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/cascade/2.2.04-pafccj3/share == env: SRT_ROOT_GCC_TOOLCHAIN_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj == env: SRT_CASCADE_PDFPATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/cascade/2.2.04-pafccj3/share == env: WORKER_DATA_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/blackhat/0.9.9-pafccj/share/blackhat/datafiles/ == env: GIT_TEMPLATE_DIR=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/git/2.19.0-pafccj/share/git-core/templates == env: G4SAIDXSDATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4SAIDDATA/1.1/data/G4SAIDDATA1.1 == env: MLAS_DYNAMIC_CPU_ARCH=0 == env: VC3_ROOT_CCTOOLS_WQ_WORKER=/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999 == env: PYTHON27PATH=/cvmfs/cms.cern.ch/share/overrides/python:/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20/python:/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/python:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/coral/CORAL_2_3_21-pafccj7/slc7_amd64_gcc700/python:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/coral/CORAL_2_3_21-pafccj7/slc7_amd64_gcc700/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/xrootd/4.8.5-pafccj2/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/fastjet/3.3.0-pafccj/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw-tool-conf/45.0-pafccj26/lib/python2.7/site-packages == env: SRT_CMSSW_FWLITE_INCLUDE_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sherpa/2.2.10-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/hepmc/2.06.07-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/clhep/2.4.0.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/boost/1.67.0-pafccj/include == env: VC3_MACHINE_TARGET=x86_64/redhat7.4.1708 == env: VC3_ROOT_SED=/usr/ == env: OASIS_CERTIFICATES=/cvmfs/oasis.opensciencegrid.org/mis/certificates == env: ROOTSYS=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/lcg/root/6.14.09-pafccj6/ == env: SRT_PYTHIA8DATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/pythia8/240-pafccj5/share/Pythia8/xmldoc == env: SRT_HERWIGPATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/herwigpp/7.2.0-pafccj3/share/Herwig == env: CXX=g++ == env: SRT_QTLIB_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/lib == env: SRT_PYTHON3PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/share/overrides/python:/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20/python:/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/python:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw-tool-conf/45.0-pafccj26/lib/python3.6/site-packages == env: SHERPA_LIBRARY_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sherpa/2.2.10-pafccj2/lib/SHERPA-MC == env: SRT_PYTHON_VALGRIND_SUPP_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/python/2.7.14-pafccj/share/valgrind/valgrind-python.supp == env: HOME=/disk/vc3-root/vc3-home == env: SHLVL=3 == env: CMSSW_SEARCH_PATH=/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20/poison:/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20/src:/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20/external/slc7_amd64_gcc700/data:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/src:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/external/slc7_amd64_gcc700/data == env: VC3_VERSION_FULL_GZIP=1.5.0 == env: VC3_VERSION_MAKE=3.82 == env: GENEVA_DATA_DIR=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geneva/1.0-RC3-pafccj8/share/Geneva == env: DISK=361 == env: PARROT_CACHE=/disk/vc3-root/tmp/worker-230547-12913/cache/tmp == env: PARROT_CVMFS_ALIEN_CACHE=/disk/vc3-root/tmp/worker-230547-12913/cache/tmp/cvmfs == env: GOSAMCONTRIB_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gosamcontrib/2.0-20150803-pafccj == env: ROOT_TTREECACHE_PREFILL=0 == env: GENREFLEX=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/lcg/root/6.14.09-pafccj6/bin/genreflex == env: VINCIADATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/vincia/2.2.04-pafccj5/share/Vincia/xmldoc == env: SRT_G4LEDATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4EMLOW/7.3/data/G4EMLOW7.3 == env: SRT_GENREFLEX_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/lcg/root/6.14.09-pafccj6/bin/genreflex == env: SRT_GOSAMCONTRIB_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gosamcontrib/2.0-20150803-pafccj == env: SRT_ROOT_TTREECACHE_SIZE_SCRAMRTDEL=0 == env: SRT_EVTGENDATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/evtgen/1.6.0-pafccj5/share == env: SRT_GENEVA_DATA_DIR_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geneva/1.0-RC3-pafccj8/share/Geneva == env: GFAL_CONFIG_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/etc/gfal2.d/ == env: VC3_VERSION_FULL_WHICH=2.20.0 == env: MEMORY=1044 == env: HTTP_PROXY=http://eddie.crc.nd.edu:3128 == env: SRT_CMSSW_DATA_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms == env: HEPPDT_PARAM_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/heppdt/3.03.00-pafccj == env: SRT_LD_LIBRARY_PATH_SCRAMRT=/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20/biglib/slc7_amd64_gcc700:/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20/external/slc7_amd64_gcc700/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/biglib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/external/slc7_amd64_gcc700/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/llvm/7.1.0-pafccj/lib64:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib64:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/cuda/10.1.105-pafccj2/drivers == env: TMP=/disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL == env: PYTHONPATH=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib/python2.7/site-packages:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/python2.7/site-packages:python:python: == env: PYTHON3PATH=/cvmfs/cms.cern.ch/share/overrides/python:/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20/python:/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/python:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw-tool-conf/45.0-pafccj26/lib/python3.6/site-packages == env: QTLIB=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/lib == env: SCONS_LIB_DIR=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/scons/3.0.1-pafccj/lib == env: VC3_INSTALL_USER_HOME=/disk/vc3-root/vc3-home == env: VC3_VERSION_FULL_VC3_BUILDER_HANDS=1.0.0 == env: COMPILER_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj == env: FRONTIER_CLIENT=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/frontier_client/2.9.1/ == env: SRT_G4NEUTRONXSDATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4NEUTRONXS/1.4/data/G4NEUTRONXS1.4 == env: SRT_SHERPA_LIBRARY_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sherpa/2.2.10-pafccj2/lib/SHERPA-MC == env: VC3_ROOT_TAR=/usr/ == env: VC3_VERSION_PKG_CONFIG=0.27 == env: CLHEP_PARAM_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/clhep/2.4.0.0-pafccj2 == env: CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 == env: VC3_SH_PROFILE_WRAPPER=/disk/vc3-root/vc3-home/.vc3_sh-DnPqmi.wrapper == env: PYTHIA8DATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/pythia8/240-pafccj5/share/Pythia8/xmldoc == env: HERWIGPATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/herwigpp/7.2.0-pafccj3/share/Herwig == env: SRT_FRONTIER_CLIENT_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/frontier_client/2.9.1/ == env: SRT_COMPILER_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj == env: SRT_CSC_TRACK_FINDER_DATA_DIR_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/CSCTrackFinderEmulation/1.2-pafccj/data/ == env: GCC_RUNTIME_ASAN=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib64/libasan.so == env: SRT_ROOT_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/madgraph5amcatnlo/2.6.0-pafccj12:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gosamcontrib/2.0-20150803-pafccj == env: CC=gcc == env: TEXMFHOME=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/rivet/2.7.0-pafccj8/share/Rivet/texmf == env: SRT_MLAS_DYNAMIC_CPU_ARCH_SCRAMRTDEL=0 == env: SRT_OPENBLAS_NUM_THREADS_SCRAMRTDEL=1 == env: SRT_GIT_TEMPLATE_DIR_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/git/2.19.0-pafccj/share/git-core/templates == env: SRT_G4SAIDXSDATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4SAIDDATA/1.1/data/G4SAIDDATA1.1 == env: CMSSW_VERSION=CMSSW_10_6_20 == env: SRT_GCC_RUNTIME_UBSAN_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib64/libubsan.so == env: SRT_LANG_SCRAMRTDEL=C == env: CCTOOLS_RESOURCE_MONITOR_INFO=1029 == env: VC3_VERSION_FULL_MAKE=3.82.0 == env: CORES=1 == env: PARROT_DEBUG_FLAGS= == env: SHERPA_SHARE_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sherpa/2.2.10-pafccj2/share/SHERPA-MC == env: VC3_MACHINE_ARCH=x86_64 == env: VC3_ROOT_VC3_BUILDER_HANDS=/disk/vc3-root/x86_64/redhat7.4.1708/vc3-builder-hands/v1.0.0 == env: CMS_OPENLOOPS_PREFIX=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/openloops/2.1.0-pafccj2 == env: GIT_EXEC_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/git/2.19.0-pafccj/libexec/git-core == env: LOCALRT=/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20 == env: CMSSW_BASE=/disk/vc3-root/tmp/worker-230547-12913/t.167220/CMSSW_10_6_20 == env: SRT_SCRAM_CXX11_ABI_SCRAMRTDEL=1 == env: SRT_VALGRIND_LIB_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/valgrind/3.13.0-pafccj/lib/valgrind == env: VC3_COMMAND_LINE=/disk/vc3-builder --var CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 --require cctools-wq-worker -- work_queue_worker -M lobster_kmohrman_EFT_CRC_postLHE_crc.* -t 300 -C catalog.cse.nd.edu,backup-catalog.cse.nd.edu -d all -o worker.log --cores=12 --memory=16000 --disk=17076 == env: _=/usr/bin/env === wrapper ready @ Fri Apr 16 00:14:19 EDT 2021 ==== working directory before execution @ Fri Apr 16 00:14:19 EDT 2021 ==== == dir: total 880 == dir: -rwxr-xr-x 2 kmohrman campus 1575 Apr 16 00:13 0008_7320_report.json == dir: -rwxr-xr-x 2 kmohrman campus 1630 Apr 16 00:13 0013_6164_report.json == dir: -rwxr-xr-x 2 kmohrman campus 1462 Apr 16 00:13 0016_0941_report.json == dir: drwxr-xr-x 18 kmohrman campus 215 Apr 16 00:14 CMSSW_10_6_20 == dir: drwxr-xr-x 2 kmohrman campus 71 Apr 16 00:13 bin == dir: -rwxr-xr-x 7 kmohrman campus 627272 Apr 15 14:28 cctools-monitor == dir: -rw-r--r-- 1 kmohrman campus 0 Apr 16 00:13 cctools-monitor.summary == dir: drwxrwxrwx 2 kmohrman campus 6 Apr 16 00:13 cctools-temp-t.167220.2PbMPL == dir: -rw------- 1 kmohrman campus 51 Apr 16 00:14 cvmfschecksum.cms-ib.cern.ch == dir: -rw------- 1 kmohrman campus 51 Apr 16 00:14 cvmfschecksum.cms.cern.ch == dir: -rw------- 1 kmohrman campus 51 Apr 16 00:14 cvmfschecksum.oasis.opensciencegrid.org == dir: drwxr-xr-x 2 kmohrman campus 33 Apr 16 00:13 lib == dir: -rwxrwxrwx 1 kmohrman campus 146912 Apr 16 00:13 librmonitor_helper.so.zzOzh6 == dir: -rw-r--r-- 2 kmohrman campus 868 Apr 15 23:26 merge_cfg.py == dir: -rw-r--r-- 2 kmohrman campus 808 Apr 15 23:26 merge_reports.py == dir: -rw-r--r-- 2 kmohrman campus 2354 Apr 16 00:13 parameters.json == dir: -rw------- 2 kmohrman campus 9917 Apr 16 00:13 proxy == dir: drwxr-xr-x 3 kmohrman campus 20 Apr 16 00:13 python == dir: -rw-r--r-- 7 kmohrman campus 570 Apr 15 14:28 report.json.in == dir: -rw-r--r-- 2 kmohrman campus 549 Apr 15 23:26 sandbox-CMSSW_10_6_20-slc7_amd64_gcc700.tar.bz2 == dir: drwxr-xr-x 4 kmohrman campus 37 Apr 16 00:13 siteconf == dir: -rw-r--r-- 1 kmohrman campus 11 Apr 16 00:14 t_wrapper_ready == dir: -rw-r--r-- 1 kmohrman campus 11 Apr 16 00:14 t_wrapper_start == dir: -rwxr-xr-x 7 kmohrman campus 47217 Apr 15 14:28 task.py == dir: -rwxr-xr-x 7 kmohrman campus 5986 Apr 15 14:28 wrapper.sh >>> data is {u'files': {u'info': {}, u'output_info': {}, u'skipped': []}, u'cpu_time': 0, u'exe_exit_code': 0, u'events_written': 0, u'output_size': 0, u'task_exit_code': 0, u'output_storage_element': u'', u'cache': {u'start_size': 0, u'type': 2, u'end_size': 0}, u'stageout_exit_code': 0, u'output_bare_size': 0, u'transfers': defaultdict(, {}), u'task_timing': {u'wrapper_ready': 0, u'wrapper_start': 0, u'prologue_end': 0, u'epilogue_end': 0, u'stage_out_end': 0, u'stage_in_end': 0, u'processing_end': 0}, u'events_per_run': 0} @ Fri Apr 16 00:14:26 2021 >>> Trying hdfs client access method @ Fri Apr 16 00:14:26 2021 >>> executing 'timeout 300 hdfs dfs -fs hdfs://eddie.crc.nd.edu:19000 -get /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_136164.root MAOD-00000_136164.root' @ Fri Apr 16 00:14:26 2021 >> using /disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL/tmpkSuBuc to store command output >> cmd: timeout: failed to run command 'hdfs': No such file or directory >>>>> Unable to copy input with hdfs client @ Fri Apr 16 00:14:26 2021 >>> Trying xrootd access method @ Fri Apr 16 00:14:26 2021 >>> executing 'env XRD_LOGLEVEL=Debug timeout 300 xrdfs deepthought.crc.nd.edu stat /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_136164.root' @ Fri Apr 16 00:14:26 2021 >> using /disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL/tmpPMwzfr to store command output >> cmd: [2021-04-16 00:14:26.389315 -0400][Warning][Utility ] Unable to process global config file: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:14:26.390294 -0400][Debug ][Utility ] Unable to process user config file: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:14:26.390447 -0400][Debug ][PlugInMgr ] Initializing plug-in manager... >> cmd: [2021-04-16 00:14:26.390495 -0400][Debug ][PlugInMgr ] No default plug-in, loading plug-in configs... >> cmd: [2021-04-16 00:14:26.390544 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /etc/xrootd/client.plugins.d... >> cmd: [2021-04-16 00:14:26.390637 -0400][Debug ][PlugInMgr ] Unable to process directory /etc/xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:14:26.390950 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /tmp/runos_singularity_home-333219632341657960793722974253667464247/.xrootd/client.plugins.d... >> cmd: [2021-04-16 00:14:26.391045 -0400][Debug ][PlugInMgr ] Unable to process directory /tmp/runos_singularity_home-333219632341657960793722974253667464247/.xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:14:26.391209 -0400][Debug ][App ] Executing: stat /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_136164.root >> cmd: [2021-04-16 00:14:26.391302 -0400][Debug ][Poller ] Available pollers: built-in >> cmd: [2021-04-16 00:14:26.391343 -0400][Debug ][Poller ] Attempting to create a poller according to preference: built-in >> cmd: [2021-04-16 00:14:26.391383 -0400][Debug ][Poller ] Creating poller: built-in >> cmd: [2021-04-16 00:14:26.391429 -0400][Debug ][Poller ] Creating and starting the built-in poller... >> cmd: [2021-04-16 00:14:26.391750 -0400][Debug ][Poller ] Using 1 poller threads >> cmd: [2021-04-16 00:14:26.391803 -0400][Debug ][TaskMgr ] Starting the task manager... >> cmd: [2021-04-16 00:14:26.391938 -0400][Debug ][TaskMgr ] Task manager started >> cmd: [2021-04-16 00:14:26.391987 -0400][Debug ][JobMgr ] Starting the job manager... >> cmd: [2021-04-16 00:14:26.392294 -0400][Debug ][JobMgr ] Job manager started, 3 workers >> cmd: [2021-04-16 00:14:26.392346 -0400][Debug ][TaskMgr ] Registering task: "FileTimer task" to be run at: [2021-04-16 00:14:26 -0400] >> cmd: [2021-04-16 00:14:26.392417 -0400][Debug ][PostMaster ] Creating new channel to: deepthought.crc.nd.edu:1094 1 stream(s) >> cmd: [2021-04-16 00:14:26.392479 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 >> cmd: [2021-04-16 00:14:26.393603 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" to be run at: [2021-04-16 00:14:41 -0400] >> cmd: [2021-04-16 00:14:26.394539 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094] Found 1 address(es): [::ffff:129.74.85.55]:1094 >> cmd: [2021-04-16 00:14:26.394685 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:129.74.85.55]:1094 >> cmd: [2021-04-16 00:14:26.394782 -0400][Debug ][Poller ] Adding socket 0x10d6960 to the poller >> cmd: [2021-04-16 00:14:26.395031 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Async connection call returned >> cmd: [2021-04-16 00:14:26.395144 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2021-04-16 00:14:26.395639 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: manager [], protocol version 400) >> cmd: [2021-04-16 00:14:26.395817 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: manager [], protocol version 400) >> cmd: [2021-04-16 00:14:26.398977 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d8civy008.crc.nd.edu&xrd.rn=v4.8.5, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2021-04-16 00:14:26.399313 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Logged in, session: 3b3a0000a60d000088020000123b0000 >> cmd: [2021-04-16 00:14:26.399383 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream 0 connected. >> cmd: [2021-04-16 00:14:26.399428 -0400][Debug ][Utility ] Monitor library name not set. No monitoring >> cmd: [2021-04-16 00:14:26.400102 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant03.crc.nd.edu:1096 1 stream(s) >> cmd: [2021-04-16 00:14:26.400164 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 >> cmd: [2021-04-16 00:14:26.401421 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant03.crc.nd.edu:1096" to be run at: [2021-04-16 00:14:41 -0400] >> cmd: [2021-04-16 00:14:26.402399 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.11]:1096 >> cmd: [2021-04-16 00:14:26.402539 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.11]:1096 >> cmd: [2021-04-16 00:14:26.402625 -0400][Debug ][Poller ] Adding socket 0x200016a0 to the poller >> cmd: [2021-04-16 00:14:26.402756 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2021-04-16 00:14:26.402847 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2021-04-16 00:14:26.410869 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 400) >> cmd: [2021-04-16 00:14:26.411067 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 400) >> cmd: [2021-04-16 00:14:26.413263 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d8civy008.crc.nd.edu&xrd.rn=v4.8.5, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2021-04-16 00:14:26.413715 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Logged in, session: 2b0100005c630000e50100002b010000 >> cmd: [2021-04-16 00:14:26.413767 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10400,c:ssl,ca:ba240aa8.0|f5f0dfc2.0 >> cmd: [2021-04-16 00:14:26.413811 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2021-04-16 00:14:26.420286 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: [2021-04-16 00:14:26.491139 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending more authentication data for gsi >> cmd: [2021-04-16 00:14:26.496310 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Authenticated with gsi. >> cmd: [2021-04-16 00:14:26.496394 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096 #0] Stream 0 connected. >> cmd: Path: /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_136164.root >> cmd: Id: 0 >> cmd: Size: 96508717 >> cmd: MTime: 2021-04-15 23:43:59 >> cmd: Flags: 24 (Offline|IsReadable) >> cmd: [2021-04-16 00:14:26.499026 -0400][Debug ][JobMgr ] Stopping the job manager... >> cmd: [2021-04-16 00:14:26.499531 -0400][Debug ][JobMgr ] Job manager stopped >> cmd: [2021-04-16 00:14:26.499587 -0400][Debug ][TaskMgr ] Stopping the task manager... >> cmd: [2021-04-16 00:14:26.499748 -0400][Debug ][TaskMgr ] Task manager stopped >> cmd: [2021-04-16 00:14:26.499793 -0400][Debug ][Poller ] Stopping the poller... >> cmd: [2021-04-16 00:14:26.500059 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" >> cmd: [2021-04-16 00:14:26.500109 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2021-04-16 00:14:26.500154 -0400][Debug ][Poller ] <[::ffff:10.32.78.135]:56176><--><[::ffff:129.74.85.55]:1094> Removing socket from the poller >> cmd: [2021-04-16 00:14:26.500231 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Destroying stream >> cmd: [2021-04-16 00:14:26.500276 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2021-04-16 00:14:26.500321 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant03.crc.nd.edu:1096" >> cmd: [2021-04-16 00:14:26.500363 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2021-04-16 00:14:26.500404 -0400][Debug ][Poller ] <[::ffff:10.32.78.135]:54894><--><[::ffff:149.165.243.11]:1096> Removing socket from the poller >> cmd: [2021-04-16 00:14:26.500473 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2021-04-16 00:14:26.500523 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Closing the socket >>> will stream using xrootd instead of copying @ Fri Apr 16 00:14:26 2021 >>> Trying hdfs client access method @ Fri Apr 16 00:14:26 2021 >>> executing 'timeout 300 hdfs dfs -fs hdfs://eddie.crc.nd.edu:19000 -get /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_160941.root MAOD-00000_160941.root' @ Fri Apr 16 00:14:26 2021 >> using /disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL/tmpypM5Xh to store command output >> cmd: timeout: failed to run command 'hdfs': No such file or directory >>>>> Unable to copy input with hdfs client @ Fri Apr 16 00:14:26 2021 >>> Trying xrootd access method @ Fri Apr 16 00:14:26 2021 >>> executing 'env XRD_LOGLEVEL=Debug timeout 300 xrdfs deepthought.crc.nd.edu stat /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_160941.root' @ Fri Apr 16 00:14:26 2021 >> using /disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL/tmp9qbu7I to store command output >> cmd: [2021-04-16 00:14:26.721580 -0400][Warning][Utility ] Unable to process global config file: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:14:26.722569 -0400][Debug ][Utility ] Unable to process user config file: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:14:26.722724 -0400][Debug ][PlugInMgr ] Initializing plug-in manager... >> cmd: [2021-04-16 00:14:26.722774 -0400][Debug ][PlugInMgr ] No default plug-in, loading plug-in configs... >> cmd: [2021-04-16 00:14:26.722816 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /etc/xrootd/client.plugins.d... >> cmd: [2021-04-16 00:14:26.722904 -0400][Debug ][PlugInMgr ] Unable to process directory /etc/xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:14:26.723226 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /tmp/runos_singularity_home-333219632341657960793722974253667464247/.xrootd/client.plugins.d... >> cmd: [2021-04-16 00:14:26.723320 -0400][Debug ][PlugInMgr ] Unable to process directory /tmp/runos_singularity_home-333219632341657960793722974253667464247/.xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:14:26.723482 -0400][Debug ][App ] Executing: stat /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_160941.root >> cmd: [2021-04-16 00:14:26.723578 -0400][Debug ][Poller ] Available pollers: built-in >> cmd: [2021-04-16 00:14:26.723627 -0400][Debug ][Poller ] Attempting to create a poller according to preference: built-in >> cmd: [2021-04-16 00:14:26.723669 -0400][Debug ][Poller ] Creating poller: built-in >> cmd: [2021-04-16 00:14:26.723716 -0400][Debug ][Poller ] Creating and starting the built-in poller... >> cmd: [2021-04-16 00:14:26.724019 -0400][Debug ][Poller ] Using 1 poller threads >> cmd: [2021-04-16 00:14:26.724068 -0400][Debug ][TaskMgr ] Starting the task manager... >> cmd: [2021-04-16 00:14:26.724201 -0400][Debug ][TaskMgr ] Task manager started >> cmd: [2021-04-16 00:14:26.724251 -0400][Debug ][JobMgr ] Starting the job manager... >> cmd: [2021-04-16 00:14:26.724567 -0400][Debug ][JobMgr ] Job manager started, 3 workers >> cmd: [2021-04-16 00:14:26.724628 -0400][Debug ][TaskMgr ] Registering task: "FileTimer task" to be run at: [2021-04-16 00:14:26 -0400] >> cmd: [2021-04-16 00:14:26.724701 -0400][Debug ][PostMaster ] Creating new channel to: deepthought.crc.nd.edu:1094 1 stream(s) >> cmd: [2021-04-16 00:14:26.724764 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 >> cmd: [2021-04-16 00:14:26.725874 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" to be run at: [2021-04-16 00:14:41 -0400] >> cmd: [2021-04-16 00:14:26.727006 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094] Found 1 address(es): [::ffff:129.74.85.55]:1094 >> cmd: [2021-04-16 00:14:26.727154 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:129.74.85.55]:1094 >> cmd: [2021-04-16 00:14:26.727252 -0400][Debug ][Poller ] Adding socket 0x1016960 to the poller >> cmd: [2021-04-16 00:14:26.727480 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Async connection call returned >> cmd: [2021-04-16 00:14:26.727599 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2021-04-16 00:14:26.728033 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: manager [], protocol version 400) >> cmd: [2021-04-16 00:14:26.728216 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: manager [], protocol version 400) >> cmd: [2021-04-16 00:14:26.730795 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d8civy008.crc.nd.edu&xrd.rn=v4.8.5, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2021-04-16 00:14:26.731174 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Logged in, session: 3e3a0000a60d000088020000153b0000 >> cmd: [2021-04-16 00:14:26.731245 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream 0 connected. >> cmd: [2021-04-16 00:14:26.731290 -0400][Debug ][Utility ] Monitor library name not set. No monitoring >> cmd: [2021-04-16 00:14:26.732034 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant04.crc.nd.edu:1096 1 stream(s) >> cmd: [2021-04-16 00:14:26.732138 -0400][Debug ][PostMaster ] [primeradiant04.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 >> cmd: [2021-04-16 00:14:26.733354 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant04.crc.nd.edu:1096" to be run at: [2021-04-16 00:14:41 -0400] >> cmd: [2021-04-16 00:14:26.734300 -0400][Debug ][PostMaster ] [primeradiant04.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.12]:1096 >> cmd: [2021-04-16 00:14:26.734437 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.12]:1096 >> cmd: [2021-04-16 00:14:26.734525 -0400][Debug ][Poller ] Adding socket 0xc40016a0 to the poller >> cmd: [2021-04-16 00:14:26.734703 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2021-04-16 00:14:26.734797 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2021-04-16 00:14:26.735256 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 400) >> cmd: [2021-04-16 00:14:26.735423 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 400) >> cmd: [2021-04-16 00:14:26.737839 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d8civy008.crc.nd.edu&xrd.rn=v4.8.5, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2021-04-16 00:14:26.738151 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Logged in, session: c80e0000de110000c5060000c80e0000 >> cmd: [2021-04-16 00:14:26.738199 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10400,c:ssl,ca:ba240aa8.0|f5f0dfc2.0 >> cmd: [2021-04-16 00:14:26.738242 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2021-04-16 00:14:26.744182 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: [2021-04-16 00:14:26.800276 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Sending more authentication data for gsi >> cmd: [2021-04-16 00:14:26.804975 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Authenticated with gsi. >> cmd: [2021-04-16 00:14:26.805059 -0400][Debug ][PostMaster ] [primeradiant04.crc.nd.edu:1096 #0] Stream 0 connected. >> cmd: Path: /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_160941.root >> cmd: Id: 0 >> cmd: Size: 69657136 >> cmd: MTime: 2021-04-16 04:02:22 >> cmd: Flags: 24 (Offline|IsReadable) >> cmd: [2021-04-16 00:14:26.807599 -0400][Debug ][JobMgr ] Stopping the job manager... >> cmd: [2021-04-16 00:14:26.808140 -0400][Debug ][JobMgr ] Job manager stopped >> cmd: [2021-04-16 00:14:26.808191 -0400][Debug ][TaskMgr ] Stopping the task manager... >> cmd: [2021-04-16 00:14:26.808349 -0400][Debug ][TaskMgr ] Task manager stopped >> cmd: [2021-04-16 00:14:26.808395 -0400][Debug ][Poller ] Stopping the poller... >> cmd: [2021-04-16 00:14:26.808672 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" >> cmd: [2021-04-16 00:14:26.808728 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2021-04-16 00:14:26.808775 -0400][Debug ][Poller ] <[::ffff:10.32.78.135]:56180><--><[::ffff:129.74.85.55]:1094> Removing socket from the poller >> cmd: [2021-04-16 00:14:26.808853 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Destroying stream >> cmd: [2021-04-16 00:14:26.808898 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2021-04-16 00:14:26.808945 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant04.crc.nd.edu:1096" >> cmd: [2021-04-16 00:14:26.808987 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2021-04-16 00:14:26.809028 -0400][Debug ][Poller ] <[::ffff:10.32.78.135]:52208><--><[::ffff:149.165.243.12]:1096> Removing socket from the poller >> cmd: [2021-04-16 00:14:26.809100 -0400][Debug ][PostMaster ] [primeradiant04.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2021-04-16 00:14:26.809144 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Closing the socket >>> will stream using xrootd instead of copying @ Fri Apr 16 00:14:26 2021 >>> Trying hdfs client access method @ Fri Apr 16 00:14:26 2021 >>> executing 'timeout 300 hdfs dfs -fs hdfs://eddie.crc.nd.edu:19000 -get /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_87320.root MAOD-00000_87320.root' @ Fri Apr 16 00:14:26 2021 >> using /disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL/tmpT7v_kb to store command output >> cmd: timeout: failed to run command 'hdfs': No such file or directory >>>>> Unable to copy input with hdfs client @ Fri Apr 16 00:14:26 2021 >>> Trying xrootd access method @ Fri Apr 16 00:14:26 2021 >>> executing 'env XRD_LOGLEVEL=Debug timeout 300 xrdfs deepthought.crc.nd.edu stat /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_87320.root' @ Fri Apr 16 00:14:26 2021 >> using /disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL/tmp_mLBVz to store command output >> cmd: [2021-04-16 00:14:27.012668 -0400][Warning][Utility ] Unable to process global config file: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:14:27.013647 -0400][Debug ][Utility ] Unable to process user config file: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:14:27.013805 -0400][Debug ][PlugInMgr ] Initializing plug-in manager... >> cmd: [2021-04-16 00:14:27.013855 -0400][Debug ][PlugInMgr ] No default plug-in, loading plug-in configs... >> cmd: [2021-04-16 00:14:27.013899 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /etc/xrootd/client.plugins.d... >> cmd: [2021-04-16 00:14:27.013988 -0400][Debug ][PlugInMgr ] Unable to process directory /etc/xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:14:27.014306 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /tmp/runos_singularity_home-333219632341657960793722974253667464247/.xrootd/client.plugins.d... >> cmd: [2021-04-16 00:14:27.014400 -0400][Debug ][PlugInMgr ] Unable to process directory /tmp/runos_singularity_home-333219632341657960793722974253667464247/.xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:14:27.014570 -0400][Debug ][App ] Executing: stat /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_87320.root >> cmd: [2021-04-16 00:14:27.014670 -0400][Debug ][Poller ] Available pollers: built-in >> cmd: [2021-04-16 00:14:27.014714 -0400][Debug ][Poller ] Attempting to create a poller according to preference: built-in >> cmd: [2021-04-16 00:14:27.014756 -0400][Debug ][Poller ] Creating poller: built-in >> cmd: [2021-04-16 00:14:27.014802 -0400][Debug ][Poller ] Creating and starting the built-in poller... >> cmd: [2021-04-16 00:14:27.015123 -0400][Debug ][Poller ] Using 1 poller threads >> cmd: [2021-04-16 00:14:27.015174 -0400][Debug ][TaskMgr ] Starting the task manager... >> cmd: [2021-04-16 00:14:27.015310 -0400][Debug ][TaskMgr ] Task manager started >> cmd: [2021-04-16 00:14:27.015359 -0400][Debug ][JobMgr ] Starting the job manager... >> cmd: [2021-04-16 00:14:27.015667 -0400][Debug ][JobMgr ] Job manager started, 3 workers >> cmd: [2021-04-16 00:14:27.015725 -0400][Debug ][TaskMgr ] Registering task: "FileTimer task" to be run at: [2021-04-16 00:14:27 -0400] >> cmd: [2021-04-16 00:14:27.015798 -0400][Debug ][PostMaster ] Creating new channel to: deepthought.crc.nd.edu:1094 1 stream(s) >> cmd: [2021-04-16 00:14:27.015862 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 >> cmd: [2021-04-16 00:14:27.016828 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" to be run at: [2021-04-16 00:14:42 -0400] >> cmd: [2021-04-16 00:14:27.017860 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094] Found 1 address(es): [::ffff:129.74.85.55]:1094 >> cmd: [2021-04-16 00:14:27.018007 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:129.74.85.55]:1094 >> cmd: [2021-04-16 00:14:27.018105 -0400][Debug ][Poller ] Adding socket 0xb3e960 to the poller >> cmd: [2021-04-16 00:14:27.018322 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Async connection call returned >> cmd: [2021-04-16 00:14:27.018435 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2021-04-16 00:14:27.018863 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: manager [], protocol version 400) >> cmd: [2021-04-16 00:14:27.019042 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: manager [], protocol version 400) >> cmd: [2021-04-16 00:14:27.021748 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d8civy008.crc.nd.edu&xrd.rn=v4.8.5, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2021-04-16 00:14:27.022118 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Logged in, session: 413a0000a60d000088020000183b0000 >> cmd: [2021-04-16 00:14:27.022189 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream 0 connected. >> cmd: [2021-04-16 00:14:27.022235 -0400][Debug ][Utility ] Monitor library name not set. No monitoring >> cmd: [2021-04-16 00:14:27.023010 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant03.crc.nd.edu:1096 1 stream(s) >> cmd: [2021-04-16 00:14:27.023075 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 >> cmd: [2021-04-16 00:14:27.024339 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant03.crc.nd.edu:1096" to be run at: [2021-04-16 00:14:42 -0400] >> cmd: [2021-04-16 00:14:27.025284 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.11]:1096 >> cmd: [2021-04-16 00:14:27.025424 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.11]:1096 >> cmd: [2021-04-16 00:14:27.025508 -0400][Debug ][Poller ] Adding socket 0xc40016a0 to the poller >> cmd: [2021-04-16 00:14:27.025632 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2021-04-16 00:14:27.025728 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2021-04-16 00:14:27.026084 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 400) >> cmd: [2021-04-16 00:14:27.026250 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 400) >> cmd: [2021-04-16 00:14:27.028825 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d8civy008.crc.nd.edu&xrd.rn=v4.8.5, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2021-04-16 00:14:27.029204 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Logged in, session: 2c0100005c630000470200002c010000 >> cmd: [2021-04-16 00:14:27.029252 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10400,c:ssl,ca:ba240aa8.0|f5f0dfc2.0 >> cmd: [2021-04-16 00:14:27.029295 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2021-04-16 00:14:27.035153 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: [2021-04-16 00:14:27.101000 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending more authentication data for gsi >> cmd: [2021-04-16 00:14:27.106306 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Authenticated with gsi. >> cmd: [2021-04-16 00:14:27.106392 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096 #0] Stream 0 connected. >> cmd: Path: /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_87320.root >> cmd: Id: 0 >> cmd: Size: 96967556 >> cmd: MTime: 2021-04-15 14:35:58 >> cmd: Flags: 24 (Offline|IsReadable) >> cmd: [2021-04-16 00:14:27.109307 -0400][Debug ][JobMgr ] Stopping the job manager... >> cmd: [2021-04-16 00:14:27.109793 -0400][Debug ][JobMgr ] Job manager stopped >> cmd: [2021-04-16 00:14:27.109848 -0400][Debug ][TaskMgr ] Stopping the task manager... >> cmd: [2021-04-16 00:14:27.110008 -0400][Debug ][TaskMgr ] Task manager stopped >> cmd: [2021-04-16 00:14:27.110054 -0400][Debug ][Poller ] Stopping the poller... >> cmd: [2021-04-16 00:14:27.110317 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" >> cmd: [2021-04-16 00:14:27.110369 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2021-04-16 00:14:27.110416 -0400][Debug ][Poller ] <[::ffff:10.32.78.135]:56184><--><[::ffff:129.74.85.55]:1094> Removing socket from the poller >> cmd: [2021-04-16 00:14:27.110495 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Destroying stream >> cmd: [2021-04-16 00:14:27.110547 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2021-04-16 00:14:27.110600 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant03.crc.nd.edu:1096" >> cmd: [2021-04-16 00:14:27.110644 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2021-04-16 00:14:27.110687 -0400][Debug ][Poller ] <[::ffff:10.32.78.135]:54902><--><[::ffff:149.165.243.11]:1096> Removing socket from the poller >> cmd: [2021-04-16 00:14:27.110759 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2021-04-16 00:14:27.110803 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Closing the socket >>> will stream using xrootd instead of copying @ Fri Apr 16 00:14:27 2021 >>> modified input files @ Fri Apr 16 00:14:27 2021 >> input: root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_136164.root >> input: root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_160941.root >> input: root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_87320.root >>> updated parameters are @ Fri Apr 16 00:14:27 2021 >> json: { >> json: "accelerate stage-in": 3, >> json: "arguments": [ >> json: "outputFile=MAOD-00000.root" >> json: ], >> json: "default ce": "T3_US_NotreDame", >> json: "default host": "earth.crc.nd.edu", >> json: "default se": "T3_US_NotreDame", >> json: "disable streaming": false, >> json: "epilogue": [ >> json: "python", >> json: "merge_reports.py", >> json: "report.json", >> json: "0008_7320_report.json", >> json: "0013_6164_report.json", >> json: "0016_0941_report.json" >> json: ], >> json: "executable": "cmsRun", >> json: "file map": { >> json: "root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_136164.root": "mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_136164.root", >> json: "root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_160941.root": "mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_160941.root", >> json: "root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_87320.root": "mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_87320.root" >> json: }, >> json: "gridpack": false, >> json: "input": [ >> json: "hdfs://eddie.crc.nd.edu:19000/store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/", >> json: "root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/", >> json: "gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1", >> json: "gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1", >> json: "file:///hadoop/store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/" >> json: ], >> json: "mask": { >> json: "events": -1, >> json: "files": [ >> json: "root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_136164.root", >> json: "root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_160941.root", >> json: "root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_87320.root" >> json: ], >> json: "lumis": null >> json: }, >> json: "monitoring": { >> json: "monitorid": null, >> json: "syncid": null, >> json: "taskid": "lobster_kmohrman_EFT_CRC_postLHE_crc_20210414_1958_7e3da5c056bebd75" >> json: }, >> json: "output": [ >> json: "hdfs://eddie.crc.nd.edu:19000/store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/", >> json: "root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/", >> json: "gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1", >> json: "gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1", >> json: "file:///hadoop/store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/" >> json: ], >> json: "output files": [ >> json: [ >> json: "MAOD-00000.root", >> json: "mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_167220.root" >> json: ] >> json: ], >> json: "prologue": null, >> json: "pset": "merge_cfg.py", >> json: "want summary": true >> json: } >>> using sync CE T3_US_NotreDame @ Fri Apr 16 00:14:27 2021 >>> config file fragment @ Fri Apr 16 00:14:28 2021 >> pset: >> pset: import FWCore.ParameterSet.Config as cms >> pset: process.Timing = cms.Service("Timing", >> pset: useJobReport = cms.untracked.bool(True), >> pset: summaryOnly = cms.untracked.bool(True)) >> pset: >> pset: process.maxEvents = cms.untracked.PSet(input = cms.untracked.int32(-1)) >> pset: >> pset: import os >> pset: _, major, minor, _ = os.environ["CMSSW_VERSION"].split('_', 3) >> pset: if int(major) >= 7 and int(minor) >= 4: >> pset: xrdstats = cms.Service("XrdAdaptor::XrdStatisticsService", cms.untracked.PSet(reportToFJR = cms.untracked.bool(True))) >> pset: process.add_(xrdstats) >> pset: >> pset: for prod in process.producers.values(): >> pset: if prod.hasParameter('nEvents') and prod.type_() == 'ExternalLHEProducer': >> pset: prod.nEvents = cms.untracked.uint32(-1) >> pset: >> pset: process.source.fileNames = cms.untracked.vstring(['root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_136164.root', 'root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_160941.root', 'root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_87320.root']) >> pset: if hasattr(process, 'options'): >> pset: process.options.wantSummary = cms.untracked.bool(True) >> pset: else: >> pset: process.options = cms.untracked.PSet(wantSummary = cms.untracked.bool(True)) >>> executing 'cmsRun -j report.xml merge_cfg_mod.py outputFile=MAOD-00000.root' @ Fri Apr 16 00:14:28 2021 >> using /disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL/tmpm9CNiK to store command output >> cmd: 16-Apr-2021 00:14:35 EDT Initiating request to open file root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_136164.root >> cmd: 16-Apr-2021 00:14:45 EDT Successfully opened file root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_136164.root >> cmd: Begin processing the 1st record. Run 1, Event 1, LumiSection 23722 on stream 0 at 16-Apr-2021 00:15:13.525 EDT >> cmd: 16-Apr-2021 00:15:28 EDT Closed file root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_136164.root >> cmd: 16-Apr-2021 00:15:28 EDT Initiating request to open file root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_160941.root >> cmd: 16-Apr-2021 00:15:30 EDT Successfully opened file root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_160941.root >> cmd: Begin processing the 1001st record. Run 1, Event 152, LumiSection 13630 on stream 0 at 16-Apr-2021 00:15:56.252 EDT >> cmd: 16-Apr-2021 00:15:59 EDT Closed file root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_160941.root >> cmd: 16-Apr-2021 00:15:59 EDT Initiating request to open file root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_87320.root >> cmd: 16-Apr-2021 00:16:01 EDT Successfully opened file root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_87320.root >> cmd: 16-Apr-2021 00:16:41 EDT Closed file root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_87320.root >> cmd: >> cmd: TrigReport ---------- Event Summary ------------ >> cmd: TrigReport Events total = 1894 passed = 1894 failed = 0 >> cmd: >> cmd: TrigReport ---------- Path Summary ------------ >> cmd: TrigReport Trig Bit# Executed Passed Failed Error Name >> cmd: >> cmd: TrigReport -------End-Path Summary ------------ >> cmd: TrigReport Trig Bit# Executed Passed Failed Error Name >> cmd: TrigReport 0 0 1894 1894 0 0 end >> cmd: >> cmd: TrigReport ------ Modules in End-Path: end ------------ >> cmd: TrigReport Trig Bit# Visited Passed Failed Error Name >> cmd: TrigReport 0 0 1894 1894 0 0 out >> cmd: >> cmd: TrigReport ---------- Module Summary ------------ >> cmd: TrigReport Visited Executed Passed Failed Error Name >> cmd: TrigReport 1894 1894 1894 0 0 end >> cmd: TrigReport 1894 1894 1894 0 0 out >> cmd: >> cmd: TimeReport ---------- Event Summary ---[sec]---- >> cmd: TimeReport event loop CPU/event = 0.015640 >> cmd: TimeReport event loop Real/event = 0.062301 >> cmd: TimeReport sum Streams Real/event = 0.020466 >> cmd: TimeReport efficiency CPU/Real/thread = 0.251031 >> cmd: >> cmd: TimeReport ---------- Path Summary ---[Real sec]---- >> cmd: TimeReport per event per exec Name >> cmd: TimeReport per event per exec Name >> cmd: >> cmd: TimeReport -------End-Path Summary ---[Real sec]---- >> cmd: TimeReport per event per exec Name >> cmd: TimeReport 0.020344 0.020344 end >> cmd: TimeReport per event per exec Name >> cmd: >> cmd: TimeReport ------ Modules in End-Path: end ---[Real sec]---- >> cmd: TimeReport per event per visit Name >> cmd: TimeReport 0.000116 0.000116 out >> cmd: TimeReport per event per visit Name >> cmd: >> cmd: TimeReport ---------- Module Summary ---[Real sec]---- >> cmd: TimeReport per event per exec per visit Name >> cmd: TimeReport 0.000004 0.000004 0.000004 end >> cmd: TimeReport 0.000116 0.000116 0.000116 out >> cmd: TimeReport per event per exec per visit Name >> cmd: >> cmd: T---Report end! >> cmd: >> cmd: TimeReport> Time report complete in 134.816 seconds >> cmd: Time Summary: >> cmd: - Min event: 0.00880289 >> cmd: - Max event: 1.6809 >> cmd: - Avg event: 0.0203489 >> cmd: - Total loop: 115.406 >> cmd: - Total init: 16.807 >> cmd: - Total job: 134.816 >> cmd: - EventSetup Lock: 0 >> cmd: - EventSetup Get: 0 >> cmd: Event Throughput: 16.4116 ev/s >> cmd: CPU Summary: >> cmd: - Total loop: 62.608 >> cmd: - Total init: 6.93714 >> cmd: - Total extra: 0 >> cmd: - Total job: 72.0424 >> cmd: Processing Summary: >> cmd: - Number of Events: 1894 >> cmd: - Number of Global Begin Lumi Calls: 8 >> cmd: - Number of Global Begin Run Calls: 8 >> cmd: >> cmd: >> cmd: ============================================= >> cmd: >> cmd: MessageLogger Summary >> cmd: >> cmd: type category sev module subroutine count total >> cmd: ---- -------------------- -- ---------------- ---------------- ----- ----- >> cmd: 1 TimeReport -e AfterModEndJob 1 1 >> cmd: 2 fileAction -s file_close 3 3 >> cmd: 3 fileAction -s file_open 6 6 >> cmd: >> cmd: type category Examples: run/evt run/evt run/evt >> cmd: ---- -------------------- ---------------- ---------------- ---------------- >> cmd: 1 TimeReport PostGlobalEndRun >> cmd: 2 fileAction PostProcessEvent PostProcessEvent PostGlobalEndRun >> cmd: 3 fileAction pre-events pre-events PostProcessEvent >> cmd: >> cmd: Severity # Occurrences Total Occurrences >> cmd: -------- ------------- ----------------- >> cmd: Error 1 1 >> cmd: System 9 9 >> cmd: >> cmd: dropped waiting message count 0 >>> executable returned with exit code 0. @ Fri Apr 16 00:16:44 2021 >>> epilogue @ Fri Apr 16 00:16:56 2021 >>> executing 'python merge_reports.py report.json 0008_7320_report.json 0013_6164_report.json 0016_0941_report.json' @ Fri Apr 16 00:16:56 2021 >> using /disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL/tmpmb0DKR to store command output >> cmd: >> merging 0008_7320_report.json >> cmd: >> merging 0013_6164_report.json >> cmd: >> merging 0016_0941_report.json >>> executing 'timeout 300 hdfs dfs -fs hdfs://eddie.crc.nd.edu:19000 -put MAOD-00000.root /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_167220.root' @ Fri Apr 16 00:17:04 2021 >> using /disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL/tmp5OklDO to store command output >> cmd: timeout: failed to run command 'hdfs': No such file or directory >>> Checking output file transfer. @ Fri Apr 16 00:17:05 2021 >>> executing 'env XRD_LOGLEVEL=Debug xrdcp MAOD-00000.root root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_167220.root' @ Fri Apr 16 00:17:05 2021 >> using /disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL/tmp7HBcfm to store command output >> cmd: [2021-04-16 00:17:05.211848 -0400][Warning][Utility ] Unable to process global config file: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:17:05.212830 -0400][Debug ][Utility ] Unable to process user config file: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:17:05.212981 -0400][Debug ][PlugInMgr ] Initializing plug-in manager... >> cmd: [2021-04-16 00:17:05.213028 -0400][Debug ][PlugInMgr ] No default plug-in, loading plug-in configs... >> cmd: [2021-04-16 00:17:05.213068 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /etc/xrootd/client.plugins.d... >> cmd: [2021-04-16 00:17:05.213152 -0400][Debug ][PlugInMgr ] Unable to process directory /etc/xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:17:05.213457 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /tmp/runos_singularity_home-333219632341657960793722974253667464247/.xrootd/client.plugins.d... >> cmd: [2021-04-16 00:17:05.213566 -0400][Debug ][PlugInMgr ] Unable to process directory /tmp/runos_singularity_home-333219632341657960793722974253667464247/.xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:17:05.213833 -0400][Debug ][Poller ] Available pollers: built-in >> cmd: [2021-04-16 00:17:05.213878 -0400][Debug ][Poller ] Attempting to create a poller according to preference: built-in >> cmd: [2021-04-16 00:17:05.213918 -0400][Debug ][Poller ] Creating poller: built-in >> cmd: [2021-04-16 00:17:05.213961 -0400][Debug ][Poller ] Creating and starting the built-in poller... >> cmd: [2021-04-16 00:17:05.214323 -0400][Debug ][Poller ] Using 1 poller threads >> cmd: [2021-04-16 00:17:05.214376 -0400][Debug ][TaskMgr ] Starting the task manager... >> cmd: [2021-04-16 00:17:05.214520 -0400][Debug ][TaskMgr ] Task manager started >> cmd: [2021-04-16 00:17:05.214578 -0400][Debug ][JobMgr ] Starting the job manager... >> cmd: [2021-04-16 00:17:05.214880 -0400][Debug ][JobMgr ] Job manager started, 3 workers >> cmd: [2021-04-16 00:17:05.214932 -0400][Debug ][TaskMgr ] Registering task: "FileTimer task" to be run at: [2021-04-16 00:17:05 -0400] >> cmd: [2021-04-16 00:17:05.215003 -0400][Debug ][PostMaster ] Creating new channel to: deepthought.crc.nd.edu:1094 1 stream(s) >> cmd: [2021-04-16 00:17:05.215065 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 >> cmd: [2021-04-16 00:17:05.216052 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" to be run at: [2021-04-16 00:17:20 -0400] >> cmd: [2021-04-16 00:17:05.217350 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094] Found 1 address(es): [::ffff:129.74.85.55]:1094 >> cmd: [2021-04-16 00:17:05.217497 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:129.74.85.55]:1094 >> cmd: [2021-04-16 00:17:05.217602 -0400][Debug ][Poller ] Adding socket 0x1514060 to the poller >> cmd: [2021-04-16 00:17:05.217929 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Async connection call returned >> cmd: [2021-04-16 00:17:05.218100 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2021-04-16 00:17:05.218719 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: manager [], protocol version 400) >> cmd: [2021-04-16 00:17:05.218970 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: manager [], protocol version 400) >> cmd: [2021-04-16 00:17:05.222221 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=d8civy008.crc.nd.edu&xrd.rn=v4.8.5, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2021-04-16 00:17:05.222622 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Logged in, session: cf3b0000a60d000034040000a63c0000 >> cmd: [2021-04-16 00:17:05.222705 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream 0 connected. >> cmd: [2021-04-16 00:17:05.222752 -0400][Debug ][Utility ] Monitor library name not set. No monitoring >> cmd: [2021-04-16 00:17:05.739716 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:17:10 -0400] >> cmd: [2021-04-16 00:17:10.215563 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:17:10.726417 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:17:15 -0400] >> cmd: [2021-04-16 00:17:15.216665 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:17:15.713337 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:17:20 -0400] >> cmd: [2021-04-16 00:17:20.217726 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:17:20.700003 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:17:25 -0400] >> cmd: [2021-04-16 00:17:25.218849 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:17:25.686745 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:17:30 -0400] >> cmd: [2021-04-16 00:17:30.219963 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:17:30.673759 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:17:35 -0400] >> cmd: [2021-04-16 00:17:35.221087 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:17:35.660608 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:17:40 -0400] >> cmd: [2021-04-16 00:17:40.222196 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:17:40.647271 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:17:45 -0400] >> cmd: [2021-04-16 00:17:45.223293 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:17:45.633072 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:17:50 -0400] >> cmd: [2021-04-16 00:17:50.224319 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:17:50.618937 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:17:55 -0400] >> cmd: [2021-04-16 00:17:55.225230 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:17:55.605783 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:18:00 -0400] >> cmd: [2021-04-16 00:18:00.226194 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:18:00.592766 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:18:05 -0400] >> cmd: [2021-04-16 00:18:05.227302 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:18:05.757625 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:18:10 -0400] >> cmd: [2021-04-16 00:18:10.228345 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:18:10.744244 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:18:15 -0400] >> cmd: [2021-04-16 00:18:15.229318 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:18:15.730763 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:18:20 -0400] >> cmd: [2021-04-16 00:18:20.230290 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:18:20.717589 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:18:25 -0400] >> cmd: [2021-04-16 00:18:25.231258 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:18:25.704254 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:18:30 -0400] >> cmd: [2021-04-16 00:18:30.232160 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:18:30.690908 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:18:35 -0400] >> cmd: [2021-04-16 00:18:35.233024 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:18:35.677899 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:18:40 -0400] >> cmd: [2021-04-16 00:18:40.233992 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:18:40.664701 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:18:45 -0400] >> cmd: [2021-04-16 00:18:45.234924 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:18:45.651119 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:18:50 -0400] >> cmd: [2021-04-16 00:18:50.235920 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:18:50.637598 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:18:55 -0400] >> cmd: [2021-04-16 00:18:55.236956 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:18:55.624194 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:19:00 -0400] >> cmd: [2021-04-16 00:19:00.237784 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:19:00.610990 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:19:05 -0400] >> cmd: [2021-04-16 00:19:05.238641 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:19:05.597782 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:19:10 -0400] >> cmd: [2021-04-16 00:19:10.239510 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:19:10.762345 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:19:15 -0400] >> cmd: [2021-04-16 00:19:15.240354 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:19:15.748438 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:19:20 -0400] >> cmd: [2021-04-16 00:19:20.241225 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:19:20.734663 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:19:25 -0400] >> cmd: [2021-04-16 00:19:25.242133 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:19:25.721332 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:19:30 -0400] >> cmd: [2021-04-16 00:19:30.243017 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:19:30.707908 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:19:35 -0400] >> cmd: [2021-04-16 00:19:35.243853 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:19:35.694840 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:19:40 -0400] >> cmd: [2021-04-16 00:19:40.244733 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:19:40.681771 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:19:45 -0400] >> cmd: [2021-04-16 00:19:45.245566 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:19:45.668708 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:19:50 -0400] >> cmd: [2021-04-16 00:19:50.246403 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:19:50.655685 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:19:55 -0400] >> cmd: [2021-04-16 00:19:55.247341 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:19:55.642609 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:20:00 -0400] >> cmd: [2021-04-16 00:20:00.248285 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:20:00.629290 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:20:05 -0400] >> cmd: [2021-04-16 00:20:05.249270 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:20:05.616081 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:20:10 -0400] >> cmd: [2021-04-16 00:20:10.250224 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:20:10.780922 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:20:15 -0400] >> cmd: [2021-04-16 00:20:15.251172 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:20:15.767702 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:20:20 -0400] >> cmd: [2021-04-16 00:20:20.252279 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:20:20.754394 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:20:25 -0400] >> cmd: [2021-04-16 00:20:25.253349 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:20:25.740834 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x150f060" to be run at: [2021-04-16 00:20:30 -0400] >> cmd: [2021-04-16 00:20:30.254337 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x150f060" >> cmd: [2021-04-16 00:20:30.254956 -0400][Debug ][XRootD ] [deepthought.crc.nd.edu:1094] Handling error while processing kXR_stat (path: /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_167220.root, flags: none): [ERROR] Error response: No such file or directory. >> cmd: [2021-04-16 00:20:30.255284 -0400][Debug ][Utility ] CopyProcess: 2 jobs to prepare >> cmd: [2021-04-16 00:20:30.255866 -0400][Debug ][XRootD ] [deepthought.crc.nd.edu:1094] Handling error while processing kXR_stat (path: /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_167220.root, flags: none): [ERROR] Error response: No such file or directory. >> cmd: [2021-04-16 00:20:30.256011 -0400][Debug ][Utility ] Creating a classic copy job, from file://localhost/disk/vc3-root/tmp/worker-230547-12913/t.167220/MAOD-00000.root to root://deepthought.crc.nd.edu:1094//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_167220.root >> cmd: [2021-04-16 00:20:30.256140 -0400][Debug ][Utility ] Opening file://localhost/disk/vc3-root/tmp/worker-230547-12913/t.167220/MAOD-00000.root for reading >> cmd: [2021-04-16 00:20:30.256225 -0400][Debug ][File ] [0x15147f0@file://localhost/disk/vc3-root/tmp/worker-230547-12913/t.167220/MAOD-00000.root] Sending an open command >> cmd: [2021-04-16 00:20:30.256831 -0400][Debug ][File ] [0x15147f0@file://localhost/disk/vc3-root/tmp/worker-230547-12913/t.167220/MAOD-00000.root] Open has returned with status [SUCCESS] >> cmd: [2021-04-16 00:20:30.256892 -0400][Debug ][File ] [0x15147f0@file://localhost/disk/vc3-root/tmp/worker-230547-12913/t.167220/MAOD-00000.root] successfully opened at localhost, handle: 0x1a, session id: 1 >> cmd: [2021-04-16 00:20:30.257027 -0400][Debug ][Utility ] Opening root://deepthought.crc.nd.edu:1094//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_167220.root?oss.asize=258736215 for writing >> cmd: [2021-04-16 00:20:30.257101 -0400][Debug ][File ] [0x1515bb0@root://deepthought.crc.nd.edu:1094//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_167220.root?oss.asize=258736215] Sending an open command >> cmd: [2021-04-16 00:20:30.257647 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant02.crc.nd.edu:1096 1 stream(s) >> cmd: [2021-04-16 00:20:30.257709 -0400][Debug ][PostMaster ] [primeradiant02.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 >> cmd: [2021-04-16 00:20:30.259086 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant02.crc.nd.edu:1096" to be run at: [2021-04-16 00:20:45 -0400] >> cmd: [2021-04-16 00:20:30.260049 -0400][Debug ][PostMaster ] [primeradiant02.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.6]:1096 >> cmd: [2021-04-16 00:20:30.260188 -0400][Debug ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.6]:1096 >> cmd: [2021-04-16 00:20:30.260275 -0400][Debug ][Poller ] Adding socket 0x6c001860 to the poller >> cmd: [2021-04-16 00:20:30.260377 -0400][Debug ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2021-04-16 00:20:30.260476 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2021-04-16 00:20:30.260826 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 400) >> cmd: [2021-04-16 00:20:30.261002 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 400) >> cmd: [2021-04-16 00:20:30.263396 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=d8civy008.crc.nd.edu&xrd.rn=v4.8.5, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2021-04-16 00:20:30.265604 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Logged in, session: b6e400003e780000370d0000b6e40000 >> cmd: [2021-04-16 00:20:30.265655 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10400,c:ssl,ca:ba240aa8.0|f5f0dfc2.0 >> cmd: [2021-04-16 00:20:30.265700 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2021-04-16 00:20:30.271104 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: [2021-04-16 00:20:30.347063 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Sending more authentication data for gsi >> cmd: [2021-04-16 00:20:30.352217 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Authenticated with gsi. >> cmd: [2021-04-16 00:20:30.352303 -0400][Debug ][PostMaster ] [primeradiant02.crc.nd.edu:1096 #0] Stream 0 connected. >> cmd: [2021-04-16 00:21:30.566717 -0400][Debug ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2021-04-16 00:21:30.566847 -0400][Debug ][Poller ] <[::ffff:10.32.78.135]:35932><--><[::ffff:149.165.243.6]:1096> Removing socket from the poller >> cmd: [2021-04-16 00:21:30.567029 -0400][Debug ][PostMaster ] [primeradiant02.crc.nd.edu:1096 #0] Recovering error for stream #0: [ERROR] Socket error. >> cmd: [2021-04-16 00:21:30.567078 -0400][Debug ][PostMaster ] [primeradiant02.crc.nd.edu:1096 #0] Reporting disconnection to queued message handlers. >> cmd: [2021-04-16 00:21:30.567129 -0400][Debug ][XRootD ] [primeradiant02.crc.nd.edu:1096] Handling error while processing kXR_open (file: /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_167220.root?oss.asize=258736215, mode: 0644, flags: kXR_new kXR_open_updt kXR_async kXR_retstat ): [ERROR] Socket error. >> cmd: [2021-04-16 00:21:30.570970 -0400][Debug ][XRootD ] [deepthought.crc.nd.edu:1094] Handling error while processing kXR_open (file: /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_167220.root?oss.asize=258736215&tried=primeradiant02.crc.nd.edu, mode: 0644, flags: kXR_new kXR_open_updt kXR_async kXR_retstat ): [ERROR] Error response: No such file or directory. >> cmd: [2021-04-16 00:21:30.571065 -0400][Debug ][File ] [0x1515bb0@root://deepthought.crc.nd.edu:1094//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_167220.root?oss.asize=258736215] Open has returned with status [ERROR] Server responded with an error: [3011] Unable to create new file; file already exists. >> cmd: [2021-04-16 00:21:30.571119 -0400][Debug ][File ] [0x1515bb0@root://deepthought.crc.nd.edu:1094//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_167220.root?oss.asize=258736215] Error while opening at deepthought.crc.nd.edu:1094: [ERROR] Server responded with an error: [3011] Unable to create new file; file already exists. >> cmd: [2021-04-16 00:21:30.571261 -0400][Debug ][File ] [0x15147f0@file://localhost/disk/vc3-root/tmp/worker-230547-12913/t.167220/MAOD-00000.root] Sending a close command for handle 0x1a to localhost >> cmd: [2021-04-16 00:21:30.571542 -0400][Debug ][File ] [0x15147f0@file://localhost/disk/vc3-root/tmp/worker-230547-12913/t.167220/MAOD-00000.root] Close returned from localhost with: [SUCCESS] >> cmd: [0B/0B][100%][==================================================][0B/s] >> cmd: Run: [ERROR] Server responded with an error: [3011] Unable to create new file; file already exists. >> cmd: >> cmd: [2021-04-16 00:21:30.604095 -0400][Debug ][JobMgr ] Stopping the job manager... >> cmd: [2021-04-16 00:21:30.604788 -0400][Debug ][JobMgr ] Job manager stopped >> cmd: [2021-04-16 00:21:30.604847 -0400][Debug ][TaskMgr ] Stopping the task manager... >> cmd: [2021-04-16 00:21:30.605055 -0400][Debug ][TaskMgr ] Task manager stopped >> cmd: [2021-04-16 00:21:30.605104 -0400][Debug ][Poller ] Stopping the poller... >> cmd: [2021-04-16 00:21:30.605410 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" >> cmd: [2021-04-16 00:21:30.605463 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2021-04-16 00:21:30.605510 -0400][Debug ][Poller ] <[::ffff:10.32.78.135]:56220><--><[::ffff:129.74.85.55]:1094> Removing socket from the poller >> cmd: [2021-04-16 00:21:30.605610 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Destroying stream >> cmd: [2021-04-16 00:21:30.605658 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2021-04-16 00:21:30.605711 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant02.crc.nd.edu:1096" >> cmd: [2021-04-16 00:21:30.605768 -0400][Debug ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2021-04-16 00:21:30.605815 -0400][Debug ][PostMaster ] [primeradiant02.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2021-04-16 00:21:30.605857 -0400][Debug ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Closing the socket >>> Checking output file transfer. @ Fri Apr 16 00:21:30 2021 >>> executing '/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/bin/gfal-copy -f file:///disk/vc3-root/tmp/worker-230547-12913/t.167220/MAOD-00000.root gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_167220.root' @ Fri Apr 16 00:21:30 2021 >> using /disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL/tmpy0tN4w to store command output >> cmd: Copying 258736215 bytes file:///disk/vc3-root/tmp/worker-230547-12913/t.167220/MAOD-00000.root => gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_167220.root >>> Checking output file transfer. @ Fri Apr 16 00:22:13 2021 >>> executing 'timeout 300 hdfs dfs -fs hdfs://eddie.crc.nd.edu:19000 -stat "Size: %b" /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_167220.root' @ Fri Apr 16 00:22:13 2021 >> using /disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL/tmpJyDOhX to store command output >> cmd: timeout: failed to run command 'hdfs': No such file or directory >>>>> checking output for MAOD-00000.root failed: timeout: failed to run command 'hdfs': No such file or directory @ Fri Apr 16 00:22:13 2021 >>> executing 'timeout 300 xrdfs deepthought.crc.nd.edu stat /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_167220.root' @ Fri Apr 16 00:22:13 2021 >> using /disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL/tmpfAMx_i to store command output >> cmd: [ERROR] Server responded with an error: [3011] No servers are available to read the file. >> cmd: >>>>> checking output for MAOD-00000.root failed: [ERROR] Server responded with an error: [3011] No servers are available to read the file. @ Fri Apr 16 00:23:14 2021 >>> executing '/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/bin/gfal-stat gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_167220.root' @ Fri Apr 16 00:23:14 2021 >> using /disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL/tmpKBX1Sj to store command output >> cmd: File: 'gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_167220.root' >> cmd: Size: 258736215 regular file >> cmd: Access: (0664/-rw-rw-r--) Uid: 230547 Gid: 0 >> cmd: Access: 1969-12-31 19:00:00.000000 >> cmd: Modify: 2021-04-16 00:22:12.000000 >> cmd: Change: 1969-12-31 19:00:00.000000 >>> File transfer successful! @ Fri Apr 16 00:23:41 2021 >>> Checking output files... @ Fri Apr 16 00:23:41 2021 >>> Checking MAOD-00000.root => mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_167220.root @ Fri Apr 16 00:23:41 2021 >>> executing 'timeout 300 hdfs dfs -fs hdfs://eddie.crc.nd.edu:19000 -stat "Size: %b" /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_167220.root' @ Fri Apr 16 00:23:41 2021 >> using /disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL/tmpYBVwdr to store command output >> cmd: timeout: failed to run command 'hdfs': No such file or directory >>>>> checking output for MAOD-00000.root failed: timeout: failed to run command 'hdfs': No such file or directory @ Fri Apr 16 00:23:41 2021 >>> executing 'timeout 300 xrdfs deepthought.crc.nd.edu stat /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_167220.root' @ Fri Apr 16 00:23:41 2021 >> using /disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL/tmp5vN5T2 to store command output >>>>> checking output for MAOD-00000.root failed: @ Fri Apr 16 00:28:41 2021 >>> executing '/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/bin/gfal-stat gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_167220.root' @ Fri Apr 16 00:28:41 2021 >> using /disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL/tmpsMEvtM to store command output >> cmd: gfal-stat error: 13 (Permission denied) - globus_ftp_client: the server responded with an error 530 530-Login incorrect. : /builddir/build/BUILD/gridftp-hdfs-1.1.1/src/gridftp_hdfs.c:hdfs_start:756: 530-Server is cancelling transfer due to over-load limit (host=deepthought.crc.nd.edu, user=kmohrman, path=(null)) 530 End. >>>>> checking output for MAOD-00000.root failed: gfal-stat error: 13 (Permission denied) - globus_ftp_client: the server responded with an error 530 530-Login incorrect. : /builddir/build/BUILD/gridftp-hdfs-1.1.1/src/gridftp_hdfs.c:hdfs_start:756: 530-Server is cancelling transfer due to over-load limit (host=deepthought.crc.nd.edu, user=kmohrman, path=(null)) 530 End. @ Fri Apr 16 00:30:56 2021 >>> executing '/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/bin/gfal-stat gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_167220.root' @ Fri Apr 16 00:30:56 2021 >> using /disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL/tmpiizEdx to store command output >> cmd: gfal-stat error: 13 (Permission denied) - globus_ftp_client: the server responded with an error 530 530-Login incorrect. : /builddir/build/BUILD/gridftp-hdfs-1.1.1/src/gridftp_hdfs.c:hdfs_start:756: 530-Server is cancelling transfer due to over-load limit (host=deepthought.crc.nd.edu, user=kmohrman, path=(null)) 530 End. >>>>> checking output for MAOD-00000.root failed: gfal-stat error: 13 (Permission denied) - globus_ftp_client: the server responded with an error 530 530-Login incorrect. : /builddir/build/BUILD/gridftp-hdfs-1.1.1/src/gridftp_hdfs.c:hdfs_start:756: 530-Server is cancelling transfer due to over-load limit (host=deepthought.crc.nd.edu, user=kmohrman, path=(null)) 530 End. @ Fri Apr 16 00:31:05 2021 >>> executing 'stat /hadoop/store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_167220.root' @ Fri Apr 16 00:31:05 2021 >> using /disk/vc3-root/tmp/worker-230547-12913/t.167220/cctools-temp-t.167220.2PbMPL/tmp_xaFJ_ to store command output >> cmd: stat: cannot stat '/hadoop/store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_167220.root': No such file or directory >>>>> checking output for MAOD-00000.root failed: stat: cannot stat '/hadoop/store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_167220.root': No such file or directory @ Fri Apr 16 00:31:05 2021 >> trace: Traceback (most recent call last): >> trace: File "task.py", line 268, in wrapper >> trace: result = fct(data, *args, **kwargs) >> trace: File "task.py", line 404, in check_outputs >> trace: raise IOError("could not verify output file '{}'".format(remote)) >> trace: IOError: could not verify output file 'mAOD_step_ttlnuJet_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_167220.root' >>>>> call to 'check_outputs' failed, exiting with exit code 211 @ Fri Apr 16 00:31:05 2021 Fri Apr 16 00:14:27 2021: ApMon[ERROR]: Error resolving cms-jobmon.cern.ch: [Errno -2] Name or service not known Fri Apr 16 00:14:27 2021: ApMon[WARNING]: Not sending parameters since no destination is defined. Fri Apr 16 00:14:27 2021: ApMon[DEBUG]: Closing UDP socket on ApMon object destroy. Fri Apr 16 00:16:44 2021: ApMon[ERROR]: Error resolving cms-jobmon.cern.ch: [Errno -2] Name or service not known Fri Apr 16 00:16:44 2021: ApMon[WARNING]: Not sending parameters since no destination is defined. Fri Apr 16 00:16:44 2021: ApMon[DEBUG]: Closing UDP socket on ApMon object destroy. >> Execution time 1017 >> Exiting with code 211 >> Reporting ExeExitCode 0 >> Reporting StageOutSE T3_US_NotreDame >> Reporting StageOutExitCode 211 Fri Apr 16 00:31:05 2021: ApMon[ERROR]: Error resolving cms-jobmon.cern.ch: [Errno -2] Name or service not known Fri Apr 16 00:31:05 2021: ApMon[WARNING]: Not sending parameters since no destination is defined. Fri Apr 16 00:31:05 2021: ApMon[DEBUG]: Closing UDP socket on ApMon object destroy. ==== working directory after execution @ Fri Apr 16 00:31:06 EDT 2021 ==== == dir: total 253632 == dir: -rwxr-xr-x 2 kmohrman campus 1575 Apr 16 00:13 0008_7320_report.json == dir: -rwxr-xr-x 2 kmohrman campus 1630 Apr 16 00:13 0013_6164_report.json == dir: -rwxr-xr-x 2 kmohrman campus 1462 Apr 16 00:13 0016_0941_report.json == dir: drwxr-xr-x 18 kmohrman campus 215 Apr 16 00:14 CMSSW_10_6_20 == dir: -rw-r--r-- 1 kmohrman campus 258736215 Apr 16 00:16 MAOD-00000.root == dir: drwxr-xr-x 2 kmohrman campus 71 Apr 16 00:13 bin == dir: -rwxr-xr-x 7 kmohrman campus 627272 Apr 15 14:28 cctools-monitor == dir: -rw-r--r-- 1 kmohrman campus 0 Apr 16 00:13 cctools-monitor.summary == dir: drwxrwxrwx 2 kmohrman campus 6 Apr 16 00:31 cctools-temp-t.167220.2PbMPL == dir: -rw------- 1 kmohrman campus 51 Apr 16 00:14 cvmfschecksum.cms-ib.cern.ch == dir: -rw------- 1 kmohrman campus 51 Apr 16 00:14 cvmfschecksum.cms.cern.ch == dir: -rw------- 1 kmohrman campus 51 Apr 16 00:14 cvmfschecksum.oasis.opensciencegrid.org == dir: drwxr-xr-x 2 kmohrman campus 33 Apr 16 00:13 lib == dir: -rwxrwxrwx 1 kmohrman campus 146912 Apr 16 00:13 librmonitor_helper.so.zzOzh6 == dir: -rw-r--r-- 2 kmohrman campus 868 Apr 15 23:26 merge_cfg.py == dir: -rw-r--r-- 1 kmohrman campus 2355 Apr 16 00:14 merge_cfg_mod.py == dir: -rw-r--r-- 2 kmohrman campus 808 Apr 15 23:26 merge_reports.py == dir: -rw-r--r-- 2 kmohrman campus 2354 Apr 16 00:13 parameters.json == dir: -rw------- 2 kmohrman campus 9917 Apr 16 00:13 proxy == dir: drwxr-xr-x 3 kmohrman campus 20 Apr 16 00:13 python == dir: -rw-r--r-- 1 kmohrman campus 2425 Apr 16 00:31 report.json == dir: -rw-r--r-- 7 kmohrman campus 570 Apr 15 14:28 report.json.in == dir: -rw-r--r-- 1 kmohrman campus 59460 Apr 16 00:16 report.xml == dir: -rw-r--r-- 1 kmohrman campus 4879 Apr 16 00:31 report.xml.gz == dir: -rw-r--r-- 2 kmohrman campus 549 Apr 15 23:26 sandbox-CMSSW_10_6_20-slc7_amd64_gcc700.tar.bz2 == dir: drwxr-xr-x 4 kmohrman campus 37 Apr 16 00:13 siteconf == dir: -rw-r--r-- 1 kmohrman campus 11 Apr 16 00:14 t_wrapper_ready == dir: -rw-r--r-- 1 kmohrman campus 11 Apr 16 00:14 t_wrapper_start == dir: -rwxr-xr-x 7 kmohrman campus 47217 Apr 15 14:28 task.py == dir: -rwxr-xr-x 7 kmohrman campus 5986 Apr 15 14:28 wrapper.sh === wrapper done @ Fri Apr 16 00:31:06 EDT 2021 === final return status = 211 @ Fri Apr 16 00:31:06 EDT 2021