==== wrapper started @ Fri Apr 16 00:17:10 EDT 2021 ==== == startup: hostname: d8civy101.crc.nd.edu == startup: kernel: Linux d8civy101.crc.nd.edu 3.10.0-1160.el7.x86_64 #1 SMP Tue Aug 18 14:50:17 EDT 2020 x86_64 x86_64 x86_64 GNU/Linux ==== tracing google @ Fri Apr 16 00:17:15 EDT 2021 ==== wrapper.sh: line 20: traceroute: command not found ==== environment at startup @ Fri Apr 16 00:17:20 EDT 2021 ==== == env: VC3_VERSION_SED=4.2 == env: CCTOOLS_RESOURCE_PROCESS_START=1618546640362176 == env: VC3_INSIDE_WRAPPER_SCRIPT=yes == env: VC3_ROOT_GZIP=/usr/ == env: SHELL=/bin/sh == env: TERM=xterm-256color == env: MAKEFLAGS=-j4 == env: TMPDIR=/disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N == env: VC3_ROOT_WHICH=/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0 == env: LOBSTER_CVMFS_PROXY=http://eddie.crc.nd.edu:3128 == env: LD_PRELOAD=/disk/vc3-root/tmp/worker-230547-29293/t.166186/librmonitor_helper.so.pJWPIv == env: LOBSTER_FRONTIER_PROXY=http://eddie.crc.nd.edu:3128 == env: LOBSTER_CMSSW_VERSION=CMSSW_10_6_20 == env: VC3_VERSION_FULL_TAR=1.26.0 == env: VC3_MACHINE_OS=Linux/redhat7.4.1708 == env: VC3_VERSION_FULL_CCTOOLS_WQ_WORKER=999.999.999 == env: USER=kmohrman == env: TEMP=/disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N == env: VC3_BUILDER_PATH=/disk/vc3-builder == env: LD_LIBRARY_PATH=/.singularity.d/libs == env: VC3_VERSION_VC3_BUILDER_HANDS=1.0 == env: VC3_ROOT_MAKE=/usr/ == env: VC3_DISTFILES=/disk/vc3-distfiles == env: WORKER_TMPDIR=/disk/vc3-root/tmp/worker-230547-29293/cache/tmp == env: VC3_VERSION_TAR=1.26 == env: ENV=/disk/vc3-root/vc3-home/.vc3_sh-K0KHdQ == env: VC3_VERSION_CCTOOLS_WQ_WORKER=999.999 == env: VC3_VERSION_FULL_PKG_CONFIG=0.27.1 == env: VC3_VERSION_GZIP=1.5 == env: VC3_ENV_TAG_GLOBAL=29040 == env: PATH=/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0/bin:/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin == env: VC3_ROOT=/disk/vc3-root == env: VC3_SH_PROFILE_ENV=/disk/vc3-root/vc3-home/.vc3_sh-K0KHdQ == env: VC3_ROOT_PKG_CONFIG=/usr/ == env: PWD=/disk/vc3-root/tmp/worker-230547-29293/t.166186 == env: LOBSTER_OSG_VERSION=3.4 == env: VC3_VERSION_FULL_SED=4.2.2 == env: VC3_VERSION_WHICH=2.20 == env: VC3_MACHINE_GLIBC_VERSION=2.17 == env: MODULEPATH=/afs/crc.nd.edu/x86_64_linux/Modules/modules/development_tools_and_libraries:/afs/crc.nd.edu/x86_64_linux/Modules/modules/general_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/restricted_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/system_modules:/afs/crc.nd.edu/x86_64_linux/Modules/modules/deprecated_software:/afs/crc.nd.edu/x86_64_linux/Modules/current/modulefiles == env: WORK_QUEUE_SANDBOX=/disk/vc3-root/tmp/worker-230547-29293 == env: VC3_ROOT_CCTOOLS_WQ_WORKER=/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999 == env: VC3_ROOT_SED=/usr/ == env: VC3_MACHINE_TARGET=x86_64/redhat7.4.1708 == env: CXX=g++ == env: SHLVL=3 == env: HOME=/disk/vc3-root/vc3-home == env: VC3_VERSION_MAKE=3.82 == env: VC3_VERSION_FULL_GZIP=1.5.0 == env: DISK=361 == env: MEMORY=1044 == env: VC3_VERSION_FULL_WHICH=2.20.0 == env: TMP=/disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N == env: VC3_INSTALL_USER_HOME=/disk/vc3-root/vc3-home == env: VC3_VERSION_FULL_VC3_BUILDER_HANDS=1.0.0 == env: VC3_VERSION_PKG_CONFIG=0.27 == env: VC3_ROOT_TAR=/usr/ == env: VC3_SH_PROFILE_WRAPPER=/disk/vc3-root/vc3-home/.vc3_sh-K0KHdQ.wrapper == env: CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 == env: CC=gcc == env: CORES=1 == env: VC3_VERSION_FULL_MAKE=3.82.0 == env: CCTOOLS_RESOURCE_MONITOR_INFO=1027 == env: VC3_ROOT_VC3_BUILDER_HANDS=/disk/vc3-root/x86_64/redhat7.4.1708/vc3-builder-hands/v1.0.0 == env: VC3_MACHINE_ARCH=x86_64 == env: VC3_COMMAND_LINE=/disk/vc3-builder --var CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 --require cctools-wq-worker -- work_queue_worker -M lobster_kmohrman_EFT_CRC_postLHE_crc.* -t 300 -C catalog.cse.nd.edu,backup-catalog.cse.nd.edu -d all -o worker.log --cores=12 --memory=16000 --disk=18566 == env: _=/usr/bin/env ==== cpu info @ Fri Apr 16 00:17:25 EDT 2021 ==== == cpu: processor : 0 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 0 == cpu: cpu cores : 8 == cpu: apicid : 0 == cpu: initial apicid : 0 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.08 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 1 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 1 == cpu: cpu cores : 8 == cpu: apicid : 2 == cpu: initial apicid : 2 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.08 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 2 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3000.537 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 2 == cpu: cpu cores : 8 == cpu: apicid : 4 == cpu: initial apicid : 4 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.08 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 3 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 3 == cpu: cpu cores : 8 == cpu: apicid : 6 == cpu: initial apicid : 6 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.08 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 4 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 4 == cpu: cpu cores : 8 == cpu: apicid : 8 == cpu: initial apicid : 8 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.08 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 5 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3000.061 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 5 == cpu: cpu cores : 8 == cpu: apicid : 10 == cpu: initial apicid : 10 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.08 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 6 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3000.854 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 6 == cpu: cpu cores : 8 == cpu: apicid : 12 == cpu: initial apicid : 12 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.08 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 7 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3000.378 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 7 == cpu: cpu cores : 8 == cpu: apicid : 14 == cpu: initial apicid : 14 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.08 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 8 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.743 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 0 == cpu: cpu cores : 8 == cpu: apicid : 32 == cpu: initial apicid : 32 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5191.96 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 9 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 1 == cpu: cpu cores : 8 == cpu: apicid : 34 == cpu: initial apicid : 34 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5191.96 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 10 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 2 == cpu: cpu cores : 8 == cpu: apicid : 36 == cpu: initial apicid : 36 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5191.96 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 11 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 3 == cpu: cpu cores : 8 == cpu: apicid : 38 == cpu: initial apicid : 38 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5191.96 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 12 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3001.965 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 4 == cpu: cpu cores : 8 == cpu: apicid : 40 == cpu: initial apicid : 40 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5191.96 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 13 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 5 == cpu: cpu cores : 8 == cpu: apicid : 42 == cpu: initial apicid : 42 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5191.96 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 14 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.743 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 6 == cpu: cpu cores : 8 == cpu: apicid : 44 == cpu: initial apicid : 44 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5191.96 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 15 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 7 == cpu: cpu cores : 8 == cpu: apicid : 46 == cpu: initial apicid : 46 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5191.96 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: === using CVMFS proxy: http://eddie.crc.nd.edu:3128 @ Fri Apr 16 00:17:50 EDT 2021 === using Frontier proxy: http://eddie.crc.nd.edu:3128 @ Fri Apr 16 00:17:50 EDT 2021 === parrot helper: /disk/vc3-root/tmp/worker-230547-29293/t.166186/lib/libparrot_helper.so @ Fri Apr 16 00:17:55 EDT 2021 ==== content of /disk/vc3-root/tmp/worker-230547-29293/cache/tmp @ Fri Apr 16 00:17:55 EDT 2021 ==== == cache: total 28 == cache: drwx------ 263 kmohrman campus 8192 Apr 16 00:06 ex_parrot_kmohrman == cache: -rw-r--r-- 1 kmohrman campus 10 Apr 15 16:00 hot_cache == cache: drwxr-x--- 260 kmohrman campus 8192 Apr 15 15:45 cvmfs === OSG certificate location: /cvmfs/oasis.opensciencegrid.org/mis/certificates @ Fri Apr 16 00:18:00 EDT 2021 === testing parrot usage @ Fri Apr 16 00:18:00 EDT 2021 === parrot OK @ Fri Apr 16 00:18:05 EDT 2021 === starting parrot to access CMSSW... @ Fri Apr 16 00:18:05 EDT 2021 ==== wrapper started @ Fri Apr 16 00:18:05 EDT 2021 ==== == startup: hostname: d8civy101.crc.nd.edu == startup: kernel: Linux d8civy101.crc.nd.edu 3.10.0-1160.el7.x86_64 #1 SMP Tue Aug 18 14:50:17 EDT 2020 x86_64 x86_64 x86_64 GNU/Linux ==== tracing google @ Fri Apr 16 00:18:05 EDT 2021 ==== wrapper.sh: line 20: traceroute: command not found ==== environment at startup @ Fri Apr 16 00:18:05 EDT 2021 ==== == env: VC3_VERSION_SED=4.2 == env: CCTOOLS_RESOURCE_PROCESS_START=1618546685818471 == env: VC3_INSIDE_WRAPPER_SCRIPT=yes == env: PARROT_HTTP_PROXY=http://eddie.crc.nd.edu:3128 == env: TERM=xterm-256color == env: SHELL=/bin/sh == env: VC3_ROOT_GZIP=/usr/ == env: LOBSTER_GFAL_COPY= == env: MAKEFLAGS=-j4 == env: PARROT_HELPER=/disk/vc3-root/tmp/worker-230547-29293/t.166186/lib/libparrot_helper.so == env: TMPDIR=/disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N == env: VC3_ROOT_WHICH=/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0 == env: LD_PRELOAD=/disk/vc3-root/tmp/worker-230547-29293/t.166186/librmonitor_helper.so.pJWPIv == env: LOBSTER_CVMFS_PROXY=http://eddie.crc.nd.edu:3128 == env: PARROT_PATH=./bin == env: LOBSTER_FRONTIER_PROXY=http://eddie.crc.nd.edu:3128 == env: LOBSTER_CMSSW_VERSION=CMSSW_10_6_20 == env: VC3_MACHINE_OS=Linux/redhat7.4.1708 == env: VC3_VERSION_FULL_TAR=1.26.0 == env: USER=kmohrman == env: VC3_VERSION_FULL_CCTOOLS_WQ_WORKER=999.999.999 == env: LD_LIBRARY_PATH=/.singularity.d/libs:lib == env: VC3_BUILDER_PATH=/disk/vc3-builder == env: TEMP=/disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N == env: VC3_VERSION_VC3_BUILDER_HANDS=1.0 == env: VC3_DISTFILES=/disk/vc3-distfiles == env: VC3_ROOT_MAKE=/usr/ == env: ENV=/disk/vc3-root/vc3-home/.vc3_sh-K0KHdQ == env: VC3_VERSION_TAR=1.26 == env: WORKER_TMPDIR=/disk/vc3-root/tmp/worker-230547-29293/cache/tmp == env: FRONTIER_PROXY=http://eddie.crc.nd.edu:3128 == env: PARROT_ALLOW_SWITCHING_CVMFS_REPOSITORIES=TRUE == env: VC3_VERSION_CCTOOLS_WQ_WORKER=999.999 == env: VC3_VERSION_FULL_PKG_CONFIG=0.27.1 == env: PARROT_CVMFS_REPO= == env: VC3_VERSION_GZIP=1.5 == env: PATH=/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0/bin:/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin == env: VC3_ENV_TAG_GLOBAL=29040 == env: VC3_ROOT=/disk/vc3-root == env: VC3_SH_PROFILE_ENV=/disk/vc3-root/vc3-home/.vc3_sh-K0KHdQ == env: PWD=/disk/vc3-root/tmp/worker-230547-29293/t.166186 == env: VC3_ROOT_PKG_CONFIG=/usr/ == env: PARROT_ENABLED=TRUE == env: VC3_VERSION_FULL_SED=4.2.2 == env: LOBSTER_OSG_VERSION=3.4 == env: VC3_VERSION_WHICH=2.20 == env: MODULEPATH=/afs/crc.nd.edu/x86_64_linux/Modules/modules/development_tools_and_libraries:/afs/crc.nd.edu/x86_64_linux/Modules/modules/general_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/restricted_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/system_modules:/afs/crc.nd.edu/x86_64_linux/Modules/modules/deprecated_software:/afs/crc.nd.edu/x86_64_linux/Modules/current/modulefiles == env: VC3_MACHINE_GLIBC_VERSION=2.17 == env: LOBSTER_LCG_CP= == env: WORK_QUEUE_SANDBOX=/disk/vc3-root/tmp/worker-230547-29293 == env: VC3_ROOT_CCTOOLS_WQ_WORKER=/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999 == env: VC3_MACHINE_TARGET=x86_64/redhat7.4.1708 == env: VC3_ROOT_SED=/usr/ == env: OASIS_CERTIFICATES=/cvmfs/oasis.opensciencegrid.org/mis/certificates == env: CXX=g++ == env: HOME=/disk/vc3-root/vc3-home == env: SHLVL=3 == env: VC3_VERSION_FULL_GZIP=1.5.0 == env: VC3_VERSION_MAKE=3.82 == env: DISK=361 == env: PARROT_CACHE=/disk/vc3-root/tmp/worker-230547-29293/cache/tmp == env: PARROT_CVMFS_ALIEN_CACHE=/disk/vc3-root/tmp/worker-230547-29293/cache/tmp/cvmfs == env: VC3_VERSION_FULL_WHICH=2.20.0 == env: MEMORY=1044 == env: HTTP_PROXY=http://eddie.crc.nd.edu:3128 == env: TMP=/disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N == env: PYTHONPATH=python: == env: VC3_INSTALL_USER_HOME=/disk/vc3-root/vc3-home == env: VC3_VERSION_FULL_VC3_BUILDER_HANDS=1.0.0 == env: VC3_ROOT_TAR=/usr/ == env: VC3_VERSION_PKG_CONFIG=0.27 == env: CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 == env: VC3_SH_PROFILE_WRAPPER=/disk/vc3-root/vc3-home/.vc3_sh-K0KHdQ.wrapper == env: CC=gcc == env: CCTOOLS_RESOURCE_MONITOR_INFO=1027 == env: VC3_VERSION_FULL_MAKE=3.82.0 == env: CORES=1 == env: PARROT_DEBUG_FLAGS= == env: VC3_MACHINE_ARCH=x86_64 == env: VC3_ROOT_VC3_BUILDER_HANDS=/disk/vc3-root/x86_64/redhat7.4.1708/vc3-builder-hands/v1.0.0 == env: VC3_COMMAND_LINE=/disk/vc3-builder --var CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 --require cctools-wq-worker -- work_queue_worker -M lobster_kmohrman_EFT_CRC_postLHE_crc.* -t 300 -C catalog.cse.nd.edu,backup-catalog.cse.nd.edu -d all -o worker.log --cores=12 --memory=16000 --disk=18566 == env: _=/usr/bin/env ==== cpu info @ Fri Apr 16 00:18:06 EDT 2021 ==== == cpu: processor : 0 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 0 == cpu: cpu cores : 8 == cpu: apicid : 0 == cpu: initial apicid : 0 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.08 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 1 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 1 == cpu: cpu cores : 8 == cpu: apicid : 2 == cpu: initial apicid : 2 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.08 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 2 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 2 == cpu: cpu cores : 8 == cpu: apicid : 4 == cpu: initial apicid : 4 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.08 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 3 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 3 == cpu: cpu cores : 8 == cpu: apicid : 6 == cpu: initial apicid : 6 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.08 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 4 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 4 == cpu: cpu cores : 8 == cpu: apicid : 8 == cpu: initial apicid : 8 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.08 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 5 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 5 == cpu: cpu cores : 8 == cpu: apicid : 10 == cpu: initial apicid : 10 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.08 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 6 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3000.219 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 6 == cpu: cpu cores : 8 == cpu: apicid : 12 == cpu: initial apicid : 12 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.08 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 7 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 7 == cpu: cpu cores : 8 == cpu: apicid : 14 == cpu: initial apicid : 14 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.08 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 8 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3000.061 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 0 == cpu: cpu cores : 8 == cpu: apicid : 32 == cpu: initial apicid : 32 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5191.96 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 9 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3005.139 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 1 == cpu: cpu cores : 8 == cpu: apicid : 34 == cpu: initial apicid : 34 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5191.96 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 10 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3002.441 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 2 == cpu: cpu cores : 8 == cpu: apicid : 36 == cpu: initial apicid : 36 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5191.96 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 11 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 3 == cpu: cpu cores : 8 == cpu: apicid : 38 == cpu: initial apicid : 38 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5191.96 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 12 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 4 == cpu: cpu cores : 8 == cpu: apicid : 40 == cpu: initial apicid : 40 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5191.96 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 13 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3007.202 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 5 == cpu: cpu cores : 8 == cpu: apicid : 42 == cpu: initial apicid : 42 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5191.96 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 14 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 6 == cpu: cpu cores : 8 == cpu: apicid : 44 == cpu: initial apicid : 44 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5191.96 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 15 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3005.139 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 7 == cpu: cpu cores : 8 == cpu: apicid : 46 == cpu: initial apicid : 46 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5191.96 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: === using parrot @ Fri Apr 16 00:18:06 EDT 2021 === sourcing CMS setup @ Fri Apr 16 00:18:06 EDT 2021 LibCvmfs version 2.4, revision 25 === sourcing OSG setup @ Fri Apr 16 00:18:08 EDT 2021 ==== environment after sourcing startup scripts @ Fri Apr 16 00:18:08 EDT 2021 ==== == env: VC3_VERSION_SED=4.2 == env: MANPATH=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/share/man:/cvmfs/cms.cern.ch/share/man: == env: CCTOOLS_RESOURCE_PROCESS_START=1618546688234881 == env: VC3_INSIDE_WRAPPER_SCRIPT=yes == env: GFAL_PLUGIN_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/gfal2-plugins/ == env: PARROT_HTTP_PROXY=http://eddie.crc.nd.edu:3128 == env: TERM=xterm-256color == env: SHELL=/bin/sh == env: VC3_ROOT_GZIP=/usr/ == env: LOBSTER_GFAL_COPY=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/bin/gfal-copy == env: VOMS_USERCONF=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/etc/vomses == env: MAKEFLAGS=-j4 == env: GLOBUS_LOCATION=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr == env: TMPDIR=/disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N == env: PERL5LIB=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/share/perl5/vendor_perl:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/share/perl5:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/perl5/vendor_perl:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/perl5 == env: VC3_ROOT_WHICH=/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0 == env: CVSROOT=:gserver:cmssw.cvs.cern.ch:/local/reps/CMSSW == env: LD_PRELOAD=/disk/vc3-root/tmp/worker-230547-29293/t.166186/librmonitor_helper.so.pJWPIv == env: LOBSTER_CVMFS_PROXY=http://eddie.crc.nd.edu:3128 == env: PARROT_PATH=./bin == env: LOBSTER_FRONTIER_PROXY=http://eddie.crc.nd.edu:3128 == env: X509_CERT_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/etc/grid-security/certificates == env: SCRAM_ARCH=slc7_amd64_gcc820 == env: LOBSTER_CMSSW_VERSION=CMSSW_10_6_20 == env: VC3_MACHINE_OS=Linux/redhat7.4.1708 == env: VC3_VERSION_FULL_TAR=1.26.0 == env: USER=kmohrman == env: VC3_VERSION_FULL_CCTOOLS_WQ_WORKER=999.999.999 == env: LD_LIBRARY_PATH=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/dcap:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/lcgdm:/.singularity.d/libs:lib:lib == env: VC3_BUILDER_PATH=/disk/vc3-builder == env: TEMP=/disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N == env: VC3_VERSION_VC3_BUILDER_HANDS=1.0 == env: VC3_DISTFILES=/disk/vc3-distfiles == env: VC3_ROOT_MAKE=/usr/ == env: OSG_LOCATION=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64 == env: ENV=/disk/vc3-root/vc3-home/.vc3_sh-K0KHdQ == env: VC3_VERSION_TAR=1.26 == env: WORKER_TMPDIR=/disk/vc3-root/tmp/worker-230547-29293/cache/tmp == env: FRONTIER_PROXY=http://eddie.crc.nd.edu:3128 == env: PARROT_ALLOW_SWITCHING_CVMFS_REPOSITORIES=TRUE == env: VC3_VERSION_CCTOOLS_WQ_WORKER=999.999 == env: VC3_VERSION_FULL_PKG_CONFIG=0.27.1 == env: SITECONFIG_PATH=/cvmfs/cms.cern.ch/SITECONF/local == env: PARROT_CVMFS_REPO= == env: VC3_VERSION_GZIP=1.5 == env: PATH=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/bin:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/sbin:/cvmfs/cms.cern.ch/common:/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0/bin:/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin == env: VC3_ENV_TAG_GLOBAL=29040 == env: CMS_PATH=/cvmfs/cms.cern.ch == env: VC3_ROOT=/disk/vc3-root == env: VC3_SH_PROFILE_ENV=/disk/vc3-root/vc3-home/.vc3_sh-K0KHdQ == env: PWD=/disk/vc3-root/tmp/worker-230547-29293/t.166186 == env: VC3_ROOT_PKG_CONFIG=/usr/ == env: PARROT_ENABLED=TRUE == env: VC3_VERSION_FULL_SED=4.2.2 == env: LOBSTER_OSG_VERSION=3.4 == env: VC3_VERSION_WHICH=2.20 == env: MODULEPATH=/afs/crc.nd.edu/x86_64_linux/Modules/modules/development_tools_and_libraries:/afs/crc.nd.edu/x86_64_linux/Modules/modules/general_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/restricted_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/system_modules:/afs/crc.nd.edu/x86_64_linux/Modules/modules/deprecated_software:/afs/crc.nd.edu/x86_64_linux/Modules/current/modulefiles == env: VC3_MACHINE_GLIBC_VERSION=2.17 == env: X509_VOMS_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/etc/grid-security/vomsdir == env: LOBSTER_LCG_CP= == env: WORK_QUEUE_SANDBOX=/disk/vc3-root/tmp/worker-230547-29293 == env: VC3_ROOT_CCTOOLS_WQ_WORKER=/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999 == env: VC3_MACHINE_TARGET=x86_64/redhat7.4.1708 == env: VC3_ROOT_SED=/usr/ == env: OASIS_CERTIFICATES=/cvmfs/oasis.opensciencegrid.org/mis/certificates == env: CXX=g++ == env: HOME=/disk/vc3-root/vc3-home == env: SHLVL=3 == env: VC3_VERSION_FULL_GZIP=1.5.0 == env: VC3_VERSION_MAKE=3.82 == env: DISK=361 == env: PARROT_CACHE=/disk/vc3-root/tmp/worker-230547-29293/cache/tmp == env: PARROT_CVMFS_ALIEN_CACHE=/disk/vc3-root/tmp/worker-230547-29293/cache/tmp/cvmfs == env: GFAL_CONFIG_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/etc/gfal2.d/ == env: VC3_VERSION_FULL_WHICH=2.20.0 == env: MEMORY=1044 == env: HTTP_PROXY=http://eddie.crc.nd.edu:3128 == env: TMP=/disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N == env: PYTHONPATH=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib/python2.7/site-packages:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/python2.7/site-packages:python:python: == env: VC3_INSTALL_USER_HOME=/disk/vc3-root/vc3-home == env: VC3_VERSION_FULL_VC3_BUILDER_HANDS=1.0.0 == env: VC3_ROOT_TAR=/usr/ == env: VC3_VERSION_PKG_CONFIG=0.27 == env: CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 == env: VC3_SH_PROFILE_WRAPPER=/disk/vc3-root/vc3-home/.vc3_sh-K0KHdQ.wrapper == env: CC=gcc == env: CCTOOLS_RESOURCE_MONITOR_INFO=1027 == env: VC3_VERSION_FULL_MAKE=3.82.0 == env: CORES=1 == env: PARROT_DEBUG_FLAGS= == env: VC3_MACHINE_ARCH=x86_64 == env: VC3_ROOT_VC3_BUILDER_HANDS=/disk/vc3-root/x86_64/redhat7.4.1708/vc3-builder-hands/v1.0.0 == env: VC3_COMMAND_LINE=/disk/vc3-builder --var CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 --require cctools-wq-worker -- work_queue_worker -M lobster_kmohrman_EFT_CRC_postLHE_crc.* -t 300 -C catalog.cse.nd.edu,backup-catalog.cse.nd.edu -d all -o worker.log --cores=12 --memory=16000 --disk=18566 == env: _=/usr/bin/env ==== proxy information @ Fri Apr 16 00:18:08 EDT 2021 ==== == proxy: subject : /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=kmohrman/CN=825714/CN=Kelci Ann Mohrman/CN=3380169703 == proxy: issuer : /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=kmohrman/CN=825714/CN=Kelci Ann Mohrman == proxy: identity : /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=kmohrman/CN=825714/CN=Kelci Ann Mohrman == proxy: type : RFC compliant proxy == proxy: strength : 1024 bits == proxy: path : proxy == proxy: timeleft : 163:24:11 ==== working directory at startup @ Fri Apr 16 00:18:08 EDT 2021 ==== == dir: total 876 == dir: -rwxr-xr-x 2 kmohrman campus 1598 Apr 16 00:17 0015_3455_report.json == dir: -rwxr-xr-x 2 kmohrman campus 1599 Apr 16 00:17 0015_4423_report.json == dir: -rwxr-xr-x 2 kmohrman campus 1602 Apr 16 00:17 0015_7630_report.json == dir: -rwxr-xr-x 2 kmohrman campus 1602 Apr 16 00:17 0015_8749_report.json == dir: drwxr-xr-x 2 kmohrman campus 71 Apr 16 00:17 bin == dir: -rwxr-xr-x 5 kmohrman campus 627272 Apr 15 15:44 cctools-monitor == dir: -rw-r--r-- 1 kmohrman campus 0 Apr 16 00:17 cctools-monitor.summary == dir: drwxrwxrwx 2 kmohrman campus 6 Apr 16 00:17 cctools-temp-t.166186.YOAD9N == dir: -rw------- 1 kmohrman campus 51 Apr 16 00:18 cvmfschecksum.cms.cern.ch == dir: -rw------- 1 kmohrman campus 51 Apr 16 00:18 cvmfschecksum.oasis.opensciencegrid.org == dir: drwxr-xr-x 2 kmohrman campus 33 Apr 16 00:17 lib == dir: -rwxrwxrwx 1 kmohrman campus 146912 Apr 16 00:17 librmonitor_helper.so.pJWPIv == dir: -rw-r--r-- 2 kmohrman campus 868 Apr 16 00:17 merge_cfg.py == dir: -rw-r--r-- 2 kmohrman campus 808 Apr 16 00:17 merge_reports.py == dir: -rw-r--r-- 2 kmohrman campus 2528 Apr 16 00:17 parameters.json == dir: -rw------- 2 kmohrman campus 9917 Apr 16 00:17 proxy == dir: drwxr-xr-x 3 kmohrman campus 20 Apr 16 00:17 python == dir: -rw-r--r-- 5 kmohrman campus 570 Apr 15 15:44 report.json.in == dir: -rw-r--r-- 2 kmohrman campus 549 Apr 16 00:17 sandbox-CMSSW_10_6_20-slc7_amd64_gcc700.tar.bz2 == dir: drwxr-xr-x 4 kmohrman campus 37 Apr 16 00:17 siteconf == dir: -rw-r--r-- 1 kmohrman campus 11 Apr 16 00:18 t_wrapper_start == dir: -rwxr-xr-x 5 kmohrman campus 47217 Apr 15 15:44 task.py == dir: -rwxr-xr-x 5 kmohrman campus 5986 Apr 15 15:44 wrapper.sh === creating new release CMSSW_10_6_20 for scram arch slc7_amd64_gcc700 @ Fri Apr 16 00:18:08 EDT 2021 WARNING: In non-interactive mode release checks e.g. deprecated releases, production architectures are disabled. === unpacking sandbox-CMSSW_10_6_20-slc7_amd64_gcc700.tar.bz2 @ Fri Apr 16 00:18:12 EDT 2021 ==== machine load @ Fri Apr 16 00:18:14 EDT 2021 ==== top: unknown option 'M' Usage: top -hv | -bcHiOSs -d secs -n max -u|U user -p pid(s) -o field -w [cols] ==== environment before execution @ Fri Apr 16 00:18:14 EDT 2021 ==== == env: G4LEVELGAMMADATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4PhotonEvaporation/5.2/data/PhotonEvaporation5.2 == env: UTM_XSD_DIR=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/utm/utm_0.7.1-pafccj == env: SRT_G4LEVELGAMMADATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4PhotonEvaporation/5.2/data/PhotonEvaporation5.2 == env: VC3_VERSION_SED=4.2 == env: CMSSW_RELEASE_BASE=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20 == env: SCRAMRT_SET=CMSSW:CMSSW_10_6_20:slc7_amd64_gcc700:V2_2_9_pre13:SRT_ == env: MANPATH=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/share/man:/cvmfs/cms.cern.ch/share/man: == env: CCTOOLS_RESOURCE_PROCESS_START=1618546694610080 == env: CMSSW_FWLITE_INCLUDE_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sherpa/2.2.10-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/hepmc/2.06.07-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/clhep/2.4.0.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/boost/1.67.0-pafccj/include == env: SRT_GCC_RUNTIME_ASAN_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib64/libasan.so == env: VC3_INSIDE_WRAPPER_SCRIPT=yes == env: ROOT_INCLUDE_PATH=/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20/src:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/src:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/coral/CORAL_2_3_21-pafccj7/include/LCG:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/mctester/1.25.0a-pafccj9/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/lcg/root/6.14.09-pafccj6/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/dd4hep/v01-10x-pafccj9/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/QtDesigner:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/herwigpp/7.2.0-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/tauolapp/1.1.5-pafccj5/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/charybdis/1.003-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sherpa/2.2.10-pafccj2/include/SHERPA-MC:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/QtOpenGL:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/QtGui:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/thepeg/2.2.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/pythia8/240-pafccj5/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/herwig/6.521-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/rivet/2.7.0-pafccj8/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/Qt3Support:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/lwtnn/2.4-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4/10.4.3-pafccj2/include/Geant4:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/classlib/3.1.3-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/lhapdf/6.2.1-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/cgal/4.2-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/tkonlinesw/4.2.0-1_gcc7-pafccj9/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/starlight/r193-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/Qt:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/QtCore:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/QtXml:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/mcdb/1.0.3-pafccj/interface:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libungif/4.1.4-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libtiff/4.0.3-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/highfive/2.0/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libpng/1.6.16-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/frontier_client/2.9.1/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/pcre/8.37-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/boost/1.67.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/xrootd/4.8.5-pafccj2/include/xrootd/private:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/vdt/0.4.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/valgrind/3.13.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/utm/utm_0.7.1-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/toprex/4.23-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/tbb/2019_U3-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/tauola/27.121.5-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sigcpp/2.6.2-pafccj/include/sigc++-2.0:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sqlite/3.22.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/protobuf/3.5.2-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/pacparser/1.3.5-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/oracle/12.1.0.2.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/py2-numpy/1.16.2-nmpfii2/c-api/core/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/meschach/1.2.pCMS1-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/lz4/1.9.2-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libuuid/2.22.2-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libunwind/1.2.1-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libhepml/0.2.1-pafccj/interface:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/ktjet/1.06-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/jimmy/4.2-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/jemalloc/5.1.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/hls/2019.08/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/madgraph5amcatnlo/2.6.0-pafccj12:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/heppdt/3.03.00-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/hector/1.3.4_patch1-pafccj9/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libjpeg-turbo/2.0.1-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/giflib/4.2.3-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gsl/2.2.1-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gdbm/1.10-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/freetype/2.5.3-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/fftw3/3.3.2-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/fftjet/1.5.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/fastjet/3.3.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/expat/2.1.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/dpm/1.8.0.1-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/hepmc/2.06.07-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/vecgeom/v00.05.00-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/xerces-c/3.1.3-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/xz/5.2.2-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/dcap/2.47.8-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libxml2/2.9.1-pafccj/include/libxml2:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/curl/7.59.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/cppunit/1.40.1-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/clhep/2.4.0.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/openssl/1.0.2d-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/pythia6/426-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/photos/215.5-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/zlib-x86_64/1.2.11-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/cascade/2.2.04-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/bz2lib/1.0.6-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/python/2.7.14-pafccj/include/python2.7:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/tinyxml2/6.2.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/ittnotify/16.06.18-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gosamcontrib/2.0-20150803-pafccj/include:/usr/local/include:/usr/include == env: INTEL_LICENSE_FILE=28518@lxlicen01.cern.ch,28518@lxlicen02.cern.ch,28518@lxlicen03.cern.ch == env: SRT_TEXMFHOME_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/rivet/2.7.0-pafccj8/share/Rivet/texmf == env: SRT_GEANT4_PARFULLCMS_CONFIG_DIR_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-parfullcms/2014.01.27-pafccj2/share/ParFullCMS == env: GFAL_PLUGIN_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/gfal2-plugins/ == env: PARROT_HTTP_PROXY=http://eddie.crc.nd.edu:3128 == env: TERM=xterm-256color == env: SHELL=/bin/sh == env: VC3_ROOT_GZIP=/usr/ == env: LOBSTER_GFAL_COPY=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/bin/gfal-copy == env: PDFPATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/rivet/2.7.0-pafccj8/share == env: SRT_TNS_ADMIN_SCRAMRTDEL=/etc == env: VOMS_USERCONF=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/etc/vomses == env: MAKEFLAGS=-j4 == env: ROOT_GCC_TOOLCHAIN=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj == env: GLOBUS_LOCATION=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr == env: TMPDIR=/disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N == env: G4FORCENUMBEROFTHREADS=max == env: OPENBLAS_NUM_THREADS=1 == env: SRT_PDFPATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/rivet/2.7.0-pafccj8/share == env: PERL5LIB=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/git/2.19.0-pafccj/share/perl5:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/share/perl5/vendor_perl:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/share/perl5:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/perl5/vendor_perl:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/perl5 == env: VC3_ROOT_WHICH=/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0 == env: CMSSW_DATA_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms == env: LHAPDF_DATA_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/lhapdf/6.2.1-pafccj3/share/LHAPDF == env: CVSROOT=:gserver:cmssw.cvs.cern.ch:/local/reps/CMSSW == env: LD_PRELOAD=/disk/vc3-root/tmp/worker-230547-29293/t.166186/librmonitor_helper.so.pJWPIv == env: LOBSTER_CVMFS_PROXY=http://eddie.crc.nd.edu:3128 == env: PARROT_PATH=./bin == env: CMSSW_GIT_HASH=CMSSW_10_6_20 == env: SRT_G4ABLADATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4ABLA/3.0/data/G4ABLA3.0 == env: QTDIR=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2 == env: SRT_SCONS_LIB_DIR_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/scons/3.0.1-pafccj/lib == env: OLDPWD=/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20 == env: LOBSTER_FRONTIER_PROXY=http://eddie.crc.nd.edu:3128 == env: G4NEUTRONXSDATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4NEUTRONXS/1.4/data/G4NEUTRONXS1.4 == env: QTINC=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include == env: X509_CERT_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/etc/grid-security/certificates == env: SCRAM_ARCH=slc7_amd64_gcc700 == env: GIT_SSL_CAINFO=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/git/2.19.0-pafccj/share/ssl/certs/ca-bundle.crt == env: SRT_CMS_OPENLOOPS_PREFIX_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/openloops/2.1.0-pafccj2 == env: SRT_GIT_EXEC_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/git/2.19.0-pafccj/libexec/git-core == env: SRT_THEPEGPATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/thepeg/2.2.0-pafccj2/share/ThePEG == env: LOBSTER_CMSSW_VERSION=CMSSW_10_6_20 == env: ROOT_TTREECACHE_SIZE=0 == env: EVTGENDATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/evtgen/1.6.0-pafccj5/share == env: SRT_INTEL_LICENSE_FILE_SCRAMRTDEL=28518@lxlicen01.cern.ch,28518@lxlicen02.cern.ch,28518@lxlicen03.cern.ch == env: SRT_ROOT_INCLUDE_PATH_SCRAMRTDEL=/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20/src:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/src:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/coral/CORAL_2_3_21-pafccj7/include/LCG:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/mctester/1.25.0a-pafccj9/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/lcg/root/6.14.09-pafccj6/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/dd4hep/v01-10x-pafccj9/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/QtDesigner:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/herwigpp/7.2.0-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/tauolapp/1.1.5-pafccj5/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/charybdis/1.003-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sherpa/2.2.10-pafccj2/include/SHERPA-MC:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/QtOpenGL:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/QtGui:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/thepeg/2.2.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/pythia8/240-pafccj5/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/herwig/6.521-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/rivet/2.7.0-pafccj8/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/Qt3Support:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/lwtnn/2.4-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4/10.4.3-pafccj2/include/Geant4:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/classlib/3.1.3-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/lhapdf/6.2.1-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/cgal/4.2-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/tkonlinesw/4.2.0-1_gcc7-pafccj9/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/starlight/r193-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/Qt:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/QtCore:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/QtXml:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/mcdb/1.0.3-pafccj/interface:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libungif/4.1.4-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libtiff/4.0.3-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/highfive/2.0/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libpng/1.6.16-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/frontier_client/2.9.1/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/pcre/8.37-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/boost/1.67.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/xrootd/4.8.5-pafccj2/include/xrootd/private:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/vdt/0.4.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/valgrind/3.13.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/utm/utm_0.7.1-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/toprex/4.23-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/tbb/2019_U3-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/tauola/27.121.5-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sigcpp/2.6.2-pafccj/include/sigc++-2.0:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sqlite/3.22.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/protobuf/3.5.2-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/pacparser/1.3.5-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/oracle/12.1.0.2.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/py2-numpy/1.16.2-nmpfii2/c-api/core/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/meschach/1.2.pCMS1-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/lz4/1.9.2-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libuuid/2.22.2-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libunwind/1.2.1-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libhepml/0.2.1-pafccj/interface:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/ktjet/1.06-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/jimmy/4.2-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/jemalloc/5.1.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/hls/2019.08/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/madgraph5amcatnlo/2.6.0-pafccj12:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/heppdt/3.03.00-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/hector/1.3.4_patch1-pafccj9/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libjpeg-turbo/2.0.1-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/giflib/4.2.3-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gsl/2.2.1-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gdbm/1.10-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/freetype/2.5.3-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/fftw3/3.3.2-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/fftjet/1.5.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/fastjet/3.3.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/expat/2.1.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/dpm/1.8.0.1-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/hepmc/2.06.07-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/vecgeom/v00.05.00-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/xerces-c/3.1.3-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/xz/5.2.2-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/dcap/2.47.8-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libxml2/2.9.1-pafccj/include/libxml2:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/curl/7.59.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/cppunit/1.40.1-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/clhep/2.4.0.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/openssl/1.0.2d-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/pythia6/426-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/photos/215.5-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/zlib-x86_64/1.2.11-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/cascade/2.2.04-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/bz2lib/1.0.6-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/python/2.7.14-pafccj/include/python2.7:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/tinyxml2/6.2.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/ittnotify/16.06.18-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gosamcontrib/2.0-20150803-pafccj/include:/usr/local/include:/usr/include == env: VC3_MACHINE_OS=Linux/redhat7.4.1708 == env: VC3_VERSION_FULL_TAR=1.26.0 == env: G4LEDATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4EMLOW/7.3/data/G4EMLOW7.3 == env: SRT_VINCIADATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/vincia/2.2.04-pafccj5/share/Vincia/xmldoc == env: SRT_ROOT_TTREECACHE_PREFILL_SCRAMRTDEL=0 == env: OPAL_PREFIX=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/openmpi/2.1.5-pafccj == env: USER=kmohrman == env: VC3_VERSION_FULL_CCTOOLS_WQ_WORKER=999.999.999 == env: ROOT_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/madgraph5amcatnlo/2.6.0-pafccj12:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gosamcontrib/2.0-20150803-pafccj == env: SHERPA_INCLUDE_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sherpa/2.2.10-pafccj2/include/SHERPA-MC == env: SRT_LHAPDF_DATA_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/lhapdf/6.2.1-pafccj3/share/LHAPDF == env: SRT_PYTHON27PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/share/overrides/python:/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20/python:/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/python:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/coral/CORAL_2_3_21-pafccj7/slc7_amd64_gcc700/python:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/coral/CORAL_2_3_21-pafccj7/slc7_amd64_gcc700/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/xrootd/4.8.5-pafccj2/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/fastjet/3.3.0-pafccj/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw-tool-conf/45.0-pafccj26/lib/python2.7/site-packages == env: LD_LIBRARY_PATH=/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20/biglib/slc7_amd64_gcc700:/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20/external/slc7_amd64_gcc700/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/biglib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/external/slc7_amd64_gcc700/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/llvm/7.1.0-pafccj/lib64:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib64:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/cuda/10.1.105-pafccj2/drivers:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/dcap:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/lcgdm:/.singularity.d/libs:lib:lib == env: VC3_BUILDER_PATH=/disk/vc3-builder == env: TEMP=/disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N == env: SRT_CMSSW_RELEASE_BASE_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20 == env: SRT_RIVET_ANALYSIS_PATH_SCRAMRTDEL=/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/rivet/2.7.0-pafccj8/lib == env: VC3_VERSION_VC3_BUILDER_HANDS=1.0 == env: SRT_PERL5LIB_SCRAMRT=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/git/2.19.0-pafccj/share/perl5 == env: SRT_CMSSW_VERSION_SCRAMRTDEL=CMSSW_10_6_20 == env: VC3_DISTFILES=/disk/vc3-distfiles == env: VC3_ROOT_MAKE=/usr/ == env: GCC_RUNTIME_LSAN=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib64/libasan.so == env: SCRAM_CXX11_ABI=1 == env: VALGRIND_LIB=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/valgrind/3.13.0-pafccj/lib/valgrind == env: SRT_CMSSW_GIT_HASH_SCRAMRTDEL=CMSSW_10_6_20 == env: SRT_ROOTSYS_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/lcg/root/6.14.09-pafccj6/ == env: OSG_LOCATION=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64 == env: ENV=/disk/vc3-root/vc3-home/.vc3_sh-K0KHdQ == env: VC3_VERSION_TAR=1.26 == env: WORKER_TMPDIR=/disk/vc3-root/tmp/worker-230547-29293/cache/tmp == env: G4NEUTRONHPDATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4NDL/4.5/data/G4NDL4.5 == env: GEANT4_PARFULLCMS_CONFIG_DIR=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-parfullcms/2014.01.27-pafccj2/share/ParFullCMS == env: FRONTIER_PROXY=http://eddie.crc.nd.edu:3128 == env: PARROT_ALLOW_SWITCHING_CVMFS_REPOSITORIES=TRUE == env: CSC_TRACK_FINDER_DATA_DIR=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/CSCTrackFinderEmulation/1.2-pafccj/data/ == env: SRT_QTINC_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include == env: VC3_VERSION_CCTOOLS_WQ_WORKER=999.999 == env: G4ENSDFSTATEDATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4ENSDFSTATE/2.2/data/G4ENSDFSTATE2.2 == env: G4RADIOACTIVEDATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4RadioactiveDecay/5.2/data/RadioactiveDecay5.2 == env: ROOFITSYS=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/lcg/root/6.14.09-pafccj6/ == env: SRT_ROOFITSYS_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/lcg/root/6.14.09-pafccj6/ == env: SRT_G4RADIOACTIVEDATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4RadioactiveDecay/5.2/data/RadioactiveDecay5.2 == env: SRT_G4ENSDFSTATEDATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4ENSDFSTATE/2.2/data/G4ENSDFSTATE2.2 == env: SRT_CLHEP_PARAM_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/clhep/2.4.0.0-pafccj2 == env: G4ABLADATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4ABLA/3.0/data/G4ABLA3.0 == env: VC3_VERSION_FULL_PKG_CONFIG=0.27.1 == env: SRT_SHERPA_INCLUDE_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sherpa/2.2.10-pafccj2/include/SHERPA-MC == env: SRT_G4FORCENUMBEROFTHREADS_SCRAMRTDEL=max == env: SRT_WORKER_DATA_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/blackhat/0.9.9-pafccj/share/blackhat/datafiles/ == env: SITECONFIG_PATH=/cvmfs/cms.cern.ch/SITECONF/local == env: PARROT_CVMFS_REPO= == env: VC3_VERSION_GZIP=1.5 == env: GCC_RUNTIME_TSAN=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib64/libtsan.so == env: TNS_ADMIN=/etc == env: SRT_HEPPDT_PARAM_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/heppdt/3.03.00-pafccj == env: PATH=/cvmfs/cms.cern.ch/share/overrides/bin:/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20/bin/slc7_amd64_gcc700:/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20/external/slc7_amd64_gcc700/bin:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/bin/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/external/slc7_amd64_gcc700/bin:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/llvm/7.1.0-pafccj/bin:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/bin:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/bin:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/sbin:/cvmfs/cms.cern.ch/common:/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0/bin:/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin == env: VC3_ENV_TAG_GLOBAL=29040 == env: THEPEGPATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/thepeg/2.2.0-pafccj2/share/ThePEG == env: SRT_CMSSW_SEARCH_PATH_SCRAMRTDEL=/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20/poison:/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20/src:/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20/external/slc7_amd64_gcc700/data:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/src:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/external/slc7_amd64_gcc700/data == env: SRT_GIT_SSL_CAINFO_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/git/2.19.0-pafccj/share/ssl/certs/ca-bundle.crt == env: SRT_GCC_RUNTIME_LSAN_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib64/libasan.so == env: CMS_PATH=/cvmfs/cms.cern.ch == env: VC3_ROOT=/disk/vc3-root == env: SRT_UTM_XSD_DIR_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/utm/utm_0.7.1-pafccj == env: SRT_PATH_SCRAMRT=/cvmfs/cms.cern.ch/share/overrides/bin:/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20/bin/slc7_amd64_gcc700:/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20/external/slc7_amd64_gcc700/bin:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/bin/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/external/slc7_amd64_gcc700/bin:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/llvm/7.1.0-pafccj/bin:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/bin == env: VC3_SH_PROFILE_ENV=/disk/vc3-root/vc3-home/.vc3_sh-K0KHdQ == env: RIVET_ANALYSIS_PATH=/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/rivet/2.7.0-pafccj8/lib == env: SRT_LOCALRT_SCRAMRTDEL=/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20 == env: SRT_CMSSW_BASE_SCRAMRTDEL=/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20 == env: PWD=/disk/vc3-root/tmp/worker-230547-29293/t.166186 == env: VC3_ROOT_PKG_CONFIG=/usr/ == env: PARROT_ENABLED=TRUE == env: VC3_VERSION_FULL_SED=4.2.2 == env: LOBSTER_OSG_VERSION=3.4 == env: PYTHON_VALGRIND_SUPP=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/python/2.7.14-pafccj/share/valgrind/valgrind-python.supp == env: SRT_OPAL_PREFIX_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/openmpi/2.1.5-pafccj == env: VC3_VERSION_WHICH=2.20 == env: GCC_RUNTIME_UBSAN=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib64/libubsan.so == env: LANG=C == env: SRT_QTDIR_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2 == env: MODULEPATH=/afs/crc.nd.edu/x86_64_linux/Modules/modules/development_tools_and_libraries:/afs/crc.nd.edu/x86_64_linux/Modules/modules/general_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/restricted_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/system_modules:/afs/crc.nd.edu/x86_64_linux/Modules/modules/deprecated_software:/afs/crc.nd.edu/x86_64_linux/Modules/current/modulefiles == env: VC3_MACHINE_GLIBC_VERSION=2.17 == env: SRT_G4NEUTRONHPDATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4NDL/4.5/data/G4NDL4.5 == env: X509_VOMS_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/etc/grid-security/vomsdir == env: LOBSTER_LCG_CP= == env: SRT_SHERPA_SHARE_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sherpa/2.2.10-pafccj2/share/SHERPA-MC == env: SRT_GCC_RUNTIME_TSAN_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib64/libtsan.so == env: WORK_QUEUE_SANDBOX=/disk/vc3-root/tmp/worker-230547-29293 == env: CASCADE_PDFPATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/cascade/2.2.04-pafccj3/share == env: SRT_ROOT_GCC_TOOLCHAIN_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj == env: SRT_CASCADE_PDFPATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/cascade/2.2.04-pafccj3/share == env: WORKER_DATA_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/blackhat/0.9.9-pafccj/share/blackhat/datafiles/ == env: GIT_TEMPLATE_DIR=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/git/2.19.0-pafccj/share/git-core/templates == env: G4SAIDXSDATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4SAIDDATA/1.1/data/G4SAIDDATA1.1 == env: MLAS_DYNAMIC_CPU_ARCH=0 == env: VC3_ROOT_CCTOOLS_WQ_WORKER=/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999 == env: PYTHON27PATH=/cvmfs/cms.cern.ch/share/overrides/python:/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20/python:/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/python:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/coral/CORAL_2_3_21-pafccj7/slc7_amd64_gcc700/python:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/coral/CORAL_2_3_21-pafccj7/slc7_amd64_gcc700/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/xrootd/4.8.5-pafccj2/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/fastjet/3.3.0-pafccj/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw-tool-conf/45.0-pafccj26/lib/python2.7/site-packages == env: SRT_CMSSW_FWLITE_INCLUDE_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sherpa/2.2.10-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/hepmc/2.06.07-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/clhep/2.4.0.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/boost/1.67.0-pafccj/include == env: VC3_MACHINE_TARGET=x86_64/redhat7.4.1708 == env: VC3_ROOT_SED=/usr/ == env: OASIS_CERTIFICATES=/cvmfs/oasis.opensciencegrid.org/mis/certificates == env: ROOTSYS=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/lcg/root/6.14.09-pafccj6/ == env: SRT_PYTHIA8DATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/pythia8/240-pafccj5/share/Pythia8/xmldoc == env: SRT_HERWIGPATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/herwigpp/7.2.0-pafccj3/share/Herwig == env: CXX=g++ == env: SRT_QTLIB_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/lib == env: SRT_PYTHON3PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/share/overrides/python:/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20/python:/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/python:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw-tool-conf/45.0-pafccj26/lib/python3.6/site-packages == env: SHERPA_LIBRARY_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sherpa/2.2.10-pafccj2/lib/SHERPA-MC == env: SRT_PYTHON_VALGRIND_SUPP_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/python/2.7.14-pafccj/share/valgrind/valgrind-python.supp == env: HOME=/disk/vc3-root/vc3-home == env: SHLVL=3 == env: CMSSW_SEARCH_PATH=/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20/poison:/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20/src:/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20/external/slc7_amd64_gcc700/data:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/src:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/external/slc7_amd64_gcc700/data == env: VC3_VERSION_FULL_GZIP=1.5.0 == env: VC3_VERSION_MAKE=3.82 == env: GENEVA_DATA_DIR=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geneva/1.0-RC3-pafccj8/share/Geneva == env: DISK=361 == env: PARROT_CACHE=/disk/vc3-root/tmp/worker-230547-29293/cache/tmp == env: PARROT_CVMFS_ALIEN_CACHE=/disk/vc3-root/tmp/worker-230547-29293/cache/tmp/cvmfs == env: GOSAMCONTRIB_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gosamcontrib/2.0-20150803-pafccj == env: ROOT_TTREECACHE_PREFILL=0 == env: GENREFLEX=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/lcg/root/6.14.09-pafccj6/bin/genreflex == env: VINCIADATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/vincia/2.2.04-pafccj5/share/Vincia/xmldoc == env: SRT_G4LEDATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4EMLOW/7.3/data/G4EMLOW7.3 == env: SRT_GENREFLEX_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/lcg/root/6.14.09-pafccj6/bin/genreflex == env: SRT_GOSAMCONTRIB_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gosamcontrib/2.0-20150803-pafccj == env: SRT_ROOT_TTREECACHE_SIZE_SCRAMRTDEL=0 == env: SRT_EVTGENDATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/evtgen/1.6.0-pafccj5/share == env: SRT_GENEVA_DATA_DIR_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geneva/1.0-RC3-pafccj8/share/Geneva == env: GFAL_CONFIG_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/etc/gfal2.d/ == env: VC3_VERSION_FULL_WHICH=2.20.0 == env: MEMORY=1044 == env: HTTP_PROXY=http://eddie.crc.nd.edu:3128 == env: SRT_CMSSW_DATA_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms == env: HEPPDT_PARAM_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/heppdt/3.03.00-pafccj == env: SRT_LD_LIBRARY_PATH_SCRAMRT=/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20/biglib/slc7_amd64_gcc700:/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20/external/slc7_amd64_gcc700/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/biglib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/external/slc7_amd64_gcc700/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/llvm/7.1.0-pafccj/lib64:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib64:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/cuda/10.1.105-pafccj2/drivers == env: TMP=/disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N == env: PYTHONPATH=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib/python2.7/site-packages:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/python2.7/site-packages:python:python: == env: PYTHON3PATH=/cvmfs/cms.cern.ch/share/overrides/python:/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20/python:/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/python:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw-tool-conf/45.0-pafccj26/lib/python3.6/site-packages == env: QTLIB=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/lib == env: SCONS_LIB_DIR=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/scons/3.0.1-pafccj/lib == env: VC3_INSTALL_USER_HOME=/disk/vc3-root/vc3-home == env: VC3_VERSION_FULL_VC3_BUILDER_HANDS=1.0.0 == env: COMPILER_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj == env: FRONTIER_CLIENT=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/frontier_client/2.9.1/ == env: SRT_G4NEUTRONXSDATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4NEUTRONXS/1.4/data/G4NEUTRONXS1.4 == env: SRT_SHERPA_LIBRARY_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sherpa/2.2.10-pafccj2/lib/SHERPA-MC == env: VC3_ROOT_TAR=/usr/ == env: VC3_VERSION_PKG_CONFIG=0.27 == env: CLHEP_PARAM_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/clhep/2.4.0.0-pafccj2 == env: CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 == env: VC3_SH_PROFILE_WRAPPER=/disk/vc3-root/vc3-home/.vc3_sh-K0KHdQ.wrapper == env: PYTHIA8DATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/pythia8/240-pafccj5/share/Pythia8/xmldoc == env: HERWIGPATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/herwigpp/7.2.0-pafccj3/share/Herwig == env: SRT_FRONTIER_CLIENT_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/frontier_client/2.9.1/ == env: SRT_COMPILER_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj == env: SRT_CSC_TRACK_FINDER_DATA_DIR_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/CSCTrackFinderEmulation/1.2-pafccj/data/ == env: GCC_RUNTIME_ASAN=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib64/libasan.so == env: SRT_ROOT_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/madgraph5amcatnlo/2.6.0-pafccj12:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gosamcontrib/2.0-20150803-pafccj == env: CC=gcc == env: TEXMFHOME=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/rivet/2.7.0-pafccj8/share/Rivet/texmf == env: SRT_MLAS_DYNAMIC_CPU_ARCH_SCRAMRTDEL=0 == env: SRT_OPENBLAS_NUM_THREADS_SCRAMRTDEL=1 == env: SRT_GIT_TEMPLATE_DIR_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/git/2.19.0-pafccj/share/git-core/templates == env: SRT_G4SAIDXSDATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4SAIDDATA/1.1/data/G4SAIDDATA1.1 == env: CMSSW_VERSION=CMSSW_10_6_20 == env: SRT_GCC_RUNTIME_UBSAN_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib64/libubsan.so == env: SRT_LANG_SCRAMRTDEL=C == env: CCTOOLS_RESOURCE_MONITOR_INFO=1027 == env: VC3_VERSION_FULL_MAKE=3.82.0 == env: CORES=1 == env: PARROT_DEBUG_FLAGS= == env: SHERPA_SHARE_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sherpa/2.2.10-pafccj2/share/SHERPA-MC == env: VC3_MACHINE_ARCH=x86_64 == env: VC3_ROOT_VC3_BUILDER_HANDS=/disk/vc3-root/x86_64/redhat7.4.1708/vc3-builder-hands/v1.0.0 == env: CMS_OPENLOOPS_PREFIX=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/openloops/2.1.0-pafccj2 == env: GIT_EXEC_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/git/2.19.0-pafccj/libexec/git-core == env: LOCALRT=/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20 == env: CMSSW_BASE=/disk/vc3-root/tmp/worker-230547-29293/t.166186/CMSSW_10_6_20 == env: SRT_SCRAM_CXX11_ABI_SCRAMRTDEL=1 == env: SRT_VALGRIND_LIB_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/valgrind/3.13.0-pafccj/lib/valgrind == env: VC3_COMMAND_LINE=/disk/vc3-builder --var CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 --require cctools-wq-worker -- work_queue_worker -M lobster_kmohrman_EFT_CRC_postLHE_crc.* -t 300 -C catalog.cse.nd.edu,backup-catalog.cse.nd.edu -d all -o worker.log --cores=12 --memory=16000 --disk=18566 == env: _=/usr/bin/env === wrapper ready @ Fri Apr 16 00:18:16 EDT 2021 ==== working directory before execution @ Fri Apr 16 00:18:16 EDT 2021 ==== == dir: total 884 == dir: -rwxr-xr-x 2 kmohrman campus 1598 Apr 16 00:17 0015_3455_report.json == dir: -rwxr-xr-x 2 kmohrman campus 1599 Apr 16 00:17 0015_4423_report.json == dir: -rwxr-xr-x 2 kmohrman campus 1602 Apr 16 00:17 0015_7630_report.json == dir: -rwxr-xr-x 2 kmohrman campus 1602 Apr 16 00:17 0015_8749_report.json == dir: drwxr-xr-x 18 kmohrman campus 215 Apr 16 00:18 CMSSW_10_6_20 == dir: drwxr-xr-x 2 kmohrman campus 71 Apr 16 00:17 bin == dir: -rwxr-xr-x 5 kmohrman campus 627272 Apr 15 15:44 cctools-monitor == dir: -rw-r--r-- 1 kmohrman campus 0 Apr 16 00:17 cctools-monitor.summary == dir: drwxrwxrwx 2 kmohrman campus 6 Apr 16 00:17 cctools-temp-t.166186.YOAD9N == dir: -rw------- 1 kmohrman campus 51 Apr 16 00:18 cvmfschecksum.cms-ib.cern.ch == dir: -rw------- 1 kmohrman campus 51 Apr 16 00:18 cvmfschecksum.cms.cern.ch == dir: -rw------- 1 kmohrman campus 51 Apr 16 00:18 cvmfschecksum.oasis.opensciencegrid.org == dir: drwxr-xr-x 2 kmohrman campus 33 Apr 16 00:17 lib == dir: -rwxrwxrwx 1 kmohrman campus 146912 Apr 16 00:17 librmonitor_helper.so.pJWPIv == dir: -rw-r--r-- 2 kmohrman campus 868 Apr 16 00:17 merge_cfg.py == dir: -rw-r--r-- 2 kmohrman campus 808 Apr 16 00:17 merge_reports.py == dir: -rw-r--r-- 2 kmohrman campus 2528 Apr 16 00:17 parameters.json == dir: -rw------- 2 kmohrman campus 9917 Apr 16 00:17 proxy == dir: drwxr-xr-x 3 kmohrman campus 20 Apr 16 00:17 python == dir: -rw-r--r-- 5 kmohrman campus 570 Apr 15 15:44 report.json.in == dir: -rw-r--r-- 2 kmohrman campus 549 Apr 16 00:17 sandbox-CMSSW_10_6_20-slc7_amd64_gcc700.tar.bz2 == dir: drwxr-xr-x 4 kmohrman campus 37 Apr 16 00:17 siteconf == dir: -rw-r--r-- 1 kmohrman campus 11 Apr 16 00:18 t_wrapper_ready == dir: -rw-r--r-- 1 kmohrman campus 11 Apr 16 00:18 t_wrapper_start == dir: -rwxr-xr-x 5 kmohrman campus 47217 Apr 15 15:44 task.py == dir: -rwxr-xr-x 5 kmohrman campus 5986 Apr 15 15:44 wrapper.sh >>> data is {u'files': {u'info': {}, u'output_info': {}, u'skipped': []}, u'cpu_time': 0, u'exe_exit_code': 0, u'events_written': 0, u'output_size': 0, u'task_exit_code': 0, u'output_storage_element': u'', u'cache': {u'start_size': 0, u'type': 2, u'end_size': 0}, u'stageout_exit_code': 0, u'output_bare_size': 0, u'transfers': defaultdict(, {}), u'task_timing': {u'wrapper_ready': 0, u'wrapper_start': 0, u'prologue_end': 0, u'epilogue_end': 0, u'stage_out_end': 0, u'stage_in_end': 0, u'processing_end': 0}, u'events_per_run': 0} @ Fri Apr 16 00:18:23 2021 >>> Trying hdfs client access method @ Fri Apr 16 00:18:23 2021 >>> executing 'timeout 300 hdfs dfs -fs hdfs://eddie.crc.nd.edu:19000 -get /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_157630.root MAOD-00000_157630.root' @ Fri Apr 16 00:18:23 2021 >> using /disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N/tmpaEYiXd to store command output >> cmd: timeout: failed to run command 'hdfs': No such file or directory >>>>> Unable to copy input with hdfs client @ Fri Apr 16 00:18:23 2021 >>> Trying xrootd access method @ Fri Apr 16 00:18:23 2021 >>> executing 'env XRD_LOGLEVEL=Debug timeout 300 xrdfs deepthought.crc.nd.edu stat /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_157630.root' @ Fri Apr 16 00:18:23 2021 >> using /disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N/tmp6I3kOO to store command output >> cmd: [2021-04-16 00:18:24.020050 -0400][Warning][Utility ] Unable to process global config file: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:18:24.021663 -0400][Debug ][Utility ] Unable to process user config file: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:18:24.021823 -0400][Debug ][PlugInMgr ] Initializing plug-in manager... >> cmd: [2021-04-16 00:18:24.021873 -0400][Debug ][PlugInMgr ] No default plug-in, loading plug-in configs... >> cmd: [2021-04-16 00:18:24.021917 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /etc/xrootd/client.plugins.d... >> cmd: [2021-04-16 00:18:24.022006 -0400][Debug ][PlugInMgr ] Unable to process directory /etc/xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:18:24.022627 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /tmp/runos_singularity_home-20839078316581324623660965507796721359/.xrootd/client.plugins.d... >> cmd: [2021-04-16 00:18:24.022724 -0400][Debug ][PlugInMgr ] Unable to process directory /tmp/runos_singularity_home-20839078316581324623660965507796721359/.xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:18:24.022902 -0400][Debug ][App ] Executing: stat /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_157630.root >> cmd: [2021-04-16 00:18:24.023000 -0400][Debug ][Poller ] Available pollers: built-in >> cmd: [2021-04-16 00:18:24.023043 -0400][Debug ][Poller ] Attempting to create a poller according to preference: built-in >> cmd: [2021-04-16 00:18:24.023085 -0400][Debug ][Poller ] Creating poller: built-in >> cmd: [2021-04-16 00:18:24.023132 -0400][Debug ][Poller ] Creating and starting the built-in poller... >> cmd: [2021-04-16 00:18:24.023453 -0400][Debug ][Poller ] Using 1 poller threads >> cmd: [2021-04-16 00:18:24.023508 -0400][Debug ][TaskMgr ] Starting the task manager... >> cmd: [2021-04-16 00:18:24.023642 -0400][Debug ][TaskMgr ] Task manager started >> cmd: [2021-04-16 00:18:24.023692 -0400][Debug ][JobMgr ] Starting the job manager... >> cmd: [2021-04-16 00:18:24.023998 -0400][Debug ][JobMgr ] Job manager started, 3 workers >> cmd: [2021-04-16 00:18:24.024050 -0400][Debug ][TaskMgr ] Registering task: "FileTimer task" to be run at: [2021-04-16 00:18:24 -0400] >> cmd: [2021-04-16 00:18:24.024123 -0400][Debug ][PostMaster ] Creating new channel to: deepthought.crc.nd.edu:1094 1 stream(s) >> cmd: [2021-04-16 00:18:24.024186 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 >> cmd: [2021-04-16 00:18:24.025479 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" to be run at: [2021-04-16 00:18:39 -0400] >> cmd: [2021-04-16 00:18:24.026784 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094] Found 1 address(es): [::ffff:129.74.85.55]:1094 >> cmd: [2021-04-16 00:18:24.026934 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:129.74.85.55]:1094 >> cmd: [2021-04-16 00:18:24.027035 -0400][Debug ][Poller ] Adding socket 0xf23a10 to the poller >> cmd: [2021-04-16 00:18:24.027312 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Async connection call returned >> cmd: [2021-04-16 00:18:24.027431 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2021-04-16 00:18:24.027940 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: manager [], protocol version 400) >> cmd: [2021-04-16 00:18:24.028114 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: manager [], protocol version 400) >> cmd: [2021-04-16 00:18:24.032173 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d8civy101.crc.nd.edu&xrd.rn=v4.8.5, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2021-04-16 00:18:24.032521 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Logged in, session: 813d0000a60d000098070000583e0000 >> cmd: [2021-04-16 00:18:24.032597 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream 0 connected. >> cmd: [2021-04-16 00:18:24.032643 -0400][Debug ][Utility ] Monitor library name not set. No monitoring >> cmd: [2021-04-16 00:18:24.033418 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant04.crc.nd.edu:1096 1 stream(s) >> cmd: [2021-04-16 00:18:24.033481 -0400][Debug ][PostMaster ] [primeradiant04.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 >> cmd: [2021-04-16 00:18:24.035274 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant04.crc.nd.edu:1096" to be run at: [2021-04-16 00:18:39 -0400] >> cmd: [2021-04-16 00:18:24.037142 -0400][Debug ][PostMaster ] [primeradiant04.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.12]:1096 >> cmd: [2021-04-16 00:18:24.037282 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.12]:1096 >> cmd: [2021-04-16 00:18:24.037369 -0400][Debug ][Poller ] Adding socket 0x480016a0 to the poller >> cmd: [2021-04-16 00:18:24.037494 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2021-04-16 00:18:24.037594 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2021-04-16 00:18:24.037945 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 400) >> cmd: [2021-04-16 00:18:24.038116 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 400) >> cmd: [2021-04-16 00:18:24.041156 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d8civy101.crc.nd.edu&xrd.rn=v4.8.5, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2021-04-16 00:18:24.041539 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Logged in, session: 790f0000de11000097030000790f0000 >> cmd: [2021-04-16 00:18:24.041596 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10400,c:ssl,ca:ba240aa8.0|f5f0dfc2.0 >> cmd: [2021-04-16 00:18:24.041642 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2021-04-16 00:18:24.049864 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: [2021-04-16 00:18:24.123074 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Sending more authentication data for gsi >> cmd: [2021-04-16 00:18:24.128586 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Authenticated with gsi. >> cmd: [2021-04-16 00:18:24.128673 -0400][Debug ][PostMaster ] [primeradiant04.crc.nd.edu:1096 #0] Stream 0 connected. >> cmd: Path: /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_157630.root >> cmd: Id: 0 >> cmd: Size: 61557441 >> cmd: MTime: 2021-04-16 02:57:51 >> cmd: Flags: 24 (Offline|IsReadable) >> cmd: [2021-04-16 00:18:24.131097 -0400][Debug ][JobMgr ] Stopping the job manager... >> cmd: [2021-04-16 00:18:24.131681 -0400][Debug ][JobMgr ] Job manager stopped >> cmd: [2021-04-16 00:18:24.131735 -0400][Debug ][TaskMgr ] Stopping the task manager... >> cmd: [2021-04-16 00:18:24.131896 -0400][Debug ][TaskMgr ] Task manager stopped >> cmd: [2021-04-16 00:18:24.131943 -0400][Debug ][Poller ] Stopping the poller... >> cmd: [2021-04-16 00:18:24.132221 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" >> cmd: [2021-04-16 00:18:24.132273 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2021-04-16 00:18:24.132321 -0400][Debug ][Poller ] <[::ffff:10.32.79.65]:47728><--><[::ffff:129.74.85.55]:1094> Removing socket from the poller >> cmd: [2021-04-16 00:18:24.132411 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Destroying stream >> cmd: [2021-04-16 00:18:24.132462 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2021-04-16 00:18:24.132510 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant04.crc.nd.edu:1096" >> cmd: [2021-04-16 00:18:24.132582 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2021-04-16 00:18:24.132630 -0400][Debug ][Poller ] <[::ffff:10.32.79.65]:42298><--><[::ffff:149.165.243.12]:1096> Removing socket from the poller >> cmd: [2021-04-16 00:18:24.132703 -0400][Debug ][PostMaster ] [primeradiant04.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2021-04-16 00:18:24.132749 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Closing the socket >>> will stream using xrootd instead of copying @ Fri Apr 16 00:18:24 2021 >>> Trying hdfs client access method @ Fri Apr 16 00:18:24 2021 >>> executing 'timeout 300 hdfs dfs -fs hdfs://eddie.crc.nd.edu:19000 -get /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_154423.root MAOD-00000_154423.root' @ Fri Apr 16 00:18:24 2021 >> using /disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N/tmp3W5vCf to store command output >> cmd: timeout: failed to run command 'hdfs': No such file or directory >>>>> Unable to copy input with hdfs client @ Fri Apr 16 00:18:24 2021 >>> Trying xrootd access method @ Fri Apr 16 00:18:24 2021 >>> executing 'env XRD_LOGLEVEL=Debug timeout 300 xrdfs deepthought.crc.nd.edu stat /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_154423.root' @ Fri Apr 16 00:18:24 2021 >> using /disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N/tmp7hsbLk to store command output >> cmd: [2021-04-16 00:18:24.371138 -0400][Warning][Utility ] Unable to process global config file: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:18:24.372747 -0400][Debug ][Utility ] Unable to process user config file: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:18:24.372906 -0400][Debug ][PlugInMgr ] Initializing plug-in manager... >> cmd: [2021-04-16 00:18:24.372954 -0400][Debug ][PlugInMgr ] No default plug-in, loading plug-in configs... >> cmd: [2021-04-16 00:18:24.372998 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /etc/xrootd/client.plugins.d... >> cmd: [2021-04-16 00:18:24.373087 -0400][Debug ][PlugInMgr ] Unable to process directory /etc/xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:18:24.373702 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /tmp/runos_singularity_home-20839078316581324623660965507796721359/.xrootd/client.plugins.d... >> cmd: [2021-04-16 00:18:24.373798 -0400][Debug ][PlugInMgr ] Unable to process directory /tmp/runos_singularity_home-20839078316581324623660965507796721359/.xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:18:24.373969 -0400][Debug ][App ] Executing: stat /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_154423.root >> cmd: [2021-04-16 00:18:24.374068 -0400][Debug ][Poller ] Available pollers: built-in >> cmd: [2021-04-16 00:18:24.374110 -0400][Debug ][Poller ] Attempting to create a poller according to preference: built-in >> cmd: [2021-04-16 00:18:24.374152 -0400][Debug ][Poller ] Creating poller: built-in >> cmd: [2021-04-16 00:18:24.374199 -0400][Debug ][Poller ] Creating and starting the built-in poller... >> cmd: [2021-04-16 00:18:24.374525 -0400][Debug ][Poller ] Using 1 poller threads >> cmd: [2021-04-16 00:18:24.374579 -0400][Debug ][TaskMgr ] Starting the task manager... >> cmd: [2021-04-16 00:18:24.374713 -0400][Debug ][TaskMgr ] Task manager started >> cmd: [2021-04-16 00:18:24.374763 -0400][Debug ][JobMgr ] Starting the job manager... >> cmd: [2021-04-16 00:18:24.375068 -0400][Debug ][JobMgr ] Job manager started, 3 workers >> cmd: [2021-04-16 00:18:24.375121 -0400][Debug ][TaskMgr ] Registering task: "FileTimer task" to be run at: [2021-04-16 00:18:24 -0400] >> cmd: [2021-04-16 00:18:24.375194 -0400][Debug ][PostMaster ] Creating new channel to: deepthought.crc.nd.edu:1094 1 stream(s) >> cmd: [2021-04-16 00:18:24.375258 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 >> cmd: [2021-04-16 00:18:24.376817 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" to be run at: [2021-04-16 00:18:39 -0400] >> cmd: [2021-04-16 00:18:24.378045 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094] Found 1 address(es): [::ffff:129.74.85.55]:1094 >> cmd: [2021-04-16 00:18:24.378193 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:129.74.85.55]:1094 >> cmd: [2021-04-16 00:18:24.378292 -0400][Debug ][Poller ] Adding socket 0x1e1ea10 to the poller >> cmd: [2021-04-16 00:18:24.378569 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Async connection call returned >> cmd: [2021-04-16 00:18:24.378734 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2021-04-16 00:18:24.379177 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: manager [], protocol version 400) >> cmd: [2021-04-16 00:18:24.379352 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: manager [], protocol version 400) >> cmd: [2021-04-16 00:18:24.383044 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d8civy101.crc.nd.edu&xrd.rn=v4.8.5, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2021-04-16 00:18:24.383420 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Logged in, session: 833d0000a60d0000980700005a3e0000 >> cmd: [2021-04-16 00:18:24.383495 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream 0 connected. >> cmd: [2021-04-16 00:18:24.383541 -0400][Debug ][Utility ] Monitor library name not set. No monitoring >> cmd: [2021-04-16 00:18:24.384290 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant05.crc.nd.edu:1096 1 stream(s) >> cmd: [2021-04-16 00:18:24.384394 -0400][Debug ][PostMaster ] [primeradiant05.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 >> cmd: [2021-04-16 00:18:24.386415 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant05.crc.nd.edu:1096" to be run at: [2021-04-16 00:18:39 -0400] >> cmd: [2021-04-16 00:18:24.387968 -0400][Debug ][PostMaster ] [primeradiant05.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.13]:1096 >> cmd: [2021-04-16 00:18:24.388111 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.13]:1096 >> cmd: [2021-04-16 00:18:24.388196 -0400][Debug ][Poller ] Adding socket 0x700016a0 to the poller >> cmd: [2021-04-16 00:18:24.388316 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2021-04-16 00:18:24.388419 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2021-04-16 00:18:24.388796 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 400) >> cmd: [2021-04-16 00:18:24.388963 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 400) >> cmd: [2021-04-16 00:18:24.392579 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d8civy101.crc.nd.edu&xrd.rn=v4.8.5, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2021-04-16 00:18:24.392893 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Logged in, session: 610c0000d141000009020000610c0000 >> cmd: [2021-04-16 00:18:24.392941 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10400,c:ssl,ca:ba240aa8.0|f5f0dfc2.0 >> cmd: [2021-04-16 00:18:24.392985 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2021-04-16 00:18:24.400497 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: [2021-04-16 00:18:24.466469 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Sending more authentication data for gsi >> cmd: [2021-04-16 00:18:24.471391 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Authenticated with gsi. >> cmd: [2021-04-16 00:18:24.471479 -0400][Debug ][PostMaster ] [primeradiant05.crc.nd.edu:1096 #0] Stream 0 connected. >> cmd: Path: /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_154423.root >> cmd: Id: 0 >> cmd: Size: 61781032 >> cmd: MTime: 2021-04-16 02:23:10 >> cmd: Flags: 24 (Offline|IsReadable) >> cmd: [2021-04-16 00:18:24.473288 -0400][Debug ][JobMgr ] Stopping the job manager... >> cmd: [2021-04-16 00:18:24.473865 -0400][Debug ][JobMgr ] Job manager stopped >> cmd: [2021-04-16 00:18:24.473934 -0400][Debug ][TaskMgr ] Stopping the task manager... >> cmd: [2021-04-16 00:18:24.474105 -0400][Debug ][TaskMgr ] Task manager stopped >> cmd: [2021-04-16 00:18:24.474152 -0400][Debug ][Poller ] Stopping the poller... >> cmd: [2021-04-16 00:18:24.474455 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" >> cmd: [2021-04-16 00:18:24.474513 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2021-04-16 00:18:24.474561 -0400][Debug ][Poller ] <[::ffff:10.32.79.65]:47732><--><[::ffff:129.74.85.55]:1094> Removing socket from the poller >> cmd: [2021-04-16 00:18:24.474644 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Destroying stream >> cmd: [2021-04-16 00:18:24.474691 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2021-04-16 00:18:24.474740 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant05.crc.nd.edu:1096" >> cmd: [2021-04-16 00:18:24.474784 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2021-04-16 00:18:24.474857 -0400][Debug ][Poller ] <[::ffff:10.32.79.65]:43016><--><[::ffff:149.165.243.13]:1096> Removing socket from the poller >> cmd: [2021-04-16 00:18:24.474930 -0400][Debug ][PostMaster ] [primeradiant05.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2021-04-16 00:18:24.474976 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Closing the socket >>> will stream using xrootd instead of copying @ Fri Apr 16 00:18:24 2021 >>> Trying hdfs client access method @ Fri Apr 16 00:18:24 2021 >>> executing 'timeout 300 hdfs dfs -fs hdfs://eddie.crc.nd.edu:19000 -get /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_153455.root MAOD-00000_153455.root' @ Fri Apr 16 00:18:24 2021 >> using /disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N/tmpM48DDp to store command output >> cmd: timeout: failed to run command 'hdfs': No such file or directory >>>>> Unable to copy input with hdfs client @ Fri Apr 16 00:18:24 2021 >>> Trying xrootd access method @ Fri Apr 16 00:18:24 2021 >>> executing 'env XRD_LOGLEVEL=Debug timeout 300 xrdfs deepthought.crc.nd.edu stat /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_153455.root' @ Fri Apr 16 00:18:24 2021 >> using /disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N/tmpEc69Yf to store command output >> cmd: [2021-04-16 00:18:24.695288 -0400][Warning][Utility ] Unable to process global config file: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:18:24.696863 -0400][Debug ][Utility ] Unable to process user config file: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:18:24.697021 -0400][Debug ][PlugInMgr ] Initializing plug-in manager... >> cmd: [2021-04-16 00:18:24.697069 -0400][Debug ][PlugInMgr ] No default plug-in, loading plug-in configs... >> cmd: [2021-04-16 00:18:24.697112 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /etc/xrootd/client.plugins.d... >> cmd: [2021-04-16 00:18:24.697199 -0400][Debug ][PlugInMgr ] Unable to process directory /etc/xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:18:24.697804 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /tmp/runos_singularity_home-20839078316581324623660965507796721359/.xrootd/client.plugins.d... >> cmd: [2021-04-16 00:18:24.697899 -0400][Debug ][PlugInMgr ] Unable to process directory /tmp/runos_singularity_home-20839078316581324623660965507796721359/.xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:18:24.698070 -0400][Debug ][App ] Executing: stat /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_153455.root >> cmd: [2021-04-16 00:18:24.698169 -0400][Debug ][Poller ] Available pollers: built-in >> cmd: [2021-04-16 00:18:24.698210 -0400][Debug ][Poller ] Attempting to create a poller according to preference: built-in >> cmd: [2021-04-16 00:18:24.698251 -0400][Debug ][Poller ] Creating poller: built-in >> cmd: [2021-04-16 00:18:24.698296 -0400][Debug ][Poller ] Creating and starting the built-in poller... >> cmd: [2021-04-16 00:18:24.698607 -0400][Debug ][Poller ] Using 1 poller threads >> cmd: [2021-04-16 00:18:24.698659 -0400][Debug ][TaskMgr ] Starting the task manager... >> cmd: [2021-04-16 00:18:24.698790 -0400][Debug ][TaskMgr ] Task manager started >> cmd: [2021-04-16 00:18:24.698834 -0400][Debug ][JobMgr ] Starting the job manager... >> cmd: [2021-04-16 00:18:24.699132 -0400][Debug ][JobMgr ] Job manager started, 3 workers >> cmd: [2021-04-16 00:18:24.699184 -0400][Debug ][TaskMgr ] Registering task: "FileTimer task" to be run at: [2021-04-16 00:18:24 -0400] >> cmd: [2021-04-16 00:18:24.699255 -0400][Debug ][PostMaster ] Creating new channel to: deepthought.crc.nd.edu:1094 1 stream(s) >> cmd: [2021-04-16 00:18:24.699318 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 >> cmd: [2021-04-16 00:18:24.700863 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" to be run at: [2021-04-16 00:18:39 -0400] >> cmd: [2021-04-16 00:18:24.702120 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094] Found 1 address(es): [::ffff:129.74.85.55]:1094 >> cmd: [2021-04-16 00:18:24.702266 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:129.74.85.55]:1094 >> cmd: [2021-04-16 00:18:24.702367 -0400][Debug ][Poller ] Adding socket 0x76aa10 to the poller >> cmd: [2021-04-16 00:18:24.702614 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Async connection call returned >> cmd: [2021-04-16 00:18:24.702765 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2021-04-16 00:18:24.703190 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: manager [], protocol version 400) >> cmd: [2021-04-16 00:18:24.703368 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: manager [], protocol version 400) >> cmd: [2021-04-16 00:18:24.707063 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d8civy101.crc.nd.edu&xrd.rn=v4.8.5, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2021-04-16 00:18:24.707420 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Logged in, session: 853d0000a60d0000980700005c3e0000 >> cmd: [2021-04-16 00:18:24.707495 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream 0 connected. >> cmd: [2021-04-16 00:18:24.707550 -0400][Debug ][Utility ] Monitor library name not set. No monitoring >> cmd: [2021-04-16 00:18:24.708169 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant05.crc.nd.edu:1096 1 stream(s) >> cmd: [2021-04-16 00:18:24.708267 -0400][Debug ][PostMaster ] [primeradiant05.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 >> cmd: [2021-04-16 00:18:24.710034 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant05.crc.nd.edu:1096" to be run at: [2021-04-16 00:18:39 -0400] >> cmd: [2021-04-16 00:18:24.711224 -0400][Debug ][PostMaster ] [primeradiant05.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.13]:1096 >> cmd: [2021-04-16 00:18:24.711368 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.13]:1096 >> cmd: [2021-04-16 00:18:24.711456 -0400][Debug ][Poller ] Adding socket 0x9c0016a0 to the poller >> cmd: [2021-04-16 00:18:24.711580 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2021-04-16 00:18:24.711698 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2021-04-16 00:18:24.712049 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 400) >> cmd: [2021-04-16 00:18:24.712216 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 400) >> cmd: [2021-04-16 00:18:24.715133 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d8civy101.crc.nd.edu&xrd.rn=v4.8.5, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2021-04-16 00:18:24.715556 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Logged in, session: 620c0000d14100001f030000620c0000 >> cmd: [2021-04-16 00:18:24.715607 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10400,c:ssl,ca:ba240aa8.0|f5f0dfc2.0 >> cmd: [2021-04-16 00:18:24.715651 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2021-04-16 00:18:24.723117 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: [2021-04-16 00:18:24.816621 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Sending more authentication data for gsi >> cmd: [2021-04-16 00:18:24.821522 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Authenticated with gsi. >> cmd: [2021-04-16 00:18:24.821608 -0400][Debug ][PostMaster ] [primeradiant05.crc.nd.edu:1096 #0] Stream 0 connected. >> cmd: Path: /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_153455.root >> cmd: Id: 0 >> cmd: Size: 62142369 >> cmd: MTime: 2021-04-16 02:13:24 >> cmd: Flags: 24 (Offline|IsReadable) >> cmd: [2021-04-16 00:18:24.823442 -0400][Debug ][JobMgr ] Stopping the job manager... >> cmd: [2021-04-16 00:18:24.824022 -0400][Debug ][JobMgr ] Job manager stopped >> cmd: [2021-04-16 00:18:24.824087 -0400][Debug ][TaskMgr ] Stopping the task manager... >> cmd: [2021-04-16 00:18:24.824275 -0400][Debug ][TaskMgr ] Task manager stopped >> cmd: [2021-04-16 00:18:24.824334 -0400][Debug ][Poller ] Stopping the poller... >> cmd: [2021-04-16 00:18:24.824634 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" >> cmd: [2021-04-16 00:18:24.824696 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2021-04-16 00:18:24.824744 -0400][Debug ][Poller ] <[::ffff:10.32.79.65]:47736><--><[::ffff:129.74.85.55]:1094> Removing socket from the poller >> cmd: [2021-04-16 00:18:24.824825 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Destroying stream >> cmd: [2021-04-16 00:18:24.824871 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2021-04-16 00:18:24.824918 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant05.crc.nd.edu:1096" >> cmd: [2021-04-16 00:18:24.824959 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2021-04-16 00:18:24.825011 -0400][Debug ][Poller ] <[::ffff:10.32.79.65]:43020><--><[::ffff:149.165.243.13]:1096> Removing socket from the poller >> cmd: [2021-04-16 00:18:24.825086 -0400][Debug ][PostMaster ] [primeradiant05.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2021-04-16 00:18:24.825130 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Closing the socket >>> will stream using xrootd instead of copying @ Fri Apr 16 00:18:24 2021 >>> Trying hdfs client access method @ Fri Apr 16 00:18:24 2021 >>> executing 'timeout 300 hdfs dfs -fs hdfs://eddie.crc.nd.edu:19000 -get /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_158749.root MAOD-00000_158749.root' @ Fri Apr 16 00:18:24 2021 >> using /disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N/tmpB59MNL to store command output >> cmd: timeout: failed to run command 'hdfs': No such file or directory >>>>> Unable to copy input with hdfs client @ Fri Apr 16 00:18:24 2021 >>> Trying xrootd access method @ Fri Apr 16 00:18:24 2021 >>> executing 'env XRD_LOGLEVEL=Debug timeout 300 xrdfs deepthought.crc.nd.edu stat /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_158749.root' @ Fri Apr 16 00:18:24 2021 >> using /disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N/tmpaKJOsz to store command output >> cmd: [2021-04-16 00:18:25.064270 -0400][Warning][Utility ] Unable to process global config file: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:18:25.065899 -0400][Debug ][Utility ] Unable to process user config file: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:18:25.066059 -0400][Debug ][PlugInMgr ] Initializing plug-in manager... >> cmd: [2021-04-16 00:18:25.066109 -0400][Debug ][PlugInMgr ] No default plug-in, loading plug-in configs... >> cmd: [2021-04-16 00:18:25.066153 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /etc/xrootd/client.plugins.d... >> cmd: [2021-04-16 00:18:25.066243 -0400][Debug ][PlugInMgr ] Unable to process directory /etc/xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:18:25.066859 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /tmp/runos_singularity_home-20839078316581324623660965507796721359/.xrootd/client.plugins.d... >> cmd: [2021-04-16 00:18:25.066955 -0400][Debug ][PlugInMgr ] Unable to process directory /tmp/runos_singularity_home-20839078316581324623660965507796721359/.xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:18:25.067124 -0400][Debug ][App ] Executing: stat /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_158749.root >> cmd: [2021-04-16 00:18:25.067221 -0400][Debug ][Poller ] Available pollers: built-in >> cmd: [2021-04-16 00:18:25.067264 -0400][Debug ][Poller ] Attempting to create a poller according to preference: built-in >> cmd: [2021-04-16 00:18:25.067307 -0400][Debug ][Poller ] Creating poller: built-in >> cmd: [2021-04-16 00:18:25.067361 -0400][Debug ][Poller ] Creating and starting the built-in poller... >> cmd: [2021-04-16 00:18:25.067691 -0400][Debug ][Poller ] Using 1 poller threads >> cmd: [2021-04-16 00:18:25.067754 -0400][Debug ][TaskMgr ] Starting the task manager... >> cmd: [2021-04-16 00:18:25.067888 -0400][Debug ][TaskMgr ] Task manager started >> cmd: [2021-04-16 00:18:25.067939 -0400][Debug ][JobMgr ] Starting the job manager... >> cmd: [2021-04-16 00:18:25.068246 -0400][Debug ][JobMgr ] Job manager started, 3 workers >> cmd: [2021-04-16 00:18:25.068299 -0400][Debug ][TaskMgr ] Registering task: "FileTimer task" to be run at: [2021-04-16 00:18:25 -0400] >> cmd: [2021-04-16 00:18:25.068377 -0400][Debug ][PostMaster ] Creating new channel to: deepthought.crc.nd.edu:1094 1 stream(s) >> cmd: [2021-04-16 00:18:25.068446 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 >> cmd: [2021-04-16 00:18:25.070006 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" to be run at: [2021-04-16 00:18:40 -0400] >> cmd: [2021-04-16 00:18:25.071417 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094] Found 1 address(es): [::ffff:129.74.85.55]:1094 >> cmd: [2021-04-16 00:18:25.071568 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:129.74.85.55]:1094 >> cmd: [2021-04-16 00:18:25.071670 -0400][Debug ][Poller ] Adding socket 0x63fa10 to the poller >> cmd: [2021-04-16 00:18:25.072002 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Async connection call returned >> cmd: [2021-04-16 00:18:25.072127 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2021-04-16 00:18:25.072546 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: manager [], protocol version 400) >> cmd: [2021-04-16 00:18:25.072723 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: manager [], protocol version 400) >> cmd: [2021-04-16 00:18:25.076661 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d8civy101.crc.nd.edu&xrd.rn=v4.8.5, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2021-04-16 00:18:25.077044 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Logged in, session: 873d0000a60d0000f80700005e3e0000 >> cmd: [2021-04-16 00:18:25.077116 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream 0 connected. >> cmd: [2021-04-16 00:18:25.077162 -0400][Debug ][Utility ] Monitor library name not set. No monitoring >> cmd: [2021-04-16 00:18:25.077951 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant01.crc.nd.edu:1096 1 stream(s) >> cmd: [2021-04-16 00:18:25.078014 -0400][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 >> cmd: [2021-04-16 00:18:25.080223 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant01.crc.nd.edu:1096" to be run at: [2021-04-16 00:18:40 -0400] >> cmd: [2021-04-16 00:18:25.081797 -0400][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.5]:1096 >> cmd: [2021-04-16 00:18:25.081935 -0400][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.5]:1096 >> cmd: [2021-04-16 00:18:25.082018 -0400][Debug ][Poller ] Adding socket 0x940016a0 to the poller >> cmd: [2021-04-16 00:18:25.082156 -0400][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2021-04-16 00:18:25.082254 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2021-04-16 00:18:25.082612 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 400) >> cmd: [2021-04-16 00:18:25.082783 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 400) >> cmd: [2021-04-16 00:18:25.085725 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d8civy101.crc.nd.edu&xrd.rn=v4.8.5, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2021-04-16 00:18:25.086092 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Logged in, session: 7e7a0000264d0000ae0100007e7a0000 >> cmd: [2021-04-16 00:18:25.086140 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10400,c:ssl,ca:ba240aa8.0|f5f0dfc2.0 >> cmd: [2021-04-16 00:18:25.086184 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2021-04-16 00:18:25.093646 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: [2021-04-16 00:18:25.167724 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Sending more authentication data for gsi >> cmd: [2021-04-16 00:18:25.173818 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Authenticated with gsi. >> cmd: [2021-04-16 00:18:25.173922 -0400][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1096 #0] Stream 0 connected. >> cmd: Path: /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_158749.root >> cmd: Id: 0 >> cmd: Size: 60546418 >> cmd: MTime: 2021-04-16 03:00:39 >> cmd: Flags: 24 (Offline|IsReadable) >> cmd: [2021-04-16 00:18:25.176288 -0400][Debug ][JobMgr ] Stopping the job manager... >> cmd: [2021-04-16 00:18:25.176875 -0400][Debug ][JobMgr ] Job manager stopped >> cmd: [2021-04-16 00:18:25.176943 -0400][Debug ][TaskMgr ] Stopping the task manager... >> cmd: [2021-04-16 00:18:25.177112 -0400][Debug ][TaskMgr ] Task manager stopped >> cmd: [2021-04-16 00:18:25.177158 -0400][Debug ][Poller ] Stopping the poller... >> cmd: [2021-04-16 00:18:25.177508 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" >> cmd: [2021-04-16 00:18:25.177568 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2021-04-16 00:18:25.177617 -0400][Debug ][Poller ] <[::ffff:10.32.79.65]:47740><--><[::ffff:129.74.85.55]:1094> Removing socket from the poller >> cmd: [2021-04-16 00:18:25.177703 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Destroying stream >> cmd: [2021-04-16 00:18:25.177750 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2021-04-16 00:18:25.177798 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant01.crc.nd.edu:1096" >> cmd: [2021-04-16 00:18:25.177842 -0400][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2021-04-16 00:18:25.177885 -0400][Debug ][Poller ] <[::ffff:10.32.79.65]:60248><--><[::ffff:149.165.243.5]:1096> Removing socket from the poller >> cmd: [2021-04-16 00:18:25.177957 -0400][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2021-04-16 00:18:25.178002 -0400][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Closing the socket >>> will stream using xrootd instead of copying @ Fri Apr 16 00:18:25 2021 >>> Bypassing further access checks and using 'root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/' for input @ Fri Apr 16 00:18:25 2021 >>> modified input files @ Fri Apr 16 00:18:25 2021 >> input: root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_157630.root >> input: root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_154423.root >> input: root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_153455.root >> input: root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_158749.root >>> updated parameters are @ Fri Apr 16 00:18:25 2021 >> json: { >> json: "accelerate stage-in": 3, >> json: "arguments": [ >> json: "outputFile=MAOD-00000.root" >> json: ], >> json: "default ce": "T3_US_NotreDame", >> json: "default host": "earth.crc.nd.edu", >> json: "default se": "T3_US_NotreDame", >> json: "disable streaming": false, >> json: "epilogue": [ >> json: "python", >> json: "merge_reports.py", >> json: "report.json", >> json: "0015_3455_report.json", >> json: "0015_4423_report.json", >> json: "0015_7630_report.json", >> json: "0015_8749_report.json" >> json: ], >> json: "executable": "cmsRun", >> json: "file map": { >> json: "root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_153455.root": "mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_153455.root", >> json: "root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_154423.root": "mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_154423.root", >> json: "root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_157630.root": "mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_157630.root", >> json: "root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_158749.root": "mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_158749.root" >> json: }, >> json: "gridpack": false, >> json: "input": [ >> json: "root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/" >> json: ], >> json: "mask": { >> json: "events": -1, >> json: "files": [ >> json: "root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_157630.root", >> json: "root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_154423.root", >> json: "root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_153455.root", >> json: "root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_158749.root" >> json: ], >> json: "lumis": null >> json: }, >> json: "monitoring": { >> json: "monitorid": null, >> json: "syncid": null, >> json: "taskid": "lobster_kmohrman_EFT_CRC_postLHE_crc_20210414_1958_7e3da5c056bebd75" >> json: }, >> json: "output": [ >> json: "hdfs://eddie.crc.nd.edu:19000/store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/", >> json: "root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/", >> json: "gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1", >> json: "gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1", >> json: "file:///hadoop/store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/" >> json: ], >> json: "output files": [ >> json: [ >> json: "MAOD-00000.root", >> json: "mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_166186.root" >> json: ] >> json: ], >> json: "prologue": null, >> json: "pset": "merge_cfg.py", >> json: "want summary": true >> json: } >>> using sync CE T3_US_NotreDame @ Fri Apr 16 00:18:25 2021 >>> config file fragment @ Fri Apr 16 00:18:27 2021 >> pset: >> pset: import FWCore.ParameterSet.Config as cms >> pset: process.Timing = cms.Service("Timing", >> pset: useJobReport = cms.untracked.bool(True), >> pset: summaryOnly = cms.untracked.bool(True)) >> pset: >> pset: process.maxEvents = cms.untracked.PSet(input = cms.untracked.int32(-1)) >> pset: >> pset: import os >> pset: _, major, minor, _ = os.environ["CMSSW_VERSION"].split('_', 3) >> pset: if int(major) >= 7 and int(minor) >= 4: >> pset: xrdstats = cms.Service("XrdAdaptor::XrdStatisticsService", cms.untracked.PSet(reportToFJR = cms.untracked.bool(True))) >> pset: process.add_(xrdstats) >> pset: >> pset: for prod in process.producers.values(): >> pset: if prod.hasParameter('nEvents') and prod.type_() == 'ExternalLHEProducer': >> pset: prod.nEvents = cms.untracked.uint32(-1) >> pset: >> pset: process.source.fileNames = cms.untracked.vstring(['root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_157630.root', 'root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_154423.root', 'root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_153455.root', 'root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_158749.root']) >> pset: if hasattr(process, 'options'): >> pset: process.options.wantSummary = cms.untracked.bool(True) >> pset: else: >> pset: process.options = cms.untracked.PSet(wantSummary = cms.untracked.bool(True)) >>> executing 'cmsRun -j report.xml merge_cfg_mod.py outputFile=MAOD-00000.root' @ Fri Apr 16 00:18:27 2021 >> using /disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N/tmpLFs_sY to store command output >> cmd: 16-Apr-2021 00:18:34 EDT Initiating request to open file root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_157630.root >> cmd: 16-Apr-2021 00:18:44 EDT Successfully opened file root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_157630.root >> cmd: Begin processing the 1st record. Run 1, Event 10, LumiSection 13670 on stream 0 at 16-Apr-2021 00:19:03.707 EDT >> cmd: 16-Apr-2021 00:19:14 EDT Closed file root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_157630.root >> cmd: 16-Apr-2021 00:19:14 EDT Initiating request to open file root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_154423.root >> cmd: 16-Apr-2021 00:19:17 EDT Successfully opened file root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_154423.root >> cmd: 16-Apr-2021 00:19:42 EDT Closed file root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_154423.root >> cmd: 16-Apr-2021 00:19:42 EDT Initiating request to open file root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_153455.root >> cmd: 16-Apr-2021 00:19:44 EDT Successfully opened file root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_153455.root >> cmd: Begin processing the 1001st record. Run 1, Event 285, LumiSection 34522 on stream 0 at 16-Apr-2021 00:20:01.636 EDT >> cmd: 16-Apr-2021 00:20:08 EDT Closed file root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_153455.root >> cmd: 16-Apr-2021 00:20:08 EDT Initiating request to open file root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_158749.root >> cmd: 16-Apr-2021 00:20:10 EDT Successfully opened file root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_158749.root >> cmd: 16-Apr-2021 00:20:32 EDT Closed file root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_158749.root >> cmd: >> cmd: TrigReport ---------- Event Summary ------------ >> cmd: TrigReport Events total = 1816 passed = 1816 failed = 0 >> cmd: >> cmd: TrigReport ---------- Path Summary ------------ >> cmd: TrigReport Trig Bit# Executed Passed Failed Error Name >> cmd: >> cmd: TrigReport -------End-Path Summary ------------ >> cmd: TrigReport Trig Bit# Executed Passed Failed Error Name >> cmd: TrigReport 0 0 1816 1816 0 0 end >> cmd: >> cmd: TrigReport ------ Modules in End-Path: end ------------ >> cmd: TrigReport Trig Bit# Visited Passed Failed Error Name >> cmd: TrigReport 0 0 1816 1816 0 0 out >> cmd: >> cmd: TrigReport ---------- Module Summary ------------ >> cmd: TrigReport Visited Executed Passed Failed Error Name >> cmd: TrigReport 1816 1816 1816 0 0 end >> cmd: TrigReport 1816 1816 1816 0 0 out >> cmd: >> cmd: TimeReport ---------- Event Summary ---[sec]---- >> cmd: TimeReport event loop CPU/event = 0.015884 >> cmd: TimeReport event loop Real/event = 0.060123 >> cmd: TimeReport sum Streams Real/event = 0.020455 >> cmd: TimeReport efficiency CPU/Real/thread = 0.264193 >> cmd: >> cmd: TimeReport ---------- Path Summary ---[Real sec]---- >> cmd: TimeReport per event per exec Name >> cmd: TimeReport per event per exec Name >> cmd: >> cmd: TimeReport -------End-Path Summary ---[Real sec]---- >> cmd: TimeReport per event per exec Name >> cmd: TimeReport 0.020299 0.020299 end >> cmd: TimeReport per event per exec Name >> cmd: >> cmd: TimeReport ------ Modules in End-Path: end ---[Real sec]---- >> cmd: TimeReport per event per visit Name >> cmd: TimeReport 0.000139 0.000139 out >> cmd: TimeReport per event per visit Name >> cmd: >> cmd: TimeReport ---------- Module Summary ---[Real sec]---- >> cmd: TimeReport per event per exec per visit Name >> cmd: TimeReport 0.000006 0.000006 0.000006 end >> cmd: TimeReport 0.000139 0.000139 0.000139 out >> cmd: TimeReport per event per exec per visit Name >> cmd: >> cmd: T---Report end! >> cmd: >> cmd: TimeReport> Time report complete in 127.711 seconds >> cmd: Time Summary: >> cmd: - Min event: 0.00954986 >> cmd: - Max event: 2.45024 >> cmd: - Avg event: 0.0203048 >> cmd: - Total loop: 106.411 >> cmd: - Total init: 18.5176 >> cmd: - Total job: 127.711 >> cmd: - EventSetup Lock: 0 >> cmd: - EventSetup Get: 0 >> cmd: Event Throughput: 17.0659 ev/s >> cmd: CPU Summary: >> cmd: - Total loop: 61.4159 >> cmd: - Total init: 8.05643 >> cmd: - Total extra: 0 >> cmd: - Total job: 71.9907 >> cmd: Processing Summary: >> cmd: - Number of Events: 1816 >> cmd: - Number of Global Begin Lumi Calls: 12 >> cmd: - Number of Global Begin Run Calls: 12 >> cmd: >> cmd: >> cmd: ============================================= >> cmd: >> cmd: MessageLogger Summary >> cmd: >> cmd: type category sev module subroutine count total >> cmd: ---- -------------------- -- ---------------- ---------------- ----- ----- >> cmd: 1 TimeReport -e AfterModEndJob 1 1 >> cmd: 2 fileAction -s file_close 4 4 >> cmd: 3 fileAction -s file_open 8 8 >> cmd: >> cmd: type category Examples: run/evt run/evt run/evt >> cmd: ---- -------------------- ---------------- ---------------- ---------------- >> cmd: 1 TimeReport PostGlobalEndRun >> cmd: 2 fileAction PostProcessEvent PostProcessEvent PostGlobalEndRun >> cmd: 3 fileAction pre-events pre-events PostProcessEvent >> cmd: >> cmd: Severity # Occurrences Total Occurrences >> cmd: -------- ------------- ----------------- >> cmd: Error 1 1 >> cmd: System 12 12 >> cmd: >> cmd: dropped waiting message count 0 >>> executable returned with exit code 0. @ Fri Apr 16 00:20:35 2021 >>> epilogue @ Fri Apr 16 00:20:48 2021 >>> executing 'python merge_reports.py report.json 0015_3455_report.json 0015_4423_report.json 0015_7630_report.json 0015_8749_report.json' @ Fri Apr 16 00:20:48 2021 >> using /disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N/tmp_otG4m to store command output >> cmd: >> merging 0015_3455_report.json >> cmd: >> merging 0015_4423_report.json >> cmd: >> merging 0015_7630_report.json >> cmd: >> merging 0015_8749_report.json >>> executing 'timeout 300 hdfs dfs -fs hdfs://eddie.crc.nd.edu:19000 -put MAOD-00000.root /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_166186.root' @ Fri Apr 16 00:20:57 2021 >> using /disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N/tmpIiS_CP to store command output >> cmd: timeout: failed to run command 'hdfs': No such file or directory >>> Checking output file transfer. @ Fri Apr 16 00:20:57 2021 >>> executing 'env XRD_LOGLEVEL=Debug xrdcp MAOD-00000.root root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_166186.root' @ Fri Apr 16 00:20:57 2021 >> using /disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N/tmpg8x5gP to store command output >> cmd: [2021-04-16 00:20:57.397191 -0400][Warning][Utility ] Unable to process global config file: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:20:57.398425 -0400][Debug ][Utility ] Unable to process user config file: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:20:57.398604 -0400][Debug ][PlugInMgr ] Initializing plug-in manager... >> cmd: [2021-04-16 00:20:57.398662 -0400][Debug ][PlugInMgr ] No default plug-in, loading plug-in configs... >> cmd: [2021-04-16 00:20:57.398714 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /etc/xrootd/client.plugins.d... >> cmd: [2021-04-16 00:20:57.398819 -0400][Debug ][PlugInMgr ] Unable to process directory /etc/xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:20:57.399223 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /tmp/runos_singularity_home-20839078316581324623660965507796721359/.xrootd/client.plugins.d... >> cmd: [2021-04-16 00:20:57.399332 -0400][Debug ][PlugInMgr ] Unable to process directory /tmp/runos_singularity_home-20839078316581324623660965507796721359/.xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2021-04-16 00:20:57.399623 -0400][Debug ][Poller ] Available pollers: built-in >> cmd: [2021-04-16 00:20:57.399684 -0400][Debug ][Poller ] Attempting to create a poller according to preference: built-in >> cmd: [2021-04-16 00:20:57.399734 -0400][Debug ][Poller ] Creating poller: built-in >> cmd: [2021-04-16 00:20:57.399790 -0400][Debug ][Poller ] Creating and starting the built-in poller... >> cmd: [2021-04-16 00:20:57.400190 -0400][Debug ][Poller ] Using 1 poller threads >> cmd: [2021-04-16 00:20:57.400245 -0400][Debug ][TaskMgr ] Starting the task manager... >> cmd: [2021-04-16 00:20:57.400431 -0400][Debug ][TaskMgr ] Task manager started >> cmd: [2021-04-16 00:20:57.400492 -0400][Debug ][JobMgr ] Starting the job manager... >> cmd: [2021-04-16 00:20:57.400921 -0400][Debug ][JobMgr ] Job manager started, 3 workers >> cmd: [2021-04-16 00:20:57.400984 -0400][Debug ][TaskMgr ] Registering task: "FileTimer task" to be run at: [2021-04-16 00:20:57 -0400] >> cmd: [2021-04-16 00:20:57.401068 -0400][Debug ][PostMaster ] Creating new channel to: deepthought.crc.nd.edu:1094 1 stream(s) >> cmd: [2021-04-16 00:20:57.401140 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 >> cmd: [2021-04-16 00:20:57.402549 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" to be run at: [2021-04-16 00:21:12 -0400] >> cmd: [2021-04-16 00:20:57.403662 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094] Found 1 address(es): [::ffff:129.74.85.55]:1094 >> cmd: [2021-04-16 00:20:57.403812 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:129.74.85.55]:1094 >> cmd: [2021-04-16 00:20:57.403913 -0400][Debug ][Poller ] Adding socket 0x1b3b100 to the poller >> cmd: [2021-04-16 00:20:57.404180 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Async connection call returned >> cmd: [2021-04-16 00:20:57.404295 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2021-04-16 00:20:57.404737 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: manager [], protocol version 400) >> cmd: [2021-04-16 00:20:57.404921 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: manager [], protocol version 400) >> cmd: [2021-04-16 00:20:57.407936 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=d8civy101.crc.nd.edu&xrd.rn=v4.8.5, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2021-04-16 00:20:57.408268 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Logged in, session: fa400000a60d000036020000d1410000 >> cmd: [2021-04-16 00:20:57.408348 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream 0 connected. >> cmd: [2021-04-16 00:20:57.408405 -0400][Debug ][Utility ] Monitor library name not set. No monitoring >> cmd: [2021-04-16 00:20:57.765607 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x1b36040" to be run at: [2021-04-16 00:21:02 -0400] >> cmd: [2021-04-16 00:21:02.401431 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x1b36040" >> cmd: [2021-04-16 00:21:02.402237 -0400][Debug ][XRootD ] [deepthought.crc.nd.edu:1094] Handling error while processing kXR_stat (path: /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_166186.root, flags: none): [ERROR] Error response: No such file or directory. >> cmd: [2021-04-16 00:21:02.402651 -0400][Debug ][Utility ] CopyProcess: 2 jobs to prepare >> cmd: [2021-04-16 00:21:02.403491 -0400][Debug ][XRootD ] [deepthought.crc.nd.edu:1094] Handling error while processing kXR_stat (path: /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_166186.root, flags: none): [ERROR] Error response: No such file or directory. >> cmd: [2021-04-16 00:21:02.403657 -0400][Debug ][Utility ] Creating a classic copy job, from file://localhost/disk/vc3-root/tmp/worker-230547-29293/t.166186/MAOD-00000.root to root://deepthought.crc.nd.edu:1094//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_166186.root >> cmd: [2021-04-16 00:21:02.403803 -0400][Debug ][Utility ] Opening file://localhost/disk/vc3-root/tmp/worker-230547-29293/t.166186/MAOD-00000.root for reading >> cmd: [2021-04-16 00:21:02.403885 -0400][Debug ][File ] [0x1b3b700@file://localhost/disk/vc3-root/tmp/worker-230547-29293/t.166186/MAOD-00000.root] Sending an open command >> cmd: [2021-04-16 00:21:02.404777 -0400][Debug ][File ] [0x1b3b700@file://localhost/disk/vc3-root/tmp/worker-230547-29293/t.166186/MAOD-00000.root] Open has returned with status [SUCCESS] >> cmd: [2021-04-16 00:21:02.404833 -0400][Debug ][File ] [0x1b3b700@file://localhost/disk/vc3-root/tmp/worker-230547-29293/t.166186/MAOD-00000.root] successfully opened at localhost, handle: 0x18, session id: 1 >> cmd: [2021-04-16 00:21:02.404986 -0400][Debug ][Utility ] Opening root://deepthought.crc.nd.edu:1094//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_166186.root?oss.asize=239173725 for writing >> cmd: [2021-04-16 00:21:02.405074 -0400][Debug ][File ] [0x1b3cbc0@root://deepthought.crc.nd.edu:1094//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_166186.root?oss.asize=239173725] Sending an open command >> cmd: [2021-04-16 00:21:02.405815 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant01.crc.nd.edu:1096 1 stream(s) >> cmd: [2021-04-16 00:21:02.405880 -0400][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 >> cmd: [2021-04-16 00:21:02.408186 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant01.crc.nd.edu:1096" to be run at: [2021-04-16 00:21:17 -0400] >> cmd: [2021-04-16 00:21:02.409603 -0400][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.5]:1096 >> cmd: [2021-04-16 00:21:02.409803 -0400][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.5]:1096 >> cmd: [2021-04-16 00:21:02.409909 -0400][Debug ][Poller ] Adding socket 0xc8001880 to the poller >> cmd: [2021-04-16 00:21:02.410072 -0400][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2021-04-16 00:21:02.410208 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2021-04-16 00:21:02.410643 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 400) >> cmd: [2021-04-16 00:21:02.410854 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 400) >> cmd: [2021-04-16 00:21:02.414205 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=d8civy101.crc.nd.edu&xrd.rn=v4.8.5, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2021-04-16 00:21:02.414545 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Logged in, session: c67b0000264d0000330c0000c67b0000 >> cmd: [2021-04-16 00:21:02.414598 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10400,c:ssl,ca:ba240aa8.0|f5f0dfc2.0 >> cmd: [2021-04-16 00:21:02.414650 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2021-04-16 00:21:02.422106 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: [2021-04-16 00:21:02.494167 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Sending more authentication data for gsi >> cmd: [2021-04-16 00:21:02.499195 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Authenticated with gsi. >> cmd: [2021-04-16 00:21:02.499296 -0400][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1096 #0] Stream 0 connected. >> cmd: [2021-04-16 00:22:02.595076 -0400][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2021-04-16 00:22:02.595229 -0400][Debug ][Poller ] <[::ffff:10.32.79.65]:60278><--><[::ffff:149.165.243.5]:1096> Removing socket from the poller >> cmd: [2021-04-16 00:22:02.595414 -0400][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1096 #0] Recovering error for stream #0: [ERROR] Socket error. >> cmd: [2021-04-16 00:22:02.595469 -0400][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1096 #0] Reporting disconnection to queued message handlers. >> cmd: [2021-04-16 00:22:02.595519 -0400][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1096] Handling error while processing kXR_open (file: /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_166186.root?oss.asize=239173725, mode: 0644, flags: kXR_new kXR_open_updt kXR_async kXR_retstat ): [ERROR] Socket error. >> cmd: [2021-04-16 00:22:02.596608 -0400][Debug ][XRootD ] [deepthought.crc.nd.edu:1094] Handling error while processing kXR_open (file: /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_166186.root?oss.asize=239173725&tried=primeradiant01.crc.nd.edu, mode: 0644, flags: kXR_new kXR_open_updt kXR_async kXR_retstat ): [ERROR] Error response: No such file or directory. >> cmd: [2021-04-16 00:22:02.596731 -0400][Debug ][File ] [0x1b3cbc0@root://deepthought.crc.nd.edu:1094//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_166186.root?oss.asize=239173725] Open has returned with status [ERROR] Server responded with an error: [3011] Unable to create new file; file already exists. >> cmd: [2021-04-16 00:22:02.596791 -0400][Debug ][File ] [0x1b3cbc0@root://deepthought.crc.nd.edu:1094//store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_166186.root?oss.asize=239173725] Error while opening at deepthought.crc.nd.edu:1094: [ERROR] Server responded with an error: [3011] Unable to create new file; file already exists. >> cmd: [2021-04-16 00:22:02.596995 -0400][Debug ][File ] [0x1b3b700@file://localhost/disk/vc3-root/tmp/worker-230547-29293/t.166186/MAOD-00000.root] Sending a close command for handle 0x18 to localhost >> cmd: [2021-04-16 00:22:02.597196 -0400][Debug ][File ] [0x1b3b700@file://localhost/disk/vc3-root/tmp/worker-230547-29293/t.166186/MAOD-00000.root] Close returned from localhost with: [SUCCESS] >> cmd: [0B/0B][100%][==================================================][0B/s] >> cmd: Run: [ERROR] Server responded with an error: [3011] Unable to create new file; file already exists. >> cmd: >> cmd: [2021-04-16 00:22:02.643425 -0400][Debug ][JobMgr ] Stopping the job manager... >> cmd: [2021-04-16 00:22:02.644210 -0400][Debug ][JobMgr ] Job manager stopped >> cmd: [2021-04-16 00:22:02.644265 -0400][Debug ][TaskMgr ] Stopping the task manager... >> cmd: [2021-04-16 00:22:02.644462 -0400][Debug ][TaskMgr ] Task manager stopped >> cmd: [2021-04-16 00:22:02.644512 -0400][Debug ][Poller ] Stopping the poller... >> cmd: [2021-04-16 00:22:02.644827 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" >> cmd: [2021-04-16 00:22:02.644880 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2021-04-16 00:22:02.644928 -0400][Debug ][Poller ] <[::ffff:10.32.79.65]:47770><--><[::ffff:129.74.85.55]:1094> Removing socket from the poller >> cmd: [2021-04-16 00:22:02.645020 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Destroying stream >> cmd: [2021-04-16 00:22:02.645067 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2021-04-16 00:22:02.645116 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant01.crc.nd.edu:1096" >> cmd: [2021-04-16 00:22:02.645160 -0400][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2021-04-16 00:22:02.645204 -0400][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2021-04-16 00:22:02.645246 -0400][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Closing the socket >>> Checking output file transfer. @ Fri Apr 16 00:22:02 2021 >>> executing '/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/bin/gfal-copy -f file:///disk/vc3-root/tmp/worker-230547-29293/t.166186/MAOD-00000.root gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_166186.root' @ Fri Apr 16 00:22:02 2021 >> using /disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N/tmpDbRd9o to store command output >> cmd: Copying 239173725 bytes file:///disk/vc3-root/tmp/worker-230547-29293/t.166186/MAOD-00000.root => gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_166186.root >>> Checking output file transfer. @ Fri Apr 16 00:23:10 2021 >>> executing 'timeout 300 hdfs dfs -fs hdfs://eddie.crc.nd.edu:19000 -stat "Size: %b" /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_166186.root' @ Fri Apr 16 00:23:10 2021 >> using /disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N/tmpulYy43 to store command output >> cmd: timeout: failed to run command 'hdfs': No such file or directory >>>>> checking output for MAOD-00000.root failed: timeout: failed to run command 'hdfs': No such file or directory @ Fri Apr 16 00:23:10 2021 >>> executing 'timeout 300 xrdfs deepthought.crc.nd.edu stat /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_166186.root' @ Fri Apr 16 00:23:10 2021 >> using /disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N/tmptUbIM1 to store command output >> cmd: [ERROR] Server responded with an error: [3011] No servers are available to read the file. >> cmd: >>>>> checking output for MAOD-00000.root failed: [ERROR] Server responded with an error: [3011] No servers are available to read the file. @ Fri Apr 16 00:24:11 2021 >>> executing '/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/bin/gfal-stat gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_166186.root' @ Fri Apr 16 00:24:11 2021 >> using /disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N/tmpbnfaZo to store command output >> cmd: File: 'gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_166186.root' >> cmd: Size: 239173725 regular file >> cmd: Access: (0664/-rw-rw-r--) Uid: 230547 Gid: 0 >> cmd: Access: 1969-12-31 19:00:00.000000 >> cmd: Modify: 2021-04-16 00:23:08.000000 >> cmd: Change: 1969-12-31 19:00:00.000000 >>> File transfer successful! @ Fri Apr 16 00:31:07 2021 >>> Checking output files... @ Fri Apr 16 00:31:07 2021 >>> Checking MAOD-00000.root => mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_166186.root @ Fri Apr 16 00:31:07 2021 >>> executing 'timeout 300 hdfs dfs -fs hdfs://eddie.crc.nd.edu:19000 -stat "Size: %b" /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_166186.root' @ Fri Apr 16 00:31:07 2021 >> using /disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N/tmpNhpNPD to store command output >> cmd: timeout: failed to run command 'hdfs': No such file or directory >>>>> checking output for MAOD-00000.root failed: timeout: failed to run command 'hdfs': No such file or directory @ Fri Apr 16 00:31:08 2021 >>> executing 'timeout 300 xrdfs deepthought.crc.nd.edu stat /store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_166186.root' @ Fri Apr 16 00:31:08 2021 >> using /disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N/tmp0QCDmb to store command output >> cmd: [ERROR] Server responded with an error: [3011] No servers are available to read the file. >> cmd: >>>>> checking output for MAOD-00000.root failed: [ERROR] Server responded with an error: [3011] No servers are available to read the file. @ Fri Apr 16 00:32:08 2021 >>> executing '/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/bin/gfal-stat gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_166186.root' @ Fri Apr 16 00:32:08 2021 >> using /disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N/tmpG1jjeh to store command output >> cmd: gfal-stat error: 13 (Permission denied) - globus_ftp_client: the server responded with an error 530 530-Login incorrect. : /builddir/build/BUILD/gridftp-hdfs-1.1.1/src/gridftp_hdfs.c:hdfs_start:756: 530-Server is cancelling transfer due to over-load limit (host=deepthought.crc.nd.edu, user=kmohrman, path=(null)) 530 End. >>>>> checking output for MAOD-00000.root failed: gfal-stat error: 13 (Permission denied) - globus_ftp_client: the server responded with an error 530 530-Login incorrect. : /builddir/build/BUILD/gridftp-hdfs-1.1.1/src/gridftp_hdfs.c:hdfs_start:756: 530-Server is cancelling transfer due to over-load limit (host=deepthought.crc.nd.edu, user=kmohrman, path=(null)) 530 End. @ Fri Apr 16 00:32:09 2021 >>> executing '/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/bin/gfal-stat gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_166186.root' @ Fri Apr 16 00:32:09 2021 >> using /disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N/tmpR4jndC to store command output >> cmd: gfal-stat error: 13 (Permission denied) - globus_ftp_client: the server responded with an error 530 530-Login incorrect. : /builddir/build/BUILD/gridftp-hdfs-1.1.1/src/gridftp_hdfs.c:hdfs_start:756: 530-Server is cancelling transfer due to over-load limit (host=deepthought.crc.nd.edu, user=kmohrman, path=(null)) 530 End. >>>>> checking output for MAOD-00000.root failed: gfal-stat error: 13 (Permission denied) - globus_ftp_client: the server responded with an error 530 530-Login incorrect. : /builddir/build/BUILD/gridftp-hdfs-1.1.1/src/gridftp_hdfs.c:hdfs_start:756: 530-Server is cancelling transfer due to over-load limit (host=deepthought.crc.nd.edu, user=kmohrman, path=(null)) 530 End. @ Fri Apr 16 00:32:10 2021 >>> executing 'stat /hadoop/store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_166186.root' @ Fri Apr 16 00:32:10 2021 >> using /disk/vc3-root/tmp/worker-230547-29293/t.166186/cctools-temp-t.166186.YOAD9N/tmpb9fqwY to store command output >> cmd: stat: cannot stat '/hadoop/store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_166186.root': No such file or directory >>>>> checking output for MAOD-00000.root failed: stat: cannot stat '/hadoop/store/user/kmohrman/FullProduction/FullR2/UL17/Round1/Batch1/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_166186.root': No such file or directory @ Fri Apr 16 00:32:10 2021 >> trace: Traceback (most recent call last): >> trace: File "task.py", line 268, in wrapper >> trace: result = fct(data, *args, **kwargs) >> trace: File "task.py", line 404, in check_outputs >> trace: raise IOError("could not verify output file '{}'".format(remote)) >> trace: IOError: could not verify output file 'mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_166186.root' >>>>> call to 'check_outputs' failed, exiting with exit code 211 @ Fri Apr 16 00:32:10 2021 Fri Apr 16 00:18:25 2021: ApMon[ERROR]: Error resolving cms-jobmon.cern.ch: [Errno -2] Name or service not known Fri Apr 16 00:18:25 2021: ApMon[WARNING]: Not sending parameters since no destination is defined. Fri Apr 16 00:18:25 2021: ApMon[DEBUG]: Closing UDP socket on ApMon object destroy. Fri Apr 16 00:20:35 2021: ApMon[ERROR]: Error resolving cms-jobmon.cern.ch: [Errno -2] Name or service not known Fri Apr 16 00:20:35 2021: ApMon[WARNING]: Not sending parameters since no destination is defined. Fri Apr 16 00:20:35 2021: ApMon[DEBUG]: Closing UDP socket on ApMon object destroy. >> Execution time 845 >> Exiting with code 211 >> Reporting ExeExitCode 0 >> Reporting StageOutSE T3_US_NotreDame >> Reporting StageOutExitCode 211 Fri Apr 16 00:32:10 2021: ApMon[ERROR]: Error resolving cms-jobmon.cern.ch: [Errno -2] Name or service not known Fri Apr 16 00:32:11 2021: ApMon[WARNING]: Not sending parameters since no destination is defined. Fri Apr 16 00:32:11 2021: ApMon[DEBUG]: Closing UDP socket on ApMon object destroy. ==== working directory after execution @ Fri Apr 16 00:32:11 EDT 2021 ==== == dir: total 234540 == dir: -rwxr-xr-x 2 kmohrman campus 1598 Apr 16 00:17 0015_3455_report.json == dir: -rwxr-xr-x 2 kmohrman campus 1599 Apr 16 00:17 0015_4423_report.json == dir: -rwxr-xr-x 2 kmohrman campus 1602 Apr 16 00:17 0015_7630_report.json == dir: -rwxr-xr-x 2 kmohrman campus 1602 Apr 16 00:17 0015_8749_report.json == dir: drwxr-xr-x 18 kmohrman campus 215 Apr 16 00:18 CMSSW_10_6_20 == dir: -rw-r--r-- 1 kmohrman campus 239173725 Apr 16 00:20 MAOD-00000.root == dir: drwxr-xr-x 2 kmohrman campus 71 Apr 16 00:17 bin == dir: -rwxr-xr-x 5 kmohrman campus 627272 Apr 15 15:44 cctools-monitor == dir: -rw-r--r-- 1 kmohrman campus 0 Apr 16 00:17 cctools-monitor.summary == dir: drwxrwxrwx 2 kmohrman campus 6 Apr 16 00:32 cctools-temp-t.166186.YOAD9N == dir: -rw------- 1 kmohrman campus 51 Apr 16 00:18 cvmfschecksum.cms-ib.cern.ch == dir: -rw------- 1 kmohrman campus 51 Apr 16 00:18 cvmfschecksum.cms.cern.ch == dir: -rw------- 1 kmohrman campus 51 Apr 16 00:18 cvmfschecksum.oasis.opensciencegrid.org == dir: drwxr-xr-x 2 kmohrman campus 33 Apr 16 00:17 lib == dir: -rwxrwxrwx 1 kmohrman campus 146912 Apr 16 00:17 librmonitor_helper.so.pJWPIv == dir: -rw-r--r-- 2 kmohrman campus 868 Apr 16 00:17 merge_cfg.py == dir: -rw-r--r-- 1 kmohrman campus 2590 Apr 16 00:18 merge_cfg_mod.py == dir: -rw-r--r-- 2 kmohrman campus 808 Apr 16 00:17 merge_reports.py == dir: -rw-r--r-- 2 kmohrman campus 2528 Apr 16 00:17 parameters.json == dir: -rw------- 2 kmohrman campus 9917 Apr 16 00:17 proxy == dir: drwxr-xr-x 3 kmohrman campus 20 Apr 16 00:17 python == dir: -rw-r--r-- 1 kmohrman campus 3240 Apr 16 00:32 report.json == dir: -rw-r--r-- 5 kmohrman campus 570 Apr 15 15:44 report.json.in == dir: -rw-r--r-- 1 kmohrman campus 68355 Apr 16 00:20 report.xml == dir: -rw-r--r-- 1 kmohrman campus 5045 Apr 16 00:32 report.xml.gz == dir: -rw-r--r-- 2 kmohrman campus 549 Apr 16 00:17 sandbox-CMSSW_10_6_20-slc7_amd64_gcc700.tar.bz2 == dir: drwxr-xr-x 4 kmohrman campus 37 Apr 16 00:17 siteconf == dir: -rw-r--r-- 1 kmohrman campus 11 Apr 16 00:18 t_wrapper_ready == dir: -rw-r--r-- 1 kmohrman campus 11 Apr 16 00:18 t_wrapper_start == dir: -rwxr-xr-x 5 kmohrman campus 47217 Apr 15 15:44 task.py == dir: -rwxr-xr-x 5 kmohrman campus 5986 Apr 15 15:44 wrapper.sh === wrapper done @ Fri Apr 16 00:32:11 EDT 2021 === final return status = 211 @ Fri Apr 16 00:32:11 EDT 2021