==== wrapper started @ Tue May 18 06:31:01 EDT 2021 ==== == startup: hostname: d8civy106.crc.nd.edu == startup: kernel: Linux d8civy106.crc.nd.edu 3.10.0-1160.el7.x86_64 #1 SMP Tue Aug 18 14:50:17 EDT 2020 x86_64 x86_64 x86_64 GNU/Linux ==== tracing google @ Tue May 18 06:31:06 EDT 2021 ==== wrapper.sh: line 20: traceroute: command not found ==== environment at startup @ Tue May 18 06:31:11 EDT 2021 ==== == env: VC3_VERSION_SED=4.2 == env: CCTOOLS_RESOURCE_PROCESS_START=1621333871802037 == env: VC3_INSIDE_WRAPPER_SCRIPT=yes == env: VC3_ROOT_GZIP=/usr/ == env: SHELL=/bin/sh == env: TERM=xterm-256color == env: MAKEFLAGS=-j4 == env: TMPDIR=/disk/vc3-root/tmp/worker-230547-23860/t.527002/cctools-temp-t.527002.lV2uGj == env: VC3_ROOT_WHICH=/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0 == env: LOBSTER_CVMFS_PROXY=http://eddie.crc.nd.edu:3128 == env: LD_PRELOAD=/disk/vc3-root/tmp/worker-230547-23860/t.527002/librmonitor_helper.so.ukIS6w == env: LOBSTER_FRONTIER_PROXY=http://eddie.crc.nd.edu:3128 == env: LOBSTER_CMSSW_VERSION=CMSSW_10_6_20 == env: VC3_VERSION_FULL_TAR=1.26.0 == env: VC3_MACHINE_OS=Linux/redhat7.4.1708 == env: VC3_VERSION_FULL_CCTOOLS_WQ_WORKER=999.999.999 == env: USER=kmohrman == env: TEMP=/disk/vc3-root/tmp/worker-230547-23860/t.527002/cctools-temp-t.527002.lV2uGj == env: VC3_BUILDER_PATH=/disk/vc3-builder == env: LD_LIBRARY_PATH=/.singularity.d/libs == env: VC3_VERSION_VC3_BUILDER_HANDS=1.0 == env: VC3_ROOT_MAKE=/usr/ == env: VC3_DISTFILES=/disk/vc3-distfiles == env: WORKER_TMPDIR=/disk/vc3-root/tmp/worker-230547-23860/cache/tmp == env: VC3_VERSION_TAR=1.26 == env: ENV=/disk/vc3-root/vc3-home/.vc3_sh-P7YvX0 == env: VC3_VERSION_CCTOOLS_WQ_WORKER=999.999 == env: VC3_VERSION_FULL_PKG_CONFIG=0.27.1 == env: VC3_VERSION_GZIP=1.5 == env: VC3_ENV_TAG_GLOBAL=23607 == env: PATH=/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0/bin:/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin == env: VC3_ROOT=/disk/vc3-root == env: VC3_SH_PROFILE_ENV=/disk/vc3-root/vc3-home/.vc3_sh-P7YvX0 == env: VC3_ROOT_PKG_CONFIG=/usr/ == env: PWD=/disk/vc3-root/tmp/worker-230547-23860/t.527002 == env: LOBSTER_OSG_VERSION=3.4 == env: VC3_VERSION_FULL_SED=4.2.2 == env: VC3_VERSION_WHICH=2.20 == env: VC3_MACHINE_GLIBC_VERSION=2.17 == env: MODULEPATH=/afs/crc.nd.edu/x86_64_linux/Modules/modules/development_tools_and_libraries:/afs/crc.nd.edu/x86_64_linux/Modules/modules/general_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/restricted_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/system_modules:/afs/crc.nd.edu/x86_64_linux/Modules/modules/deprecated_software:/afs/crc.nd.edu/x86_64_linux/Modules/current/modulefiles == env: WORK_QUEUE_SANDBOX=/disk/vc3-root/tmp/worker-230547-23860 == env: VC3_ROOT_CCTOOLS_WQ_WORKER=/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999 == env: VC3_ROOT_SED=/usr/ == env: VC3_MACHINE_TARGET=x86_64/redhat7.4.1708 == env: CXX=g++ == env: SHLVL=3 == env: HOME=/disk/vc3-root/vc3-home == env: VC3_VERSION_MAKE=3.82 == env: VC3_VERSION_FULL_GZIP=1.5.0 == env: DISK=350 == env: MEMORY=1050 == env: VC3_VERSION_FULL_WHICH=2.20.0 == env: TMP=/disk/vc3-root/tmp/worker-230547-23860/t.527002/cctools-temp-t.527002.lV2uGj == env: VC3_INSTALL_USER_HOME=/disk/vc3-root/vc3-home == env: VC3_VERSION_FULL_VC3_BUILDER_HANDS=1.0.0 == env: VC3_VERSION_PKG_CONFIG=0.27 == env: VC3_ROOT_TAR=/usr/ == env: VC3_SH_PROFILE_WRAPPER=/disk/vc3-root/vc3-home/.vc3_sh-P7YvX0.wrapper == env: CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 == env: CC=gcc == env: CORES=1 == env: VC3_VERSION_FULL_MAKE=3.82.0 == env: CCTOOLS_RESOURCE_MONITOR_INFO=1027 == env: VC3_ROOT_VC3_BUILDER_HANDS=/disk/vc3-root/x86_64/redhat7.4.1708/vc3-builder-hands/v1.0.0 == env: VC3_MACHINE_ARCH=x86_64 == env: VC3_COMMAND_LINE=/disk/vc3-builder --var CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 --require cctools-wq-worker -- work_queue_worker -M lobster_kmohrman_EFT_CRC_postLHE.* -t 300 -C catalog.cse.nd.edu,backup-catalog.cse.nd.edu -d all -o worker.log --cores=12 --memory=16000 --disk=18419 == env: _=/usr/bin/env ==== cpu info @ Tue May 18 06:31:16 EDT 2021 ==== == cpu: processor : 0 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3241.271 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 0 == cpu: cpu cores : 8 == cpu: apicid : 0 == cpu: initial apicid : 0 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.70 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 1 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3257.617 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 1 == cpu: cpu cores : 8 == cpu: apicid : 2 == cpu: initial apicid : 2 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.70 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 2 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3191.760 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 2 == cpu: cpu cores : 8 == cpu: apicid : 4 == cpu: initial apicid : 4 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.70 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 3 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3011.804 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 3 == cpu: cpu cores : 8 == cpu: apicid : 6 == cpu: initial apicid : 6 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.70 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 4 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3047.985 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 4 == cpu: cpu cores : 8 == cpu: apicid : 8 == cpu: initial apicid : 8 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.70 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 5 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3102.734 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 5 == cpu: cpu cores : 8 == cpu: apicid : 10 == cpu: initial apicid : 10 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.70 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 6 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3048.620 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 6 == cpu: cpu cores : 8 == cpu: apicid : 12 == cpu: initial apicid : 12 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.70 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 7 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3137.170 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 7 == cpu: cpu cores : 8 == cpu: apicid : 14 == cpu: initial apicid : 14 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.70 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 8 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3194.775 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 0 == cpu: cpu cores : 8 == cpu: apicid : 32 == cpu: initial apicid : 32 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.77 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 9 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3197.155 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 1 == cpu: cpu cores : 8 == cpu: apicid : 34 == cpu: initial apicid : 34 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.77 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 10 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3000.219 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 2 == cpu: cpu cores : 8 == cpu: apicid : 36 == cpu: initial apicid : 36 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.77 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 11 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3100.671 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 3 == cpu: cpu cores : 8 == cpu: apicid : 38 == cpu: initial apicid : 38 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.77 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 12 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3083.374 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 4 == cpu: cpu cores : 8 == cpu: apicid : 40 == cpu: initial apicid : 40 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.77 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 13 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3199.694 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 5 == cpu: cpu cores : 8 == cpu: apicid : 42 == cpu: initial apicid : 42 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.77 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 14 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3099.719 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 6 == cpu: cpu cores : 8 == cpu: apicid : 44 == cpu: initial apicid : 44 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.77 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 15 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3160.498 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 7 == cpu: cpu cores : 8 == cpu: apicid : 46 == cpu: initial apicid : 46 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.77 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: === using CVMFS proxy: http://eddie.crc.nd.edu:3128 @ Tue May 18 06:31:41 EDT 2021 === using Frontier proxy: http://eddie.crc.nd.edu:3128 @ Tue May 18 06:31:41 EDT 2021 === parrot helper: /disk/vc3-root/tmp/worker-230547-23860/t.527002/lib/libparrot_helper.so @ Tue May 18 06:31:47 EDT 2021 ==== content of /disk/vc3-root/tmp/worker-230547-23860/cache/tmp @ Tue May 18 06:31:47 EDT 2021 ==== == cache: total 28 == cache: drwx------ 262 kmohrman campus 8192 May 18 06:31 ex_parrot_kmohrman == cache: -rw-r--r-- 1 kmohrman campus 10 May 17 21:42 hot_cache == cache: drwxr-x--- 260 kmohrman campus 8192 May 17 21:34 cvmfs === OSG certificate location: /cvmfs/oasis.opensciencegrid.org/mis/certificates @ Tue May 18 06:31:52 EDT 2021 === testing parrot usage @ Tue May 18 06:31:52 EDT 2021 === parrot OK @ Tue May 18 06:31:57 EDT 2021 === starting parrot to access CMSSW... @ Tue May 18 06:31:57 EDT 2021 ==== wrapper started @ Tue May 18 06:31:57 EDT 2021 ==== == startup: hostname: d8civy106.crc.nd.edu == startup: kernel: Linux d8civy106.crc.nd.edu 3.10.0-1160.el7.x86_64 #1 SMP Tue Aug 18 14:50:17 EDT 2020 x86_64 x86_64 x86_64 GNU/Linux ==== tracing google @ Tue May 18 06:31:57 EDT 2021 ==== wrapper.sh: line 20: traceroute: command not found ==== environment at startup @ Tue May 18 06:31:57 EDT 2021 ==== == env: VC3_VERSION_SED=4.2 == env: CCTOOLS_RESOURCE_PROCESS_START=1621333917233861 == env: VC3_INSIDE_WRAPPER_SCRIPT=yes == env: PARROT_HTTP_PROXY=http://eddie.crc.nd.edu:3128 == env: TERM=xterm-256color == env: SHELL=/bin/sh == env: VC3_ROOT_GZIP=/usr/ == env: LOBSTER_GFAL_COPY= == env: MAKEFLAGS=-j4 == env: PARROT_HELPER=/disk/vc3-root/tmp/worker-230547-23860/t.527002/lib/libparrot_helper.so == env: TMPDIR=/disk/vc3-root/tmp/worker-230547-23860/t.527002/cctools-temp-t.527002.lV2uGj == env: VC3_ROOT_WHICH=/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0 == env: LD_PRELOAD=/disk/vc3-root/tmp/worker-230547-23860/t.527002/librmonitor_helper.so.ukIS6w == env: LOBSTER_CVMFS_PROXY=http://eddie.crc.nd.edu:3128 == env: PARROT_PATH=./bin == env: LOBSTER_FRONTIER_PROXY=http://eddie.crc.nd.edu:3128 == env: LOBSTER_CMSSW_VERSION=CMSSW_10_6_20 == env: VC3_MACHINE_OS=Linux/redhat7.4.1708 == env: VC3_VERSION_FULL_TAR=1.26.0 == env: USER=kmohrman == env: VC3_VERSION_FULL_CCTOOLS_WQ_WORKER=999.999.999 == env: LD_LIBRARY_PATH=/.singularity.d/libs:lib == env: VC3_BUILDER_PATH=/disk/vc3-builder == env: TEMP=/disk/vc3-root/tmp/worker-230547-23860/t.527002/cctools-temp-t.527002.lV2uGj == env: VC3_VERSION_VC3_BUILDER_HANDS=1.0 == env: VC3_DISTFILES=/disk/vc3-distfiles == env: VC3_ROOT_MAKE=/usr/ == env: ENV=/disk/vc3-root/vc3-home/.vc3_sh-P7YvX0 == env: VC3_VERSION_TAR=1.26 == env: WORKER_TMPDIR=/disk/vc3-root/tmp/worker-230547-23860/cache/tmp == env: FRONTIER_PROXY=http://eddie.crc.nd.edu:3128 == env: PARROT_ALLOW_SWITCHING_CVMFS_REPOSITORIES=TRUE == env: VC3_VERSION_CCTOOLS_WQ_WORKER=999.999 == env: VC3_VERSION_FULL_PKG_CONFIG=0.27.1 == env: PARROT_CVMFS_REPO= == env: VC3_VERSION_GZIP=1.5 == env: PATH=/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0/bin:/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin == env: VC3_ENV_TAG_GLOBAL=23607 == env: VC3_ROOT=/disk/vc3-root == env: VC3_SH_PROFILE_ENV=/disk/vc3-root/vc3-home/.vc3_sh-P7YvX0 == env: PWD=/disk/vc3-root/tmp/worker-230547-23860/t.527002 == env: VC3_ROOT_PKG_CONFIG=/usr/ == env: PARROT_ENABLED=TRUE == env: VC3_VERSION_FULL_SED=4.2.2 == env: LOBSTER_OSG_VERSION=3.4 == env: VC3_VERSION_WHICH=2.20 == env: MODULEPATH=/afs/crc.nd.edu/x86_64_linux/Modules/modules/development_tools_and_libraries:/afs/crc.nd.edu/x86_64_linux/Modules/modules/general_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/restricted_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/system_modules:/afs/crc.nd.edu/x86_64_linux/Modules/modules/deprecated_software:/afs/crc.nd.edu/x86_64_linux/Modules/current/modulefiles == env: VC3_MACHINE_GLIBC_VERSION=2.17 == env: LOBSTER_LCG_CP= == env: WORK_QUEUE_SANDBOX=/disk/vc3-root/tmp/worker-230547-23860 == env: VC3_ROOT_CCTOOLS_WQ_WORKER=/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999 == env: VC3_MACHINE_TARGET=x86_64/redhat7.4.1708 == env: VC3_ROOT_SED=/usr/ == env: OASIS_CERTIFICATES=/cvmfs/oasis.opensciencegrid.org/mis/certificates == env: CXX=g++ == env: HOME=/disk/vc3-root/vc3-home == env: SHLVL=3 == env: VC3_VERSION_FULL_GZIP=1.5.0 == env: VC3_VERSION_MAKE=3.82 == env: DISK=350 == env: PARROT_CACHE=/disk/vc3-root/tmp/worker-230547-23860/cache/tmp == env: PARROT_CVMFS_ALIEN_CACHE=/disk/vc3-root/tmp/worker-230547-23860/cache/tmp/cvmfs == env: VC3_VERSION_FULL_WHICH=2.20.0 == env: MEMORY=1050 == env: HTTP_PROXY=http://eddie.crc.nd.edu:3128 == env: TMP=/disk/vc3-root/tmp/worker-230547-23860/t.527002/cctools-temp-t.527002.lV2uGj == env: PYTHONPATH=python: == env: VC3_INSTALL_USER_HOME=/disk/vc3-root/vc3-home == env: VC3_VERSION_FULL_VC3_BUILDER_HANDS=1.0.0 == env: VC3_ROOT_TAR=/usr/ == env: VC3_VERSION_PKG_CONFIG=0.27 == env: CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 == env: VC3_SH_PROFILE_WRAPPER=/disk/vc3-root/vc3-home/.vc3_sh-P7YvX0.wrapper == env: CC=gcc == env: CCTOOLS_RESOURCE_MONITOR_INFO=1027 == env: VC3_VERSION_FULL_MAKE=3.82.0 == env: CORES=1 == env: PARROT_DEBUG_FLAGS= == env: VC3_MACHINE_ARCH=x86_64 == env: VC3_ROOT_VC3_BUILDER_HANDS=/disk/vc3-root/x86_64/redhat7.4.1708/vc3-builder-hands/v1.0.0 == env: VC3_COMMAND_LINE=/disk/vc3-builder --var CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 --require cctools-wq-worker -- work_queue_worker -M lobster_kmohrman_EFT_CRC_postLHE.* -t 300 -C catalog.cse.nd.edu,backup-catalog.cse.nd.edu -d all -o worker.log --cores=12 --memory=16000 --disk=18419 == env: _=/usr/bin/env ==== cpu info @ Tue May 18 06:31:57 EDT 2021 ==== == cpu: processor : 0 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 0 == cpu: cpu cores : 8 == cpu: apicid : 0 == cpu: initial apicid : 0 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.70 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 1 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 1 == cpu: cpu cores : 8 == cpu: apicid : 2 == cpu: initial apicid : 2 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.70 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 2 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 2 == cpu: cpu cores : 8 == cpu: apicid : 4 == cpu: initial apicid : 4 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.70 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 3 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 3 == cpu: cpu cores : 8 == cpu: apicid : 6 == cpu: initial apicid : 6 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.70 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 4 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 4 == cpu: cpu cores : 8 == cpu: apicid : 8 == cpu: initial apicid : 8 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.70 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 5 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 5 == cpu: cpu cores : 8 == cpu: apicid : 10 == cpu: initial apicid : 10 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.70 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 6 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 6 == cpu: cpu cores : 8 == cpu: apicid : 12 == cpu: initial apicid : 12 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.70 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 7 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 0 == cpu: siblings : 8 == cpu: core id : 7 == cpu: cpu cores : 8 == cpu: apicid : 14 == cpu: initial apicid : 14 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5187.70 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 8 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 0 == cpu: cpu cores : 8 == cpu: apicid : 32 == cpu: initial apicid : 32 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.77 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 9 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 1 == cpu: cpu cores : 8 == cpu: apicid : 34 == cpu: initial apicid : 34 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.77 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 10 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 2 == cpu: cpu cores : 8 == cpu: apicid : 36 == cpu: initial apicid : 36 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.77 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 11 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 3 == cpu: cpu cores : 8 == cpu: apicid : 38 == cpu: initial apicid : 38 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.77 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 12 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 4 == cpu: cpu cores : 8 == cpu: apicid : 40 == cpu: initial apicid : 40 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.77 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 13 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 5 == cpu: cpu cores : 8 == cpu: apicid : 42 == cpu: initial apicid : 42 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.77 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 14 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 2999.902 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 6 == cpu: cpu cores : 8 == cpu: apicid : 44 == cpu: initial apicid : 44 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.77 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 15 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 62 == cpu: model name : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz == cpu: stepping : 4 == cpu: microcode : 0x42e == cpu: cpu MHz : 3000.061 == cpu: cache size : 20480 KB == cpu: physical id : 1 == cpu: siblings : 8 == cpu: core id : 7 == cpu: cpu cores : 8 == cpu: apicid : 46 == cpu: initial apicid : 46 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 13 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5192.77 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: === using parrot @ Tue May 18 06:31:58 EDT 2021 === sourcing CMS setup @ Tue May 18 06:31:58 EDT 2021 LibCvmfs version 2.4, revision 25 === sourcing OSG setup @ Tue May 18 06:31:59 EDT 2021 ==== environment after sourcing startup scripts @ Tue May 18 06:31:59 EDT 2021 ==== == env: VC3_VERSION_SED=4.2 == env: MANPATH=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/share/man:/cvmfs/cms.cern.ch/share/man: == env: CCTOOLS_RESOURCE_PROCESS_START=1621333919655853 == env: VC3_INSIDE_WRAPPER_SCRIPT=yes == env: GFAL_PLUGIN_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/gfal2-plugins/ == env: PARROT_HTTP_PROXY=http://eddie.crc.nd.edu:3128 == env: TERM=xterm-256color == env: SHELL=/bin/sh == env: VC3_ROOT_GZIP=/usr/ == env: LOBSTER_GFAL_COPY=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/bin/gfal-copy == env: VOMS_USERCONF=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/etc/vomses == env: MAKEFLAGS=-j4 == env: GLOBUS_LOCATION=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr == env: TMPDIR=/disk/vc3-root/tmp/worker-230547-23860/t.527002/cctools-temp-t.527002.lV2uGj == env: PERL5LIB=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/share/perl5/vendor_perl:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/share/perl5:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/perl5/vendor_perl:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/perl5 == env: VC3_ROOT_WHICH=/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0 == env: CVSROOT=:gserver:cmssw.cvs.cern.ch:/local/reps/CMSSW == env: LD_PRELOAD=/disk/vc3-root/tmp/worker-230547-23860/t.527002/librmonitor_helper.so.ukIS6w == env: LOBSTER_CVMFS_PROXY=http://eddie.crc.nd.edu:3128 == env: PARROT_PATH=./bin == env: LOBSTER_FRONTIER_PROXY=http://eddie.crc.nd.edu:3128 == env: X509_CERT_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/etc/grid-security/certificates == env: SCRAM_ARCH=slc7_amd64_gcc820 == env: LOBSTER_CMSSW_VERSION=CMSSW_10_6_20 == env: VC3_MACHINE_OS=Linux/redhat7.4.1708 == env: VC3_VERSION_FULL_TAR=1.26.0 == env: USER=kmohrman == env: VC3_VERSION_FULL_CCTOOLS_WQ_WORKER=999.999.999 == env: LD_LIBRARY_PATH=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/dcap:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/lcgdm:/.singularity.d/libs:lib:lib == env: VC3_BUILDER_PATH=/disk/vc3-builder == env: TEMP=/disk/vc3-root/tmp/worker-230547-23860/t.527002/cctools-temp-t.527002.lV2uGj == env: VC3_VERSION_VC3_BUILDER_HANDS=1.0 == env: VC3_DISTFILES=/disk/vc3-distfiles == env: VC3_ROOT_MAKE=/usr/ == env: OSG_LOCATION=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64 == env: ENV=/disk/vc3-root/vc3-home/.vc3_sh-P7YvX0 == env: VC3_VERSION_TAR=1.26 == env: WORKER_TMPDIR=/disk/vc3-root/tmp/worker-230547-23860/cache/tmp == env: FRONTIER_PROXY=http://eddie.crc.nd.edu:3128 == env: PARROT_ALLOW_SWITCHING_CVMFS_REPOSITORIES=TRUE == env: VC3_VERSION_CCTOOLS_WQ_WORKER=999.999 == env: VC3_VERSION_FULL_PKG_CONFIG=0.27.1 == env: SITECONFIG_PATH=/cvmfs/cms.cern.ch/SITECONF/local == env: PARROT_CVMFS_REPO= == env: VC3_VERSION_GZIP=1.5 == env: PATH=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/bin:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/sbin:/cvmfs/cms.cern.ch/common:/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0/bin:/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin == env: VC3_ENV_TAG_GLOBAL=23607 == env: CMS_PATH=/cvmfs/cms.cern.ch == env: VC3_ROOT=/disk/vc3-root == env: VC3_SH_PROFILE_ENV=/disk/vc3-root/vc3-home/.vc3_sh-P7YvX0 == env: PWD=/disk/vc3-root/tmp/worker-230547-23860/t.527002 == env: VC3_ROOT_PKG_CONFIG=/usr/ == env: PARROT_ENABLED=TRUE == env: VC3_VERSION_FULL_SED=4.2.2 == env: LOBSTER_OSG_VERSION=3.4 == env: VC3_VERSION_WHICH=2.20 == env: MODULEPATH=/afs/crc.nd.edu/x86_64_linux/Modules/modules/development_tools_and_libraries:/afs/crc.nd.edu/x86_64_linux/Modules/modules/general_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/restricted_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/system_modules:/afs/crc.nd.edu/x86_64_linux/Modules/modules/deprecated_software:/afs/crc.nd.edu/x86_64_linux/Modules/current/modulefiles == env: VC3_MACHINE_GLIBC_VERSION=2.17 == env: X509_VOMS_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/etc/grid-security/vomsdir == env: LOBSTER_LCG_CP= == env: WORK_QUEUE_SANDBOX=/disk/vc3-root/tmp/worker-230547-23860 == env: VC3_ROOT_CCTOOLS_WQ_WORKER=/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999 == env: VC3_MACHINE_TARGET=x86_64/redhat7.4.1708 == env: VC3_ROOT_SED=/usr/ == env: OASIS_CERTIFICATES=/cvmfs/oasis.opensciencegrid.org/mis/certificates == env: CXX=g++ == env: HOME=/disk/vc3-root/vc3-home == env: SHLVL=3 == env: VC3_VERSION_FULL_GZIP=1.5.0 == env: VC3_VERSION_MAKE=3.82 == env: DISK=350 == env: PARROT_CACHE=/disk/vc3-root/tmp/worker-230547-23860/cache/tmp == env: PARROT_CVMFS_ALIEN_CACHE=/disk/vc3-root/tmp/worker-230547-23860/cache/tmp/cvmfs == env: GFAL_CONFIG_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/etc/gfal2.d/ == env: VC3_VERSION_FULL_WHICH=2.20.0 == env: MEMORY=1050 == env: HTTP_PROXY=http://eddie.crc.nd.edu:3128 == env: TMP=/disk/vc3-root/tmp/worker-230547-23860/t.527002/cctools-temp-t.527002.lV2uGj == env: PYTHONPATH=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib/python2.7/site-packages:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/python2.7/site-packages:python:python: == env: VC3_INSTALL_USER_HOME=/disk/vc3-root/vc3-home == env: VC3_VERSION_FULL_VC3_BUILDER_HANDS=1.0.0 == env: VC3_ROOT_TAR=/usr/ == env: VC3_VERSION_PKG_CONFIG=0.27 == env: CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 == env: VC3_SH_PROFILE_WRAPPER=/disk/vc3-root/vc3-home/.vc3_sh-P7YvX0.wrapper == env: CC=gcc == env: CCTOOLS_RESOURCE_MONITOR_INFO=1027 == env: VC3_VERSION_FULL_MAKE=3.82.0 == env: CORES=1 == env: PARROT_DEBUG_FLAGS= == env: VC3_MACHINE_ARCH=x86_64 == env: VC3_ROOT_VC3_BUILDER_HANDS=/disk/vc3-root/x86_64/redhat7.4.1708/vc3-builder-hands/v1.0.0 == env: VC3_COMMAND_LINE=/disk/vc3-builder --var CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 --require cctools-wq-worker -- work_queue_worker -M lobster_kmohrman_EFT_CRC_postLHE.* -t 300 -C catalog.cse.nd.edu,backup-catalog.cse.nd.edu -d all -o worker.log --cores=12 --memory=16000 --disk=18419 == env: _=/usr/bin/env ==== proxy information @ Tue May 18 06:31:59 EDT 2021 ==== == proxy: subject : /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=kmohrman/CN=825714/CN=Kelci Ann Mohrman/CN=3724584890 == proxy: issuer : /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=kmohrman/CN=825714/CN=Kelci Ann Mohrman == proxy: identity : /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=kmohrman/CN=825714/CN=Kelci Ann Mohrman == proxy: type : RFC compliant proxy == proxy: strength : 1024 bits == proxy: path : proxy == proxy: timeleft : 73:52:25 ==== working directory at startup @ Tue May 18 06:32:00 EDT 2021 ==== == dir: total 876 == dir: -rwxr-xr-x 2 kmohrman campus 1600 May 18 06:31 0052_4392_report.json == dir: -rwxr-xr-x 2 kmohrman campus 1598 May 18 06:31 0052_4395_report.json == dir: -rwxr-xr-x 2 kmohrman campus 1602 May 18 06:31 0052_4397_report.json == dir: -rwxr-xr-x 2 kmohrman campus 1600 May 18 06:31 0052_4426_report.json == dir: drwxr-xr-x 2 kmohrman campus 71 May 18 06:31 bin == dir: -rwxr-xr-x 5 kmohrman campus 627272 May 17 21:33 cctools-monitor == dir: -rw-r--r-- 1 kmohrman campus 0 May 18 06:31 cctools-monitor.summary == dir: drwxrwxrwx 2 kmohrman campus 6 May 18 06:31 cctools-temp-t.527002.lV2uGj == dir: -rw------- 1 kmohrman campus 51 May 18 06:31 cvmfschecksum.cms.cern.ch == dir: -rw------- 1 kmohrman campus 51 May 18 06:31 cvmfschecksum.oasis.opensciencegrid.org == dir: drwxr-xr-x 2 kmohrman campus 33 May 18 06:31 lib == dir: -rwxrwxrwx 1 kmohrman campus 146912 May 18 06:31 librmonitor_helper.so.ukIS6w == dir: -rw-r--r-- 2 kmohrman campus 868 May 18 06:31 merge_cfg.py == dir: -rw-r--r-- 2 kmohrman campus 808 May 18 06:31 merge_reports.py == dir: -rw-r--r-- 2 kmohrman campus 2528 May 18 06:31 parameters.json == dir: -rw------- 2 kmohrman campus 9921 May 18 06:31 proxy == dir: drwxr-xr-x 3 kmohrman campus 20 May 18 06:31 python == dir: -rw-r--r-- 5 kmohrman campus 570 May 17 21:33 report.json.in == dir: -rw-r--r-- 3 kmohrman campus 549 May 17 23:48 sandbox-CMSSW_10_6_20-slc7_amd64_gcc700.tar.bz2 == dir: drwxr-xr-x 4 kmohrman campus 37 May 18 06:31 siteconf == dir: -rw-r--r-- 1 kmohrman campus 11 May 18 06:31 t_wrapper_start == dir: -rwxr-xr-x 5 kmohrman campus 47217 May 17 21:33 task.py == dir: -rwxr-xr-x 5 kmohrman campus 5986 May 17 21:33 wrapper.sh === creating new release CMSSW_10_6_20 for scram arch slc7_amd64_gcc700 @ Tue May 18 06:32:00 EDT 2021 WARNING: In non-interactive mode release checks e.g. deprecated releases, production architectures are disabled. === unpacking sandbox-CMSSW_10_6_20-slc7_amd64_gcc700.tar.bz2 @ Tue May 18 06:32:03 EDT 2021 ==== machine load @ Tue May 18 06:32:05 EDT 2021 ==== top: unknown option 'M' Usage: top -hv | -bcHiOSs -d secs -n max -u|U user -p pid(s) -o field -w [cols] ==== environment before execution @ Tue May 18 06:32:05 EDT 2021 ==== == env: G4LEVELGAMMADATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4PhotonEvaporation/5.2/data/PhotonEvaporation5.2 == env: UTM_XSD_DIR=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/utm/utm_0.7.1-pafccj == env: SRT_G4LEVELGAMMADATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4PhotonEvaporation/5.2/data/PhotonEvaporation5.2 == env: VC3_VERSION_SED=4.2 == env: CMSSW_RELEASE_BASE=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20 == env: SCRAMRT_SET=CMSSW:CMSSW_10_6_20:slc7_amd64_gcc700:V2_2_9_pre13:SRT_ == env: MANPATH=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/share/man:/cvmfs/cms.cern.ch/share/man: == env: CCTOOLS_RESOURCE_PROCESS_START=1621333925548519 == env: CMSSW_FWLITE_INCLUDE_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sherpa/2.2.10-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/hepmc/2.06.07-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/clhep/2.4.0.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/boost/1.67.0-pafccj/include == env: SRT_GCC_RUNTIME_ASAN_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib64/libasan.so == env: VC3_INSIDE_WRAPPER_SCRIPT=yes == env: ROOT_INCLUDE_PATH=/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20/src:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/src:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/coral/CORAL_2_3_21-pafccj7/include/LCG:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/mctester/1.25.0a-pafccj9/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/lcg/root/6.14.09-pafccj6/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/dd4hep/v01-10x-pafccj9/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/QtDesigner:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/herwigpp/7.2.0-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/tauolapp/1.1.5-pafccj5/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/charybdis/1.003-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sherpa/2.2.10-pafccj2/include/SHERPA-MC:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/QtOpenGL:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/QtGui:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/thepeg/2.2.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/pythia8/240-pafccj5/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/herwig/6.521-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/rivet/2.7.0-pafccj8/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/Qt3Support:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/lwtnn/2.4-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4/10.4.3-pafccj2/include/Geant4:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/classlib/3.1.3-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/lhapdf/6.2.1-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/cgal/4.2-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/tkonlinesw/4.2.0-1_gcc7-pafccj9/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/starlight/r193-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/Qt:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/QtCore:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/QtXml:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/mcdb/1.0.3-pafccj/interface:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libungif/4.1.4-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libtiff/4.0.3-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/highfive/2.0/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libpng/1.6.16-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/frontier_client/2.9.1/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/pcre/8.37-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/boost/1.67.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/xrootd/4.8.5-pafccj2/include/xrootd/private:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/vdt/0.4.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/valgrind/3.13.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/utm/utm_0.7.1-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/toprex/4.23-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/tbb/2019_U3-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/tauola/27.121.5-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sigcpp/2.6.2-pafccj/include/sigc++-2.0:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sqlite/3.22.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/protobuf/3.5.2-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/pacparser/1.3.5-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/oracle/12.1.0.2.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/py2-numpy/1.16.2-nmpfii2/c-api/core/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/meschach/1.2.pCMS1-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/lz4/1.9.2-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libuuid/2.22.2-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libunwind/1.2.1-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libhepml/0.2.1-pafccj/interface:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/ktjet/1.06-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/jimmy/4.2-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/jemalloc/5.1.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/hls/2019.08/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/madgraph5amcatnlo/2.6.0-pafccj12:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/heppdt/3.03.00-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/hector/1.3.4_patch1-pafccj9/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libjpeg-turbo/2.0.1-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/giflib/4.2.3-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gsl/2.2.1-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gdbm/1.10-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/freetype/2.5.3-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/fftw3/3.3.2-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/fftjet/1.5.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/fastjet/3.3.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/expat/2.1.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/dpm/1.8.0.1-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/hepmc/2.06.07-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/vecgeom/v00.05.00-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/xerces-c/3.1.3-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/xz/5.2.2-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/dcap/2.47.8-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libxml2/2.9.1-pafccj/include/libxml2:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/curl/7.59.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/cppunit/1.40.1-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/clhep/2.4.0.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/openssl/1.0.2d-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/pythia6/426-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/photos/215.5-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/zlib-x86_64/1.2.11-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/cascade/2.2.04-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/bz2lib/1.0.6-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/python/2.7.14-pafccj/include/python2.7:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/tinyxml2/6.2.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/ittnotify/16.06.18-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gosamcontrib/2.0-20150803-pafccj/include:/usr/local/include:/usr/include == env: INTEL_LICENSE_FILE=28518@lxlicen01.cern.ch,28518@lxlicen02.cern.ch,28518@lxlicen03.cern.ch == env: SRT_TEXMFHOME_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/rivet/2.7.0-pafccj8/share/Rivet/texmf == env: SRT_GEANT4_PARFULLCMS_CONFIG_DIR_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-parfullcms/2014.01.27-pafccj2/share/ParFullCMS == env: GFAL_PLUGIN_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/gfal2-plugins/ == env: PARROT_HTTP_PROXY=http://eddie.crc.nd.edu:3128 == env: TERM=xterm-256color == env: SHELL=/bin/sh == env: VC3_ROOT_GZIP=/usr/ == env: LOBSTER_GFAL_COPY=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/bin/gfal-copy == env: PDFPATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/rivet/2.7.0-pafccj8/share == env: SRT_TNS_ADMIN_SCRAMRTDEL=/etc == env: VOMS_USERCONF=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/etc/vomses == env: MAKEFLAGS=-j4 == env: ROOT_GCC_TOOLCHAIN=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj == env: GLOBUS_LOCATION=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr == env: TMPDIR=/disk/vc3-root/tmp/worker-230547-23860/t.527002/cctools-temp-t.527002.lV2uGj == env: G4FORCENUMBEROFTHREADS=max == env: OPENBLAS_NUM_THREADS=1 == env: SRT_PDFPATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/rivet/2.7.0-pafccj8/share == env: PERL5LIB=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/git/2.19.0-pafccj/share/perl5:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/share/perl5/vendor_perl:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/share/perl5:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/perl5/vendor_perl:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/perl5 == env: VC3_ROOT_WHICH=/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0 == env: CMSSW_DATA_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms == env: LHAPDF_DATA_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/lhapdf/6.2.1-pafccj3/share/LHAPDF == env: CVSROOT=:gserver:cmssw.cvs.cern.ch:/local/reps/CMSSW == env: LD_PRELOAD=/disk/vc3-root/tmp/worker-230547-23860/t.527002/librmonitor_helper.so.ukIS6w == env: LOBSTER_CVMFS_PROXY=http://eddie.crc.nd.edu:3128 == env: PARROT_PATH=./bin == env: CMSSW_GIT_HASH=CMSSW_10_6_20 == env: SRT_G4ABLADATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4ABLA/3.0/data/G4ABLA3.0 == env: QTDIR=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2 == env: SRT_SCONS_LIB_DIR_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/scons/3.0.1-pafccj/lib == env: OLDPWD=/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20 == env: LOBSTER_FRONTIER_PROXY=http://eddie.crc.nd.edu:3128 == env: G4NEUTRONXSDATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4NEUTRONXS/1.4/data/G4NEUTRONXS1.4 == env: QTINC=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include == env: X509_CERT_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/etc/grid-security/certificates == env: SCRAM_ARCH=slc7_amd64_gcc700 == env: GIT_SSL_CAINFO=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/git/2.19.0-pafccj/share/ssl/certs/ca-bundle.crt == env: SRT_CMS_OPENLOOPS_PREFIX_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/openloops/2.1.0-pafccj2 == env: SRT_GIT_EXEC_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/git/2.19.0-pafccj/libexec/git-core == env: SRT_THEPEGPATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/thepeg/2.2.0-pafccj2/share/ThePEG == env: LOBSTER_CMSSW_VERSION=CMSSW_10_6_20 == env: ROOT_TTREECACHE_SIZE=0 == env: EVTGENDATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/evtgen/1.6.0-pafccj5/share == env: SRT_INTEL_LICENSE_FILE_SCRAMRTDEL=28518@lxlicen01.cern.ch,28518@lxlicen02.cern.ch,28518@lxlicen03.cern.ch == env: SRT_ROOT_INCLUDE_PATH_SCRAMRTDEL=/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20/src:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/src:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/coral/CORAL_2_3_21-pafccj7/include/LCG:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/mctester/1.25.0a-pafccj9/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/lcg/root/6.14.09-pafccj6/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/dd4hep/v01-10x-pafccj9/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/QtDesigner:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/herwigpp/7.2.0-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/tauolapp/1.1.5-pafccj5/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/charybdis/1.003-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sherpa/2.2.10-pafccj2/include/SHERPA-MC:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/QtOpenGL:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/QtGui:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/thepeg/2.2.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/pythia8/240-pafccj5/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/herwig/6.521-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/rivet/2.7.0-pafccj8/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/Qt3Support:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/lwtnn/2.4-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4/10.4.3-pafccj2/include/Geant4:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/classlib/3.1.3-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/lhapdf/6.2.1-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/cgal/4.2-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/tkonlinesw/4.2.0-1_gcc7-pafccj9/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/starlight/r193-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/Qt:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/QtCore:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include/QtXml:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/mcdb/1.0.3-pafccj/interface:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libungif/4.1.4-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libtiff/4.0.3-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/highfive/2.0/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libpng/1.6.16-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/frontier_client/2.9.1/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/pcre/8.37-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/boost/1.67.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/xrootd/4.8.5-pafccj2/include/xrootd/private:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/vdt/0.4.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/valgrind/3.13.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/utm/utm_0.7.1-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/toprex/4.23-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/tbb/2019_U3-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/tauola/27.121.5-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sigcpp/2.6.2-pafccj/include/sigc++-2.0:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sqlite/3.22.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/protobuf/3.5.2-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/pacparser/1.3.5-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/oracle/12.1.0.2.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/py2-numpy/1.16.2-nmpfii2/c-api/core/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/meschach/1.2.pCMS1-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/lz4/1.9.2-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libuuid/2.22.2-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libunwind/1.2.1-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libhepml/0.2.1-pafccj/interface:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/ktjet/1.06-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/jimmy/4.2-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/jemalloc/5.1.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/hls/2019.08/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/madgraph5amcatnlo/2.6.0-pafccj12:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/heppdt/3.03.00-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/hector/1.3.4_patch1-pafccj9/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libjpeg-turbo/2.0.1-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/giflib/4.2.3-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gsl/2.2.1-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gdbm/1.10-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/freetype/2.5.3-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/fftw3/3.3.2-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/fftjet/1.5.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/fastjet/3.3.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/expat/2.1.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/dpm/1.8.0.1-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/hepmc/2.06.07-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/vecgeom/v00.05.00-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/xerces-c/3.1.3-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/xz/5.2.2-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/dcap/2.47.8-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/libxml2/2.9.1-pafccj/include/libxml2:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/curl/7.59.0-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/cppunit/1.40.1-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/clhep/2.4.0.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/openssl/1.0.2d-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/pythia6/426-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/photos/215.5-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/zlib-x86_64/1.2.11-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/cascade/2.2.04-pafccj3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/bz2lib/1.0.6-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/python/2.7.14-pafccj/include/python2.7:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/tinyxml2/6.2.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/ittnotify/16.06.18-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gosamcontrib/2.0-20150803-pafccj/include:/usr/local/include:/usr/include == env: VC3_MACHINE_OS=Linux/redhat7.4.1708 == env: VC3_VERSION_FULL_TAR=1.26.0 == env: G4LEDATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4EMLOW/7.3/data/G4EMLOW7.3 == env: SRT_VINCIADATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/vincia/2.2.04-pafccj5/share/Vincia/xmldoc == env: SRT_ROOT_TTREECACHE_PREFILL_SCRAMRTDEL=0 == env: OPAL_PREFIX=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/openmpi/2.1.5-pafccj == env: USER=kmohrman == env: VC3_VERSION_FULL_CCTOOLS_WQ_WORKER=999.999.999 == env: ROOT_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/madgraph5amcatnlo/2.6.0-pafccj12:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gosamcontrib/2.0-20150803-pafccj == env: SHERPA_INCLUDE_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sherpa/2.2.10-pafccj2/include/SHERPA-MC == env: SRT_LHAPDF_DATA_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/lhapdf/6.2.1-pafccj3/share/LHAPDF == env: SRT_PYTHON27PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/share/overrides/python:/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20/python:/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/python:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/coral/CORAL_2_3_21-pafccj7/slc7_amd64_gcc700/python:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/coral/CORAL_2_3_21-pafccj7/slc7_amd64_gcc700/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/xrootd/4.8.5-pafccj2/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/fastjet/3.3.0-pafccj/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw-tool-conf/45.0-pafccj26/lib/python2.7/site-packages == env: LD_LIBRARY_PATH=/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20/biglib/slc7_amd64_gcc700:/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20/external/slc7_amd64_gcc700/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/biglib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/external/slc7_amd64_gcc700/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/llvm/7.1.0-pafccj/lib64:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib64:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/cuda/10.1.105-pafccj2/drivers:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/dcap:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/lcgdm:/.singularity.d/libs:lib:lib == env: VC3_BUILDER_PATH=/disk/vc3-builder == env: TEMP=/disk/vc3-root/tmp/worker-230547-23860/t.527002/cctools-temp-t.527002.lV2uGj == env: SRT_CMSSW_RELEASE_BASE_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20 == env: SRT_RIVET_ANALYSIS_PATH_SCRAMRTDEL=/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/rivet/2.7.0-pafccj8/lib == env: VC3_VERSION_VC3_BUILDER_HANDS=1.0 == env: SRT_PERL5LIB_SCRAMRT=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/git/2.19.0-pafccj/share/perl5 == env: SRT_CMSSW_VERSION_SCRAMRTDEL=CMSSW_10_6_20 == env: VC3_DISTFILES=/disk/vc3-distfiles == env: VC3_ROOT_MAKE=/usr/ == env: GCC_RUNTIME_LSAN=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib64/libasan.so == env: SCRAM_CXX11_ABI=1 == env: VALGRIND_LIB=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/valgrind/3.13.0-pafccj/lib/valgrind == env: SRT_CMSSW_GIT_HASH_SCRAMRTDEL=CMSSW_10_6_20 == env: SRT_ROOTSYS_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/lcg/root/6.14.09-pafccj6/ == env: OSG_LOCATION=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64 == env: ENV=/disk/vc3-root/vc3-home/.vc3_sh-P7YvX0 == env: VC3_VERSION_TAR=1.26 == env: WORKER_TMPDIR=/disk/vc3-root/tmp/worker-230547-23860/cache/tmp == env: G4NEUTRONHPDATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4NDL/4.5/data/G4NDL4.5 == env: GEANT4_PARFULLCMS_CONFIG_DIR=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-parfullcms/2014.01.27-pafccj2/share/ParFullCMS == env: FRONTIER_PROXY=http://eddie.crc.nd.edu:3128 == env: PARROT_ALLOW_SWITCHING_CVMFS_REPOSITORIES=TRUE == env: CSC_TRACK_FINDER_DATA_DIR=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/CSCTrackFinderEmulation/1.2-pafccj/data/ == env: SRT_QTINC_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/include == env: VC3_VERSION_CCTOOLS_WQ_WORKER=999.999 == env: G4ENSDFSTATEDATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4ENSDFSTATE/2.2/data/G4ENSDFSTATE2.2 == env: G4RADIOACTIVEDATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4RadioactiveDecay/5.2/data/RadioactiveDecay5.2 == env: ROOFITSYS=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/lcg/root/6.14.09-pafccj6/ == env: SRT_ROOFITSYS_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/lcg/root/6.14.09-pafccj6/ == env: SRT_G4RADIOACTIVEDATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4RadioactiveDecay/5.2/data/RadioactiveDecay5.2 == env: SRT_G4ENSDFSTATEDATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4ENSDFSTATE/2.2/data/G4ENSDFSTATE2.2 == env: SRT_CLHEP_PARAM_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/clhep/2.4.0.0-pafccj2 == env: G4ABLADATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4ABLA/3.0/data/G4ABLA3.0 == env: VC3_VERSION_FULL_PKG_CONFIG=0.27.1 == env: SRT_SHERPA_INCLUDE_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sherpa/2.2.10-pafccj2/include/SHERPA-MC == env: SRT_G4FORCENUMBEROFTHREADS_SCRAMRTDEL=max == env: SRT_WORKER_DATA_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/blackhat/0.9.9-pafccj/share/blackhat/datafiles/ == env: SITECONFIG_PATH=/cvmfs/cms.cern.ch/SITECONF/local == env: PARROT_CVMFS_REPO= == env: VC3_VERSION_GZIP=1.5 == env: GCC_RUNTIME_TSAN=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib64/libtsan.so == env: TNS_ADMIN=/etc == env: SRT_HEPPDT_PARAM_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/heppdt/3.03.00-pafccj == env: PATH=/cvmfs/cms.cern.ch/share/overrides/bin:/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20/bin/slc7_amd64_gcc700:/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20/external/slc7_amd64_gcc700/bin:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/bin/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/external/slc7_amd64_gcc700/bin:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/llvm/7.1.0-pafccj/bin:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/bin:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/bin:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/sbin:/cvmfs/cms.cern.ch/common:/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0/bin:/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin == env: VC3_ENV_TAG_GLOBAL=23607 == env: THEPEGPATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/thepeg/2.2.0-pafccj2/share/ThePEG == env: SRT_CMSSW_SEARCH_PATH_SCRAMRTDEL=/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20/poison:/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20/src:/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20/external/slc7_amd64_gcc700/data:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/src:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/external/slc7_amd64_gcc700/data == env: SRT_GIT_SSL_CAINFO_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/git/2.19.0-pafccj/share/ssl/certs/ca-bundle.crt == env: SRT_GCC_RUNTIME_LSAN_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib64/libasan.so == env: CMS_PATH=/cvmfs/cms.cern.ch == env: VC3_ROOT=/disk/vc3-root == env: SRT_UTM_XSD_DIR_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/utm/utm_0.7.1-pafccj == env: SRT_PATH_SCRAMRT=/cvmfs/cms.cern.ch/share/overrides/bin:/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20/bin/slc7_amd64_gcc700:/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20/external/slc7_amd64_gcc700/bin:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/bin/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/external/slc7_amd64_gcc700/bin:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/llvm/7.1.0-pafccj/bin:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/bin == env: VC3_SH_PROFILE_ENV=/disk/vc3-root/vc3-home/.vc3_sh-P7YvX0 == env: RIVET_ANALYSIS_PATH=/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/rivet/2.7.0-pafccj8/lib == env: SRT_LOCALRT_SCRAMRTDEL=/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20 == env: SRT_CMSSW_BASE_SCRAMRTDEL=/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20 == env: PWD=/disk/vc3-root/tmp/worker-230547-23860/t.527002 == env: VC3_ROOT_PKG_CONFIG=/usr/ == env: PARROT_ENABLED=TRUE == env: VC3_VERSION_FULL_SED=4.2.2 == env: LOBSTER_OSG_VERSION=3.4 == env: PYTHON_VALGRIND_SUPP=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/python/2.7.14-pafccj/share/valgrind/valgrind-python.supp == env: SRT_OPAL_PREFIX_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/openmpi/2.1.5-pafccj == env: VC3_VERSION_WHICH=2.20 == env: GCC_RUNTIME_UBSAN=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib64/libubsan.so == env: LANG=C == env: SRT_QTDIR_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2 == env: MODULEPATH=/afs/crc.nd.edu/x86_64_linux/Modules/modules/development_tools_and_libraries:/afs/crc.nd.edu/x86_64_linux/Modules/modules/general_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/restricted_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/system_modules:/afs/crc.nd.edu/x86_64_linux/Modules/modules/deprecated_software:/afs/crc.nd.edu/x86_64_linux/Modules/current/modulefiles == env: VC3_MACHINE_GLIBC_VERSION=2.17 == env: SRT_G4NEUTRONHPDATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4NDL/4.5/data/G4NDL4.5 == env: X509_VOMS_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/etc/grid-security/vomsdir == env: LOBSTER_LCG_CP= == env: SRT_SHERPA_SHARE_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sherpa/2.2.10-pafccj2/share/SHERPA-MC == env: SRT_GCC_RUNTIME_TSAN_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib64/libtsan.so == env: WORK_QUEUE_SANDBOX=/disk/vc3-root/tmp/worker-230547-23860 == env: CASCADE_PDFPATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/cascade/2.2.04-pafccj3/share == env: SRT_ROOT_GCC_TOOLCHAIN_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj == env: SRT_CASCADE_PDFPATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/cascade/2.2.04-pafccj3/share == env: WORKER_DATA_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/blackhat/0.9.9-pafccj/share/blackhat/datafiles/ == env: GIT_TEMPLATE_DIR=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/git/2.19.0-pafccj/share/git-core/templates == env: G4SAIDXSDATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4SAIDDATA/1.1/data/G4SAIDDATA1.1 == env: MLAS_DYNAMIC_CPU_ARCH=0 == env: VC3_ROOT_CCTOOLS_WQ_WORKER=/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999 == env: PYTHON27PATH=/cvmfs/cms.cern.ch/share/overrides/python:/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20/python:/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/python:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/coral/CORAL_2_3_21-pafccj7/slc7_amd64_gcc700/python:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/coral/CORAL_2_3_21-pafccj7/slc7_amd64_gcc700/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/xrootd/4.8.5-pafccj2/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/fastjet/3.3.0-pafccj/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw-tool-conf/45.0-pafccj26/lib/python2.7/site-packages == env: SRT_CMSSW_FWLITE_INCLUDE_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sherpa/2.2.10-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/hepmc/2.06.07-pafccj/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/clhep/2.4.0.0-pafccj2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/boost/1.67.0-pafccj/include == env: VC3_MACHINE_TARGET=x86_64/redhat7.4.1708 == env: VC3_ROOT_SED=/usr/ == env: OASIS_CERTIFICATES=/cvmfs/oasis.opensciencegrid.org/mis/certificates == env: ROOTSYS=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/lcg/root/6.14.09-pafccj6/ == env: SRT_PYTHIA8DATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/pythia8/240-pafccj5/share/Pythia8/xmldoc == env: SRT_HERWIGPATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/herwigpp/7.2.0-pafccj3/share/Herwig == env: CXX=g++ == env: SRT_QTLIB_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/lib == env: SRT_PYTHON3PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/share/overrides/python:/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20/python:/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/python:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw-tool-conf/45.0-pafccj26/lib/python3.6/site-packages == env: SHERPA_LIBRARY_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sherpa/2.2.10-pafccj2/lib/SHERPA-MC == env: SRT_PYTHON_VALGRIND_SUPP_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/python/2.7.14-pafccj/share/valgrind/valgrind-python.supp == env: HOME=/disk/vc3-root/vc3-home == env: SHLVL=3 == env: CMSSW_SEARCH_PATH=/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20/poison:/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20/src:/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20/external/slc7_amd64_gcc700/data:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/src:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/external/slc7_amd64_gcc700/data == env: VC3_VERSION_FULL_GZIP=1.5.0 == env: VC3_VERSION_MAKE=3.82 == env: GENEVA_DATA_DIR=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geneva/1.0-RC3-pafccj8/share/Geneva == env: DISK=350 == env: PARROT_CACHE=/disk/vc3-root/tmp/worker-230547-23860/cache/tmp == env: PARROT_CVMFS_ALIEN_CACHE=/disk/vc3-root/tmp/worker-230547-23860/cache/tmp/cvmfs == env: GOSAMCONTRIB_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gosamcontrib/2.0-20150803-pafccj == env: ROOT_TTREECACHE_PREFILL=0 == env: GENREFLEX=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/lcg/root/6.14.09-pafccj6/bin/genreflex == env: VINCIADATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/vincia/2.2.04-pafccj5/share/Vincia/xmldoc == env: SRT_G4LEDATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4EMLOW/7.3/data/G4EMLOW7.3 == env: SRT_GENREFLEX_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/lcg/root/6.14.09-pafccj6/bin/genreflex == env: SRT_GOSAMCONTRIB_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gosamcontrib/2.0-20150803-pafccj == env: SRT_ROOT_TTREECACHE_SIZE_SCRAMRTDEL=0 == env: SRT_EVTGENDATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/evtgen/1.6.0-pafccj5/share == env: SRT_GENEVA_DATA_DIR_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geneva/1.0-RC3-pafccj8/share/Geneva == env: GFAL_CONFIG_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/etc/gfal2.d/ == env: VC3_VERSION_FULL_WHICH=2.20.0 == env: MEMORY=1050 == env: HTTP_PROXY=http://eddie.crc.nd.edu:3128 == env: SRT_CMSSW_DATA_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms == env: HEPPDT_PARAM_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/heppdt/3.03.00-pafccj == env: SRT_LD_LIBRARY_PATH_SCRAMRT=/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20/biglib/slc7_amd64_gcc700:/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20/external/slc7_amd64_gcc700/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/biglib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/external/slc7_amd64_gcc700/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/llvm/7.1.0-pafccj/lib64:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib64:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/cuda/10.1.105-pafccj2/drivers == env: TMP=/disk/vc3-root/tmp/worker-230547-23860/t.527002/cctools-temp-t.527002.lV2uGj == env: PYTHONPATH=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib/python2.7/site-packages:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/lib64/python2.7/site-packages:python:python: == env: PYTHON3PATH=/cvmfs/cms.cern.ch/share/overrides/python:/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20/python:/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/python:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_20/lib/slc7_amd64_gcc700:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/cmssw-tool-conf/45.0-pafccj26/lib/python3.6/site-packages == env: QTLIB=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/qt/4.8.7-pafccj2/lib == env: SCONS_LIB_DIR=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/scons/3.0.1-pafccj/lib == env: VC3_INSTALL_USER_HOME=/disk/vc3-root/vc3-home == env: VC3_VERSION_FULL_VC3_BUILDER_HANDS=1.0.0 == env: COMPILER_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj == env: FRONTIER_CLIENT=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/frontier_client/2.9.1/ == env: SRT_G4NEUTRONXSDATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4NEUTRONXS/1.4/data/G4NEUTRONXS1.4 == env: SRT_SHERPA_LIBRARY_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sherpa/2.2.10-pafccj2/lib/SHERPA-MC == env: VC3_ROOT_TAR=/usr/ == env: VC3_VERSION_PKG_CONFIG=0.27 == env: CLHEP_PARAM_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/clhep/2.4.0.0-pafccj2 == env: CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 == env: VC3_SH_PROFILE_WRAPPER=/disk/vc3-root/vc3-home/.vc3_sh-P7YvX0.wrapper == env: PYTHIA8DATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/pythia8/240-pafccj5/share/Pythia8/xmldoc == env: HERWIGPATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/herwigpp/7.2.0-pafccj3/share/Herwig == env: SRT_FRONTIER_CLIENT_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/frontier_client/2.9.1/ == env: SRT_COMPILER_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj == env: SRT_CSC_TRACK_FINDER_DATA_DIR_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/cms/CSCTrackFinderEmulation/1.2-pafccj/data/ == env: GCC_RUNTIME_ASAN=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib64/libasan.so == env: SRT_ROOT_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/madgraph5amcatnlo/2.6.0-pafccj12:/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gosamcontrib/2.0-20150803-pafccj == env: CC=gcc == env: TEXMFHOME=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/rivet/2.7.0-pafccj8/share/Rivet/texmf == env: SRT_MLAS_DYNAMIC_CPU_ARCH_SCRAMRTDEL=0 == env: SRT_OPENBLAS_NUM_THREADS_SCRAMRTDEL=1 == env: SRT_GIT_TEMPLATE_DIR_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/git/2.19.0-pafccj/share/git-core/templates == env: SRT_G4SAIDXSDATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/geant4-G4SAIDDATA/1.1/data/G4SAIDDATA1.1 == env: CMSSW_VERSION=CMSSW_10_6_20 == env: SRT_GCC_RUNTIME_UBSAN_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/lib64/libubsan.so == env: SRT_LANG_SCRAMRTDEL=C == env: CCTOOLS_RESOURCE_MONITOR_INFO=1027 == env: VC3_VERSION_FULL_MAKE=3.82.0 == env: CORES=1 == env: PARROT_DEBUG_FLAGS= == env: SHERPA_SHARE_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/sherpa/2.2.10-pafccj2/share/SHERPA-MC == env: VC3_MACHINE_ARCH=x86_64 == env: VC3_ROOT_VC3_BUILDER_HANDS=/disk/vc3-root/x86_64/redhat7.4.1708/vc3-builder-hands/v1.0.0 == env: CMS_OPENLOOPS_PREFIX=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/openloops/2.1.0-pafccj2 == env: GIT_EXEC_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/git/2.19.0-pafccj/libexec/git-core == env: LOCALRT=/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20 == env: CMSSW_BASE=/disk/vc3-root/tmp/worker-230547-23860/t.527002/CMSSW_10_6_20 == env: SRT_SCRAM_CXX11_ABI_SCRAMRTDEL=1 == env: SRT_VALGRIND_LIB_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/valgrind/3.13.0-pafccj/lib/valgrind == env: VC3_COMMAND_LINE=/disk/vc3-builder --var CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 --require cctools-wq-worker -- work_queue_worker -M lobster_kmohrman_EFT_CRC_postLHE.* -t 300 -C catalog.cse.nd.edu,backup-catalog.cse.nd.edu -d all -o worker.log --cores=12 --memory=16000 --disk=18419 == env: _=/usr/bin/env === wrapper ready @ Tue May 18 06:32:07 EDT 2021 ==== working directory before execution @ Tue May 18 06:32:07 EDT 2021 ==== == dir: total 884 == dir: -rwxr-xr-x 2 kmohrman campus 1600 May 18 06:31 0052_4392_report.json == dir: -rwxr-xr-x 2 kmohrman campus 1598 May 18 06:31 0052_4395_report.json == dir: -rwxr-xr-x 2 kmohrman campus 1602 May 18 06:31 0052_4397_report.json == dir: -rwxr-xr-x 2 kmohrman campus 1600 May 18 06:31 0052_4426_report.json == dir: drwxr-xr-x 18 kmohrman campus 215 May 18 06:32 CMSSW_10_6_20 == dir: drwxr-xr-x 2 kmohrman campus 71 May 18 06:31 bin == dir: -rwxr-xr-x 5 kmohrman campus 627272 May 17 21:33 cctools-monitor == dir: -rw-r--r-- 1 kmohrman campus 0 May 18 06:31 cctools-monitor.summary == dir: drwxrwxrwx 2 kmohrman campus 6 May 18 06:31 cctools-temp-t.527002.lV2uGj == dir: -rw------- 1 kmohrman campus 51 May 18 06:32 cvmfschecksum.cms-ib.cern.ch == dir: -rw------- 1 kmohrman campus 51 May 18 06:31 cvmfschecksum.cms.cern.ch == dir: -rw------- 1 kmohrman campus 51 May 18 06:31 cvmfschecksum.oasis.opensciencegrid.org == dir: drwxr-xr-x 2 kmohrman campus 33 May 18 06:31 lib == dir: -rwxrwxrwx 1 kmohrman campus 146912 May 18 06:31 librmonitor_helper.so.ukIS6w == dir: -rw-r--r-- 2 kmohrman campus 868 May 18 06:31 merge_cfg.py == dir: -rw-r--r-- 2 kmohrman campus 808 May 18 06:31 merge_reports.py == dir: -rw-r--r-- 2 kmohrman campus 2528 May 18 06:31 parameters.json == dir: -rw------- 2 kmohrman campus 9921 May 18 06:31 proxy == dir: drwxr-xr-x 3 kmohrman campus 20 May 18 06:31 python == dir: -rw-r--r-- 5 kmohrman campus 570 May 17 21:33 report.json.in == dir: -rw-r--r-- 3 kmohrman campus 549 May 17 23:48 sandbox-CMSSW_10_6_20-slc7_amd64_gcc700.tar.bz2 == dir: drwxr-xr-x 4 kmohrman campus 37 May 18 06:31 siteconf == dir: -rw-r--r-- 1 kmohrman campus 11 May 18 06:32 t_wrapper_ready == dir: -rw-r--r-- 1 kmohrman campus 11 May 18 06:31 t_wrapper_start == dir: -rwxr-xr-x 5 kmohrman campus 47217 May 17 21:33 task.py == dir: -rwxr-xr-x 5 kmohrman campus 5986 May 17 21:33 wrapper.sh >>> data is {u'files': {u'info': {}, u'output_info': {}, u'skipped': []}, u'cpu_time': 0, u'exe_exit_code': 0, u'events_written': 0, u'output_size': 0, u'task_exit_code': 0, u'output_storage_element': u'', u'cache': {u'start_size': 0, u'type': 2, u'end_size': 0}, u'stageout_exit_code': 0, u'output_bare_size': 0, u'transfers': defaultdict(, {}), u'task_timing': {u'wrapper_ready': 0, u'wrapper_start': 0, u'prologue_end': 0, u'epilogue_end': 0, u'stage_out_end': 0, u'stage_in_end': 0, u'processing_end': 0}, u'events_per_run': 0} @ Tue May 18 06:32:14 2021 >>> Trying hdfs client access method @ Tue May 18 06:32:14 2021 >>> executing 'timeout 300 hdfs dfs -fs hdfs://eddie.crc.nd.edu:19000 -get /store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524392.root MAOD-00000_524392.root' @ Tue May 18 06:32:14 2021 >> using /disk/vc3-root/tmp/worker-230547-23860/t.527002/cctools-temp-t.527002.lV2uGj/tmpB47oDu to store command output >> cmd: timeout: failed to run command 'hdfs': No such file or directory >>>>> Unable to copy input with hdfs client @ Tue May 18 06:32:14 2021 >>> Trying xrootd access method @ Tue May 18 06:32:14 2021 >>> executing 'env XRD_LOGLEVEL=Debug timeout 300 xrdfs deepthought.crc.nd.edu stat /store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524392.root' @ Tue May 18 06:32:14 2021 >> using /disk/vc3-root/tmp/worker-230547-23860/t.527002/cctools-temp-t.527002.lV2uGj/tmpj6h4Qe to store command output >> cmd: [2021-05-18 06:32:14.709829 -0400][Warning][Utility ] Unable to process global config file: [ERROR] OS Error: No such file or directory >> cmd: [2021-05-18 06:32:14.711053 -0400][Debug ][Utility ] Unable to process user config file: [ERROR] OS Error: No such file or directory >> cmd: [2021-05-18 06:32:14.711250 -0400][Debug ][PlugInMgr ] Initializing plug-in manager... >> cmd: [2021-05-18 06:32:14.711309 -0400][Debug ][PlugInMgr ] No default plug-in, loading plug-in configs... >> cmd: [2021-05-18 06:32:14.711361 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /etc/xrootd/client.plugins.d... >> cmd: [2021-05-18 06:32:14.711468 -0400][Debug ][PlugInMgr ] Unable to process directory /etc/xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2021-05-18 06:32:14.711879 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /tmp/runos_singularity_home-193812410179019821928898941535482125024/.xrootd/client.plugins.d... >> cmd: [2021-05-18 06:32:14.711993 -0400][Debug ][PlugInMgr ] Unable to process directory /tmp/runos_singularity_home-193812410179019821928898941535482125024/.xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2021-05-18 06:32:14.712175 -0400][Debug ][App ] Executing: stat /store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524392.root >> cmd: [2021-05-18 06:32:14.712286 -0400][Debug ][Poller ] Available pollers: built-in >> cmd: [2021-05-18 06:32:14.712349 -0400][Debug ][Poller ] Attempting to create a poller according to preference: built-in >> cmd: [2021-05-18 06:32:14.712398 -0400][Debug ][Poller ] Creating poller: built-in >> cmd: [2021-05-18 06:32:14.712452 -0400][Debug ][Poller ] Creating and starting the built-in poller... >> cmd: [2021-05-18 06:32:14.712861 -0400][Debug ][Poller ] Using 1 poller threads >> cmd: [2021-05-18 06:32:14.712921 -0400][Debug ][TaskMgr ] Starting the task manager... >> cmd: [2021-05-18 06:32:14.713095 -0400][Debug ][TaskMgr ] Task manager started >> cmd: [2021-05-18 06:32:14.713148 -0400][Debug ][JobMgr ] Starting the job manager... >> cmd: [2021-05-18 06:32:14.713554 -0400][Debug ][JobMgr ] Job manager started, 3 workers >> cmd: [2021-05-18 06:32:14.713610 -0400][Debug ][TaskMgr ] Registering task: "FileTimer task" to be run at: [2021-05-18 06:32:14 -0400] >> cmd: [2021-05-18 06:32:14.713694 -0400][Debug ][PostMaster ] Creating new channel to: deepthought.crc.nd.edu:1094 1 stream(s) >> cmd: [2021-05-18 06:32:14.713769 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 >> cmd: [2021-05-18 06:32:14.714884 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" to be run at: [2021-05-18 06:32:29 -0400] >> cmd: [2021-05-18 06:32:14.716421 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094] Found 1 address(es): [::ffff:129.74.85.55]:1094 >> cmd: [2021-05-18 06:32:14.716612 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:129.74.85.55]:1094 >> cmd: [2021-05-18 06:32:14.716726 -0400][Debug ][Poller ] Adding socket 0xce0a10 to the poller >> cmd: [2021-05-18 06:32:14.717010 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Async connection call returned >> cmd: [2021-05-18 06:32:14.717159 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2021-05-18 06:32:14.717607 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: manager [], protocol version 500) >> cmd: [2021-05-18 06:32:14.717789 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: manager [], protocol version 500) >> cmd: [2021-05-18 06:32:14.720728 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d8civy106.crc.nd.edu&xrd.rn=v4.8.5, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2021-05-18 06:32:14.721117 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Logged in, session: 50bf62005c0e0000c40100000be06200 >> cmd: [2021-05-18 06:32:14.721192 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream 0 connected. >> cmd: [2021-05-18 06:32:14.721239 -0400][Debug ][Utility ] Monitor library name not set. No monitoring >> cmd: [2021-05-18 06:32:14.721967 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant04.crc.nd.edu:1096 1 stream(s) >> cmd: [2021-05-18 06:32:14.722033 -0400][Debug ][PostMaster ] [primeradiant04.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 >> cmd: [2021-05-18 06:32:14.723217 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant04.crc.nd.edu:1096" to be run at: [2021-05-18 06:32:29 -0400] >> cmd: [2021-05-18 06:32:14.724235 -0400][Debug ][PostMaster ] [primeradiant04.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.12]:1096 >> cmd: [2021-05-18 06:32:14.724375 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.12]:1096 >> cmd: [2021-05-18 06:32:14.724462 -0400][Debug ][Poller ] Adding socket 0x400016a0 to the poller >> cmd: [2021-05-18 06:32:14.724758 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2021-05-18 06:32:14.724876 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2021-05-18 06:32:14.725539 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 500) >> cmd: [2021-05-18 06:32:14.725708 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 500) >> cmd: [2021-05-18 06:32:14.728080 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d8civy106.crc.nd.edu&xrd.rn=v4.8.5, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2021-05-18 06:32:14.728779 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Logged in, session: f1230a001f670000bb00000048250a00 >> cmd: [2021-05-18 06:32:14.728831 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10400,c:ssl,ca:ba240aa8.0|f5f0dfc2.0 >> cmd: [2021-05-18 06:32:14.728880 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2021-05-18 06:32:14.735227 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: [2021-05-18 06:32:14.799867 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Sending more authentication data for gsi >> cmd: [2021-05-18 06:32:14.805178 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Authenticated with gsi. >> cmd: [2021-05-18 06:32:14.805274 -0400][Debug ][PostMaster ] [primeradiant04.crc.nd.edu:1096 #0] Stream 0 connected. >> cmd: Path: /store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524392.root >> cmd: Id: 0 >> cmd: Size: 67275741 >> cmd: MTime: 2021-05-18 10:21:51 >> cmd: Flags: 24 (Offline|IsReadable) >> cmd: [2021-05-18 06:32:14.808679 -0400][Debug ][JobMgr ] Stopping the job manager... >> cmd: [2021-05-18 06:32:14.809312 -0400][Debug ][JobMgr ] Job manager stopped >> cmd: [2021-05-18 06:32:14.809388 -0400][Debug ][TaskMgr ] Stopping the task manager... >> cmd: [2021-05-18 06:32:14.809575 -0400][Debug ][TaskMgr ] Task manager stopped >> cmd: [2021-05-18 06:32:14.809631 -0400][Debug ][Poller ] Stopping the poller... >> cmd: [2021-05-18 06:32:14.809971 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" >> cmd: [2021-05-18 06:32:14.810036 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2021-05-18 06:32:14.810094 -0400][Debug ][Poller ] <[::ffff:10.32.79.75]:41972><--><[::ffff:129.74.85.55]:1094> Removing socket from the poller >> cmd: [2021-05-18 06:32:14.810196 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Destroying stream >> cmd: [2021-05-18 06:32:14.810252 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2021-05-18 06:32:14.810310 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant04.crc.nd.edu:1096" >> cmd: [2021-05-18 06:32:14.810362 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2021-05-18 06:32:14.810415 -0400][Debug ][Poller ] <[::ffff:10.32.79.75]:39194><--><[::ffff:149.165.243.12]:1096> Removing socket from the poller >> cmd: [2021-05-18 06:32:14.810504 -0400][Debug ][PostMaster ] [primeradiant04.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2021-05-18 06:32:14.810558 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Closing the socket >>> will stream using xrootd instead of copying @ Tue May 18 06:32:14 2021 >>> Trying hdfs client access method @ Tue May 18 06:32:14 2021 >>> executing 'timeout 300 hdfs dfs -fs hdfs://eddie.crc.nd.edu:19000 -get /store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524395.root MAOD-00000_524395.root' @ Tue May 18 06:32:14 2021 >> using /disk/vc3-root/tmp/worker-230547-23860/t.527002/cctools-temp-t.527002.lV2uGj/tmpY64Dk_ to store command output >> cmd: timeout: failed to run command 'hdfs': No such file or directory >>>>> Unable to copy input with hdfs client @ Tue May 18 06:32:14 2021 >>> Trying xrootd access method @ Tue May 18 06:32:14 2021 >>> executing 'env XRD_LOGLEVEL=Debug timeout 300 xrdfs deepthought.crc.nd.edu stat /store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524395.root' @ Tue May 18 06:32:14 2021 >> using /disk/vc3-root/tmp/worker-230547-23860/t.527002/cctools-temp-t.527002.lV2uGj/tmpwM4kmL to store command output >> cmd: [2021-05-18 06:32:15.031656 -0400][Warning][Utility ] Unable to process global config file: [ERROR] OS Error: No such file or directory >> cmd: [2021-05-18 06:32:15.032913 -0400][Debug ][Utility ] Unable to process user config file: [ERROR] OS Error: No such file or directory >> cmd: [2021-05-18 06:32:15.033102 -0400][Debug ][PlugInMgr ] Initializing plug-in manager... >> cmd: [2021-05-18 06:32:15.033160 -0400][Debug ][PlugInMgr ] No default plug-in, loading plug-in configs... >> cmd: [2021-05-18 06:32:15.033211 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /etc/xrootd/client.plugins.d... >> cmd: [2021-05-18 06:32:15.033317 -0400][Debug ][PlugInMgr ] Unable to process directory /etc/xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2021-05-18 06:32:15.033704 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /tmp/runos_singularity_home-193812410179019821928898941535482125024/.xrootd/client.plugins.d... >> cmd: [2021-05-18 06:32:15.033823 -0400][Debug ][PlugInMgr ] Unable to process directory /tmp/runos_singularity_home-193812410179019821928898941535482125024/.xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2021-05-18 06:32:15.034023 -0400][Debug ][App ] Executing: stat /store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524395.root >> cmd: [2021-05-18 06:32:15.034149 -0400][Debug ][Poller ] Available pollers: built-in >> cmd: [2021-05-18 06:32:15.034199 -0400][Debug ][Poller ] Attempting to create a poller according to preference: built-in >> cmd: [2021-05-18 06:32:15.034248 -0400][Debug ][Poller ] Creating poller: built-in >> cmd: [2021-05-18 06:32:15.034304 -0400][Debug ][Poller ] Creating and starting the built-in poller... >> cmd: [2021-05-18 06:32:15.034841 -0400][Debug ][Poller ] Using 1 poller threads >> cmd: [2021-05-18 06:32:15.034933 -0400][Debug ][TaskMgr ] Starting the task manager... >> cmd: [2021-05-18 06:32:15.035085 -0400][Debug ][TaskMgr ] Task manager started >> cmd: [2021-05-18 06:32:15.035136 -0400][Debug ][JobMgr ] Starting the job manager... >> cmd: [2021-05-18 06:32:15.035497 -0400][Debug ][JobMgr ] Job manager started, 3 workers >> cmd: [2021-05-18 06:32:15.035551 -0400][Debug ][TaskMgr ] Registering task: "FileTimer task" to be run at: [2021-05-18 06:32:15 -0400] >> cmd: [2021-05-18 06:32:15.035633 -0400][Debug ][PostMaster ] Creating new channel to: deepthought.crc.nd.edu:1094 1 stream(s) >> cmd: [2021-05-18 06:32:15.035701 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 >> cmd: [2021-05-18 06:32:15.036737 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" to be run at: [2021-05-18 06:32:30 -0400] >> cmd: [2021-05-18 06:32:15.037609 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094] Found 1 address(es): [::ffff:129.74.85.55]:1094 >> cmd: [2021-05-18 06:32:15.037757 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:129.74.85.55]:1094 >> cmd: [2021-05-18 06:32:15.037864 -0400][Debug ][Poller ] Adding socket 0x232da10 to the poller >> cmd: [2021-05-18 06:32:15.038161 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Async connection call returned >> cmd: [2021-05-18 06:32:15.038328 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2021-05-18 06:32:15.038904 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: manager [], protocol version 500) >> cmd: [2021-05-18 06:32:15.039138 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: manager [], protocol version 500) >> cmd: [2021-05-18 06:32:15.043185 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d8civy106.crc.nd.edu&xrd.rn=v4.8.5, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2021-05-18 06:32:15.043647 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Logged in, session: 5abf62005c0e0000ca01000014e06200 >> cmd: [2021-05-18 06:32:15.043738 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream 0 connected. >> cmd: [2021-05-18 06:32:15.043792 -0400][Debug ][Utility ] Monitor library name not set. No monitoring >> cmd: [2021-05-18 06:32:15.044657 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant05.crc.nd.edu:1096 1 stream(s) >> cmd: [2021-05-18 06:32:15.044730 -0400][Debug ][PostMaster ] [primeradiant05.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 >> cmd: [2021-05-18 06:32:15.046655 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant05.crc.nd.edu:1096" to be run at: [2021-05-18 06:32:30 -0400] >> cmd: [2021-05-18 06:32:15.048108 -0400][Debug ][PostMaster ] [primeradiant05.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.13]:1096 >> cmd: [2021-05-18 06:32:15.048292 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.13]:1096 >> cmd: [2021-05-18 06:32:15.048395 -0400][Debug ][Poller ] Adding socket 0x1c0016a0 to the poller >> cmd: [2021-05-18 06:32:15.048559 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2021-05-18 06:32:15.048687 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2021-05-18 06:32:15.049371 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 500) >> cmd: [2021-05-18 06:32:15.049600 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 500) >> cmd: [2021-05-18 06:32:15.052500 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d8civy106.crc.nd.edu&xrd.rn=v4.8.5, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2021-05-18 06:32:15.053291 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Logged in, session: 5fc00500ac530000300100008ec10500 >> cmd: [2021-05-18 06:32:15.053350 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10400,c:ssl,ca:ba240aa8.0|f5f0dfc2.0 >> cmd: [2021-05-18 06:32:15.053403 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2021-05-18 06:32:15.060339 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: [2021-05-18 06:32:15.162009 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Sending more authentication data for gsi >> cmd: [2021-05-18 06:32:15.167503 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Authenticated with gsi. >> cmd: [2021-05-18 06:32:15.167608 -0400][Debug ][PostMaster ] [primeradiant05.crc.nd.edu:1096 #0] Stream 0 connected. >> cmd: Path: /store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524395.root >> cmd: Id: 0 >> cmd: Size: 64271312 >> cmd: MTime: 2021-05-18 10:22:01 >> cmd: Flags: 24 (Offline|IsReadable) >> cmd: [2021-05-18 06:32:15.170151 -0400][Debug ][JobMgr ] Stopping the job manager... >> cmd: [2021-05-18 06:32:15.170795 -0400][Debug ][JobMgr ] Job manager stopped >> cmd: [2021-05-18 06:32:15.170855 -0400][Debug ][TaskMgr ] Stopping the task manager... >> cmd: [2021-05-18 06:32:15.171052 -0400][Debug ][TaskMgr ] Task manager stopped >> cmd: [2021-05-18 06:32:15.171104 -0400][Debug ][Poller ] Stopping the poller... >> cmd: [2021-05-18 06:32:15.171457 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" >> cmd: [2021-05-18 06:32:15.171511 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2021-05-18 06:32:15.171560 -0400][Debug ][Poller ] <[::ffff:10.32.79.75]:41976><--><[::ffff:129.74.85.55]:1094> Removing socket from the poller >> cmd: [2021-05-18 06:32:15.171644 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Destroying stream >> cmd: [2021-05-18 06:32:15.171691 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2021-05-18 06:32:15.171740 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant05.crc.nd.edu:1096" >> cmd: [2021-05-18 06:32:15.171783 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2021-05-18 06:32:15.171832 -0400][Debug ][Poller ] <[::ffff:10.32.79.75]:39794><--><[::ffff:149.165.243.13]:1096> Removing socket from the poller >> cmd: [2021-05-18 06:32:15.171909 -0400][Debug ][PostMaster ] [primeradiant05.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2021-05-18 06:32:15.171953 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Closing the socket >>> will stream using xrootd instead of copying @ Tue May 18 06:32:15 2021 >>> Trying hdfs client access method @ Tue May 18 06:32:15 2021 >>> executing 'timeout 300 hdfs dfs -fs hdfs://eddie.crc.nd.edu:19000 -get /store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524397.root MAOD-00000_524397.root' @ Tue May 18 06:32:15 2021 >> using /disk/vc3-root/tmp/worker-230547-23860/t.527002/cctools-temp-t.527002.lV2uGj/tmpfACH4G to store command output >> cmd: timeout: failed to run command 'hdfs': No such file or directory >>>>> Unable to copy input with hdfs client @ Tue May 18 06:32:15 2021 >>> Trying xrootd access method @ Tue May 18 06:32:15 2021 >>> executing 'env XRD_LOGLEVEL=Debug timeout 300 xrdfs deepthought.crc.nd.edu stat /store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524397.root' @ Tue May 18 06:32:15 2021 >> using /disk/vc3-root/tmp/worker-230547-23860/t.527002/cctools-temp-t.527002.lV2uGj/tmpkSGlL4 to store command output >> cmd: [2021-05-18 06:32:15.407001 -0400][Warning][Utility ] Unable to process global config file: [ERROR] OS Error: No such file or directory >> cmd: [2021-05-18 06:32:15.408007 -0400][Debug ][Utility ] Unable to process user config file: [ERROR] OS Error: No such file or directory >> cmd: [2021-05-18 06:32:15.408168 -0400][Debug ][PlugInMgr ] Initializing plug-in manager... >> cmd: [2021-05-18 06:32:15.408217 -0400][Debug ][PlugInMgr ] No default plug-in, loading plug-in configs... >> cmd: [2021-05-18 06:32:15.408260 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /etc/xrootd/client.plugins.d... >> cmd: [2021-05-18 06:32:15.408347 -0400][Debug ][PlugInMgr ] Unable to process directory /etc/xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2021-05-18 06:32:15.408661 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /tmp/runos_singularity_home-193812410179019821928898941535482125024/.xrootd/client.plugins.d... >> cmd: [2021-05-18 06:32:15.408752 -0400][Debug ][PlugInMgr ] Unable to process directory /tmp/runos_singularity_home-193812410179019821928898941535482125024/.xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2021-05-18 06:32:15.408949 -0400][Debug ][App ] Executing: stat /store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524397.root >> cmd: [2021-05-18 06:32:15.409055 -0400][Debug ][Poller ] Available pollers: built-in >> cmd: [2021-05-18 06:32:15.409098 -0400][Debug ][Poller ] Attempting to create a poller according to preference: built-in >> cmd: [2021-05-18 06:32:15.409139 -0400][Debug ][Poller ] Creating poller: built-in >> cmd: [2021-05-18 06:32:15.409186 -0400][Debug ][Poller ] Creating and starting the built-in poller... >> cmd: [2021-05-18 06:32:15.409545 -0400][Debug ][Poller ] Using 1 poller threads >> cmd: [2021-05-18 06:32:15.409600 -0400][Debug ][TaskMgr ] Starting the task manager... >> cmd: [2021-05-18 06:32:15.409749 -0400][Debug ][TaskMgr ] Task manager started >> cmd: [2021-05-18 06:32:15.409800 -0400][Debug ][JobMgr ] Starting the job manager... >> cmd: [2021-05-18 06:32:15.410181 -0400][Debug ][JobMgr ] Job manager started, 3 workers >> cmd: [2021-05-18 06:32:15.410236 -0400][Debug ][TaskMgr ] Registering task: "FileTimer task" to be run at: [2021-05-18 06:32:15 -0400] >> cmd: [2021-05-18 06:32:15.410313 -0400][Debug ][PostMaster ] Creating new channel to: deepthought.crc.nd.edu:1094 1 stream(s) >> cmd: [2021-05-18 06:32:15.410378 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 >> cmd: [2021-05-18 06:32:15.411657 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" to be run at: [2021-05-18 06:32:30 -0400] >> cmd: [2021-05-18 06:32:15.412702 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094] Found 1 address(es): [::ffff:129.74.85.55]:1094 >> cmd: [2021-05-18 06:32:15.412854 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:129.74.85.55]:1094 >> cmd: [2021-05-18 06:32:15.412954 -0400][Debug ][Poller ] Adding socket 0xc06a10 to the poller >> cmd: [2021-05-18 06:32:15.413223 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Async connection call returned >> cmd: [2021-05-18 06:32:15.413396 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2021-05-18 06:32:15.413998 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: manager [], protocol version 500) >> cmd: [2021-05-18 06:32:15.414234 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: manager [], protocol version 500) >> cmd: [2021-05-18 06:32:15.417298 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d8civy106.crc.nd.edu&xrd.rn=v4.8.5, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2021-05-18 06:32:15.417709 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Logged in, session: 63bf62005c0e0000ca0100001ee06200 >> cmd: [2021-05-18 06:32:15.417807 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream 0 connected. >> cmd: [2021-05-18 06:32:15.417883 -0400][Debug ][Utility ] Monitor library name not set. No monitoring >> cmd: [2021-05-18 06:32:15.418762 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant02.crc.nd.edu:1096 1 stream(s) >> cmd: [2021-05-18 06:32:15.418837 -0400][Debug ][PostMaster ] [primeradiant02.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 >> cmd: [2021-05-18 06:32:15.420390 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant02.crc.nd.edu:1096" to be run at: [2021-05-18 06:32:30 -0400] >> cmd: [2021-05-18 06:32:15.421856 -0400][Debug ][PostMaster ] [primeradiant02.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.6]:1096 >> cmd: [2021-05-18 06:32:15.422043 -0400][Debug ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.6]:1096 >> cmd: [2021-05-18 06:32:15.422154 -0400][Debug ][Poller ] Adding socket 0x200016a0 to the poller >> cmd: [2021-05-18 06:32:15.422346 -0400][Debug ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2021-05-18 06:32:15.422473 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2021-05-18 06:32:15.422949 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 500) >> cmd: [2021-05-18 06:32:15.423198 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 500) >> cmd: [2021-05-18 06:32:15.426219 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d8civy106.crc.nd.edu&xrd.rn=v4.8.5, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2021-05-18 06:32:15.426610 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Logged in, session: 1db30100ae5d0000b70000001db30100 >> cmd: [2021-05-18 06:32:15.426664 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10400,c:ssl,ca:ba240aa8.0|f5f0dfc2.0 >> cmd: [2021-05-18 06:32:15.426716 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2021-05-18 06:32:15.433536 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: [2021-05-18 06:32:15.495185 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Sending more authentication data for gsi >> cmd: [2021-05-18 06:32:15.500103 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Authenticated with gsi. >> cmd: [2021-05-18 06:32:15.500214 -0400][Debug ][PostMaster ] [primeradiant02.crc.nd.edu:1096 #0] Stream 0 connected. >> cmd: Path: /store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524397.root >> cmd: Id: 0 >> cmd: Size: 67083177 >> cmd: MTime: 2021-05-18 10:21:45 >> cmd: Flags: 24 (Offline|IsReadable) >> cmd: [2021-05-18 06:32:15.502352 -0400][Debug ][JobMgr ] Stopping the job manager... >> cmd: [2021-05-18 06:32:15.502979 -0400][Debug ][JobMgr ] Job manager stopped >> cmd: [2021-05-18 06:32:15.503036 -0400][Debug ][TaskMgr ] Stopping the task manager... >> cmd: [2021-05-18 06:32:15.503324 -0400][Debug ][TaskMgr ] Task manager stopped >> cmd: [2021-05-18 06:32:15.503370 -0400][Debug ][Poller ] Stopping the poller... >> cmd: [2021-05-18 06:32:15.503717 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" >> cmd: [2021-05-18 06:32:15.503770 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2021-05-18 06:32:15.503824 -0400][Debug ][Poller ] <[::ffff:10.32.79.75]:41980><--><[::ffff:129.74.85.55]:1094> Removing socket from the poller >> cmd: [2021-05-18 06:32:15.503915 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Destroying stream >> cmd: [2021-05-18 06:32:15.503963 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2021-05-18 06:32:15.504012 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant02.crc.nd.edu:1096" >> cmd: [2021-05-18 06:32:15.504055 -0400][Debug ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2021-05-18 06:32:15.504098 -0400][Debug ][Poller ] <[::ffff:10.32.79.75]:36992><--><[::ffff:149.165.243.6]:1096> Removing socket from the poller >> cmd: [2021-05-18 06:32:15.504169 -0400][Debug ][PostMaster ] [primeradiant02.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2021-05-18 06:32:15.504213 -0400][Debug ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Closing the socket >>> will stream using xrootd instead of copying @ Tue May 18 06:32:15 2021 >>> Trying hdfs client access method @ Tue May 18 06:32:15 2021 >>> executing 'timeout 300 hdfs dfs -fs hdfs://eddie.crc.nd.edu:19000 -get /store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524426.root MAOD-00000_524426.root' @ Tue May 18 06:32:15 2021 >> using /disk/vc3-root/tmp/worker-230547-23860/t.527002/cctools-temp-t.527002.lV2uGj/tmpAAIX7q to store command output >> cmd: timeout: failed to run command 'hdfs': No such file or directory >>>>> Unable to copy input with hdfs client @ Tue May 18 06:32:15 2021 >>> Trying xrootd access method @ Tue May 18 06:32:15 2021 >>> executing 'env XRD_LOGLEVEL=Debug timeout 300 xrdfs deepthought.crc.nd.edu stat /store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524426.root' @ Tue May 18 06:32:15 2021 >> using /disk/vc3-root/tmp/worker-230547-23860/t.527002/cctools-temp-t.527002.lV2uGj/tmpZOo7EL to store command output >> cmd: [2021-05-18 06:32:15.778298 -0400][Warning][Utility ] Unable to process global config file: [ERROR] OS Error: No such file or directory >> cmd: [2021-05-18 06:32:15.779423 -0400][Debug ][Utility ] Unable to process user config file: [ERROR] OS Error: No such file or directory >> cmd: [2021-05-18 06:32:15.779584 -0400][Debug ][PlugInMgr ] Initializing plug-in manager... >> cmd: [2021-05-18 06:32:15.779642 -0400][Debug ][PlugInMgr ] No default plug-in, loading plug-in configs... >> cmd: [2021-05-18 06:32:15.779688 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /etc/xrootd/client.plugins.d... >> cmd: [2021-05-18 06:32:15.779776 -0400][Debug ][PlugInMgr ] Unable to process directory /etc/xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2021-05-18 06:32:15.785139 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /tmp/runos_singularity_home-193812410179019821928898941535482125024/.xrootd/client.plugins.d... >> cmd: [2021-05-18 06:32:15.785246 -0400][Debug ][PlugInMgr ] Unable to process directory /tmp/runos_singularity_home-193812410179019821928898941535482125024/.xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2021-05-18 06:32:15.785424 -0400][Debug ][App ] Executing: stat /store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524426.root >> cmd: [2021-05-18 06:32:15.785527 -0400][Debug ][Poller ] Available pollers: built-in >> cmd: [2021-05-18 06:32:15.785569 -0400][Debug ][Poller ] Attempting to create a poller according to preference: built-in >> cmd: [2021-05-18 06:32:15.785609 -0400][Debug ][Poller ] Creating poller: built-in >> cmd: [2021-05-18 06:32:15.785657 -0400][Debug ][Poller ] Creating and starting the built-in poller... >> cmd: [2021-05-18 06:32:15.786027 -0400][Debug ][Poller ] Using 1 poller threads >> cmd: [2021-05-18 06:32:15.786094 -0400][Debug ][TaskMgr ] Starting the task manager... >> cmd: [2021-05-18 06:32:15.786241 -0400][Debug ][TaskMgr ] Task manager started >> cmd: [2021-05-18 06:32:15.786291 -0400][Debug ][JobMgr ] Starting the job manager... >> cmd: [2021-05-18 06:32:15.786644 -0400][Debug ][JobMgr ] Job manager started, 3 workers >> cmd: [2021-05-18 06:32:15.786698 -0400][Debug ][TaskMgr ] Registering task: "FileTimer task" to be run at: [2021-05-18 06:32:15 -0400] >> cmd: [2021-05-18 06:32:15.786775 -0400][Debug ][PostMaster ] Creating new channel to: deepthought.crc.nd.edu:1094 1 stream(s) >> cmd: [2021-05-18 06:32:15.786854 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 >> cmd: [2021-05-18 06:32:15.787827 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" to be run at: [2021-05-18 06:32:30 -0400] >> cmd: [2021-05-18 06:32:15.788795 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094] Found 1 address(es): [::ffff:129.74.85.55]:1094 >> cmd: [2021-05-18 06:32:15.788947 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:129.74.85.55]:1094 >> cmd: [2021-05-18 06:32:15.789045 -0400][Debug ][Poller ] Adding socket 0x1803a10 to the poller >> cmd: [2021-05-18 06:32:15.789326 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Async connection call returned >> cmd: [2021-05-18 06:32:15.789486 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2021-05-18 06:32:15.790080 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: manager [], protocol version 500) >> cmd: [2021-05-18 06:32:15.790313 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: manager [], protocol version 500) >> cmd: [2021-05-18 06:32:15.802128 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d8civy106.crc.nd.edu&xrd.rn=v4.8.5, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2021-05-18 06:32:15.802599 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Logged in, session: 6bbf62005c0e0000cf01000026e06200 >> cmd: [2021-05-18 06:32:15.802686 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream 0 connected. >> cmd: [2021-05-18 06:32:15.802740 -0400][Debug ][Utility ] Monitor library name not set. No monitoring >> cmd: [2021-05-18 06:32:15.803598 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x17fccb0" to be run at: [2021-05-18 06:32:30 -0400] >> cmd: [2021-05-18 06:32:30.789152 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x17fccb0" >> cmd: [2021-05-18 06:32:30.789993 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x17fccb0" to be run at: [2021-05-18 06:32:45 -0400] >> cmd: [2021-05-18 06:32:45.791988 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x17fccb0" >> cmd: [2021-05-18 06:32:45.792804 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x17fccb0" to be run at: [2021-05-18 06:33:00 -0400] >> cmd: [2021-05-18 06:33:00.795017 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x17fccb0" >> cmd: [2021-05-18 06:33:00.795961 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x17fccb0" to be run at: [2021-05-18 06:33:15 -0400] >> cmd: [2021-05-18 06:33:15.798033 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x17fccb0" >> cmd: [2021-05-18 06:33:15.798860 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x17fccb0" to be run at: [2021-05-18 06:33:30 -0400] >> cmd: [2021-05-18 06:33:30.800989 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x17fccb0" >> cmd: [2021-05-18 06:33:30.801834 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x17fccb0" to be run at: [2021-05-18 06:33:45 -0400] >> cmd: [2021-05-18 06:33:45.803824 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x17fccb0" >> cmd: [2021-05-18 06:33:45.804732 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x17fccb0" to be run at: [2021-05-18 06:34:00 -0400] >> cmd: [2021-05-18 06:34:00.806728 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x17fccb0" >> cmd: [2021-05-18 06:34:00.807504 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x17fccb0" to be run at: [2021-05-18 06:34:15 -0400] >> cmd: [2021-05-18 06:34:15.809493 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x17fccb0" >> cmd: [2021-05-18 06:34:15.810382 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x17fccb0" to be run at: [2021-05-18 06:34:30 -0400] >> cmd: [2021-05-18 06:34:30.811927 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x17fccb0" >> cmd: [2021-05-18 06:34:30.812710 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x17fccb0" to be run at: [2021-05-18 06:34:45 -0400] >> cmd: [2021-05-18 06:34:45.814443 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x17fccb0" >> cmd: [2021-05-18 06:34:45.815301 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x17fccb0" to be run at: [2021-05-18 06:35:00 -0400] >> cmd: [2021-05-18 06:35:00.817094 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x17fccb0" >> cmd: [2021-05-18 06:35:00.817953 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x17fccb0" to be run at: [2021-05-18 06:35:15 -0400] >> cmd: [2021-05-18 06:35:15.819761 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x17fccb0" >> cmd: [2021-05-18 06:35:15.820610 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x17fccb0" to be run at: [2021-05-18 06:35:30 -0400] >> cmd: [2021-05-18 06:35:30.823001 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x17fccb0" >> cmd: [2021-05-18 06:35:30.823801 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x17fccb0" to be run at: [2021-05-18 06:35:45 -0400] >> cmd: [2021-05-18 06:35:45.826055 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x17fccb0" >> cmd: [2021-05-18 06:35:45.826898 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x17fccb0" to be run at: [2021-05-18 06:36:00 -0400] >> cmd: [2021-05-18 06:36:00.829097 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x17fccb0" >> cmd: [2021-05-18 06:36:00.829943 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x17fccb0" to be run at: [2021-05-18 06:36:15 -0400] >> cmd: [2021-05-18 06:36:15.831893 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x17fccb0" >> cmd: [2021-05-18 06:36:15.832699 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x17fccb0" to be run at: [2021-05-18 06:36:30 -0400] >> cmd: [2021-05-18 06:36:30.834435 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x17fccb0" >> cmd: [2021-05-18 06:36:30.835308 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x17fccb0" to be run at: [2021-05-18 06:36:45 -0400] >> cmd: [2021-05-18 06:36:45.837333 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x17fccb0" >> cmd: [2021-05-18 06:36:45.838115 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x17fccb0" to be run at: [2021-05-18 06:37:00 -0400] >> cmd: [2021-05-18 06:37:00.840086 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x17fccb0" >> cmd: [2021-05-18 06:37:00.840789 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x17fccb0" to be run at: [2021-05-18 06:37:15 -0400] >>> xrootd access to input file unavailable @ Tue May 18 06:37:15 2021 >>> Trying srm access method @ Tue May 18 06:37:15 2021 >>> executing '/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/bin/gfal-copy gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524426.root MAOD-00000_524426.root' @ Tue May 18 06:37:15 2021 >> using /disk/vc3-root/tmp/worker-230547-23860/t.527002/cctools-temp-t.527002.lV2uGj/tmpE1vkXr to store command output >> cmd: Copying 67481295 bytes gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524426.root => file:///disk/vc3-root/tmp/worker-230547-23860/t.527002/MAOD-00000_524426.root >>> Successfully copied input with SRM @ Tue May 18 06:37:24 2021 >>> modified input files @ Tue May 18 06:37:24 2021 >> input: root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524392.root >> input: root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524395.root >> input: root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524397.root >> input: file:MAOD-00000_524426.root >>> updated parameters are @ Tue May 18 06:37:24 2021 >> json: { >> json: "accelerate stage-in": 3, >> json: "arguments": [ >> json: "outputFile=MAOD-00000.root" >> json: ], >> json: "default ce": "T3_US_NotreDame", >> json: "default host": "earth.crc.nd.edu", >> json: "default se": "T3_US_NotreDame", >> json: "disable streaming": false, >> json: "epilogue": [ >> json: "python", >> json: "merge_reports.py", >> json: "report.json", >> json: "0052_4426_report.json", >> json: "0052_4392_report.json", >> json: "0052_4397_report.json", >> json: "0052_4395_report.json" >> json: ], >> json: "executable": "cmsRun", >> json: "file map": { >> json: "file:MAOD-00000_524426.root": "mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524426.root", >> json: "root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524392.root": "mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524392.root", >> json: "root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524395.root": "mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524395.root", >> json: "root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524397.root": "mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524397.root" >> json: }, >> json: "gridpack": false, >> json: "input": [ >> json: "hdfs://eddie.crc.nd.edu:19000/store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/", >> json: "root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/", >> json: "gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1", >> json: "gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1", >> json: "file:///hadoop/store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/" >> json: ], >> json: "mask": { >> json: "events": -1, >> json: "files": [ >> json: "root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524392.root", >> json: "root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524395.root", >> json: "root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524397.root", >> json: "file:MAOD-00000_524426.root" >> json: ], >> json: "lumis": null >> json: }, >> json: "monitoring": { >> json: "monitorid": null, >> json: "syncid": null, >> json: "taskid": "lobster_kmohrman_EFT_CRC_postLHE_crc_20210513_2345_be4f8d09b389a8f7" >> json: }, >> json: "output": [ >> json: "hdfs://eddie.crc.nd.edu:19000/store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/", >> json: "root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/", >> json: "gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1", >> json: "gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1", >> json: "file:///hadoop/store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/" >> json: ], >> json: "output files": [ >> json: [ >> json: "MAOD-00000.root", >> json: "mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_527002.root" >> json: ] >> json: ], >> json: "prologue": null, >> json: "pset": "merge_cfg.py", >> json: "want summary": true >> json: } >>> using sync CE T3_US_NotreDame @ Tue May 18 06:37:24 2021 >>> config file fragment @ Tue May 18 06:37:26 2021 >> pset: >> pset: import FWCore.ParameterSet.Config as cms >> pset: process.Timing = cms.Service("Timing", >> pset: useJobReport = cms.untracked.bool(True), >> pset: summaryOnly = cms.untracked.bool(True)) >> pset: >> pset: process.maxEvents = cms.untracked.PSet(input = cms.untracked.int32(-1)) >> pset: >> pset: import os >> pset: _, major, minor, _ = os.environ["CMSSW_VERSION"].split('_', 3) >> pset: if int(major) >= 7 and int(minor) >= 4: >> pset: xrdstats = cms.Service("XrdAdaptor::XrdStatisticsService", cms.untracked.PSet(reportToFJR = cms.untracked.bool(True))) >> pset: process.add_(xrdstats) >> pset: >> pset: for prod in process.producers.values(): >> pset: if prod.hasParameter('nEvents') and prod.type_() == 'ExternalLHEProducer': >> pset: prod.nEvents = cms.untracked.uint32(-1) >> pset: >> pset: process.source.fileNames = cms.untracked.vstring(['root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524392.root', 'root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524395.root', 'root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524397.root', 'file:MAOD-00000_524426.root']) >> pset: if hasattr(process, 'options'): >> pset: process.options.wantSummary = cms.untracked.bool(True) >> pset: else: >> pset: process.options = cms.untracked.PSet(wantSummary = cms.untracked.bool(True)) >>> executing 'cmsRun -j report.xml merge_cfg_mod.py outputFile=MAOD-00000.root' @ Tue May 18 06:37:26 2021 >> using /disk/vc3-root/tmp/worker-230547-23860/t.527002/cctools-temp-t.527002.lV2uGj/tmpJaI2zR to store command output >> cmd: 18-May-2021 06:37:33 EDT Initiating request to open file root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524392.root >> cmd: 18-May-2021 06:37:43 EDT Successfully opened file root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524392.root >> cmd: Begin processing the 1st record. Run 1, Event 31, LumiSection 31543 on stream 0 at 18-May-2021 06:37:59.260 EDT >> cmd: 18-May-2021 06:38:11 EDT Closed file root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524392.root >> cmd: 18-May-2021 06:38:11 EDT Initiating request to open file root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524395.root >> cmd: 18-May-2021 06:38:13 EDT Successfully opened file root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524395.root >> cmd: 18-May-2021 06:38:43 EDT Closed file root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524395.root >> cmd: 18-May-2021 06:38:43 EDT Initiating request to open file root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524397.root >> cmd: 18-May-2021 06:38:45 EDT Successfully opened file root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524397.root >> cmd: Begin processing the 1001st record. Run 1, Event 101, LumiSection 18418 on stream 0 at 18-May-2021 06:39:06.281 EDT >> cmd: 18-May-2021 06:39:14 EDT Closed file root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_524397.root >> cmd: 18-May-2021 06:39:14 EDT Initiating request to open file file:MAOD-00000_524426.root >> cmd: 18-May-2021 06:39:14 EDT Successfully opened file file:MAOD-00000_524426.root >> cmd: 18-May-2021 06:39:32 EDT Closed file file:MAOD-00000_524426.root >> cmd: >> cmd: TrigReport ---------- Event Summary ------------ >> cmd: TrigReport Events total = 1948 passed = 1948 failed = 0 >> cmd: >> cmd: TrigReport ---------- Path Summary ------------ >> cmd: TrigReport Trig Bit# Executed Passed Failed Error Name >> cmd: >> cmd: TrigReport -------End-Path Summary ------------ >> cmd: TrigReport Trig Bit# Executed Passed Failed Error Name >> cmd: TrigReport 0 0 1948 1948 0 0 end >> cmd: >> cmd: TrigReport ------ Modules in End-Path: end ------------ >> cmd: TrigReport Trig Bit# Visited Passed Failed Error Name >> cmd: TrigReport 0 0 1948 1948 0 0 out >> cmd: >> cmd: TrigReport ---------- Module Summary ------------ >> cmd: TrigReport Visited Executed Passed Failed Error Name >> cmd: TrigReport 1948 1948 1948 0 0 end >> cmd: TrigReport 1948 1948 1948 0 0 out >> cmd: >> cmd: TimeReport ---------- Event Summary ---[sec]---- >> cmd: TimeReport event loop CPU/event = 0.014415 >> cmd: TimeReport event loop Real/event = 0.057168 >> cmd: TimeReport sum Streams Real/event = 0.022510 >> cmd: TimeReport efficiency CPU/Real/thread = 0.252152 >> cmd: >> cmd: TimeReport ---------- Path Summary ---[Real sec]---- >> cmd: TimeReport per event per exec Name >> cmd: TimeReport per event per exec Name >> cmd: >> cmd: TimeReport -------End-Path Summary ---[Real sec]---- >> cmd: TimeReport per event per exec Name >> cmd: TimeReport 0.022322 0.022322 end >> cmd: TimeReport per event per exec Name >> cmd: >> cmd: TimeReport ------ Modules in End-Path: end ---[Real sec]---- >> cmd: TimeReport per event per visit Name >> cmd: TimeReport 0.000140 0.000140 out >> cmd: TimeReport per event per visit Name >> cmd: >> cmd: TimeReport ---------- Module Summary ---[Real sec]---- >> cmd: TimeReport per event per exec per visit Name >> cmd: TimeReport 0.000007 0.000007 0.000007 end >> cmd: TimeReport 0.000140 0.000140 0.000140 out >> cmd: TimeReport per event per exec per visit Name >> cmd: >> cmd: T---Report end! >> cmd: >> cmd: TimeReport> Time report complete in 129.259 seconds >> cmd: Time Summary: >> cmd: - Min event: 0.0100648 >> cmd: - Max event: 1.76616 >> cmd: - Avg event: 0.0223293 >> cmd: - Total loop: 108.206 >> cmd: - Total init: 17.8848 >> cmd: - Total job: 129.259 >> cmd: - EventSetup Lock: 0 >> cmd: - EventSetup Get: 0 >> cmd: Event Throughput: 18.0027 ev/s >> cmd: CPU Summary: >> cmd: - Total loop: 62.7501 >> cmd: - Total init: 7.11666 >> cmd: - Total extra: 0 >> cmd: - Total job: 72.8156 >> cmd: Processing Summary: >> cmd: - Number of Events: 1948 >> cmd: - Number of Global Begin Lumi Calls: 12 >> cmd: - Number of Global Begin Run Calls: 12 >> cmd: >> cmd: >> cmd: ============================================= >> cmd: >> cmd: MessageLogger Summary >> cmd: >> cmd: type category sev module subroutine count total >> cmd: ---- -------------------- -- ---------------- ---------------- ----- ----- >> cmd: 1 TimeReport -e AfterModEndJob 1 1 >> cmd: 2 fileAction -s file_close 4 4 >> cmd: 3 fileAction -s file_open 8 8 >> cmd: >> cmd: type category Examples: run/evt run/evt run/evt >> cmd: ---- -------------------- ---------------- ---------------- ---------------- >> cmd: 1 TimeReport PostGlobalEndRun >> cmd: 2 fileAction PostProcessEvent PostProcessEvent PostGlobalEndRun >> cmd: 3 fileAction pre-events pre-events PostProcessEvent >> cmd: >> cmd: Severity # Occurrences Total Occurrences >> cmd: -------- ------------- ----------------- >> cmd: Error 1 1 >> cmd: System 12 12 >> cmd: >> cmd: dropped waiting message count 0 >>> executable returned with exit code 0. @ Tue May 18 06:39:36 2021 >>> epilogue @ Tue May 18 06:39:48 2021 >>> executing 'python merge_reports.py report.json 0052_4426_report.json 0052_4392_report.json 0052_4397_report.json 0052_4395_report.json' @ Tue May 18 06:39:48 2021 >> using /disk/vc3-root/tmp/worker-230547-23860/t.527002/cctools-temp-t.527002.lV2uGj/tmpqQFskI to store command output >> cmd: >> merging 0052_4426_report.json >> cmd: >> merging 0052_4392_report.json >> cmd: >> merging 0052_4397_report.json >> cmd: >> merging 0052_4395_report.json >>> executing 'timeout 300 hdfs dfs -fs hdfs://eddie.crc.nd.edu:19000 -put MAOD-00000.root /store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_527002.root' @ Tue May 18 06:39:57 2021 >> using /disk/vc3-root/tmp/worker-230547-23860/t.527002/cctools-temp-t.527002.lV2uGj/tmph8gXuW to store command output >> cmd: timeout: failed to run command 'hdfs': No such file or directory >>> Checking output file transfer. @ Tue May 18 06:39:58 2021 >>> executing 'env XRD_LOGLEVEL=Debug xrdcp MAOD-00000.root root://deepthought.crc.nd.edu//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_527002.root' @ Tue May 18 06:39:58 2021 >> using /disk/vc3-root/tmp/worker-230547-23860/t.527002/cctools-temp-t.527002.lV2uGj/tmpet7amV to store command output >> cmd: [2021-05-18 06:39:58.139782 -0400][Warning][Utility ] Unable to process global config file: [ERROR] OS Error: No such file or directory >> cmd: [2021-05-18 06:39:58.140781 -0400][Debug ][Utility ] Unable to process user config file: [ERROR] OS Error: No such file or directory >> cmd: [2021-05-18 06:39:58.140948 -0400][Debug ][PlugInMgr ] Initializing plug-in manager... >> cmd: [2021-05-18 06:39:58.141001 -0400][Debug ][PlugInMgr ] No default plug-in, loading plug-in configs... >> cmd: [2021-05-18 06:39:58.141043 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /etc/xrootd/client.plugins.d... >> cmd: [2021-05-18 06:39:58.141130 -0400][Debug ][PlugInMgr ] Unable to process directory /etc/xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2021-05-18 06:39:58.141446 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /tmp/runos_singularity_home-193812410179019821928898941535482125024/.xrootd/client.plugins.d... >> cmd: [2021-05-18 06:39:58.141538 -0400][Debug ][PlugInMgr ] Unable to process directory /tmp/runos_singularity_home-193812410179019821928898941535482125024/.xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2021-05-18 06:39:58.141822 -0400][Debug ][Poller ] Available pollers: built-in >> cmd: [2021-05-18 06:39:58.141876 -0400][Debug ][Poller ] Attempting to create a poller according to preference: built-in >> cmd: [2021-05-18 06:39:58.141917 -0400][Debug ][Poller ] Creating poller: built-in >> cmd: [2021-05-18 06:39:58.141963 -0400][Debug ][Poller ] Creating and starting the built-in poller... >> cmd: [2021-05-18 06:39:58.142344 -0400][Debug ][Poller ] Using 1 poller threads >> cmd: [2021-05-18 06:39:58.142399 -0400][Debug ][TaskMgr ] Starting the task manager... >> cmd: [2021-05-18 06:39:58.142537 -0400][Debug ][TaskMgr ] Task manager started >> cmd: [2021-05-18 06:39:58.142585 -0400][Debug ][JobMgr ] Starting the job manager... >> cmd: [2021-05-18 06:39:58.142938 -0400][Debug ][JobMgr ] Job manager started, 3 workers >> cmd: [2021-05-18 06:39:58.142993 -0400][Debug ][TaskMgr ] Registering task: "FileTimer task" to be run at: [2021-05-18 06:39:58 -0400] >> cmd: [2021-05-18 06:39:58.143068 -0400][Debug ][PostMaster ] Creating new channel to: deepthought.crc.nd.edu:1094 1 stream(s) >> cmd: [2021-05-18 06:39:58.143131 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 >> cmd: [2021-05-18 06:39:58.144481 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" to be run at: [2021-05-18 06:40:13 -0400] >> cmd: [2021-05-18 06:39:58.145465 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094] Found 1 address(es): [::ffff:129.74.85.55]:1094 >> cmd: [2021-05-18 06:39:58.145610 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:129.74.85.55]:1094 >> cmd: [2021-05-18 06:39:58.145705 -0400][Debug ][Poller ] Adding socket 0x1068100 to the poller >> cmd: [2021-05-18 06:39:58.146028 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Async connection call returned >> cmd: [2021-05-18 06:39:58.146202 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2021-05-18 06:39:58.146795 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: manager [], protocol version 500) >> cmd: [2021-05-18 06:39:58.147040 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: manager [], protocol version 500) >> cmd: [2021-05-18 06:39:58.150350 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=d8civy106.crc.nd.edu&xrd.rn=v4.8.5, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2021-05-18 06:39:58.150795 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Logged in, session: 29c762005c0e00002d060000e4e76200 >> cmd: [2021-05-18 06:39:58.150899 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream 0 connected. >> cmd: [2021-05-18 06:39:58.150955 -0400][Debug ][Utility ] Monitor library name not set. No monitoring >> cmd: [2021-05-18 06:39:58.511914 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x1063040" to be run at: [2021-05-18 06:40:03 -0400] >> cmd: [2021-05-18 06:40:03.143851 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x1063040" >> cmd: [2021-05-18 06:40:03.676711 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x1063040" to be run at: [2021-05-18 06:40:08 -0400] >> cmd: [2021-05-18 06:40:08.145141 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x1063040" >> cmd: [2021-05-18 06:40:08.145913 -0400][Debug ][XRootD ] [deepthought.crc.nd.edu:1094] Handling error while processing kXR_stat (path: /store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_527002.root, flags: none): [ERROR] Error response: No such file or directory. >> cmd: [2021-05-18 06:40:08.146279 -0400][Debug ][Utility ] CopyProcess: 2 jobs to prepare >> cmd: [2021-05-18 06:40:08.147105 -0400][Debug ][XRootD ] [deepthought.crc.nd.edu:1094] Handling error while processing kXR_stat (path: /store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_527002.root, flags: none): [ERROR] Error response: No such file or directory. >> cmd: [2021-05-18 06:40:08.147269 -0400][Debug ][Utility ] Creating a classic copy job, from file://localhost/disk/vc3-root/tmp/worker-230547-23860/t.527002/MAOD-00000.root to root://deepthought.crc.nd.edu:1094//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_527002.root >> cmd: [2021-05-18 06:40:08.147409 -0400][Debug ][Utility ] Opening file://localhost/disk/vc3-root/tmp/worker-230547-23860/t.527002/MAOD-00000.root for reading >> cmd: [2021-05-18 06:40:08.147483 -0400][Debug ][File ] [0x1068700@file://localhost/disk/vc3-root/tmp/worker-230547-23860/t.527002/MAOD-00000.root] Sending an open command >> cmd: [2021-05-18 06:40:08.148041 -0400][Debug ][File ] [0x1068700@file://localhost/disk/vc3-root/tmp/worker-230547-23860/t.527002/MAOD-00000.root] Open has returned with status [SUCCESS] >> cmd: [2021-05-18 06:40:08.148095 -0400][Debug ][File ] [0x1068700@file://localhost/disk/vc3-root/tmp/worker-230547-23860/t.527002/MAOD-00000.root] successfully opened at localhost, handle: 0x18, session id: 1 >> cmd: [2021-05-18 06:40:08.148229 -0400][Debug ][Utility ] Opening root://deepthought.crc.nd.edu:1094//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_527002.root?oss.asize=259009527 for writing >> cmd: [2021-05-18 06:40:08.148304 -0400][Debug ][File ] [0x1069bc0@root://deepthought.crc.nd.edu:1094//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_527002.root?oss.asize=259009527] Sending an open command >> cmd: [2021-05-18 06:40:08.149060 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant03.crc.nd.edu:1096 1 stream(s) >> cmd: [2021-05-18 06:40:08.149121 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 >> cmd: [2021-05-18 06:40:08.150433 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant03.crc.nd.edu:1096" to be run at: [2021-05-18 06:40:23 -0400] >> cmd: [2021-05-18 06:40:08.151662 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.11]:1096 >> cmd: [2021-05-18 06:40:08.151822 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.11]:1096 >> cmd: [2021-05-18 06:40:08.151916 -0400][Debug ][Poller ] Adding socket 0xec001880 to the poller >> cmd: [2021-05-18 06:40:08.152046 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2021-05-18 06:40:08.152178 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2021-05-18 06:40:08.152679 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 500) >> cmd: [2021-05-18 06:40:08.152854 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 500) >> cmd: [2021-05-18 06:40:08.155644 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=d8civy106.crc.nd.edu&xrd.rn=v4.8.5, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2021-05-18 06:40:08.156421 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Logged in, session: a7c60500602b0000ff000000a4c70500 >> cmd: [2021-05-18 06:40:08.156474 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10400,c:ssl,ca:ba240aa8.0|f5f0dfc2.0 >> cmd: [2021-05-18 06:40:08.156518 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2021-05-18 06:40:08.162357 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: [2021-05-18 06:40:08.230630 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending more authentication data for gsi >> cmd: [2021-05-18 06:40:08.235992 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Authenticated with gsi. >> cmd: [2021-05-18 06:40:08.236076 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096 #0] Stream 0 connected. >> cmd: [2021-05-18 06:40:09.071775 -0400][Debug ][File ] [0x1069bc0@root://deepthought.crc.nd.edu:1094//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_527002.root?oss.asize=259009527] Open has returned with status [SUCCESS] >> cmd: [2021-05-18 06:40:09.071878 -0400][Debug ][File ] [0x1069bc0@root://deepthought.crc.nd.edu:1094//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_527002.root?oss.asize=259009527] successfully opened at primeradiant03.crc.nd.edu:1096, handle: 0x0, session id: 1 >> cmd: [2021-05-18 06:40:09.072507 -0400][Debug ][File ] [0x1068700@file://localhost/disk/vc3-root/tmp/worker-230547-23860/t.527002/MAOD-00000.root] Sending a read command for handle 0x18 to localhost >> cmd: [2021-05-18 06:40:09.085785 -0400][Debug ][File ] [0x1068700@file://localhost/disk/vc3-root/tmp/worker-230547-23860/t.527002/MAOD-00000.root] Sending a read command for handle 0x18 to localhost >> cmd: [2021-05-18 06:40:09.101055 -0400][Debug ][File ] [0x1068700@file://localhost/disk/vc3-root/tmp/worker-230547-23860/t.527002/MAOD-00000.root] Sending a read command for handle 0x18 to localhost >> cmd: [2021-05-18 06:40:09.112057 -0400][Debug ][File ] [0x1068700@file://localhost/disk/vc3-root/tmp/worker-230547-23860/t.527002/MAOD-00000.root] Sending a read command for handle 0x18 to localhost >> cmd: [2021-05-18 06:40:09.122991 -0400][Debug ][File ] [0x1069bc0@root://deepthought.crc.nd.edu:1094//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_527002.root?oss.asize=259009527] Sending a write command for handle 0x0 to primeradiant03.crc.nd.edu:1096 >> cmd: [16MB/247MB][ 6%][===> ][16MB/s] [2021-05-18 06:40:09.123403 -0400][Debug ][File ] [0x1068700@file://localhost/disk/vc3-root/tmp/worker-230547-23860/t.527002/MAOD-00000.root] Sending a read command for handle 0x18 to localhost >> cmd: [2021-05-18 06:40:09.134258 -0400][Debug ][File ] [0x1069bc0@root://deepthought.crc.nd.edu:1094//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_527002.root?oss.asize=259009527] Sending a write command for handle 0x0 to primeradiant03.crc.nd.edu:1096 >> cmd: [2021-05-18 06:40:09.134457 -0400][Debug ][File ] [0x1068700@file://localhost/disk/vc3-root/tmp/worker-230547-23860/t.527002/MAOD-00000.root] Sending a read command for handle 0x18 to localhost >> cmd: [2021-05-18 06:40:09.145182 -0400][Debug ][File ] [0x1069bc0@root://deepthought.crc.nd.edu:1094//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_527002.root?oss.asize=259009527] Sending a write command for handle 0x0 to primeradiant03.crc.nd.edu:1096 >> cmd: [2021-05-18 06:40:09.145397 -0400][Debug ][File ] [0x1068700@file://localhost/disk/vc3-root/tmp/worker-230547-23860/t.527002/MAOD-00000.root] Sending a read command for handle 0x18 to localhost >> cmd: [2021-05-18 06:40:09.156208 -0400][Debug ][File ] [0x1069bc0@root://deepthought.crc.nd.edu:1094//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_527002.root?oss.asize=259009527] Sending a write command for handle 0x0 to primeradiant03.crc.nd.edu:1096 >> cmd: [2021-05-18 06:40:09.156414 -0400][Debug ][File ] [0x1068700@file://localhost/disk/vc3-root/tmp/worker-230547-23860/t.527002/MAOD-00000.root] Sending a read command for handle 0x18 to localhost >> cmd: [2021-05-18 06:42:09.074149 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2021-05-18 06:42:09.074304 -0400][Debug ][Poller ] <[::ffff:10.32.79.75]:55440><--><[::ffff:149.165.243.11]:1096> Removing socket from the poller >> cmd: [2021-05-18 06:42:09.074472 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096 #0] Recovering error for stream #0: [ERROR] Socket error. >> cmd: [2021-05-18 06:42:09.074526 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096 #0] Reporting disconnection to queued message handlers. >> cmd: [2021-05-18 06:42:09.074578 -0400][Error ][XRootD ] [primeradiant03.crc.nd.edu:1096] Impossible to send message kXR_write (handle: 0x00000000, offset: 0, size: 16777216). Trying to recover. >> cmd: [2021-05-18 06:42:09.074627 -0400][Debug ][XRootD ] [primeradiant03.crc.nd.edu:1096] Handling error while processing kXR_write (handle: 0x00000000, offset: 0, size: 16777216): [ERROR] Socket error. >> cmd: [2021-05-18 06:42:09.074674 -0400][Error ][XRootD ] [primeradiant03.crc.nd.edu:1096] Unable to get the response to request kXR_write (handle: 0x00000000, offset: 0, size: 16777216) >> cmd: [2021-05-18 06:42:09.074753 -0400][Error ][XRootD ] [primeradiant03.crc.nd.edu:1096] Impossible to send message kXR_write (handle: 0x00000000, offset: 16777216, size: 16777216). Trying to recover. >> cmd: [2021-05-18 06:42:09.074804 -0400][Debug ][XRootD ] [primeradiant03.crc.nd.edu:1096] Handling error while processing kXR_write (handle: 0x00000000, offset: 16777216, size: 16777216): [ERROR] Socket error. >> cmd: [2021-05-18 06:42:09.074876 -0400][Error ][XRootD ] [primeradiant03.crc.nd.edu:1096] Unable to get the response to request kXR_write (handle: 0x00000000, offset: 16777216, size: 16777216) >> cmd: [2021-05-18 06:42:09.075004 -0400][Error ][XRootD ] [primeradiant03.crc.nd.edu:1096] Impossible to send message kXR_write (handle: 0x00000000, offset: 33554432, size: 16777216). Trying to recover. >> cmd: [2021-05-18 06:42:09.075087 -0400][Debug ][XRootD ] [primeradiant03.crc.nd.edu:1096] Handling error while processing kXR_write (handle: 0x00000000, offset: 33554432, size: 16777216): [ERROR] Socket error. >> cmd: [2021-05-18 06:42:09.075136 -0400][Error ][XRootD ] [primeradiant03.crc.nd.edu:1096] Unable to get the response to request kXR_write (handle: 0x00000000, offset: 33554432, size: 16777216) >> cmd: [2021-05-18 06:42:09.075254 -0400][Error ][XRootD ] [primeradiant03.crc.nd.edu:1096] Impossible to send message kXR_write (handle: 0x00000000, offset: 50331648, size: 16777216). Trying to recover. >> cmd: [2021-05-18 06:42:09.075300 -0400][Debug ][XRootD ] [primeradiant03.crc.nd.edu:1096] Handling error while processing kXR_write (handle: 0x00000000, offset: 50331648, size: 16777216): [ERROR] Socket error. >> cmd: [2021-05-18 06:42:09.075348 -0400][Error ][XRootD ] [primeradiant03.crc.nd.edu:1096] Unable to get the response to request kXR_write (handle: 0x00000000, offset: 50331648, size: 16777216) >> cmd: [2021-05-18 06:42:09.075477 -0400][Debug ][File ] [0x1069bc0@root://deepthought.crc.nd.edu:1094//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_527002.root?oss.asize=259009527] Running the recovery procedure >> cmd: [2021-05-18 06:42:09.077846 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.11]:1096 >> cmd: [2021-05-18 06:42:09.078013 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.11]:1096 >> cmd: [2021-05-18 06:42:09.078116 -0400][Debug ][Poller ] Adding socket 0xec001880 to the poller >> cmd: [2021-05-18 06:42:09.078228 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2021-05-18 06:42:09.078330 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2021-05-18 06:42:09.078729 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 500) >> cmd: [2021-05-18 06:42:09.078912 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 500) >> cmd: [2021-05-18 06:42:09.081389 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=d8civy106.crc.nd.edu&xrd.rn=v4.8.5, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2021-05-18 06:42:09.082000 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Logged in, session: 90c70500602b0000150700008dc80500 >> cmd: [2021-05-18 06:42:09.082053 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10400,c:ssl,ca:ba240aa8.0|f5f0dfc2.0 >> cmd: [2021-05-18 06:42:09.082111 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2021-05-18 06:42:09.083419 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: [2021-05-18 06:42:09.112897 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending more authentication data for gsi >> cmd: [2021-05-18 06:42:09.117567 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Authenticated with gsi. >> cmd: [2021-05-18 06:42:09.117633 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out kXR_endsess for session: a7c60500602b0000ff000000a4c70500 >> cmd: [2021-05-18 06:42:09.118108 -0400][Info ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Got wait response to kXR_endsess: session still active >> cmd: [2021-05-18 06:42:09.118164 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out kXR_endsess for session: a7c60500602b0000ff000000a4c70500 >> cmd: [2021-05-18 06:42:09.118222 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x2b95dc03b540" to be run at: [2021-05-18 06:42:13 -0400] >> cmd: [2021-05-18 06:42:13.169550 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x2b95dc03b540" >> cmd: [2021-05-18 06:42:13.170030 -0400][Info ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Got wait response to kXR_endsess: session still active >> cmd: [2021-05-18 06:42:13.170114 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out kXR_endsess for session: a7c60500602b0000ff000000a4c70500 >> cmd: [2021-05-18 06:42:13.170170 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x2b95dc02c5a0" to be run at: [2021-05-18 06:42:19 -0400] >> cmd: [2021-05-18 06:42:19.170835 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x2b95dc02c5a0" >> cmd: [2021-05-18 06:42:19.173756 -0400][Info ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Got wait response to kXR_endsess: session still active >> cmd: [2021-05-18 06:42:19.173851 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out kXR_endsess for session: a7c60500602b0000ff000000a4c70500 >> cmd: [2021-05-18 06:42:19.173913 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x2b95dc03b3d0" to be run at: [2021-05-18 06:42:27 -0400] >> cmd: [2021-05-18 06:42:27.172622 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x2b95dc03b3d0" >> cmd: [2021-05-18 06:42:27.172989 -0400][Info ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Got wait response to kXR_endsess: session still active >> cmd: [2021-05-18 06:42:27.173074 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out kXR_endsess for session: a7c60500602b0000ff000000a4c70500 >> cmd: [2021-05-18 06:42:27.173128 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x2b95dc02c5a0" to be run at: [2021-05-18 06:42:37 -0400] >> cmd: [2021-05-18 06:42:37.175007 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x2b95dc02c5a0" >> cmd: [2021-05-18 06:42:37.175699 -0400][Info ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Got wait response to kXR_endsess: session still active >> cmd: [2021-05-18 06:42:37.175785 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out kXR_endsess for session: a7c60500602b0000ff000000a4c70500 >> cmd: [2021-05-18 06:42:37.175845 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x2b95dc03b3d0" to be run at: [2021-05-18 06:42:49 -0400] >> cmd: [2021-05-18 06:42:49.177715 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x2b95dc03b3d0" >> cmd: [2021-05-18 06:42:49.178272 -0400][Info ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Got wait response to kXR_endsess: session still active >> cmd: [2021-05-18 06:42:49.178354 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out kXR_endsess for session: a7c60500602b0000ff000000a4c70500 >> cmd: [2021-05-18 06:42:49.178405 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x2b95dc02c5a0" to be run at: [2021-05-18 06:43:03 -0400] >> cmd: [2021-05-18 06:43:03.180729 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x2b95dc02c5a0" >> cmd: [2021-05-18 06:43:03.181105 -0400][Info ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Got wait response to kXR_endsess: session still active >> cmd: [2021-05-18 06:43:03.181189 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out kXR_endsess for session: a7c60500602b0000ff000000a4c70500 >> cmd: [2021-05-18 06:43:03.181245 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x2b95dc03b3d0" to be run at: [2021-05-18 06:43:19 -0400] >> cmd: [2021-05-18 06:43:19.183700 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x2b95dc03b3d0" >> cmd: [2021-05-18 06:43:19.184375 -0400][Info ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Got wait response to kXR_endsess: session still active >> cmd: [2021-05-18 06:43:19.184468 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out kXR_endsess for session: a7c60500602b0000ff000000a4c70500 >> cmd: [2021-05-18 06:43:19.184550 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x2b95dc02c5a0" to be run at: [2021-05-18 06:43:37 -0400] >> cmd: [2021-05-18 06:43:37.186761 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x2b95dc02c5a0" >> cmd: [2021-05-18 06:43:37.187209 -0400][Info ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Got wait response to kXR_endsess: session still active >> cmd: [2021-05-18 06:43:37.187304 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out kXR_endsess for session: a7c60500602b0000ff000000a4c70500 >> cmd: [2021-05-18 06:43:37.187373 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x2b95dc03b3d0" to be run at: [2021-05-18 06:43:57 -0400] >> cmd: [2021-05-18 06:43:57.190314 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x2b95dc03b3d0" >> cmd: [2021-05-18 06:43:57.190678 -0400][Info ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Got wait response to kXR_endsess: session still active >> cmd: [2021-05-18 06:43:57.190760 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out kXR_endsess for session: a7c60500602b0000ff000000a4c70500 >> cmd: [2021-05-18 06:43:57.190809 -0400][Error ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Wont retry kXR_endsess request because wouldreach connection timeout. >> cmd: [2021-05-18 06:43:57.190873 -0400][Error ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Socket error while handshaking: [ERROR] Socket timeout >> cmd: [2021-05-18 06:43:57.190921 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2021-05-18 06:43:57.190970 -0400][Debug ][Poller ] <[::ffff:10.32.79.75]:55474><--><[::ffff:149.165.243.11]:1096> Removing socket from the poller >> cmd: [2021-05-18 06:43:57.191107 -0400][Error ][PostMaster ] [primeradiant03.crc.nd.edu:1096 #0] elapsed = 108, pConnectionWindow = 120 seconds. >> cmd: [2021-05-18 06:43:57.191156 -0400][Info ][PostMaster ] [primeradiant03.crc.nd.edu:1096 #0] Attempting reconnection in 12 seconds. >> cmd: [2021-05-18 06:43:57.191204 -0400][Debug ][TaskMgr ] Registering task: "StreamConnectorTask for primeradiant03.crc.nd.edu:1096 #0" to be run at: [2021-05-18 06:44:09 -0400] >> cmd: [2021-05-18 06:44:09.195015 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.11]:1096 >> cmd: [2021-05-18 06:44:09.195261 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.11]:1096 >> cmd: [2021-05-18 06:44:09.195372 -0400][Debug ][Poller ] Adding socket 0xec001880 to the poller >> cmd: [2021-05-18 06:44:09.195524 -0400][Debug ][TaskMgr ] Done with task: "StreamConnectorTask for primeradiant03.crc.nd.edu:1096 #0" >> cmd: [2021-05-18 06:44:09.195586 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2021-05-18 06:44:09.195760 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2021-05-18 06:44:09.196222 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 500) >> cmd: [2021-05-18 06:44:09.196407 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 500) >> cmd: [2021-05-18 06:44:09.198964 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=d8civy106.crc.nd.edu&xrd.rn=v4.8.5, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2021-05-18 06:44:09.199328 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Logged in, session: fec80500602b0000ec000000fbc90500 >> cmd: [2021-05-18 06:44:09.199396 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10400,c:ssl,ca:ba240aa8.0|f5f0dfc2.0 >> cmd: [2021-05-18 06:44:09.199453 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2021-05-18 06:44:09.201675 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: [2021-05-18 06:44:09.215521 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending more authentication data for gsi >> cmd: [2021-05-18 06:44:09.221697 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Authenticated with gsi. >> cmd: [2021-05-18 06:44:09.221773 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out kXR_endsess for session: 90c70500602b0000150700008dc80500 >> cmd: [2021-05-18 06:44:09.222578 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096 #0] Stream 0 connected. >> cmd: [2021-05-18 06:44:09.223071 -0400][Debug ][XRootD ] [primeradiant03.crc.nd.edu:1096] Handling error while processing kXR_open (file: /store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_527002.root?oss.asize=259009527, mode: 0644, flags: kXR_open_updt ): [ERROR] Error response: Resource deadlock avoided. >> cmd: [2021-05-18 06:44:09.223195 -0400][Debug ][File ] [0x1069bc0@root://deepthought.crc.nd.edu:1094//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_527002.root?oss.asize=259009527] Open has returned with status [ERROR] Server responded with an error: [3003] Output file /store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_527002.root is already opened by 1 writer; open denied. >> cmd: [2021-05-18 06:44:09.223256 -0400][Debug ][File ] [0x1069bc0@root://deepthought.crc.nd.edu:1094//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_527002.root?oss.asize=259009527] Error while opening at primeradiant03.crc.nd.edu:1096: [ERROR] Server responded with an error: [3003] Output file /store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_527002.root is already opened by 1 writer; open denied. >> cmd: [2021-05-18 06:44:09.223595 -0400][Debug ][Utility ] Unable write 16777216 bytes at 0 from root://deepthought.crc.nd.edu:1094//store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_527002.root?oss.asize=259009527: [ERROR] Server responded with an error: [3003] Output file /store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_527002.root is already opened by 1 writer; open denied. >> cmd: [2021-05-18 06:44:09.225321 -0400][Debug ][File ] [0x1068700@file://localhost/disk/vc3-root/tmp/worker-230547-23860/t.527002/MAOD-00000.root] Sending a close command for handle 0x18 to localhost >> cmd: [2021-05-18 06:44:09.225537 -0400][Debug ][File ] [0x1068700@file://localhost/disk/vc3-root/tmp/worker-230547-23860/t.527002/MAOD-00000.root] Close returned from localhost with: [SUCCESS] >> cmd: [16MB/247MB][ 6%][===> ][67.98kB/s] >> cmd: Run: [ERROR] Server responded with an error: [3003] Output file /store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_527002.root is already opened by 1 writer; open denied. >> cmd: >> cmd: [2021-05-18 06:44:09.263047 -0400][Debug ][JobMgr ] Stopping the job manager... >> cmd: [2021-05-18 06:44:09.263718 -0400][Debug ][JobMgr ] Job manager stopped >> cmd: [2021-05-18 06:44:09.263772 -0400][Debug ][TaskMgr ] Stopping the task manager... >> cmd: [2021-05-18 06:44:09.264059 -0400][Debug ][TaskMgr ] Task manager stopped >> cmd: [2021-05-18 06:44:09.264112 -0400][Debug ][Poller ] Stopping the poller... >> cmd: [2021-05-18 06:44:09.264443 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" >> cmd: [2021-05-18 06:44:09.264497 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2021-05-18 06:44:09.264545 -0400][Debug ][Poller ] <[::ffff:10.32.79.75]:42488><--><[::ffff:129.74.85.55]:1094> Removing socket from the poller >> cmd: [2021-05-18 06:44:09.264634 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Destroying stream >> cmd: [2021-05-18 06:44:09.264682 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2021-05-18 06:44:09.264731 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant03.crc.nd.edu:1096" >> cmd: [2021-05-18 06:44:09.264774 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2021-05-18 06:44:09.264823 -0400][Debug ][Poller ] <[::ffff:10.32.79.75]:55498><--><[::ffff:149.165.243.11]:1096> Removing socket from the poller >> cmd: [2021-05-18 06:44:09.264905 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2021-05-18 06:44:09.264953 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Closing the socket >>> Checking output file transfer. @ Tue May 18 06:44:09 2021 >>> executing '/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/bin/gfal-copy -f file:///disk/vc3-root/tmp/worker-230547-23860/t.527002/MAOD-00000.root gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_527002.root' @ Tue May 18 06:44:09 2021 >> using /disk/vc3-root/tmp/worker-230547-23860/t.527002/cctools-temp-t.527002.lV2uGj/tmpcP60ln to store command output >> cmd: gfal-copy error: 70 (Communication error on send) - globus_xio: Unable to connect to deepthought.crc.nd.edu:2811 globus_xio: System error in connect: Connection timed out globus_xio: A system call failed: Connection timed out >> cmd: Copying 259009527 bytes file:///disk/vc3-root/tmp/worker-230547-23860/t.527002/MAOD-00000.root => gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_527002.root >>> Checking output file transfer. @ Tue May 18 07:01:49 2021 >>> executing '/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.57/el7-x86_64/usr/bin/gfal-copy -f file:///disk/vc3-root/tmp/worker-230547-23860/t.527002/MAOD-00000.root gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_527002.root' @ Tue May 18 07:01:49 2021 >> using /disk/vc3-root/tmp/worker-230547-23860/t.527002/cctools-temp-t.527002.lV2uGj/tmpA7iD4W to store command output >> cmd: gfal-copy error: 70 (Communication error on send) - globus_xio: Unable to connect to deepthought.crc.nd.edu:2811 globus_xio: System error in connect: Connection timed out globus_xio: A system call failed: Connection timed out >> cmd: Copying 259009527 bytes file:///disk/vc3-root/tmp/worker-230547-23860/t.527002/MAOD-00000.root => gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/FullProduction/FullR2/UL18/Round1/Batch2/postLHE_step/v1/mAOD_step_ttllNuNuJetNoHiggs_all22WCsStartPtCheckdim6TopMay20GST_run0/MAOD-00000_527002.root >>> Checking output file transfer. @ Tue May 18 07:08:11 2021 >> trace: Traceback (most recent call last): >> trace: File "task.py", line 268, in wrapper >> trace: result = fct(data, *args, **kwargs) >> trace: File "task.py", line 821, in copy_outputs >> trace: raise RuntimeError("no stage-out method succeeded") >> trace: RuntimeError: no stage-out method succeeded >>>>> call to 'copy_outputs' failed, exiting with exit code 210 @ Tue May 18 07:08:11 2021 Tue May 18 06:37:24 2021: ApMon[ERROR]: Error resolving cms-jobmon.cern.ch: [Errno -2] Name or service not known Tue May 18 06:37:24 2021: ApMon[WARNING]: Not sending parameters since no destination is defined. Tue May 18 06:37:24 2021: ApMon[DEBUG]: Closing UDP socket on ApMon object destroy. Tue May 18 06:39:36 2021: ApMon[ERROR]: Error resolving cms-jobmon.cern.ch: [Errno -2] Name or service not known Tue May 18 06:39:36 2021: ApMon[WARNING]: Not sending parameters since no destination is defined. Tue May 18 06:39:36 2021: ApMon[DEBUG]: Closing UDP socket on ApMon object destroy. >> Execution time 2174 >> Exiting with code 210 >> Reporting ExeExitCode 0 >> Reporting StageOutSE >> Reporting StageOutExitCode 210 Tue May 18 07:08:11 2021: ApMon[ERROR]: Error resolving cms-jobmon.cern.ch: [Errno -2] Name or service not known Tue May 18 07:08:12 2021: ApMon[WARNING]: Not sending parameters since no destination is defined. Tue May 18 07:08:12 2021: ApMon[DEBUG]: Closing UDP socket on ApMon object destroy. ==== working directory after execution @ Tue May 18 07:08:13 EDT 2021 ==== == dir: total 319808 == dir: -rwxr-xr-x 2 kmohrman campus 1600 May 18 06:31 0052_4392_report.json == dir: -rwxr-xr-x 2 kmohrman campus 1598 May 18 06:31 0052_4395_report.json == dir: -rwxr-xr-x 2 kmohrman campus 1602 May 18 06:31 0052_4397_report.json == dir: -rwxr-xr-x 2 kmohrman campus 1600 May 18 06:31 0052_4426_report.json == dir: drwxr-xr-x 18 kmohrman campus 215 May 18 06:32 CMSSW_10_6_20 == dir: -rw-r--r-- 1 kmohrman campus 259009527 May 18 06:39 MAOD-00000.root == dir: -rwxr-xr-x 1 kmohrman campus 67481295 May 18 06:37 MAOD-00000_524426.root == dir: drwxr-xr-x 2 kmohrman campus 71 May 18 06:31 bin == dir: -rwxr-xr-x 6 kmohrman campus 627272 May 17 21:33 cctools-monitor == dir: -rw-r--r-- 1 kmohrman campus 0 May 18 06:31 cctools-monitor.summary == dir: drwxrwxrwx 2 kmohrman campus 6 May 18 07:08 cctools-temp-t.527002.lV2uGj == dir: -rw------- 1 kmohrman campus 51 May 18 06:32 cvmfschecksum.cms-ib.cern.ch == dir: -rw------- 1 kmohrman campus 51 May 18 06:31 cvmfschecksum.cms.cern.ch == dir: -rw------- 1 kmohrman campus 51 May 18 06:31 cvmfschecksum.oasis.opensciencegrid.org == dir: drwxr-xr-x 2 kmohrman campus 33 May 18 06:31 lib == dir: -rwxrwxrwx 1 kmohrman campus 146912 May 18 06:31 librmonitor_helper.so.ukIS6w == dir: -rw-r--r-- 3 kmohrman campus 868 May 18 06:31 merge_cfg.py == dir: -rw-r--r-- 1 kmohrman campus 2417 May 18 06:37 merge_cfg_mod.py == dir: -rw-r--r-- 3 kmohrman campus 808 May 18 06:31 merge_reports.py == dir: -rw-r--r-- 2 kmohrman campus 2528 May 18 06:31 parameters.json == dir: -rw------- 2 kmohrman campus 9921 May 18 06:31 proxy == dir: drwxr-xr-x 3 kmohrman campus 20 May 18 06:31 python == dir: -rw-r--r-- 1 kmohrman campus 3199 May 18 07:08 report.json == dir: -rw-r--r-- 6 kmohrman campus 570 May 17 21:33 report.json.in == dir: -rw-r--r-- 1 kmohrman campus 68474 May 18 06:39 report.xml == dir: -rw-r--r-- 1 kmohrman campus 5223 May 18 07:08 report.xml.gz == dir: -rw-r--r-- 4 kmohrman campus 549 May 17 23:48 sandbox-CMSSW_10_6_20-slc7_amd64_gcc700.tar.bz2 == dir: drwxr-xr-x 4 kmohrman campus 37 May 18 06:31 siteconf == dir: -rw-r--r-- 1 kmohrman campus 11 May 18 06:32 t_wrapper_ready == dir: -rw-r--r-- 1 kmohrman campus 11 May 18 06:31 t_wrapper_start == dir: -rwxr-xr-x 6 kmohrman campus 47217 May 17 21:33 task.py == dir: -rwxr-xr-x 6 kmohrman campus 5986 May 17 21:33 wrapper.sh === wrapper done @ Tue May 18 07:08:13 EDT 2021 === final return status = 210 @ Tue May 18 07:08:13 EDT 2021