==== wrapper started @ Sat Sep 14 20:02:54 EDT 2019 ==== == startup: hostname: d12chas574.crc.nd.edu == startup: kernel: Linux d12chas574.crc.nd.edu 3.10.0-957.el7.x86_64 #1 SMP Thu Oct 4 20:48:51 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux ==== tracing google @ Sat Sep 14 20:02:59 EDT 2019 ==== wrapper.sh: line 20: traceroute: command not found ==== environment at startup @ Sat Sep 14 20:03:04 EDT 2019 ==== == env: VC3_VERSION_SED=4.2 == env: CCTOOLS_RESOURCE_PROCESS_START=1568505784195670 == env: VC3_INSIDE_WRAPPER_SCRIPT=yes == env: VC3_ROOT_GZIP=/ == env: SHELL=/bin/sh == env: TERM=xterm-256color == env: MAKEFLAGS=-j4 == env: TMPDIR=/disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME == env: VC3_ROOT_WHICH=/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0 == env: LOBSTER_CVMFS_PROXY=http://eddie.crc.nd.edu:3128 == env: LD_PRELOAD=/disk/vc3-root/tmp/worker-230547-21579/t.730/librmonitor_helper.so.BWZ9YF == env: LOBSTER_FRONTIER_PROXY=http://eddie.crc.nd.edu:3128 == env: LOBSTER_CMSSW_VERSION=CMSSW_9_3_1 == env: VC3_VERSION_FULL_TAR=1.26.0 == env: VC3_MACHINE_OS=Linux/redhat7.4.1708 == env: VC3_VERSION_FULL_CCTOOLS_WQ_WORKER=999.999.999 == env: USER=kmohrman == env: TEMP=/disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME == env: VC3_BUILDER_PATH=/disk/vc3-builder == env: LD_LIBRARY_PATH=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64/dcap:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64/lcgdm:/afs/crc.nd.edu/user/k/kmohrman/CMSSW_Releases/CMSSW_9_4_6/biglib/slc7_amd64_gcc630:/afs/crc.nd.edu/user/k/kmohrman/CMSSW_Releases/CMSSW_9_4_6/lib/slc7_amd64_gcc630:/afs/crc.nd.edu/user/k/kmohrman/CMSSW_Releases/CMSSW_9_4_6/external/slc7_amd64_gcc630/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_4_6/biglib/slc7_amd64_gcc630:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_4_6/lib/slc7_amd64_gcc630:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_4_6/external/slc7_amd64_gcc630/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/llvm/4.0.1/lib64:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0/lib64:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0/lib:/opt/crc/usr/local/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64/dcap:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64/lcgdm:/opt/condor/RedHat7/lib/condor:/opt/condor/RedHat7/lib/condor:/.singularity.d/libs == env: VC3_VERSION_VC3_BUILDER_HANDS=1.0 == env: VC3_ROOT_MAKE=/ == env: VC3_DISTFILES=/disk/vc3-distfiles == env: WORKER_TMPDIR=/disk/vc3-root/tmp/worker-230547-21579/cache/tmp == env: VC3_VERSION_TAR=1.26 == env: ENV=/disk/vc3-root/vc3-home/.vc3_sh-eIReEh == env: VC3_VERSION_CCTOOLS_WQ_WORKER=999.999 == env: VC3_VERSION_FULL_PKG_CONFIG=0.27.1 == env: VC3_VERSION_GZIP=1.5 == env: VC3_ENV_TAG_GLOBAL=21325 == env: PATH=/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0/bin:/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/bin == env: VC3_ROOT=/disk/vc3-root == env: VC3_SH_PROFILE_ENV=/disk/vc3-root/vc3-home/.vc3_sh-eIReEh == env: VC3_ROOT_PKG_CONFIG=/ == env: PWD=/disk/vc3-root/tmp/worker-230547-21579/t.730 == env: LOBSTER_OSG_VERSION=3.4 == env: VC3_VERSION_FULL_SED=4.2.2 == env: VC3_VERSION_WHICH=2.20 == env: VC3_MACHINE_GLIBC_VERSION=2.17 == env: MODULEPATH=/afs/crc.nd.edu/x86_64_linux/Modules/modules/development_tools_and_libraries:/afs/crc.nd.edu/x86_64_linux/Modules/modules/general_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/restricted_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/system_modules:/afs/crc.nd.edu/x86_64_linux/Modules/modules/deprecated_software:/afs/crc.nd.edu/x86_64_linux/Modules/current/modulefiles == env: WORK_QUEUE_SANDBOX=/disk/vc3-root/tmp/worker-230547-21579 == env: VC3_ROOT_CCTOOLS_WQ_WORKER=/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999 == env: VC3_ROOT_SED=/ == env: VC3_MACHINE_TARGET=x86_64/redhat7.4.1708 == env: CXX=g++ == env: SHLVL=3 == env: HOME=/disk/vc3-root/vc3-home == env: VC3_VERSION_MAKE=3.82 == env: VC3_VERSION_FULL_GZIP=1.5.0 == env: DISK=1000 == env: MEMORY=1200 == env: VC3_VERSION_FULL_WHICH=2.20.0 == env: TMP=/disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME == env: VC3_INSTALL_USER_HOME=/disk/vc3-root/vc3-home == env: VC3_VERSION_FULL_VC3_BUILDER_HANDS=1.0.0 == env: VC3_VERSION_PKG_CONFIG=0.27 == env: VC3_ROOT_TAR=/ == env: VC3_SH_PROFILE_WRAPPER=/disk/vc3-root/vc3-home/.vc3_sh-eIReEh.wrapper == env: CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 == env: CC=gcc == env: CORES=1 == env: VC3_VERSION_FULL_MAKE=3.82.0 == env: CCTOOLS_RESOURCE_MONITOR_INFO=1033 == env: VC3_ROOT_VC3_BUILDER_HANDS=/disk/vc3-root/x86_64/redhat7.4.1708/vc3-builder-hands/v1.0.0 == env: VC3_MACHINE_ARCH=x86_64 == env: VC3_COMMAND_LINE=/disk/vc3-builder --var CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 --require cctools-wq-worker -- work_queue_worker -M lobster_kmohrman_EFT_ALL_genOnly.* -t 300 -C catalog.cse.nd.edu,backup-catalog.cse.nd.edu -d all -o worker.log --cores=12 --memory=16000 --disk=18864 == env: _=/bin/env ==== cpu info @ Sat Sep 14 20:03:09 EDT 2019 ==== == cpu: processor : 0 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 1199.951 == cpu: cache size : 15360 KB == cpu: physical id : 0 == cpu: siblings : 12 == cpu: core id : 0 == cpu: cpu cores : 12 == cpu: apicid : 0 == cpu: initial apicid : 0 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5000.40 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 1 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 1498.718 == cpu: cache size : 15360 KB == cpu: physical id : 0 == cpu: siblings : 12 == cpu: core id : 1 == cpu: cpu cores : 12 == cpu: apicid : 2 == cpu: initial apicid : 2 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5000.40 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 2 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 1396.026 == cpu: cache size : 15360 KB == cpu: physical id : 0 == cpu: siblings : 12 == cpu: core id : 2 == cpu: cpu cores : 12 == cpu: apicid : 4 == cpu: initial apicid : 4 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5000.40 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 3 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 1496.124 == cpu: cache size : 15360 KB == cpu: physical id : 0 == cpu: siblings : 12 == cpu: core id : 3 == cpu: cpu cores : 12 == cpu: apicid : 6 == cpu: initial apicid : 6 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5000.40 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 4 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 1491.699 == cpu: cache size : 15360 KB == cpu: physical id : 0 == cpu: siblings : 12 == cpu: core id : 4 == cpu: cpu cores : 12 == cpu: apicid : 8 == cpu: initial apicid : 8 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5000.40 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 5 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 1494.903 == cpu: cache size : 15360 KB == cpu: physical id : 0 == cpu: siblings : 12 == cpu: core id : 5 == cpu: cpu cores : 12 == cpu: apicid : 10 == cpu: initial apicid : 10 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5000.40 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 6 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 1497.650 == cpu: cache size : 15360 KB == cpu: physical id : 0 == cpu: siblings : 12 == cpu: core id : 8 == cpu: cpu cores : 12 == cpu: apicid : 16 == cpu: initial apicid : 16 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5000.40 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 7 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 1490.173 == cpu: cache size : 15360 KB == cpu: physical id : 0 == cpu: siblings : 12 == cpu: core id : 9 == cpu: cpu cores : 12 == cpu: apicid : 18 == cpu: initial apicid : 18 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5000.40 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 8 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 1595.764 == cpu: cache size : 15360 KB == cpu: physical id : 0 == cpu: siblings : 12 == cpu: core id : 10 == cpu: cpu cores : 12 == cpu: apicid : 20 == cpu: initial apicid : 20 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5000.40 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 9 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 1595.306 == cpu: cache size : 15360 KB == cpu: physical id : 0 == cpu: siblings : 12 == cpu: core id : 11 == cpu: cpu cores : 12 == cpu: apicid : 22 == cpu: initial apicid : 22 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5000.40 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 10 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 1498.565 == cpu: cache size : 15360 KB == cpu: physical id : 0 == cpu: siblings : 12 == cpu: core id : 12 == cpu: cpu cores : 12 == cpu: apicid : 24 == cpu: initial apicid : 24 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5000.40 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 11 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 1499.633 == cpu: cache size : 15360 KB == cpu: physical id : 0 == cpu: siblings : 12 == cpu: core id : 13 == cpu: cpu cores : 12 == cpu: apicid : 26 == cpu: initial apicid : 26 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5000.40 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 12 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 1392.211 == cpu: cache size : 15360 KB == cpu: physical id : 1 == cpu: siblings : 12 == cpu: core id : 0 == cpu: cpu cores : 12 == cpu: apicid : 32 == cpu: initial apicid : 32 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5016.20 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 13 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 1398.773 == cpu: cache size : 15360 KB == cpu: physical id : 1 == cpu: siblings : 12 == cpu: core id : 1 == cpu: cpu cores : 12 == cpu: apicid : 34 == cpu: initial apicid : 34 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5016.20 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 14 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 1495.819 == cpu: cache size : 15360 KB == cpu: physical id : 1 == cpu: siblings : 12 == cpu: core id : 2 == cpu: cpu cores : 12 == cpu: apicid : 36 == cpu: initial apicid : 36 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5016.20 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 15 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 1399.993 == cpu: cache size : 15360 KB == cpu: physical id : 1 == cpu: siblings : 12 == cpu: core id : 3 == cpu: cpu cores : 12 == cpu: apicid : 38 == cpu: initial apicid : 38 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5016.20 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 16 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 1595.306 == cpu: cache size : 15360 KB == cpu: physical id : 1 == cpu: siblings : 12 == cpu: core id : 4 == cpu: cpu cores : 12 == cpu: apicid : 40 == cpu: initial apicid : 40 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5016.20 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 17 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 1397.857 == cpu: cache size : 15360 KB == cpu: physical id : 1 == cpu: siblings : 12 == cpu: core id : 5 == cpu: cpu cores : 12 == cpu: apicid : 42 == cpu: initial apicid : 42 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5016.20 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 18 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 1498.718 == cpu: cache size : 15360 KB == cpu: physical id : 1 == cpu: siblings : 12 == cpu: core id : 8 == cpu: cpu cores : 12 == cpu: apicid : 48 == cpu: initial apicid : 48 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5016.20 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 19 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 1399.993 == cpu: cache size : 15360 KB == cpu: physical id : 1 == cpu: siblings : 12 == cpu: core id : 9 == cpu: cpu cores : 12 == cpu: apicid : 50 == cpu: initial apicid : 50 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5016.20 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 20 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 1595.001 == cpu: cache size : 15360 KB == cpu: physical id : 1 == cpu: siblings : 12 == cpu: core id : 10 == cpu: cpu cores : 12 == cpu: apicid : 52 == cpu: initial apicid : 52 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5016.20 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 21 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 1495.361 == cpu: cache size : 15360 KB == cpu: physical id : 1 == cpu: siblings : 12 == cpu: core id : 11 == cpu: cpu cores : 12 == cpu: apicid : 54 == cpu: initial apicid : 54 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5016.20 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 22 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 1398.315 == cpu: cache size : 15360 KB == cpu: physical id : 1 == cpu: siblings : 12 == cpu: core id : 12 == cpu: cpu cores : 12 == cpu: apicid : 56 == cpu: initial apicid : 56 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5016.20 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 23 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 1294.403 == cpu: cache size : 15360 KB == cpu: physical id : 1 == cpu: siblings : 12 == cpu: core id : 13 == cpu: cpu cores : 12 == cpu: apicid : 58 == cpu: initial apicid : 58 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5016.20 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: === using CVMFS proxy: http://eddie.crc.nd.edu:3128 @ Sat Sep 14 20:03:34 EDT 2019 === using Frontier proxy: http://eddie.crc.nd.edu:3128 @ Sat Sep 14 20:03:34 EDT 2019 === parrot helper: /disk/vc3-root/tmp/worker-230547-21579/t.730/lib/libparrot_helper.so @ Sat Sep 14 20:03:39 EDT 2019 ==== content of /disk/vc3-root/tmp/worker-230547-21579/cache/tmp @ Sat Sep 14 20:03:39 EDT 2019 ==== == cache: total 0 === OSG certificate location: /cvmfs/oasis.opensciencegrid.org/mis/certificates @ Sat Sep 14 20:03:44 EDT 2019 === testing parrot usage @ Sat Sep 14 20:03:44 EDT 2019 === parrot OK @ Sat Sep 14 20:03:49 EDT 2019 === starting parrot to access CMSSW... @ Sat Sep 14 20:03:49 EDT 2019 LibCvmfs version 2.4, revision 25 ==== wrapper started @ Sat Sep 14 20:03:50 EDT 2019 ==== == startup: hostname: d12chas574.crc.nd.edu == startup: kernel: Linux d12chas574.crc.nd.edu 3.10.0-957.el7.x86_64 #1 SMP Thu Oct 4 20:48:51 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux ==== tracing google @ Sat Sep 14 20:03:50 EDT 2019 ==== wrapper.sh: line 20: traceroute: command not found ==== environment at startup @ Sat Sep 14 20:03:50 EDT 2019 ==== == env: VC3_VERSION_SED=4.2 == env: CCTOOLS_RESOURCE_PROCESS_START=1568505830520193 == env: VC3_INSIDE_WRAPPER_SCRIPT=yes == env: PARROT_HTTP_PROXY=http://eddie.crc.nd.edu:3128 == env: TERM=xterm-256color == env: SHELL=/bin/sh == env: VC3_ROOT_GZIP=/ == env: LOBSTER_GFAL_COPY= == env: MAKEFLAGS=-j4 == env: PARROT_HELPER=/disk/vc3-root/tmp/worker-230547-21579/t.730/lib/libparrot_helper.so == env: TMPDIR=/disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME == env: VC3_ROOT_WHICH=/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0 == env: LD_PRELOAD=/disk/vc3-root/tmp/worker-230547-21579/t.730/librmonitor_helper.so.BWZ9YF == env: LOBSTER_CVMFS_PROXY=http://eddie.crc.nd.edu:3128 == env: PARROT_PATH=./bin == env: LOBSTER_FRONTIER_PROXY=http://eddie.crc.nd.edu:3128 == env: LOBSTER_CMSSW_VERSION=CMSSW_9_3_1 == env: VC3_MACHINE_OS=Linux/redhat7.4.1708 == env: VC3_VERSION_FULL_TAR=1.26.0 == env: USER=kmohrman == env: VC3_VERSION_FULL_CCTOOLS_WQ_WORKER=999.999.999 == env: LD_LIBRARY_PATH=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64/dcap:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64/lcgdm:/afs/crc.nd.edu/user/k/kmohrman/CMSSW_Releases/CMSSW_9_4_6/biglib/slc7_amd64_gcc630:/afs/crc.nd.edu/user/k/kmohrman/CMSSW_Releases/CMSSW_9_4_6/lib/slc7_amd64_gcc630:/afs/crc.nd.edu/user/k/kmohrman/CMSSW_Releases/CMSSW_9_4_6/external/slc7_amd64_gcc630/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_4_6/biglib/slc7_amd64_gcc630:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_4_6/lib/slc7_amd64_gcc630:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_4_6/external/slc7_amd64_gcc630/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/llvm/4.0.1/lib64:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0/lib64:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0/lib:/opt/crc/usr/local/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64/dcap:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64/lcgdm:/opt/condor/RedHat7/lib/condor:/opt/condor/RedHat7/lib/condor:/.singularity.d/libs:lib == env: VC3_BUILDER_PATH=/disk/vc3-builder == env: TEMP=/disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME == env: VC3_VERSION_VC3_BUILDER_HANDS=1.0 == env: VC3_DISTFILES=/disk/vc3-distfiles == env: VC3_ROOT_MAKE=/ == env: ENV=/disk/vc3-root/vc3-home/.vc3_sh-eIReEh == env: VC3_VERSION_TAR=1.26 == env: WORKER_TMPDIR=/disk/vc3-root/tmp/worker-230547-21579/cache/tmp == env: FRONTIER_PROXY=http://eddie.crc.nd.edu:3128 == env: PARROT_ALLOW_SWITCHING_CVMFS_REPOSITORIES=TRUE == env: VC3_VERSION_CCTOOLS_WQ_WORKER=999.999 == env: VC3_VERSION_FULL_PKG_CONFIG=0.27.1 == env: PARROT_CVMFS_REPO=*:try_local_filesystem == env: *.cern.ch:pubkey=,url=http://cvmfs.fnal.gov:8000/opt/*;http://cvmfs.racf.bnl.gov:8000/opt/* == env: *.opensciencegrid.org:pubkey=,url=http://oasis-replica.opensciencegrid.org:8000/cvmfs/*;http://cvmfs.fnal.gov:8000/cvmfs/*;http://cvmfs.racf.bnl.gov:8000/cvmfs/* == env: VC3_VERSION_GZIP=1.5 == env: PATH=/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0/bin:/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/bin == env: VC3_ENV_TAG_GLOBAL=21325 == env: VC3_ROOT=/disk/vc3-root == env: VC3_SH_PROFILE_ENV=/disk/vc3-root/vc3-home/.vc3_sh-eIReEh == env: PWD=/disk/vc3-root/tmp/worker-230547-21579/t.730 == env: VC3_ROOT_PKG_CONFIG=/ == env: PARROT_ENABLED=TRUE == env: VC3_VERSION_FULL_SED=4.2.2 == env: LOBSTER_OSG_VERSION=3.4 == env: VC3_VERSION_WHICH=2.20 == env: MODULEPATH=/afs/crc.nd.edu/x86_64_linux/Modules/modules/development_tools_and_libraries:/afs/crc.nd.edu/x86_64_linux/Modules/modules/general_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/restricted_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/system_modules:/afs/crc.nd.edu/x86_64_linux/Modules/modules/deprecated_software:/afs/crc.nd.edu/x86_64_linux/Modules/current/modulefiles == env: VC3_MACHINE_GLIBC_VERSION=2.17 == env: LOBSTER_LCG_CP= == env: WORK_QUEUE_SANDBOX=/disk/vc3-root/tmp/worker-230547-21579 == env: VC3_ROOT_CCTOOLS_WQ_WORKER=/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999 == env: VC3_MACHINE_TARGET=x86_64/redhat7.4.1708 == env: VC3_ROOT_SED=/ == env: OASIS_CERTIFICATES=/cvmfs/oasis.opensciencegrid.org/mis/certificates == env: CXX=g++ == env: HOME=/disk/vc3-root/vc3-home == env: SHLVL=3 == env: VC3_VERSION_FULL_GZIP=1.5.0 == env: VC3_VERSION_MAKE=3.82 == env: DISK=1000 == env: PARROT_CACHE=/disk/vc3-root/tmp/worker-230547-21579/cache/tmp == env: PARROT_CVMFS_ALIEN_CACHE=/disk/vc3-root/tmp/worker-230547-21579/cache/tmp/cvmfs == env: VC3_VERSION_FULL_WHICH=2.20.0 == env: MEMORY=1200 == env: HTTP_PROXY=http://eddie.crc.nd.edu:3128 == env: TMP=/disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME == env: PYTHONPATH=python: == env: VC3_INSTALL_USER_HOME=/disk/vc3-root/vc3-home == env: VC3_VERSION_FULL_VC3_BUILDER_HANDS=1.0.0 == env: VC3_ROOT_TAR=/ == env: VC3_VERSION_PKG_CONFIG=0.27 == env: CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 == env: VC3_SH_PROFILE_WRAPPER=/disk/vc3-root/vc3-home/.vc3_sh-eIReEh.wrapper == env: CC=gcc == env: CCTOOLS_RESOURCE_MONITOR_INFO=1033 == env: VC3_VERSION_FULL_MAKE=3.82.0 == env: CORES=1 == env: PARROT_DEBUG_FLAGS= == env: VC3_MACHINE_ARCH=x86_64 == env: VC3_ROOT_VC3_BUILDER_HANDS=/disk/vc3-root/x86_64/redhat7.4.1708/vc3-builder-hands/v1.0.0 == env: VC3_COMMAND_LINE=/disk/vc3-builder --var CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 --require cctools-wq-worker -- work_queue_worker -M lobster_kmohrman_EFT_ALL_genOnly.* -t 300 -C catalog.cse.nd.edu,backup-catalog.cse.nd.edu -d all -o worker.log --cores=12 --memory=16000 --disk=18864 == env: _=/bin/env ==== cpu info @ Sat Sep 14 20:03:51 EDT 2019 ==== == cpu: processor : 0 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 2900.085 == cpu: cache size : 15360 KB == cpu: physical id : 0 == cpu: siblings : 12 == cpu: core id : 0 == cpu: cpu cores : 12 == cpu: apicid : 0 == cpu: initial apicid : 0 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5000.40 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 1 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 2899.932 == cpu: cache size : 15360 KB == cpu: physical id : 0 == cpu: siblings : 12 == cpu: core id : 1 == cpu: cpu cores : 12 == cpu: apicid : 2 == cpu: initial apicid : 2 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5000.40 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 2 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 2900.085 == cpu: cache size : 15360 KB == cpu: physical id : 0 == cpu: siblings : 12 == cpu: core id : 2 == cpu: cpu cores : 12 == cpu: apicid : 4 == cpu: initial apicid : 4 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5000.40 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 3 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 2900.085 == cpu: cache size : 15360 KB == cpu: physical id : 0 == cpu: siblings : 12 == cpu: core id : 3 == cpu: cpu cores : 12 == cpu: apicid : 6 == cpu: initial apicid : 6 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5000.40 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 4 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 2900.085 == cpu: cache size : 15360 KB == cpu: physical id : 0 == cpu: siblings : 12 == cpu: core id : 4 == cpu: cpu cores : 12 == cpu: apicid : 8 == cpu: initial apicid : 8 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5000.40 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 5 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 2900.085 == cpu: cache size : 15360 KB == cpu: physical id : 0 == cpu: siblings : 12 == cpu: core id : 5 == cpu: cpu cores : 12 == cpu: apicid : 10 == cpu: initial apicid : 10 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5000.40 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 6 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 2899.932 == cpu: cache size : 15360 KB == cpu: physical id : 0 == cpu: siblings : 12 == cpu: core id : 8 == cpu: cpu cores : 12 == cpu: apicid : 16 == cpu: initial apicid : 16 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5000.40 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 7 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 2899.932 == cpu: cache size : 15360 KB == cpu: physical id : 0 == cpu: siblings : 12 == cpu: core id : 9 == cpu: cpu cores : 12 == cpu: apicid : 18 == cpu: initial apicid : 18 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5000.40 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 8 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 2899.932 == cpu: cache size : 15360 KB == cpu: physical id : 0 == cpu: siblings : 12 == cpu: core id : 10 == cpu: cpu cores : 12 == cpu: apicid : 20 == cpu: initial apicid : 20 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5000.40 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 9 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 2899.932 == cpu: cache size : 15360 KB == cpu: physical id : 0 == cpu: siblings : 12 == cpu: core id : 11 == cpu: cpu cores : 12 == cpu: apicid : 22 == cpu: initial apicid : 22 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5000.40 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 10 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 2899.932 == cpu: cache size : 15360 KB == cpu: physical id : 0 == cpu: siblings : 12 == cpu: core id : 12 == cpu: cpu cores : 12 == cpu: apicid : 24 == cpu: initial apicid : 24 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5000.40 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 11 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 2899.932 == cpu: cache size : 15360 KB == cpu: physical id : 0 == cpu: siblings : 12 == cpu: core id : 13 == cpu: cpu cores : 12 == cpu: apicid : 26 == cpu: initial apicid : 26 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5000.40 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 12 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 2899.932 == cpu: cache size : 15360 KB == cpu: physical id : 1 == cpu: siblings : 12 == cpu: core id : 0 == cpu: cpu cores : 12 == cpu: apicid : 32 == cpu: initial apicid : 32 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5016.20 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 13 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 2900.085 == cpu: cache size : 15360 KB == cpu: physical id : 1 == cpu: siblings : 12 == cpu: core id : 1 == cpu: cpu cores : 12 == cpu: apicid : 34 == cpu: initial apicid : 34 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5016.20 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 14 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 2899.932 == cpu: cache size : 15360 KB == cpu: physical id : 1 == cpu: siblings : 12 == cpu: core id : 2 == cpu: cpu cores : 12 == cpu: apicid : 36 == cpu: initial apicid : 36 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5016.20 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 15 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 2900.085 == cpu: cache size : 15360 KB == cpu: physical id : 1 == cpu: siblings : 12 == cpu: core id : 3 == cpu: cpu cores : 12 == cpu: apicid : 38 == cpu: initial apicid : 38 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5016.20 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 16 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 2899.932 == cpu: cache size : 15360 KB == cpu: physical id : 1 == cpu: siblings : 12 == cpu: core id : 4 == cpu: cpu cores : 12 == cpu: apicid : 40 == cpu: initial apicid : 40 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5016.20 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 17 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 2899.932 == cpu: cache size : 15360 KB == cpu: physical id : 1 == cpu: siblings : 12 == cpu: core id : 5 == cpu: cpu cores : 12 == cpu: apicid : 42 == cpu: initial apicid : 42 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5016.20 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 18 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 2899.932 == cpu: cache size : 15360 KB == cpu: physical id : 1 == cpu: siblings : 12 == cpu: core id : 8 == cpu: cpu cores : 12 == cpu: apicid : 48 == cpu: initial apicid : 48 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5016.20 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 19 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 2899.932 == cpu: cache size : 15360 KB == cpu: physical id : 1 == cpu: siblings : 12 == cpu: core id : 9 == cpu: cpu cores : 12 == cpu: apicid : 50 == cpu: initial apicid : 50 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5016.20 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 20 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 2899.932 == cpu: cache size : 15360 KB == cpu: physical id : 1 == cpu: siblings : 12 == cpu: core id : 10 == cpu: cpu cores : 12 == cpu: apicid : 52 == cpu: initial apicid : 52 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5016.20 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 21 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 2899.932 == cpu: cache size : 15360 KB == cpu: physical id : 1 == cpu: siblings : 12 == cpu: core id : 11 == cpu: cpu cores : 12 == cpu: apicid : 54 == cpu: initial apicid : 54 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5016.20 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 22 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 2900.085 == cpu: cache size : 15360 KB == cpu: physical id : 1 == cpu: siblings : 12 == cpu: core id : 12 == cpu: cpu cores : 12 == cpu: apicid : 56 == cpu: initial apicid : 56 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5016.20 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: == cpu: processor : 23 == cpu: vendor_id : GenuineIntel == cpu: cpu family : 6 == cpu: model : 63 == cpu: model name : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz == cpu: stepping : 2 == cpu: microcode : 0x41 == cpu: cpu MHz : 2900.085 == cpu: cache size : 15360 KB == cpu: physical id : 1 == cpu: siblings : 12 == cpu: core id : 13 == cpu: cpu cores : 12 == cpu: apicid : 58 == cpu: initial apicid : 58 == cpu: fpu : yes == cpu: fpu_exception : yes == cpu: cpuid level : 15 == cpu: wp : yes == cpu: flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d == cpu: bogomips : 5016.20 == cpu: clflush size : 64 == cpu: cache_alignment : 64 == cpu: address sizes : 46 bits physical, 48 bits virtual == cpu: power management: == cpu: === using parrot @ Sat Sep 14 20:03:53 EDT 2019 === sourcing CMS setup @ Sat Sep 14 20:03:53 EDT 2019 === sourcing OSG setup @ Sat Sep 14 20:03:53 EDT 2019 ==== environment after sourcing startup scripts @ Sat Sep 14 20:03:53 EDT 2019 ==== == env: VC3_VERSION_SED=4.2 == env: MANPATH=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/share/man:/cvmfs/cms.cern.ch/share/man: == env: CCTOOLS_RESOURCE_PROCESS_START=1568505833833420 == env: VC3_INSIDE_WRAPPER_SCRIPT=yes == env: GFAL_PLUGIN_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64/gfal2-plugins/ == env: PARROT_HTTP_PROXY=http://eddie.crc.nd.edu:3128 == env: TERM=xterm-256color == env: SHELL=/bin/sh == env: VC3_ROOT_GZIP=/ == env: LOBSTER_GFAL_COPY=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/bin/gfal-copy == env: VOMS_USERCONF=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/etc/vomses == env: MAKEFLAGS=-j4 == env: GLOBUS_LOCATION=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr == env: TMPDIR=/disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME == env: PERL5LIB=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/share/perl5/vendor_perl:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/share/perl5:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64/perl5/vendor_perl:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64/perl5 == env: VC3_ROOT_WHICH=/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0 == env: CVSROOT=:gserver:cmssw.cvs.cern.ch:/local/reps/CMSSW == env: LD_PRELOAD=/disk/vc3-root/tmp/worker-230547-21579/t.730/librmonitor_helper.so.BWZ9YF == env: LOBSTER_CVMFS_PROXY=http://eddie.crc.nd.edu:3128 == env: PARROT_PATH=./bin == env: LOBSTER_FRONTIER_PROXY=http://eddie.crc.nd.edu:3128 == env: X509_CERT_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/etc/grid-security/certificates == env: SCRAM_ARCH=slc7_amd64_gcc493 == env: LOBSTER_CMSSW_VERSION=CMSSW_9_3_1 == env: VC3_MACHINE_OS=Linux/redhat7.4.1708 == env: VC3_VERSION_FULL_TAR=1.26.0 == env: USER=kmohrman == env: VC3_VERSION_FULL_CCTOOLS_WQ_WORKER=999.999.999 == env: LD_LIBRARY_PATH=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64/dcap:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64/lcgdm:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64/dcap:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64/lcgdm:/afs/crc.nd.edu/user/k/kmohrman/CMSSW_Releases/CMSSW_9_4_6/biglib/slc7_amd64_gcc630:/afs/crc.nd.edu/user/k/kmohrman/CMSSW_Releases/CMSSW_9_4_6/lib/slc7_amd64_gcc630:/afs/crc.nd.edu/user/k/kmohrman/CMSSW_Releases/CMSSW_9_4_6/external/slc7_amd64_gcc630/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_4_6/biglib/slc7_amd64_gcc630:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_4_6/lib/slc7_amd64_gcc630:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_4_6/external/slc7_amd64_gcc630/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/llvm/4.0.1/lib64:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0/lib64:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0/lib:/opt/crc/usr/local/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64/dcap:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64/lcgdm:/opt/condor/RedHat7/lib/condor:/opt/condor/RedHat7/lib/condor:/.singularity.d/libs:lib:lib == env: VC3_BUILDER_PATH=/disk/vc3-builder == env: TEMP=/disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME == env: VC3_VERSION_VC3_BUILDER_HANDS=1.0 == env: VC3_DISTFILES=/disk/vc3-distfiles == env: VC3_ROOT_MAKE=/ == env: OSG_LOCATION=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64 == env: ENV=/disk/vc3-root/vc3-home/.vc3_sh-eIReEh == env: VC3_VERSION_TAR=1.26 == env: WORKER_TMPDIR=/disk/vc3-root/tmp/worker-230547-21579/cache/tmp == env: FRONTIER_PROXY=http://eddie.crc.nd.edu:3128 == env: PARROT_ALLOW_SWITCHING_CVMFS_REPOSITORIES=TRUE == env: VC3_VERSION_CCTOOLS_WQ_WORKER=999.999 == env: VC3_VERSION_FULL_PKG_CONFIG=0.27.1 == env: PARROT_CVMFS_REPO=*:try_local_filesystem == env: *.cern.ch:pubkey=,url=http://cvmfs.fnal.gov:8000/opt/*;http://cvmfs.racf.bnl.gov:8000/opt/* == env: *.opensciencegrid.org:pubkey=,url=http://oasis-replica.opensciencegrid.org:8000/cvmfs/*;http://cvmfs.fnal.gov:8000/cvmfs/*;http://cvmfs.racf.bnl.gov:8000/cvmfs/* == env: VC3_VERSION_GZIP=1.5 == env: PATH=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/bin:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/sbin:/cvmfs/cms.cern.ch/common:/cvmfs/cms.cern.ch/bin:/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0/bin:/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/bin == env: VC3_ENV_TAG_GLOBAL=21325 == env: CMS_PATH=/cvmfs/cms.cern.ch == env: VC3_ROOT=/disk/vc3-root == env: VC3_SH_PROFILE_ENV=/disk/vc3-root/vc3-home/.vc3_sh-eIReEh == env: PWD=/disk/vc3-root/tmp/worker-230547-21579/t.730 == env: VC3_ROOT_PKG_CONFIG=/ == env: PARROT_ENABLED=TRUE == env: VC3_VERSION_FULL_SED=4.2.2 == env: LOBSTER_OSG_VERSION=3.4 == env: VC3_VERSION_WHICH=2.20 == env: MODULEPATH=/afs/crc.nd.edu/x86_64_linux/Modules/modules/development_tools_and_libraries:/afs/crc.nd.edu/x86_64_linux/Modules/modules/general_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/restricted_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/system_modules:/afs/crc.nd.edu/x86_64_linux/Modules/modules/deprecated_software:/afs/crc.nd.edu/x86_64_linux/Modules/current/modulefiles == env: VC3_MACHINE_GLIBC_VERSION=2.17 == env: X509_VOMS_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/etc/grid-security/vomsdir == env: LOBSTER_LCG_CP= == env: WORK_QUEUE_SANDBOX=/disk/vc3-root/tmp/worker-230547-21579 == env: VC3_ROOT_CCTOOLS_WQ_WORKER=/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999 == env: VC3_MACHINE_TARGET=x86_64/redhat7.4.1708 == env: VC3_ROOT_SED=/ == env: OASIS_CERTIFICATES=/cvmfs/oasis.opensciencegrid.org/mis/certificates == env: CXX=g++ == env: HOME=/disk/vc3-root/vc3-home == env: SHLVL=3 == env: VC3_VERSION_FULL_GZIP=1.5.0 == env: VC3_VERSION_MAKE=3.82 == env: DISK=1000 == env: PARROT_CACHE=/disk/vc3-root/tmp/worker-230547-21579/cache/tmp == env: PARROT_CVMFS_ALIEN_CACHE=/disk/vc3-root/tmp/worker-230547-21579/cache/tmp/cvmfs == env: GFAL_CONFIG_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/etc/gfal2.d/ == env: VC3_VERSION_FULL_WHICH=2.20.0 == env: MEMORY=1200 == env: HTTP_PROXY=http://eddie.crc.nd.edu:3128 == env: TMP=/disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME == env: PYTHONPATH=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib/python2.7/site-packages:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64/python2.7/site-packages:python:python: == env: VC3_INSTALL_USER_HOME=/disk/vc3-root/vc3-home == env: VC3_VERSION_FULL_VC3_BUILDER_HANDS=1.0.0 == env: VC3_ROOT_TAR=/ == env: VC3_VERSION_PKG_CONFIG=0.27 == env: CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 == env: VC3_SH_PROFILE_WRAPPER=/disk/vc3-root/vc3-home/.vc3_sh-eIReEh.wrapper == env: CC=gcc == env: CCTOOLS_RESOURCE_MONITOR_INFO=1033 == env: VC3_VERSION_FULL_MAKE=3.82.0 == env: CORES=1 == env: PARROT_DEBUG_FLAGS= == env: VC3_MACHINE_ARCH=x86_64 == env: VC3_ROOT_VC3_BUILDER_HANDS=/disk/vc3-root/x86_64/redhat7.4.1708/vc3-builder-hands/v1.0.0 == env: VC3_COMMAND_LINE=/disk/vc3-builder --var CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 --require cctools-wq-worker -- work_queue_worker -M lobster_kmohrman_EFT_ALL_genOnly.* -t 300 -C catalog.cse.nd.edu,backup-catalog.cse.nd.edu -d all -o worker.log --cores=12 --memory=16000 --disk=18864 == env: _=/bin/env ==== proxy information @ Sat Sep 14 20:03:54 EDT 2019 ==== == proxy: subject : /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=kmohrman/CN=825714/CN=Kelci Ann Mohrman/CN=419255808 == proxy: issuer : /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=kmohrman/CN=825714/CN=Kelci Ann Mohrman == proxy: identity : /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=kmohrman/CN=825714/CN=Kelci Ann Mohrman == proxy: type : RFC compliant proxy == proxy: strength : 1024 bits == proxy: path : proxy == proxy: timeleft : 189:34:31 ==== working directory at startup @ Sat Sep 14 20:03:54 EDT 2019 ==== == dir: total 860 == dir: -rw-r--r-- 13 kmohrman campus 7693 Sep 14 20:02 GEN-00000-ttlnuJets_1_qCut25-1_cfg.py == dir: drwxr-xr-x 2 kmohrman campus 71 Sep 14 20:02 bin == dir: -rwxr-xr-x 13 kmohrman campus 627272 Sep 14 20:02 cctools-monitor == dir: -rw-r--r-- 1 kmohrman campus 0 Sep 14 20:02 cctools-monitor.summary == dir: drwxrwxrwx 2 kmohrman campus 6 Sep 14 20:02 cctools-temp-t.730.ig3lME == dir: -rw------- 1 kmohrman campus 51 Sep 14 20:03 cvmfschecksum.cms.cern.ch == dir: -rw------- 1 kmohrman campus 51 Sep 14 20:03 cvmfschecksum.oasis.opensciencegrid.org == dir: drwxr-xr-x 2 kmohrman campus 33 Sep 14 20:02 lib == dir: -rwxrwxrwx 1 kmohrman campus 146912 Sep 14 20:02 librmonitor_helper.so.BWZ9YF == dir: -rw-r--r-- 2 kmohrman campus 2604 Sep 14 20:02 parameters.json == dir: -rw------- 2 kmohrman campus 9165 Sep 14 20:02 proxy == dir: drwxr-xr-x 3 kmohrman campus 20 Sep 14 20:02 python == dir: -rw-r--r-- 13 kmohrman campus 570 Sep 14 20:02 report.json.in == dir: -rw-r--r-- 13 kmohrman campus 461 Sep 14 20:02 sandbox-CMSSW_9_3_1-slc7_amd64_gcc630.tar.bz2 == dir: drwxr-xr-x 4 kmohrman campus 37 Sep 14 20:02 siteconf == dir: -rw-r--r-- 1 kmohrman campus 11 Sep 14 20:03 t_wrapper_start == dir: -rwxr-xr-x 13 kmohrman campus 47217 Sep 14 20:02 task.py == dir: -rwxr-xr-x 13 kmohrman campus 6296 Sep 14 20:02 wrapper.sh === creating new release CMSSW_9_3_1 for scram arch slc7_amd64_gcc630 @ Sat Sep 14 20:03:54 EDT 2019 Attaching cms-ib.cern.ch failed: Failed to initialize root file catalog (16) Attaching cms-ib.cern.ch failed: Failed to initialize root file catalog (16) Attaching cms-ib.cern.ch failed: Failed to initialize root file catalog (16) Attaching cms-ib.cern.ch failed: Failed to initialize root file catalog (16) WARNING: In non-interactive mode release checks e.g. deprecated releases, production architectures are disabled. === unpacking sandbox-CMSSW_9_3_1-slc7_amd64_gcc630.tar.bz2 @ Sat Sep 14 20:03:59 EDT 2019 ==== machine load @ Sat Sep 14 20:04:00 EDT 2019 ==== top: unknown option 'M' Usage: top -hv | -bcHiOSs -d secs -n max -u|U user -p pid(s) -o field -w [cols] ==== environment before execution @ Sat Sep 14 20:04:02 EDT 2019 ==== == env: G4LEVELGAMMADATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/geant4-G4PhotonEvaporation/3.2/data/PhotonEvaporation3.2 == env: UTM_XSD_DIR=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/utm/utm_0.6.5 == env: SRT_G4LEVELGAMMADATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/geant4-G4PhotonEvaporation/3.2/data/PhotonEvaporation3.2 == env: VC3_VERSION_SED=4.2 == env: CMSSW_RELEASE_BASE=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_3_1 == env: SCRAMRT_SET=CMSSW:CMSSW_9_3_1:slc7_amd64_gcc630:V2_2_9_pre04:SRT_ == env: MANPATH=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/share/man:/cvmfs/cms.cern.ch/share/man: == env: CCTOOLS_RESOURCE_PROCESS_START=1568505842056895 == env: CMSSW_FWLITE_INCLUDE_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/sherpa/2.2.4-ghjeda/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/hepmc/2.06.07/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/clhep/2.3.4.2-mlhled2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/boost/1.63.0-mlhled2/include == env: COMPILER_RUNTIME_OBJECTS=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0 == env: SRT_MALLOC_CONF_SCRAMRTDEL=lg_chunk:18,lg_dirty_mult:4 == env: VC3_INSIDE_WRAPPER_SCRIPT=yes == env: ROOT_INCLUDE_PATH=/disk/vc3-root/tmp/worker-230547-21579/t.730/CMSSW_9_3_1/src:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_3_1/src:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/coral/CORAL_2_3_21-ghjeda5/include/LCG:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/mctester/1.25.0a-ghjeda4/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/lcg/root/6.10.04-ghjeda/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/qt/4.8.7/include/QtDesigner:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/tauolapp/1.1.5-ghjeda2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/charybdis/1.003-ghjeda2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/thepeg/1.9.2p1/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/sherpa/2.2.4-ghjeda/include/SHERPA-MC:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/qt/4.8.7/include/QtOpenGL:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/qt/4.8.7/include/QtGui:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/pythia8/226-ghjeda2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/herwig/6.521-ghjeda2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/rivet/2.5.2-ghjeda4/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/qt/4.8.7/include/Qt3Support:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/lwtnn/1.0-mlhled2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/classlib/3.1.3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/lhapdf/6.2.1/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/cgal/4.2-mlhled2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/tkonlinesw/4.1.0-1-ghjeda4/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/starlight/r193-mlhled2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/qt/4.8.7/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/qt/4.8.7/include/Qt:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/qt/4.8.7/include/QtCore:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/qt/4.8.7/include/QtXml:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/mcdb/1.0.3/interface:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/libungif/4.1.4/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/libtiff/4.0.3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/libpng/1.6.16/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/geant4/10.02.p02-ghjeda2/include/Geant4:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/frontier_client/2.8.20-mlhled2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/pcre/8.37/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/boost/1.63.0-mlhled2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/xrootd/4.6.1/include/xrootd/private:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/vdt/v0.3.2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/valgrind/3.12.0/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/utm/utm_0.6.5/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/toprex/4.23/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/tbb/2017_20161004oss/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/tauola/27.121.5/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/sigcpp/2.6.2/include/sigc++-2.0:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/sqlite/3.15.1/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/protobuf/3.2.0/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/pacparser/1.3.5/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/oracle/12.1.0.2.0/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/meschach/1.2.pCMS1/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/madgraph5amcatnlo/2.4.3-ghjeda5:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/libhepml/0.2.1/interface:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/ktjet/1.06-mlhled2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/jimmy/4.2-ghjeda2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/jemalloc/4.5.0/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/xz/5.2.2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/heppdt/3.03.00/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/hector/1.3.4_patch1-ghjeda4/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gsl/2.2.1/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/libjpeg-turbo/1.3.1/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/giflib/4.2.3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/xerces-c/3.1.3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gdbm/1.10/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/freetype/2.5.3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/fftw3/3.3.2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/fftjet/1.5.0/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/fastjet/3.1.0/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/expat/2.1.0/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/hepmc/2.06.07/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/dpm/1.8.0.1/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/dcap/2.47.8/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/libxml2/2.9.1/include/libxml2:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/curl/7.51.0/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/cppunit/1.12.1/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/clhep/2.3.4.2-mlhled2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/openssl/1.0.2d/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/pythia6/426/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/photos/215.5/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/zlib-x86_64/1.2.11/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/libuuid/2.22.2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/castor/2.1.13.9/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/castor/2.1.13.9/include/shift:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/cascade/2.2.04-ghjeda2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/bz2lib/1.0.6/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/python/2.7.11-mlhled2/include/python2.7:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/ittnotify/16.06.18/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0/include/c++/6.3.0:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0/include/c++/6.3.0/x86_64-unknown-linux-gnu:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0/include/c++/6.3.0/backward:/usr/local/include:/usr/include == env: INTEL_LICENSE_FILE=28518@lxlicen01.cern.ch,28518@lxlicen02.cern.ch,28518@lxlicen03.cern.ch == env: SRT_TEXMFHOME_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/rivet/2.5.2-ghjeda4/share/Rivet/texmf == env: SRT_GEANT4_PARFULLCMS_CONFIG_DIR_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/geant4-parfullcms/2014.01.27-ghjeda2/share/ParFullCMS == env: GFAL_PLUGIN_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64/gfal2-plugins/ == env: PARROT_HTTP_PROXY=http://eddie.crc.nd.edu:3128 == env: TERM=xterm-256color == env: SHELL=/bin/sh == env: VC3_ROOT_GZIP=/ == env: LOBSTER_GFAL_COPY=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/bin/gfal-copy == env: PDFPATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/rivet/2.5.2-ghjeda4/share == env: DBSCMD_HOME=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/dbs-client/DBS_2_1_9-mlhled2/lib/DBSAPI == env: SRT_TNS_ADMIN_SCRAMRTDEL=/etc == env: VOMS_USERCONF=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/etc/vomses == env: MAKEFLAGS=-j4 == env: ROOT_GCC_TOOLCHAIN=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0 == env: GLOBUS_LOCATION=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr == env: TMPDIR=/disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME == env: G4FORCENUMBEROFTHREADS=max == env: OPENBLAS_NUM_THREADS=1 == env: SRT_PDFPATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/rivet/2.5.2-ghjeda4/share == env: PERL5LIB=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/git/1.8.3.1/share/perl5:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/share/perl5/vendor_perl:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/share/perl5:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64/perl5/vendor_perl:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64/perl5 == env: VC3_ROOT_WHICH=/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0 == env: CMSSW_DATA_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms == env: LHAPDF_DATA_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/lhapdf/6.2.1/share/LHAPDF == env: CVSROOT=:gserver:cmssw.cvs.cern.ch:/local/reps/CMSSW == env: LD_PRELOAD=/disk/vc3-root/tmp/worker-230547-21579/t.730/librmonitor_helper.so.BWZ9YF == env: LOBSTER_CVMFS_PROXY=http://eddie.crc.nd.edu:3128 == env: PARROT_PATH=./bin == env: CMSSW_GIT_HASH=CMSSW_9_3_1 == env: SRT_G4ABLADATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/geant4-G4ABLA/3.0/data/G4ABLA3.0 == env: QTDIR=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/qt/4.8.7 == env: SRT_SCONS_LIB_DIR_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/scons/1.2.0-mlhled2/lib == env: OLDPWD=/disk/vc3-root/tmp/worker-230547-21579/t.730/CMSSW_9_3_1 == env: LOBSTER_FRONTIER_PROXY=http://eddie.crc.nd.edu:3128 == env: G4NEUTRONXSDATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/geant4-G4NEUTRONXS/1.4/data/G4NEUTRONXS1.4 == env: QTINC=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/qt/4.8.7/include == env: SRT_COMPILER_RUNTIME_OBJECTS_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0 == env: X509_CERT_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/etc/grid-security/certificates == env: SCRAM_ARCH=slc7_amd64_gcc630 == env: GIT_SSL_CAINFO=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/git/1.8.3.1/share/ssl/certs/ca-bundle.crt == env: DBS_CLIENT_CONFIG=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/dbs-client/DBS_2_1_9-mlhled2/lib/DBSAPI/dbs.config == env: SRT_CMS_OPENLOOPS_PREFIX_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/openloops/1.3.1-mlhled2 == env: SRT_GIT_EXEC_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/git/1.8.3.1/libexec/git-core == env: LOBSTER_CMSSW_VERSION=CMSSW_9_3_1 == env: ROOT_TTREECACHE_SIZE=0 == env: EVTGENDATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/evtgen/1.6.0-ghjeda2/share == env: SRT_INTEL_LICENSE_FILE_SCRAMRTDEL=28518@lxlicen01.cern.ch,28518@lxlicen02.cern.ch,28518@lxlicen03.cern.ch == env: SRT_ROOT_INCLUDE_PATH_SCRAMRTDEL=/disk/vc3-root/tmp/worker-230547-21579/t.730/CMSSW_9_3_1/src:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_3_1/src:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/coral/CORAL_2_3_21-ghjeda5/include/LCG:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/mctester/1.25.0a-ghjeda4/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/lcg/root/6.10.04-ghjeda/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/qt/4.8.7/include/QtDesigner:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/tauolapp/1.1.5-ghjeda2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/charybdis/1.003-ghjeda2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/thepeg/1.9.2p1/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/sherpa/2.2.4-ghjeda/include/SHERPA-MC:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/qt/4.8.7/include/QtOpenGL:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/qt/4.8.7/include/QtGui:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/pythia8/226-ghjeda2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/herwig/6.521-ghjeda2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/rivet/2.5.2-ghjeda4/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/qt/4.8.7/include/Qt3Support:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/lwtnn/1.0-mlhled2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/classlib/3.1.3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/lhapdf/6.2.1/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/cgal/4.2-mlhled2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/tkonlinesw/4.1.0-1-ghjeda4/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/starlight/r193-mlhled2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/qt/4.8.7/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/qt/4.8.7/include/Qt:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/qt/4.8.7/include/QtCore:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/qt/4.8.7/include/QtXml:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/mcdb/1.0.3/interface:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/libungif/4.1.4/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/libtiff/4.0.3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/libpng/1.6.16/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/geant4/10.02.p02-ghjeda2/include/Geant4:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/frontier_client/2.8.20-mlhled2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/pcre/8.37/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/boost/1.63.0-mlhled2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/xrootd/4.6.1/include/xrootd/private:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/vdt/v0.3.2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/valgrind/3.12.0/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/utm/utm_0.6.5/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/toprex/4.23/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/tbb/2017_20161004oss/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/tauola/27.121.5/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/sigcpp/2.6.2/include/sigc++-2.0:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/sqlite/3.15.1/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/protobuf/3.2.0/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/pacparser/1.3.5/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/oracle/12.1.0.2.0/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/meschach/1.2.pCMS1/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/madgraph5amcatnlo/2.4.3-ghjeda5:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/libhepml/0.2.1/interface:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/ktjet/1.06-mlhled2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/jimmy/4.2-ghjeda2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/jemalloc/4.5.0/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/xz/5.2.2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/heppdt/3.03.00/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/hector/1.3.4_patch1-ghjeda4/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gsl/2.2.1/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/libjpeg-turbo/1.3.1/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/giflib/4.2.3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/xerces-c/3.1.3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gdbm/1.10/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/freetype/2.5.3/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/fftw3/3.3.2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/fftjet/1.5.0/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/fastjet/3.1.0/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/expat/2.1.0/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/hepmc/2.06.07/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/dpm/1.8.0.1/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/dcap/2.47.8/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/libxml2/2.9.1/include/libxml2:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/curl/7.51.0/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/cppunit/1.12.1/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/clhep/2.3.4.2-mlhled2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/openssl/1.0.2d/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/pythia6/426/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/photos/215.5/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/zlib-x86_64/1.2.11/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/libuuid/2.22.2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/castor/2.1.13.9/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/castor/2.1.13.9/include/shift:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/cascade/2.2.04-ghjeda2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/bz2lib/1.0.6/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/python/2.7.11-mlhled2/include/python2.7:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/ittnotify/16.06.18/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0/include/c++/6.3.0:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0/include/c++/6.3.0/x86_64-unknown-linux-gnu:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0/include/c++/6.3.0/backward:/usr/local/include:/usr/include == env: VC3_MACHINE_OS=Linux/redhat7.4.1708 == env: VC3_VERSION_FULL_TAR=1.26.0 == env: G4LEDATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/geant4-G4EMLOW/6.48/data/G4EMLOW6.48 == env: SRT_VINCIADATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/vincia/1.2.02-ghjeda2/share/Vincia/xmldoc == env: SRT_ROOT_TTREECACHE_PREFILL_SCRAMRTDEL=0 == env: OPAL_PREFIX=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/openmpi/2.1.2rc4 == env: USER=kmohrman == env: VC3_VERSION_FULL_CCTOOLS_WQ_WORKER=999.999.999 == env: ROOT_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/madgraph5amcatnlo/2.4.3-ghjeda5 == env: SHERPA_INCLUDE_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/sherpa/2.2.4-ghjeda/include/SHERPA-MC == env: SRT_LHAPDF_DATA_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/lhapdf/6.2.1/share/LHAPDF == env: LD_LIBRARY_PATH=/disk/vc3-root/tmp/worker-230547-21579/t.730/CMSSW_9_3_1/biglib/slc7_amd64_gcc630:/disk/vc3-root/tmp/worker-230547-21579/t.730/CMSSW_9_3_1/lib/slc7_amd64_gcc630:/disk/vc3-root/tmp/worker-230547-21579/t.730/CMSSW_9_3_1/external/slc7_amd64_gcc630/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_3_1/biglib/slc7_amd64_gcc630:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_3_1/lib/slc7_amd64_gcc630:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_3_1/external/slc7_amd64_gcc630/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/llvm/4.0.0-njopjo/lib64:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0/lib64:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64/dcap:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64/lcgdm:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64/dcap:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64/lcgdm:/afs/crc.nd.edu/user/k/kmohrman/CMSSW_Releases/CMSSW_9_4_6/biglib/slc7_amd64_gcc630:/afs/crc.nd.edu/user/k/kmohrman/CMSSW_Releases/CMSSW_9_4_6/lib/slc7_amd64_gcc630:/afs/crc.nd.edu/user/k/kmohrman/CMSSW_Releases/CMSSW_9_4_6/external/slc7_amd64_gcc630/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_4_6/biglib/slc7_amd64_gcc630:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_4_6/lib/slc7_amd64_gcc630:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_4_6/external/slc7_amd64_gcc630/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/llvm/4.0.1/lib64:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0/lib64:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0/lib:/opt/crc/usr/local/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64/dcap:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64/lcgdm:/opt/condor/RedHat7/lib/condor:/opt/condor/RedHat7/lib/condor:/.singularity.d/libs:lib:lib == env: VC3_BUILDER_PATH=/disk/vc3-builder == env: TEMP=/disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME == env: SRT_CMSSW_RELEASE_BASE_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_3_1 == env: SRT_RIVET_ANALYSIS_PATH_SCRAMRTDEL=/disk/vc3-root/tmp/worker-230547-21579/t.730/CMSSW_9_3_1/lib/slc7_amd64_gcc630:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_3_1/lib/slc7_amd64_gcc630:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/rivet/2.5.2-ghjeda4/lib == env: VC3_VERSION_VC3_BUILDER_HANDS=1.0 == env: SRT_PERL5LIB_SCRAMRT=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/git/1.8.3.1/share/perl5 == env: SRT_CMSSW_VERSION_SCRAMRTDEL=CMSSW_9_3_1 == env: VC3_DISTFILES=/disk/vc3-distfiles == env: VC3_ROOT_MAKE=/ == env: VALGRIND_LIB=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/valgrind/3.12.0/lib/valgrind == env: SRT_CMSSW_GIT_HASH_SCRAMRTDEL=CMSSW_9_3_1 == env: SRT_ROOTSYS_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/lcg/root/6.10.04-ghjeda/ == env: OSG_LOCATION=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64 == env: ENV=/disk/vc3-root/vc3-home/.vc3_sh-eIReEh == env: VC3_VERSION_TAR=1.26 == env: WORKER_TMPDIR=/disk/vc3-root/tmp/worker-230547-21579/cache/tmp == env: G4NEUTRONHPDATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/geant4-G4NDL/4.5/data/G4NDL4.5 == env: GEANT4_PARFULLCMS_CONFIG_DIR=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/geant4-parfullcms/2014.01.27-ghjeda2/share/ParFullCMS == env: FRONTIER_PROXY=http://eddie.crc.nd.edu:3128 == env: PARROT_ALLOW_SWITCHING_CVMFS_REPOSITORIES=TRUE == env: CSC_TRACK_FINDER_DATA_DIR=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/CSCTrackFinderEmulation/1.2/data/ == env: SRT_QTINC_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/qt/4.8.7/include == env: VC3_VERSION_CCTOOLS_WQ_WORKER=999.999 == env: G4ENSDFSTATEDATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/geant4-G4ENSDFSTATE/1.2.3/data/G4ENSDFSTATE1.2.3 == env: G4RADIOACTIVEDATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/geant4-G4RadioactiveDecay/4.3.2/data/RadioactiveDecay4.3.2 == env: ROOFITSYS=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/lcg/root/6.10.04-ghjeda/ == env: SRT_ROOFITSYS_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/lcg/root/6.10.04-ghjeda/ == env: SRT_G4RADIOACTIVEDATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/geant4-G4RadioactiveDecay/4.3.2/data/RadioactiveDecay4.3.2 == env: SRT_G4ENSDFSTATEDATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/geant4-G4ENSDFSTATE/1.2.3/data/G4ENSDFSTATE1.2.3 == env: SRT_CLHEP_PARAM_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/clhep/2.3.4.2-mlhled2 == env: G4ABLADATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/geant4-G4ABLA/3.0/data/G4ABLA3.0 == env: VC3_VERSION_FULL_PKG_CONFIG=0.27.1 == env: SRT_SHERPA_INCLUDE_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/sherpa/2.2.4-ghjeda/include/SHERPA-MC == env: SRT_G4FORCENUMBEROFTHREADS_SCRAMRTDEL=max == env: SRT_WORKER_DATA_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/blackhat/0.9.9-mlhled2/share/blackhat/datafiles/ == env: PARROT_CVMFS_REPO=*:try_local_filesystem == env: *.cern.ch:pubkey=,url=http://cvmfs.fnal.gov:8000/opt/*;http://cvmfs.racf.bnl.gov:8000/opt/* == env: *.opensciencegrid.org:pubkey=,url=http://oasis-replica.opensciencegrid.org:8000/cvmfs/*;http://cvmfs.fnal.gov:8000/cvmfs/*;http://cvmfs.racf.bnl.gov:8000/cvmfs/* == env: VC3_VERSION_GZIP=1.5 == env: TNS_ADMIN=/etc == env: SRT_HEPPDT_PARAM_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/heppdt/3.03.00 == env: PATH=/cvmfs/cms.cern.ch/share/overrides/bin:/disk/vc3-root/tmp/worker-230547-21579/t.730/CMSSW_9_3_1/bin/slc7_amd64_gcc630:/disk/vc3-root/tmp/worker-230547-21579/t.730/CMSSW_9_3_1/external/slc7_amd64_gcc630/bin:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_3_1/bin/slc7_amd64_gcc630:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_3_1/external/slc7_amd64_gcc630/bin:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/llvm/4.0.0-njopjo/bin:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0/bin:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/bin:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/sbin:/cvmfs/cms.cern.ch/common:/cvmfs/cms.cern.ch/bin:/disk/vc3-root/x86_64/redhat7.4.1708/which/v2.20.0/bin:/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/bin == env: VC3_ENV_TAG_GLOBAL=21325 == env: SRT_DBS_CLIENT_CONFIG_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/dbs-client/DBS_2_1_9-mlhled2/lib/DBSAPI/dbs.config == env: SRT_CMSSW_SEARCH_PATH_SCRAMRTDEL=/disk/vc3-root/tmp/worker-230547-21579/t.730/CMSSW_9_3_1/src:/disk/vc3-root/tmp/worker-230547-21579/t.730/CMSSW_9_3_1/external/slc7_amd64_gcc630/data:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_3_1/src:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_3_1/external/slc7_amd64_gcc630/data == env: SRT_GIT_SSL_CAINFO_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/git/1.8.3.1/share/ssl/certs/ca-bundle.crt == env: CMS_PATH=/cvmfs/cms.cern.ch == env: VC3_ROOT=/disk/vc3-root == env: SRT_UTM_XSD_DIR_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/utm/utm_0.6.5 == env: SRT_PATH_SCRAMRT=/cvmfs/cms.cern.ch/share/overrides/bin:/disk/vc3-root/tmp/worker-230547-21579/t.730/CMSSW_9_3_1/bin/slc7_amd64_gcc630:/disk/vc3-root/tmp/worker-230547-21579/t.730/CMSSW_9_3_1/external/slc7_amd64_gcc630/bin:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_3_1/bin/slc7_amd64_gcc630:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_3_1/external/slc7_amd64_gcc630/bin:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/llvm/4.0.0-njopjo/bin:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0/bin == env: VC3_SH_PROFILE_ENV=/disk/vc3-root/vc3-home/.vc3_sh-eIReEh == env: RIVET_ANALYSIS_PATH=/disk/vc3-root/tmp/worker-230547-21579/t.730/CMSSW_9_3_1/lib/slc7_amd64_gcc630:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_3_1/lib/slc7_amd64_gcc630:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/rivet/2.5.2-ghjeda4/lib == env: SRT_LOCALRT_SCRAMRTDEL=/disk/vc3-root/tmp/worker-230547-21579/t.730/CMSSW_9_3_1 == env: SRT_CMSSW_BASE_SCRAMRTDEL=/disk/vc3-root/tmp/worker-230547-21579/t.730/CMSSW_9_3_1 == env: PWD=/disk/vc3-root/tmp/worker-230547-21579/t.730 == env: VC3_ROOT_PKG_CONFIG=/ == env: PARROT_ENABLED=TRUE == env: VC3_VERSION_FULL_SED=4.2.2 == env: LOBSTER_OSG_VERSION=3.4 == env: SRT_OPAL_PREFIX_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/openmpi/2.1.2rc4 == env: VC3_VERSION_WHICH=2.20 == env: LANG=C == env: SRT_QTDIR_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/qt/4.8.7 == env: MODULEPATH=/afs/crc.nd.edu/x86_64_linux/Modules/modules/development_tools_and_libraries:/afs/crc.nd.edu/x86_64_linux/Modules/modules/general_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/restricted_software:/afs/crc.nd.edu/x86_64_linux/Modules/modules/system_modules:/afs/crc.nd.edu/x86_64_linux/Modules/modules/deprecated_software:/afs/crc.nd.edu/x86_64_linux/Modules/current/modulefiles == env: VC3_MACHINE_GLIBC_VERSION=2.17 == env: SRT_G4NEUTRONHPDATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/geant4-G4NDL/4.5/data/G4NDL4.5 == env: X509_VOMS_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/etc/grid-security/vomsdir == env: LOBSTER_LCG_CP= == env: SRT_SHERPA_SHARE_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/sherpa/2.2.4-ghjeda/share/SHERPA-MC == env: WORK_QUEUE_SANDBOX=/disk/vc3-root/tmp/worker-230547-21579 == env: CASCADE_PDFPATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/cascade/2.2.04-ghjeda2/share == env: SRT_ROOT_GCC_TOOLCHAIN_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0 == env: SRT_CASCADE_PDFPATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/cascade/2.2.04-ghjeda2/share == env: WORKER_DATA_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/blackhat/0.9.9-mlhled2/share/blackhat/datafiles/ == env: GIT_TEMPLATE_DIR=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/git/1.8.3.1/share/git-core/templates == env: G4SAIDXSDATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/geant4-G4SAIDDATA/1.1/data/G4SAIDDATA1.1 == env: SRT_DBSCMD_HOME_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/dbs-client/DBS_2_1_9-mlhled2/lib/DBSAPI == env: VC3_ROOT_CCTOOLS_WQ_WORKER=/disk/vc3-root/x86_64/redhat7.4.1708/cctools-wq-worker/v999.999.999 == env: SRT_CMSSW_FWLITE_INCLUDE_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/sherpa/2.2.4-ghjeda/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/hepmc/2.06.07/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/clhep/2.3.4.2-mlhled2/include:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/boost/1.63.0-mlhled2/include == env: SRT_PYTHONPATH_SCRAMRT=/disk/vc3-root/tmp/worker-230547-21579/t.730/CMSSW_9_3_1/python:/disk/vc3-root/tmp/worker-230547-21579/t.730/CMSSW_9_3_1/lib/slc7_amd64_gcc630:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_3_1/python:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_3_1/lib/slc7_amd64_gcc630:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/coral/CORAL_2_3_21-ghjeda5/slc7_amd64_gcc630/python:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/coral/CORAL_2_3_21-ghjeda5/slc7_amd64_gcc630/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/professor2/2.2.1-ghjeda5/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/pyqt/4.11.4-mlhled2/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/sherpa/2.2.4-ghjeda/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/rivet/2.5.2-ghjeda4/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/python-ldap/2.4.10-mlhled2/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/py2-matplotlib/1.5.2-njopjo/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/sip/4.17-mlhled2/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/llvm/4.0.0-njopjo/lib64/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/py2-sqlalchemy/1.1.4-mlhled2/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/py2-lint/0.25.1-mlhled2/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/py2-dxr/1.0-ghjeda5/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/py2-cx-oracle/5.2.1-mlhled2/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/py2-numpy/1.12.1-mlhled2/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/frontier_client/2.8.20-mlhled2/python/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/das_client/v03.01.00-mlhled2/bin:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/xrootd/4.6.1/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/yoda/1.6.5-ghjeda4/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/lcg/root/6.10.04-ghjeda/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/pyminuit2/0.0.1-ghjeda4/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/py2-PyYAML/3.11-mlhled2/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/py2-pygithub/1.23.0-mlhled2/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/py2-pip/9.0.1-mlhled2/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/py2-dablooms/0.9.1-mlhled2/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/py2-pippkgs_depscipy/3.0-ghjeda5/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/py2-pippkgs/5.0-ghjeda5/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/professor/1.4.0-ghjeda4/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/dbs-client/DBS_2_1_9-mlhled2/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/dbs-client/DBS_2_1_9-mlhled2/lib/DBSAPI:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/cvs2git/5419-mlhled2/lib == env: VC3_MACHINE_TARGET=x86_64/redhat7.4.1708 == env: VC3_ROOT_SED=/ == env: OASIS_CERTIFICATES=/cvmfs/oasis.opensciencegrid.org/mis/certificates == env: ROOTSYS=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/lcg/root/6.10.04-ghjeda/ == env: SRT_PYTHIA8DATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/pythia8/226-ghjeda2/share/Pythia8/xmldoc == env: SRT_HERWIGPATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/herwigpp/2.7.1/share/Herwig++ == env: CXX=g++ == env: SRT_QTLIB_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/qt/4.8.7/lib == env: SHERPA_LIBRARY_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/sherpa/2.2.4-ghjeda/lib/SHERPA-MC == env: HOME=/disk/vc3-root/vc3-home == env: SHLVL=3 == env: CMSSW_SEARCH_PATH=/disk/vc3-root/tmp/worker-230547-21579/t.730/CMSSW_9_3_1/src:/disk/vc3-root/tmp/worker-230547-21579/t.730/CMSSW_9_3_1/external/slc7_amd64_gcc630/data:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_3_1/src:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_3_1/external/slc7_amd64_gcc630/data == env: VC3_VERSION_FULL_GZIP=1.5.0 == env: VC3_VERSION_MAKE=3.82 == env: DISK=1000 == env: PARROT_CACHE=/disk/vc3-root/tmp/worker-230547-21579/cache/tmp == env: PARROT_CVMFS_ALIEN_CACHE=/disk/vc3-root/tmp/worker-230547-21579/cache/tmp/cvmfs == env: ROOT_TTREECACHE_PREFILL=0 == env: GENREFLEX=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/lcg/root/6.10.04-ghjeda/bin/genreflex == env: VINCIADATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/vincia/1.2.02-ghjeda2/share/Vincia/xmldoc == env: SRT_G4LEDATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/geant4-G4EMLOW/6.48/data/G4EMLOW6.48 == env: SRT_GENREFLEX_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/lcg/root/6.10.04-ghjeda/bin/genreflex == env: MALLOC_CONF=lg_chunk:18,lg_dirty_mult:4 == env: SRT_ROOT_TTREECACHE_SIZE_SCRAMRTDEL=0 == env: SRT_EVTGENDATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/evtgen/1.6.0-ghjeda2/share == env: GFAL_CONFIG_DIR=/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/etc/gfal2.d/ == env: VC3_VERSION_FULL_WHICH=2.20.0 == env: MEMORY=1200 == env: HTTP_PROXY=http://eddie.crc.nd.edu:3128 == env: SRT_CMSSW_DATA_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms == env: HEPPDT_PARAM_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/heppdt/3.03.00 == env: SRT_LD_LIBRARY_PATH_SCRAMRT=/disk/vc3-root/tmp/worker-230547-21579/t.730/CMSSW_9_3_1/biglib/slc7_amd64_gcc630:/disk/vc3-root/tmp/worker-230547-21579/t.730/CMSSW_9_3_1/lib/slc7_amd64_gcc630:/disk/vc3-root/tmp/worker-230547-21579/t.730/CMSSW_9_3_1/external/slc7_amd64_gcc630/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_3_1/biglib/slc7_amd64_gcc630:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_3_1/lib/slc7_amd64_gcc630:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_3_1/external/slc7_amd64_gcc630/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/llvm/4.0.0-njopjo/lib64:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0/lib64:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0/lib == env: TMP=/disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME == env: PYTHONPATH=/disk/vc3-root/tmp/worker-230547-21579/t.730/CMSSW_9_3_1/python:/disk/vc3-root/tmp/worker-230547-21579/t.730/CMSSW_9_3_1/lib/slc7_amd64_gcc630:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_3_1/python:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_3_1/lib/slc7_amd64_gcc630:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/coral/CORAL_2_3_21-ghjeda5/slc7_amd64_gcc630/python:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/coral/CORAL_2_3_21-ghjeda5/slc7_amd64_gcc630/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/professor2/2.2.1-ghjeda5/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/pyqt/4.11.4-mlhled2/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/sherpa/2.2.4-ghjeda/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/rivet/2.5.2-ghjeda4/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/python-ldap/2.4.10-mlhled2/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/py2-matplotlib/1.5.2-njopjo/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/sip/4.17-mlhled2/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/llvm/4.0.0-njopjo/lib64/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/py2-sqlalchemy/1.1.4-mlhled2/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/py2-lint/0.25.1-mlhled2/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/py2-dxr/1.0-ghjeda5/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/py2-cx-oracle/5.2.1-mlhled2/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/py2-numpy/1.12.1-mlhled2/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/frontier_client/2.8.20-mlhled2/python/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/das_client/v03.01.00-mlhled2/bin:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/xrootd/4.6.1/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/yoda/1.6.5-ghjeda4/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/lcg/root/6.10.04-ghjeda/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/pyminuit2/0.0.1-ghjeda4/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/py2-PyYAML/3.11-mlhled2/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/py2-pygithub/1.23.0-mlhled2/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/py2-pip/9.0.1-mlhled2/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/py2-dablooms/0.9.1-mlhled2/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/py2-pippkgs_depscipy/3.0-ghjeda5/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/py2-pippkgs/5.0-ghjeda5/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/professor/1.4.0-ghjeda4/lib/python2.7/site-packages:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/dbs-client/DBS_2_1_9-mlhled2/lib:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/dbs-client/DBS_2_1_9-mlhled2/lib/DBSAPI:/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/cvs2git/5419-mlhled2/lib:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib/python2.7/site-packages:/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/lib64/python2.7/site-packages:python:python: == env: QTLIB=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/qt/4.8.7/lib == env: SCONS_LIB_DIR=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/scons/1.2.0-mlhled2/lib == env: VC3_INSTALL_USER_HOME=/disk/vc3-root/vc3-home == env: VC3_VERSION_FULL_VC3_BUILDER_HANDS=1.0.0 == env: FRONTIER_CLIENT=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/frontier_client/2.8.20-mlhled2/ == env: SRT_G4NEUTRONXSDATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/geant4-G4NEUTRONXS/1.4/data/G4NEUTRONXS1.4 == env: SRT_SHERPA_LIBRARY_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/sherpa/2.2.4-ghjeda/lib/SHERPA-MC == env: VC3_ROOT_TAR=/ == env: VC3_VERSION_PKG_CONFIG=0.27 == env: CLHEP_PARAM_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/clhep/2.3.4.2-mlhled2 == env: CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 == env: VC3_SH_PROFILE_WRAPPER=/disk/vc3-root/vc3-home/.vc3_sh-eIReEh.wrapper == env: HERWIGPATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/herwigpp/2.7.1/share/Herwig++ == env: PYTHIA8DATA=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/pythia8/226-ghjeda2/share/Pythia8/xmldoc == env: SRT_FRONTIER_CLIENT_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/frontier_client/2.8.20-mlhled2/ == env: SRT_CSC_TRACK_FINDER_DATA_DIR_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/CSCTrackFinderEmulation/1.2/data/ == env: SRT_ROOT_PATH_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/madgraph5amcatnlo/2.4.3-ghjeda5 == env: CC=gcc == env: TEXMFHOME=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/rivet/2.5.2-ghjeda4/share/Rivet/texmf == env: SRT_OPENBLAS_NUM_THREADS_SCRAMRTDEL=1 == env: SRT_GIT_TEMPLATE_DIR_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/git/1.8.3.1/share/git-core/templates == env: SRT_G4SAIDXSDATA_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/geant4-G4SAIDDATA/1.1/data/G4SAIDDATA1.1 == env: CMSSW_VERSION=CMSSW_9_3_1 == env: SRT_LANG_SCRAMRTDEL=C == env: CCTOOLS_RESOURCE_MONITOR_INFO=1033 == env: VC3_VERSION_FULL_MAKE=3.82.0 == env: CORES=1 == env: PARROT_DEBUG_FLAGS= == env: SHERPA_SHARE_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/sherpa/2.2.4-ghjeda/share/SHERPA-MC == env: VC3_MACHINE_ARCH=x86_64 == env: VC3_ROOT_VC3_BUILDER_HANDS=/disk/vc3-root/x86_64/redhat7.4.1708/vc3-builder-hands/v1.0.0 == env: CMS_OPENLOOPS_PREFIX=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/openloops/1.3.1-mlhled2 == env: GIT_EXEC_PATH=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/git/1.8.3.1/libexec/git-core == env: LOCALRT=/disk/vc3-root/tmp/worker-230547-21579/t.730/CMSSW_9_3_1 == env: CMSSW_BASE=/disk/vc3-root/tmp/worker-230547-21579/t.730/CMSSW_9_3_1 == env: SRT_VALGRIND_LIB_SCRAMRTDEL=/cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/valgrind/3.12.0/lib/valgrind == env: VC3_COMMAND_LINE=/disk/vc3-builder --var CCTOOLS_ND=lobster-171-cd5e3e2c-cvmfs-70dfa0d6 --require cctools-wq-worker -- work_queue_worker -M lobster_kmohrman_EFT_ALL_genOnly.* -t 300 -C catalog.cse.nd.edu,backup-catalog.cse.nd.edu -d all -o worker.log --cores=12 --memory=16000 --disk=18864 == env: _=/bin/env === wrapper ready @ Sat Sep 14 20:04:05 EDT 2019 ==== working directory before execution @ Sat Sep 14 20:04:05 EDT 2019 ==== == dir: total 864 == dir: drwxr-xr-x 17 kmohrman campus 201 Sep 14 20:03 CMSSW_9_3_1 == dir: -rw-r--r-- 13 kmohrman campus 7693 Sep 14 20:02 GEN-00000-ttlnuJets_1_qCut25-1_cfg.py == dir: drwxr-xr-x 2 kmohrman campus 71 Sep 14 20:02 bin == dir: -rwxr-xr-x 13 kmohrman campus 627272 Sep 14 20:02 cctools-monitor == dir: -rw-r--r-- 1 kmohrman campus 0 Sep 14 20:02 cctools-monitor.summary == dir: drwxrwxrwx 2 kmohrman campus 6 Sep 14 20:02 cctools-temp-t.730.ig3lME == dir: -rw------- 1 kmohrman campus 51 Sep 14 20:03 cvmfschecksum.cms.cern.ch == dir: -rw------- 1 kmohrman campus 51 Sep 14 20:03 cvmfschecksum.oasis.opensciencegrid.org == dir: drwxr-xr-x 2 kmohrman campus 33 Sep 14 20:02 lib == dir: -rwxrwxrwx 1 kmohrman campus 146912 Sep 14 20:02 librmonitor_helper.so.BWZ9YF == dir: -rw-r--r-- 2 kmohrman campus 2604 Sep 14 20:02 parameters.json == dir: -rw------- 2 kmohrman campus 9165 Sep 14 20:02 proxy == dir: drwxr-xr-x 3 kmohrman campus 20 Sep 14 20:02 python == dir: -rw-r--r-- 13 kmohrman campus 570 Sep 14 20:02 report.json.in == dir: -rw-r--r-- 13 kmohrman campus 461 Sep 14 20:02 sandbox-CMSSW_9_3_1-slc7_amd64_gcc630.tar.bz2 == dir: drwxr-xr-x 4 kmohrman campus 37 Sep 14 20:02 siteconf == dir: -rw-r--r-- 1 kmohrman campus 11 Sep 14 20:04 t_wrapper_ready == dir: -rw-r--r-- 1 kmohrman campus 11 Sep 14 20:03 t_wrapper_start == dir: -rwxr-xr-x 13 kmohrman campus 47217 Sep 14 20:02 task.py == dir: -rwxr-xr-x 13 kmohrman campus 6296 Sep 14 20:02 wrapper.sh Warning in : class BSTRecord found in libDataFormatsTCDS.so is already in libDataFormatsScalers.so Warning in : class edm::Wrapper found in libDataFormatsTCDS.so is already in libDataFormatsScalers.so >>> data is {u'files': {u'info': {}, u'output_info': {}, u'skipped': []}, u'cpu_time': 0, u'exe_exit_code': 0, u'events_written': 0, u'output_size': 0, u'task_exit_code': 0, u'output_storage_element': u'', u'cache': {u'start_size': 0, u'type': 2, u'end_size': 0}, u'stageout_exit_code': 0, u'output_bare_size': 0, u'transfers': defaultdict(, {}), u'task_timing': {u'wrapper_ready': 0, u'wrapper_start': 0, u'prologue_end': 0, u'epilogue_end': 0, u'stage_out_end': 0, u'stage_in_end': 0, u'processing_end': 0}, u'events_per_run': 0} @ Sat Sep 14 20:04:15 2019 >>> Trying hdfs client access method @ Sat Sep 14 20:04:15 2019 >>> executing 'timeout 300 hdfs dfs -fs hdfs://eddie.crc.nd.edu:19000 -get /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1556.root HIG-RunIIFall17wmLHE-00000ND_1556.root' @ Sat Sep 14 20:04:15 2019 >> using /disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME/tmpV1zjVX to store command output >> cmd: timeout: failed to run command 'hdfs': No such file or directory >>>>> Unable to copy input with hdfs client @ Sat Sep 14 20:04:16 2019 >>> Trying xrootd access method @ Sat Sep 14 20:04:16 2019 >>> executing 'env XRD_LOGLEVEL=Debug timeout 300 xrdfs deepthought.crc.nd.edu stat /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1556.root' @ Sat Sep 14 20:04:16 2019 >> using /disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME/tmpy4B0GA to store command output >> cmd: [2019-09-14 20:04:16.321523 -0400][Warning][Utility ] Unable to process global config file: [ERROR] OS Error: No such file or directory >> cmd: [2019-09-14 20:04:16.322462 -0400][Debug ][Utility ] Unable to process user config file: [ERROR] OS Error: No such file or directory >> cmd: [2019-09-14 20:04:16.322604 -0400][Debug ][PlugInMgr ] Initializing plug-in manager... >> cmd: [2019-09-14 20:04:16.322646 -0400][Debug ][PlugInMgr ] No default plug-in, loading plug-in configs... >> cmd: [2019-09-14 20:04:16.322687 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /etc/xrootd/client.plugins.d... >> cmd: [2019-09-14 20:04:16.322779 -0400][Debug ][PlugInMgr ] Unable to process directory /etc/xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2019-09-14 20:04:16.323071 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /tmp/runos_singularity_home-310711236289025070240217978766146744805/.xrootd/client.plugins.d... >> cmd: [2019-09-14 20:04:16.323153 -0400][Debug ][PlugInMgr ] Unable to process directory /tmp/runos_singularity_home-310711236289025070240217978766146744805/.xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2019-09-14 20:04:16.323292 -0400][Debug ][App ] Executing: stat /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1556.root >> cmd: [2019-09-14 20:04:16.323366 -0400][Debug ][Poller ] Available pollers: built-in >> cmd: [2019-09-14 20:04:16.323400 -0400][Debug ][Poller ] Attempting to create a poller according to preference: built-in >> cmd: [2019-09-14 20:04:16.323433 -0400][Debug ][Poller ] Creating poller: built-in >> cmd: [2019-09-14 20:04:16.323470 -0400][Debug ][Poller ] Creating and starting the built-in poller... >> cmd: [2019-09-14 20:04:16.323765 -0400][Debug ][Poller ] Using 1 poller threads >> cmd: [2019-09-14 20:04:16.323810 -0400][Debug ][TaskMgr ] Starting the task manager... >> cmd: [2019-09-14 20:04:16.323934 -0400][Debug ][TaskMgr ] Task manager started >> cmd: [2019-09-14 20:04:16.323982 -0400][Debug ][JobMgr ] Starting the job manager... >> cmd: [2019-09-14 20:04:16.324320 -0400][Debug ][JobMgr ] Job manager started, 3 workers >> cmd: [2019-09-14 20:04:16.324362 -0400][Debug ][TaskMgr ] Registering task: "FileTimer task" to be run at: [2019-09-14 20:04:16 -0400] >> cmd: [2019-09-14 20:04:16.324421 -0400][Debug ][PostMaster ] Creating new channel to: deepthought.crc.nd.edu:1094 1 stream(s) >> cmd: [2019-09-14 20:04:16.324472 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800 >> cmd: [2019-09-14 20:04:16.326054 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" to be run at: [2019-09-14 20:04:31 -0400] >> cmd: [2019-09-14 20:04:16.327169 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094] Found 1 address(es): [::ffff:129.74.85.55]:1094 >> cmd: [2019-09-14 20:04:16.327286 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:129.74.85.55]:1094 >> cmd: [2019-09-14 20:04:16.327365 -0400][Debug ][Poller ] Adding socket 0x1d2b8a0 to the poller >> cmd: [2019-09-14 20:04:16.327618 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Async connection call returned >> cmd: [2019-09-14 20:04:16.327754 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2019-09-14 20:04:47.154745 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: manager [], protocol version 310) >> cmd: [2019-09-14 20:04:47.155243 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: manager [], protocol version 310) >> cmd: [2019-09-14 20:04:47.161274 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d12chas574.crc.nd.edu&xrd.rn=v4.6.1, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2019-09-14 20:04:47.165728 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Logged in, session: a00800008556370024000000a9080000 >> cmd: [2019-09-14 20:04:47.165901 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream 0 connected. >> cmd: [2019-09-14 20:04:47.166044 -0400][Debug ][Utility ] Monitor library name not set. No monitoring >> cmd: [2019-09-14 20:04:47.168100 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant03.crc.nd.edu:1096 1 stream(s) >> cmd: [2019-09-14 20:04:47.168220 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800 >> cmd: [2019-09-14 20:04:47.170524 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant03.crc.nd.edu:1096" to be run at: [2019-09-14 20:05:02 -0400] >> cmd: [2019-09-14 20:04:47.172684 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.11]:1096 >> cmd: [2019-09-14 20:04:47.172993 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.11]:1096 >> cmd: [2019-09-14 20:04:47.173176 -0400][Debug ][Poller ] Adding socket 0x680016a0 to the poller >> cmd: [2019-09-14 20:04:47.173469 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2019-09-14 20:04:47.173682 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2019-09-14 20:04:47.181686 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 310) >> cmd: [2019-09-14 20:04:47.182130 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 310) >> cmd: [2019-09-14 20:04:47.187190 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d12chas574.crc.nd.edu&xrd.rn=v4.6.1, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2019-09-14 20:04:47.188057 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Logged in, session: 4d090000860c000024000000480a0000 >> cmd: [2019-09-14 20:04:47.188182 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10300,c:ssl,ca: >> cmd: [2019-09-14 20:04:47.188285 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2019-09-14 20:04:47.230961 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: 190914 20:04:47 24793 secgsi_ParseCAlist: nothing to parse >> cmd: [2019-09-14 20:04:47.287471 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Cannot get credentials for protocol gsi: Secgsi: ErrParseBuffer: unknown CA: cannot verify server certificate: kXGS_init >> cmd: [2019-09-14 20:04:47.287555 -0400][Error ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] No protocols left to try >> cmd: [2019-09-14 20:04:47.287624 -0400][Error ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Socket error while handshaking: [FATAL] Auth failed >> cmd: [2019-09-14 20:04:47.287683 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:04:47.287743 -0400][Debug ][Poller ] <[::ffff:10.32.86.143]:44000><--><[::ffff:149.165.243.11]:1096> Removing socket from the poller >> cmd: [2019-09-14 20:04:47.287945 -0400][Error ][PostMaster ] [primeradiant03.crc.nd.edu:1096 #0] Unable to recover: [FATAL] Auth failed. >> cmd: [2019-09-14 20:04:47.288028 -0400][Error ][XRootD ] [primeradiant03.crc.nd.edu:1096] Impossible to send message kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1556.root, flags: none). Trying to recover. >> cmd: [2019-09-14 20:04:47.288094 -0400][Debug ][XRootD ] [primeradiant03.crc.nd.edu:1096] Handling error while processing kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1556.root, flags: none): [FATAL] Auth failed. >> cmd: [2019-09-14 20:04:50.289371 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x1d25ea0" to be run at: [2019-09-14 20:04:55 -0400] >> cmd: [2019-09-14 20:04:55.333683 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x1d25ea0" >> cmd: [2019-09-14 20:04:58.335111 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x1d25ea0" to be run at: [2019-09-14 20:05:03 -0400] >> cmd: [2019-09-14 20:05:03.335785 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x1d25ea0" >> cmd: [2019-09-14 20:05:03.357282 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x1d25ea0" to be run at: [2019-09-14 20:05:13 -0400] >> cmd: [2019-09-14 20:05:13.338362 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x1d25ea0" >> cmd: [2019-09-14 20:05:16.610402 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x1d25ea0" to be run at: [2019-09-14 20:05:26 -0400] >> cmd: [2019-09-14 20:05:26.342039 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x1d25ea0" >> cmd: [2019-09-14 20:05:34.833866 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x1d25ea0" to be run at: [2019-09-14 20:05:39 -0400] >> cmd: [2019-09-14 20:05:39.345643 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x1d25ea0" >> cmd: [2019-09-14 20:05:39.346688 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x1d25ea0" to be run at: [2019-09-14 20:05:49 -0400] >> cmd: [2019-09-14 20:05:49.348614 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x1d25ea0" >> cmd: [2019-09-14 20:05:57.984889 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x1d25ea0" to be run at: [2019-09-14 20:06:07 -0400] >> cmd: [2019-09-14 20:06:07.353482 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x1d25ea0" >> cmd: [2019-09-14 20:06:10.959883 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x1d25ea0" to be run at: [2019-09-14 20:06:20 -0400] >> cmd: [2019-09-14 20:06:20.356845 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x1d25ea0" >> cmd: [2019-09-14 20:06:22.358518 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x1d25ea0" to be run at: [2019-09-14 20:06:27 -0400] >> cmd: [2019-09-14 20:06:27.358989 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x1d25ea0" >> cmd: [2019-09-14 20:06:37.884439 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x1d25ea0" to be run at: [2019-09-14 20:06:47 -0400] >> cmd: [2019-09-14 20:06:47.363887 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x1d25ea0" >> cmd: [2019-09-14 20:06:55.842861 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x1d25ea0" to be run at: [2019-09-14 20:07:05 -0400] >> cmd: [2019-09-14 20:07:05.368207 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x1d25ea0" >> cmd: [2019-09-14 20:07:09.453218 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x1d25ea0" to be run at: [2019-09-14 20:07:19 -0400] >> cmd: [2019-09-14 20:07:19.371711 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x1d25ea0" >> cmd: [2019-09-14 20:07:21.373683 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x1d25ea0" to be run at: [2019-09-14 20:07:26 -0400] >> cmd: [2019-09-14 20:07:26.373894 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x1d25ea0" >> cmd: [2019-09-14 20:07:29.375033 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x1d25ea0" to be run at: [2019-09-14 20:07:34 -0400] >> cmd: [2019-09-14 20:07:34.376320 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x1d25ea0" >> cmd: [2019-09-14 20:07:34.377240 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x1d25ea0" to be run at: [2019-09-14 20:07:44 -0400] >> cmd: [2019-09-14 20:07:44.379004 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x1d25ea0" >> cmd: [2019-09-14 20:07:48.453584 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x1d25ea0" to be run at: [2019-09-14 20:07:58 -0400] >> cmd: [2019-09-14 20:07:58.382702 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x1d25ea0" >> cmd: [2019-09-14 20:08:13.091140 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x1d25ea0" to be run at: [2019-09-14 20:08:23 -0400] >> cmd: [2019-09-14 20:08:23.388527 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x1d25ea0" >> cmd: [2019-09-14 20:08:33.607054 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x1d25ea0" to be run at: [2019-09-14 20:08:43 -0400] >> cmd: [2019-09-14 20:08:43.393562 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x1d25ea0" >> cmd: [2019-09-14 20:09:02.909200 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x1d25ea0" to be run at: [2019-09-14 20:09:12 -0400] >> cmd: [2019-09-14 20:09:12.400922 -0400][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x1d25ea0" >> cmd: [2019-09-14 20:09:14.402354 -0400][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x1d25ea0" to be run at: [2019-09-14 20:09:19 -0400] >>> xrootd access to input file unavailable @ Sat Sep 14 20:09:16 2019 >>> Trying srm access method @ Sat Sep 14 20:09:16 2019 >>> executing '/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/bin/gfal-copy gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1556.root HIG-RunIIFall17wmLHE-00000ND_1556.root' @ Sat Sep 14 20:09:16 2019 >> using /disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME/tmp9o6Tjx to store command output >> cmd: gfal-copy error: 13 (Permission denied) - Could not stat the source: globus_ftp_client: the server responded with an error 530 530-Login incorrect. : /builddir/build/BUILD/gridftp-hdfs-1.1.1/src/gridftp_hdfs.c:hdfs_start:756: 530-Server is cancelling transfer due to over-load limit (host=deepthought.crc.nd.edu, user=uscms01, path=(null)) 530 End. >>>>> Unable to copy input with SRM @ Sat Sep 14 20:11:18 2019 >>> Trying srm access method @ Sat Sep 14 20:11:18 2019 >>> executing '/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/bin/gfal-copy gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1556.root HIG-RunIIFall17wmLHE-00000ND_1556.root' @ Sat Sep 14 20:11:18 2019 >> using /disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME/tmpBswU8Z to store command output >> cmd: gfal-copy error: 13 (Permission denied) - Could not stat the source: globus_ftp_client: the server responded with an error 530 530-Login incorrect. : /builddir/build/BUILD/gridftp-hdfs-1.1.1/src/gridftp_hdfs.c:hdfs_start:756: 530-Server is cancelling transfer due to over-load limit (host=deepthought.crc.nd.edu, user=uscms01, path=(null)) 530 End. >>>>> Unable to copy input with SRM @ Sat Sep 14 20:11:36 2019 >>>>>> no stage out method succeeded for: kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1556.root @ Sat Sep 14 20:11:36 2019 >>> Trying hdfs client access method @ Sat Sep 14 20:11:36 2019 >>> executing 'timeout 300 hdfs dfs -fs hdfs://eddie.crc.nd.edu:19000 -get /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_906.root HIG-RunIIFall17wmLHE-00000ND_906.root' @ Sat Sep 14 20:11:36 2019 >> using /disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME/tmph6U8NJ to store command output >> cmd: timeout: failed to run command 'hdfs': No such file or directory >>>>> Unable to copy input with hdfs client @ Sat Sep 14 20:11:36 2019 >>> Trying xrootd access method @ Sat Sep 14 20:11:36 2019 >>> executing 'env XRD_LOGLEVEL=Debug timeout 300 xrdfs deepthought.crc.nd.edu stat /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_906.root' @ Sat Sep 14 20:11:36 2019 >> using /disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME/tmpdt_nJX to store command output >> cmd: [2019-09-14 20:11:36.904279 -0400][Warning][Utility ] Unable to process global config file: [ERROR] OS Error: No such file or directory >> cmd: [2019-09-14 20:11:36.905353 -0400][Debug ][Utility ] Unable to process user config file: [ERROR] OS Error: No such file or directory >> cmd: [2019-09-14 20:11:36.905528 -0400][Debug ][PlugInMgr ] Initializing plug-in manager... >> cmd: [2019-09-14 20:11:36.905631 -0400][Debug ][PlugInMgr ] No default plug-in, loading plug-in configs... >> cmd: [2019-09-14 20:11:36.905696 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /etc/xrootd/client.plugins.d... >> cmd: [2019-09-14 20:11:36.905812 -0400][Debug ][PlugInMgr ] Unable to process directory /etc/xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2019-09-14 20:11:36.906203 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /tmp/runos_singularity_home-310711236289025070240217978766146744805/.xrootd/client.plugins.d... >> cmd: [2019-09-14 20:11:36.906310 -0400][Debug ][PlugInMgr ] Unable to process directory /tmp/runos_singularity_home-310711236289025070240217978766146744805/.xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2019-09-14 20:11:36.906473 -0400][Debug ][App ] Executing: stat /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_906.root >> cmd: [2019-09-14 20:11:36.906562 -0400][Debug ][Poller ] Available pollers: built-in >> cmd: [2019-09-14 20:11:36.906603 -0400][Debug ][Poller ] Attempting to create a poller according to preference: built-in >> cmd: [2019-09-14 20:11:36.906642 -0400][Debug ][Poller ] Creating poller: built-in >> cmd: [2019-09-14 20:11:36.906686 -0400][Debug ][Poller ] Creating and starting the built-in poller... >> cmd: [2019-09-14 20:11:36.907030 -0400][Debug ][Poller ] Using 1 poller threads >> cmd: [2019-09-14 20:11:36.907111 -0400][Debug ][TaskMgr ] Starting the task manager... >> cmd: [2019-09-14 20:11:36.907312 -0400][Debug ][TaskMgr ] Task manager started >> cmd: [2019-09-14 20:11:36.907387 -0400][Debug ][JobMgr ] Starting the job manager... >> cmd: [2019-09-14 20:11:36.907768 -0400][Debug ][JobMgr ] Job manager started, 3 workers >> cmd: [2019-09-14 20:11:36.907858 -0400][Debug ][TaskMgr ] Registering task: "FileTimer task" to be run at: [2019-09-14 20:11:36 -0400] >> cmd: [2019-09-14 20:11:36.907959 -0400][Debug ][PostMaster ] Creating new channel to: deepthought.crc.nd.edu:1094 1 stream(s) >> cmd: [2019-09-14 20:11:36.908031 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800 >> cmd: [2019-09-14 20:11:36.909176 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" to be run at: [2019-09-14 20:11:51 -0400] >> cmd: [2019-09-14 20:11:36.910635 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094] Found 1 address(es): [::ffff:129.74.85.55]:1094 >> cmd: [2019-09-14 20:11:36.910810 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:129.74.85.55]:1094 >> cmd: [2019-09-14 20:11:36.910919 -0400][Debug ][Poller ] Adding socket 0x4f27f0 to the poller >> cmd: [2019-09-14 20:11:36.911213 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Async connection call returned >> cmd: [2019-09-14 20:11:36.911339 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2019-09-14 20:11:36.911842 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: manager [], protocol version 310) >> cmd: [2019-09-14 20:11:36.912042 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: manager [], protocol version 310) >> cmd: [2019-09-14 20:11:36.915744 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d12chas574.crc.nd.edu&xrd.rn=v4.6.1, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2019-09-14 20:11:36.916204 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Logged in, session: 57090000855637001100000061090000 >> cmd: [2019-09-14 20:11:36.916307 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream 0 connected. >> cmd: [2019-09-14 20:11:36.916365 -0400][Debug ][Utility ] Monitor library name not set. No monitoring >> cmd: [2019-09-14 20:11:36.917097 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant03.crc.nd.edu:1096 1 stream(s) >> cmd: [2019-09-14 20:11:36.917176 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800 >> cmd: [2019-09-14 20:11:36.919037 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant03.crc.nd.edu:1096" to be run at: [2019-09-14 20:11:51 -0400] >> cmd: [2019-09-14 20:11:36.920156 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.11]:1096 >> cmd: [2019-09-14 20:11:36.920317 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.11]:1096 >> cmd: [2019-09-14 20:11:36.920416 -0400][Debug ][Poller ] Adding socket 0x580016a0 to the poller >> cmd: [2019-09-14 20:11:36.920549 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2019-09-14 20:11:36.920676 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2019-09-14 20:11:36.921089 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 310) >> cmd: [2019-09-14 20:11:36.921339 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 310) >> cmd: [2019-09-14 20:11:36.924701 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d12chas574.crc.nd.edu&xrd.rn=v4.6.1, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2019-09-14 20:11:36.925086 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Logged in, session: e8090000860c00001b000000e30a0000 >> cmd: [2019-09-14 20:11:36.925148 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10300,c:ssl,ca: >> cmd: [2019-09-14 20:11:36.925192 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2019-09-14 20:11:36.930966 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: 190914 20:11:36 29836 secgsi_ParseCAlist: nothing to parse >> cmd: [2019-09-14 20:11:36.959086 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Cannot get credentials for protocol gsi: Secgsi: ErrParseBuffer: unknown CA: cannot verify server certificate: kXGS_init >> cmd: [2019-09-14 20:11:36.959133 -0400][Error ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] No protocols left to try >> cmd: [2019-09-14 20:11:36.959175 -0400][Error ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Socket error while handshaking: [FATAL] Auth failed >> cmd: [2019-09-14 20:11:36.959220 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:36.959275 -0400][Debug ][Poller ] <[::ffff:10.32.86.143]:44790><--><[::ffff:149.165.243.11]:1096> Removing socket from the poller >> cmd: [2019-09-14 20:11:36.959440 -0400][Error ][PostMaster ] [primeradiant03.crc.nd.edu:1096 #0] Unable to recover: [FATAL] Auth failed. >> cmd: [2019-09-14 20:11:36.959517 -0400][Error ][XRootD ] [primeradiant03.crc.nd.edu:1096] Impossible to send message kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_906.root, flags: none). Trying to recover. >> cmd: [2019-09-14 20:11:36.959578 -0400][Debug ][XRootD ] [primeradiant03.crc.nd.edu:1096] Handling error while processing kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_906.root, flags: none): [FATAL] Auth failed. >> cmd: [2019-09-14 20:11:36.966329 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant02.crc.nd.edu:1096 1 stream(s) >> cmd: [2019-09-14 20:11:36.966427 -0400][Debug ][PostMaster ] [primeradiant02.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800 >> cmd: [2019-09-14 20:11:36.968022 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant02.crc.nd.edu:1096" to be run at: [2019-09-14 20:11:51 -0400] >> cmd: [2019-09-14 20:11:36.969512 -0400][Debug ][PostMaster ] [primeradiant02.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.6]:1096 >> cmd: [2019-09-14 20:11:36.969653 -0400][Debug ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.6]:1096 >> cmd: [2019-09-14 20:11:36.969747 -0400][Debug ][Poller ] Adding socket 0x60001540 to the poller >> cmd: [2019-09-14 20:11:36.969868 -0400][Debug ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2019-09-14 20:11:36.969990 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2019-09-14 20:11:36.970506 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 310) >> cmd: [2019-09-14 20:11:36.970715 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 310) >> cmd: [2019-09-14 20:11:36.973513 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d12chas574.crc.nd.edu&xrd.rn=v4.6.1, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2019-09-14 20:11:36.973904 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Logged in, session: dc0900009b0c00001b000000160b0000 >> cmd: [2019-09-14 20:11:36.973965 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10300,c:ssl,ca: >> cmd: [2019-09-14 20:11:36.974007 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2019-09-14 20:11:36.975370 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: 190914 20:11:36 29836 secgsi_ParseCAlist: nothing to parse >> cmd: [2019-09-14 20:11:36.975687 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Cannot get credentials for protocol gsi: Secgsi: ErrParseBuffer: unknown CA: cannot verify server certificate: kXGS_init >> cmd: [2019-09-14 20:11:36.975731 -0400][Error ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] No protocols left to try >> cmd: [2019-09-14 20:11:36.975782 -0400][Error ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Socket error while handshaking: [FATAL] Auth failed >> cmd: [2019-09-14 20:11:36.975824 -0400][Debug ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:36.975864 -0400][Debug ][Poller ] <[::ffff:10.32.86.143]:46464><--><[::ffff:149.165.243.6]:1096> Removing socket from the poller >> cmd: [2019-09-14 20:11:36.976015 -0400][Error ][PostMaster ] [primeradiant02.crc.nd.edu:1096 #0] Unable to recover: [FATAL] Auth failed. >> cmd: [2019-09-14 20:11:36.976070 -0400][Error ][XRootD ] [primeradiant02.crc.nd.edu:1096] Impossible to send message kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_906.root?tried=primeradiant03.crc.nd.edu, flags: none). Trying to recover. >> cmd: [2019-09-14 20:11:36.976122 -0400][Debug ][XRootD ] [primeradiant02.crc.nd.edu:1096] Handling error while processing kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_906.root?tried=primeradiant03.crc.nd.edu, flags: none): [FATAL] Auth failed. >> cmd: [2019-09-14 20:11:36.985882 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant05.crc.nd.edu:1096 1 stream(s) >> cmd: [2019-09-14 20:11:36.985959 -0400][Debug ][PostMaster ] [primeradiant05.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800 >> cmd: [2019-09-14 20:11:36.987689 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant05.crc.nd.edu:1096" to be run at: [2019-09-14 20:11:51 -0400] >> cmd: [2019-09-14 20:11:36.989216 -0400][Debug ][PostMaster ] [primeradiant05.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.13]:1096 >> cmd: [2019-09-14 20:11:36.989363 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.13]:1096 >> cmd: [2019-09-14 20:11:36.989453 -0400][Debug ][Poller ] Adding socket 0x64001540 to the poller >> cmd: [2019-09-14 20:11:36.989594 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2019-09-14 20:11:36.989719 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2019-09-14 20:11:36.990185 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 310) >> cmd: [2019-09-14 20:11:36.990412 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 310) >> cmd: [2019-09-14 20:11:36.993253 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d12chas574.crc.nd.edu&xrd.rn=v4.6.1, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2019-09-14 20:11:36.993680 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Logged in, session: d7090000780c00001b000000d60a0000 >> cmd: [2019-09-14 20:11:36.993731 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10300,c:ssl,ca: >> cmd: [2019-09-14 20:11:36.993784 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2019-09-14 20:11:36.994825 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: 190914 20:11:36 29836 secgsi_ParseCAlist: nothing to parse >> cmd: [2019-09-14 20:11:36.995212 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Cannot get credentials for protocol gsi: Secgsi: ErrParseBuffer: unknown CA: cannot verify server certificate: kXGS_init >> cmd: [2019-09-14 20:11:36.995301 -0400][Error ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] No protocols left to try >> cmd: [2019-09-14 20:11:36.995385 -0400][Error ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Socket error while handshaking: [FATAL] Auth failed >> cmd: [2019-09-14 20:11:36.995438 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:36.995477 -0400][Debug ][Poller ] <[::ffff:10.32.86.143]:57172><--><[::ffff:149.165.243.13]:1096> Removing socket from the poller >> cmd: [2019-09-14 20:11:36.995599 -0400][Error ][PostMaster ] [primeradiant05.crc.nd.edu:1096 #0] Unable to recover: [FATAL] Auth failed. >> cmd: [2019-09-14 20:11:36.995646 -0400][Error ][XRootD ] [primeradiant05.crc.nd.edu:1096] Impossible to send message kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_906.root?tried=primeradiant03.crc.nd.edu,primeradiant02.crc.nd.edu, flags: none). Trying to recover. >> cmd: [2019-09-14 20:11:36.995686 -0400][Debug ][XRootD ] [primeradiant05.crc.nd.edu:1096] Handling error while processing kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_906.root?tried=primeradiant03.crc.nd.edu,primeradiant02.crc.nd.edu, flags: none): [FATAL] Auth failed. >> cmd: [2019-09-14 20:11:37.014377 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant04.crc.nd.edu:1096 1 stream(s) >> cmd: [2019-09-14 20:11:37.014439 -0400][Debug ][PostMaster ] [primeradiant04.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800 >> cmd: [2019-09-14 20:11:37.015426 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant04.crc.nd.edu:1096" to be run at: [2019-09-14 20:11:52 -0400] >> cmd: [2019-09-14 20:11:37.016391 -0400][Debug ][PostMaster ] [primeradiant04.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.12]:1096 >> cmd: [2019-09-14 20:11:37.016511 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.12]:1096 >> cmd: [2019-09-14 20:11:37.016584 -0400][Debug ][Poller ] Adding socket 0x58001d20 to the poller >> cmd: [2019-09-14 20:11:37.016706 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2019-09-14 20:11:37.016808 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2019-09-14 20:11:37.017159 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 310) >> cmd: [2019-09-14 20:11:37.017395 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 310) >> cmd: [2019-09-14 20:11:37.020577 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d12chas574.crc.nd.edu&xrd.rn=v4.6.1, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2019-09-14 20:11:37.021006 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Logged in, session: 11090000a00c00001c00000011090000 >> cmd: [2019-09-14 20:11:37.021069 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10300,c:ssl,ca: >> cmd: [2019-09-14 20:11:37.021112 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2019-09-14 20:11:37.022090 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: 190914 20:11:37 29836 secgsi_ParseCAlist: nothing to parse >> cmd: [2019-09-14 20:11:37.022431 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Cannot get credentials for protocol gsi: Secgsi: ErrParseBuffer: unknown CA: cannot verify server certificate: kXGS_init >> cmd: [2019-09-14 20:11:37.022474 -0400][Error ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] No protocols left to try >> cmd: [2019-09-14 20:11:37.022530 -0400][Error ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Socket error while handshaking: [FATAL] Auth failed >> cmd: [2019-09-14 20:11:37.022590 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:37.022633 -0400][Debug ][Poller ] <[::ffff:10.32.86.143]:56232><--><[::ffff:149.165.243.12]:1096> Removing socket from the poller >> cmd: [2019-09-14 20:11:37.022791 -0400][Error ][PostMaster ] [primeradiant04.crc.nd.edu:1096 #0] Unable to recover: [FATAL] Auth failed. >> cmd: [2019-09-14 20:11:37.022850 -0400][Error ][XRootD ] [primeradiant04.crc.nd.edu:1096] Impossible to send message kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_906.root?tried=primeradiant03.crc.nd.edu,primeradiant02.crc.nd.edu,primeradiant05.crc.nd.edu, flags: none). Trying to recover. >> cmd: [2019-09-14 20:11:37.022894 -0400][Debug ][XRootD ] [primeradiant04.crc.nd.edu:1096] Handling error while processing kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_906.root?tried=primeradiant03.crc.nd.edu,primeradiant02.crc.nd.edu,primeradiant05.crc.nd.edu, flags: none): [FATAL] Auth failed. >> cmd: [2019-09-14 20:11:37.034896 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant01.crc.nd.edu:1096 1 stream(s) >> cmd: [2019-09-14 20:11:37.034994 -0400][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800 >> cmd: [2019-09-14 20:11:37.036586 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant01.crc.nd.edu:1096" to be run at: [2019-09-14 20:11:52 -0400] >> cmd: [2019-09-14 20:11:37.038198 -0400][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.5]:1096 >> cmd: [2019-09-14 20:11:37.038404 -0400][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.5]:1096 >> cmd: [2019-09-14 20:11:37.038520 -0400][Debug ][Poller ] Adding socket 0x60001e10 to the poller >> cmd: [2019-09-14 20:11:37.039109 -0400][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2019-09-14 20:11:37.039252 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2019-09-14 20:11:37.039658 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 310) >> cmd: [2019-09-14 20:11:37.039883 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 310) >> cmd: [2019-09-14 20:11:37.042506 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d12chas574.crc.nd.edu&xrd.rn=v4.6.1, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2019-09-14 20:11:37.042886 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Logged in, session: 470a00009d0c00001d000000760b0000 >> cmd: [2019-09-14 20:11:37.042940 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10300,c:ssl,ca: >> cmd: [2019-09-14 20:11:37.042986 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2019-09-14 20:11:37.044247 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: 190914 20:11:37 29836 secgsi_ParseCAlist: nothing to parse >> cmd: [2019-09-14 20:11:37.044669 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Cannot get credentials for protocol gsi: Secgsi: ErrParseBuffer: unknown CA: cannot verify server certificate: kXGS_init >> cmd: [2019-09-14 20:11:37.044722 -0400][Error ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] No protocols left to try >> cmd: [2019-09-14 20:11:37.044769 -0400][Error ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Socket error while handshaking: [FATAL] Auth failed >> cmd: [2019-09-14 20:11:37.044812 -0400][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:37.044848 -0400][Debug ][Poller ] <[::ffff:10.32.86.143]:53398><--><[::ffff:149.165.243.5]:1096> Removing socket from the poller >> cmd: [2019-09-14 20:11:37.045011 -0400][Error ][PostMaster ] [primeradiant01.crc.nd.edu:1096 #0] Unable to recover: [FATAL] Auth failed. >> cmd: [2019-09-14 20:11:37.045077 -0400][Error ][XRootD ] [primeradiant01.crc.nd.edu:1096] Impossible to send message kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_906.root?tried=primeradiant03.crc.nd.edu,primeradiant02.crc.nd.edu,primeradiant05.crc.nd.edu,primeradiant04.crc.nd.edu, flags: none). Trying to recover. >> cmd: [2019-09-14 20:11:37.045142 -0400][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1096] Handling error while processing kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_906.root?tried=primeradiant03.crc.nd.edu,primeradiant02.crc.nd.edu,primeradiant05.crc.nd.edu,primeradiant04.crc.nd.edu, flags: none): [FATAL] Auth failed. >> cmd: [2019-09-14 20:11:37.101233 -0400][Debug ][XRootD ] [deepthought.crc.nd.edu:1094] Handling error while processing kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_906.root?tried=primeradiant03.crc.nd.edu,primeradiant02.crc.nd.edu,primeradiant05.crc.nd.edu,primeradiant04.crc.nd.edu,primeradiant01.crc.nd.edu, flags: none): [ERROR] Error response: No such file or directory. >> cmd: [2019-09-14 20:11:37.101466 -0400][Error ][App ] Unable stat /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_906.root: [ERROR] Server responded with an error: [3011] No servers have read access to the file >> cmd: [ERROR] Server responded with an error: [3011] No servers have read access to the file >> cmd: >> cmd: [2019-09-14 20:11:37.121064 -0400][Debug ][JobMgr ] Stopping the job manager... >> cmd: [2019-09-14 20:11:37.121797 -0400][Debug ][JobMgr ] Job manager stopped >> cmd: [2019-09-14 20:11:37.121893 -0400][Debug ][TaskMgr ] Stopping the task manager... >> cmd: [2019-09-14 20:11:37.122116 -0400][Debug ][TaskMgr ] Task manager stopped >> cmd: [2019-09-14 20:11:37.122177 -0400][Debug ][Poller ] Stopping the poller... >> cmd: [2019-09-14 20:11:37.122505 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" >> cmd: [2019-09-14 20:11:37.122586 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:37.122641 -0400][Debug ][Poller ] <[::ffff:10.32.86.143]:50154><--><[::ffff:129.74.85.55]:1094> Removing socket from the poller >> cmd: [2019-09-14 20:11:37.122741 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Destroying stream >> cmd: [2019-09-14 20:11:37.122803 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:37.122874 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant01.crc.nd.edu:1096" >> cmd: [2019-09-14 20:11:37.122931 -0400][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:37.122998 -0400][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2019-09-14 20:11:37.123068 -0400][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:37.123135 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant02.crc.nd.edu:1096" >> cmd: [2019-09-14 20:11:37.123190 -0400][Debug ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:37.123235 -0400][Debug ][PostMaster ] [primeradiant02.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2019-09-14 20:11:37.123270 -0400][Debug ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:37.123309 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant03.crc.nd.edu:1096" >> cmd: [2019-09-14 20:11:37.123344 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:37.123386 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2019-09-14 20:11:37.123422 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:37.123458 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant04.crc.nd.edu:1096" >> cmd: [2019-09-14 20:11:37.123493 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:37.123527 -0400][Debug ][PostMaster ] [primeradiant04.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2019-09-14 20:11:37.123561 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:37.123619 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant05.crc.nd.edu:1096" >> cmd: [2019-09-14 20:11:37.123673 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:37.123713 -0400][Debug ][PostMaster ] [primeradiant05.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2019-09-14 20:11:37.123747 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Closing the socket >>> xrootd access to input file unavailable @ Sat Sep 14 20:11:37 2019 >>> Trying srm access method @ Sat Sep 14 20:11:37 2019 >>> executing '/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/bin/gfal-copy gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_906.root HIG-RunIIFall17wmLHE-00000ND_906.root' @ Sat Sep 14 20:11:37 2019 >> using /disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME/tmpsMuxJr to store command output >> cmd: gfal-copy error: 13 (Permission denied) - Could not stat the source: globus_ftp_client: the server responded with an error 530 530-Login incorrect. : /builddir/build/BUILD/gridftp-hdfs-1.1.1/src/gridftp_hdfs.c:hdfs_start:756: 530-Server is cancelling transfer due to over-load limit (host=deepthought.crc.nd.edu, user=uscms01, path=(null)) 530 End. >>>>> Unable to copy input with SRM @ Sat Sep 14 20:11:43 2019 >>> Trying srm access method @ Sat Sep 14 20:11:43 2019 >>> executing '/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/bin/gfal-copy gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_906.root HIG-RunIIFall17wmLHE-00000ND_906.root' @ Sat Sep 14 20:11:43 2019 >> using /disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME/tmpw3EM_m to store command output >> cmd: gfal-copy error: 13 (Permission denied) - Could not stat the source: globus_ftp_client: the server responded with an error 530 530-Login incorrect. : /builddir/build/BUILD/gridftp-hdfs-1.1.1/src/gridftp_hdfs.c:hdfs_start:756: 530-Server is cancelling transfer due to over-load limit (host=deepthought.crc.nd.edu, user=uscms01, path=(null)) 530 End. >>>>> Unable to copy input with SRM @ Sat Sep 14 20:11:46 2019 >>>>>> no stage out method succeeded for: kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_906.root @ Sat Sep 14 20:11:46 2019 >>> Trying hdfs client access method @ Sat Sep 14 20:11:46 2019 >>> executing 'timeout 300 hdfs dfs -fs hdfs://eddie.crc.nd.edu:19000 -get /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1680.root HIG-RunIIFall17wmLHE-00000ND_1680.root' @ Sat Sep 14 20:11:46 2019 >> using /disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME/tmp7Hhtdk to store command output >> cmd: timeout: failed to run command 'hdfs': No such file or directory >>>>> Unable to copy input with hdfs client @ Sat Sep 14 20:11:46 2019 >>> Trying xrootd access method @ Sat Sep 14 20:11:46 2019 >>> executing 'env XRD_LOGLEVEL=Debug timeout 300 xrdfs deepthought.crc.nd.edu stat /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1680.root' @ Sat Sep 14 20:11:46 2019 >> using /disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME/tmpovPZmt to store command output >> cmd: [2019-09-14 20:11:46.718553 -0400][Warning][Utility ] Unable to process global config file: [ERROR] OS Error: No such file or directory >> cmd: [2019-09-14 20:11:46.719741 -0400][Debug ][Utility ] Unable to process user config file: [ERROR] OS Error: No such file or directory >> cmd: [2019-09-14 20:11:46.720011 -0400][Debug ][PlugInMgr ] Initializing plug-in manager... >> cmd: [2019-09-14 20:11:46.720100 -0400][Debug ][PlugInMgr ] No default plug-in, loading plug-in configs... >> cmd: [2019-09-14 20:11:46.720168 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /etc/xrootd/client.plugins.d... >> cmd: [2019-09-14 20:11:46.720312 -0400][Debug ][PlugInMgr ] Unable to process directory /etc/xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2019-09-14 20:11:46.720756 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /tmp/runos_singularity_home-310711236289025070240217978766146744805/.xrootd/client.plugins.d... >> cmd: [2019-09-14 20:11:46.720911 -0400][Debug ][PlugInMgr ] Unable to process directory /tmp/runos_singularity_home-310711236289025070240217978766146744805/.xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2019-09-14 20:11:46.721101 -0400][Debug ][App ] Executing: stat /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1680.root >> cmd: [2019-09-14 20:11:46.721228 -0400][Debug ][Poller ] Available pollers: built-in >> cmd: [2019-09-14 20:11:46.721303 -0400][Debug ][Poller ] Attempting to create a poller according to preference: built-in >> cmd: [2019-09-14 20:11:46.721368 -0400][Debug ][Poller ] Creating poller: built-in >> cmd: [2019-09-14 20:11:46.721447 -0400][Debug ][Poller ] Creating and starting the built-in poller... >> cmd: [2019-09-14 20:11:46.721832 -0400][Debug ][Poller ] Using 1 poller threads >> cmd: [2019-09-14 20:11:46.721884 -0400][Debug ][TaskMgr ] Starting the task manager... >> cmd: [2019-09-14 20:11:46.722076 -0400][Debug ][TaskMgr ] Task manager started >> cmd: [2019-09-14 20:11:46.722139 -0400][Debug ][JobMgr ] Starting the job manager... >> cmd: [2019-09-14 20:11:46.722495 -0400][Debug ][JobMgr ] Job manager started, 3 workers >> cmd: [2019-09-14 20:11:46.722560 -0400][Debug ][TaskMgr ] Registering task: "FileTimer task" to be run at: [2019-09-14 20:11:46 -0400] >> cmd: [2019-09-14 20:11:46.722656 -0400][Debug ][PostMaster ] Creating new channel to: deepthought.crc.nd.edu:1094 1 stream(s) >> cmd: [2019-09-14 20:11:46.722732 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800 >> cmd: [2019-09-14 20:11:46.724167 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" to be run at: [2019-09-14 20:12:01 -0400] >> cmd: [2019-09-14 20:11:46.725618 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094] Found 1 address(es): [::ffff:129.74.85.55]:1094 >> cmd: [2019-09-14 20:11:46.725757 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:129.74.85.55]:1094 >> cmd: [2019-09-14 20:11:46.725850 -0400][Debug ][Poller ] Adding socket 0x1a568a0 to the poller >> cmd: [2019-09-14 20:11:46.726098 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Async connection call returned >> cmd: [2019-09-14 20:11:46.726239 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2019-09-14 20:11:46.726915 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: manager [], protocol version 310) >> cmd: [2019-09-14 20:11:46.727347 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: manager [], protocol version 310) >> cmd: [2019-09-14 20:11:46.730849 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d12chas574.crc.nd.edu&xrd.rn=v4.6.1, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2019-09-14 20:11:46.731203 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Logged in, session: 770a00008556370057000000810a0000 >> cmd: [2019-09-14 20:11:46.731280 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream 0 connected. >> cmd: [2019-09-14 20:11:46.731329 -0400][Debug ][Utility ] Monitor library name not set. No monitoring >> cmd: [2019-09-14 20:11:46.731933 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant03.crc.nd.edu:1096 1 stream(s) >> cmd: [2019-09-14 20:11:46.732021 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800 >> cmd: [2019-09-14 20:11:46.733607 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant03.crc.nd.edu:1096" to be run at: [2019-09-14 20:12:01 -0400] >> cmd: [2019-09-14 20:11:46.735049 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.11]:1096 >> cmd: [2019-09-14 20:11:46.735217 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.11]:1096 >> cmd: [2019-09-14 20:11:46.735323 -0400][Debug ][Poller ] Adding socket 0x5c0016a0 to the poller >> cmd: [2019-09-14 20:11:46.735438 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2019-09-14 20:11:46.735552 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2019-09-14 20:11:46.735927 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 310) >> cmd: [2019-09-14 20:11:46.736187 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 310) >> cmd: [2019-09-14 20:11:46.739704 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d12chas574.crc.nd.edu&xrd.rn=v4.6.1, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2019-09-14 20:11:46.740304 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Logged in, session: ca0a0000860c00001b000000c50b0000 >> cmd: [2019-09-14 20:11:46.740381 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10300,c:ssl,ca: >> cmd: [2019-09-14 20:11:46.740445 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2019-09-14 20:11:46.746646 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: 190914 20:11:46 30726 secgsi_ParseCAlist: nothing to parse >> cmd: [2019-09-14 20:11:46.778163 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Cannot get credentials for protocol gsi: Secgsi: ErrParseBuffer: unknown CA: cannot verify server certificate: kXGS_init >> cmd: [2019-09-14 20:11:46.778214 -0400][Error ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] No protocols left to try >> cmd: [2019-09-14 20:11:46.778266 -0400][Error ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Socket error while handshaking: [FATAL] Auth failed >> cmd: [2019-09-14 20:11:46.778338 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:46.778475 -0400][Debug ][Poller ] <[::ffff:10.32.86.143]:45198><--><[::ffff:149.165.243.11]:1096> Removing socket from the poller >> cmd: [2019-09-14 20:11:46.778675 -0400][Error ][PostMaster ] [primeradiant03.crc.nd.edu:1096 #0] Unable to recover: [FATAL] Auth failed. >> cmd: [2019-09-14 20:11:46.778735 -0400][Error ][XRootD ] [primeradiant03.crc.nd.edu:1096] Impossible to send message kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1680.root, flags: none). Trying to recover. >> cmd: [2019-09-14 20:11:46.778783 -0400][Debug ][XRootD ] [primeradiant03.crc.nd.edu:1096] Handling error while processing kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1680.root, flags: none): [FATAL] Auth failed. >> cmd: [2019-09-14 20:11:46.780366 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant02.crc.nd.edu:1096 1 stream(s) >> cmd: [2019-09-14 20:11:46.780439 -0400][Debug ][PostMaster ] [primeradiant02.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800 >> cmd: [2019-09-14 20:11:46.782888 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant02.crc.nd.edu:1096" to be run at: [2019-09-14 20:12:01 -0400] >> cmd: [2019-09-14 20:11:46.784832 -0400][Debug ][PostMaster ] [primeradiant02.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.6]:1096 >> cmd: [2019-09-14 20:11:46.784968 -0400][Debug ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.6]:1096 >> cmd: [2019-09-14 20:11:46.785079 -0400][Debug ][Poller ] Adding socket 0x60001540 to the poller >> cmd: [2019-09-14 20:11:46.785256 -0400][Debug ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2019-09-14 20:11:46.785371 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2019-09-14 20:11:46.785816 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 310) >> cmd: [2019-09-14 20:11:46.785992 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 310) >> cmd: [2019-09-14 20:11:46.789313 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d12chas574.crc.nd.edu&xrd.rn=v4.6.1, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2019-09-14 20:11:46.790179 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Logged in, session: c20a00009b0c00001b000000fc0b0000 >> cmd: [2019-09-14 20:11:46.790257 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10300,c:ssl,ca: >> cmd: [2019-09-14 20:11:46.790307 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2019-09-14 20:11:46.791802 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: 190914 20:11:46 30726 secgsi_ParseCAlist: nothing to parse >> cmd: [2019-09-14 20:11:46.792183 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Cannot get credentials for protocol gsi: Secgsi: ErrParseBuffer: unknown CA: cannot verify server certificate: kXGS_init >> cmd: [2019-09-14 20:11:46.792232 -0400][Error ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] No protocols left to try >> cmd: [2019-09-14 20:11:46.792287 -0400][Error ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Socket error while handshaking: [FATAL] Auth failed >> cmd: [2019-09-14 20:11:46.792341 -0400][Debug ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:46.792409 -0400][Debug ][Poller ] <[::ffff:10.32.86.143]:46880><--><[::ffff:149.165.243.6]:1096> Removing socket from the poller >> cmd: [2019-09-14 20:11:46.792583 -0400][Error ][PostMaster ] [primeradiant02.crc.nd.edu:1096 #0] Unable to recover: [FATAL] Auth failed. >> cmd: [2019-09-14 20:11:46.792640 -0400][Error ][XRootD ] [primeradiant02.crc.nd.edu:1096] Impossible to send message kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1680.root?tried=primeradiant03.crc.nd.edu, flags: none). Trying to recover. >> cmd: [2019-09-14 20:11:46.792709 -0400][Debug ][XRootD ] [primeradiant02.crc.nd.edu:1096] Handling error while processing kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1680.root?tried=primeradiant03.crc.nd.edu, flags: none): [FATAL] Auth failed. >> cmd: [2019-09-14 20:11:46.794898 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant05.crc.nd.edu:1096 1 stream(s) >> cmd: [2019-09-14 20:11:46.794962 -0400][Debug ][PostMaster ] [primeradiant05.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800 >> cmd: [2019-09-14 20:11:46.796940 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant05.crc.nd.edu:1096" to be run at: [2019-09-14 20:12:01 -0400] >> cmd: [2019-09-14 20:11:46.798363 -0400][Debug ][PostMaster ] [primeradiant05.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.13]:1096 >> cmd: [2019-09-14 20:11:46.798482 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.13]:1096 >> cmd: [2019-09-14 20:11:46.798557 -0400][Debug ][Poller ] Adding socket 0x64001540 to the poller >> cmd: [2019-09-14 20:11:46.798684 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2019-09-14 20:11:46.798783 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2019-09-14 20:11:46.799160 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 310) >> cmd: [2019-09-14 20:11:46.799352 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 310) >> cmd: [2019-09-14 20:11:46.802676 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d12chas574.crc.nd.edu&xrd.rn=v4.6.1, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2019-09-14 20:11:46.803026 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Logged in, session: be0a0000780c00001c000000bd0b0000 >> cmd: [2019-09-14 20:11:46.803083 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10300,c:ssl,ca: >> cmd: [2019-09-14 20:11:46.803120 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2019-09-14 20:11:46.804344 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: 190914 20:11:46 30726 secgsi_ParseCAlist: nothing to parse >> cmd: [2019-09-14 20:11:46.804641 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Cannot get credentials for protocol gsi: Secgsi: ErrParseBuffer: unknown CA: cannot verify server certificate: kXGS_init >> cmd: [2019-09-14 20:11:46.804683 -0400][Error ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] No protocols left to try >> cmd: [2019-09-14 20:11:46.804721 -0400][Error ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Socket error while handshaking: [FATAL] Auth failed >> cmd: [2019-09-14 20:11:46.804756 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:46.804792 -0400][Debug ][Poller ] <[::ffff:10.32.86.143]:57588><--><[::ffff:149.165.243.13]:1096> Removing socket from the poller >> cmd: [2019-09-14 20:11:46.804907 -0400][Error ][PostMaster ] [primeradiant05.crc.nd.edu:1096 #0] Unable to recover: [FATAL] Auth failed. >> cmd: [2019-09-14 20:11:46.804956 -0400][Error ][XRootD ] [primeradiant05.crc.nd.edu:1096] Impossible to send message kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1680.root?tried=primeradiant03.crc.nd.edu,primeradiant02.crc.nd.edu, flags: none). Trying to recover. >> cmd: [2019-09-14 20:11:46.804998 -0400][Debug ][XRootD ] [primeradiant05.crc.nd.edu:1096] Handling error while processing kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1680.root?tried=primeradiant03.crc.nd.edu,primeradiant02.crc.nd.edu, flags: none): [FATAL] Auth failed. >> cmd: [2019-09-14 20:11:46.807305 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant04.crc.nd.edu:1096 1 stream(s) >> cmd: [2019-09-14 20:11:46.807379 -0400][Debug ][PostMaster ] [primeradiant04.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800 >> cmd: [2019-09-14 20:11:46.808830 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant04.crc.nd.edu:1096" to be run at: [2019-09-14 20:12:01 -0400] >> cmd: [2019-09-14 20:11:46.809758 -0400][Debug ][PostMaster ] [primeradiant04.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.12]:1096 >> cmd: [2019-09-14 20:11:46.809901 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.12]:1096 >> cmd: [2019-09-14 20:11:46.810000 -0400][Debug ][Poller ] Adding socket 0x5c001d20 to the poller >> cmd: [2019-09-14 20:11:46.810149 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2019-09-14 20:11:46.810267 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2019-09-14 20:11:46.810655 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 310) >> cmd: [2019-09-14 20:11:46.810854 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 310) >> cmd: [2019-09-14 20:11:46.813631 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d12chas574.crc.nd.edu&xrd.rn=v4.6.1, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2019-09-14 20:11:46.813975 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Logged in, session: 260a0000a00c00001b000000260a0000 >> cmd: [2019-09-14 20:11:46.814020 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10300,c:ssl,ca: >> cmd: [2019-09-14 20:11:46.814056 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2019-09-14 20:11:46.815366 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: 190914 20:11:46 30726 secgsi_ParseCAlist: nothing to parse >> cmd: [2019-09-14 20:11:46.815796 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Cannot get credentials for protocol gsi: Secgsi: ErrParseBuffer: unknown CA: cannot verify server certificate: kXGS_init >> cmd: [2019-09-14 20:11:46.815868 -0400][Error ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] No protocols left to try >> cmd: [2019-09-14 20:11:46.815925 -0400][Error ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Socket error while handshaking: [FATAL] Auth failed >> cmd: [2019-09-14 20:11:46.816028 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:46.816096 -0400][Debug ][Poller ] <[::ffff:10.32.86.143]:56648><--><[::ffff:149.165.243.12]:1096> Removing socket from the poller >> cmd: [2019-09-14 20:11:46.816514 -0400][Error ][PostMaster ] [primeradiant04.crc.nd.edu:1096 #0] Unable to recover: [FATAL] Auth failed. >> cmd: [2019-09-14 20:11:46.816565 -0400][Error ][XRootD ] [primeradiant04.crc.nd.edu:1096] Impossible to send message kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1680.root?tried=primeradiant03.crc.nd.edu,primeradiant02.crc.nd.edu,primeradiant05.crc.nd.edu, flags: none). Trying to recover. >> cmd: [2019-09-14 20:11:46.816607 -0400][Debug ][XRootD ] [primeradiant04.crc.nd.edu:1096] Handling error while processing kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1680.root?tried=primeradiant03.crc.nd.edu,primeradiant02.crc.nd.edu,primeradiant05.crc.nd.edu, flags: none): [FATAL] Auth failed. >> cmd: [2019-09-14 20:11:46.819086 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant01.crc.nd.edu:1096 1 stream(s) >> cmd: [2019-09-14 20:11:46.819150 -0400][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800 >> cmd: [2019-09-14 20:11:46.820247 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant01.crc.nd.edu:1096" to be run at: [2019-09-14 20:12:01 -0400] >> cmd: [2019-09-14 20:11:46.821285 -0400][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.5]:1096 >> cmd: [2019-09-14 20:11:46.821400 -0400][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.5]:1096 >> cmd: [2019-09-14 20:11:46.821470 -0400][Debug ][Poller ] Adding socket 0x60001e10 to the poller >> cmd: [2019-09-14 20:11:46.821586 -0400][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2019-09-14 20:11:46.821679 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2019-09-14 20:11:46.822027 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 310) >> cmd: [2019-09-14 20:11:46.822284 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 310) >> cmd: [2019-09-14 20:11:46.825148 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d12chas574.crc.nd.edu&xrd.rn=v4.6.1, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2019-09-14 20:11:46.825454 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Logged in, session: 5c0b00009d0c00001f0000008c0c0000 >> cmd: [2019-09-14 20:11:46.825516 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10300,c:ssl,ca: >> cmd: [2019-09-14 20:11:46.825616 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2019-09-14 20:11:46.826764 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: 190914 20:11:46 30726 secgsi_ParseCAlist: nothing to parse >> cmd: [2019-09-14 20:11:46.827042 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Cannot get credentials for protocol gsi: Secgsi: ErrParseBuffer: unknown CA: cannot verify server certificate: kXGS_init >> cmd: [2019-09-14 20:11:46.827091 -0400][Error ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] No protocols left to try >> cmd: [2019-09-14 20:11:46.827171 -0400][Error ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Socket error while handshaking: [FATAL] Auth failed >> cmd: [2019-09-14 20:11:46.827230 -0400][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:46.827300 -0400][Debug ][Poller ] <[::ffff:10.32.86.143]:53812><--><[::ffff:149.165.243.5]:1096> Removing socket from the poller >> cmd: [2019-09-14 20:11:46.827466 -0400][Error ][PostMaster ] [primeradiant01.crc.nd.edu:1096 #0] Unable to recover: [FATAL] Auth failed. >> cmd: [2019-09-14 20:11:46.827532 -0400][Error ][XRootD ] [primeradiant01.crc.nd.edu:1096] Impossible to send message kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1680.root?tried=primeradiant03.crc.nd.edu,primeradiant02.crc.nd.edu,primeradiant05.crc.nd.edu,primeradiant04.crc.nd.edu, flags: none). Trying to recover. >> cmd: [2019-09-14 20:11:46.827609 -0400][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1096] Handling error while processing kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1680.root?tried=primeradiant03.crc.nd.edu,primeradiant02.crc.nd.edu,primeradiant05.crc.nd.edu,primeradiant04.crc.nd.edu, flags: none): [FATAL] Auth failed. >> cmd: [2019-09-14 20:11:46.832377 -0400][Debug ][XRootD ] [deepthought.crc.nd.edu:1094] Handling error while processing kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1680.root?tried=primeradiant03.crc.nd.edu,primeradiant02.crc.nd.edu,primeradiant05.crc.nd.edu,primeradiant04.crc.nd.edu,primeradiant01.crc.nd.edu, flags: none): [ERROR] Error response: No such file or directory. >> cmd: [2019-09-14 20:11:46.832534 -0400][Error ][App ] Unable stat /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1680.root: [ERROR] Server responded with an error: [3011] No servers have read access to the file >> cmd: [ERROR] Server responded with an error: [3011] No servers have read access to the file >> cmd: >> cmd: [2019-09-14 20:11:46.832790 -0400][Debug ][JobMgr ] Stopping the job manager... >> cmd: [2019-09-14 20:11:46.833306 -0400][Debug ][JobMgr ] Job manager stopped >> cmd: [2019-09-14 20:11:46.833357 -0400][Debug ][TaskMgr ] Stopping the task manager... >> cmd: [2019-09-14 20:11:46.833511 -0400][Debug ][TaskMgr ] Task manager stopped >> cmd: [2019-09-14 20:11:46.833546 -0400][Debug ][Poller ] Stopping the poller... >> cmd: [2019-09-14 20:11:46.833807 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" >> cmd: [2019-09-14 20:11:46.833857 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:46.833892 -0400][Debug ][Poller ] <[::ffff:10.32.86.143]:50560><--><[::ffff:129.74.85.55]:1094> Removing socket from the poller >> cmd: [2019-09-14 20:11:46.833965 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Destroying stream >> cmd: [2019-09-14 20:11:46.834007 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:46.834046 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant01.crc.nd.edu:1096" >> cmd: [2019-09-14 20:11:46.834080 -0400][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:46.834112 -0400][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2019-09-14 20:11:46.834145 -0400][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:46.834178 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant02.crc.nd.edu:1096" >> cmd: [2019-09-14 20:11:46.834210 -0400][Debug ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:46.834249 -0400][Debug ][PostMaster ] [primeradiant02.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2019-09-14 20:11:46.834282 -0400][Debug ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:46.834315 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant03.crc.nd.edu:1096" >> cmd: [2019-09-14 20:11:46.834347 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:46.834380 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2019-09-14 20:11:46.834412 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:46.834445 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant04.crc.nd.edu:1096" >> cmd: [2019-09-14 20:11:46.834477 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:46.834509 -0400][Debug ][PostMaster ] [primeradiant04.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2019-09-14 20:11:46.834541 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:46.834574 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant05.crc.nd.edu:1096" >> cmd: [2019-09-14 20:11:46.834607 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:46.834639 -0400][Debug ][PostMaster ] [primeradiant05.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2019-09-14 20:11:46.834671 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Closing the socket >>> xrootd access to input file unavailable @ Sat Sep 14 20:11:46 2019 >>> Trying srm access method @ Sat Sep 14 20:11:46 2019 >>> executing '/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/bin/gfal-copy gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1680.root HIG-RunIIFall17wmLHE-00000ND_1680.root' @ Sat Sep 14 20:11:46 2019 >> using /disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME/tmpR9pmp_ to store command output >> cmd: gfal-copy error: 13 (Permission denied) - Could not stat the source: globus_ftp_client: the server responded with an error 530 530-Login incorrect. : /builddir/build/BUILD/gridftp-hdfs-1.1.1/src/gridftp_hdfs.c:hdfs_start:756: 530-Server is cancelling transfer due to over-load limit (host=deepthought.crc.nd.edu, user=uscms01, path=(null)) 530 End. >>>>> Unable to copy input with SRM @ Sat Sep 14 20:11:50 2019 >>> Trying srm access method @ Sat Sep 14 20:11:50 2019 >>> executing '/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/bin/gfal-copy gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1680.root HIG-RunIIFall17wmLHE-00000ND_1680.root' @ Sat Sep 14 20:11:50 2019 >> using /disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME/tmpX0omWq to store command output >> cmd: gfal-copy error: 13 (Permission denied) - Could not stat the source: globus_ftp_client: the server responded with an error 530 530-Login incorrect. : /builddir/build/BUILD/gridftp-hdfs-1.1.1/src/gridftp_hdfs.c:hdfs_start:756: 530-Server is cancelling transfer due to over-load limit (host=deepthought.crc.nd.edu, user=uscms01, path=(null)) 530 End. >>>>> Unable to copy input with SRM @ Sat Sep 14 20:11:53 2019 >>>>>> no stage out method succeeded for: kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1680.root @ Sat Sep 14 20:11:53 2019 >>> Trying hdfs client access method @ Sat Sep 14 20:11:53 2019 >>> executing 'timeout 300 hdfs dfs -fs hdfs://eddie.crc.nd.edu:19000 -get /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1728.root HIG-RunIIFall17wmLHE-00000ND_1728.root' @ Sat Sep 14 20:11:53 2019 >> using /disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME/tmpxarKQ8 to store command output >> cmd: timeout: failed to run command 'hdfs': No such file or directory >>>>> Unable to copy input with hdfs client @ Sat Sep 14 20:11:53 2019 >>> Trying xrootd access method @ Sat Sep 14 20:11:53 2019 >>> executing 'env XRD_LOGLEVEL=Debug timeout 300 xrdfs deepthought.crc.nd.edu stat /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1728.root' @ Sat Sep 14 20:11:53 2019 >> using /disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME/tmpYofZkK to store command output >> cmd: [2019-09-14 20:11:53.448716 -0400][Warning][Utility ] Unable to process global config file: [ERROR] OS Error: No such file or directory >> cmd: [2019-09-14 20:11:53.449828 -0400][Debug ][Utility ] Unable to process user config file: [ERROR] OS Error: No such file or directory >> cmd: [2019-09-14 20:11:53.449985 -0400][Debug ][PlugInMgr ] Initializing plug-in manager... >> cmd: [2019-09-14 20:11:53.450056 -0400][Debug ][PlugInMgr ] No default plug-in, loading plug-in configs... >> cmd: [2019-09-14 20:11:53.450137 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /etc/xrootd/client.plugins.d... >> cmd: [2019-09-14 20:11:53.450254 -0400][Debug ][PlugInMgr ] Unable to process directory /etc/xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2019-09-14 20:11:53.450642 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /tmp/runos_singularity_home-310711236289025070240217978766146744805/.xrootd/client.plugins.d... >> cmd: [2019-09-14 20:11:53.450763 -0400][Debug ][PlugInMgr ] Unable to process directory /tmp/runos_singularity_home-310711236289025070240217978766146744805/.xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2019-09-14 20:11:53.450934 -0400][Debug ][App ] Executing: stat /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1728.root >> cmd: [2019-09-14 20:11:53.451052 -0400][Debug ][Poller ] Available pollers: built-in >> cmd: [2019-09-14 20:11:53.451273 -0400][Debug ][Poller ] Attempting to create a poller according to preference: built-in >> cmd: [2019-09-14 20:11:53.451331 -0400][Debug ][Poller ] Creating poller: built-in >> cmd: [2019-09-14 20:11:53.451377 -0400][Debug ][Poller ] Creating and starting the built-in poller... >> cmd: [2019-09-14 20:11:53.451729 -0400][Debug ][Poller ] Using 1 poller threads >> cmd: [2019-09-14 20:11:53.451826 -0400][Debug ][TaskMgr ] Starting the task manager... >> cmd: [2019-09-14 20:11:53.452031 -0400][Debug ][TaskMgr ] Task manager started >> cmd: [2019-09-14 20:11:53.452105 -0400][Debug ][JobMgr ] Starting the job manager... >> cmd: [2019-09-14 20:11:53.452500 -0400][Debug ][JobMgr ] Job manager started, 3 workers >> cmd: [2019-09-14 20:11:53.452573 -0400][Debug ][TaskMgr ] Registering task: "FileTimer task" to be run at: [2019-09-14 20:11:53 -0400] >> cmd: [2019-09-14 20:11:53.452641 -0400][Debug ][PostMaster ] Creating new channel to: deepthought.crc.nd.edu:1094 1 stream(s) >> cmd: [2019-09-14 20:11:53.452695 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800 >> cmd: [2019-09-14 20:11:53.453712 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" to be run at: [2019-09-14 20:12:08 -0400] >> cmd: [2019-09-14 20:11:53.454881 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094] Found 1 address(es): [::ffff:129.74.85.55]:1094 >> cmd: [2019-09-14 20:11:53.455025 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:129.74.85.55]:1094 >> cmd: [2019-09-14 20:11:53.455135 -0400][Debug ][Poller ] Adding socket 0x74b8a0 to the poller >> cmd: [2019-09-14 20:11:53.455413 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Async connection call returned >> cmd: [2019-09-14 20:11:53.455553 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2019-09-14 20:11:53.456068 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: manager [], protocol version 310) >> cmd: [2019-09-14 20:11:53.456295 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: manager [], protocol version 310) >> cmd: [2019-09-14 20:11:53.459609 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d12chas574.crc.nd.edu&xrd.rn=v4.6.1, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2019-09-14 20:11:53.459971 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Logged in, session: fa0a00008556370035000000040b0000 >> cmd: [2019-09-14 20:11:53.460059 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream 0 connected. >> cmd: [2019-09-14 20:11:53.460116 -0400][Debug ][Utility ] Monitor library name not set. No monitoring >> cmd: [2019-09-14 20:11:53.460739 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant03.crc.nd.edu:1096 1 stream(s) >> cmd: [2019-09-14 20:11:53.460812 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800 >> cmd: [2019-09-14 20:11:53.462186 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant03.crc.nd.edu:1096" to be run at: [2019-09-14 20:12:08 -0400] >> cmd: [2019-09-14 20:11:53.463161 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.11]:1096 >> cmd: [2019-09-14 20:11:53.463465 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.11]:1096 >> cmd: [2019-09-14 20:11:53.463546 -0400][Debug ][Poller ] Adding socket 0x3c0016a0 to the poller >> cmd: [2019-09-14 20:11:53.463693 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2019-09-14 20:11:53.463788 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2019-09-14 20:11:53.464218 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 310) >> cmd: [2019-09-14 20:11:53.464415 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 310) >> cmd: [2019-09-14 20:11:53.467139 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d12chas574.crc.nd.edu&xrd.rn=v4.6.1, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2019-09-14 20:11:53.467692 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Logged in, session: 6a0b0000860c00001b000000650c0000 >> cmd: [2019-09-14 20:11:53.467754 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10300,c:ssl,ca: >> cmd: [2019-09-14 20:11:53.467822 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2019-09-14 20:11:53.474266 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: 190914 20:11:53 31318 secgsi_ParseCAlist: nothing to parse >> cmd: [2019-09-14 20:11:53.508357 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Cannot get credentials for protocol gsi: Secgsi: ErrParseBuffer: unknown CA: cannot verify server certificate: kXGS_init >> cmd: [2019-09-14 20:11:53.508422 -0400][Error ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] No protocols left to try >> cmd: [2019-09-14 20:11:53.508468 -0400][Error ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Socket error while handshaking: [FATAL] Auth failed >> cmd: [2019-09-14 20:11:53.508505 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:53.508580 -0400][Debug ][Poller ] <[::ffff:10.32.86.143]:45490><--><[::ffff:149.165.243.11]:1096> Removing socket from the poller >> cmd: [2019-09-14 20:11:53.508770 -0400][Error ][PostMaster ] [primeradiant03.crc.nd.edu:1096 #0] Unable to recover: [FATAL] Auth failed. >> cmd: [2019-09-14 20:11:53.508827 -0400][Error ][XRootD ] [primeradiant03.crc.nd.edu:1096] Impossible to send message kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1728.root, flags: none). Trying to recover. >> cmd: [2019-09-14 20:11:53.508874 -0400][Debug ][XRootD ] [primeradiant03.crc.nd.edu:1096] Handling error while processing kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1728.root, flags: none): [FATAL] Auth failed. >> cmd: [2019-09-14 20:11:53.510184 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant02.crc.nd.edu:1096 1 stream(s) >> cmd: [2019-09-14 20:11:53.510283 -0400][Debug ][PostMaster ] [primeradiant02.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800 >> cmd: [2019-09-14 20:11:53.512026 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant02.crc.nd.edu:1096" to be run at: [2019-09-14 20:12:08 -0400] >> cmd: [2019-09-14 20:11:53.513709 -0400][Debug ][PostMaster ] [primeradiant02.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.6]:1096 >> cmd: [2019-09-14 20:11:53.513882 -0400][Debug ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.6]:1096 >> cmd: [2019-09-14 20:11:53.514012 -0400][Debug ][Poller ] Adding socket 0x40001540 to the poller >> cmd: [2019-09-14 20:11:53.514313 -0400][Debug ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2019-09-14 20:11:53.514446 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2019-09-14 20:11:53.514975 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 310) >> cmd: [2019-09-14 20:11:53.515466 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 310) >> cmd: [2019-09-14 20:11:53.519185 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d12chas574.crc.nd.edu&xrd.rn=v4.6.1, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2019-09-14 20:11:53.520062 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Logged in, session: 5d0b00009b0c00001b000000970c0000 >> cmd: [2019-09-14 20:11:53.520144 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10300,c:ssl,ca: >> cmd: [2019-09-14 20:11:53.520234 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2019-09-14 20:11:53.522357 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: 190914 20:11:53 31318 secgsi_ParseCAlist: nothing to parse >> cmd: [2019-09-14 20:11:53.522732 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Cannot get credentials for protocol gsi: Secgsi: ErrParseBuffer: unknown CA: cannot verify server certificate: kXGS_init >> cmd: [2019-09-14 20:11:53.522962 -0400][Error ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] No protocols left to try >> cmd: [2019-09-14 20:11:53.523031 -0400][Error ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Socket error while handshaking: [FATAL] Auth failed >> cmd: [2019-09-14 20:11:53.523083 -0400][Debug ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:53.523188 -0400][Debug ][Poller ] <[::ffff:10.32.86.143]:47164><--><[::ffff:149.165.243.6]:1096> Removing socket from the poller >> cmd: [2019-09-14 20:11:53.523331 -0400][Error ][PostMaster ] [primeradiant02.crc.nd.edu:1096 #0] Unable to recover: [FATAL] Auth failed. >> cmd: [2019-09-14 20:11:53.523370 -0400][Error ][XRootD ] [primeradiant02.crc.nd.edu:1096] Impossible to send message kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1728.root?tried=primeradiant03.crc.nd.edu, flags: none). Trying to recover. >> cmd: [2019-09-14 20:11:53.523411 -0400][Debug ][XRootD ] [primeradiant02.crc.nd.edu:1096] Handling error while processing kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1728.root?tried=primeradiant03.crc.nd.edu, flags: none): [FATAL] Auth failed. >> cmd: [2019-09-14 20:11:53.525289 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant05.crc.nd.edu:1096 1 stream(s) >> cmd: [2019-09-14 20:11:53.525360 -0400][Debug ][PostMaster ] [primeradiant05.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800 >> cmd: [2019-09-14 20:11:53.526699 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant05.crc.nd.edu:1096" to be run at: [2019-09-14 20:12:08 -0400] >> cmd: [2019-09-14 20:11:53.527764 -0400][Debug ][PostMaster ] [primeradiant05.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.13]:1096 >> cmd: [2019-09-14 20:11:53.527911 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.13]:1096 >> cmd: [2019-09-14 20:11:53.528018 -0400][Debug ][Poller ] Adding socket 0x44001540 to the poller >> cmd: [2019-09-14 20:11:53.528138 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2019-09-14 20:11:53.528246 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2019-09-14 20:11:53.528642 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 310) >> cmd: [2019-09-14 20:11:53.528871 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 310) >> cmd: [2019-09-14 20:11:53.531695 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d12chas574.crc.nd.edu&xrd.rn=v4.6.1, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2019-09-14 20:11:53.532119 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Logged in, session: 590b0000780c00001c000000580c0000 >> cmd: [2019-09-14 20:11:53.532183 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10300,c:ssl,ca: >> cmd: [2019-09-14 20:11:53.532256 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2019-09-14 20:11:53.533378 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: 190914 20:11:53 31318 secgsi_ParseCAlist: nothing to parse >> cmd: [2019-09-14 20:11:53.533844 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Cannot get credentials for protocol gsi: Secgsi: ErrParseBuffer: unknown CA: cannot verify server certificate: kXGS_init >> cmd: [2019-09-14 20:11:53.533924 -0400][Error ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] No protocols left to try >> cmd: [2019-09-14 20:11:53.533997 -0400][Error ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Socket error while handshaking: [FATAL] Auth failed >> cmd: [2019-09-14 20:11:53.534051 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:53.534121 -0400][Debug ][Poller ] <[::ffff:10.32.86.143]:57872><--><[::ffff:149.165.243.13]:1096> Removing socket from the poller >> cmd: [2019-09-14 20:11:53.534288 -0400][Error ][PostMaster ] [primeradiant05.crc.nd.edu:1096 #0] Unable to recover: [FATAL] Auth failed. >> cmd: [2019-09-14 20:11:53.534344 -0400][Error ][XRootD ] [primeradiant05.crc.nd.edu:1096] Impossible to send message kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1728.root?tried=primeradiant03.crc.nd.edu,primeradiant02.crc.nd.edu, flags: none). Trying to recover. >> cmd: [2019-09-14 20:11:53.534382 -0400][Debug ][XRootD ] [primeradiant05.crc.nd.edu:1096] Handling error while processing kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1728.root?tried=primeradiant03.crc.nd.edu,primeradiant02.crc.nd.edu, flags: none): [FATAL] Auth failed. >> cmd: [2019-09-14 20:11:53.536327 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant04.crc.nd.edu:1096 1 stream(s) >> cmd: [2019-09-14 20:11:53.536404 -0400][Debug ][PostMaster ] [primeradiant04.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800 >> cmd: [2019-09-14 20:11:53.537841 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant04.crc.nd.edu:1096" to be run at: [2019-09-14 20:12:08 -0400] >> cmd: [2019-09-14 20:11:53.538832 -0400][Debug ][PostMaster ] [primeradiant04.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.12]:1096 >> cmd: [2019-09-14 20:11:53.538961 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.12]:1096 >> cmd: [2019-09-14 20:11:53.539070 -0400][Debug ][Poller ] Adding socket 0x3c001d20 to the poller >> cmd: [2019-09-14 20:11:53.539192 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2019-09-14 20:11:53.539298 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2019-09-14 20:11:53.539677 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 310) >> cmd: [2019-09-14 20:11:53.539861 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 310) >> cmd: [2019-09-14 20:11:53.542253 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d12chas574.crc.nd.edu&xrd.rn=v4.6.1, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2019-09-14 20:11:53.542729 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Logged in, session: a70a0000a00c00001b000000a70a0000 >> cmd: [2019-09-14 20:11:53.542787 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10300,c:ssl,ca: >> cmd: [2019-09-14 20:11:53.542825 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2019-09-14 20:11:53.544159 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: 190914 20:11:53 31318 secgsi_ParseCAlist: nothing to parse >> cmd: [2019-09-14 20:11:53.544532 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Cannot get credentials for protocol gsi: Secgsi: ErrParseBuffer: unknown CA: cannot verify server certificate: kXGS_init >> cmd: [2019-09-14 20:11:53.544575 -0400][Error ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] No protocols left to try >> cmd: [2019-09-14 20:11:53.544632 -0400][Error ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Socket error while handshaking: [FATAL] Auth failed >> cmd: [2019-09-14 20:11:53.544669 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:53.544703 -0400][Debug ][Poller ] <[::ffff:10.32.86.143]:56932><--><[::ffff:149.165.243.12]:1096> Removing socket from the poller >> cmd: [2019-09-14 20:11:53.544843 -0400][Error ][PostMaster ] [primeradiant04.crc.nd.edu:1096 #0] Unable to recover: [FATAL] Auth failed. >> cmd: [2019-09-14 20:11:53.544881 -0400][Error ][XRootD ] [primeradiant04.crc.nd.edu:1096] Impossible to send message kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1728.root?tried=primeradiant03.crc.nd.edu,primeradiant02.crc.nd.edu,primeradiant05.crc.nd.edu, flags: none). Trying to recover. >> cmd: [2019-09-14 20:11:53.544916 -0400][Debug ][XRootD ] [primeradiant04.crc.nd.edu:1096] Handling error while processing kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1728.root?tried=primeradiant03.crc.nd.edu,primeradiant02.crc.nd.edu,primeradiant05.crc.nd.edu, flags: none): [FATAL] Auth failed. >> cmd: [2019-09-14 20:11:53.547755 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant01.crc.nd.edu:1096 1 stream(s) >> cmd: [2019-09-14 20:11:53.547829 -0400][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800 >> cmd: [2019-09-14 20:11:53.549104 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant01.crc.nd.edu:1096" to be run at: [2019-09-14 20:12:08 -0400] >> cmd: [2019-09-14 20:11:53.550228 -0400][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.5]:1096 >> cmd: [2019-09-14 20:11:53.550359 -0400][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.5]:1096 >> cmd: [2019-09-14 20:11:53.550475 -0400][Debug ][Poller ] Adding socket 0x40001e10 to the poller >> cmd: [2019-09-14 20:11:53.550614 -0400][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2019-09-14 20:11:53.550744 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2019-09-14 20:11:53.551356 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 310) >> cmd: [2019-09-14 20:11:53.551595 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 310) >> cmd: [2019-09-14 20:11:53.554911 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d12chas574.crc.nd.edu&xrd.rn=v4.6.1, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2019-09-14 20:11:53.555386 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Logged in, session: da0b00009d0c00001d0000000a0d0000 >> cmd: [2019-09-14 20:11:53.555446 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10300,c:ssl,ca: >> cmd: [2019-09-14 20:11:53.555518 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2019-09-14 20:11:53.556701 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: 190914 20:11:53 31318 secgsi_ParseCAlist: nothing to parse >> cmd: [2019-09-14 20:11:53.557033 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Cannot get credentials for protocol gsi: Secgsi: ErrParseBuffer: unknown CA: cannot verify server certificate: kXGS_init >> cmd: [2019-09-14 20:11:53.557085 -0400][Error ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] No protocols left to try >> cmd: [2019-09-14 20:11:53.557126 -0400][Error ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Socket error while handshaking: [FATAL] Auth failed >> cmd: [2019-09-14 20:11:53.557179 -0400][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:53.557229 -0400][Debug ][Poller ] <[::ffff:10.32.86.143]:54096><--><[::ffff:149.165.243.5]:1096> Removing socket from the poller >> cmd: [2019-09-14 20:11:53.557375 -0400][Error ][PostMaster ] [primeradiant01.crc.nd.edu:1096 #0] Unable to recover: [FATAL] Auth failed. >> cmd: [2019-09-14 20:11:53.557416 -0400][Error ][XRootD ] [primeradiant01.crc.nd.edu:1096] Impossible to send message kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1728.root?tried=primeradiant03.crc.nd.edu,primeradiant02.crc.nd.edu,primeradiant05.crc.nd.edu,primeradiant04.crc.nd.edu, flags: none). Trying to recover. >> cmd: [2019-09-14 20:11:53.557464 -0400][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1096] Handling error while processing kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1728.root?tried=primeradiant03.crc.nd.edu,primeradiant02.crc.nd.edu,primeradiant05.crc.nd.edu,primeradiant04.crc.nd.edu, flags: none): [FATAL] Auth failed. >> cmd: [2019-09-14 20:11:53.560320 -0400][Debug ][XRootD ] [deepthought.crc.nd.edu:1094] Handling error while processing kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1728.root?tried=primeradiant03.crc.nd.edu,primeradiant02.crc.nd.edu,primeradiant05.crc.nd.edu,primeradiant04.crc.nd.edu,primeradiant01.crc.nd.edu, flags: none): [ERROR] Error response: No such file or directory. >> cmd: [2019-09-14 20:11:53.560496 -0400][Error ][App ] Unable stat /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1728.root: [ERROR] Server responded with an error: [3011] No servers have read access to the file >> cmd: [ERROR] Server responded with an error: [3011] No servers have read access to the file >> cmd: >> cmd: [2019-09-14 20:11:53.560749 -0400][Debug ][JobMgr ] Stopping the job manager... >> cmd: [2019-09-14 20:11:53.561578 -0400][Debug ][JobMgr ] Job manager stopped >> cmd: [2019-09-14 20:11:53.561644 -0400][Debug ][TaskMgr ] Stopping the task manager... >> cmd: [2019-09-14 20:11:53.561873 -0400][Debug ][TaskMgr ] Task manager stopped >> cmd: [2019-09-14 20:11:53.561941 -0400][Debug ][Poller ] Stopping the poller... >> cmd: [2019-09-14 20:11:53.562292 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" >> cmd: [2019-09-14 20:11:53.562346 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:53.562396 -0400][Debug ][Poller ] <[::ffff:10.32.86.143]:50854><--><[::ffff:129.74.85.55]:1094> Removing socket from the poller >> cmd: [2019-09-14 20:11:53.562490 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Destroying stream >> cmd: [2019-09-14 20:11:53.562534 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:53.562584 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant01.crc.nd.edu:1096" >> cmd: [2019-09-14 20:11:53.562640 -0400][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:53.562684 -0400][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2019-09-14 20:11:53.562731 -0400][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:53.562774 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant02.crc.nd.edu:1096" >> cmd: [2019-09-14 20:11:53.562812 -0400][Debug ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:53.562869 -0400][Debug ][PostMaster ] [primeradiant02.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2019-09-14 20:11:53.562910 -0400][Debug ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:53.562968 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant03.crc.nd.edu:1096" >> cmd: [2019-09-14 20:11:53.563016 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:53.563055 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2019-09-14 20:11:53.563104 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:53.563147 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant04.crc.nd.edu:1096" >> cmd: [2019-09-14 20:11:53.563184 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:53.563221 -0400][Debug ][PostMaster ] [primeradiant04.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2019-09-14 20:11:53.563272 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:53.563325 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant05.crc.nd.edu:1096" >> cmd: [2019-09-14 20:11:53.563367 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:11:53.563403 -0400][Debug ][PostMaster ] [primeradiant05.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2019-09-14 20:11:53.563455 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Closing the socket >>> xrootd access to input file unavailable @ Sat Sep 14 20:11:53 2019 >>> Trying srm access method @ Sat Sep 14 20:11:53 2019 >>> executing '/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/bin/gfal-copy gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1728.root HIG-RunIIFall17wmLHE-00000ND_1728.root' @ Sat Sep 14 20:11:53 2019 >> using /disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME/tmpfkZrzx to store command output >> cmd: gfal-copy error: 13 (Permission denied) - Could not stat the source: globus_ftp_client: the server responded with an error 530 530-Login incorrect. : /builddir/build/BUILD/gridftp-hdfs-1.1.1/src/gridftp_hdfs.c:hdfs_start:756: 530-Server is cancelling transfer due to over-load limit (host=deepthought.crc.nd.edu, user=uscms01, path=(null)) 530 End. >>>>> Unable to copy input with SRM @ Sat Sep 14 20:11:56 2019 >>> Trying srm access method @ Sat Sep 14 20:11:56 2019 >>> executing '/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/bin/gfal-copy gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1728.root HIG-RunIIFall17wmLHE-00000ND_1728.root' @ Sat Sep 14 20:11:56 2019 >> using /disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME/tmpafcoNh to store command output >> cmd: gfal-copy error: 13 (Permission denied) - Could not stat the source: globus_ftp_client: the server responded with an error 530 530-Login incorrect. : /builddir/build/BUILD/gridftp-hdfs-1.1.1/src/gridftp_hdfs.c:hdfs_start:756: 530-Server is cancelling transfer due to over-load limit (host=deepthought.crc.nd.edu, user=uscms01, path=(null)) 530 End. >>>>> Unable to copy input with SRM @ Sat Sep 14 20:11:59 2019 >>>>>> no stage out method succeeded for: kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1728.root @ Sat Sep 14 20:11:59 2019 >>> Trying hdfs client access method @ Sat Sep 14 20:11:59 2019 >>> executing 'timeout 300 hdfs dfs -fs hdfs://eddie.crc.nd.edu:19000 -get /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1924.root HIG-RunIIFall17wmLHE-00000ND_1924.root' @ Sat Sep 14 20:11:59 2019 >> using /disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME/tmp1SZROY to store command output >> cmd: timeout: failed to run command 'hdfs': No such file or directory >>>>> Unable to copy input with hdfs client @ Sat Sep 14 20:12:00 2019 >>> Trying xrootd access method @ Sat Sep 14 20:12:00 2019 >>> executing 'env XRD_LOGLEVEL=Debug timeout 300 xrdfs deepthought.crc.nd.edu stat /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1924.root' @ Sat Sep 14 20:12:00 2019 >> using /disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME/tmpz5_QTO to store command output >> cmd: [2019-09-14 20:12:00.188321 -0400][Warning][Utility ] Unable to process global config file: [ERROR] OS Error: No such file or directory >> cmd: [2019-09-14 20:12:00.189378 -0400][Debug ][Utility ] Unable to process user config file: [ERROR] OS Error: No such file or directory >> cmd: [2019-09-14 20:12:00.189549 -0400][Debug ][PlugInMgr ] Initializing plug-in manager... >> cmd: [2019-09-14 20:12:00.189630 -0400][Debug ][PlugInMgr ] No default plug-in, loading plug-in configs... >> cmd: [2019-09-14 20:12:00.189677 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /etc/xrootd/client.plugins.d... >> cmd: [2019-09-14 20:12:00.189802 -0400][Debug ][PlugInMgr ] Unable to process directory /etc/xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2019-09-14 20:12:00.190148 -0400][Debug ][PlugInMgr ] Processing plug-in definitions in /tmp/runos_singularity_home-310711236289025070240217978766146744805/.xrootd/client.plugins.d... >> cmd: [2019-09-14 20:12:00.190252 -0400][Debug ][PlugInMgr ] Unable to process directory /tmp/runos_singularity_home-310711236289025070240217978766146744805/.xrootd/client.plugins.d: [ERROR] OS Error: No such file or directory >> cmd: [2019-09-14 20:12:00.190427 -0400][Debug ][App ] Executing: stat /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1924.root >> cmd: [2019-09-14 20:12:00.190528 -0400][Debug ][Poller ] Available pollers: built-in >> cmd: [2019-09-14 20:12:00.190582 -0400][Debug ][Poller ] Attempting to create a poller according to preference: built-in >> cmd: [2019-09-14 20:12:00.190639 -0400][Debug ][Poller ] Creating poller: built-in >> cmd: [2019-09-14 20:12:00.190704 -0400][Debug ][Poller ] Creating and starting the built-in poller... >> cmd: [2019-09-14 20:12:00.191198 -0400][Debug ][Poller ] Using 1 poller threads >> cmd: [2019-09-14 20:12:00.191269 -0400][Debug ][TaskMgr ] Starting the task manager... >> cmd: [2019-09-14 20:12:00.191447 -0400][Debug ][TaskMgr ] Task manager started >> cmd: [2019-09-14 20:12:00.191505 -0400][Debug ][JobMgr ] Starting the job manager... >> cmd: [2019-09-14 20:12:00.191951 -0400][Debug ][JobMgr ] Job manager started, 3 workers >> cmd: [2019-09-14 20:12:00.192021 -0400][Debug ][TaskMgr ] Registering task: "FileTimer task" to be run at: [2019-09-14 20:12:00 -0400] >> cmd: [2019-09-14 20:12:00.192089 -0400][Debug ][PostMaster ] Creating new channel to: deepthought.crc.nd.edu:1094 1 stream(s) >> cmd: [2019-09-14 20:12:00.192149 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800 >> cmd: [2019-09-14 20:12:00.193634 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" to be run at: [2019-09-14 20:12:15 -0400] >> cmd: [2019-09-14 20:12:00.195028 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094] Found 1 address(es): [::ffff:129.74.85.55]:1094 >> cmd: [2019-09-14 20:12:00.195190 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:129.74.85.55]:1094 >> cmd: [2019-09-14 20:12:00.195310 -0400][Debug ][Poller ] Adding socket 0x11968a0 to the poller >> cmd: [2019-09-14 20:12:00.197066 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Async connection call returned >> cmd: [2019-09-14 20:12:00.197208 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2019-09-14 20:12:00.197661 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: manager [], protocol version 310) >> cmd: [2019-09-14 20:12:00.197885 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: manager [], protocol version 310) >> cmd: [2019-09-14 20:12:00.201435 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d12chas574.crc.nd.edu&xrd.rn=v4.6.1, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2019-09-14 20:12:00.201836 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094 #0.0] Logged in, session: 420b000085563700180000004c0b0000 >> cmd: [2019-09-14 20:12:00.201924 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Stream 0 connected. >> cmd: [2019-09-14 20:12:00.201989 -0400][Debug ][Utility ] Monitor library name not set. No monitoring >> cmd: [2019-09-14 20:12:00.202618 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant03.crc.nd.edu:1096 1 stream(s) >> cmd: [2019-09-14 20:12:00.202676 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800 >> cmd: [2019-09-14 20:12:00.203895 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant03.crc.nd.edu:1096" to be run at: [2019-09-14 20:12:15 -0400] >> cmd: [2019-09-14 20:12:00.204967 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.11]:1096 >> cmd: [2019-09-14 20:12:00.205120 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.11]:1096 >> cmd: [2019-09-14 20:12:00.205211 -0400][Debug ][Poller ] Adding socket 0x940016a0 to the poller >> cmd: [2019-09-14 20:12:00.205341 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2019-09-14 20:12:00.205440 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2019-09-14 20:12:00.205799 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 310) >> cmd: [2019-09-14 20:12:00.206023 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 310) >> cmd: [2019-09-14 20:12:00.208843 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d12chas574.crc.nd.edu&xrd.rn=v4.6.1, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2019-09-14 20:12:00.209240 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Logged in, session: b30b0000860c00001b000000ae0c0000 >> cmd: [2019-09-14 20:12:00.209313 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10300,c:ssl,ca: >> cmd: [2019-09-14 20:12:00.209371 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2019-09-14 20:12:00.215448 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: 190914 20:12:00 31906 secgsi_ParseCAlist: nothing to parse >> cmd: [2019-09-14 20:12:00.246124 -0400][Debug ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] Cannot get credentials for protocol gsi: Secgsi: ErrParseBuffer: unknown CA: cannot verify server certificate: kXGS_init >> cmd: [2019-09-14 20:12:00.246210 -0400][Error ][XRootDTransport ] [primeradiant03.crc.nd.edu:1096 #0.0] No protocols left to try >> cmd: [2019-09-14 20:12:00.246274 -0400][Error ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Socket error while handshaking: [FATAL] Auth failed >> cmd: [2019-09-14 20:12:00.246334 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:12:00.246400 -0400][Debug ][Poller ] <[::ffff:10.32.86.143]:45766><--><[::ffff:149.165.243.11]:1096> Removing socket from the poller >> cmd: [2019-09-14 20:12:00.246540 -0400][Error ][PostMaster ] [primeradiant03.crc.nd.edu:1096 #0] Unable to recover: [FATAL] Auth failed. >> cmd: [2019-09-14 20:12:00.246606 -0400][Error ][XRootD ] [primeradiant03.crc.nd.edu:1096] Impossible to send message kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1924.root, flags: none). Trying to recover. >> cmd: [2019-09-14 20:12:00.246654 -0400][Debug ][XRootD ] [primeradiant03.crc.nd.edu:1096] Handling error while processing kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1924.root, flags: none): [FATAL] Auth failed. >> cmd: [2019-09-14 20:12:00.248282 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant02.crc.nd.edu:1096 1 stream(s) >> cmd: [2019-09-14 20:12:00.248370 -0400][Debug ][PostMaster ] [primeradiant02.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800 >> cmd: [2019-09-14 20:12:00.249965 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant02.crc.nd.edu:1096" to be run at: [2019-09-14 20:12:15 -0400] >> cmd: [2019-09-14 20:12:00.251388 -0400][Debug ][PostMaster ] [primeradiant02.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.6]:1096 >> cmd: [2019-09-14 20:12:00.251719 -0400][Debug ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.6]:1096 >> cmd: [2019-09-14 20:12:00.251842 -0400][Debug ][Poller ] Adding socket 0x9c001540 to the poller >> cmd: [2019-09-14 20:12:00.252021 -0400][Debug ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2019-09-14 20:12:00.252156 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2019-09-14 20:12:00.252801 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 310) >> cmd: [2019-09-14 20:12:00.253080 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 310) >> cmd: [2019-09-14 20:12:00.257104 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d12chas574.crc.nd.edu&xrd.rn=v4.6.1, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2019-09-14 20:12:00.257746 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Logged in, session: a80b00009b0c00001b000000e20c0000 >> cmd: [2019-09-14 20:12:00.257804 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10300,c:ssl,ca: >> cmd: [2019-09-14 20:12:00.257869 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2019-09-14 20:12:00.259503 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: 190914 20:12:00 31906 secgsi_ParseCAlist: nothing to parse >> cmd: [2019-09-14 20:12:00.260303 -0400][Debug ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] Cannot get credentials for protocol gsi: Secgsi: ErrParseBuffer: unknown CA: cannot verify server certificate: kXGS_init >> cmd: [2019-09-14 20:12:00.260541 -0400][Error ][XRootDTransport ] [primeradiant02.crc.nd.edu:1096 #0.0] No protocols left to try >> cmd: [2019-09-14 20:12:00.260607 -0400][Error ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Socket error while handshaking: [FATAL] Auth failed >> cmd: [2019-09-14 20:12:00.260680 -0400][Debug ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:12:00.260734 -0400][Debug ][Poller ] <[::ffff:10.32.86.143]:47440><--><[::ffff:149.165.243.6]:1096> Removing socket from the poller >> cmd: [2019-09-14 20:12:00.260936 -0400][Error ][PostMaster ] [primeradiant02.crc.nd.edu:1096 #0] Unable to recover: [FATAL] Auth failed. >> cmd: [2019-09-14 20:12:00.261009 -0400][Error ][XRootD ] [primeradiant02.crc.nd.edu:1096] Impossible to send message kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1924.root?tried=primeradiant03.crc.nd.edu, flags: none). Trying to recover. >> cmd: [2019-09-14 20:12:00.261088 -0400][Debug ][XRootD ] [primeradiant02.crc.nd.edu:1096] Handling error while processing kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1924.root?tried=primeradiant03.crc.nd.edu, flags: none): [FATAL] Auth failed. >> cmd: [2019-09-14 20:12:00.262685 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant05.crc.nd.edu:1096 1 stream(s) >> cmd: [2019-09-14 20:12:00.262765 -0400][Debug ][PostMaster ] [primeradiant05.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800 >> cmd: [2019-09-14 20:12:00.264429 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant05.crc.nd.edu:1096" to be run at: [2019-09-14 20:12:15 -0400] >> cmd: [2019-09-14 20:12:00.265830 -0400][Debug ][PostMaster ] [primeradiant05.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.13]:1096 >> cmd: [2019-09-14 20:12:00.265992 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.13]:1096 >> cmd: [2019-09-14 20:12:00.266100 -0400][Debug ][Poller ] Adding socket 0xa0001540 to the poller >> cmd: [2019-09-14 20:12:00.266232 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2019-09-14 20:12:00.266346 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2019-09-14 20:12:00.266756 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 310) >> cmd: [2019-09-14 20:12:00.266977 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 310) >> cmd: [2019-09-14 20:12:00.270029 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d12chas574.crc.nd.edu&xrd.rn=v4.6.1, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2019-09-14 20:12:00.270473 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Logged in, session: a30b0000780c00001b000000a20c0000 >> cmd: [2019-09-14 20:12:00.270533 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10300,c:ssl,ca: >> cmd: [2019-09-14 20:12:00.270587 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2019-09-14 20:12:00.271885 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: 190914 20:12:00 31906 secgsi_ParseCAlist: nothing to parse >> cmd: [2019-09-14 20:12:00.272265 -0400][Debug ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] Cannot get credentials for protocol gsi: Secgsi: ErrParseBuffer: unknown CA: cannot verify server certificate: kXGS_init >> cmd: [2019-09-14 20:12:00.272330 -0400][Error ][XRootDTransport ] [primeradiant05.crc.nd.edu:1096 #0.0] No protocols left to try >> cmd: [2019-09-14 20:12:00.272394 -0400][Error ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Socket error while handshaking: [FATAL] Auth failed >> cmd: [2019-09-14 20:12:00.272439 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:12:00.272492 -0400][Debug ][Poller ] <[::ffff:10.32.86.143]:58148><--><[::ffff:149.165.243.13]:1096> Removing socket from the poller >> cmd: [2019-09-14 20:12:00.272827 -0400][Error ][PostMaster ] [primeradiant05.crc.nd.edu:1096 #0] Unable to recover: [FATAL] Auth failed. >> cmd: [2019-09-14 20:12:00.272892 -0400][Error ][XRootD ] [primeradiant05.crc.nd.edu:1096] Impossible to send message kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1924.root?tried=primeradiant03.crc.nd.edu,primeradiant02.crc.nd.edu, flags: none). Trying to recover. >> cmd: [2019-09-14 20:12:00.273214 -0400][Debug ][XRootD ] [primeradiant05.crc.nd.edu:1096] Handling error while processing kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1924.root?tried=primeradiant03.crc.nd.edu,primeradiant02.crc.nd.edu, flags: none): [FATAL] Auth failed. >> cmd: [2019-09-14 20:12:00.275776 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant04.crc.nd.edu:1096 1 stream(s) >> cmd: [2019-09-14 20:12:00.275844 -0400][Debug ][PostMaster ] [primeradiant04.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800 >> cmd: [2019-09-14 20:12:00.277392 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant04.crc.nd.edu:1096" to be run at: [2019-09-14 20:12:15 -0400] >> cmd: [2019-09-14 20:12:00.278841 -0400][Debug ][PostMaster ] [primeradiant04.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.12]:1096 >> cmd: [2019-09-14 20:12:00.279007 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.12]:1096 >> cmd: [2019-09-14 20:12:00.279107 -0400][Debug ][Poller ] Adding socket 0x94001d20 to the poller >> cmd: [2019-09-14 20:12:00.279224 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2019-09-14 20:12:00.279343 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2019-09-14 20:12:00.279793 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 310) >> cmd: [2019-09-14 20:12:00.280010 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 310) >> cmd: [2019-09-14 20:12:00.283169 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d12chas574.crc.nd.edu&xrd.rn=v4.6.1, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2019-09-14 20:12:00.283614 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Logged in, session: ef0a0000a00c00001b000000ef0a0000 >> cmd: [2019-09-14 20:12:00.283669 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10300,c:ssl,ca: >> cmd: [2019-09-14 20:12:00.283741 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2019-09-14 20:12:00.284939 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: 190914 20:12:00 31906 secgsi_ParseCAlist: nothing to parse >> cmd: [2019-09-14 20:12:00.285420 -0400][Debug ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] Cannot get credentials for protocol gsi: Secgsi: ErrParseBuffer: unknown CA: cannot verify server certificate: kXGS_init >> cmd: [2019-09-14 20:12:00.285490 -0400][Error ][XRootDTransport ] [primeradiant04.crc.nd.edu:1096 #0.0] No protocols left to try >> cmd: [2019-09-14 20:12:00.285538 -0400][Error ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Socket error while handshaking: [FATAL] Auth failed >> cmd: [2019-09-14 20:12:00.285594 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:12:00.285667 -0400][Debug ][Poller ] <[::ffff:10.32.86.143]:57208><--><[::ffff:149.165.243.12]:1096> Removing socket from the poller >> cmd: [2019-09-14 20:12:00.285835 -0400][Error ][PostMaster ] [primeradiant04.crc.nd.edu:1096 #0] Unable to recover: [FATAL] Auth failed. >> cmd: [2019-09-14 20:12:00.285907 -0400][Error ][XRootD ] [primeradiant04.crc.nd.edu:1096] Impossible to send message kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1924.root?tried=primeradiant03.crc.nd.edu,primeradiant02.crc.nd.edu,primeradiant05.crc.nd.edu, flags: none). Trying to recover. >> cmd: [2019-09-14 20:12:00.285967 -0400][Debug ][XRootD ] [primeradiant04.crc.nd.edu:1096] Handling error while processing kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1924.root?tried=primeradiant03.crc.nd.edu,primeradiant02.crc.nd.edu,primeradiant05.crc.nd.edu, flags: none): [FATAL] Auth failed. >> cmd: [2019-09-14 20:12:00.296878 -0400][Debug ][PostMaster ] Creating new channel to: primeradiant01.crc.nd.edu:1096 1 stream(s) >> cmd: [2019-09-14 20:12:00.296970 -0400][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1096 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Widnow: 1800 >> cmd: [2019-09-14 20:12:00.298460 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant01.crc.nd.edu:1096" to be run at: [2019-09-14 20:12:15 -0400] >> cmd: [2019-09-14 20:12:00.300050 -0400][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1096] Found 1 address(es): [::ffff:149.165.243.5]:1096 >> cmd: [2019-09-14 20:12:00.300226 -0400][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Attempting connection to [::ffff:149.165.243.5]:1096 >> cmd: [2019-09-14 20:12:00.300334 -0400][Debug ][Poller ] Adding socket 0x9c001e10 to the poller >> cmd: [2019-09-14 20:12:00.300497 -0400][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Async connection call returned >> cmd: [2019-09-14 20:12:00.300648 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Sending out the initial hand shake + kXR_protocol >> cmd: [2019-09-14 20:12:00.301439 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Got the server hand shake response (type: server [], protocol version 310) >> cmd: [2019-09-14 20:12:00.301650 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] kXR_protocol successful (type: server [], protocol version 310) >> cmd: [2019-09-14 20:12:00.304166 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdfs&xrd.info=&xrd.hostname=d12chas574.crc.nd.edu&xrd.rn=v4.6.1, dual-stack: false, private IPv4: true, private IPv6: false >> cmd: [2019-09-14 20:12:00.304516 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Logged in, session: 220c00009d0c00001d000000520d0000 >> cmd: [2019-09-14 20:12:00.304575 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Authentication is required: &P=gsi,v:10300,c:ssl,ca: >> cmd: [2019-09-14 20:12:00.304617 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Sending authentication data >> cmd: [2019-09-14 20:12:00.305820 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Trying to authenticate using gsi >> cmd: 190914 20:12:00 31906 secgsi_ParseCAlist: nothing to parse >> cmd: [2019-09-14 20:12:00.306379 -0400][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] Cannot get credentials for protocol gsi: Secgsi: ErrParseBuffer: unknown CA: cannot verify server certificate: kXGS_init >> cmd: [2019-09-14 20:12:00.306624 -0400][Error ][XRootDTransport ] [primeradiant01.crc.nd.edu:1096 #0.0] No protocols left to try >> cmd: [2019-09-14 20:12:00.306697 -0400][Error ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Socket error while handshaking: [FATAL] Auth failed >> cmd: [2019-09-14 20:12:00.306769 -0400][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:12:00.306839 -0400][Debug ][Poller ] <[::ffff:10.32.86.143]:54372><--><[::ffff:149.165.243.5]:1096> Removing socket from the poller >> cmd: [2019-09-14 20:12:00.307063 -0400][Error ][PostMaster ] [primeradiant01.crc.nd.edu:1096 #0] Unable to recover: [FATAL] Auth failed. >> cmd: [2019-09-14 20:12:00.307134 -0400][Error ][XRootD ] [primeradiant01.crc.nd.edu:1096] Impossible to send message kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1924.root?tried=primeradiant03.crc.nd.edu,primeradiant02.crc.nd.edu,primeradiant05.crc.nd.edu,primeradiant04.crc.nd.edu, flags: none). Trying to recover. >> cmd: [2019-09-14 20:12:00.307248 -0400][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1096] Handling error while processing kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1924.root?tried=primeradiant03.crc.nd.edu,primeradiant02.crc.nd.edu,primeradiant05.crc.nd.edu,primeradiant04.crc.nd.edu, flags: none): [FATAL] Auth failed. >> cmd: [2019-09-14 20:12:00.311065 -0400][Debug ][XRootD ] [deepthought.crc.nd.edu:1094] Handling error while processing kXR_stat (path: /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1924.root?tried=primeradiant03.crc.nd.edu,primeradiant02.crc.nd.edu,primeradiant05.crc.nd.edu,primeradiant04.crc.nd.edu,primeradiant01.crc.nd.edu, flags: none): [ERROR] Error response: No such file or directory. >> cmd: [2019-09-14 20:12:00.311237 -0400][Error ][App ] Unable stat /store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1924.root: [ERROR] Server responded with an error: [3011] No servers have read access to the file >> cmd: [ERROR] Server responded with an error: [3011] No servers have read access to the file >> cmd: >> cmd: [2019-09-14 20:12:00.311597 -0400][Debug ][JobMgr ] Stopping the job manager... >> cmd: [2019-09-14 20:12:00.312459 -0400][Debug ][JobMgr ] Job manager stopped >> cmd: [2019-09-14 20:12:00.312524 -0400][Debug ][TaskMgr ] Stopping the task manager... >> cmd: [2019-09-14 20:12:00.312754 -0400][Debug ][TaskMgr ] Task manager stopped >> cmd: [2019-09-14 20:12:00.312815 -0400][Debug ][Poller ] Stopping the poller... >> cmd: [2019-09-14 20:12:00.313209 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: deepthought.crc.nd.edu:1094" >> cmd: [2019-09-14 20:12:00.313267 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2019-09-14 20:12:00.313304 -0400][Debug ][Poller ] <[::ffff:10.32.86.143]:51130><--><[::ffff:129.74.85.55]:1094> Removing socket from the poller >> cmd: [2019-09-14 20:12:00.313379 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094 #0] Destroying stream >> cmd: [2019-09-14 20:12:00.313445 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094 #0.0] Closing the socket >> cmd: [2019-09-14 20:12:00.313499 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant01.crc.nd.edu:1096" >> cmd: [2019-09-14 20:12:00.313551 -0400][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:12:00.313621 -0400][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2019-09-14 20:12:00.313687 -0400][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:12:00.313753 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant02.crc.nd.edu:1096" >> cmd: [2019-09-14 20:12:00.313839 -0400][Debug ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:12:00.313916 -0400][Debug ][PostMaster ] [primeradiant02.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2019-09-14 20:12:00.313997 -0400][Debug ][AsyncSock ] [primeradiant02.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:12:00.314047 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant03.crc.nd.edu:1096" >> cmd: [2019-09-14 20:12:00.314083 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:12:00.314131 -0400][Debug ][PostMaster ] [primeradiant03.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2019-09-14 20:12:00.314209 -0400][Debug ][AsyncSock ] [primeradiant03.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:12:00.314281 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant04.crc.nd.edu:1096" >> cmd: [2019-09-14 20:12:00.314365 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:12:00.314427 -0400][Debug ][PostMaster ] [primeradiant04.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2019-09-14 20:12:00.314505 -0400][Debug ][AsyncSock ] [primeradiant04.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:12:00.314735 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant05.crc.nd.edu:1096" >> cmd: [2019-09-14 20:12:00.314800 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Closing the socket >> cmd: [2019-09-14 20:12:00.314877 -0400][Debug ][PostMaster ] [primeradiant05.crc.nd.edu:1096 #0] Destroying stream >> cmd: [2019-09-14 20:12:00.314940 -0400][Debug ][AsyncSock ] [primeradiant05.crc.nd.edu:1096 #0.0] Closing the socket >>> xrootd access to input file unavailable @ Sat Sep 14 20:12:00 2019 >>> Trying srm access method @ Sat Sep 14 20:12:00 2019 >>> executing '/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/bin/gfal-copy gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1924.root HIG-RunIIFall17wmLHE-00000ND_1924.root' @ Sat Sep 14 20:12:00 2019 >> using /disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME/tmpWe1lb7 to store command output >> cmd: gfal-copy error: 13 (Permission denied) - Could not stat the source: globus_ftp_client: the server responded with an error 530 530-Login incorrect. : /builddir/build/BUILD/gridftp-hdfs-1.1.1/src/gridftp_hdfs.c:hdfs_start:756: 530-Server is cancelling transfer due to over-load limit (host=deepthought.crc.nd.edu, user=uscms01, path=(null)) 530 End. >>>>> Unable to copy input with SRM @ Sat Sep 14 20:12:03 2019 >>> Trying srm access method @ Sat Sep 14 20:12:03 2019 >>> executing '/cvmfs/oasis.opensciencegrid.org/mis/osg-wn-client/3.4/3.4.34/el7-x86_64/usr/bin/gfal-copy gsiftp://deepthought.crc.nd.edu/store/user/kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1924.root HIG-RunIIFall17wmLHE-00000ND_1924.root' @ Sat Sep 14 20:12:03 2019 >> using /disk/vc3-root/tmp/worker-230547-21579/t.730/cctools-temp-t.730.ig3lME/tmpI7H3j5 to store command output >> cmd: gfal-copy error: 13 (Permission denied) - Could not stat the source: globus_ftp_client: the server responded with an error 530 530-Login incorrect. : /builddir/build/BUILD/gridftp-hdfs-1.1.1/src/gridftp_hdfs.c:hdfs_start:756: 530-Server is cancelling transfer due to over-load limit (host=deepthought.crc.nd.edu, user=uscms01, path=(null)) 530 End. >>>>> Unable to copy input with SRM @ Sat Sep 14 20:12:06 2019 >>>>>> no stage out method succeeded for: kmohrman/LHE_step/2019_04_19/ttXJetTests-HanV4Model-xqcut10/v1/lhe_step_ttllNuNuJetNoHiggs_HanV4lModel16DttllScanpoints_run1/HIG-RunIIFall17wmLHE-00000ND_1924.root @ Sat Sep 14 20:12:06 2019 >> trace: Traceback (most recent call last): >> trace: File "task.py", line 268, in wrapper >> trace: result = fct(data, *args, **kwargs) >> trace: File "task.py", line 638, in copy_inputs >> trace: raise RuntimeError("no stage-in method succeeded") >> trace: RuntimeError: no stage-in method succeeded >>>>> call to 'copy_inputs' failed, exiting with exit code 179 @ Sat Sep 14 20:12:06 2019 >> Execution time -1568505830 >> Exiting with code 179 >> Reporting ExeExitCode 0 >> Reporting StageOutSE >> Reporting StageOutExitCode 0 Sat Sep 14 20:12:06 2019: ApMon[INFO]: Adding destination 188.184.29.203:8884 Sat Sep 14 20:12:06 2019: ApMon[DEBUG]: Overwritting option: sys_monitoring = 0 Sat Sep 14 20:12:06 2019: ApMon[DEBUG]: Overwritting option: general_info = 0 Sat Sep 14 20:12:06 2019: ApMon[DEBUG]: Overwritting option: job_monitoring = 0 Sat Sep 14 20:12:06 2019: ApMon[NOTICE]: Adding parameter StageOutExitStatusReason = Copy succedeed with srm-lcg utils Sat Sep 14 20:12:06 2019: ApMon[NOTICE]: Adding parameter sid = https://T3_US_NotreDame//lobster_kmohrman_EFT_ALL_genOnly_20190914_1944_0bc210367fc00647//12345.730 Sat Sep 14 20:12:06 2019: ApMon[NOTICE]: Adding parameter StageOutExitStatus = 0 Sat Sep 14 20:12:06 2019: ApMon[NOTICE]: Adding parameter jobId = 730_https://T3_US_NotreDame/5ea63027c133f59a/730 Sat Sep 14 20:12:06 2019: ApMon[NOTICE]: Adding parameter ExeTime = 0 Sat Sep 14 20:12:06 2019: ApMon[NOTICE]: Adding parameter taskId = lobster_kmohrman_EFT_ALL_genOnly_20190914_1944_0bc210367fc00647 Sat Sep 14 20:12:06 2019: ApMon[NOTICE]: Adding parameter CrabWrapperTime = -1568505830 Sat Sep 14 20:12:06 2019: ApMon[NOTICE]: Adding parameter NoEventsPerRun = 0 Sat Sep 14 20:12:06 2019: ApMon[NOTICE]: Adding parameter JobExitReason = Sat Sep 14 20:12:06 2019: ApMon[NOTICE]: Adding parameter StageOutSE = Sat Sep 14 20:12:06 2019: ApMon[NOTICE]: Adding parameter CrabUserCpuTime = 0 Sat Sep 14 20:12:06 2019: ApMon[NOTICE]: Adding parameter MessageType = jobRuntime Sat Sep 14 20:12:06 2019: ApMon[NOTICE]: Adding parameter NEventsProcessed = 0 Sat Sep 14 20:12:06 2019: ApMon[NOTICE]: Adding parameter MessageTS = 2019-09-15 00:12:06 Sat Sep 14 20:12:06 2019: ApMon[NOTICE]: Adding parameter SyncGridJobId = https://T3_US_NotreDame//lobster_kmohrman_EFT_ALL_genOnly_20190914_1944_0bc210367fc00647//12345.730 Sat Sep 14 20:12:06 2019: ApMon[NOTICE]: Adding parameter JobExitCode = 179 Sat Sep 14 20:12:06 2019: ApMon[NOTICE]: Adding parameter CrabCpuPercentage = -0.0 Sat Sep 14 20:12:06 2019: ApMon[NOTICE]: Adding parameter WCCPU = -1568505830 Sat Sep 14 20:12:06 2019: ApMon[NOTICE]: Adding parameter NbEvPerRun = 0 Sat Sep 14 20:12:06 2019: ApMon[NOTICE]: Adding parameter ExeExitCode = 0 Sat Sep 14 20:12:06 2019: ApMon[DEBUG]: previously sent: 0; dropped: 0 Sat Sep 14 20:12:06 2019: ApMon[NOTICE]: Building XDR packet [lobster_kmohrman_EFT_ALL_genOnly_20190914_1944_0bc210367fc00647/730_https://T3_US_NotreDame/5ea63027c133f59a/730] <1/14672095> 20 params, 1100 bytes. Sat Sep 14 20:12:06 2019: ApMon[NOTICE]: Packet sent to 188.184.29.203:8884 Sat Sep 14 20:12:06 2019: ApMon[DEBUG]: Closing UDP socket on ApMon object destroy. ==== working directory after execution @ Sat Sep 14 20:12:07 EDT 2019 ==== == dir: total 868 == dir: drwxr-xr-x 17 kmohrman campus 201 Sep 14 20:03 CMSSW_9_3_1 == dir: -rw-r--r-- 11 kmohrman campus 7693 Sep 14 20:02 GEN-00000-ttlnuJets_1_qCut25-1_cfg.py == dir: drwxr-xr-x 2 kmohrman campus 71 Sep 14 20:02 bin == dir: -rwxr-xr-x 13 kmohrman campus 627272 Sep 14 20:02 cctools-monitor == dir: -rw-r--r-- 1 kmohrman campus 0 Sep 14 20:02 cctools-monitor.summary == dir: drwxrwxrwx 2 kmohrman campus 6 Sep 14 20:12 cctools-temp-t.730.ig3lME == dir: -rw------- 1 kmohrman campus 51 Sep 14 20:03 cvmfschecksum.cms.cern.ch == dir: -rw------- 1 kmohrman campus 51 Sep 14 20:03 cvmfschecksum.oasis.opensciencegrid.org == dir: drwxr-xr-x 2 kmohrman campus 33 Sep 14 20:02 lib == dir: -rwxrwxrwx 1 kmohrman campus 146912 Sep 14 20:02 librmonitor_helper.so.BWZ9YF == dir: -rw-r--r-- 2 kmohrman campus 2604 Sep 14 20:02 parameters.json == dir: -rw------- 2 kmohrman campus 9165 Sep 14 20:02 proxy == dir: drwxr-xr-x 3 kmohrman campus 20 Sep 14 20:02 python == dir: -rw-r--r-- 1 kmohrman campus 722 Sep 14 20:12 report.json == dir: -rw-r--r-- 13 kmohrman campus 570 Sep 14 20:02 report.json.in == dir: -rw-r--r-- 13 kmohrman campus 461 Sep 14 20:02 sandbox-CMSSW_9_3_1-slc7_amd64_gcc630.tar.bz2 == dir: drwxr-xr-x 4 kmohrman campus 37 Sep 14 20:02 siteconf == dir: -rw-r--r-- 1 kmohrman campus 11 Sep 14 20:04 t_wrapper_ready == dir: -rw-r--r-- 1 kmohrman campus 11 Sep 14 20:03 t_wrapper_start == dir: -rwxr-xr-x 13 kmohrman campus 47217 Sep 14 20:02 task.py == dir: -rwxr-xr-x 13 kmohrman campus 6296 Sep 14 20:02 wrapper.sh === wrapper done @ Sat Sep 14 20:12:07 EDT 2019 === final return status = 179 @ Sat Sep 14 20:12:07 EDT 2019