[kmohrman@earth: file_lsts]$ xrdcp -d 2 -f root://ndcms.crc.nd.edu//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root /dev/null [2022-03-02 14:37:44.027974 -0500][Debug ][Utility ] CopyProcess: 2 jobs to prepare [2022-03-02 14:37:44.028272 -0500][Debug ][Utility ] Creating a classic copy job, from root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root to file://localhost/dev/null [2022-03-02 14:37:44.028327 -0500][Debug ][Utility ] Monitor library name not set. No monitoring [2022-03-02 14:37:44.028523 -0500][Debug ][Poller ] Available pollers: built-in [2022-03-02 14:37:44.028549 -0500][Debug ][Poller ] Attempting to create a poller according to preference: built-in [2022-03-02 14:37:44.028572 -0500][Debug ][Poller ] Creating poller: built-in [2022-03-02 14:37:44.028600 -0500][Debug ][Poller ] Creating and starting the built-in poller... [2022-03-02 14:37:44.028925 -0500][Debug ][Poller ] Using 1 poller threads [2022-03-02 14:37:44.028956 -0500][Debug ][TaskMgr ] Starting the task manager... [2022-03-02 14:37:44.029030 -0500][Debug ][TaskMgr ] Task manager started [2022-03-02 14:37:44.029078 -0500][Debug ][JobMgr ] Starting the job manager... [2022-03-02 14:37:44.029249 -0500][Debug ][JobMgr ] Job manager started, 3 workers [2022-03-02 14:37:44.029280 -0500][Debug ][TaskMgr ] Registering task: "FileTimer task" to be run at: [2022-03-02 14:37:44 -0500] [2022-03-02 14:37:44.029320 -0500][Debug ][Utility ] Opening root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root for reading [2022-03-02 14:37:44.029511 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Sending an open command [2022-03-02 14:37:44.029610 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0x1114a70 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) ). [2022-03-02 14:37:44.029701 -0500][Debug ][PostMaster ] Creating new channel to: ndcms.crc.nd.edu:1094 1 stream(s) [2022-03-02 14:37:44.029750 -0500][Debug ][PostMaster ] [ndcms.crc.nd.edu:1094 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 [2022-03-02 14:37:44.030657 -0500][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: ndcms.crc.nd.edu:1094" to be run at: [2022-03-02 14:37:59 -0500] [2022-03-02 14:37:44.031486 -0500][Debug ][PostMaster ] [ndcms.crc.nd.edu:1094] Found 1 address(es): [::ffff:129.74.85.4]:1094 [2022-03-02 14:37:44.031539 -0500][Debug ][AsyncSock ] [ndcms.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:129.74.85.4]:1094 [2022-03-02 14:37:44.031592 -0500][Debug ][Poller ] Adding socket 0x1115860 to the poller [2022-03-02 14:37:44.031910 -0500][Debug ][AsyncSock ] [ndcms.crc.nd.edu:1094 #0.0] Async connection call returned [2022-03-02 14:37:44.031991 -0500][Debug ][XRootDTransport ] [ndcms.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol [2022-03-02 14:37:44.032629 -0500][Debug ][XRootDTransport ] [ndcms.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: manager [], protocol version 500) [2022-03-02 14:37:44.032657 -0500][Debug ][XRootDTransport ] [ndcms.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: manager [], protocol version 500) [2022-03-02 14:37:44.033667 -0500][Debug ][XRootDTransport ] [ndcms.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=earth.crc.nd.edu&xrd.rn=v4.12.6, dual-stack: false, private IPv4: true, private IPv6: false [2022-03-02 14:37:44.033905 -0500][Debug ][XRootDTransport ] [ndcms.crc.nd.edu:1094 #0.0] Logged in, session: 5b550000ec080000170000005c550000 [2022-03-02 14:37:44.033944 -0500][Debug ][PostMaster ] [ndcms.crc.nd.edu:1094 #0] Stream 0 connected. [2022-03-02 14:37:44.034008 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0x1114a70 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) ) from out-queu to in-queue. [2022-03-02 14:37:44.034562 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0x1114a70 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) ). [2022-03-02 14:37:44.034633 -0500][Debug ][PostMaster ] Creating new channel to: primeradiant01.crc.nd.edu:1094 1 stream(s) [2022-03-02 14:37:44.034666 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 [2022-03-02 14:37:44.035247 -0500][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: primeradiant01.crc.nd.edu:1094" to be run at: [2022-03-02 14:37:59 -0500] [2022-03-02 14:37:44.035767 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094] Found 1 address(es): [::ffff:149.165.243.5]:1094 [2022-03-02 14:37:44.035808 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:149.165.243.5]:1094 [2022-03-02 14:37:44.035854 -0500][Debug ][Poller ] Adding socket 0xc8001b30 to the poller [2022-03-02 14:37:44.035962 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Async connection call returned [2022-03-02 14:37:44.036009 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol [2022-03-02 14:37:44.043664 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: server [], protocol version 500) [2022-03-02 14:37:44.043722 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: server [proxy], protocol version 500) [2022-03-02 14:37:44.045301 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=earth.crc.nd.edu&xrd.rn=v4.12.6, dual-stack: false, private IPv4: true, private IPv6: false [2022-03-02 14:37:44.045485 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Logged in, session: 7d180000070d0000330000007d180000 [2022-03-02 14:37:44.045563 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Stream 0 connected. [2022-03-02 14:37:44.045643 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0x1114a70 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) ) from out-queu to in-queue. [2022-03-02 14:37:44.045843 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0x1114a70 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) ) with status: [SUCCESS] . [2022-03-02 14:37:44.045946 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Open has returned with status [SUCCESS] [2022-03-02 14:37:44.045976 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] successfully opened at primeradiant01.crc.nd.edu:1094, handle: 0x0, session id: 1 [2022-03-02 14:37:44.046065 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 14:37:44.046065 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 14:37:44.046125 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0x1114a70. [2022-03-02 14:37:44.046146 -0500][Debug ][Utility ] Opening file://localhost/dev/null?oss.asize=157882637 for writing [2022-03-02 14:37:44.046277 -0500][Debug ][File ] [0x1114910@file://localhost/dev/null?oss.asize=157882637&xrdcl.requuid=ae1adf29-47c1-43e2-ad7c-427e260856fe] Sending an open command [2022-03-02 14:37:44.046497 -0500][Debug ][File ] [0x1114910@file://localhost/dev/null?oss.asize=157882637&xrdcl.requuid=ae1adf29-47c1-43e2-ad7c-427e260856fe] Open has returned with status [SUCCESS] [2022-03-02 14:37:44.046529 -0500][Debug ][File ] [0x1114910@file://localhost/dev/null?oss.asize=157882637&xrdcl.requuid=ae1adf29-47c1-43e2-ad7c-427e260856fe] successfully opened at localhost, handle: 0xb, session id: 1 [2022-03-02 14:37:44.046607 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Sending a read command for handle 0x0 to primeradiant01.crc.nd.edu:1094 [2022-03-02 14:37:44.046667 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0x11165d0 (message: kXR_read (handle: 0x00000000, offset: 0, size: 8388608) ). [2022-03-02 14:37:44.046725 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Sending a read command for handle 0x0 to primeradiant01.crc.nd.edu:1094 [2022-03-02 14:37:44.046738 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0x11165d0 (message: kXR_read (handle: 0x00000000, offset: 0, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:37:44.046763 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0x1116f40 (message: kXR_read (handle: 0x00000000, offset: 8388608, size: 8388608) ). [2022-03-02 14:37:44.046857 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Sending a read command for handle 0x0 to primeradiant01.crc.nd.edu:1094 [2022-03-02 14:37:44.046900 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0x1117680 (message: kXR_read (handle: 0x00000000, offset: 16777216, size: 8388608) ). [2022-03-02 14:37:44.046866 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0x1116f40 (message: kXR_read (handle: 0x00000000, offset: 8388608, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:37:44.046958 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Sending a read command for handle 0x0 to primeradiant01.crc.nd.edu:1094 [2022-03-02 14:37:44.047060 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0x1117ba0 (message: kXR_read (handle: 0x00000000, offset: 25165824, size: 8388608) ). [2022-03-02 14:37:44.047170 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0x1117680 (message: kXR_read (handle: 0x00000000, offset: 16777216, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:37:44.047339 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0x1117ba0 (message: kXR_read (handle: 0x00000000, offset: 25165824, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:38:44.093946 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 14:38:44.094012 -0500][Debug ][Poller ] <[::ffff:10.32.77.18]:13746><--><[::ffff:149.165.243.5]:1094> Removing socket from the poller [2022-03-02 14:38:44.094125 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Recovering error for stream #0: [ERROR] Socket error. [2022-03-02 14:38:44.094142 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Reporting disconnection to queued message handlers. [2022-03-02 14:38:44.094161 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x00000000, offset: 0, size: 8388608): [ERROR] Socket error. [2022-03-02 14:38:44.094174 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x00000000, offset: 0, size: 8388608) [2022-03-02 14:38:44.094192 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0x11165d0 (message: kXR_read (handle: 0x00000000, offset: 0, size: 8388608) ). [2022-03-02 14:38:44.094215 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x00000000, offset: 8388608, size: 8388608): [ERROR] Socket error. [2022-03-02 14:38:44.094226 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x00000000, offset: 8388608, size: 8388608) [2022-03-02 14:38:44.094238 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0x1116f40 (message: kXR_read (handle: 0x00000000, offset: 8388608, size: 8388608) ). [2022-03-02 14:38:44.094256 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x00000000, offset: 16777216, size: 8388608): [ERROR] Socket error. [2022-03-02 14:38:44.094267 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x00000000, offset: 16777216, size: 8388608) [2022-03-02 14:38:44.094279 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0x1117680 (message: kXR_read (handle: 0x00000000, offset: 16777216, size: 8388608) ). [2022-03-02 14:38:44.094303 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x00000000, offset: 25165824, size: 8388608): [ERROR] Socket error. [2022-03-02 14:38:44.094315 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x00000000, offset: 25165824, size: 8388608) [2022-03-02 14:38:44.094311 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0x11165d0 (message: kXR_read (handle: 0x00000000, offset: 0, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:38:44.094373 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0x1116f40 (message: kXR_read (handle: 0x00000000, offset: 8388608, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:38:44.094403 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0x11165d0. [2022-03-02 14:38:44.094442 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0x1116f40. [2022-03-02 14:38:44.094442 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0x1117680 (message: kXR_read (handle: 0x00000000, offset: 16777216, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:38:44.094328 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0x1117ba0 (message: kXR_read (handle: 0x00000000, offset: 25165824, size: 8388608) ). [2022-03-02 14:38:44.094504 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0x1117680. [2022-03-02 14:38:44.094591 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0x1117ba0 (message: kXR_read (handle: 0x00000000, offset: 25165824, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:38:44.094658 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 14:38:44.094718 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xcc001550 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:38:44.094797 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0x1117ba0. [2022-03-02 14:38:44.094873 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xcc001550 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:38:44.095594 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xcc001550 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:38:44.096791 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094] Found 1 address(es): [::ffff:149.165.243.5]:1094 [2022-03-02 14:38:44.096828 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:149.165.243.5]:1094 [2022-03-02 14:38:44.096867 -0500][Debug ][Poller ] Adding socket 0xc8001b30 to the poller [2022-03-02 14:38:44.097088 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Async connection call returned [2022-03-02 14:38:44.097133 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol [2022-03-02 14:38:44.097280 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: server [], protocol version 500) [2022-03-02 14:38:44.097309 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: server [proxy], protocol version 500) [2022-03-02 14:38:44.099121 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=earth.crc.nd.edu&xrd.rn=v4.12.6, dual-stack: false, private IPv4: true, private IPv6: false [2022-03-02 14:38:44.099364 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Logged in, session: 7e180000070d0000380000007e180000 [2022-03-02 14:38:44.099446 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 7d180000070d0000330000007d180000 [2022-03-02 14:38:44.099596 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:38:44.099628 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 7d180000070d0000330000007d180000 [2022-03-02 14:38:44.099671 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000c90" to be run at: [2022-03-02 14:38:48 -0500] [2022-03-02 14:38:48.038852 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000c90" [2022-03-02 14:38:48.039327 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:38:48.039393 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 7d180000070d0000330000007d180000 [2022-03-02 14:38:48.039418 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000bd0" to be run at: [2022-03-02 14:38:54 -0500] [2022-03-02 14:38:54.039790 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000bd0" [2022-03-02 14:38:54.040151 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:38:54.040219 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 7d180000070d0000330000007d180000 [2022-03-02 14:38:54.040242 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000c10" to be run at: [2022-03-02 14:39:02 -0500] [2022-03-02 14:39:02.041178 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000c10" [2022-03-02 14:39:02.041599 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:39:02.041673 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 7d180000070d0000330000007d180000 [2022-03-02 14:39:02.041697 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000c90" to be run at: [2022-03-02 14:39:12 -0500] [2022-03-02 14:39:12.042851 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000c90" [2022-03-02 14:39:12.043321 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Stream 0 connected. [2022-03-02 14:39:12.043461 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc001550 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:39:12.043803 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc001550 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 14:39:12.043900 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Open has returned with status [SUCCESS] [2022-03-02 14:39:12.043932 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] successfully opened at primeradiant01.crc.nd.edu:1094, handle: 0x0, session id: 2 [2022-03-02 14:39:12.043980 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xc00013d0 (message: kXR_read (handle: 0x00000000, offset: 0, size: 8388608) ). [2022-03-02 14:39:12.044072 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xc0001660 (message: kXR_read (handle: 0x00000000, offset: 8388608, size: 8388608) ). [2022-03-02 14:39:12.044090 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc00013d0 (message: kXR_read (handle: 0x00000000, offset: 0, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:39:12.044149 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xc0001dc0 (message: kXR_read (handle: 0x00000000, offset: 16777216, size: 8388608) ). [2022-03-02 14:39:12.044220 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc0001660 (message: kXR_read (handle: 0x00000000, offset: 8388608, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:39:12.044239 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xc0002050 (message: kXR_read (handle: 0x00000000, offset: 25165824, size: 8388608) ). [2022-03-02 14:39:12.044299 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc0001dc0 (message: kXR_read (handle: 0x00000000, offset: 16777216, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:39:12.044330 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 14:39:12.044330 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 14:39:12.044350 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc0002050 (message: kXR_read (handle: 0x00000000, offset: 25165824, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:39:12.044393 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc001550. [2022-03-02 14:40:12.105384 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 14:40:12.105466 -0500][Debug ][Poller ] <[::ffff:10.32.77.18]:13754><--><[::ffff:149.165.243.5]:1094> Removing socket from the poller [2022-03-02 14:40:12.105553 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Recovering error for stream #0: [ERROR] Socket error. [2022-03-02 14:40:12.105608 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Reporting disconnection to queued message handlers. [2022-03-02 14:40:12.105665 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x00000000, offset: 25165824, size: 8388608): [ERROR] Socket error. [2022-03-02 14:40:12.105714 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x00000000, offset: 25165824, size: 8388608) [2022-03-02 14:40:12.105736 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xc0002050 (message: kXR_read (handle: 0x00000000, offset: 25165824, size: 8388608) ). [2022-03-02 14:40:12.105800 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x00000000, offset: 0, size: 8388608): [ERROR] Socket error. [2022-03-02 14:40:12.105820 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x00000000, offset: 0, size: 8388608) [2022-03-02 14:40:12.105847 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc0002050 (message: kXR_read (handle: 0x00000000, offset: 25165824, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:40:12.105869 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xc00013d0 (message: kXR_read (handle: 0x00000000, offset: 0, size: 8388608) ). [2022-03-02 14:40:12.105919 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 14:40:12.105931 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x00000000, offset: 8388608, size: 8388608): [ERROR] Socket error. [2022-03-02 14:40:12.105979 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x00000000, offset: 8388608, size: 8388608) [2022-03-02 14:40:12.105992 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xc0001660 (message: kXR_read (handle: 0x00000000, offset: 8388608, size: 8388608) ). [2022-03-02 14:40:12.106001 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xcc001060 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:40:12.106015 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x00000000, offset: 16777216, size: 8388608): [ERROR] Socket error. [2022-03-02 14:40:12.106029 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x00000000, offset: 16777216, size: 8388608) [2022-03-02 14:40:12.106058 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xc0001dc0 (message: kXR_read (handle: 0x00000000, offset: 16777216, size: 8388608) ). [2022-03-02 14:40:12.106111 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xcc001060 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:40:12.106060 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc0002050. [2022-03-02 14:40:12.106106 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc00013d0 (message: kXR_read (handle: 0x00000000, offset: 0, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:40:12.106170 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc0001660 (message: kXR_read (handle: 0x00000000, offset: 8388608, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:40:12.106294 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 14:40:12.106291 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc0001dc0 (message: kXR_read (handle: 0x00000000, offset: 16777216, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:40:12.106418 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xc8000930 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:40:12.106487 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc00013d0. [2022-03-02 14:40:12.106502 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 14:40:12.106493 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xc8000930 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:40:12.106590 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xc00013d0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:40:12.106671 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc0001660. [2022-03-02 14:40:12.106692 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 14:40:12.106685 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xc00013d0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:40:12.106762 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xcc001060 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:40:12.106785 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xcc0017e0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:40:12.106833 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xc8000930 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:40:12.107000 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc0001dc0. [2022-03-02 14:40:12.107069 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xcc0017e0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:40:12.107188 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xc00013d0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:40:12.107897 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094] Found 1 address(es): [::ffff:149.165.243.5]:1094 [2022-03-02 14:40:12.107953 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:149.165.243.5]:1094 [2022-03-02 14:40:12.108006 -0500][Debug ][Poller ] Adding socket 0xc8001b30 to the poller [2022-03-02 14:40:12.108169 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xcc0017e0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:40:12.108201 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Async connection call returned [2022-03-02 14:40:12.108241 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol [2022-03-02 14:40:12.108392 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: server [], protocol version 500) [2022-03-02 14:40:12.108416 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: server [proxy], protocol version 500) [2022-03-02 14:40:12.109424 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=earth.crc.nd.edu&xrd.rn=v4.12.6, dual-stack: false, private IPv4: true, private IPv6: false [2022-03-02 14:40:12.109603 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Logged in, session: 80180000070d00003900000080180000 [2022-03-02 14:40:12.109627 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 7e180000070d0000380000007e180000 [2022-03-02 14:40:12.109743 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:40:12.109762 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 7e180000070d0000380000007e180000 [2022-03-02 14:40:12.109780 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000c90" to be run at: [2022-03-02 14:40:16 -0500] [2022-03-02 14:40:16.052930 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000c90" [2022-03-02 14:40:16.053309 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:40:16.053389 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 7e180000070d0000380000007e180000 [2022-03-02 14:40:16.053414 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000bd0" to be run at: [2022-03-02 14:40:22 -0500] [2022-03-02 14:40:22.053940 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000bd0" [2022-03-02 14:40:22.054264 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:40:22.054337 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 7e180000070d0000380000007e180000 [2022-03-02 14:40:22.054360 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000c10" to be run at: [2022-03-02 14:40:30 -0500] [2022-03-02 14:40:30.054918 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000c10" [2022-03-02 14:40:30.055336 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:40:30.055419 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 7e180000070d0000380000007e180000 [2022-03-02 14:40:30.055452 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000c90" to be run at: [2022-03-02 14:40:40 -0500] [2022-03-02 14:40:40.056297 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000c90" [2022-03-02 14:40:40.056685 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:40:40.056746 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 7e180000070d0000380000007e180000 [2022-03-02 14:40:40.056769 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000bd0" to be run at: [2022-03-02 14:40:52 -0500] [2022-03-02 14:40:52.057474 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000bd0" [2022-03-02 14:40:52.057830 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:40:52.057906 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 7e180000070d0000380000007e180000 [2022-03-02 14:40:52.057969 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000c10" to be run at: [2022-03-02 14:41:06 -0500] [2022-03-02 14:41:06.058570 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000c10" [2022-03-02 14:41:06.058923 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Stream 0 connected. [2022-03-02 14:41:06.059018 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc001060 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:41:06.059136 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc8000930 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:41:06.059189 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc00013d0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:41:06.059307 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc0017e0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:41:06.059337 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc001060 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 14:41:06.059391 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 14:41:06.059391 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 14:41:06.059410 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc001060. [2022-03-02 14:41:06.059434 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc8000930 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 14:41:06.059458 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 14:41:06.059458 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 14:41:06.059474 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc8000930. [2022-03-02 14:41:06.059498 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc00013d0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 14:41:06.059525 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 14:41:06.059525 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 14:41:06.059522 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc0017e0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 14:41:06.059645 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Open has returned with status [SUCCESS] [2022-03-02 14:41:06.059680 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] successfully opened at primeradiant01.crc.nd.edu:1094, handle: 0x3, session id: 3 [2022-03-02 14:41:06.059565 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc00013d0. [2022-03-02 14:41:06.059746 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xcc000f90 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ). [2022-03-02 14:41:06.059826 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xcc002510 (message: kXR_read (handle: 0x03000000, offset: 0, size: 8388608) ). [2022-03-02 14:41:06.059836 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc000f90 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:41:06.059899 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xcc0027a0 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ). [2022-03-02 14:41:06.059931 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc002510 (message: kXR_read (handle: 0x03000000, offset: 0, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:41:06.059958 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xcc002a30 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ). [2022-03-02 14:41:06.059988 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc0027a0 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:41:06.060020 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 14:41:06.060020 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 14:41:06.060054 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc002a30 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:41:06.060100 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc0017e0. [2022-03-02 14:42:06.121360 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 14:42:06.121420 -0500][Debug ][Poller ] <[::ffff:10.32.77.18]:13778><--><[::ffff:149.165.243.5]:1094> Removing socket from the poller [2022-03-02 14:42:06.121487 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Recovering error for stream #0: [ERROR] Socket error. [2022-03-02 14:42:06.121501 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Reporting disconnection to queued message handlers. [2022-03-02 14:42:06.121518 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608): [ERROR] Socket error. [2022-03-02 14:42:06.121529 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) [2022-03-02 14:42:06.121542 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xcc000f90 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ). [2022-03-02 14:42:06.121568 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 0, size: 8388608): [ERROR] Socket error. [2022-03-02 14:42:06.121579 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 0, size: 8388608) [2022-03-02 14:42:06.121590 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xcc002510 (message: kXR_read (handle: 0x03000000, offset: 0, size: 8388608) ). [2022-03-02 14:42:06.121608 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608): [ERROR] Socket error. [2022-03-02 14:42:06.121620 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) [2022-03-02 14:42:06.121631 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xcc0027a0 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ). [2022-03-02 14:42:06.121757 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc0027a0 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:42:06.121763 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc002510 (message: kXR_read (handle: 0x03000000, offset: 0, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:42:06.121797 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 14:42:06.121719 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc000f90 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:42:06.121854 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xcc0017e0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:42:06.121908 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc0027a0. [2022-03-02 14:42:06.121921 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 14:42:06.121651 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608): [ERROR] Socket error. [2022-03-02 14:42:06.121982 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xc00013d0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:42:06.122094 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc002510. [2022-03-02 14:42:06.122063 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) [2022-03-02 14:42:06.122107 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 14:42:06.122167 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xcc002a30 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ). [2022-03-02 14:42:06.122227 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xc8000930 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:42:06.122238 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc002a30 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:42:06.122277 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xcc0017e0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:42:06.122295 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 14:42:06.122324 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xc00013d0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:42:06.122344 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xcc002610 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:42:06.122394 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xc8000930 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:42:06.122286 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc000f90. [2022-03-02 14:42:06.122519 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xcc002610 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:42:06.122508 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc002a30. [2022-03-02 14:42:06.122758 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xcc0017e0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:42:06.123058 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xc00013d0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:42:06.123063 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xc8000930 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:42:06.123483 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094] Found 1 address(es): [::ffff:149.165.243.5]:1094 [2022-03-02 14:42:06.123567 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:149.165.243.5]:1094 [2022-03-02 14:42:06.123620 -0500][Debug ][Poller ] Adding socket 0xc8001b30 to the poller [2022-03-02 14:42:06.123746 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Async connection call returned [2022-03-02 14:42:06.123803 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xcc002610 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:42:06.123809 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol [2022-03-02 14:42:06.124082 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: server [], protocol version 500) [2022-03-02 14:42:06.124120 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: server [proxy], protocol version 500) [2022-03-02 14:42:06.125147 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=earth.crc.nd.edu&xrd.rn=v4.12.6, dual-stack: false, private IPv4: true, private IPv6: false [2022-03-02 14:42:06.125313 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Logged in, session: 82180000070d00003100000082180000 [2022-03-02 14:42:06.125343 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 80180000070d00003900000080180000 [2022-03-02 14:42:06.125452 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:42:06.125483 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 80180000070d00003900000080180000 [2022-03-02 14:42:06.125515 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000c90" to be run at: [2022-03-02 14:42:10 -0500] [2022-03-02 14:42:10.064271 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000c90" [2022-03-02 14:42:10.064550 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:42:10.064633 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 80180000070d00003900000080180000 [2022-03-02 14:42:10.064667 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000bd0" to be run at: [2022-03-02 14:42:16 -0500] [2022-03-02 14:42:16.064739 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000bd0" [2022-03-02 14:42:16.065088 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:42:16.065160 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 80180000070d00003900000080180000 [2022-03-02 14:42:16.065192 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000c10" to be run at: [2022-03-02 14:42:24 -0500] [2022-03-02 14:42:24.065271 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000c10" [2022-03-02 14:42:24.065640 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:42:24.065734 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 80180000070d00003900000080180000 [2022-03-02 14:42:24.065769 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000c90" to be run at: [2022-03-02 14:42:34 -0500] [2022-03-02 14:42:34.066031 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000c90" [2022-03-02 14:42:34.066338 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:42:34.066417 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 80180000070d00003900000080180000 [2022-03-02 14:42:34.066451 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000bd0" to be run at: [2022-03-02 14:42:46 -0500] [2022-03-02 14:42:46.067079 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000bd0" [2022-03-02 14:42:46.067385 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:42:46.067472 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 80180000070d00003900000080180000 [2022-03-02 14:42:46.067507 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000c10" to be run at: [2022-03-02 14:43:00 -0500] [2022-03-02 14:43:00.068303 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000c10" [2022-03-02 14:43:00.068627 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:43:00.068717 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 80180000070d00003900000080180000 [2022-03-02 14:43:00.068751 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000c90" to be run at: [2022-03-02 14:43:16 -0500] [2022-03-02 14:43:16.069476 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000c90" [2022-03-02 14:43:16.069828 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:43:16.069912 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 80180000070d00003900000080180000 [2022-03-02 14:43:16.069946 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000bd0" to be run at: [2022-03-02 14:43:34 -0500] [2022-03-02 14:43:34.070764 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000bd0" [2022-03-02 14:43:34.071147 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:43:34.071232 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 80180000070d00003900000080180000 [2022-03-02 14:43:34.071267 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000c10" to be run at: [2022-03-02 14:43:54 -0500] [2022-03-02 14:43:54.072437 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000c10" [2022-03-02 14:43:54.072743 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:43:54.072830 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 80180000070d00003900000080180000 [2022-03-02 14:43:54.072857 -0500][Error ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Wont retry kXR_endsess request because wouldreach connection timeout. [2022-03-02 14:43:54.072893 -0500][Error ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Socket error while handshaking: [ERROR] Socket timeout [2022-03-02 14:43:54.072917 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 14:43:54.072940 -0500][Debug ][Poller ] <[::ffff:10.32.77.18]:13838><--><[::ffff:149.165.243.5]:1094> Removing socket from the poller [2022-03-02 14:43:54.072994 -0500][Error ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] elapsed = 108, pConnectionWindow = 120 seconds. [2022-03-02 14:43:54.073054 -0500][Info ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Attempting reconnection in 12 seconds. [2022-03-02 14:43:54.073113 -0500][Debug ][TaskMgr ] Registering task: "StreamConnectorTask for primeradiant01.crc.nd.edu:1094 #0" to be run at: [2022-03-02 14:44:06 -0500] [2022-03-02 14:44:06.074010 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094] Found 1 address(es): [::ffff:149.165.243.5]:1094 [2022-03-02 14:44:06.074112 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:149.165.243.5]:1094 [2022-03-02 14:44:06.074161 -0500][Debug ][Poller ] Adding socket 0xc8001b30 to the poller [2022-03-02 14:44:06.074202 -0500][Debug ][TaskMgr ] Done with task: "StreamConnectorTask for primeradiant01.crc.nd.edu:1094 #0" [2022-03-02 14:44:06.074299 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Async connection call returned [2022-03-02 14:44:06.074326 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol [2022-03-02 14:44:06.074477 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: server [], protocol version 500) [2022-03-02 14:44:06.074517 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: server [proxy], protocol version 500) [2022-03-02 14:44:06.075652 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=earth.crc.nd.edu&xrd.rn=v4.12.6, dual-stack: false, private IPv4: true, private IPv6: false [2022-03-02 14:44:06.075837 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Logged in, session: 85180000070d00003b00000085180000 [2022-03-02 14:44:06.075856 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 82180000070d00003100000082180000 [2022-03-02 14:44:06.075981 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Stream 0 connected. [2022-03-02 14:44:06.076015 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc0017e0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:44:06.076073 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc00013d0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:44:06.076104 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc8000930 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:44:06.076134 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc002610 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:44:06.076385 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc0017e0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 14:44:06.076463 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 14:44:06.076463 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 14:44:06.076481 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc0017e0. [2022-03-02 14:44:06.076509 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc002610 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 14:44:06.076539 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc8000930 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 14:44:06.076539 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc00013d0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 14:44:06.076565 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Open has returned with status [SUCCESS] [2022-03-02 14:44:06.076630 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 14:44:06.076630 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 14:44:06.076636 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] successfully opened at primeradiant01.crc.nd.edu:1094, handle: 0x3, session id: 4 [2022-03-02 14:44:06.076685 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc8000930. [2022-03-02 14:44:06.076718 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xc8002ce0 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ). [2022-03-02 14:44:06.076638 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 14:44:06.076638 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 14:44:06.076750 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc00013d0. [2022-03-02 14:44:06.076768 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xc8002f70 (message: kXR_read (handle: 0x03000000, offset: 0, size: 8388608) ). [2022-03-02 14:44:06.076787 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc8002ce0 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:44:06.076802 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xc80025d0 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ). [2022-03-02 14:44:06.076821 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc8002f70 (message: kXR_read (handle: 0x03000000, offset: 0, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:44:06.076829 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xc8003200 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ). [2022-03-02 14:44:06.076857 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc80025d0 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:44:06.076889 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc8003200 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:44:06.076893 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 14:44:06.076893 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 14:44:06.076932 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc002610. [2022-03-02 14:45:06.126194 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 14:45:06.126255 -0500][Debug ][Poller ] <[::ffff:10.32.77.18]:13862><--><[::ffff:149.165.243.5]:1094> Removing socket from the poller [2022-03-02 14:45:06.126320 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Recovering error for stream #0: [ERROR] Socket error. [2022-03-02 14:45:06.126334 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Reporting disconnection to queued message handlers. [2022-03-02 14:45:06.126350 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608): [ERROR] Socket error. [2022-03-02 14:45:06.126362 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) [2022-03-02 14:45:06.126375 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xc80025d0 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ). [2022-03-02 14:45:06.126395 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608): [ERROR] Socket error. [2022-03-02 14:45:06.126407 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) [2022-03-02 14:45:06.126418 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xc8003200 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ). [2022-03-02 14:45:06.126436 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608): [ERROR] Socket error. [2022-03-02 14:45:06.126447 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) [2022-03-02 14:45:06.126458 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xc8002ce0 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ). [2022-03-02 14:45:06.126477 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 0, size: 8388608): [ERROR] Socket error. [2022-03-02 14:45:06.126489 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 0, size: 8388608) [2022-03-02 14:45:06.126500 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xc8002f70 (message: kXR_read (handle: 0x03000000, offset: 0, size: 8388608) ). [2022-03-02 14:45:06.126630 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc8002ce0 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:45:06.126628 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc8003200 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:45:06.126695 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 14:45:06.126628 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc80025d0 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:45:06.126788 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xc8003490 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:45:06.126841 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc8002ce0. [2022-03-02 14:45:06.126855 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 14:45:06.126858 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xc8003490 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:45:06.126862 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc8002f70 (message: kXR_read (handle: 0x03000000, offset: 0, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:45:06.126937 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xc00013d0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:45:06.127031 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc8003200. [2022-03-02 14:45:06.127059 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xc00013d0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:45:06.127062 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 14:45:06.127150 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:45:06.127229 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc80025d0. [2022-03-02 14:45:06.127236 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:45:06.127239 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 14:45:06.127339 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xc8002ce0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:45:06.127369 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc8002f70. [2022-03-02 14:45:06.127381 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xc8002ce0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:45:06.127797 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xc00013d0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:45:06.127799 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xc8003490 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:45:06.127851 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:45:06.128498 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094] Found 1 address(es): [::ffff:149.165.243.5]:1094 [2022-03-02 14:45:06.128561 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:149.165.243.5]:1094 [2022-03-02 14:45:06.128611 -0500][Debug ][Poller ] Adding socket 0xc8001b30 to the poller [2022-03-02 14:45:06.128709 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Async connection call returned [2022-03-02 14:45:06.128774 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol [2022-03-02 14:45:06.128788 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xc8002ce0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:45:06.128983 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: server [], protocol version 500) [2022-03-02 14:45:06.129021 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: server [proxy], protocol version 500) [2022-03-02 14:45:06.130159 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=earth.crc.nd.edu&xrd.rn=v4.12.6, dual-stack: false, private IPv4: true, private IPv6: false [2022-03-02 14:45:06.130359 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Logged in, session: 86180000070d00003100000086180000 [2022-03-02 14:45:06.130390 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 85180000070d00003b00000085180000 [2022-03-02 14:45:06.130497 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:45:06.130529 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 85180000070d00003b00000085180000 [2022-03-02 14:45:06.130564 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000e10" to be run at: [2022-03-02 14:45:10 -0500] [2022-03-02 14:45:10.084373 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000e10" [2022-03-02 14:45:10.084741 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:45:10.084803 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 85180000070d00003b00000085180000 [2022-03-02 14:45:10.084865 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000bd0" to be run at: [2022-03-02 14:45:16 -0500] [2022-03-02 14:45:16.085453 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000bd0" [2022-03-02 14:45:16.085809 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:45:16.085886 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 85180000070d00003b00000085180000 [2022-03-02 14:45:16.085951 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000c10" to be run at: [2022-03-02 14:45:24 -0500] [2022-03-02 14:45:24.086823 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000c10" [2022-03-02 14:45:24.087091 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:45:24.087224 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 85180000070d00003b00000085180000 [2022-03-02 14:45:24.087264 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000e10" to be run at: [2022-03-02 14:45:34 -0500] [2022-03-02 14:45:34.088593 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000e10" [2022-03-02 14:45:34.088953 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:45:34.089060 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 85180000070d00003b00000085180000 [2022-03-02 14:45:34.089098 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000c10" to be run at: [2022-03-02 14:45:46 -0500] [2022-03-02 14:45:46.090651 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000c10" [2022-03-02 14:45:46.091002 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:45:46.091107 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 85180000070d00003b00000085180000 [2022-03-02 14:45:46.091142 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000bd0" to be run at: [2022-03-02 14:46:00 -0500] [2022-03-02 14:46:00.093006 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000bd0" [2022-03-02 14:46:00.093373 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:46:00.093455 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 85180000070d00003b00000085180000 [2022-03-02 14:46:00.093491 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000e10" to be run at: [2022-03-02 14:46:16 -0500] [2022-03-02 14:46:16.095652 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000e10" [2022-03-02 14:46:16.095943 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:46:16.096018 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 85180000070d00003b00000085180000 [2022-03-02 14:46:16.096073 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000c10" to be run at: [2022-03-02 14:46:34 -0500] [2022-03-02 14:46:34.098668 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000c10" [2022-03-02 14:46:34.098874 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:46:34.098970 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 85180000070d00003b00000085180000 [2022-03-02 14:46:34.099003 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000bd0" to be run at: [2022-03-02 14:46:54 -0500] [2022-03-02 14:46:54.102099 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000bd0" [2022-03-02 14:46:54.102379 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:46:54.102454 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 85180000070d00003b00000085180000 [2022-03-02 14:46:54.102510 -0500][Error ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Wont retry kXR_endsess request because wouldreach connection timeout. [2022-03-02 14:46:54.102536 -0500][Error ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Socket error while handshaking: [ERROR] Socket timeout [2022-03-02 14:46:54.102557 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 14:46:54.102606 -0500][Debug ][Poller ] <[::ffff:10.32.77.18]:13870><--><[::ffff:149.165.243.5]:1094> Removing socket from the poller [2022-03-02 14:46:54.102659 -0500][Error ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] elapsed = 108, pConnectionWindow = 120 seconds. [2022-03-02 14:46:54.102695 -0500][Info ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Attempting reconnection in 12 seconds. [2022-03-02 14:46:54.102720 -0500][Debug ][TaskMgr ] Registering task: "StreamConnectorTask for primeradiant01.crc.nd.edu:1094 #0" to be run at: [2022-03-02 14:47:06 -0500] [2022-03-02 14:47:06.104824 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094] Found 1 address(es): [::ffff:149.165.243.5]:1094 [2022-03-02 14:47:06.104906 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:149.165.243.5]:1094 [2022-03-02 14:47:06.104963 -0500][Debug ][Poller ] Adding socket 0xc8001b30 to the poller [2022-03-02 14:47:06.105013 -0500][Debug ][TaskMgr ] Done with task: "StreamConnectorTask for primeradiant01.crc.nd.edu:1094 #0" [2022-03-02 14:47:06.105198 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Async connection call returned [2022-03-02 14:47:06.105323 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol [2022-03-02 14:47:06.105556 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: server [], protocol version 500) [2022-03-02 14:47:06.105599 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: server [proxy], protocol version 500) [2022-03-02 14:47:06.106629 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=earth.crc.nd.edu&xrd.rn=v4.12.6, dual-stack: false, private IPv4: true, private IPv6: false [2022-03-02 14:47:06.106786 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Logged in, session: 89180000070d00004000000089180000 [2022-03-02 14:47:06.106816 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 86180000070d00003100000086180000 [2022-03-02 14:47:06.106936 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Stream 0 connected. [2022-03-02 14:47:06.106980 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc00013d0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:47:06.107070 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc8003490 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:47:06.107156 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:47:06.107206 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc8002ce0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:47:06.107259 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc00013d0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 14:47:06.107330 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc8003490 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 14:47:06.107333 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 14:47:06.107333 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 14:47:06.107387 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc00013d0. [2022-03-02 14:47:06.107394 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 14:47:06.107394 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 14:47:06.107415 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc8002ce0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 14:47:06.107390 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 14:47:06.107468 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Open has returned with status [SUCCESS] [2022-03-02 14:47:06.107493 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 14:47:06.107493 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 14:47:06.107499 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] successfully opened at primeradiant01.crc.nd.edu:1094, handle: 0x3, session id: 5 [2022-03-02 14:47:06.107439 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc8003490. [2022-03-02 14:47:06.107577 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xc0002510 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ). [2022-03-02 14:47:06.107538 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc001350. [2022-03-02 14:47:06.107621 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xc00027a0 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ). [2022-03-02 14:47:06.107641 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc0002510 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:47:06.107659 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xc0002a30 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ). [2022-03-02 14:47:06.107712 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc00027a0 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:47:06.107737 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xc0002cc0 (message: kXR_read (handle: 0x03000000, offset: 0, size: 8388608) ). [2022-03-02 14:47:06.107785 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc0002a30 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:47:06.107813 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 14:47:06.107813 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 14:47:06.107832 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc0002cc0 (message: kXR_read (handle: 0x03000000, offset: 0, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:47:06.107876 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc8002ce0. [2022-03-02 14:48:06.168284 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 14:48:06.168361 -0500][Debug ][Poller ] <[::ffff:10.32.77.18]:13888><--><[::ffff:149.165.243.5]:1094> Removing socket from the poller [2022-03-02 14:48:06.168433 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Recovering error for stream #0: [ERROR] Socket error. [2022-03-02 14:48:06.168456 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Reporting disconnection to queued message handlers. [2022-03-02 14:48:06.168482 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 0, size: 8388608): [ERROR] Socket error. [2022-03-02 14:48:06.168502 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 0, size: 8388608) [2022-03-02 14:48:06.168527 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xc0002cc0 (message: kXR_read (handle: 0x03000000, offset: 0, size: 8388608) ). [2022-03-02 14:48:06.168643 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608): [ERROR] Socket error. [2022-03-02 14:48:06.168661 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc0002cc0 (message: kXR_read (handle: 0x03000000, offset: 0, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:48:06.168705 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) [2022-03-02 14:48:06.168763 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xc00027a0 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ). [2022-03-02 14:48:06.168793 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608): [ERROR] Socket error. [2022-03-02 14:48:06.168828 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) [2022-03-02 14:48:06.168763 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 14:48:06.168876 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xc0002510 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ). [2022-03-02 14:48:06.168936 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc00027a0 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:48:06.169000 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608): [ERROR] Socket error. [2022-03-02 14:48:06.169112 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) [2022-03-02 14:48:06.169142 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xc0002a30 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ). [2022-03-02 14:48:06.169140 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc0002510 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:48:06.169148 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xc8002570 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:48:06.169295 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc0002cc0. [2022-03-02 14:48:06.169310 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 14:48:06.169311 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xc8002570 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:48:06.169362 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc0002a30 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:48:06.169427 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:48:06.169532 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc00027a0. [2022-03-02 14:48:06.169537 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:48:06.169547 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 14:48:06.169646 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xc0002140 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:48:06.169694 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc0002510. [2022-03-02 14:48:06.169701 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xc0002140 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:48:06.169709 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 14:48:06.169809 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xc8002ce0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:48:06.169890 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc0002a30. [2022-03-02 14:48:06.169896 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xc8002570 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:48:06.169900 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xc8002ce0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:48:06.169999 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:48:06.170167 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xc0002140 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:48:06.171205 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094] Found 1 address(es): [::ffff:149.165.243.5]:1094 [2022-03-02 14:48:06.171257 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:149.165.243.5]:1094 [2022-03-02 14:48:06.171312 -0500][Debug ][Poller ] Adding socket 0xc8001b30 to the poller [2022-03-02 14:48:06.171406 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Async connection call returned [2022-03-02 14:48:06.171456 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol [2022-03-02 14:48:06.171489 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xc8002ce0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:48:06.171649 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: server [], protocol version 500) [2022-03-02 14:48:06.171687 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: server [proxy], protocol version 500) [2022-03-02 14:48:06.172662 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=earth.crc.nd.edu&xrd.rn=v4.12.6, dual-stack: false, private IPv4: true, private IPv6: false [2022-03-02 14:48:06.172866 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Logged in, session: 8b180000070d0000410000008b180000 [2022-03-02 14:48:06.172898 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 89180000070d00004000000089180000 [2022-03-02 14:48:06.173015 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:48:06.173068 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 89180000070d00004000000089180000 [2022-03-02 14:48:06.173104 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000fa0" to be run at: [2022-03-02 14:48:10 -0500] [2022-03-02 14:48:10.113852 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000fa0" [2022-03-02 14:48:10.114135 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:48:10.114205 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 89180000070d00004000000089180000 [2022-03-02 14:48:10.114238 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000fa0" to be run at: [2022-03-02 14:48:16 -0500] [2022-03-02 14:48:16.114885 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000fa0" [2022-03-02 14:48:16.115244 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:48:16.115326 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 89180000070d00004000000089180000 [2022-03-02 14:48:16.115361 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000fa0" to be run at: [2022-03-02 14:48:24 -0500] [2022-03-02 14:48:24.116258 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000fa0" [2022-03-02 14:48:24.116592 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:48:24.116659 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 89180000070d00004000000089180000 [2022-03-02 14:48:24.116720 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000e10" to be run at: [2022-03-02 14:48:34 -0500] [2022-03-02 14:48:34.117993 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000e10" [2022-03-02 14:48:34.118398 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:48:34.118436 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 89180000070d00004000000089180000 [2022-03-02 14:48:34.118503 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000fa0" to be run at: [2022-03-02 14:48:46 -0500] [2022-03-02 14:48:46.120053 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000fa0" [2022-03-02 14:48:46.120323 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:48:46.120404 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 89180000070d00004000000089180000 [2022-03-02 14:48:46.120438 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000fa0" to be run at: [2022-03-02 14:49:00 -0500] [2022-03-02 14:49:00.122489 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000fa0" [2022-03-02 14:49:00.122962 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:49:00.123057 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 89180000070d00004000000089180000 [2022-03-02 14:49:00.123120 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001030" to be run at: [2022-03-02 14:49:16 -0500] [2022-03-02 14:49:16.125165 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001030" [2022-03-02 14:49:16.125513 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:49:16.125593 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 89180000070d00004000000089180000 [2022-03-02 14:49:16.125626 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000fa0" to be run at: [2022-03-02 14:49:34 -0500] [2022-03-02 14:49:34.128167 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000fa0" [2022-03-02 14:49:34.128467 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:49:34.128548 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 89180000070d00004000000089180000 [2022-03-02 14:49:34.128583 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000e10" to be run at: [2022-03-02 14:49:54 -0500] [2022-03-02 14:49:54.131699 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000e10" [2022-03-02 14:49:54.132077 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:49:54.132215 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 89180000070d00004000000089180000 [2022-03-02 14:49:54.132243 -0500][Error ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Wont retry kXR_endsess request because wouldreach connection timeout. [2022-03-02 14:49:54.132300 -0500][Error ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Socket error while handshaking: [ERROR] Socket timeout [2022-03-02 14:49:54.132351 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 14:49:54.132374 -0500][Debug ][Poller ] <[::ffff:10.32.77.18]:13900><--><[::ffff:149.165.243.5]:1094> Removing socket from the poller [2022-03-02 14:49:54.132434 -0500][Error ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] elapsed = 108, pConnectionWindow = 120 seconds. [2022-03-02 14:49:54.132461 -0500][Info ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Attempting reconnection in 12 seconds. [2022-03-02 14:49:54.132485 -0500][Debug ][TaskMgr ] Registering task: "StreamConnectorTask for primeradiant01.crc.nd.edu:1094 #0" to be run at: [2022-03-02 14:50:06 -0500] [2022-03-02 14:50:06.134468 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094] Found 1 address(es): [::ffff:149.165.243.5]:1094 [2022-03-02 14:50:06.134554 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:149.165.243.5]:1094 [2022-03-02 14:50:06.134610 -0500][Debug ][Poller ] Adding socket 0xc8001b30 to the poller [2022-03-02 14:50:06.134651 -0500][Debug ][TaskMgr ] Done with task: "StreamConnectorTask for primeradiant01.crc.nd.edu:1094 #0" [2022-03-02 14:50:06.134847 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Async connection call returned [2022-03-02 14:50:06.134920 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol [2022-03-02 14:50:06.135373 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: server [], protocol version 500) [2022-03-02 14:50:06.135431 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: server [proxy], protocol version 500) [2022-03-02 14:50:06.136426 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=earth.crc.nd.edu&xrd.rn=v4.12.6, dual-stack: false, private IPv4: true, private IPv6: false [2022-03-02 14:50:06.136601 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Logged in, session: 8d180000070d0000310000008d180000 [2022-03-02 14:50:06.136628 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 8b180000070d0000410000008b180000 [2022-03-02 14:50:06.136735 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Stream 0 connected. [2022-03-02 14:50:06.136774 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc8002570 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:50:06.136810 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:50:06.136840 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc0002140 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:50:06.136887 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc8002ce0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:50:06.137097 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc0002140 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 14:50:06.137100 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 14:50:06.137103 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc8002570 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 14:50:06.137196 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 14:50:06.137196 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 14:50:06.137168 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 14:50:06.137168 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 14:50:06.137217 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc001350. [2022-03-02 14:50:06.137230 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc0002140. [2022-03-02 14:50:06.137249 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc8002ce0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 14:50:06.137247 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 14:50:06.137247 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 14:50:06.137288 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc8002570. [2022-03-02 14:50:06.137318 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Open has returned with status [SUCCESS] [2022-03-02 14:50:06.137335 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] successfully opened at primeradiant01.crc.nd.edu:1094, handle: 0x3, session id: 6 [2022-03-02 14:50:06.137365 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xcc001350 (message: kXR_read (handle: 0x03000000, offset: 0, size: 8388608) ). [2022-03-02 14:50:06.137404 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xcc002ae0 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ). [2022-03-02 14:50:06.137417 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc001350 (message: kXR_read (handle: 0x03000000, offset: 0, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:50:06.137429 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xcc002d70 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ). [2022-03-02 14:50:06.137453 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc002ae0 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:50:06.137458 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xcc003000 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ). [2022-03-02 14:50:06.137494 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc002d70 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:50:06.137570 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 14:50:06.137570 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 14:50:06.137574 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc003000 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:50:06.137635 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc8002ce0. [2022-03-02 14:50:06.159206 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc001350 (message: kXR_read (handle: 0x03000000, offset: 0, size: 8388608) ) with status: [SUCCESS] . [2022-03-02 14:50:06.159280 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc001350. [2022-03-02 14:50:06.159420 -0500][Debug ][File ] [0x1114910@file://localhost/dev/null?oss.asize=157882637&xrdcl.requuid=ae1adf29-47c1-43e2-ad7c-427e260856fe] Sending a write command for handle 0xb to localhost [8MB/150.6MB][ 5%][==> ][11.04kB/s] [2022-03-02 14:50:06.159837 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Sending a read command for handle 0x3 to primeradiant01.crc.nd.edu:1094 [2022-03-02 14:50:06.159883 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0x11165d0 (message: kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608) ). [2022-03-02 14:50:06.159978 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0x11165d0 (message: kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:51:06.220689 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 14:51:06.220763 -0500][Debug ][Poller ] <[::ffff:10.32.77.18]:13928><--><[::ffff:149.165.243.5]:1094> Removing socket from the poller [2022-03-02 14:51:06.220843 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Recovering error for stream #0: [ERROR] Socket error. [2022-03-02 14:51:06.220868 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Reporting disconnection to queued message handlers. [2022-03-02 14:51:06.220894 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608): [ERROR] Socket error. [2022-03-02 14:51:06.220916 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) [2022-03-02 14:51:06.220974 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xcc002d70 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ). [2022-03-02 14:51:06.221009 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608): [ERROR] Socket error. [2022-03-02 14:51:06.221030 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) [2022-03-02 14:51:06.221072 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xcc002ae0 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ). [2022-03-02 14:51:06.221134 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608): [ERROR] Socket error. [2022-03-02 14:51:06.221163 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc002d70 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:51:06.221186 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608) [2022-03-02 14:51:06.221266 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc002d70. [2022-03-02 14:51:06.221274 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0x11165d0 (message: kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608) ). [2022-03-02 14:51:06.221267 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc002ae0 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:51:06.221385 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0x11165d0 (message: kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:51:06.221367 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608): [ERROR] Socket error. [2022-03-02 14:51:06.221412 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 14:51:06.221416 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) [2022-03-02 14:51:06.221480 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xcc003000 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ). [2022-03-02 14:51:06.221491 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xc0002140 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:51:06.221551 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc003000 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:51:06.221601 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0x11165d0. [2022-03-02 14:51:06.221612 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 14:51:06.221618 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xc0002140 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:51:06.221709 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xc8002290 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:51:06.221764 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc002ae0. [2022-03-02 14:51:06.221776 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 14:51:06.221774 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xc8002290 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:51:06.221840 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:51:06.221887 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc003000. [2022-03-02 14:51:06.221898 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:51:06.222131 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xc0002140 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:51:06.222397 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xc8002290 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:51:06.222408 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:51:06.223068 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094] Found 1 address(es): [::ffff:149.165.243.5]:1094 [2022-03-02 14:51:06.223124 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:149.165.243.5]:1094 [2022-03-02 14:51:06.223178 -0500][Debug ][Poller ] Adding socket 0xc8001b30 to the poller [2022-03-02 14:51:06.223423 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Async connection call returned [2022-03-02 14:51:06.223482 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol [2022-03-02 14:51:06.223688 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: server [], protocol version 500) [2022-03-02 14:51:06.223742 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: server [proxy], protocol version 500) [2022-03-02 14:51:06.225274 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=earth.crc.nd.edu&xrd.rn=v4.12.6, dual-stack: false, private IPv4: true, private IPv6: false [2022-03-02 14:51:06.225523 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Logged in, session: 8f180000070d0000430000008f180000 [2022-03-02 14:51:06.225567 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 8d180000070d0000310000008d180000 [2022-03-02 14:51:06.225700 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:51:06.225733 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 8d180000070d0000310000008d180000 [2022-03-02 14:51:06.225767 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000ef0" to be run at: [2022-03-02 14:51:10 -0500] [2022-03-02 14:51:10.144192 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000ef0" [2022-03-02 14:51:10.144507 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:51:10.144580 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 8d180000070d0000310000008d180000 [2022-03-02 14:51:10.144614 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000e50" to be run at: [2022-03-02 14:51:16 -0500] [2022-03-02 14:51:16.145255 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000e50" [2022-03-02 14:51:16.145604 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:51:16.145685 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 8d180000070d0000310000008d180000 [2022-03-02 14:51:16.145719 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000e10" to be run at: [2022-03-02 14:51:24 -0500] [2022-03-02 14:51:24.146430 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000e10" [2022-03-02 14:51:24.146745 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:51:24.146819 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 8d180000070d0000310000008d180000 [2022-03-02 14:51:24.146853 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000e90" to be run at: [2022-03-02 14:51:34 -0500] [2022-03-02 14:51:34.147994 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000e90" [2022-03-02 14:51:34.148374 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:51:34.148468 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 8d180000070d0000310000008d180000 [2022-03-02 14:51:34.148503 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000e50" to be run at: [2022-03-02 14:51:46 -0500] [2022-03-02 14:51:46.149598 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000e50" [2022-03-02 14:51:46.149884 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:51:46.149964 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 8d180000070d0000310000008d180000 [2022-03-02 14:51:46.149998 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000e10" to be run at: [2022-03-02 14:52:00 -0500] [2022-03-02 14:52:00.151311 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000e10" [2022-03-02 14:52:00.151605 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:52:00.151682 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 8d180000070d0000310000008d180000 [2022-03-02 14:52:00.151717 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000e90" to be run at: [2022-03-02 14:52:16 -0500] [2022-03-02 14:52:16.153470 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000e90" [2022-03-02 14:52:16.153758 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:52:16.153823 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 8d180000070d0000310000008d180000 [2022-03-02 14:52:16.153845 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000e50" to be run at: [2022-03-02 14:52:34 -0500] [2022-03-02 14:52:34.156332 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000e50" [2022-03-02 14:52:34.156692 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:52:34.156760 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 8d180000070d0000310000008d180000 [2022-03-02 14:52:34.156783 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000e10" to be run at: [2022-03-02 14:52:54 -0500] [2022-03-02 14:52:54.159640 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000e10" [2022-03-02 14:52:54.159968 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:52:54.160069 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 8d180000070d0000310000008d180000 [2022-03-02 14:52:54.160096 -0500][Error ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Wont retry kXR_endsess request because wouldreach connection timeout. [2022-03-02 14:52:54.160154 -0500][Error ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Socket error while handshaking: [ERROR] Socket timeout [2022-03-02 14:52:54.160181 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 14:52:54.160236 -0500][Debug ][Poller ] <[::ffff:10.32.77.18]:13942><--><[::ffff:149.165.243.5]:1094> Removing socket from the poller [2022-03-02 14:52:54.160318 -0500][Error ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] elapsed = 108, pConnectionWindow = 120 seconds. [2022-03-02 14:52:54.160345 -0500][Info ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Attempting reconnection in 12 seconds. [2022-03-02 14:52:54.160376 -0500][Debug ][TaskMgr ] Registering task: "StreamConnectorTask for primeradiant01.crc.nd.edu:1094 #0" to be run at: [2022-03-02 14:53:06 -0500] [2022-03-02 14:53:06.162198 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094] Found 1 address(es): [::ffff:149.165.243.5]:1094 [2022-03-02 14:53:06.162370 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:149.165.243.5]:1094 [2022-03-02 14:53:06.162430 -0500][Debug ][Poller ] Adding socket 0xc8001b30 to the poller [2022-03-02 14:53:06.162511 -0500][Debug ][TaskMgr ] Done with task: "StreamConnectorTask for primeradiant01.crc.nd.edu:1094 #0" [2022-03-02 14:53:06.162683 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Async connection call returned [2022-03-02 14:53:06.162819 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol [2022-03-02 14:53:06.163189 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: server [], protocol version 500) [2022-03-02 14:53:06.163240 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: server [proxy], protocol version 500) [2022-03-02 14:53:06.164407 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=earth.crc.nd.edu&xrd.rn=v4.12.6, dual-stack: false, private IPv4: true, private IPv6: false [2022-03-02 14:53:06.164614 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Logged in, session: 91180000070d00001f00000091180000 [2022-03-02 14:53:06.164653 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 8f180000070d0000430000008f180000 [2022-03-02 14:53:06.164807 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Stream 0 connected. [2022-03-02 14:53:06.164857 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc0002140 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:53:06.164915 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc8002290 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:53:06.165061 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:53:06.165067 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc0002140 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 14:53:06.165192 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 14:53:06.165192 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 14:53:06.165255 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc0002140. [2022-03-02 14:53:06.165277 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc8002290 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 14:53:06.165298 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 14:53:06.165372 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 14:53:06.165372 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 14:53:06.165461 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Open has returned with status [SUCCESS] [2022-03-02 14:53:06.165468 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc8002290. [2022-03-02 14:53:06.165509 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] successfully opened at primeradiant01.crc.nd.edu:1094, handle: 0x2, session id: 7 [2022-03-02 14:53:06.165595 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xcc0030d0 (message: kXR_read (handle: 0x02000000, offset: 8388608, size: 8388608) ). [2022-03-02 14:53:06.165651 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xcc003360 (message: kXR_read (handle: 0x02000000, offset: 33554432, size: 8388608) ). [2022-03-02 14:53:06.165667 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc0030d0 (message: kXR_read (handle: 0x02000000, offset: 8388608, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:53:06.165691 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xcc003650 (message: kXR_read (handle: 0x02000000, offset: 16777216, size: 8388608) ). [2022-03-02 14:53:06.165743 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc003360 (message: kXR_read (handle: 0x02000000, offset: 33554432, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:53:06.165766 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xcc0038e0 (message: kXR_read (handle: 0x02000000, offset: 25165824, size: 8388608) ). [2022-03-02 14:53:06.165791 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc003650 (message: kXR_read (handle: 0x02000000, offset: 16777216, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:53:06.165852 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 14:53:06.165852 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 14:53:06.165872 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc0038e0 (message: kXR_read (handle: 0x02000000, offset: 25165824, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:53:06.165906 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc001350. [2022-03-02 14:54:06.213197 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 14:54:06.213274 -0500][Debug ][Poller ] <[::ffff:10.32.77.18]:13982><--><[::ffff:149.165.243.5]:1094> Removing socket from the poller [2022-03-02 14:54:06.213350 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Recovering error for stream #0: [ERROR] Socket error. [2022-03-02 14:54:06.213410 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Reporting disconnection to queued message handlers. [2022-03-02 14:54:06.213436 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x02000000, offset: 33554432, size: 8388608): [ERROR] Socket error. [2022-03-02 14:54:06.213458 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x02000000, offset: 33554432, size: 8388608) [2022-03-02 14:54:06.213481 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xcc003360 (message: kXR_read (handle: 0x02000000, offset: 33554432, size: 8388608) ). [2022-03-02 14:54:06.213512 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x02000000, offset: 25165824, size: 8388608): [ERROR] Socket error. [2022-03-02 14:54:06.213530 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc003360 (message: kXR_read (handle: 0x02000000, offset: 33554432, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:54:06.213540 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x02000000, offset: 25165824, size: 8388608) [2022-03-02 14:54:06.213584 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 14:54:06.213595 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xcc0038e0 (message: kXR_read (handle: 0x02000000, offset: 25165824, size: 8388608) ). [2022-03-02 14:54:06.213708 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x02000000, offset: 16777216, size: 8388608): [ERROR] Socket error. [2022-03-02 14:54:06.213713 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xc0002140 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:54:06.213755 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x02000000, offset: 16777216, size: 8388608) [2022-03-02 14:54:06.213818 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xcc003650 (message: kXR_read (handle: 0x02000000, offset: 16777216, size: 8388608) ). [2022-03-02 14:54:06.213816 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc003360. [2022-03-02 14:54:06.213853 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc0038e0 (message: kXR_read (handle: 0x02000000, offset: 25165824, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:54:06.213889 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x02000000, offset: 8388608, size: 8388608): [ERROR] Socket error. [2022-03-02 14:54:06.213931 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc003650 (message: kXR_read (handle: 0x02000000, offset: 16777216, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:54:06.213972 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x02000000, offset: 8388608, size: 8388608) [2022-03-02 14:54:06.214064 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xcc0030d0 (message: kXR_read (handle: 0x02000000, offset: 8388608, size: 8388608) ). [2022-03-02 14:54:06.213964 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 14:54:06.214125 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xc0002140 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:54:06.214153 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc0030d0 (message: kXR_read (handle: 0x02000000, offset: 8388608, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:54:06.214218 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xc8002290 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:54:06.214289 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc0038e0. [2022-03-02 14:54:06.214296 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xc8002290 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:54:06.214303 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 14:54:06.214402 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xc0002610 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:54:06.214453 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc003650. [2022-03-02 14:54:06.214467 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 14:54:06.214468 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xc0002610 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:54:06.214570 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:54:06.214636 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc0030d0. [2022-03-02 14:54:06.214647 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:54:06.214838 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xc0002140 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:54:06.215076 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xc8002290 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:54:06.215309 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xc0002610 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:54:06.215386 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094] Found 1 address(es): [::ffff:149.165.243.5]:1094 [2022-03-02 14:54:06.215435 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:149.165.243.5]:1094 [2022-03-02 14:54:06.215489 -0500][Debug ][Poller ] Adding socket 0xc8001b30 to the poller [2022-03-02 14:54:06.215613 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:54:06.215720 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Async connection call returned [2022-03-02 14:54:06.215756 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol [2022-03-02 14:54:06.215900 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: server [], protocol version 500) [2022-03-02 14:54:06.215924 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: server [proxy], protocol version 500) [2022-03-02 14:54:06.216952 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=earth.crc.nd.edu&xrd.rn=v4.12.6, dual-stack: false, private IPv4: true, private IPv6: false [2022-03-02 14:54:06.217187 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Logged in, session: 93180000070d00002f00000093180000 [2022-03-02 14:54:06.217213 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 91180000070d00001f00000091180000 [2022-03-02 14:54:06.217307 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:54:06.217326 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 91180000070d00001f00000091180000 [2022-03-02 14:54:06.217344 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001190" to be run at: [2022-03-02 14:54:10 -0500] [2022-03-02 14:54:10.172588 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001190" [2022-03-02 14:54:10.173050 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:54:10.173104 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 91180000070d00001f00000091180000 [2022-03-02 14:54:10.173126 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000ee0" to be run at: [2022-03-02 14:54:16 -0500] [2022-03-02 14:54:16.173661 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000ee0" [2022-03-02 14:54:16.174203 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:54:16.174255 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 91180000070d00001f00000091180000 [2022-03-02 14:54:16.174318 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000ea0" to be run at: [2022-03-02 14:54:24 -0500] [2022-03-02 14:54:24.174751 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000ea0" [2022-03-02 14:54:24.175177 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:54:24.175256 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 91180000070d00001f00000091180000 [2022-03-02 14:54:24.175323 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001160" to be run at: [2022-03-02 14:54:34 -0500] [2022-03-02 14:54:34.176169 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001160" [2022-03-02 14:54:34.176516 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:54:34.176592 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 91180000070d00001f00000091180000 [2022-03-02 14:54:34.176626 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000ee0" to be run at: [2022-03-02 14:54:46 -0500] [2022-03-02 14:54:46.177482 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000ee0" [2022-03-02 14:54:46.177931 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:54:46.178070 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 91180000070d00001f00000091180000 [2022-03-02 14:54:46.178113 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000ea0" to be run at: [2022-03-02 14:55:00 -0500] [2022-03-02 14:55:00.179367 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000ea0" [2022-03-02 14:55:00.179841 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:55:00.179950 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 91180000070d00001f00000091180000 [2022-03-02 14:55:00.180014 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001160" to be run at: [2022-03-02 14:55:16 -0500] [2022-03-02 14:55:16.181145 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001160" [2022-03-02 14:55:16.181421 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:55:16.181503 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 91180000070d00001f00000091180000 [2022-03-02 14:55:16.181536 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0000ee0" to be run at: [2022-03-02 14:55:34 -0500] [2022-03-02 14:55:34.183748 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0000ee0" [2022-03-02 14:55:34.184146 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:55:34.184287 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 91180000070d00001f00000091180000 [2022-03-02 14:55:34.184330 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001160" to be run at: [2022-03-02 14:55:54 -0500] [2022-03-02 14:55:54.186528 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001160" [2022-03-02 14:55:54.186968 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:55:54.187102 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 91180000070d00001f00000091180000 [2022-03-02 14:55:54.187149 -0500][Error ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Wont retry kXR_endsess request because wouldreach connection timeout. [2022-03-02 14:55:54.187187 -0500][Error ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Socket error while handshaking: [ERROR] Socket timeout [2022-03-02 14:55:54.187210 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 14:55:54.187233 -0500][Debug ][Poller ] <[::ffff:10.32.77.18]:13992><--><[::ffff:149.165.243.5]:1094> Removing socket from the poller [2022-03-02 14:55:54.187293 -0500][Error ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] elapsed = 108, pConnectionWindow = 120 seconds. [2022-03-02 14:55:54.187316 -0500][Info ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Attempting reconnection in 12 seconds. [2022-03-02 14:55:54.187377 -0500][Debug ][TaskMgr ] Registering task: "StreamConnectorTask for primeradiant01.crc.nd.edu:1094 #0" to be run at: [2022-03-02 14:56:06 -0500] [2022-03-02 14:56:06.189320 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094] Found 1 address(es): [::ffff:149.165.243.5]:1094 [2022-03-02 14:56:06.189395 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:149.165.243.5]:1094 [2022-03-02 14:56:06.189446 -0500][Debug ][Poller ] Adding socket 0xc8001b30 to the poller [2022-03-02 14:56:06.189491 -0500][Debug ][TaskMgr ] Done with task: "StreamConnectorTask for primeradiant01.crc.nd.edu:1094 #0" [2022-03-02 14:56:06.189650 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Async connection call returned [2022-03-02 14:56:06.189746 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol [2022-03-02 14:56:06.189967 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: server [], protocol version 500) [2022-03-02 14:56:06.190004 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: server [proxy], protocol version 500) [2022-03-02 14:56:06.192030 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=earth.crc.nd.edu&xrd.rn=v4.12.6, dual-stack: false, private IPv4: true, private IPv6: false [2022-03-02 14:56:06.192263 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Logged in, session: 96180000070d00003500000096180000 [2022-03-02 14:56:06.192307 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 93180000070d00002f00000093180000 [2022-03-02 14:56:06.192447 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Stream 0 connected. [2022-03-02 14:56:06.192494 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc0002140 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:56:06.192561 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc8002290 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:56:06.192600 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc0002610 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:56:06.192790 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc0002140 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 14:56:06.192789 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:56:06.192844 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 14:56:06.192844 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 14:56:06.192867 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc0002140. [2022-03-02 14:56:06.192894 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc8002290 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 14:56:06.192952 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 14:56:06.192952 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 14:56:06.193003 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc8002290. [2022-03-02 14:56:06.192901 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc0002610 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 14:56:06.193090 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 14:56:06.193146 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 14:56:06.193146 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 14:56:06.193245 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Open has returned with status [SUCCESS] [2022-03-02 14:56:06.193261 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc0002610. [2022-03-02 14:56:06.193314 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] successfully opened at primeradiant01.crc.nd.edu:1094, handle: 0x3, session id: 8 [2022-03-02 14:56:06.193439 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xcc002b00 (message: kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608) ). [2022-03-02 14:56:06.193519 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xcc0030d0 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ). [2022-03-02 14:56:06.193541 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc002b00 (message: kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:56:06.193568 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xcc003770 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ). [2022-03-02 14:56:06.193604 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc0030d0 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:56:06.193631 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xcc003a00 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ). [2022-03-02 14:56:06.193648 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc003770 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:56:06.193711 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 14:56:06.193711 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 14:56:06.193728 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc003a00 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:56:06.193775 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc001350. [2022-03-02 14:57:06.254431 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 14:57:06.254509 -0500][Debug ][Poller ] <[::ffff:10.32.77.18]:14074><--><[::ffff:149.165.243.5]:1094> Removing socket from the poller [2022-03-02 14:57:06.254584 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Recovering error for stream #0: [ERROR] Socket error. [2022-03-02 14:57:06.254651 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Reporting disconnection to queued message handlers. [2022-03-02 14:57:06.254677 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608): [ERROR] Socket error. [2022-03-02 14:57:06.254698 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608) [2022-03-02 14:57:06.254723 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xcc002b00 (message: kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608) ). [2022-03-02 14:57:06.254757 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608): [ERROR] Socket error. [2022-03-02 14:57:06.254780 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) [2022-03-02 14:57:06.254811 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xcc0030d0 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ). [2022-03-02 14:57:06.254810 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc002b00 (message: kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:57:06.254948 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608): [ERROR] Socket error. [2022-03-02 14:57:06.255012 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 14:57:06.255076 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) [2022-03-02 14:57:06.255108 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xcc003770 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ). [2022-03-02 14:57:06.255108 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc0030d0 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:57:06.255124 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xc8002570 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:57:06.255174 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608): [ERROR] Socket error. [2022-03-02 14:57:06.255279 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc003770 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:57:06.255332 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) [2022-03-02 14:57:06.255340 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc002b00. [2022-03-02 14:57:06.255353 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 14:57:06.255405 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xcc003a00 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ). [2022-03-02 14:57:06.255513 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xc0002140 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:57:06.255515 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc003a00 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 14:57:06.255535 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xc8002570 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:57:06.255571 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc0030d0. [2022-03-02 14:57:06.255642 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xc0002140 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:57:06.255586 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 14:57:06.255739 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:57:06.255792 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc003770. [2022-03-02 14:57:06.255797 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:57:06.255808 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 14:57:06.255909 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xc8002de0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:57:06.255954 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc003a00. [2022-03-02 14:57:06.255983 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xc8002de0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:57:06.256090 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xc8002570 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:57:06.256317 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xc0002140 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:57:06.256329 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:57:06.256966 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094] Found 1 address(es): [::ffff:149.165.243.5]:1094 [2022-03-02 14:57:06.257018 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:149.165.243.5]:1094 [2022-03-02 14:57:06.257093 -0500][Debug ][Poller ] Adding socket 0xc8001b30 to the poller [2022-03-02 14:57:06.257226 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xc8002de0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 14:57:06.257300 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Async connection call returned [2022-03-02 14:57:06.257356 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol [2022-03-02 14:57:06.257550 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: server [], protocol version 500) [2022-03-02 14:57:06.257586 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: server [proxy], protocol version 500) [2022-03-02 14:57:06.258687 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=earth.crc.nd.edu&xrd.rn=v4.12.6, dual-stack: false, private IPv4: true, private IPv6: false [2022-03-02 14:57:06.258882 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Logged in, session: 97180000070d00002f00000097180000 [2022-03-02 14:57:06.258925 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 96180000070d00003500000096180000 [2022-03-02 14:57:06.259067 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:57:06.259104 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 96180000070d00003500000096180000 [2022-03-02 14:57:06.259136 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad00012a0" to be run at: [2022-03-02 14:57:10 -0500] [2022-03-02 14:57:10.199337 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad00012a0" [2022-03-02 14:57:10.199711 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:57:10.199754 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 96180000070d00003500000096180000 [2022-03-02 14:57:10.199816 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad00012a0" to be run at: [2022-03-02 14:57:16 -0500] [2022-03-02 14:57:16.200464 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad00012a0" [2022-03-02 14:57:16.200910 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:57:16.200954 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 96180000070d00003500000096180000 [2022-03-02 14:57:16.201018 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001250" to be run at: [2022-03-02 14:57:24 -0500] [2022-03-02 14:57:24.201817 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001250" [2022-03-02 14:57:24.202098 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:57:24.202134 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 96180000070d00003500000096180000 [2022-03-02 14:57:24.202196 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001290" to be run at: [2022-03-02 14:57:34 -0500] [2022-03-02 14:57:34.203224 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001290" [2022-03-02 14:57:34.203518 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:57:34.203554 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 96180000070d00003500000096180000 [2022-03-02 14:57:34.203621 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad00012d0" to be run at: [2022-03-02 14:57:46 -0500] [2022-03-02 14:57:46.205357 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad00012d0" [2022-03-02 14:57:46.205635 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:57:46.205658 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 96180000070d00003500000096180000 [2022-03-02 14:57:46.205680 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001250" to be run at: [2022-03-02 14:58:00 -0500] [2022-03-02 14:58:00.207770 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001250" [2022-03-02 14:58:00.208185 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:58:00.208269 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 96180000070d00003500000096180000 [2022-03-02 14:58:00.208304 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001290" to be run at: [2022-03-02 14:58:16 -0500] [2022-03-02 14:58:16.210598 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001290" [2022-03-02 14:58:16.211101 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:58:16.211151 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 96180000070d00003500000096180000 [2022-03-02 14:58:16.211191 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad00012d0" to be run at: [2022-03-02 14:58:34 -0500] [2022-03-02 14:58:34.213793 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad00012d0" [2022-03-02 14:58:34.214144 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:58:34.214231 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 96180000070d00003500000096180000 [2022-03-02 14:58:34.214265 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001250" to be run at: [2022-03-02 14:58:54 -0500] [2022-03-02 14:58:54.217191 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001250" [2022-03-02 14:58:54.217662 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 14:58:54.217707 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 96180000070d00003500000096180000 [2022-03-02 14:58:54.217764 -0500][Error ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Wont retry kXR_endsess request because wouldreach connection timeout. [2022-03-02 14:58:54.217823 -0500][Error ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Socket error while handshaking: [ERROR] Socket timeout [2022-03-02 14:58:54.217873 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 14:58:54.217925 -0500][Debug ][Poller ] <[::ffff:10.32.77.18]:14096><--><[::ffff:149.165.243.5]:1094> Removing socket from the poller [2022-03-02 14:58:54.218003 -0500][Error ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] elapsed = 108, pConnectionWindow = 120 seconds. [2022-03-02 14:58:54.218029 -0500][Info ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Attempting reconnection in 12 seconds. [2022-03-02 14:58:54.218079 -0500][Debug ][TaskMgr ] Registering task: "StreamConnectorTask for primeradiant01.crc.nd.edu:1094 #0" to be run at: [2022-03-02 14:59:06 -0500] [2022-03-02 14:59:06.220324 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094] Found 1 address(es): [::ffff:149.165.243.5]:1094 [2022-03-02 14:59:06.220381 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:149.165.243.5]:1094 [2022-03-02 14:59:06.220423 -0500][Debug ][Poller ] Adding socket 0xc8001b30 to the poller [2022-03-02 14:59:06.220462 -0500][Debug ][TaskMgr ] Done with task: "StreamConnectorTask for primeradiant01.crc.nd.edu:1094 #0" [2022-03-02 14:59:06.220618 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Async connection call returned [2022-03-02 14:59:06.220668 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol [2022-03-02 14:59:06.220880 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: server [], protocol version 500) [2022-03-02 14:59:06.220909 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: server [proxy], protocol version 500) [2022-03-02 14:59:06.221982 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=earth.crc.nd.edu&xrd.rn=v4.12.6, dual-stack: false, private IPv4: true, private IPv6: false [2022-03-02 14:59:06.222236 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Logged in, session: 99180000070d00003400000099180000 [2022-03-02 14:59:06.222267 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 97180000070d00002f00000097180000 [2022-03-02 14:59:06.222367 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Stream 0 connected. [2022-03-02 14:59:06.222402 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc8002570 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:59:06.222438 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc0002140 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:59:06.222468 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:59:06.222509 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc8002de0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 14:59:06.222644 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc8002570 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 14:59:06.222684 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 14:59:06.222684 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 14:59:06.222683 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc0002140 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 14:59:06.222752 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 14:59:06.222806 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 14:59:06.222806 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 14:59:06.222824 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc001350. [2022-03-02 14:59:06.222732 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc8002570. [2022-03-02 14:59:06.222847 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc8002de0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 14:59:06.222880 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 14:59:06.222880 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 14:59:06.222912 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc0002140. [2022-03-02 14:59:06.222939 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Open has returned with status [SUCCESS] [2022-03-02 14:59:06.222955 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] successfully opened at primeradiant01.crc.nd.edu:1094, handle: 0x3, session id: 9 [2022-03-02 14:59:06.222984 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xc0002510 (message: kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608) ). [2022-03-02 14:59:06.223023 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xc00032f0 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ). [2022-03-02 14:59:06.223058 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc0002510 (message: kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:59:06.223076 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xc0003580 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ). [2022-03-02 14:59:06.223098 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc00032f0 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:59:06.223105 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xc0003810 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ). [2022-03-02 14:59:06.223127 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc0003580 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:59:06.223141 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 14:59:06.223141 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 14:59:06.223155 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc0003810 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ) from out-queu to in-queue. [2022-03-02 14:59:06.223162 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc8002de0. [2022-03-02 15:00:06.281219 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 15:00:06.281306 -0500][Debug ][Poller ] <[::ffff:10.32.77.18]:14146><--><[::ffff:149.165.243.5]:1094> Removing socket from the poller [2022-03-02 15:00:06.281394 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Recovering error for stream #0: [ERROR] Socket error. [2022-03-02 15:00:06.281419 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Reporting disconnection to queued message handlers. [2022-03-02 15:00:06.281445 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608): [ERROR] Socket error. [2022-03-02 15:00:06.281467 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608) [2022-03-02 15:00:06.281528 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xc0002510 (message: kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608) ). [2022-03-02 15:00:06.281561 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608): [ERROR] Socket error. [2022-03-02 15:00:06.281583 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) [2022-03-02 15:00:06.281603 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xc0003580 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ). [2022-03-02 15:00:06.281632 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608): [ERROR] Socket error. [2022-03-02 15:00:06.281652 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) [2022-03-02 15:00:06.281674 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xc00032f0 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ). [2022-03-02 15:00:06.281709 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc0002510 (message: kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 15:00:06.281777 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc0003580 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 15:00:06.281737 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608): [ERROR] Socket error. [2022-03-02 15:00:06.281822 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 15:00:06.281882 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc00032f0 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 15:00:06.281932 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) [2022-03-02 15:00:06.282010 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xc0003810 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ). [2022-03-02 15:00:06.282014 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:00:06.282136 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc0002510. [2022-03-02 15:00:06.282150 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:00:06.282151 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 15:00:06.282203 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc0003810 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 15:00:06.282296 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xc8002570 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:00:06.282362 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc0003580. [2022-03-02 15:00:06.282367 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xc8002570 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:00:06.282378 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 15:00:06.282483 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xc0002510 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:00:06.282537 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc00032f0. [2022-03-02 15:00:06.282546 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xc0002510 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:00:06.282548 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 15:00:06.282650 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xcc0030d0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:00:06.282700 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc0003810. [2022-03-02 15:00:06.282800 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xcc0030d0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:00:06.282880 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:00:06.283019 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xc8002570 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:00:06.283022 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xc0002510 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:00:06.283575 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094] Found 1 address(es): [::ffff:149.165.243.5]:1094 [2022-03-02 15:00:06.283624 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:149.165.243.5]:1094 [2022-03-02 15:00:06.283674 -0500][Debug ][Poller ] Adding socket 0xc8001b30 to the poller [2022-03-02 15:00:06.283805 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xcc0030d0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:00:06.283820 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Async connection call returned [2022-03-02 15:00:06.283862 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol [2022-03-02 15:00:06.284053 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: server [], protocol version 500) [2022-03-02 15:00:06.284078 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: server [proxy], protocol version 500) [2022-03-02 15:00:06.285119 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=earth.crc.nd.edu&xrd.rn=v4.12.6, dual-stack: false, private IPv4: true, private IPv6: false [2022-03-02 15:00:06.285342 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Logged in, session: 9a180000070d00001d0000009a180000 [2022-03-02 15:00:06.285366 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 99180000070d00003400000099180000 [2022-03-02 15:00:06.285515 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:00:06.285568 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 99180000070d00003400000099180000 [2022-03-02 15:00:06.285604 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001240" to be run at: [2022-03-02 15:00:10 -0500] [2022-03-02 15:00:10.230876 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001240" [2022-03-02 15:00:10.231273 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:00:10.231341 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 99180000070d00003400000099180000 [2022-03-02 15:00:10.231386 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001240" to be run at: [2022-03-02 15:00:16 -0500] [2022-03-02 15:00:16.231913 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001240" [2022-03-02 15:00:16.232301 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:00:16.232393 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 99180000070d00003400000099180000 [2022-03-02 15:00:16.232454 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001240" to be run at: [2022-03-02 15:00:24 -0500] [2022-03-02 15:00:24.233243 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001240" [2022-03-02 15:00:24.233608 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:00:24.233701 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 99180000070d00003400000099180000 [2022-03-02 15:00:24.233765 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001240" to be run at: [2022-03-02 15:00:34 -0500] [2022-03-02 15:00:34.235008 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001240" [2022-03-02 15:00:34.235471 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:00:34.235542 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 99180000070d00003400000099180000 [2022-03-02 15:00:34.235565 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001240" to be run at: [2022-03-02 15:00:46 -0500] [2022-03-02 15:00:46.237153 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001240" [2022-03-02 15:00:46.237600 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:00:46.237673 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 99180000070d00003400000099180000 [2022-03-02 15:00:46.237697 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001240" to be run at: [2022-03-02 15:01:00 -0500] [2022-03-02 15:01:00.239521 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001240" [2022-03-02 15:01:00.239969 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:01:00.240000 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 99180000070d00003400000099180000 [2022-03-02 15:01:00.240021 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001240" to be run at: [2022-03-02 15:01:16 -0500] [2022-03-02 15:01:16.242171 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001240" [2022-03-02 15:01:16.242406 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:01:16.242436 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 99180000070d00003400000099180000 [2022-03-02 15:01:16.242458 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001240" to be run at: [2022-03-02 15:01:34 -0500] [2022-03-02 15:01:34.245262 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001240" [2022-03-02 15:01:34.245487 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:01:34.245517 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 99180000070d00003400000099180000 [2022-03-02 15:01:34.245541 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001240" to be run at: [2022-03-02 15:01:54 -0500] [2022-03-02 15:01:54.248745 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001240" [2022-03-02 15:01:54.248981 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:01:54.249010 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 99180000070d00003400000099180000 [2022-03-02 15:01:54.249025 -0500][Error ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Wont retry kXR_endsess request because wouldreach connection timeout. [2022-03-02 15:01:54.249066 -0500][Error ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Socket error while handshaking: [ERROR] Socket timeout [2022-03-02 15:01:54.249079 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 15:01:54.249091 -0500][Debug ][Poller ] <[::ffff:10.32.77.18]:14154><--><[::ffff:149.165.243.5]:1094> Removing socket from the poller [2022-03-02 15:01:54.249131 -0500][Error ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] elapsed = 108, pConnectionWindow = 120 seconds. [2022-03-02 15:01:54.249144 -0500][Info ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Attempting reconnection in 12 seconds. [2022-03-02 15:01:54.249158 -0500][Debug ][TaskMgr ] Registering task: "StreamConnectorTask for primeradiant01.crc.nd.edu:1094 #0" to be run at: [2022-03-02 15:02:06 -0500] [2022-03-02 15:02:06.251407 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094] Found 1 address(es): [::ffff:149.165.243.5]:1094 [2022-03-02 15:02:06.251507 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:149.165.243.5]:1094 [2022-03-02 15:02:06.251563 -0500][Debug ][Poller ] Adding socket 0xc8001b30 to the poller [2022-03-02 15:02:06.251608 -0500][Debug ][TaskMgr ] Done with task: "StreamConnectorTask for primeradiant01.crc.nd.edu:1094 #0" [2022-03-02 15:02:06.251813 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Async connection call returned [2022-03-02 15:02:06.251882 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol [2022-03-02 15:02:06.252207 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: server [], protocol version 500) [2022-03-02 15:02:06.252295 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: server [proxy], protocol version 500) [2022-03-02 15:02:06.253723 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=earth.crc.nd.edu&xrd.rn=v4.12.6, dual-stack: false, private IPv4: true, private IPv6: false [2022-03-02 15:02:06.253977 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Logged in, session: 9b180000070d00002f0000009b180000 [2022-03-02 15:02:06.254022 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9a180000070d00001d0000009a180000 [2022-03-02 15:02:06.254183 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Stream 0 connected. [2022-03-02 15:02:06.254233 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:02:06.254280 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc8002570 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:02:06.254319 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc0002510 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:02:06.254451 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc0030d0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:02:06.254565 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 15:02:06.254628 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc8002570 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 15:02:06.254628 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc0002510 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 15:02:06.254646 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 15:02:06.254646 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 15:02:06.254720 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 15:02:06.254720 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 15:02:06.254746 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 15:02:06.254746 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 15:02:06.254786 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc001350. [2022-03-02 15:02:06.254841 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc8002570. [2022-03-02 15:02:06.254900 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc0002510. [2022-03-02 15:02:06.254943 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc0030d0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 15:02:06.255082 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Open has returned with status [SUCCESS] [2022-03-02 15:02:06.255112 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] successfully opened at primeradiant01.crc.nd.edu:1094, handle: 0x3, session id: 10 [2022-03-02 15:02:06.255155 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xc0002510 (message: kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608) ). [2022-03-02 15:02:06.255206 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xc0002cc0 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ). [2022-03-02 15:02:06.255223 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc0002510 (message: kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608) ) from out-queu to in-queue. [2022-03-02 15:02:06.255278 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xc0003060 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ). [2022-03-02 15:02:06.255331 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc0002cc0 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ) from out-queu to in-queue. [2022-03-02 15:02:06.255353 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xc00032f0 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ). [2022-03-02 15:02:06.255373 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc0003060 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ) from out-queu to in-queue. [2022-03-02 15:02:06.255397 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 15:02:06.255397 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 15:02:06.255417 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc00032f0 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ) from out-queu to in-queue. [2022-03-02 15:02:06.255467 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc0030d0. [2022-03-02 15:03:06.316124 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 15:03:06.316205 -0500][Debug ][Poller ] <[::ffff:10.32.77.18]:14168><--><[::ffff:149.165.243.5]:1094> Removing socket from the poller [2022-03-02 15:03:06.316280 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Recovering error for stream #0: [ERROR] Socket error. [2022-03-02 15:03:06.316340 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Reporting disconnection to queued message handlers. [2022-03-02 15:03:06.316366 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608): [ERROR] Socket error. [2022-03-02 15:03:06.316388 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608) [2022-03-02 15:03:06.316410 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xc0002510 (message: kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608) ). [2022-03-02 15:03:06.316441 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608): [ERROR] Socket error. [2022-03-02 15:03:06.316463 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) [2022-03-02 15:03:06.316490 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xc0002cc0 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ). [2022-03-02 15:03:06.316520 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608): [ERROR] Socket error. [2022-03-02 15:03:06.316541 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) [2022-03-02 15:03:06.316561 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xc0003060 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ). [2022-03-02 15:03:06.316504 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc0002510 (message: kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 15:03:06.316619 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608): [ERROR] Socket error. [2022-03-02 15:03:06.316630 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc0002cc0 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 15:03:06.316720 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 15:03:06.316756 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) [2022-03-02 15:03:06.316759 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc0003060 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 15:03:06.316868 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xc00032f0 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ). [2022-03-02 15:03:06.316882 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:03:06.317011 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc0002510. [2022-03-02 15:03:06.317022 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 15:03:06.317027 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:03:06.317141 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xc8002ce0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:03:06.317081 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc00032f0 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 15:03:06.317217 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc0002cc0. [2022-03-02 15:03:06.317224 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xc8002ce0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:03:06.317233 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 15:03:06.317363 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xc0002cc0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:03:06.317433 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc0003060. [2022-03-02 15:03:06.317442 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xc0002cc0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:03:06.317445 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 15:03:06.317544 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xcc0030d0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:03:06.317586 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc00032f0. [2022-03-02 15:03:06.317602 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xcc0030d0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:03:06.317779 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:03:06.318016 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xc8002ce0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:03:06.318019 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xc0002cc0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:03:06.318569 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094] Found 1 address(es): [::ffff:149.165.243.5]:1094 [2022-03-02 15:03:06.318614 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:149.165.243.5]:1094 [2022-03-02 15:03:06.318662 -0500][Debug ][Poller ] Adding socket 0xc8001b30 to the poller [2022-03-02 15:03:06.318768 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Async connection call returned [2022-03-02 15:03:06.318803 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xcc0030d0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:03:06.318832 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol [2022-03-02 15:03:06.319062 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: server [], protocol version 500) [2022-03-02 15:03:06.319100 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: server [proxy], protocol version 500) [2022-03-02 15:03:06.320713 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=earth.crc.nd.edu&xrd.rn=v4.12.6, dual-stack: false, private IPv4: true, private IPv6: false [2022-03-02 15:03:06.320953 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Logged in, session: 9c180000070d00001d0000009c180000 [2022-03-02 15:03:06.320997 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9b180000070d00002f0000009b180000 [2022-03-02 15:03:06.321175 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:03:06.321207 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9b180000070d00002f0000009b180000 [2022-03-02 15:03:06.321242 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad00013d0" to be run at: [2022-03-02 15:03:10 -0500] [2022-03-02 15:03:10.261950 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad00013d0" [2022-03-02 15:03:10.262309 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:03:10.262383 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9b180000070d00002f0000009b180000 [2022-03-02 15:03:10.262416 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001390" to be run at: [2022-03-02 15:03:16 -0500] [2022-03-02 15:03:16.262995 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001390" [2022-03-02 15:03:16.263332 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:03:16.263390 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9b180000070d00002f0000009b180000 [2022-03-02 15:03:16.263449 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001350" to be run at: [2022-03-02 15:03:24 -0500] [2022-03-02 15:03:24.264344 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001350" [2022-03-02 15:03:24.264659 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:03:24.264700 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9b180000070d00002f0000009b180000 [2022-03-02 15:03:24.264767 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad00013d0" to be run at: [2022-03-02 15:03:34 -0500] [2022-03-02 15:03:34.266112 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad00013d0" [2022-03-02 15:03:34.266554 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:03:34.266646 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9b180000070d00002f0000009b180000 [2022-03-02 15:03:34.266681 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001390" to be run at: [2022-03-02 15:03:46 -0500] [2022-03-02 15:03:46.268191 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001390" [2022-03-02 15:03:46.268675 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:03:46.268765 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9b180000070d00002f0000009b180000 [2022-03-02 15:03:46.268800 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001350" to be run at: [2022-03-02 15:04:00 -0500] [2022-03-02 15:04:00.270576 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001350" [2022-03-02 15:04:00.270987 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:04:00.271118 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9b180000070d00002f0000009b180000 [2022-03-02 15:04:00.271188 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad00013d0" to be run at: [2022-03-02 15:04:16 -0500] [2022-03-02 15:04:16.273276 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad00013d0" [2022-03-02 15:04:16.273585 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:04:16.273622 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9b180000070d00002f0000009b180000 [2022-03-02 15:04:16.273686 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001390" to be run at: [2022-03-02 15:04:34 -0500] [2022-03-02 15:04:34.276311 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001390" [2022-03-02 15:04:34.276797 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:04:34.276888 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9b180000070d00002f0000009b180000 [2022-03-02 15:04:34.276922 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001350" to be run at: [2022-03-02 15:04:54 -0500] [2022-03-02 15:04:54.279656 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001350" [2022-03-02 15:04:54.280064 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:04:54.280143 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9b180000070d00002f0000009b180000 [2022-03-02 15:04:54.280162 -0500][Error ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Wont retry kXR_endsess request because wouldreach connection timeout. [2022-03-02 15:04:54.280184 -0500][Error ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Socket error while handshaking: [ERROR] Socket timeout [2022-03-02 15:04:54.280197 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 15:04:54.280214 -0500][Debug ][Poller ] <[::ffff:10.32.77.18]:14182><--><[::ffff:149.165.243.5]:1094> Removing socket from the poller [2022-03-02 15:04:54.280268 -0500][Error ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] elapsed = 108, pConnectionWindow = 120 seconds. [2022-03-02 15:04:54.280285 -0500][Info ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Attempting reconnection in 12 seconds. [2022-03-02 15:04:54.280299 -0500][Debug ][TaskMgr ] Registering task: "StreamConnectorTask for primeradiant01.crc.nd.edu:1094 #0" to be run at: [2022-03-02 15:05:06 -0500] [2022-03-02 15:05:06.282431 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094] Found 1 address(es): [::ffff:149.165.243.5]:1094 [2022-03-02 15:05:06.282528 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:149.165.243.5]:1094 [2022-03-02 15:05:06.282584 -0500][Debug ][Poller ] Adding socket 0xc8001b30 to the poller [2022-03-02 15:05:06.282630 -0500][Debug ][TaskMgr ] Done with task: "StreamConnectorTask for primeradiant01.crc.nd.edu:1094 #0" [2022-03-02 15:05:06.282821 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Async connection call returned [2022-03-02 15:05:06.282906 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol [2022-03-02 15:05:06.283160 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: server [], protocol version 500) [2022-03-02 15:05:06.283197 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: server [proxy], protocol version 500) [2022-03-02 15:05:06.284211 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=earth.crc.nd.edu&xrd.rn=v4.12.6, dual-stack: false, private IPv4: true, private IPv6: false [2022-03-02 15:05:06.284405 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Logged in, session: 9d180000070d0000320000009d180000 [2022-03-02 15:05:06.284443 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9c180000070d00001d0000009c180000 [2022-03-02 15:05:06.284581 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Stream 0 connected. [2022-03-02 15:05:06.284627 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:05:06.284678 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc8002ce0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:05:06.284725 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc0002cc0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:05:06.284809 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc0030d0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:05:06.284836 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc001350 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 15:05:06.284895 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 15:05:06.284895 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 15:05:06.284914 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc001350. [2022-03-02 15:05:06.284938 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc8002ce0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 15:05:06.284963 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 15:05:06.284963 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 15:05:06.284979 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc8002ce0. [2022-03-02 15:05:06.285024 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc0002cc0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 15:05:06.285024 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc0030d0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 15:05:06.285121 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 15:05:06.285121 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 15:05:06.285144 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc0002cc0. [2022-03-02 15:05:06.285158 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Open has returned with status [SUCCESS] [2022-03-02 15:05:06.285180 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] successfully opened at primeradiant01.crc.nd.edu:1094, handle: 0x3, session id: 11 [2022-03-02 15:05:06.285215 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xcc003950 (message: kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608) ). [2022-03-02 15:05:06.285255 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xcc003be0 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ). [2022-03-02 15:05:06.285264 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc003950 (message: kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608) ) from out-queu to in-queue. [2022-03-02 15:05:06.285281 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xcc003e70 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ). [2022-03-02 15:05:06.285299 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc003be0 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ) from out-queu to in-queue. [2022-03-02 15:05:06.285309 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xcc004100 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ). [2022-03-02 15:05:06.285340 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc003e70 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ) from out-queu to in-queue. [2022-03-02 15:05:06.285371 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 15:05:06.285371 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 15:05:06.285371 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc004100 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ) from out-queu to in-queue. [2022-03-02 15:05:06.285398 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc0030d0. [2022-03-02 15:06:06.312794 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 15:06:06.312873 -0500][Debug ][Poller ] <[::ffff:10.32.77.18]:14194><--><[::ffff:149.165.243.5]:1094> Removing socket from the poller [2022-03-02 15:06:06.312953 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Recovering error for stream #0: [ERROR] Socket error. [2022-03-02 15:06:06.312978 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Reporting disconnection to queued message handlers. [2022-03-02 15:06:06.313004 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608): [ERROR] Socket error. [2022-03-02 15:06:06.313083 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608) [2022-03-02 15:06:06.313106 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xcc003950 (message: kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608) ). [2022-03-02 15:06:06.313136 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608): [ERROR] Socket error. [2022-03-02 15:06:06.313186 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) [2022-03-02 15:06:06.313208 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xcc003be0 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ). [2022-03-02 15:06:06.313270 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608): [ERROR] Socket error. [2022-03-02 15:06:06.313286 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc003950 (message: kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 15:06:06.313342 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) [2022-03-02 15:06:06.313411 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc003be0 (message: kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 15:06:06.313427 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 15:06:06.313468 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xcc003e70 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ). [2022-03-02 15:06:06.313588 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608): [ERROR] Socket error. [2022-03-02 15:06:06.313593 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xc0002cc0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:06:06.313632 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) [2022-03-02 15:06:06.313687 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xcc004100 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ). [2022-03-02 15:06:06.313706 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 15:06:06.313694 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc003e70 (message: kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 15:06:06.313689 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc003950. [2022-03-02 15:06:06.313763 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xc0002cc0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:06:06.313800 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xc8002290 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:06:06.313840 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc004100 (message: kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 15:06:06.313960 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc003be0. [2022-03-02 15:06:06.313969 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xc8002290 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:06:06.313973 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 15:06:06.314090 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xcc0030d0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:06:06.314139 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc003e70. [2022-03-02 15:06:06.314149 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 15:06:06.314147 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xcc0030d0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:06:06.314220 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xc00033e0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:06:06.314279 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc004100. [2022-03-02 15:06:06.314289 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xc00033e0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:06:06.314373 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xc0002cc0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:06:06.314469 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xc8002290 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:06:06.314649 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xcc0030d0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:06:06.315144 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094] Found 1 address(es): [::ffff:149.165.243.5]:1094 [2022-03-02 15:06:06.315214 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:149.165.243.5]:1094 [2022-03-02 15:06:06.315272 -0500][Debug ][Poller ] Adding socket 0xc8001b30 to the poller [2022-03-02 15:06:06.315454 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xc00033e0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:06:06.315457 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Async connection call returned [2022-03-02 15:06:06.315545 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol [2022-03-02 15:06:06.315736 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: server [], protocol version 500) [2022-03-02 15:06:06.315776 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: server [proxy], protocol version 500) [2022-03-02 15:06:06.317634 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=earth.crc.nd.edu&xrd.rn=v4.12.6, dual-stack: false, private IPv4: true, private IPv6: false [2022-03-02 15:06:06.317856 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Logged in, session: 9e180000070d00001d0000009e180000 [2022-03-02 15:06:06.317888 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9d180000070d0000320000009d180000 [2022-03-02 15:06:06.318020 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:06:06.318062 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9d180000070d0000320000009d180000 [2022-03-02 15:06:06.318100 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001450" to be run at: [2022-03-02 15:06:10 -0500] [2022-03-02 15:06:10.292810 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001450" [2022-03-02 15:06:10.293173 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:06:10.293207 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9d180000070d0000320000009d180000 [2022-03-02 15:06:10.293270 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001490" to be run at: [2022-03-02 15:06:16 -0500] [2022-03-02 15:06:16.293929 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001490" [2022-03-02 15:06:16.294282 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:06:16.294318 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9d180000070d0000320000009d180000 [2022-03-02 15:06:16.294385 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad00014d0" to be run at: [2022-03-02 15:06:24 -0500] [2022-03-02 15:06:24.295292 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad00014d0" [2022-03-02 15:06:24.295610 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:06:24.295646 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9d180000070d0000320000009d180000 [2022-03-02 15:06:24.295685 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001450" to be run at: [2022-03-02 15:06:34 -0500] [2022-03-02 15:06:34.297026 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001450" [2022-03-02 15:06:34.297411 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:06:34.297450 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9d180000070d0000320000009d180000 [2022-03-02 15:06:34.297520 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001490" to be run at: [2022-03-02 15:06:46 -0500] [2022-03-02 15:06:46.299097 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001490" [2022-03-02 15:06:46.299405 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:06:46.299441 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9d180000070d0000320000009d180000 [2022-03-02 15:06:46.299505 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad00015b0" to be run at: [2022-03-02 15:07:00 -0500] [2022-03-02 15:07:00.301467 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad00015b0" [2022-03-02 15:07:00.301800 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:07:00.301865 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9d180000070d0000320000009d180000 [2022-03-02 15:07:00.301887 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001450" to be run at: [2022-03-02 15:07:16 -0500] [2022-03-02 15:07:16.304106 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001450" [2022-03-02 15:07:16.304448 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:07:16.304485 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9d180000070d0000320000009d180000 [2022-03-02 15:07:16.304550 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001490" to be run at: [2022-03-02 15:07:34 -0500] [2022-03-02 15:07:34.307184 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001490" [2022-03-02 15:07:34.307511 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:07:34.307585 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9d180000070d0000320000009d180000 [2022-03-02 15:07:34.307609 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad00015b0" to be run at: [2022-03-02 15:07:54 -0500] [2022-03-02 15:07:45.309199 -0500][Error ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Fatal file state error. Message kXR_read (handle: 0x03000000, offset: 16777216, size: 8388608) returned with [ERROR] Operation expired [2022-03-02 15:07:45.309303 -0500][Error ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Fatal file state error. Message kXR_read (handle: 0x03000000, offset: 25165824, size: 8388608) returned with [ERROR] Operation expired [2022-03-02 15:07:45.309417 -0500][Error ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Fatal file state error. Message kXR_read (handle: 0x03000000, offset: 8388608, size: 8388608) returned with [ERROR] Operation expired [2022-03-02 15:07:45.309523 -0500][Debug ][Utility ] Unable read 8388608 bytes at 8388608 from root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root: [ERROR] Operation expired [2022-03-02 15:07:54.310607 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad00015b0" [2022-03-02 15:07:54.311006 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:07:54.311121 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9d180000070d0000320000009d180000 [2022-03-02 15:07:54.311147 -0500][Error ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Wont retry kXR_endsess request because wouldreach connection timeout. [2022-03-02 15:07:54.311173 -0500][Error ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Socket error while handshaking: [ERROR] Socket timeout [2022-03-02 15:07:54.311194 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 15:07:54.311218 -0500][Debug ][Poller ] <[::ffff:10.32.77.18]:14340><--><[::ffff:149.165.243.5]:1094> Removing socket from the poller [2022-03-02 15:07:54.311278 -0500][Error ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] elapsed = 108, pConnectionWindow = 120 seconds. [2022-03-02 15:07:54.311337 -0500][Info ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Attempting reconnection in 12 seconds. [2022-03-02 15:07:54.311361 -0500][Debug ][TaskMgr ] Registering task: "StreamConnectorTask for primeradiant01.crc.nd.edu:1094 #0" to be run at: [2022-03-02 15:08:06 -0500] [2022-03-02 15:08:06.313269 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094] Found 1 address(es): [::ffff:149.165.243.5]:1094 [2022-03-02 15:08:06.313339 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:149.165.243.5]:1094 [2022-03-02 15:08:06.313390 -0500][Debug ][Poller ] Adding socket 0xc8001b30 to the poller [2022-03-02 15:08:06.313435 -0500][Debug ][TaskMgr ] Done with task: "StreamConnectorTask for primeradiant01.crc.nd.edu:1094 #0" [2022-03-02 15:08:06.313639 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Async connection call returned [2022-03-02 15:08:06.313775 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol [2022-03-02 15:08:06.314009 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: server [], protocol version 500) [2022-03-02 15:08:06.314060 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: server [proxy], protocol version 500) [2022-03-02 15:08:06.315086 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=earth.crc.nd.edu&xrd.rn=v4.12.6, dual-stack: false, private IPv4: true, private IPv6: false [2022-03-02 15:08:06.315323 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Logged in, session: 9f180000070d00003a0000009f180000 [2022-03-02 15:08:06.315360 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9e180000070d00001d0000009e180000 [2022-03-02 15:08:06.315509 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Stream 0 connected. [2022-03-02 15:08:06.315555 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc0002cc0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:08:06.315602 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc8002290 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:08:06.315646 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xcc0030d0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:08:06.315820 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc0002cc0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 15:08:06.315832 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc00033e0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:08:06.315904 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 15:08:06.315904 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 15:08:06.315927 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc0002cc0. [2022-03-02 15:08:06.315957 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc8002290 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 15:08:06.315989 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 15:08:06.315989 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 15:08:06.316004 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc8002290. [2022-03-02 15:08:06.316025 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xcc0030d0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 15:08:06.316067 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 15:08:06.316067 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 15:08:06.316084 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xcc0030d0. [2022-03-02 15:08:06.316080 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc00033e0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 15:08:06.316195 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Open has returned with status [SUCCESS] [2022-03-02 15:08:06.316219 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] successfully opened at primeradiant01.crc.nd.edu:1094, handle: 0x3, session id: 12 [2022-03-02 15:08:06.316251 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xc80027a0 (message: kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608) ). [2022-03-02 15:08:06.316295 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 15:08:06.316295 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 15:08:06.316300 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc80027a0 (message: kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608) ) from out-queu to in-queue. [2022-03-02 15:08:06.316313 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc00033e0. [2022-03-02 15:09:06.376929 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 15:09:06.377006 -0500][Debug ][Poller ] <[::ffff:10.32.77.18]:14360><--><[::ffff:149.165.243.5]:1094> Removing socket from the poller [2022-03-02 15:09:06.377101 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Recovering error for stream #0: [ERROR] Socket error. [2022-03-02 15:09:06.377169 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Reporting disconnection to queued message handlers. [2022-03-02 15:09:06.377198 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608): [ERROR] Socket error. [2022-03-02 15:09:06.377251 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608) [2022-03-02 15:09:06.377273 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xc80027a0 (message: kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608) ). [2022-03-02 15:09:06.377475 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc80027a0 (message: kXR_read (handle: 0x03000000, offset: 33554432, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 15:09:06.377576 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 15:09:06.377676 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xc0002cc0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:09:06.377744 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc80027a0. [2022-03-02 15:09:06.377762 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xc0002cc0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:09:06.378420 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xc0002cc0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:09:06.379092 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094] Found 1 address(es): [::ffff:149.165.243.5]:1094 [2022-03-02 15:09:06.379135 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:149.165.243.5]:1094 [2022-03-02 15:09:06.379185 -0500][Debug ][Poller ] Adding socket 0xc8001b30 to the poller [2022-03-02 15:09:06.379433 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Async connection call returned [2022-03-02 15:09:06.379503 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol [2022-03-02 15:09:06.379706 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: server [], protocol version 500) [2022-03-02 15:09:06.379743 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: server [proxy], protocol version 500) [2022-03-02 15:09:06.381698 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=earth.crc.nd.edu&xrd.rn=v4.12.6, dual-stack: false, private IPv4: true, private IPv6: false [2022-03-02 15:09:06.381965 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Logged in, session: a0180000070d00001d000000a0180000 [2022-03-02 15:09:06.382008 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9f180000070d00003a0000009f180000 [2022-03-02 15:09:06.382151 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:09:06.382182 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9f180000070d00003a0000009f180000 [2022-03-02 15:09:06.382216 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad00015a0" to be run at: [2022-03-02 15:09:10 -0500] [2022-03-02 15:09:10.323570 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad00015a0" [2022-03-02 15:09:10.323762 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:09:10.323804 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9f180000070d00003a0000009f180000 [2022-03-02 15:09:10.323837 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001560" to be run at: [2022-03-02 15:09:16 -0500] [2022-03-02 15:09:16.324787 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001560" [2022-03-02 15:09:16.325033 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:09:16.325093 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9f180000070d00003a0000009f180000 [2022-03-02 15:09:16.325126 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad00015a0" to be run at: [2022-03-02 15:09:24 -0500] [2022-03-02 15:09:24.326236 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad00015a0" [2022-03-02 15:09:24.326426 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:09:24.326496 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9f180000070d00003a0000009f180000 [2022-03-02 15:09:24.326527 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001560" to be run at: [2022-03-02 15:09:34 -0500] [2022-03-02 15:09:34.328120 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001560" [2022-03-02 15:09:34.328383 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:09:34.328426 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9f180000070d00003a0000009f180000 [2022-03-02 15:09:34.328499 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad00015a0" to be run at: [2022-03-02 15:09:46 -0500] [2022-03-02 15:09:46.330341 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad00015a0" [2022-03-02 15:09:46.330582 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:09:46.330626 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9f180000070d00003a0000009f180000 [2022-03-02 15:09:46.330659 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001560" to be run at: [2022-03-02 15:10:00 -0500] [2022-03-02 15:10:00.332913 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001560" [2022-03-02 15:10:00.333158 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:10:00.333203 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9f180000070d00003a0000009f180000 [2022-03-02 15:10:00.333236 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad00015a0" to be run at: [2022-03-02 15:10:16 -0500] [2022-03-02 15:10:16.335653 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad00015a0" [2022-03-02 15:10:16.335819 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:10:16.335862 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9f180000070d00003a0000009f180000 [2022-03-02 15:10:16.335894 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001560" to be run at: [2022-03-02 15:10:34 -0500] [2022-03-02 15:10:34.338722 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001560" [2022-03-02 15:10:34.339015 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:10:34.339117 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9f180000070d00003a0000009f180000 [2022-03-02 15:10:34.339181 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad00015a0" to be run at: [2022-03-02 15:10:54 -0500] [2022-03-02 15:10:54.342232 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad00015a0" [2022-03-02 15:10:54.342743 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:10:54.342787 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: 9f180000070d00003a0000009f180000 [2022-03-02 15:10:54.342812 -0500][Error ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Wont retry kXR_endsess request because wouldreach connection timeout. [2022-03-02 15:10:54.342837 -0500][Error ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Socket error while handshaking: [ERROR] Socket timeout [2022-03-02 15:10:54.342858 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 15:10:54.342881 -0500][Debug ][Poller ] <[::ffff:10.32.77.18]:14368><--><[::ffff:149.165.243.5]:1094> Removing socket from the poller [2022-03-02 15:10:54.342975 -0500][Error ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] elapsed = 108, pConnectionWindow = 120 seconds. [2022-03-02 15:10:54.343006 -0500][Info ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Attempting reconnection in 12 seconds. [2022-03-02 15:10:54.343054 -0500][Debug ][TaskMgr ] Registering task: "StreamConnectorTask for primeradiant01.crc.nd.edu:1094 #0" to be run at: [2022-03-02 15:11:06 -0500] [2022-03-02 15:11:06.344842 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094] Found 1 address(es): [::ffff:149.165.243.5]:1094 [2022-03-02 15:11:06.344915 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:149.165.243.5]:1094 [2022-03-02 15:11:06.344964 -0500][Debug ][Poller ] Adding socket 0xc8001b30 to the poller [2022-03-02 15:11:06.345009 -0500][Debug ][TaskMgr ] Done with task: "StreamConnectorTask for primeradiant01.crc.nd.edu:1094 #0" [2022-03-02 15:11:06.345211 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Async connection call returned [2022-03-02 15:11:06.345316 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol [2022-03-02 15:11:06.345586 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: server [], protocol version 500) [2022-03-02 15:11:06.345623 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: server [proxy], protocol version 500) [2022-03-02 15:11:06.347250 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=earth.crc.nd.edu&xrd.rn=v4.12.6, dual-stack: false, private IPv4: true, private IPv6: false [2022-03-02 15:11:06.347502 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Logged in, session: a1180000070d000028000000a1180000 [2022-03-02 15:11:06.347542 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a0180000070d00001d000000a0180000 [2022-03-02 15:11:06.347675 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Stream 0 connected. [2022-03-02 15:11:06.347721 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc0002cc0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:11:06.347950 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc0002cc0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 15:11:06.348030 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Open has returned with status [SUCCESS] [2022-03-02 15:11:06.348074 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] successfully opened at primeradiant01.crc.nd.edu:1094, handle: 0x0, session id: 13 [2022-03-02 15:11:06.348120 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xc8002ce0 (message: kXR_read (handle: 0x00000000, offset: 33554432, size: 8388608) ). [2022-03-02 15:11:06.348179 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 15:11:06.348179 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 15:11:06.348181 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc8002ce0 (message: kXR_read (handle: 0x00000000, offset: 33554432, size: 8388608) ) from out-queu to in-queue. [2022-03-02 15:11:06.348246 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc0002cc0. [2022-03-02 15:12:06.394605 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 15:12:06.394683 -0500][Debug ][Poller ] <[::ffff:10.32.77.18]:14386><--><[::ffff:149.165.243.5]:1094> Removing socket from the poller [2022-03-02 15:12:06.394759 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Recovering error for stream #0: [ERROR] Socket error. [2022-03-02 15:12:06.394816 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Reporting disconnection to queued message handlers. [2022-03-02 15:12:06.394842 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x00000000, offset: 33554432, size: 8388608): [ERROR] Socket error. [2022-03-02 15:12:06.394863 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x00000000, offset: 33554432, size: 8388608) [2022-03-02 15:12:06.394885 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xc8002ce0 (message: kXR_read (handle: 0x00000000, offset: 33554432, size: 8388608) ). [2022-03-02 15:12:06.395073 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc8002ce0 (message: kXR_read (handle: 0x00000000, offset: 33554432, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 15:12:06.395145 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 15:12:06.395242 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xc0002cc0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:12:06.395304 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc8002ce0. [2022-03-02 15:12:06.395326 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xc0002cc0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:12:06.396001 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xc0002cc0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:12:06.397000 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094] Found 1 address(es): [::ffff:149.165.243.5]:1094 [2022-03-02 15:12:06.397057 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:149.165.243.5]:1094 [2022-03-02 15:12:06.397096 -0500][Debug ][Poller ] Adding socket 0xc8001b30 to the poller [2022-03-02 15:12:06.397233 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Async connection call returned [2022-03-02 15:12:06.397284 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol [2022-03-02 15:12:06.397581 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: server [], protocol version 500) [2022-03-02 15:12:06.397637 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: server [proxy], protocol version 500) [2022-03-02 15:12:06.399061 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=earth.crc.nd.edu&xrd.rn=v4.12.6, dual-stack: false, private IPv4: true, private IPv6: false [2022-03-02 15:12:06.399240 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Logged in, session: a2180000070d00001d000000a2180000 [2022-03-02 15:12:06.399279 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a1180000070d000028000000a1180000 [2022-03-02 15:12:06.399415 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:12:06.399447 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a1180000070d000028000000a1180000 [2022-03-02 15:12:06.399505 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001710" to be run at: [2022-03-02 15:12:10 -0500] [2022-03-02 15:12:10.355535 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001710" [2022-03-02 15:12:10.356016 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:12:10.356104 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a1180000070d000028000000a1180000 [2022-03-02 15:12:10.356141 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001750" to be run at: [2022-03-02 15:12:16 -0500] [2022-03-02 15:12:16.356666 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001750" [2022-03-02 15:12:16.357049 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:12:16.357134 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a1180000070d000028000000a1180000 [2022-03-02 15:12:16.357168 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad00016d0" to be run at: [2022-03-02 15:12:24 -0500] [2022-03-02 15:12:24.358073 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad00016d0" [2022-03-02 15:12:24.358364 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:12:24.358426 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a1180000070d000028000000a1180000 [2022-03-02 15:12:24.358448 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001710" to be run at: [2022-03-02 15:12:34 -0500] [2022-03-02 15:12:34.359670 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001710" [2022-03-02 15:12:34.360024 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:12:34.360110 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a1180000070d000028000000a1180000 [2022-03-02 15:12:34.360133 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001750" to be run at: [2022-03-02 15:12:46 -0500] [2022-03-02 15:12:46.361784 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001750" [2022-03-02 15:12:46.362256 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:12:46.362341 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a1180000070d000028000000a1180000 [2022-03-02 15:12:46.362409 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad00016d0" to be run at: [2022-03-02 15:13:00 -0500] [2022-03-02 15:13:00.364167 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad00016d0" [2022-03-02 15:13:00.364605 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:13:00.364684 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a1180000070d000028000000a1180000 [2022-03-02 15:13:00.364722 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001710" to be run at: [2022-03-02 15:13:16 -0500] [2022-03-02 15:13:16.366871 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001710" [2022-03-02 15:13:16.367266 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:13:16.367383 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a1180000070d000028000000a1180000 [2022-03-02 15:13:16.367446 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001750" to be run at: [2022-03-02 15:13:34 -0500] [2022-03-02 15:13:34.369859 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001750" [2022-03-02 15:13:34.370127 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:13:34.370242 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a1180000070d000028000000a1180000 [2022-03-02 15:13:34.370283 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad00016d0" to be run at: [2022-03-02 15:13:54 -0500] [2022-03-02 15:13:54.373319 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad00016d0" [2022-03-02 15:13:54.373769 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:13:54.373878 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a1180000070d000028000000a1180000 [2022-03-02 15:13:54.373936 -0500][Error ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Wont retry kXR_endsess request because wouldreach connection timeout. [2022-03-02 15:13:54.373998 -0500][Error ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Socket error while handshaking: [ERROR] Socket timeout [2022-03-02 15:13:54.374026 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 15:13:54.374119 -0500][Debug ][Poller ] <[::ffff:10.32.77.18]:14396><--><[::ffff:149.165.243.5]:1094> Removing socket from the poller [2022-03-02 15:13:54.374213 -0500][Error ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] elapsed = 108, pConnectionWindow = 120 seconds. [2022-03-02 15:13:54.374243 -0500][Info ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Attempting reconnection in 12 seconds. [2022-03-02 15:13:54.374272 -0500][Debug ][TaskMgr ] Registering task: "StreamConnectorTask for primeradiant01.crc.nd.edu:1094 #0" to be run at: [2022-03-02 15:14:06 -0500] [2022-03-02 15:14:06.376184 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094] Found 1 address(es): [::ffff:149.165.243.5]:1094 [2022-03-02 15:14:06.376256 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:149.165.243.5]:1094 [2022-03-02 15:14:06.376308 -0500][Debug ][Poller ] Adding socket 0xc8001b30 to the poller [2022-03-02 15:14:06.376358 -0500][Debug ][TaskMgr ] Done with task: "StreamConnectorTask for primeradiant01.crc.nd.edu:1094 #0" [2022-03-02 15:14:06.376458 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Async connection call returned [2022-03-02 15:14:06.376505 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol [2022-03-02 15:14:06.376669 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: server [], protocol version 500) [2022-03-02 15:14:06.376721 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: server [proxy], protocol version 500) [2022-03-02 15:14:06.378746 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=earth.crc.nd.edu&xrd.rn=v4.12.6, dual-stack: false, private IPv4: true, private IPv6: false [2022-03-02 15:14:06.378966 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Logged in, session: a3180000070d000029000000a3180000 [2022-03-02 15:14:06.379006 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a2180000070d00001d000000a2180000 [2022-03-02 15:14:06.379213 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Stream 0 connected. [2022-03-02 15:14:06.379261 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc0002cc0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:14:06.379529 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc0002cc0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 15:14:06.379606 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Open has returned with status [SUCCESS] [2022-03-02 15:14:06.379622 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] successfully opened at primeradiant01.crc.nd.edu:1094, handle: 0x0, session id: 14 [2022-03-02 15:14:06.379653 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xc8002700 (message: kXR_read (handle: 0x00000000, offset: 33554432, size: 8388608) ). [2022-03-02 15:14:06.379701 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 15:14:06.379701 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 15:14:06.379701 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc8002700 (message: kXR_read (handle: 0x00000000, offset: 33554432, size: 8388608) ) from out-queu to in-queue. [2022-03-02 15:14:06.379746 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc0002cc0. [2022-03-02 15:15:06.440049 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 15:15:06.440122 -0500][Debug ][Poller ] <[::ffff:10.32.77.18]:14412><--><[::ffff:149.165.243.5]:1094> Removing socket from the poller [2022-03-02 15:15:06.440197 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Recovering error for stream #0: [ERROR] Socket error. [2022-03-02 15:15:06.440221 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Reporting disconnection to queued message handlers. [2022-03-02 15:15:06.440247 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x00000000, offset: 33554432, size: 8388608): [ERROR] Socket error. [2022-03-02 15:15:06.440268 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x00000000, offset: 33554432, size: 8388608) [2022-03-02 15:15:06.440290 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xc8002700 (message: kXR_read (handle: 0x00000000, offset: 33554432, size: 8388608) ). [2022-03-02 15:15:06.440481 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc8002700 (message: kXR_read (handle: 0x00000000, offset: 33554432, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 15:15:06.440533 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 15:15:06.440587 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xc0002cc0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:15:06.440636 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc8002700. [2022-03-02 15:15:06.440659 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xc0002cc0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:15:06.441345 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xc0002cc0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:15:06.441941 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094] Found 1 address(es): [::ffff:149.165.243.5]:1094 [2022-03-02 15:15:06.441974 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:149.165.243.5]:1094 [2022-03-02 15:15:06.442005 -0500][Debug ][Poller ] Adding socket 0xc8001b30 to the poller [2022-03-02 15:15:06.442231 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Async connection call returned [2022-03-02 15:15:06.442312 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol [2022-03-02 15:15:06.442504 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: server [], protocol version 500) [2022-03-02 15:15:06.442542 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: server [proxy], protocol version 500) [2022-03-02 15:15:06.444016 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=earth.crc.nd.edu&xrd.rn=v4.12.6, dual-stack: false, private IPv4: true, private IPv6: false [2022-03-02 15:15:06.444260 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Logged in, session: a4180000070d00001d000000a4180000 [2022-03-02 15:15:06.444303 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a3180000070d000029000000a3180000 [2022-03-02 15:15:06.444457 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:15:06.444489 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a3180000070d000029000000a3180000 [2022-03-02 15:15:06.444522 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001810" to be run at: [2022-03-02 15:15:10 -0500] [2022-03-02 15:15:10.386831 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001810" [2022-03-02 15:15:10.387166 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:15:10.387247 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a3180000070d000029000000a3180000 [2022-03-02 15:15:10.387285 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001880" to be run at: [2022-03-02 15:15:16 -0500] [2022-03-02 15:15:16.387599 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001880" [2022-03-02 15:15:16.387942 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:15:16.388025 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a3180000070d000029000000a3180000 [2022-03-02 15:15:16.388090 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad00017d0" to be run at: [2022-03-02 15:15:24 -0500] [2022-03-02 15:15:24.388903 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad00017d0" [2022-03-02 15:15:24.389238 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:15:24.389323 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a3180000070d000029000000a3180000 [2022-03-02 15:15:24.389361 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001810" to be run at: [2022-03-02 15:15:34 -0500] [2022-03-02 15:15:34.390375 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001810" [2022-03-02 15:15:34.390789 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:15:34.390857 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a3180000070d000029000000a3180000 [2022-03-02 15:15:34.390879 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001850" to be run at: [2022-03-02 15:15:46 -0500] [2022-03-02 15:15:46.392220 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001850" [2022-03-02 15:15:46.392533 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:15:46.392598 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a3180000070d000029000000a3180000 [2022-03-02 15:15:46.392624 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad00017d0" to be run at: [2022-03-02 15:16:00 -0500] [2022-03-02 15:16:00.394176 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad00017d0" [2022-03-02 15:16:00.394577 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:16:00.394661 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a3180000070d000029000000a3180000 [2022-03-02 15:16:00.394733 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001810" to be run at: [2022-03-02 15:16:16 -0500] [2022-03-02 15:16:16.396837 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001810" [2022-03-02 15:16:16.397316 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:16:16.397430 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a3180000070d000029000000a3180000 [2022-03-02 15:16:16.397492 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001850" to be run at: [2022-03-02 15:16:34 -0500] [2022-03-02 15:16:34.399791 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001850" [2022-03-02 15:16:34.400282 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:16:34.400353 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a3180000070d000029000000a3180000 [2022-03-02 15:16:34.400416 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad00017d0" to be run at: [2022-03-02 15:16:54 -0500] [2022-03-02 15:16:54.403176 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad00017d0" [2022-03-02 15:16:54.403525 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:16:54.403599 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a3180000070d000029000000a3180000 [2022-03-02 15:16:54.403613 -0500][Error ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Wont retry kXR_endsess request because wouldreach connection timeout. [2022-03-02 15:16:54.403630 -0500][Error ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Socket error while handshaking: [ERROR] Socket timeout [2022-03-02 15:16:54.403642 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 15:16:54.403655 -0500][Debug ][Poller ] <[::ffff:10.32.77.18]:14422><--><[::ffff:149.165.243.5]:1094> Removing socket from the poller [2022-03-02 15:16:54.403694 -0500][Error ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] elapsed = 108, pConnectionWindow = 120 seconds. [2022-03-02 15:16:54.403707 -0500][Info ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Attempting reconnection in 12 seconds. [2022-03-02 15:16:54.403721 -0500][Debug ][TaskMgr ] Registering task: "StreamConnectorTask for primeradiant01.crc.nd.edu:1094 #0" to be run at: [2022-03-02 15:17:06 -0500] [2022-03-02 15:17:06.405880 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094] Found 1 address(es): [::ffff:149.165.243.5]:1094 [2022-03-02 15:17:06.405946 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:149.165.243.5]:1094 [2022-03-02 15:17:06.405988 -0500][Debug ][Poller ] Adding socket 0xc8001b30 to the poller [2022-03-02 15:17:06.406022 -0500][Debug ][TaskMgr ] Done with task: "StreamConnectorTask for primeradiant01.crc.nd.edu:1094 #0" [2022-03-02 15:17:06.406226 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Async connection call returned [2022-03-02 15:17:06.406299 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol [2022-03-02 15:17:06.406499 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: server [], protocol version 500) [2022-03-02 15:17:06.406523 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: server [proxy], protocol version 500) [2022-03-02 15:17:06.407599 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=earth.crc.nd.edu&xrd.rn=v4.12.6, dual-stack: false, private IPv4: true, private IPv6: false [2022-03-02 15:17:06.407772 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Logged in, session: a5180000070d00002b000000a5180000 [2022-03-02 15:17:06.407797 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a4180000070d00001d000000a4180000 [2022-03-02 15:17:06.407907 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Stream 0 connected. [2022-03-02 15:17:06.407943 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc0002cc0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:17:06.408215 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc0002cc0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 15:17:06.408309 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Open has returned with status [SUCCESS] [2022-03-02 15:17:06.408333 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] successfully opened at primeradiant01.crc.nd.edu:1094, handle: 0x0, session id: 15 [2022-03-02 15:17:06.408367 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xc8003420 (message: kXR_read (handle: 0x00000000, offset: 33554432, size: 8388608) ). [2022-03-02 15:17:06.408418 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc8003420 (message: kXR_read (handle: 0x00000000, offset: 33554432, size: 8388608) ) from out-queu to in-queue. [2022-03-02 15:17:06.408419 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 15:17:06.408419 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 15:17:06.408566 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc0002cc0. [2022-03-02 15:18:06.452218 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 15:18:06.452298 -0500][Debug ][Poller ] <[::ffff:10.32.77.18]:14620><--><[::ffff:149.165.243.5]:1094> Removing socket from the poller [2022-03-02 15:18:06.452404 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Recovering error for stream #0: [ERROR] Socket error. [2022-03-02 15:18:06.452419 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Reporting disconnection to queued message handlers. [2022-03-02 15:18:06.452434 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x00000000, offset: 33554432, size: 8388608): [ERROR] Socket error. [2022-03-02 15:18:06.452446 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x00000000, offset: 33554432, size: 8388608) [2022-03-02 15:18:06.452460 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xc8003420 (message: kXR_read (handle: 0x00000000, offset: 33554432, size: 8388608) ). [2022-03-02 15:18:06.452658 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc8003420 (message: kXR_read (handle: 0x00000000, offset: 33554432, size: 8388608) ) with status: [ERROR] Socket error. [2022-03-02 15:18:06.452751 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Running the recovery procedure [2022-03-02 15:18:06.452823 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] MsgHandler created: 0xc0002cc0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:18:06.452916 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc8003420. [2022-03-02 15:18:06.452939 -0500][Debug ][ExDbgMsg ] [ndcms.crc.nd.edu:1094] Moving MsgHandler: 0xc0002cc0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:18:06.453637 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Retry at server MsgHandler: 0xc0002cc0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ). [2022-03-02 15:18:06.454350 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094] Found 1 address(es): [::ffff:149.165.243.5]:1094 [2022-03-02 15:18:06.454453 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:149.165.243.5]:1094 [2022-03-02 15:18:06.454548 -0500][Debug ][Poller ] Adding socket 0xc8001b30 to the poller [2022-03-02 15:18:06.454660 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Async connection call returned [2022-03-02 15:18:06.454703 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol [2022-03-02 15:18:06.454833 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: server [], protocol version 500) [2022-03-02 15:18:06.454879 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: server [proxy], protocol version 500) [2022-03-02 15:18:06.455929 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=earth.crc.nd.edu&xrd.rn=v4.12.6, dual-stack: false, private IPv4: true, private IPv6: false [2022-03-02 15:18:06.456116 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Logged in, session: a6180000070d00001d000000a6180000 [2022-03-02 15:18:06.456157 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a5180000070d00002b000000a5180000 [2022-03-02 15:18:06.456277 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:18:06.456318 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a5180000070d00002b000000a5180000 [2022-03-02 15:18:06.456356 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad00018e0" to be run at: [2022-03-02 15:18:10 -0500] [2022-03-02 15:18:10.416965 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad00018e0" [2022-03-02 15:18:10.417461 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:18:10.417598 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a5180000070d00002b000000a5180000 [2022-03-02 15:18:10.417640 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001950" to be run at: [2022-03-02 15:18:16 -0500] [2022-03-02 15:18:16.418224 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001950" [2022-03-02 15:18:16.418752 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:18:16.418778 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a5180000070d00002b000000a5180000 [2022-03-02 15:18:16.418800 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad00018a0" to be run at: [2022-03-02 15:18:24 -0500] [2022-03-02 15:18:24.419709 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad00018a0" [2022-03-02 15:18:24.420080 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:18:24.420105 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a5180000070d00002b000000a5180000 [2022-03-02 15:18:24.420129 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001990" to be run at: [2022-03-02 15:18:34 -0500] [2022-03-02 15:18:34.421486 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001990" [2022-03-02 15:18:34.421791 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:18:34.421873 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a5180000070d00002b000000a5180000 [2022-03-02 15:18:34.421905 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad00018e0" to be run at: [2022-03-02 15:18:46 -0500] [2022-03-02 15:18:46.423366 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad00018e0" [2022-03-02 15:18:46.423814 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:18:46.423879 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a5180000070d00002b000000a5180000 [2022-03-02 15:18:46.423903 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad00018a0" to be run at: [2022-03-02 15:19:00 -0500] [2022-03-02 15:19:00.425226 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad00018a0" [2022-03-02 15:19:00.425522 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:19:00.425545 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a5180000070d00002b000000a5180000 [2022-03-02 15:19:00.425565 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad0001920" to be run at: [2022-03-02 15:19:16 -0500] [2022-03-02 15:19:16.427734 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad0001920" [2022-03-02 15:19:16.428021 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:19:16.428126 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a5180000070d00002b000000a5180000 [2022-03-02 15:19:16.428158 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad00018e0" to be run at: [2022-03-02 15:19:34 -0500] [2022-03-02 15:19:34.430754 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad00018e0" [2022-03-02 15:19:34.431228 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:19:34.431360 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a5180000070d00002b000000a5180000 [2022-03-02 15:19:34.431425 -0500][Debug ][TaskMgr ] Registering task: "WaitTask for: 0x0x7f4ad00018a0" to be run at: [2022-03-02 15:19:54 -0500] [2022-03-02 15:19:54.434127 -0500][Debug ][TaskMgr ] Done with task: "WaitTask for: 0x0x7f4ad00018a0" [2022-03-02 15:19:54.434538 -0500][Info ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got wait response to kXR_endsess: session still active [2022-03-02 15:19:54.434628 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a5180000070d00002b000000a5180000 [2022-03-02 15:19:54.434693 -0500][Error ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Wont retry kXR_endsess request because wouldreach connection timeout. [2022-03-02 15:19:54.434719 -0500][Error ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Socket error while handshaking: [ERROR] Socket timeout [2022-03-02 15:19:54.434740 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 15:19:54.434765 -0500][Debug ][Poller ] <[::ffff:10.32.77.18]:15198><--><[::ffff:149.165.243.5]:1094> Removing socket from the poller [2022-03-02 15:19:54.434852 -0500][Error ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] elapsed = 108, pConnectionWindow = 120 seconds. [2022-03-02 15:19:54.434888 -0500][Info ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Attempting reconnection in 12 seconds. [2022-03-02 15:19:54.434914 -0500][Debug ][TaskMgr ] Registering task: "StreamConnectorTask for primeradiant01.crc.nd.edu:1094 #0" to be run at: [2022-03-02 15:20:06 -0500] [2022-03-02 15:20:06.437245 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094] Found 1 address(es): [::ffff:149.165.243.5]:1094 [2022-03-02 15:20:06.437351 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Attempting connection to [::ffff:149.165.243.5]:1094 [2022-03-02 15:20:06.437416 -0500][Debug ][Poller ] Adding socket 0xc8001b30 to the poller [2022-03-02 15:20:06.437466 -0500][Debug ][TaskMgr ] Done with task: "StreamConnectorTask for primeradiant01.crc.nd.edu:1094 #0" [2022-03-02 15:20:06.437666 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Async connection call returned [2022-03-02 15:20:06.437761 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out the initial hand shake + kXR_protocol [2022-03-02 15:20:06.438085 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Got the server hand shake response (type: server [], protocol version 500) [2022-03-02 15:20:06.438174 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] kXR_protocol successful (type: server [proxy], protocol version 500) [2022-03-02 15:20:06.439904 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_login request, username: kmohrman, cgi: ?xrd.cc=us&xrd.tz=-5&xrd.appname=xrdcp&xrd.info=&xrd.hostname=earth.crc.nd.edu&xrd.rn=v4.12.6, dual-stack: false, private IPv4: true, private IPv6: false [2022-03-02 15:20:06.440092 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Logged in, session: a7180000070d000030000000a7180000 [2022-03-02 15:20:06.440131 -0500][Debug ][XRootDTransport ] [primeradiant01.crc.nd.edu:1094 #0.0] Sending out kXR_endsess for session: a6180000070d00001d000000a6180000 [2022-03-02 15:20:06.440295 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Stream 0 connected. [2022-03-02 15:20:06.440351 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc0002cc0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) from out-queu to in-queue. [2022-03-02 15:20:06.440612 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc0002cc0 (message: kXR_open (file: /store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root, mode: 00, flags: kXR_open_read ) ) with status: [SUCCESS] . [2022-03-02 15:20:06.440693 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Open has returned with status [SUCCESS] [2022-03-02 15:20:06.440716 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] successfully opened at primeradiant01.crc.nd.edu:1094, handle: 0x0, session id: 16 [2022-03-02 15:20:06.440751 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0xc8003540 (message: kXR_read (handle: 0x00000000, offset: 33554432, size: 8388608) ). [2022-03-02 15:20:06.440805 -0500][Debug ][XRootD ] Redirect trace-back: [2022-03-02 15:20:06.440805 -0500][Debug ][XRootD ] 0. Redirected from: root://ndcms.crc.nd.edu:1094/ to: root://primeradiant01.crc.nd.edu:1094/ [2022-03-02 15:20:06.440806 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0xc8003540 (message: kXR_read (handle: 0x00000000, offset: 33554432, size: 8388608) ) from out-queu to in-queue. [2022-03-02 15:20:06.440858 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc0002cc0. [2022-03-02 15:20:14.438847 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_read (handle: 0x00000000, offset: 33554432, size: 8388608): [ERROR] Operation expired. [2022-03-02 15:20:14.438906 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_read (handle: 0x00000000, offset: 33554432, size: 8388608) [2022-03-02 15:20:14.438931 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0xc8003540 (message: kXR_read (handle: 0x00000000, offset: 33554432, size: 8388608) ). [2022-03-02 15:20:14.439132 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0xc8003540 (message: kXR_read (handle: 0x00000000, offset: 33554432, size: 8388608) ) with status: [ERROR] Operation expired. [2022-03-02 15:20:14.439253 -0500][Error ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Fatal file state error. Message kXR_read (handle: 0x00000000, offset: 33554432, size: 8388608) returned with [ERROR] Operation expired [2022-03-02 15:20:14.439324 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0xc8003540. [2022-03-02 15:20:14.441622 -0500][Debug ][File ] [0x1114910@file://localhost/dev/null?oss.asize=157882637&xrdcl.requuid=ae1adf29-47c1-43e2-ad7c-427e260856fe] Sending a close command for handle 0xb to localhost [2022-03-02 15:20:14.441819 -0500][Debug ][File ] [0x1114910@file://localhost/dev/null?oss.asize=157882637&xrdcl.requuid=ae1adf29-47c1-43e2-ad7c-427e260856fe] Close returned from localhost with: [SUCCESS] [2022-03-02 15:20:14.441898 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Sending a close command for handle 0x0 to primeradiant01.crc.nd.edu:1094 [2022-03-02 15:20:14.441943 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] MsgHandler created: 0x1114910 (message: kXR_close (handle: 0x00000000) ). [2022-03-02 15:20:14.442074 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Moving MsgHandler: 0x1114910 (message: kXR_close (handle: 0x00000000) ) from out-queu to in-queue. [2022-03-02 15:21:21.509192 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 15:21:21.509276 -0500][Debug ][Poller ] <[::ffff:10.32.77.18]:15220><--><[::ffff:149.165.243.5]:1094> Removing socket from the poller [2022-03-02 15:21:21.509348 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Recovering error for stream #0: [ERROR] Socket error. [2022-03-02 15:21:21.509372 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Reporting disconnection to queued message handlers. [2022-03-02 15:21:21.509398 -0500][Debug ][XRootD ] [primeradiant01.crc.nd.edu:1094] Handling error while processing kXR_close (handle: 0x00000000): [ERROR] Socket error. [2022-03-02 15:21:21.509419 -0500][Error ][XRootD ] [primeradiant01.crc.nd.edu:1094] Unable to get the response to request kXR_close (handle: 0x00000000) [2022-03-02 15:21:21.509479 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0x1114910 (message: kXR_close (handle: 0x00000000) ). [2022-03-02 15:21:21.509665 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Calling MsgHandler: 0x1114910 (message: kXR_close (handle: 0x00000000) ) with status: [ERROR] Socket error. [2022-03-02 15:21:21.509782 -0500][Debug ][File ] [0x110fe10@root://ndcms.crc.nd.edu:1094//store/mc/RunIISummer20UL16NanoAODv9/WWTo2L2Nu_TuneCP5_13TeV-powheg-pythia8/NANOAODSIM/106X_mcRun2_asymptotic_v17-v1/130000/80849F69-0DEE-3048-A2CF-32EC495D6ECF.root?xrdcl.requuid=6ff2d270-1788-475e-921e-a604a4d03742] Close returned from primeradiant01.crc.nd.edu:1094 with: [ERROR] Socket error [2022-03-02 15:21:21.509859 -0500][Debug ][ExDbgMsg ] [primeradiant01.crc.nd.edu:1094] Destroying MsgHandler: 0x1114910. [8MB/150.6MB][ 5%][==> ][3.13kB/s] Run: [ERROR] Operation expired: (source) [2022-03-02 15:21:21.510339 -0500][Debug ][JobMgr ] Stopping the job manager... [2022-03-02 15:21:21.510781 -0500][Debug ][JobMgr ] Job manager stopped [2022-03-02 15:21:21.510805 -0500][Debug ][TaskMgr ] Stopping the task manager... [2022-03-02 15:21:21.511006 -0500][Debug ][TaskMgr ] Task manager stopped [2022-03-02 15:21:21.511027 -0500][Debug ][Poller ] Stopping the poller... [2022-03-02 15:21:21.511278 -0500][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: ndcms.crc.nd.edu:1094" [2022-03-02 15:21:21.511307 -0500][Debug ][AsyncSock ] [ndcms.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 15:21:21.511323 -0500][Debug ][Poller ] <[::ffff:10.32.77.18]:19200><--><[::ffff:129.74.85.4]:1094> Removing socket from the poller [2022-03-02 15:21:21.511364 -0500][Debug ][PostMaster ] [ndcms.crc.nd.edu:1094 #0] Destroying stream [2022-03-02 15:21:21.511384 -0500][Debug ][AsyncSock ] [ndcms.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 15:21:21.511415 -0500][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: primeradiant01.crc.nd.edu:1094" [2022-03-02 15:21:21.511433 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [2022-03-02 15:21:21.511448 -0500][Debug ][PostMaster ] [primeradiant01.crc.nd.edu:1094 #0] Destroying stream [2022-03-02 15:21:21.511461 -0500][Debug ][AsyncSock ] [primeradiant01.crc.nd.edu:1094 #0.0] Closing the socket [kmohrman@earth: file_lsts]$