[kmohrman@earth: ~]$ xrdcp -d 2 -f root://deepthought.crc.nd.edu//store/user/awightma/skims/mc/new-lepMVA-v2/central_bkgd_p1/FullRun2/v2/UL18_ZZTo4L/output_3238.root /dev/null [2022-04-01 14:58:30.932366 -0400][Debug ][Utility ] CopyProcess: 2 jobs to prepare [2022-04-01 14:58:30.932632 -0400][Debug ][Utility ] Creating a classic copy job, from root://deepthought.crc.nd.edu:1094//store/user/awightma/skims/mc/new-lepMVA-v2/central_bkgd_p1/FullRun2/v2/UL18_ZZTo4L/output_3238.root to file://localhost/dev/null [2022-04-01 14:58:30.932688 -0400][Debug ][Utility ] Monitor library name not set. No monitoring [2022-04-01 14:58:30.932830 -0400][Debug ][Utility ] Env: trying to get a non-existent string entry: pollerpreference [2022-04-01 14:58:30.932888 -0400][Debug ][Poller ] Available pollers: built-in [2022-04-01 14:58:30.932909 -0400][Debug ][Poller ] Attempting to create a poller according to preference: built-in [2022-04-01 14:58:30.932930 -0400][Debug ][Poller ] Creating poller: built-in [2022-04-01 14:58:30.932962 -0400][Debug ][Poller ] Creating and starting the built-in poller... [2022-04-01 14:58:30.933319 -0400][Debug ][Poller ] Using 1 poller threads [2022-04-01 14:58:30.933349 -0400][Debug ][TaskMgr ] Starting the task manager... [2022-04-01 14:58:30.933436 -0400][Debug ][TaskMgr ] Task manager started [2022-04-01 14:58:30.933466 -0400][Debug ][JobMgr ] Starting the job manager... [2022-04-01 14:58:30.933614 -0400][Debug ][JobMgr ] Job manager started, 3 workers [2022-04-01 14:58:30.933647 -0400][Debug ][TaskMgr ] Registering task: "FileTimer task" to be run at: [2022-04-01 14:58:30 -0400] [2022-04-01 14:58:30.933682 -0400][Debug ][Utility ] Opening root://deepthought.crc.nd.edu:1094//store/user/awightma/skims/mc/new-lepMVA-v2/central_bkgd_p1/FullRun2/v2/UL18_ZZTo4L/output_3238.root for reading [2022-04-01 14:58:30.933903 -0400][Debug ][File ] [0x126af70@root://deepthought.crc.nd.edu:1094//store/user/awightma/skims/mc/new-lepMVA-v2/central_bkgd_p1/FullRun2/v2/UL18_ZZTo4L/output_3238.root?xrdcl.requuid=cb15a4f0-5ce1-4cc9-9125-6c2ccd071933] Sending an open command [2022-04-01 14:58:30.934020 -0400][Debug ][ExDbgMsg ] [deepthought.crc.nd.edu:1094] MsgHandler created: 0x12709d0 (message: kXR_open (file: /store/user/awightma/skims/mc/new-lepMVA-v2/central_bkgd_p1/FullRun2/v2/UL18_ZZTo4L/output_3238.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) ). [2022-04-01 14:58:30.934125 -0400][Debug ][PostMaster ] Creating new channel to: root://deepthought.crc.nd.edu:1094 [2022-04-01 14:58:30.934191 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094] Stream parameters: Network Stack: IPAuto, Connection Window: 120, ConnectionRetry: 5, Stream Error Window: 1800 [2022-04-01 14:58:30.934241 -0400][Debug ][TaskMgr ] Registering task: "TickGeneratorTask for: root://deepthought.crc.nd.edu:1094" to be run at: [2022-04-01 14:58:45 -0400] [2022-04-01 14:58:30.935115 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094] Found 1 address(es): [::ffff:129.74.85.55]:1094 [2022-04-01 14:58:30.935173 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094.0] Attempting connection to [::ffff:129.74.85.55]:1094 [2022-04-01 14:58:30.935228 -0400][Debug ][Poller ] Adding socket 0x1270ea0 to the poller [2022-04-01 15:00:31.052243 -0400][Error ][AsyncSock ] [deepthought.crc.nd.edu:1094.0] Socket error while handshaking: [ERROR] Socket timeout [2022-04-01 15:00:31.052336 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094.0] Closing the socket [2022-04-01 15:00:31.052376 -0400][Debug ][Poller ] <--> Removing socket from the poller [2022-04-01 15:00:31.052472 -0400][Error ][PostMaster ] [deepthought.crc.nd.edu:1094] elapsed = 121, pConnectionWindow = 120 seconds. [2022-04-01 15:00:31.053602 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094] Found 1 address(es): [::ffff:129.74.85.55]:1094 [2022-04-01 15:00:31.053720 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094.0] Attempting connection to [::ffff:129.74.85.55]:1094 [2022-04-01 15:00:31.053842 -0400][Debug ][Poller ] Adding socket 0x1270ea0 to the poller [2022-04-01 15:01:02.106117 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094.0] Async connection call returned [2022-04-01 15:01:02.106283 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094.0] Sending out the initial hand shake + kXR_protocol [2022-04-01 15:02:32.186730 -0400][Error ][AsyncSock ] [deepthought.crc.nd.edu:1094.0] Socket error while handshaking: [ERROR] Socket timeout [2022-04-01 15:02:32.186916 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094.0] Closing the socket [2022-04-01 15:02:32.186970 -0400][Debug ][Poller ] <[::ffff:10.32.77.18]:9450><--><[::ffff:129.74.85.55]:1094> Removing socket from the poller [2022-04-01 15:02:32.187052 -0400][Error ][PostMaster ] [deepthought.crc.nd.edu:1094] elapsed = 121, pConnectionWindow = 120 seconds. [2022-04-01 15:02:32.187608 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094] Found 1 address(es): [::ffff:129.74.85.55]:1094 [2022-04-01 15:02:32.187659 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094.0] Attempting connection to [::ffff:129.74.85.55]:1094 [2022-04-01 15:02:32.187721 -0400][Debug ][Poller ] Adding socket 0x1270ea0 to the poller [2022-04-01 15:02:32.187989 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094.0] Async connection call returned [2022-04-01 15:02:32.188081 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094.0] Sending out the initial hand shake + kXR_protocol [2022-04-01 15:04:47.305283 -0400][Error ][AsyncSock ] [deepthought.crc.nd.edu:1094.0] Socket error while handshaking: [ERROR] Socket timeout [2022-04-01 15:04:47.305358 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094.0] Closing the socket [2022-04-01 15:04:47.305388 -0400][Debug ][Poller ] <[::ffff:10.32.77.18]:9520><--><[::ffff:129.74.85.55]:1094> Removing socket from the poller [2022-04-01 15:04:47.305501 -0400][Error ][PostMaster ] [deepthought.crc.nd.edu:1094] elapsed = 135, pConnectionWindow = 120 seconds. [2022-04-01 15:04:47.306527 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094] Found 1 address(es): [::ffff:129.74.85.55]:1094 [2022-04-01 15:04:47.306588 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094.0] Attempting connection to [::ffff:129.74.85.55]:1094 [2022-04-01 15:04:47.306686 -0400][Debug ][Poller ] Adding socket 0x1270ea0 to the poller [2022-04-01 15:06:54.554022 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094.0] Async connection call returned [2022-04-01 15:06:54.554149 -0400][Error ][AsyncSock ] [deepthought.crc.nd.edu:1094.0] Unable to connect: connection timed out [2022-04-01 15:06:54.554178 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094.0] Closing the socket [2022-04-01 15:06:54.554211 -0400][Debug ][Poller ] <--> Removing socket from the poller [2022-04-01 15:06:54.554379 -0400][Error ][PostMaster ] [deepthought.crc.nd.edu:1094] elapsed = 127, pConnectionWindow = 120 seconds. [2022-04-01 15:06:54.555127 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094] Found 1 address(es): [::ffff:129.74.85.55]:1094 [2022-04-01 15:06:54.555176 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094.0] Attempting connection to [::ffff:129.74.85.55]:1094 [2022-04-01 15:06:54.555271 -0400][Debug ][Poller ] Adding socket 0x1270ea0 to the poller [2022-04-01 15:06:54.555387 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094.0] Async connection call returned [2022-04-01 15:06:54.555419 -0400][Debug ][XRootDTransport ] [deepthought.crc.nd.edu:1094.0] Sending out the initial hand shake + kXR_protocol [2022-04-01 15:09:09.667039 -0400][Error ][AsyncSock ] [deepthought.crc.nd.edu:1094.0] Socket error while handshaking: [ERROR] Socket timeout [2022-04-01 15:09:09.667232 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094.0] Closing the socket [2022-04-01 15:09:09.667272 -0400][Debug ][Poller ] <[::ffff:10.32.77.18]:9688><--><[::ffff:129.74.85.55]:1094> Removing socket from the poller [2022-04-01 15:09:09.667403 -0400][Error ][PostMaster ] [deepthought.crc.nd.edu:1094] elapsed = 135, pConnectionWindow = 120 seconds. [2022-04-01 15:09:09.667531 -0400][Error ][PostMaster ] [deepthought.crc.nd.edu:1094] Unable to recover: [ERROR] Socket timeout. [2022-04-01 15:09:09.667628 -0400][Error ][XRootD ] [deepthought.crc.nd.edu:1094] Impossible to send message kXR_open (file: /store/user/awightma/skims/mc/new-lepMVA-v2/central_bkgd_p1/FullRun2/v2/UL18_ZZTo4L/output_3238.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ). Trying to recover. [2022-04-01 15:09:09.667679 -0400][Debug ][XRootD ] [deepthought.crc.nd.edu:1094] Handling error while processing kXR_open (file: /store/user/awightma/skims/mc/new-lepMVA-v2/central_bkgd_p1/FullRun2/v2/UL18_ZZTo4L/output_3238.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ): [FATAL] Socket timeout. [2022-04-01 15:09:09.667750 -0400][Debug ][ExDbgMsg ] [deepthought.crc.nd.edu:1094] Passing to the thread-pool MsgHandler: 0x12709d0 (message: kXR_open (file: /store/user/awightma/skims/mc/new-lepMVA-v2/central_bkgd_p1/FullRun2/v2/UL18_ZZTo4L/output_3238.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) ). [2022-04-01 15:09:09.668120 -0400][Debug ][ExDbgMsg ] [deepthought.crc.nd.edu:1094] Calling MsgHandler: 0x12709d0 (message: kXR_open (file: /store/user/awightma/skims/mc/new-lepMVA-v2/central_bkgd_p1/FullRun2/v2/UL18_ZZTo4L/output_3238.root, mode: 00, flags: kXR_open_read kXR_async kXR_retstat ) ) with status: [FATAL] Socket timeout. [2022-04-01 15:09:09.668293 -0400][Debug ][File ] [0x126af70@root://deepthought.crc.nd.edu:1094//store/user/awightma/skims/mc/new-lepMVA-v2/central_bkgd_p1/FullRun2/v2/UL18_ZZTo4L/output_3238.root?xrdcl.requuid=cb15a4f0-5ce1-4cc9-9125-6c2ccd071933] Open has returned with status [FATAL] Socket timeout [2022-04-01 15:09:09.668348 -0400][Debug ][File ] [0x126af70@root://deepthought.crc.nd.edu:1094//store/user/awightma/skims/mc/new-lepMVA-v2/central_bkgd_p1/FullRun2/v2/UL18_ZZTo4L/output_3238.root?xrdcl.requuid=cb15a4f0-5ce1-4cc9-9125-6c2ccd071933] Error while opening at deepthought.crc.nd.edu:1094: [FATAL] Socket timeout [2022-04-01 15:09:09.668414 -0400][Debug ][ExDbgMsg ] [deepthought.crc.nd.edu:1094] Destroying MsgHandler: 0x12709d0. [0B/0B][100%][==================================================][0B/s] Run: [FATAL] Socket timeout: (source) [2022-04-01 15:09:09.669780 -0400][Debug ][JobMgr ] Stopping the job manager... [2022-04-01 15:09:09.670371 -0400][Debug ][JobMgr ] Job manager stopped [2022-04-01 15:09:09.670407 -0400][Debug ][TaskMgr ] Stopping the task manager... [2022-04-01 15:09:09.670715 -0400][Debug ][TaskMgr ] Task manager stopped [2022-04-01 15:09:09.670756 -0400][Debug ][Poller ] Stopping the poller... [2022-04-01 15:09:09.670950 -0400][Debug ][TaskMgr ] Requesting unregistration of: "TickGeneratorTask for: root://deepthought.crc.nd.edu:1094" [2022-04-01 15:09:09.670998 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094.0] Closing the socket [2022-04-01 15:09:09.671033 -0400][Debug ][PostMaster ] [deepthought.crc.nd.edu:1094] Destroying stream [2022-04-01 15:09:09.671067 -0400][Debug ][AsyncSock ] [deepthought.crc.nd.edu:1094.0] Closing the socket [kmohrman@earth: ~]$