Network Automation Software We are currently migrating this forum
over to our HelpSystems domain. Please
post all new threads in our new
HelpSystems Community Portal.
Post to the HelpSystems Forum
You are not currently logged on. You must be logged on in order to post. Log on
Or Create a new account
AutoMate Discussion
Decrease font size
Increase font size
Topic Title: ftp download resume failure
Topic Summary:
Created On: 12/13/2015 09:33 AM
Status: Read Only
Linear : Threading : Single : Branch
Search Topic Search Topic
Topic Tools Topic Tools
View similar topics View similar topics
View topic in raw text format. Print this topic.
 12/13/2015 09:33 AM
User is offline View Users Profile Print this message

Author Icon
MacTurner
Student (5-19 posts)

Posts: 16
Joined: 06/07/2011

running automate 8.0.1.10 on windows 7 pc. I have ftp download task that has worked fine for months but after remote host started dropping connection every 5 minutes, I cannot get automate to resume download.

I created loop on ftp download error to try to work around issue. automate loops fine but does not appear to resume download.

The file is little over 2 gig in size.

I downloaded the file using filezilla and it worked. The log shows the disconnects every 300 seconds, then resumes the download. I believe the disconnects are causing the automate task to fail.

Here is snippit of filzilla log:

Command: RETR 2016_ADM_YTD.zip
Response: 150 Opening BINARY mode data connection for 2016_ADM_YTD.zip(2335389680 bytes).
Error: Disconnected from server: ECONNABORTED - Connection aborted
Error: Disconnected from server: ECONNABORTED - Connection aborted
Error: File transfer failed after transferring 400,556,032 bytes in 301 seconds

Status: Disconnected from server
Status: Resolving address of www.rma.usda.gov
Status: Connecting to 165.221.68.23:21...
Status: Connection established, waiting for welcome message...
Response: 220-Microsoft FTP Service
Response: ********************************************
Response: Unauthorized access to this United States Government Computer System and software is prohibited by Public Law 99-474, Title 18, United States Code. Public Law 99-74 and Chapter XXI, Section 1030 states that: Whoever knowingly, or intentionally accesses a computer without authorization or exceeds authorized access, and by means such conduct, obtains, alters, damages, destroys, or discloses information or prevents authorized use of (data or a computer owned by or operated for) the Government of the United States shall be punished by a fine under this title or imprisonment for not more than 10 years, or both.
Response: All activities on this system and network may be monitored, intercepted, recorded, read, copied, or captured in any manner and disclosed in any manner, by authorized personnel. THERE IS NO RIGHT OF PRIVACY IN THIS SYSTEM. System personnel may give to law enforcement officials any potential evidence of crime found on USDA computer systems. USE OF THIS SYSTEM BY ANY USER, AUTHORIZED OR UNAUTHORIZED, CONSTITUTES CONSENT TO THIS MONITORING, INTERCEPTION, RECORDING, READING, COPYING OR CAPTURING AND DISCLOSURE. REPORT UNAUTHORIZED USE TO AN INFORMATION SYSTEMS SECURITY OFFICER.
Response: 220 ********************************************
Command: AUTH TLS
Response: 500 'AUTH TLS': command not understood
Command: AUTH SSL
Response: 500 'AUTH SSL': command not understood
Status: Insecure server, it does not support FTP over TLS.
Command: USER anonymous
Response: 331 Anonymous access allowed, send identity (e-mail name) as password.
Command: PASS **************
Response: 230-********************************************
Response: This is the USDA Risk Management Agency-FCIC Anonymous FTP server.
Response: --------------------------------------------
Response: ALL transfers from this FTP server are logged. AND Activities while connected MAY BE MONITORED. If you don't like this then disconnect NOW! Send problem reports regarding this server to: webteam@rma.usda.gov
Response: ********************************************
Response: 230 Anonymous user logged in.
Status: Server does not support non-ASCII characters.
Status: Connected
Status: Starting download of /pub/References/actuarial_data_master/2016/2016_ADM_YTD.zip
Command: CWD /pub/References/actuarial_data_master/2016
Response: 250 CWD command successful.
Command: TYPE I
Response: 200 Type set to I.
Command: PORT 192,168,135,104,199,111
Response: 200 PORT command successful.
Command: REST 400556032
Response: 350 Restarting at 400556032.
Command: RETR 2016_ADM_YTD.zip
Response: 150 Opening BINARY mode data connection for 2016_ADM_YTD.zip(2335389680 bytes).
Error: Disconnected from server: ECONNABORTED - Connection aborted
Error: File transfer failed after transferring 398,917,632 bytes in 306 seconds
Status: Disconnected from server

Status: Resolving address of www.rma.usda.gov
Status: Connecting to 165.221.68.23:21...


thanks for the help!



 Category Survey
AutoMate 8 version: 8.0.1
Windows version: Windows 7



Task Steps: (Right click, Select All, Copy to copy to the clipboard. Steps can then be pasted into the AutoMate Task Builder)
 12/15/2015 11:26 AM
User is offline View Users Profile Print this message

Author Icon
Valentineo McGowan
HelpSystems

Posts: 766
Joined: 09/03/2013

Hello Mac,

Edit the FTP Logon step and add the following syntax 'exactly' to the 'FTP options' field under the 'Advanced' tab at the bottom of the action properties window:

DoNotSendAbort,KeepAliveDuringTransfer,UseLargeBuffers

And please re-test.

If an error generates; please provide that exact error message in your text post.

 Category Survey
AutoMate 8 version:
Windows version:


-------------------------

Valentineo McGowan | Technical Consultant
AutoMate | Division of HelpSystems
HelpSystems, LLC | T: +1 213.738.6966 | F: +1.213.738.7665
 12/15/2015 12:44 PM
User is offline View Users Profile Print this message

Author Icon
MacTurner
Student (5-19 posts)

Posts: 16
Joined: 06/07/2011

I attached the updated automate code. Here is the error I get:

This report is to inform you that an AutoMate Task step has failed to execute properly:

Task name: TEST ADM FTP DOWNLOAD
Step number: 8
Time of error: 12/15/2015 2:32:51 PM
Error message: FTP Error : An existing connection was forcibly closed by the remote host.
Computer name: MYCOMPUTER


 Category Survey
AutoMate 8 version: 8.0.1
Windows version: Windows 7



Task Steps: (Right click, Select All, Copy to copy to the clipboard. Steps can then be pasted into the AutoMate Task Builder)
 12/17/2015 03:57 PM
User is offline View Users Profile Print this message

Author Icon
Ricardo Castaneda
HelpSystems

Posts: 3424
Joined: 01/11/2008

Hello,

Can you please double check that the FTP Server does not have a timeout set?

 Category Survey
AutoMate 8 version:
Windows version:


-------------------------

Ricardo Castaneda | Principal Support Analyst
AutoMate | Division of HelpSystems
HelpSystems, LLC | T: +1.213.738.1700
 12/18/2015 05:21 AM
User is offline View Users Profile Print this message

Author Icon
MacTurner
Student (5-19 posts)

Posts: 16
Joined: 06/07/2011

Originally posted by: Ricardo Castaneda

Hello,



Can you please double check that the FTP Server does not have a timeout set?



The server starting having a 5 minute disconnect a couple weeks ago. I have no control over this. As I said in my original post, filzilla can handle these disconnects and resumes where automate does not.

Did you try to run my code on your end? It is a public ftp site so you should be able to do this.


 Category Survey
AutoMate 8 version: 8.0.1
Windows version: Windows 7
 12/21/2015 04:22 PM
User is offline View Users Profile Print this message

Author Icon
Valentineo McGowan
HelpSystems

Posts: 766
Joined: 09/03/2013

Hello MacTurner,

1 We are unable to connect to your FTP Server using the AML code provided.

[Main](Step 0) Task Started.
[Main](Step 1) Logging on to ftp server "www.rma.usda.gov". Session "RMA" started.
[Main](Step 1) Could not connect to server.
[Main](Step 1) Task Ended.

2 Also, Select 'Use compression' under Advanced.

3 Was the error message: "FTP Error : An existing connection was forcibly closed by the remote host." for Step 8 on the Logon or Upload?

 Category Survey
AutoMate 8 version:
Windows version:


-------------------------

Valentineo McGowan | Technical Consultant
AutoMate | Division of HelpSystems
HelpSystems, LLC | T: +1 213.738.6966 | F: +1.213.738.7665
 12/23/2015 07:54 AM
User is offline View Users Profile Print this message

Author Icon
MacTurner
Student (5-19 posts)

Posts: 16
Joined: 06/07/2011

the error I get is when downloading. It looks like the host ftp server is down at the moment. I will let you know when it is back up for testing. probably after christmas.

 Category Survey
AutoMate 8 version: 8.0.1
Windows version: Windows 7
 12/28/2015 03:26 PM
User is offline View Users Profile Print this message

Author Icon
Ricardo Castaneda
HelpSystems

Posts: 3424
Joined: 01/11/2008

Hello,

We look forward to hear from you.

 Category Survey
AutoMate 8 version:
Windows version:


-------------------------

Ricardo Castaneda | Principal Support Analyst
AutoMate | Division of HelpSystems
HelpSystems, LLC | T: +1.213.738.1700
 01/05/2016 07:30 AM
User is offline View Users Profile Print this message

Author Icon
MacTurner
Student (5-19 posts)

Posts: 16
Joined: 06/07/2011

The host ftp server is back up so you can try from your end again.

 Category Survey
AutoMate 8 version: 8.0.1
Windows version: Windows 7
 01/11/2016 07:36 AM
User is offline View Users Profile Print this message

Author Icon
MacTurner
Student (5-19 posts)

Posts: 16
Joined: 06/07/2011

have you had a chance to look at this?

 Category Survey
AutoMate 8 version: 8.0.1
Windows version: Windows 7
 01/14/2016 08:56 AM
User is offline View Users Profile Print this message

Author Icon
Valentineo McGowan
HelpSystems

Posts: 766
Joined: 09/03/2013

Hello Mac,

Thank you for the update.
I am presently re-testing your task steps now.
I was able to connect to the FTP server without issue.
Currently the '2016_ADM_YTD.zip' file is downloading. I will let you know if it aborts after 300 seconds (5 minutes).

 Category Survey
AutoMate 8 version:
Windows version:


-------------------------

Valentineo McGowan | Technical Consultant
AutoMate | Division of HelpSystems
HelpSystems, LLC | T: +1 213.738.6966 | F: +1.213.738.7665
 01/14/2016 09:59 AM
User is offline View Users Profile Print this message

Author Icon
MacTurner
Student (5-19 posts)

Posts: 16
Joined: 06/07/2011

It looks like they changed the timeout on host server. It now times out one time after 10 minutes or so. Automate seems to handle the file downloads now.

I believe automate still has an issue with timeouts but now that they updated the server, it is currently working.



 Category Survey
AutoMate 8 version: 8.0.1
Windows version: Windows 7
 01/14/2016 10:11 AM
User is offline View Users Profile Print this message

Author Icon
Valentineo McGowan
HelpSystems

Posts: 766
Joined: 09/03/2013

Hello Mac,

As you can see from the attached screenshot I was able to download your entire 2GB file without issue. There were no errors.

[Main](Step 0) Task Started.
[Main](Step 1) Logging on to ftp server "www.rma.usda.gov". Session "RMA" started.
[Main](Step 2) Total File(s) to Download : 1
[Main](Step 2) Downloading file "/pub/References/actuarial_data_master/2016/2016_ADM_YTD.zip" to "c:\temp\ADM\2016_ADM_YTD.zip"...
[Main](Step 3) Logging off from ftp server "www.rma.usda.gov".
[Main](Step 3) Task Ended.

 Category Survey
AutoMate 8 version:
Windows version:


Click for full image
Click for larger image



-------------------------

Valentineo McGowan | Technical Consultant
AutoMate | Division of HelpSystems
HelpSystems, LLC | T: +1 213.738.6966 | F: +1.213.738.7665
Statistics
18258 users are registered to the AutoMate Discussion forum.
There are currently 0 users logged in.
The most users ever online was 5551 on 01/08/2018 at 11:11 AM.
There are currently 1822 guests browsing this forum, which makes a total of 1822 users using this forum.

FuseTalk Enterprise Edition v4.0 - © 1999-2020 FuseTalk Inc. All rights reserved.

Sitemap Network Automation Software Blog