Put Time Out Issue

Feb 6, 2010 at 11:07 PM

I started using the command line tool the other day to perform backups of my photos, docs, code, etc.  Shortly after running my first backup, I ran into an issue where I received the cancelled error message.  After doing some googling, I started to find out what the issue is.  The error message is actually caused by timeout.  Since, I'm using my home connection (which is limited to 200-300k/sec), I'm limited to a file size which can be sent within the timeout * 200k.  I pulled down the code and as of v1.5, the put timeout is limited to 3 minutes.  So the max size file I can send is about 36,000k.  

While many people may be pushing files with a faster upload, some will need to be able to use it over a slower connection.  To resolve this issue there are a couple things we can do to make it more flexible ...

  1. Add a timeout parameter to allow the user to increase the timeout if needed
  2. Set the timeout to infinite
  3. Increase the timeout to a larger value

I downloaded the source code and simply increased the timeout to 45 minutes, which resolved the issue.  However, it would probably be better to make it a parameter or simply set it to infinite.  In any case, I don't mind making the code changes and submitting a patch; however, I'm curious what others think of this issue and the best solution ...

Coordinator
Feb 7, 2010 at 11:45 PM

Hi, thanks so much for this (and for being so polite about what is a pretty basic error).

Here's my suggested change, I hope you agree:

                    request.Timeout = Timeout.Infinite;
                    (request as HttpWebRequest).ReadWriteTimeout = 3 * 60 * 1000;  // 3 minutes
                    request.Timeout = Timeout.Infinite;
                    (request as HttpWebRequest).ReadWriteTimeout = 3 * 60 * 1000;  // 3 minutes

The latter does what I intended the .Timeout property to do, which is to throw an exception when the transfer has stalled so a retry can begin.

 

http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.timeout.aspx

http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.readwritetimeout.aspx

Feb 8, 2010 at 2:36 AM

Yes, I would agree.  That should provide the best of both worlds.

Great utility you've created!

Jul 23, 2010 at 10:09 AM
Edited Jul 23, 2010 at 1:38 PM
Very cool utility. However, I was getting timeouts while uploading large files to my bucket in my (currently) slow connection. To fix this, I removed the Timeout assignment of 30000 from Response.Response(WebRequest request). As far as I can see, the timeout should already be set up by the above code, so resetting it to a fixed number does not seem right. Edit: removing the Timeout from Response improved things, but with large files I may still see errors. I got was when uploading a 5.5MB file; after around 90 seconds the upload timed out and S3.exe didn't retry. The server had responded with a 400 error, which I unfortunately lost before copying it somewhere. It may have been a fluke with my connection, but FYI...