Fraser Speirs (![[info]](http://l-stat.livejournal.com/img/userinfo.gif){kind=small} fraserspeirs) wrote,@ 2006-04-24 09:47:00 |
|
HTTP Nerdery, or, Why NSURLConnection Sucks For POST.
I spent some time at the weekend re-plumbing a major component of FlickrExport. There's one class that does all the hard work of hauling your images up to Flickr, it's called FKPhotoUploadRequest and it uses NSURLConnection behind the scenes. NSURLConnection is a great class and I love it to bits, at least when I'm attempting an HTTP GET. FlickrExport, unfortunately but necessarily, uses HTTP POST to send images and NSURLConnection is distinctly buggy when used in this mode of operation.
At the nerd level, the problem can be seen by running tcpflow whilst uploading in FlickrExport. Just occasionally, you'll see the headers being sent and then .... nothing. Where there should be a very large MIME-encoded body, there's just nothing. At the user level, the problem manifests as FlickrExport just stopping and never making progress. This is because it's waiting on NSURLConnection to tell it that the post succeeded or failed. When this bug bites, the connection goes off into an interminable waiting state and never calls back success or failure. It's not that NSURLConnection is erroring and we're not catching it.
It appears to be a total heisenbug. I have innumerable support emails saying things like "FlickrExport stalls at image 7!" or "I can never get past image 2!!" - each time a different number, but the same problem.
Anyway, I took matters into my own hands and wrote a replacement class, built on the super-duper SmallSockets. It's not a perfect replacement - it's mostly designed around the needs of uploading rather than downloading - but it's already more reliable than NSURLConnection for this problem.
Another flaw in NSURLConnection, and this time by design, is that it only reports progress in retrieving large objects from the server. It tells you nothing until it's sent the entire NSURLRequest and only then starts to call back about the number of bytes that have come down. When you're POSTing requests containing images which may be 3-4Mb, this can be a very long delay between starting the upload and getting any feedback. This is why FlickrExport doesn't report to you the progress of uploading any individual image, only the overall progress. I don't like this and users don't like it either.
In my new HTTP class, I made it call back every time it uploads 10k of the request, as well as calling back on download progress. This means we can do a fine-grained progress meter for each image as it uploads as well as an overall progress meter for the batch.
I spent some time at the weekend re-plumbing a major component of FlickrExport. There's one class that does all the hard work of hauling your images up to Flickr, it's called FKPhotoUploadRequest and it uses NSURLConnection behind the scenes. NSURLConnection is a great class and I love it to bits, at least when I'm attempting an HTTP GET. FlickrExport, unfortunately but necessarily, uses HTTP POST to send images and NSURLConnection is distinctly buggy when used in this mode of operation.
At the nerd level, the problem can be seen by running tcpflow whilst uploading in FlickrExport. Just occasionally, you'll see the headers being sent and then .... nothing. Where there should be a very large MIME-encoded body, there's just nothing. At the user level, the problem manifests as FlickrExport just stopping and never making progress. This is because it's waiting on NSURLConnection to tell it that the post succeeded or failed. When this bug bites, the connection goes off into an interminable waiting state and never calls back success or failure. It's not that NSURLConnection is erroring and we're not catching it.
It appears to be a total heisenbug. I have innumerable support emails saying things like "FlickrExport stalls at image 7!" or "I can never get past image 2!!" - each time a different number, but the same problem.
Anyway, I took matters into my own hands and wrote a replacement class, built on the super-duper SmallSockets. It's not a perfect replacement - it's mostly designed around the needs of uploading rather than downloading - but it's already more reliable than NSURLConnection for this problem.
Another flaw in NSURLConnection, and this time by design, is that it only reports progress in retrieving large objects from the server. It tells you nothing until it's sent the entire NSURLRequest and only then starts to call back about the number of bytes that have come down. When you're POSTing requests containing images which may be 3-4Mb, this can be a very long delay between starting the upload and getting any feedback. This is why FlickrExport doesn't report to you the progress of uploading any individual image, only the overall progress. I don't like this and users don't like it either.
In my new HTTP class, I made it call back every time it uploads 10k of the request, as well as calling back on download progress. This means we can do a fine-grained progress meter for each image as it uploads as well as an overall progress meter for the batch.
