Skip to content

Upload multiple files cause server side error : "Stream ended unexpectedly" #332

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
solon-yu opened this issue Oct 16, 2013 · 12 comments
Closed

Upload multiple files cause server side error : "Stream ended unexpectedly" #332

solon-yu opened this issue Oct 16, 2013 · 12 comments
Assignees
@smarek
Labels
critical
Milestone
1.4.5

Comments

@solon-yu
Copy link

With latest code, when I uploading lots of image files to server, cuase this problem, server console shows log:

errlog

It always happen when uploading progress goes to 100%.

Device: Sony Ericsson Xperia Arc S
Server: Apache Tomcat 6.0

thanks!
@ghost ghost assigned smarek Oct 16, 2013
@smarek
Copy link
Member

smarek commented Oct 16, 2013

Hello, are you using maven provided snapshot or repository?

@solon-yu
Copy link
Author

Hi, smarek , I got latest code from the repository.
"Add upload progress request..." ( Issue #118 ), one solution by jensh ( jensh@256ec9425b4) does not have this problem, but sometimes, it cuase OutOfMemory Exception.

@smarek
Copy link
Member

smarek commented Oct 16, 2013

Is it possible, you'd test out how many of the file was submitted ? Compare files in bytes on both ends of communication if necessary.

And one more thing, you say when I uploading lots of image files to server, what is the count estimate? Is it tens or hundreds or thousands? How large files, over how fast network?

If I were you, I'd try to send one file at a time and keep the buffer, and probably you can try reducing the count of parallel threads ran by android-async-http, to test out if parallelism doesn't cause any issues in your case.

@smarek
Copy link
Member

smarek commented Oct 21, 2013

@solon-yu could you provide more info, so I can make a proper fix? I'm not going to use @jensh solution, as it was buffering whole file into memory, which is unfeasible.

@smarek
Copy link
Member

smarek commented Oct 21, 2013

Maybe try to find out how many bytes was supposed to be transfered and how many was transfered, I expect possibly some problem with writing bounds in multipart post.

@solon-yu
Copy link
Author

@smarek , I'm sorry for long time no replying your sug, I was busy with a project in last few days.
as your metion:
I was uploading about fifteen images(photo toke by camera, each about 2MB sizes) to server,
If I try to upload just one file, there's no problem.

@smarek
Copy link
Member

smarek commented Oct 24, 2013

Upload was parallel I suppose, can you find out how to upload proceeded? Meaning if the data transfered correctly, if there was start or end missing, on how many of the 15 files?

@smarek
Copy link
Member

smarek commented Oct 24, 2013

You should be able to monitor upload progress via onProgress callback on AsyncHttpResponseHandler, you say it's near to 100%, is it repeatable error? How much close to 100% Please try to investigate a little bit more, I'd really like to help you, but I need more information.

@smarek
Copy link
Member

smarek commented Oct 27, 2013

And one more question, do you use single request for each file or single request with all files?

@sozonnyk
Copy link

sozonnyk commented Nov 1, 2013

I have the same issue;

RequestParams params = new RequestParams();

final File icon = new File(new URI(element.getIcon()));
final File preview = new File(new URI(element.getPreview()));
final File file = new File(new URI(element.getFile()));

params.put("file", file);
params.put("preview", preview);
params.put("icon", icon);

client.post(API_URL, params,
        new AsyncHttpResponseHandler() {

Exception on the server side (Spring MVC + Commons Fileupload)

org.apache.commons.fileupload.MultipartStream$MalformedStreamException: Stream ended unexpectedly
        at org.apache.commons.fileupload.MultipartStream$ItemInputStream.makeAvailable(MultipartStream.java:982)
        at org.apache.commons.fileupload.MultipartStream$ItemInputStream.read(MultipartStream.java:886)
        at java.io.InputStream.read(InputStream.java:101)

In this case total size of 3 files was 433795 bytes.
Looks like something is wrong with multipart because network stream has one full file and part of second. No trace of third file. Content-lenght seems OK (434407), but i didnt recalculate it precisely.
The same code is working fine with 1.4.3

@smarek
Copy link
Member

smarek commented Nov 5, 2013

@sozonnyk where did you get those sizes? (433795, 434407) Could you possibly try to dump whole data on server side and see what part of request is malformed? I'll try to set up some debug environment in meantime.

@Dream74
Copy link

Dream74 commented Dec 24, 2013

Android Upload File Code :

for( String address : imgAddressList ) 
{
   // URL exmpale : file:///sdcard/20.png
   File file = new File(address.substring("file://".length()));
   try 
   {
    params.put(file.getName(), file) ;
   } catch (FileNotFoundException e) 
   {
    e.printStackTrace();
   }
}

Problem:

When you upload equal and more than three, some characters of boundary
will be missing in the end .

When you upload two files :

POST /CYCUFood/Image HTTP/1.1
Content-Length: 2231512
Content-Type: multipart/form-data; boundary=2bSkanUMJSquLPjFKv1T_3Be6DwpSQ
Connection: Keep-Alive
User-Agent: android-async-http/1.4.4 (http://loopj.com/android-async-http)
Cookie: JSESSIONID=0E01CDE1A5377B47F911E206F5C7EF30
Cookie2: $Version=1
Accept-Encoding: gzip

--2bSkanUMJSquLPjFKv1T_3Be6DwpSQ
Content-Disposition: form-data; name="storeUUID"
Content-Type: text/plain; charset=UTF-8

0231b1e8-76fe-4395-9bb2-b45913486155
--2bSkanUMJSquLPjFKv1T_3Be6DwpSQ
Content-Disposition: form-data; name="device_id"
Content-Type: text/plain; charset=UTF-8

28bf603b-c161-4be3-bd30-3806bcb90c4f
--2bSkanUMJSquLPjFKv1T_3Be6DwpSQ
Content-Disposition: form-data; name="03.jpg"; filename="03.jpg"
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary

<file byte>
--2bSkanUMJSquLPjFKv1T_3Be6DwpSQ
Content-Disposition: form-data; name="04.jpg"; filename="04.jpg"
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary

<file byte>
--2bSkanUMJSquLPjFKv1T_3Be6DwpSQ

When you upload three files :

POST /CYCUFood/Image HTTP/1.1
Content-Length: 2692826
Content-Type: multipart/form-data; boundary=t2EUCvDU_6jrO1VA6BuRosBD4xYMHE
Connection: Keep-Alive
User-Agent: android-async-http/1.4.4 (http://loopj.com/android-async-http)
Cookie: JSESSIONID=0E01CDE1A5377B47F911E206F5C7EF30
Cookie2: $Version=1
Accept-Encoding: gzip

--t2EUCvDU_6jrO1VA6BuRosBD4xYMHE
Content-Disposition: form-data; name="storeUUID"
Content-Type: text/plain; charset=UTF-8

0231b1e8-76fe-4395-9bb2-b45913486155
--t2EUCvDU_6jrO1VA6BuRosBD4xYMHE
Content-Disposition: form-data; name="device_id"
Content-Type: text/plain; charset=UTF-8

28bf603b-c161-4be3-bd30-3806bcb90c4f
--t2EUCvDU_6jrO1VA6BuRosBD4xYMHE
Content-Disposition: form-data; name="03.jpg"; filename="03.jpg"
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary

<file byte>
--t2EUCvDU_6jrO1VA6BuRosBD4xYMHE
Content-Disposition: form-data; name="04.jpg"; filename="04.jpg"
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary

<file byte>
--t2EUCvDU_6jrO1VA6BuRosBD4xYMHE
Content-Disposition: form-data; name="2012129174113560.jpg"; filename="2012129174113560.jpg"
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary

<file byte>
--t2EUCvDU_6jrO1VA6BuRosBD4xYM

smarek added a commit that referenced this issue Dec 28, 2013
@smarek
Merge pull request #414 from realxu/master
656a244 
Wrong length calculated when uploading files, Closing #332
@smarek smarek closed this as completed Dec 28, 2013
@gaobq gaobq mentioned this issue May 21, 2014
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@smarek smarek

Labels
critical
Projects
None yet
Milestone
1.4.5
Development

No branches or pull requests
4 participants
@smarek @Dream74 @sozonnyk @solon-yu