Skip to content

onProgress() not working correctly when add Stream object to RequestParams #380

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
VerachadW opened this issue Nov 12, 2013 · 7 comments
Closed
Assignees
Milestone

Comments

@VerachadW
Copy link

When I upload a file by adding a Stream object instead of File object, the progress was jumped from 0 to 100% instantly.

PS. I found that this issue was reported in #118, but I think create as a new issue would be easier to track.

@ghost ghost assigned smarek Nov 16, 2013
@smarek
Copy link
Member

smarek commented Nov 16, 2013

Hello @VerachadW, the problem is in my opinion that size of blocks to be reported (chunk size) is greater than your file size. #118 is request to add the feature, so it's right to create new issue.

@hillold
Copy link

hillold commented Nov 20, 2013

@VerachadW Thanks for creating the new issue! I was facing the same problem and was discussing with Adam in #118 . I was about to open a separate issue. Thanks for doing that!

@hateum
Copy link

hateum commented Dec 30, 2013

try this it work, it's very simple to fix:

public void onProgress (int bytesWritten, int totalSize)
{
Log.d(LOG_TAG, String.format("Progress %d from %d (%d%%)", bytesWritten, totalSize, (totalSize > 0) ? (int) (((float) bytesWritten / (float) totalSize) * 100f) : -1));
}

@xujinyang
Copy link

@hateum when l use your method l got this:01-06 13:Progress 6818 from 4096 (166%),l don't know why total size is 4096 but bytesWritten is bigger than it.

@smarek smarek closed this as completed Feb 18, 2014
@smarek
Copy link
Member

smarek commented Feb 18, 2014

Long inactivity

@Clearhu
Copy link

Clearhu commented Feb 20, 2014

I fixed this issue, should @smarek merge code in main version?

/**
 * Adds an input stream to the request.
 *
 * @param key         the key name for the new param.
 * @param stream      the input stream to add.
 * @param name        the name of the stream.
 * @param contentType the content type of the file, eg. application/json
 * @param length      the content length, if set can support upload progress
 */
public void put(String key, InputStream stream, String name, String contentType, long length) {
    if (key != null && stream != null) {
        streamParams.put(key, new StreamWrapper(stream, name, contentType, length));
    }
}


    // Add stream params
    for (ConcurrentHashMap.Entry<String, StreamWrapper> entry : streamParams.entrySet()) {
        StreamWrapper stream = entry.getValue();
        if (stream.inputStream != null) {
            entity.addPart(entry.getKey(), stream.name, stream.inputStream,
                    stream.contentType, stream.length);
        }
    }


public static class StreamWrapper {
    public InputStream inputStream;
    public String name;
    public String contentType;
    public long length;

    public StreamWrapper(InputStream inputStream, String name, String contentType) {
        this(inputStream, name, contentType, 0);
    }

    public StreamWrapper(InputStream inputStream, String name, String contentType, long length) {
        this.inputStream = inputStream;
        this.name = name;
        this.contentType = contentType;
        this.length = length;
    }
}

private List inputStreamParts = new ArrayList();

public void addPart(final String key, File file, String type) {
    if (type == null) {
        type = APPLICATION_OCTET_STREAM;
    }
    // fileParts.add(new FilePart(key, file, type));

    FileInputStream inputStream = null;
    try
    {
        inputStream = new FileInputStream(file);
    }
    catch (FileNotFoundException e)
    {
        Log.w(LOG_TAG, "Cannot find file exception", e);
    }

    if (inputStream == null)
    {
        return;
    }

    inputStreamParts.add(new InputStreamPart(key, file.getName(), inputStream, type, file.length()));
}

public void addPart(String key, String streamName, InputStream inputStream, String type, long length)
        throws IOException {
    if (type == null) {
        type = APPLICATION_OCTET_STREAM;
    }

    if (length > 0)
    {
        inputStreamParts.add(new InputStreamPart(key, streamName, inputStream, type, length));
        return;
    }


    out.write(boundaryLine);

    // Headers
    out.write(createContentDisposition(key, streamName));
    out.write(createContentType(type));
    out.write(TRANSFER_ENCODING_BINARY);
    out.write(CR_LF);

    // Stream (file)
    final byte[] tmp = new byte[4096];
    int l;
    while ((l = inputStream.read(tmp)) != -1) {
        out.write(tmp, 0, l);
    }

    out.write(CR_LF);
    out.flush();
    try {
        inputStream.close();
    } catch (final IOException e) {
        // Not important, just log it
        Log.w(LOG_TAG, "Cannot close input stream", e);
    }
}

private byte[] createContentType(String type) {
    String result = "Content-Type: " + type + "\r\n";
    return result.getBytes();
}

private byte[] createContentDisposition(final String key) {
    return ("Content-Disposition: form-data; name=\"" + key + "\"\r\n")
            .getBytes();
}

private byte[] createContentDisposition(final String key, final String fileName) {
    return ("Content-Disposition: form-data; name=\"" + key + "\"; filename=\"" + fileName + "\"\r\n")
            .getBytes();
}

private void updateProgress(int count) {
    bytesWritten += count;
    progressHandler.sendProgressMessage(bytesWritten, totalSize);
}

private class InputStreamPart
{
    public InputStream inputStream;
    public byte[] header;
    public long length;

    public InputStreamPart(String key, String streamName, InputStream inputStream, String type, long length)
    {
         header = createHeader(key, streamName, type);
         this.inputStream = inputStream;
         this.length = length;
    }

    private byte[] createHeader(String key, String streamName, String type) {
        ByteArrayOutputStream headerStream = new ByteArrayOutputStream();
        try {
            headerStream.write(boundaryLine);

            // Headers
            headerStream.write(createContentDisposition(key, streamName));
            headerStream.write(createContentType(type));
            headerStream.write(TRANSFER_ENCODING_BINARY);
            headerStream.write(CR_LF);
        } catch (IOException e) {
            // Can't happen on ByteArrayOutputStream
            Log.e(LOG_TAG, "createHeader ByteArrayOutputStream exception", e);
        }
        return headerStream.toByteArray();
    }

    public long getTotalLength() {
        long streamLength = this.length + CR_LF.length;
        return header.length + streamLength;
    }

    public void writeTo(OutputStream out) throws IOException {
        out.write(header);
        updateProgress(header.length);

        final byte[] tmp = new byte[4096];
        int l;
        while ((l = inputStream.read(tmp)) != -1) {
            out.write(tmp, 0, l);
            updateProgress(l);
        }
        out.write(CR_LF);
        updateProgress(CR_LF.length);
        out.flush();
        try {
            inputStream.close();
        } catch (final IOException e) {
            // Not important, just log it
            Log.w(LOG_TAG, "Cannot close input stream", e);
        }
    }
}

// The following methods are from the HttpEntity interface

@Override
public long getContentLength() {
    long contentLen = out.size();

    for (InputStreamPart inputStreamPart : inputStreamParts) {
        long len = inputStreamPart.getTotalLength();
        if (len < 0) {
            return -1; // Should normally not happen
        }
        contentLen += len;
    }
    contentLen += boundaryEnd.length;
    return contentLen;
}

@Override
public Header getContentType() {
    return new BasicHeader("Content-Type", "multipart/form-data; boundary=" + boundary);
}

@Override
public boolean isChunked() {
    return false;
}

public void setIsRepeatable(boolean isRepeatable) {
    this.isRepeatable = isRepeatable;
}

@Override
public boolean isRepeatable() {
    return isRepeatable;
}

@Override
public boolean isStreaming() {
    return false;
}

@Override
public void writeTo(final OutputStream outstream) throws IOException {
    bytesWritten = 0;
    totalSize = (int) getContentLength();
    out.writeTo(outstream);
    updateProgress(out.size());

    for (InputStreamPart inputStreamPart : inputStreamParts) {
        inputStreamPart.writeTo(outstream);
    }
    outstream.write(boundaryEnd);
    updateProgress(boundaryEnd.length);
}

@Override
public Header getContentEncoding() {
    return null;
}

@Override
public void consumeContent() throws IOException, UnsupportedOperationException {
    if (isStreaming()) {
        throw new UnsupportedOperationException(
                "Streaming entity does not implement #consumeContent()");
    }
}

@Override
public InputStream getContent() throws IOException, UnsupportedOperationException {
    throw new UnsupportedOperationException(
            "getContent() is not supported. Use writeTo() instead.");
}

Use:

params.put("file", stream, "photo.jpg", "image/jpeg", length);

@okrosa
Copy link

okrosa commented Aug 18, 2014

Guys what is the status of this ticket? As @VerachadW said, the progress is still not working correctly for streams, just for files. If it won't be fixed I cannot really give meaningful feedback for the user most of the time...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants