Automatically sending Transfer-Encoding: chunked but I want to send Content-Length #1835
-
|
Hello! I'm trying to send a large file back to the client, and want to have a download progress bar so I need to send the Content-Length header. I know the file size before initiating the transfer, so that's fine. However, I read and send the file in chunks (while checking for backpressure) and this causes uWebsockets to automatically send the Transfer-Encoding: chunked header, with which it is forbidden to send the Content-Length header according to HTTP specification. I know I could send a custom header such as X-Content-Length, but for sake of general compatibilty reasons I would like to avoid that. Here is my code to send the file, am I missing something / is there another way to do this? void initSendFile(HttpResponse<true> *clientConnection, std::string filePath) {
//start sending a file in chunks to the client (handles backpressure, creates appropriate onwritable callback)
std::shared_ptr<std::ifstream> in = std::make_shared<std::ifstream>(filePath, std::ios::in | std::ios::binary);
uintmax_t fileSize = std::filesystem::file_size(std::filesystem::path(filePath));
std::shared_ptr<uintmax_t> bytesWritten = std::make_shared<uintmax_t>(0);
//clientConnection->writeHeader("Content-Length", std::to_string(fileSize));
std::cout << fileSize << " bytes size" << std::endl;
//see shared_ptr_cyclic_ownership.txt for why this is safe
std::function<bool(uintmax_t)> trySendTillEnd = [clientConnection, in, fileSize, bytesWritten](uintmax_t a) -> bool{
bool keepGoing = true;
while (keepGoing) {
uintmax_t totalBytesToSend = fileSize - *bytesWritten;
//std::cout << totalBytesToSend << std::endl;
if (totalBytesToSend > bigResponseChunkSize) {
char buffer[bigResponseChunkSize];
in->read(buffer, bigResponseChunkSize);
keepGoing = clientConnection->write(std::string(buffer, bigResponseChunkSize));
*bytesWritten += bigResponseChunkSize;
}
else {
//final write
char buffer[totalBytesToSend];
in->read(buffer, totalBytesToSend);
clientConnection->write(std::string(buffer, totalBytesToSend));
clientConnection->end();
in->close();
std::cout << "end transfer" << std::endl;
keepGoing = false;
}
}
return true;
};
trySendTillEnd(0); //the zero is just there as a dummy argument, doesn't do anything (it helps because then I can just use one lambda for both :D )
clientConnection->onWritable(trySendTillEnd);
} |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
It turns out I was going about this wrong, I re-wrote it to use tryEnd and onWritable instead of write: -- EDIT- IMPORTANT NOTE: I'm not too sure about this anymore because I noticed the while loop could potentially block the event loop for way too long if no backpressure occurs, so I don't recommend using this solution -- void initSendFile(HttpResponse<true> *clientConnection, std::string filePath) {
//start sending a file in chunks to the client (handles backpressure, creates appropriate onwritable callback)
std::shared_ptr<std::ifstream> in = std::make_shared<std::ifstream>(filePath, std::ios::in | std::ios::binary);
uintmax_t fileSize = std::filesystem::file_size(std::filesystem::path(filePath));
std::shared_ptr<char[]> buffer = std::make_shared<char[]>(bigResponseChunkSize);
std::cout << fileSize << " bytes size" << std::endl;
std::function trySendTillEnd = [clientConnection, in, buffer, fileSize](uintmax_t offset) -> bool {
//std::cout << offset << " offset backpressure triggered" << std::endl;
uintmax_t chunkPosition = offset % bigResponseChunkSize;
//Todo- chunkPosition is always zero? do we even need to account for partial chunks? How does tryEnd and onWritable work?
//std::cout << chunkPosition << " chunk position" << std::endl;
std::pair<bool, bool> resp = clientConnection->tryEnd(std::string(std::string(buffer.get(), bigResponseChunkSize), chunkPosition), fileSize);
bool cont = std::get<0>(resp);
bool finished = std::get<1>(resp);
while (cont == true && (!finished)) {
in->read(buffer.get(), bigResponseChunkSize);
resp = clientConnection->tryEnd(std::string(buffer.get(), bigResponseChunkSize), fileSize);
cont = std::get<0>(resp);
finished = std::get<1>(resp);
}
return true;
};
clientConnection->onWritable(trySendTillEnd);
//prefill buffer with data
in->read(buffer.get(), bigResponseChunkSize);
trySendTillEnd(0); //start sending
}This solution (while maybe not perfect) seems to work great, I haven't run into any issues so far (but do take it with a grain of salt, I am far from an expert) This also reads the file in chunks so you can send over large files without using huge amounts of RAM. |
Beta Was this translation helpful? Give feedback.
It turns out I was going about this wrong, I re-wrote it to use tryEnd and onWritable instead of write:
-- EDIT- IMPORTANT NOTE: I'm not too sure about this anymore because I noticed the while loop could potentially block the event loop for way too long if no backpressure occurs, so I don't recommend using this solution --