Draft:Tus (protocol)

For the protein, see Tus (biology).

tus

Developed by	Transloadit et al.
Introduced	April 2013 (2013-04)
Website	tus.io

Tus is an open-source protocol designed to optimize and standardize the process of file uploads using the Hypertext Transfer Protocol (HTTP). Tus addresses the challenge of uploading large files by enabling resumable uploads. If an upload is interrupted due to connection issues or other disruptions, tus can automatically resume the process from where it left off, without the need to restart the entire upload. This functionality is achieved by utilizing a server-side component, which saves the received data right up to the point of interruption and allows the client to only transmit the remaining part of the file.

The tus protocol is applicable to a broad range of use cases, including but not limited to cloud storage services, content sharing platforms, and data backup and recovery solutions. Tus is utilized across various industries by developers and companies aiming to enhance their platforms' upload systems, offering a more reliable and user-friendly interface for managing large file transfers.

Description

Tus is an open-source network protocol that facilitates resumable file uploads, i.e., the ability to interrupt uploads at any time and resume them afterwards from the state where the failure began. This interruption can be accidentally (e.g., a connection drop or server crash) or voluntarily if the user decides to pause the upload. In traditional uploading implementations the user’s progress would be lost in such situations, but tus allows for recovery from these interruptions and continuation from the point where the upload was disrupted.

The tus protocol is built upon HTTP/HTTPS, making it available on every platform including web browsers, desktop software and mobile applications. Tus may be useful to those who:

• operate on partly unreliable networks
• handle large files and want to avoid having to re-upload parts of it
• want to provide their users with the ability to pause and resume uploads

Protocol overview

The tus protocol provides the ability for resumable file uploads as part of the standard HTTP/HTTPS protocol. Tus achieves this by breaking an upload down into different HTTP requests.

Firstly, a POST request is sent by the client to initiate the upload. This will provide basic information about the upload to the server, which the server can choose to accept. In response, the server will send the upload URL as part of the Location header.

The client will then send the file data as part of a PATCH request to the upload URL provided by the server. The Upload-Offset header tells the server which byte offset to start writing the data from.

However, if the PATCH request fails, the upload can be resumed by sending a HEAD request to the upload URL. This tells the client the current Upload-Offset.

From this, the client can continue the upload with further PATCH requests until it is completed.

There is also the option to cancel an in-progress upload using a DELETE request. The server will clean up the upload resource and the upload cannot be resumed.

Example

An example of communication between a client and server using the tus protocol is shown below. This example was obtained through the tus demo page.

The client will create an empty file resource and then send a subsequent PATCH request to populate the resource with the file data:

# Client:
> POST /files HTTP/1.1
> Host: tus.example.org
> Tus-Resumable: 1.0.0
> Content-Length: 0
> Upload-Length: 100
> Upload-Metadata: filename d29ybGRfZG9taW5hdGlvbl9wbGFuLnBkZg==

The server responds to the client with the location of the empty resource:

# Server:
< HTTP/1.1 201 Created
< Location: http://tus.example.org/files/24e533e02ec3bc40c387f1a0e460e216
< Tus-Resumable: 1.0.0

The client now sends the file data to the empty resource created by the server:

# Client:
> PATCH /files/24e533e02ec3bc40c387f1a0e460e216 HTTP/1.1
> Host: tus.example.org
< Tus-Resumable: 1.0.0
> Content-Type: application/offset+octet-stream
> Content-Length: 30
> Upload-Offset: 0
>
> [first 30 bytes]

The server confirms it has received the first 30 bytes:

# Server:
< HTTP/1.1 204 No Content
< Tus-Resumable: 1.0.0

Protocol extensions

Tus has several protocol extensions available that clients and servers can implement.

Creation

The creation extension allows clients to create an empty upload resource on the server. The server then returns the location of this resource, with an implicit offset of 0. The client is then able to use the core protocol to perform the actual file upload. While the creation extension is optional, clients and servers are strongly advised to implement it. Most tus deployments initialize uploads using the creation extension, although it is also possible for clients to retrieve the upload URL through proprietary, non-standardized APIs.

Creation With Upload

The creation-with-upload extension allows the client to upload a portion of the file with the initial creation request. This extension directly depends upon the creation extension and therefore cannot be implemented by itself.

Expiration

The server can remove unfinished uploads once they expire. The expiration time is specified in the Upload-Expires header field from the response.

Checksum

The client and server can implement the checksum extension to ensure data integrity for each PATCH request. For each chunk upload, the server verifies the chunk against the client-supplied checksum using a pre-agreed algorithm.

Comparison with other file transfer methods

Popular protocols for file transmission include FTP, rsync and SCP. While these are built directly on top of TCP and therefore require access to the operating system's network stack, tus only utilizes HTTP. This allows tus to be used in environments where no direct TCP access is possible and HTTP is the only possible networking method. This is the case for websites, where browser do not provide TCP functionality.

Other HTTP-based upload methods, such as S3 multipart uploads, require the client to split the file into multiple chunks, which can then be uploaded individually. Whenever a chunk transfer fails, the entire chunk can be retried. Once all chunks are transferred, they are concatenated and form the complete upload. This chunked approach introduces inefficiencies, because retrying entire chunks requires retransmission of data that the server has already received. In addition, even a successful upload can stretch across multiple HTTP requests, which introduces additional overhead for the client and server. The chunk size must therefore be carefully chosen to strike a balance between reducing the number of HTTP requests, while also keeping chunks small enough to reduce the volume of retransmitted data.

Tus circumvents this problem by not employing chunking at all. Instead, it utilizes an optimistic approach where the client transmits and the server receives as much data as possible until the upload completes or is interrupted. If the transmission is completed without issues, no unnecessary HTTP requests were issued. In cases of interruptions, the server allows the client to resume the upload directly from the offset of the last byte that the server received, thereby avoiding retransmission of data.

Adoption

The tus protocol is used internally in many applications. Thanks to its nature as an open standard, it is also part of public APIs, such as the ones from Vimeo, Cloudflare, Supabase, Git LFS, Uppy and ownCloud.

Implementations

The tus protocol developers maintain server implementations in Go, Node.js and .NET, as well as client implementations for JavaScript, Java, Swift and Python. In addition, the community also provides a wide range of implementations for other platforms.

Development history

Development of the tus protocol was initiated in 2013 by file uploading and encoding company Transloadit as an open-source initiative to create a freely available protocol and implementations that are suitable for a wide range of applications. This attracted contributions of engineers from Google and the then CTO of Vimeo, Naren Venkataraman, who also took over maintenance of the project for a year.

Transloadit's Marius Kleidl took over leadership in 2015 and oversaw the production-ready 1.0 release of the tus protocol in 2015.

Since 2022, the tus project has joined efforts with Apple, Vimeo and Cloudflare in the IETF HTTP Working Group, the governing body over HTTP, to make resumable uploads an official extension to HTTP itself. While these efforts no longer continue to carry the name tus, the approach and concepts have largely been unchanged.

Future

The intention of the tus project is for 2.0 to be compatible with the protocol that will be standardized in IETF. It will continue to offer extensions around this core as well as implementations for different platforms for which browsers and servers do not yet offer native support.

Etymology

Originally an acronym for Transloadit Upload Server, and later The Upload Server, tus is now an orphan initialism and used as a standalone term. It is often written in lowercase as a tribute to Unix naming conventions.

See also

• Hypertext Transfer Protocol (HTTP)
• File Transfer Protocol (FTP)
• Rsync
• Secure Copy Protocol (SCP)

External links

• Official website

* Category:Internet protocols Category:Lists of network protocols