CE303

Advanced Programming

Somdip Dey
Today:
Web services: Exchanging information over the web
 Overview

The HTTP Protocol

Web Services

RESTful Web Services

Demo of RESTful Web Application using JAX-RS / Jersey

and Tomcat
Part I:
The HyperText Transfer Protocol (HTTP)
 Hypertext Transfer Protocol (HTTP)

 An application layer protocol

- Part of the Internet Protocol Suite
- Normally run on top of TCP

 A response-request style protocol

- I.e. the interaction protocol in a client-server model.

 Supports many features:

- E.g. authentication, encryption (e.g. https), caching, use of proxy servers

 Multiple versions exist (and in use)

- HTTP/1.0: A separate connection is made to the same server for every resource request
- HTTP/1.1: Various improvements: reusable connection; caching, content negotiation, etc. Human readable.
- HTTP/2: Improvements over 1.1 (compression, parallelization, etc); notably, no longer human readable.
- HTTP/3: Does not rely on TCP, but on QUIC instead (which is datagram based).
Slide 6/N
 What are “Resources”

 Concept of a Universal Resource Identifier (URI)

 Two main forms:
For more information see:
- Universal Resource Locator (URL), a.k.a. web address. https://developer.mozilla.org/en-
- Represents information on where / how the resource can be accessed. US/docs/Web/HTTP/Basics_of_
HTTP/Identifying_resources_on
- E.g. http://example.com:80/path/to/myfile.html?key1=value1&key2=value2#SomewhereInTheDocument _the_Web
(Syntax: protocol, domain name (a.k.a. authority), port, path, queries, fragment
https://stackoverflow.com/questi
ons/4913343/what-is-the-differe
- Universal Resource Name (URN) nce-between-uri-url-and-urn
- Uniquely identifies a resource by name, in the context of a particular namespace.
- The resource may or may not be accessible – clients decide how or where to look for the resource.
https://developer.mozilla.org/en-
- E.g. urn:isbn:9780141036144 (The book Nineteen Eighty-Four by George Orwell) US/docs/Web/HTTP/Basics_of_
HTTP/MIME_Types
- Popular with Peer-to-Peer protocols, including the IPFS.

 The HTTP protocol relies on URLs.

- A number of other popular protocols rely on URLs too. E.g. data, file, ftp, http/https, javascript, mailto, ssh, smb, tel, view-source, ws/wss

 Underlying resource can represent different media content types (MIME type)
- E.g. text/plain, text/html, text/javascript, image/png, video/mp4, audio/mpeg, application/pdf, application/octet-stream, multipart/form-data

Slide 7/N
 HTTP Protocol specification - Requests
 A single line consisting of:
- An HTTP method, that describes the action to be performed.
- E.g. a ‘verb’ such as GET, POST, PUT, DELETE, CONNECT or a ‘noun’ such as HEAD, OPTIONS, TRACE, PATCH.
- For a full list see: https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods

- A request target, which could be a URL, or an absolute path to a file, relative to the host server’s web-root
- Typically a path, as the remaining components, i.e. protocol, domain, and port, are obvious from the context of the connection
- Specifying only a directory typically implies requesting the index.html file from the relevant directory by default

- The specific version of the HTTP protocol used

- Full example: GET / HTTP/1.1

 Optional headers
- Syntax: “Key: value”, one per line. E.g. “Host: example.com” (Host is the only mandatory header for HTTP/1.1), Content-type, Accept, etc.
- The start-line and optional headers are often collectively referred to as the ‘head’ of the request.

 Optionally, a message body

- Only when required by the specific method used, e.g. submitting form content via POST requests

For more information, see: - https://developer.mozilla.org/en-US/docs/Web/HTTP/Overview#requests

- https://developer.mozilla.org/en-US/docs/Web/HTTP/Messages - https://en.wikipedia.org/wiki/List_of_HTTP_header_fields

Slide 8/N
 HTTP Request Example

POST /examples/rest/bank/transfer/cust1/ HTTP/1.1

Host: localhost:8080
Accept: application/json
Content-Type: application/json

{ "fromAccount":105, "toAccount":100, "amount":50.0}

Slide 9/N
 HTTP Protocol specification - Responses
 A single line consisting of Some HTTP Status Codes
- The version of the HTTP protocol used.
- A status code, indicating if the request was successful or not, and why. 2xx Success
200 OK
- A status message, acting as a short description of the status code.
204 No Content

 HTTP headers 3xx Redirection

- ‘Key: Value’ pairs, like those for requests. 301 Moved Permanently
- See: https://en.wikipedia.org/wiki/List_of_HTTP_header_fields for a full list
4xx Client Error
400 Bad Request
 Optionally, a body containing the fetched resource. 401 Unauthorized
- Example: Status 200 OK 404 Not Found
Content-Length: 17 406 Not acceptable
Content-Type: application/json 418 I'm a teapot
Date: Sat, 05 Aug 2017 16:09:33
GMT 5xx Server Error
500 Internal Server Error
4795.847610355551

Slide 10/N
 HTTP Request Methods
 GET
- The GET method requests a representation of the specified resource. Requests using GET should only retrieve data.

 POST
- The POST method submits an entity to the specified resource, often causing a change in state or side effects on the server.

 PUT
- The PUT method replaces all current representations of the target resource with the request payload.

 DELETE
- The DELETE method deletes the specified resource.

 HEAD
- The HEAD method asks for a response identical to a GET request, but without the response body.

 CONNECT
- The CONNECT method establishes a tunnel to the server identified by the target resource.

 OPTIONS
- The OPTIONS method describes the communication options for the target resource.

 TRACE
- The TRACE method performs a message loop-back test along the path to the target resource.

 PATCH
- The PATCH method applies partial modifications to a resource.

For more details, see: https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods

Slide 11/N
 GET Requests

 GET request means: “give me this resource”

 GET requests should be preferred choice for queries that do not modify data

 GET requests often originate from

- typing a URL into a web browser
- clicking on a hyper link in a web page
- redirecting a web page
- submitted forms with method set to GET (e.g. Google)

 The URL can encode parameters (“query string”) which are decomposed on the
server (“URL rewriting”):
- http://www.mysite.org/myservice?fName=Fred&sName=Flintstone

Slide 12/N
 POST Requests

 POST requests have a ‘payload’ attached

- i.e. they ‘send’ data.

 The payload is not visible in the URL

- Also not limited to how much can fit in an address line

 They often originate from HTML elements where the method is set to POST:

<FORM action="ProcessServlet" method = "POST">

…
</FORM>

 POST requests should be used preferably when you want to create or modify data

Slide 13/N
 GET vs POST comparison

 GET
- Appends the form data to the URL, in name/value pairs
- NEVER use GET to send sensitive data! (the submitted form data is visible in the URL!)

- The length of a URL is limited (2048 characters)

- Useful for form submissions where a user wants to bookmark the result
- GET is good for non-secure data, like query strings in Google

 POST
- Appends the form data inside the body of the HTTP request
- The submitted form data is not shown in the URL

- POST has no size limitations, and can be used to send large amounts of data.
- Form submissions with POST cannot be bookmarked

Slide 14/N
 Media Types and Content Negotiation
 HTTP clients can specify:
- the media type they are sending, and
- the media type they expect in the response of the server
- E.g. see Content-Type / Accept headers

 HTTP servers can specify:

- the media types they can provide for a certain resource
- the media types they expect in the request sent by the client

 HTTP supports “content negotiation”:

- selecting the “best” representation for a given response

 Some basic media types:

- application/json, application/x-www-form-urlencoded
- text/plain, text/html, text/css, image/png, …

For more information, see: https://en.wikipedia.org/wiki/List_of_HTTP_header_fields / https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_Types

Slide 15/N
Part II:
Web Services
Slide 17/N
Slide 18/N
 Let’s think about this for a second …

 If you log in to the airline’s website directly, the process is straightforward

- Receive the request
- Query a local database
- Prepare a response
- Send it to the customer

 But. Does the travel agent have access to the airline’s database?
- No. The airline provides an ‘interface’ (interaction protocol?) for requests
- The travel agent transforms your request to the kind expected by the airline’s API.
- The airline provider responds appropriately with a useful resource (JSON?)

 No requirement for the travel agent and many different airlines to be implemented
in the same language / stack.
- As long as they can handle requests and respond in a useful manner

Slide 19/N
 Web Applications vs Web Services

 A Web Application:
- Is for human users.
- Typically a complete application
- Involves interaction with a browser or other front-end / Graphical User Interface (GUI).

 A Web Service:
- Generally not intended for direct human consumption
- Provides some “computational service” for the purpose of machine-to-machine interaction via HTTP(S), e.g. data transfer between Web Applications.

- Typically a smaller component within a larger application.

- Involves an machine-to-machine interaction protocol, rather than a user interface
- Generally platform / language agnostic
- Transferred data typically in the form of JSON, XML or text

Slide 20/N
 Web Service Communication styles

 Two main approaches used

- Remote Procedure Call (RCP) based – protocol oriented
- Most established protocol: Simple Object Access Protocol (SOAP)
- More cumbersome, but can have benefits, e.g. considerations / assurances on security, identity verification, privacy, data integrity, etc. Common in banking.

- Web APIs – resource oriented

- Most common architecture: Representational State Transfer (REST)
- Uses the existing facilities already provided by the HTTP protocol, but extending use beyond original intent of simply transferring webpages / images, etc.

 Why “REST”?
- Representational – because it leverages HTTPs existing ability to deal with many ‘representation types’ (i.e.
html, xml, json)
- Each object at the end of a URI corresponds to a particular ‘representation’ of a resource, which can be of a specific type of resource.

- State, because these representations can have dynamic state

- E.g. the specific object may take many states, depending on the specific URI / query parameters used

- Transfer, because this representation in this specific state can be transferred between web services.

Slide 21/N
 So what exactly classifies as RESTful?

 W3 consortium definition:
“The REST Web is the subset of the WWW (based on HTTP) in which agents provide uniform interface semantics -- essentially
create, retrieve, update and delete -- rather than arbitrary or application-specific interfaces, and manipulate resources only by the
exchange of representations. Furthermore, the REST interactions are "stateless" in the sense that the meaning of a message does not
depend on the state of the conversation.

We can identify two major classes of Web services:

 REST-compliant Web services, in which the primary purpose of the service is to manipulate XML representations of Web
resources using a uniform set of "stateless" operations; and

 Arbitrary Web services, in which the service may expose an arbitrary set of operations.

Both classes of Web services use URIs to identify resources and use Web protocols (such as HTTP and SOAP 1.2) and XML data
formats for messaging”

Source: https://www.w3.org/TR/2004/NOTE-ws-arch-20040211/#relwwwrest

Slide 22/N
 So what exactly classifies as RESTful?

 Some guy at StackOverflow:

“A RESTful web service (also called a RESTful web API) is a web service implemented using HTTP
and the principles of REST. It is a collection of resources, with four defined aspects:

 The base URI for the web service, such as http://example.com/resources/

 An API, which must be hypertext driven.
 The Internet media type of the data supported by the web service.
This is often XML but can be any other valid Internet media type providing that it is a valid
hypertext standard.
 A set of operations supported by the web service using HTTP methods (e.g., GET, PUT, POST, or
DELETE).”

Source: https://stackoverflow.com/a/12728166

Slide 23/N
 REST Architectural elements

 Communication approach based on resources and standard web infrastructure

- As opposed to, e.g. SOAP, which relies on transmitting information that is application-logic specific.

 Resources are identified by URIs

 Use standard HTTP verbs to operate on resources

- Notably: POST, GET, PUT, DELETE

 Stateless protocols
- Each interaction is independent; a response/request pair is all that is required to complete that interaction.
- Therefore no session information (i.e. on past requests) needs to be stored by the server.

 Other:
- Use of standard content types: JSON, XML, text, ...
- Use of web features: HTTP status codes, caching, …
- Aims: simplicity, efficiency/scalability, re-use
Slide 24/N
 RESTful principles – URIs and HTTP Methods

 Clear, robust, consistent interfaces

- In terms of URIs: http(s)://{your-server}/rest-api/service/api/{version}/{resource} .
- In terms of consistency and disciplined use of HTTP Methods

 Consistent and Disciplined use of HTTP operations:

- E.g. GET should not change resources on the server
- For example, GET http://myserver/myresource/delete should NOT delete the resource

- POST: create or update / modify resource

- PUT: create or replace resource at specified URL

- PUT and DELETE operations should be idempotent

- Repetition of operation should have no further effect

Slide 25/N
Part III:
Web Services in Java, using JAX-RS / Jersey
 Web Services using JAX-RS / Jersey

 JAX-RS is a Java API specification for “RESTful” web services

- It uses annotations (lots!) to define ‘resources’ and the operations that can be performed on them.
- Annotations are also used to ‘inject’ request parameters and path-segments into the operations
- The resource ‘entities’ (i.e. content type) can be POJOs (“plain old java objects”)
- I.e. they should not implement any particular interface or extend a particular class
- These will be transformed accordingly, e.g. to JSON.

 Jersey is the canonical implementation of JAX-RS

- Now also part of the Jakarta EE (formerly Java EE / J2EE), a set of specifications, extending Java SE

 We will deploy it on Tomcat, launched from a Java program

- Tomcat is a webserver capable of serving both normal webpages, as well as ‘servlets’
- Tomcat is distributed as a standalone application, or “embedded” for launching programmatically.

Slide 27/N
 JAX-RS Fundamentals

 JAX-RS relies heavily on annotations…

 Resource methods are annotated with a request method designator

@GET, @PUT, @POST, or @DELETE

 These are the methods which are invoked in response to HTTP requests

 @Path annotations determine the mapping from URLs to resource methods

 Class level @Path value is prefixed to URL if present

Slide 28/N
Slide 29/N
What are annotations?

Slide 30/N
What are annotations?

Slide 31/N
What are annotations?

Ok. Back to Java

Slide 32/N
 What are annotations?
What are annotations?

public class NewClass extends OldClass {

@Override
public void oldMethod() {
// Do something different
}
}

Slide 33/N
 JAX-RS Fundamentals

 JAX-RS relies heavily on annotations

 Resource methods are annotated with a request method designator

@GET, @PUT, @POST, or @DELETE

 These are the methods which are invoked in response to HTTP requests
- Note: The methods themselves are never invoked by name. They are invoked instead via reflection.

 @Path annotations determine the mapping from URLs to resource methods

 Class level @Path value is prefixed to URL if present

Slide 34/N
 Some Basic JAX-RS Annotations

 @Path("/{id}") path parameter declaration

 @PathParam("…") path parameter binding
 @QueryParam("…") query parameter binding
 @FormParam("…") form parameter binding

 Content negotiation
- @Consumes specifies one or more media types a resource can accept (“consume”) from the client
- @Produces specifies one or more media types a resource can “produce” and send back to the client

Slide 35/N
 Deployment, Testing,
and Clients for RESTful Web Services
 Jersey web services need to be deployed on a web server
 Both the web server and Jersey can be configured extensively
 Once the server has started, the web services can be accessed via HTTP

 Clients need to generate appropriate HTTP requests and parse responses

 Web browsers can do this to some extend

- but they are not built for this purpose
- for example you cannot directly issue a POST request by entering it in the address bar of Chrome

 The curl command line tool can easily generate HTTP requests for testing
curl -H "Accept: text/plain" http://localhost:8080/examples/rest/hello/Sam

Slide 36/N
 Invoking RESTful Services from Java

 JAX-RS Jersey also has a Client API

- See: https://jersey.java.net/documentation/latest/client.html

 Typical usage
- Create a Web Target pointing at a Web resource
- Build the request
- Submit request and process the response

 Usage considerations:
- Web target creation includes building a client
- This can have an associated configuration such as setting of message handlers and logging levels

- Request building may involve construction of entities that are passed in the body
- Especially relevant for POST and PUT requests

- “Fluent API” with chaining of method calls

Slide 37/N
package webserver;

import java.io.File;

import javax.servlet.ServletException;

import org.apache.catalina.Context;
import org.apache.catalina.LifecycleException;
import org.apache.catalina.startup.Tomcat;
import org.glassfish.jersey.logging.LoggingFeature;
import org.glassfish.jersey.server.ResourceConfig;
import org.glassfish.jersey.servlet.ServletContainer;

public class TomcatServer {

public static final int TOMCAT_PORT = 8080;

public static final String EXAMPLES_URL = "http://localhost:8080/examples/rest";
public static final String BANK_URL = EXAMPLES_URL + "/bank";
public static final String BOOK_URL = EXAMPLES_URL + "/book";
public static final String HELLO_URL = EXAMPLES_URL + "/hello";

public TomcatServer() {
}

public void start() throws ServletException, LifecycleException { Slide 38/N

package hello;

import javax.ws.rs.Consumes;
import javax.ws.rs.FormParam;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.QueryParam;
import javax.ws.rs.core.MediaType;

@Path("/hello")
public class HelloServer {

@GET
@Path("/{name}")
@Produces(MediaType.TEXT_PLAIN)
public String helloPlain(@PathParam("name") String name) {
return "Hello, " + name + "!";
}

@GET
@Produces(MediaType.APPLICATION_JSON)
Slide 39/N
package hello;

import javax.ws.rs.client.ClientBuilder;
import javax.ws.rs.client.Entity;
import javax.ws.rs.client.WebTarget;
import javax.ws.rs.core.Form;
import javax.ws.rs.core.MediaType;

import webserver.TomcatServer;

public class HelloClient {

public WebTarget webTarget;

public HelloClient() {
webTarget = ClientBuilder.newClient().target(TomcatServer.HELLO_URL);
}

public String helloPathParam(String name) {

return webTarget.path(name).request().get(String.class);
}

public String helloQueryParam(String name) {

return webTarget.queryParam("name", name).request().get(String.class);
} Slide 40/N
 Final thoughts

 RESTful web services have become popular in recent years, especially for “public
facing” web services

 They are supported in many different programming languages and web application
frameworks

 JAX-RS / Jersey makes it reasonably straightforward to code web services in Java

 Supports programming of both servers and clients

 Worth reading further on JAX-RS and Jersey if interested in webservices,

- E.g. asynchronous client API and server-sent events

Slide 41/N
 Summary

The HTTP Protocol

Web Services

RESTful Web Services

Demo of RESTful Web Application using JAX-RS / Jersey

and Tomcat
Any Academic hours:
Wednesdays 14:00-15:00

questions? (please email if you can)

essex.ac.uk