Skip to content

Commit 9063336

Browse files
Shrirang97ohbusiluwatar
authored
feature: Claim check pattern azure (iluwatar#1897)
* Archietecture Diagram added * Added pom.xml * Architecture Diagram Updated * Added Microservices and kafka * ReadME File Added * ReadME file Updated * iluwatar#1329 ReadMe file updated and working pattern code added * iluwatar#1329 readme file updated * iluwatar#1329 readme file updated and java documentation added * iluwatar#1329 repository merged * Update claim-check-pattern/ReadME.md iluwatar#1329 Real world description updated Co-authored-by: Subhrodip Mohanta <subhrodipmohanta@gmail.com> * iluwatar#1329 Update claim-check-pattern/ReadME.md Co-authored-by: Subhrodip Mohanta <subhrodipmohanta@gmail.com> * iluwatar#1329 Applicability section in ReadMe file updated Co-authored-by: Subhrodip Mohanta <subhrodipmohanta@gmail.com> * iluwatar#1329 Tutorial section in ReadMe file updated Co-authored-by: Subhrodip Mohanta <subhrodipmohanta@gmail.com> * iluwatar#1329 Storage Data section in ReadMe File updated Co-authored-by: Subhrodip Mohanta <subhrodipmohanta@gmail.com> * iluwatar#1329 workflow section Update claim-check-pattern/ReadME.md Co-authored-by: Subhrodip Mohanta <subhrodipmohanta@gmail.com> * iluwatar#1329 Update claim-check-pattern/pom.xml Co-authored-by: Subhrodip Mohanta <subhrodipmohanta@gmail.com> * iluwatar#1329 deleted mvnw.cmd file * iluwatar#1329 deleted drawio file * iluwatar#1329 deleted mvnw file * iluwatar#1329 deleted mvnw file * iluwatar#1329 mvnw.cmd file deleted * iluwatar#1329 Update claim-check-pattern/usage-cost-processor/src/main/java/com/callusage/domain/MessageHeader.java Co-authored-by: Subhrodip Mohanta <subhrodipmohanta@gmail.com> * iluwatar#1329 Update claim-check-pattern/usage-cost-processor/src/main/java/com/callusage/domain/UsageCostDetail.java Co-authored-by: Subhrodip Mohanta <subhrodipmohanta@gmail.com> * iluwatar#1329 Update claim-check-pattern/usage-cost-processor/src/main/java/com/callusage/domain/UsageDetail.java Co-authored-by: Subhrodip Mohanta <subhrodipmohanta@gmail.com> * iluwatar#1329 deleted mvnw file * iluwatar#1329 deleted mvnw.cmd file * iluwatar#1329 Update claim-check-pattern/usage-detail-sender/src/main/java/com/callusage/domain/Message.java Co-authored-by: Subhrodip Mohanta <subhrodipmohanta@gmail.com> * iluwatar#1329 Update claim-check-pattern/ReadME.md Co-authored-by: Subhrodip Mohanta <subhrodipmohanta@gmail.com> * iluwatar#1329 pom file dependencies fixed, readmeflie updated, removed unused imports * iluwatar#1329 Readfile updated, class javadoc added * iluwatar#1329 UML class diagrams added, readme file updated * iluwatar#1329 lombok annotations used on model classes, common dependencies moved to parent pom * iluwatar#1329 test cases added * include claim-check-pattern in parent pom * iluwatar#1329 code smells fixed * iluwatar#1329 code smells removed * iluwatar#1329 security issues fixed * iluwatar#1329 updated pom files and refactored packages name * iluwatar#1329 checkstyle warning fixed * iluwatar#1329 code coverage increased * iluwatar#1329 minor changed. * checkpoint created with common utility * Claim-Check-Pattern | Shrirang97 | Implemented using Java Azure Functions * Claim-Check-Pattern | Shrirang97 | Updated Functions logic * Update MessageHandlerUtility.java * Update UsageCostProcessorFunction.java * claim-check-pattern | Shrirang97 | Added test cases * claim-check-pattern | Shrirang97 | Test cases for Azure functions fixed * Claim-Check-Pattern | Shrirang | Used string as request body * claim-check-pattern | Shrirang | Working test cases * claim-check-pattern | Shrirang | Issue fixed while deserializing * claim-check-pattern | Shrirang | removed unused import * claim-check-pattern | Shrirang | fixed refactoring * claim-chek-pattern | Shrirang | added lombok | fixed dependencies & test cases * Delete .DS_Store * claim-check-pattern | Shrirang | Fixed unrelated file * Update BookService.java * Update BookService.java * claim-check-pattern | Shrirang | Fixed unrelated files * claim-check-pattern | Shrirang | Fixed review comments * Update UsageCostProcessorFunction.java * Update ReadME.md * Rename ReadME.md to README.md * claim-check-pattern | Shrirang | Incorporated review comments. * Update cloud-claim-check-pattern/README.md Co-authored-by: Ilkka Seppälä <iluwatar@users.noreply.github.com> * Update cloud-claim-check-pattern/README.md Co-authored-by: Ilkka Seppälä <iluwatar@users.noreply.github.com> * Update cloud-claim-check-pattern/README.md Co-authored-by: Ilkka Seppälä <iluwatar@users.noreply.github.com> * Update cloud-claim-check-pattern/README.md Co-authored-by: Ilkka Seppälä <iluwatar@users.noreply.github.com> * Update cloud-claim-check-pattern/README.md Co-authored-by: Ilkka Seppälä <iluwatar@users.noreply.github.com> * Update cloud-claim-check-pattern/README.md Co-authored-by: Ilkka Seppälä <iluwatar@users.noreply.github.com> * Update cloud-claim-check-pattern/README.md Co-authored-by: Ilkka Seppälä <iluwatar@users.noreply.github.com> * Updated readme file * Read me file updated | Added more description Co-authored-by: Subhrodip Mohanta <subhrodipmohanta@gmail.com> Co-authored-by: Subhrodip Mohanta <hello@subho.xyz> Co-authored-by: Ilkka Seppälä <iluwatar@users.noreply.github.com>
1 parent b22c8bc commit 9063336

28 files changed

+2010
-0
lines changed

‎cloud-claim-check-pattern/.gitignore

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# Build output
2+
target/
3+
*.class
4+
5+
# Log file
6+
*.log
7+
8+
# BlueJ files
9+
*.ctxt
10+
11+
# Mobile Tools for Java (J2ME)
12+
.mtj.tmp/
13+
14+
# Package Files #
15+
*.jar
16+
*.war
17+
*.ear
18+
*.zip
19+
*.tar.gz
20+
*.rar
21+
22+
# virtual machine crash logs, see http://www.java.com/en/download/help/error_hotspot.xml
23+
hs_err_pid*
24+
25+
# IDE
26+
.idea/
27+
*.iml
28+
.settings/
29+
.project
30+
.classpath
31+
32+
# macOS
33+
.DS_Store
34+
35+
# Azure Functions
36+
local.settings.json
37+
bin/
38+
obj/

‎cloud-claim-check-pattern/README.md

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
---
2+
layout: pattern
3+
title: Claim Check Pattern
4+
folder: cloud-claim-check-pattern
5+
permalink: /patterns/cloud-claim-check-pattern/
6+
categories: Cloud
7+
tags:
8+
- Cloud distributed
9+
- Microservices
10+
---
11+
12+
## Name
13+
14+
[Claim Check Pattern](https://docs.microsoft.com/en-us/azure/architecture/patterns/claim-check)
15+
16+
## Also known as
17+
18+
[Reference-Based Messaging](https://www.enterpriseintegrationpatterns.com/patterns/messaging/StoreInLibrary.html)
19+
20+
## Intent
21+
22+
- Reduce the load of data transfer through the Internet. Instead of sending actual data directly, just send the message reference.
23+
- Improve data security. As only message reference is shared, no data is exposed to the Internet.
24+
25+
## Explanation
26+
27+
Real-World Example
28+
29+
> Suppose if you want to build a photo processing system. A photo processing system takes an image as input, processes it, and outputs a different set of images. Consider system has one persistent storage, one input component, ten processing components, messaging platform. Once a photo is given to the input component, it stores that image on persistent storage. It then creates ten different messages/events with the same image location and publishes them to the messaging platform. The messaging platform triggers ten different processing components. The ten processing components extract information about image location from the received event and then read an image from persistent storage. They generate ten different images from the original image and drop these images again to persistent storage.
30+
31+
In Plain words
32+
33+
> Split a large message into a claim check and a payload. Send the claim check to the messaging platform and store the payload to an external service. This pattern allows large messages to be processed while protecting the message bus and the client from being overwhelmed or slowed down. This pattern also helps to reduce costs, as storage is usually cheaper than resource units used by the messaging platform.([ref](https://docs.microsoft.com/en-us/azure/architecture/patterns/claim-check))
34+
35+
## Architecture Diagram
36+
37+
![alt text](./etc/Claim-Check-Pattern.png "Claim Check Pattern")
38+
39+
## Applicability
40+
41+
Use the Claim Check Pattern when
42+
43+
- Huge processing data causes a lot of bandwidth consumption to transfer data through the Internet.
44+
- To secure your data transfer by storing in common persistent storage.
45+
- Using a cloud platform - Azure Functions or AWS Lambda, Azure EventGrid or AWS Event Bridge, Azure Blob Storage or AWS S3 Bucket.
46+
- Each service must be independent and idempotent. Output data is dropped to persistent storage by the service.
47+
- Publish-subscribe messaging pattern needs to be used.
48+
49+
## Consequences
50+
51+
- This pattern is stateless. Any compute API will not store any data.
52+
- You must have persistent storage and a reliable messaging platform.
53+
54+
## Tutorials
55+
56+
### Workflow
57+
58+
Suppose a telecom company wants to build call cost calculator system which generate the call cost daily. At the end of each day, details of the calls made by the consumers are stored somewhere. The call calculator system will read this data and generate call cost data for each user. Consumers will be billed using this generated data in case of postpaid service.
59+
60+
Producer class( `UsageDetailPublisherFunction` Azure Function) will generate call usage details (here we are generating call data in producer class itself. In real world scenario, it will read from storage). `UsageDetailPublisherFunction` creates a message. Message consists of message header and message body. Message header is basically an event grid event or claim or message reference. Message body contains actual data. `UsageDetailPublisherFunction` sends a message header to Event Grid topic `usage-detail` and drops an entire message to the blob storage. Event Grid then sent this message header to the `UsageCostProcessorFunction` Azure function. It will read the entire message from blob storage with the help of the header, will calculate call cost and drop the result to the blob storage.
61+
62+
### Class Diagrams
63+
64+
![alt text](./etc/class-diagram.png "Claim-Check-Class-Diagram")
65+
66+
### Setup
67+
68+
- Any operating system can be used macOS, Windows, Linux as everything is deployed on Azure.
69+
- Install Java JDK 11 and set up Java environmental variables.
70+
- Install Git.
71+
- Install Visual Studio Code.
72+
- Install [ Azure Functions extension](https://marketplace.visualstudio.com/items?itemName=ms-azuretools.vscode-azurefunctions) to be able to deploy using Visual studio.
73+
74+
### Storage Data
75+
76+
The data is stored in the Azure blob storage in the container `callusageapp`. For every trigger, one GUID is created. Under the `GUID folder`, 2 files will be created `input.json` and `output.json`.
77+
`Input.json` is dropped `producer` azure function which contains call usage details.` Output.json` contains call cost details which are dropped by the `consumer` azure function.
78+
79+
## Credits
80+
81+
- [Messaging Pattern - Claim Check](https://www.enterpriseintegrationpatterns.com/patterns/messaging/StoreInLibrary.html)
82+
- [Azure Architecture Pattern - Claim Check Pattern](https://docs.microsoft.com/en-us/azure/architecture/patterns/claim-check)

‎cloud-claim-check-pattern/call-usage-app/.gitignore

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# Build output
2+
target/
3+
*.class
4+
5+
# Log file
6+
*.log
7+
8+
# BlueJ files
9+
*.ctxt
10+
11+
# Mobile Tools for Java (J2ME)
12+
.mtj.tmp/
13+
14+
# Package Files #
15+
*.jar
16+
*.war
17+
*.ear
18+
*.zip
19+
*.tar.gz
20+
*.rar
21+
22+
# virtual machine crash logs, see http://www.java.com/en/download/help/error_hotspot.xml
23+
hs_err_pid*
24+
25+
# IDE
26+
.idea/
27+
*.iml
28+
.settings/
29+
.project
30+
.classpath
31+
32+
# macOS
33+
.DS_Store
34+
35+
# Azure Functions
36+
local.settings.json
37+
bin/
38+
obj/

‎cloud-claim-check-pattern/call-usage-app/etc/call-usage-app.urm.puml

Lines changed: 117 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,117 @@
1+
@startuml
2+
package com.iluwatar.claimcheckpattern.producer.calldetails.functions {
3+
class UsageDetailPublisherFunction {
4+
- eventHandlerUtility : EventHandlerUtility<MessageHeader>
5+
- messageHandlerUtility : MessageHandlerUtility<UsageDetail>
6+
+ UsageDetailPublisherFunction()
7+
+ UsageDetailPublisherFunction(messageHandlerUtility : MessageHandlerUtility<UsageDetail>, eventHandlerUtility : EventHandlerUtility<MessageHeader>)
8+
+ run(request : HttpRequestMessage<Optional<String>>, context : ExecutionContext) : HttpResponseMessage
9+
}
10+
}
11+
package com.iluwatar.claimcheckpattern.domain {
12+
class Message<T> {
13+
- messageBody : MessageBody<T>
14+
- messageHeader : MessageHeader
15+
+ Message<T>()
16+
+ getMessageBody() : MessageBody<T>
17+
+ getMessageHeader() : MessageHeader
18+
+ setMessageBody(messageBody : MessageBody<T>)
19+
+ setMessageHeader(messageHeader : MessageHeader)
20+
}
21+
class MessageBody<T> {
22+
- data : List<T>
23+
+ MessageBody<T>()
24+
+ getData() : List<T>
25+
+ setData(data : List<T>)
26+
}
27+
class MessageHeader {
28+
- data : Object
29+
- dataVersion : String
30+
- eventTime : String
31+
- eventType : String
32+
- id : String
33+
- subject : String
34+
- topic : String
35+
+ MessageHeader()
36+
+ getData() : Object
37+
+ getDataVersion() : String
38+
+ getEventTime() : String
39+
+ getEventType() : String
40+
+ getId() : String
41+
+ getSubject() : String
42+
+ getTopic() : String
43+
+ setData(data : Object)
44+
+ setDataVersion(dataVersion : String)
45+
+ setEventTime(eventTime : String)
46+
+ setEventType(eventType : String)
47+
+ setId(id : String)
48+
+ setSubject(subject : String)
49+
+ setTopic(topic : String)
50+
}
51+
class MessageReference {
52+
- dataFileName : String
53+
- dataLocation : String
54+
+ MessageReference()
55+
+ MessageReference(dataLocation : String, dataFileName : String)
56+
+ getDataFileName() : String
57+
+ getDataLocation() : String
58+
+ setDataFileName(dataFileName : String)
59+
+ setDataLocation(dataLocation : String)
60+
}
61+
class UsageCostDetail {
62+
- callCost : double
63+
- dataCost : double
64+
- userId : String
65+
+ UsageCostDetail()
66+
+ getCallCost() : double
67+
+ getDataCost() : double
68+
+ getUserId() : String
69+
+ setCallCost(callCost : double)
70+
+ setDataCost(dataCost : double)
71+
+ setUserId(userId : String)
72+
}
73+
class UsageDetail {
74+
- data : int
75+
- duration : int
76+
- userId : String
77+
+ UsageDetail()
78+
+ getData() : int
79+
+ getDuration() : int
80+
+ getUserId() : String
81+
+ setData(data : int)
82+
+ setDuration(duration : int)
83+
+ setUserId(userId : String)
84+
}
85+
}
86+
package com.iluwatar.claimcheckpattern.utility {
87+
class EventHandlerUtility<T> {
88+
- customEventClient : EventGridPublisherClient<BinaryData>
89+
+ EventHandlerUtility<T>()
90+
+ EventHandlerUtility<T>(customEventClient : EventGridPublisherClient<BinaryData>)
91+
+ publishEvent(customEvent : T, logger : Logger)
92+
}
93+
class MessageHandlerUtility<T> {
94+
- blobServiceClient : BlobServiceClient
95+
+ MessageHandlerUtility<T>()
96+
+ MessageHandlerUtility<T>(blobServiceClient : BlobServiceClient)
97+
+ dropToPersistantStorage(message : Message<T>, logger : Logger)
98+
+ readFromPersistantStorage(messageReference : MessageReference, logger : Logger) : Message<T>
99+
}
100+
}
101+
package com.iluwatar.claimcheckpattern.consumer.callcostprocessor.functions {
102+
class UsageCostProcessorFunction {
103+
- messageHandlerUtilityForUsageCostDetail : MessageHandlerUtility<UsageCostDetail>
104+
- messageHandlerUtilityForUsageDetail : MessageHandlerUtility<UsageDetail>
105+
+ UsageCostProcessorFunction()
106+
+ UsageCostProcessorFunction(messageHandlerUtilityForUsageDetail : MessageHandlerUtility<UsageDetail>, messageHandlerUtilityForUsageCostDetail : MessageHandlerUtility<UsageCostDetail>)
107+
- calculateUsageCostDetails(usageDetailsList : List<UsageDetail>) : List<UsageCostDetail>
108+
+ run(request : HttpRequestMessage<Optional<String>>, context : ExecutionContext) : HttpResponseMessage
109+
}
110+
}
111+
UsageCostProcessorFunction --> "-messageHandlerUtilityForUsageDetail" MessageHandlerUtility
112+
Message --> "-messageBody" MessageBody
113+
UsageDetailPublisherFunction --> "-eventHandlerUtility" EventHandlerUtility
114+
Builder ..+ HttpResponseMessage
115+
UsageDetailPublisherFunction --> "-messageHandlerUtility" MessageHandlerUtility
116+
Message --> "-messageHeader" MessageHeader
117+
@enduml

‎cloud-claim-check-pattern/call-usage-app/host.json

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
{
2+
"version": "2.0",
3+
"extensionBundle": {
4+
"id": "Microsoft.Azure.Functions.ExtensionBundle",
5+
"version": "[1.*, 2.0.0)"
6+
}
7+
}

0 commit comments

Comments
 (0)