Scribd Logo
Upload

Welcome to Scribd!

  • 38 views

    09 Green Deal Data Space (GDDS) : Infrastructure Components

    Uploaded by

    MD.Razaul Haque Subho

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    09 Green Deal Data Space (GDDS) : Infrastructure Components

    Uploaded by

    MD.Razaul Haque Subho
    38 views10 pages

    Original Title

    A2-102268947-140824-0938-15

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    38 views10 pages

    09 Green Deal Data Space (GDDS) : Infrastructure Components

    Uploaded by

    MD.Razaul Haque Subho

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    You are on page 1of 10

    09 Green Deal Data Space (GDDS)

    Infrastructure components
    Cloning repositories
    Starting the frontend
    Software Requirements
    Starting the backend
    Software Requirements
    Ports diagram
    Connecting the two Connectors with the Catalog
    Transfer process
    Database Schema (draft, please improve)
    Links
    Video courses
    Debugging
    Frontend
    Backend

    German version: https://wiki.advaneo.info/pages/viewpage.action?spaceKey=A2&title=Einrichtung+Entwicklungsumgebung

    Infrastructure components

    Cloning repositories
    For the development of the backend and frontend of this connector setup, we will start the following in our IDE:

    two Connector instances with their UI and backends


    A catalogue

    Please clone (cloning explanation) the following Repositories :

    1. Repository edc-connector-demo (https://repo.advaneo.net/development/edc/edc-connector-demo), Branch dev


    1. for the setup of the docker environment (see docker-compose.yml) and
    2. the configuration of the vaults for the database connections
    3. it generates the development environment
    2. Repository advaneo-edc-connector (https://repo.advaneo.net/development/edc/advaneo-edc-connector), Branch dev
    1. for the setup of the two Connectors (one will be used as a data source and the other as a data sink)
    3. Repository advaneo-edc-catalog (https://repo.advaneo.net/development/edc/advaneo-edc-catalog.git), Branch dev
    1. for a catalog of the assets of the different data providers
    4. Repository advaneo-edc-ui (https://repo.advaneo.net/development/edc/advaneo-edc-ui.git), Branch dev
    1. for the development of the frontend

    Annotation: This is the link of our actual edc repositories: https://repo.advaneo.net/development/edc


    Starting the docker-compose.yml in the edc-connector-demo ("project path"\edc-connector-demo>docker-compose up -d) will start the following:

    two Connectors (Docker-Images built via Gitlab)


    two HashiCorp-Vaults (Dev-Modus)
    two PostgreSQL-databases
    data source (Webserver, NGINX)
    data sink (Spring-Boot-App, lokales Projekt)
    Keycloak (25.0.0)
    Federated Catalog (0.7.0)

    Starting the frontend


    We want to start the two Front Ends (in green). All the other parts are running in the docker.

    Software Requirements
    You can download:

    Node JS here: https://nodejs.org/en/download/prebuilt-installer


    .Net here https://dotnet.microsoft.com/en-us/download/dotnet/thank-you/sdk-8.0.303-windows-x64-installer
    Visual Studio Code here https://code.visualstudio.com/docs/?dv=win64user

    Now open the project "advaneo-edc-ui" in Visual Studio Code.

    We want to start two instances of the frontend therefore we define run scripts in "package.json"

    "start1": "ng serve --proxy-config ./src/proxy.conf.json --port 4201",

    "start2": "ng serve --proxy-config ./src/proxy.conf2.json --port 4202",

    Through those we can run each front end with:

    npm run start1 and

    npm run start2

    Now we needed to define a second "proxy.configuration.json" file.


    Here we adapted the configuration of the management and the public endpoints:

    http://localhost:19193 => http://localhost:29193


    http://localhost:19195 => http://localhost:29195

    To start the two instances we can now start the two scripts in the project folder in a command line.

    For the first User Interface, it should look like this.

    Now you can access the UI via HTTP://localhost:4201/

    Starting the backend


    We want to start the backends of the two connectors. Everything else should by now be running.
    Software Requirements
    Intellij for the backend development
    pgAdmin if we want to change the databases
    Docker Desktop to verify the status of the instances running in the docker environment.

    The setup guides for the necessary software can be found here: Tools

    We first need to stop the running instances of our backends in the docker container:

    You can check now the status in your docker-desktop:

    Here you can see that connector1 and connector2 are stopped (exited) and the two databases (postgresql1 and postgresql2) are still running in
    the docker environment.

    Edit the run configuration to create a configuration for each connector instance

    A ProviderConnector and a CosumerConnector with:


    EDC_FS_CONFIG=./config/configuration/provider-configuration.properties
    EDC_FS_CONFIG=./config/configuration/consumer-configuration.properties

    Ports diagram
    By now the project should be configured like this diagram.

    The diagram can be opened in draw.io

    Connecting the two Connectors with the Catalog


    We now want to replace the Catalog running in the docker with one running in our IDE to be able to develop it and debug the communication
    between the two connectors.

    Therefore we run the catalog project in our IDE.

    The Catalog in the docker (8184) is listening on another port than the Catalog of our project (7184).

    As both connectors are registering themselves on the port on which the Catalog in the docker is running we need to change that port.

    edc.catalog.registration.url=http://localhost:8184/registry

    edc.catalog.registration.url=http://localhost:7184/registry

    If both connectors are registered on the right port we can see these two lines in the logging of the catalog:

    Transfer process
    Now we can step by step go through the transfer process

    We create a TestAsset

    We create a TestPolicy
    We create a TestOffer using the TestAsset and the TestPolicy

    Now our offer is created.

    In regular intervals, the catalog is scanning through all registered connectors. He will list these offers so consumers can request them.

    On the consumer side, we will now see this new Asset being listed.

    We now need to request the data of this Asset.


    It will now appear in the REQUESTED state.

    The state machine from the EDC Framework will now try to advance the different Assets from the REQUESTED state step by step to the
    TERMINATED step.

    We can now transfer this contract which makes it appear in the transfer part.

    Database Schema (draft, please improve)


    Links
    Data flow diagrams can be found here:

    https://github.com/eclipse-edc/Connector/blob/main/docs/developer/diagrams/transfer-data-plane-consumer-pull.png

    Video courses
    EDC

    Debugging

    Frontend
    400 Bad Request

    Solution
    A restart of database and backends solves the problem.
    Explanation
    The cross-origin request is a direct, internal access from a website on server A to a second server B, which may be in a different domain.

    request was malformed


    EdcConnectorClientError(BadRequest): request was malformed: [{"message":"https://w3id.org/edc/v0.0.1/ns/policy/http://www.w3.
    org/ns/odrl/2/permission/http://www.w3.org/ns/odrl/2/action is mandatory but missing or null","type":"ValidationFailure","path":"http://www.w3.
    org/ns/odrl/2/action","invalidValue":null},{"message":"

    Solution

    Backend

    500 (Server Error)


    POST http://localhost:4201/management/v3/assets/request? 500 (Server Error)

    Solution
    Is your server running?

    If not, restart it

    Solution

    Stop and delete connectors and databases

    Delete the volumes of the databases

    Start the backend and then the frontend again.

    You might also like