Test Data Generator for EPCIS 2.0 Events and JSON-LD

Quick links

Please find below the links to the OpenEPCIS tools and codes referenced on this documentation page. To enhance your understanding and gain practical experience with these tools, we recommend visiting and utilizing them:

• Test data generator web application : https://test-data-generator.openepcis.io/

• Test data generator API endpoint : https://test-data-generator.openepcis.io/q/swagger-ui

• Test data generator application code : https://github.com/openepcis/epcis-testdata-generator

Introduction

For various business requirements, including traceability (of goods, assets, etc.), process automation, compliance, and supply chain management, you and your trading partners must capture and share visibility data in addition to master data (such as the description and weight of a trade item) and transactional data (such as an order, invoice, etc.). This data category’s fundamental and open-source standard, EPCIS has been already widely used across a number of industries.

The open standard for capturing and exchanging visibility data is EPCIS. It has been endorsed by GS1, ISO, and IEC. It includes a ready-to-use data model that enables you to consolidate every stage of a business process into a single dataset. The basic details of what happened, when, where, why, and how are recorded as EPCIS events. The key to making your items traceable is to keep track of all pertinent events (such as harvesting, producing, packing, shipping, and selling) throughout your supply network.

What sets our Testdata-Generator apart is its use of reactive streaming. This helps ensure efficient data flow handling and optimum resource usage across platforms. With this native image, the application delivers reliable performance and platform versatility, a practical choice for applications in cloud-based microservice architectures.

Need for EPCIS Test Data Generator

End users, solution providers, research institutions, and other parties frequently need test data for load/performance tests, PoCs/research projects (for example, to populate test databases), initiating data interchange between business partners, and more. However, creating EPCIS test data can literally take you days of effort and is usually tedious. If you are in such a situation, you’ll be relieved to learn that our new open-source tool OpenEPCIS test data generator. It is now available that can assist you in completing this task in a matter of minutes.

OpenEPCIS test data generator

The initial version of the EPCIS test data generator was developed as part of master thesis with the collaboration of GS1 Germany from June to November 2020. Various users of the EPCIS community were involved in the development process, which enabled the application to be customised according to their needs and requirements. The latest version of the tool has been developed as part of an organisational effort at benelog GmbH & Co. KG with many improvements and more advanced functionality. Anyone interested in adopting or implementing EPCIS can get assistance from the interactive application OpenEPCIS Test Data Generator. It is being offered as an open-source project. This essentially means that anyone is welcome to use the tool.

Regardless of the industry, use case, or application domain, the OpenEPCIS test data generator tool offers a flexible and potent solution for creating test events. The generated events adhere to the most recent EPCIS specification, i.e. EPCIS 2.0. Events are produced in JSON/JSON-LD, but if businesses need EPCIS events in XML (either in accordance with EPCIS 1.2 or the most recent version, i.e., 2.0), there are additional open source tools that can transform them into the required data format.

Key Features

The following are some of the tool’s key characteristics:

• Adheres to the most recent revisions of CBV 2.0 and EPCIS 2.0.

• Supports object/location IDs using both GS1 Digital Link URIs and EPC/EPC Class URIs.

• Generates JSON/JSON-LD events (which in turn can be translated to XML through other tools).

• Contains a randomization technique for elements like timestamps and serial numbers.

• Supports the use of both UUIDs and EPCIS Event Hash IDs to identify EPCIS events.

• Offers functionality for modelling uniquely tailored event sequences.

• Offers copying or exporting of designs or events for later use.

• Supports object/location IDs using both GS1 Digital Link URIs and EPC/EPC Class URIs.

• Generates JSON/JSON-LD events (which in turn can be translated to XML through other tools).

• Contains a randomization technique for elements like timestamps and serial numbers.

• Supports the use of both UUIDs and EPCIS Event Hash IDs to identify EPCIS events.

• Offers functionality for modelling uniquely tailored event sequences.

• Offers copying or exporting of designs or events for later use.

Local set-up

On your local system, the OpenEPCIS Test Data Generator can be set up in various ways. The following section explains how to quickly set up the application using Podman and Docker.

Running with Podman

As a prerequisite, you must install Podman on your system (Podman is an open-source software platform for fast-developing, testing, and deploying applications). Please refer to the following links for further information about Podman and for installation guidelines:
 

Official Podman: https://github.com/containers/podman
Documentation: https://github.com/containers/podman/tree/main/docs
Installation instruction for Windows/macOS: https://podman.io/getting-started/installation
 

Run the following command in a terminal or command prompt after installing Podman on your local machine:

podman run --rm -t --name testdata-generator -p 8080:8080 openepcis/testdata-generator

Running with Docker or Podman

Docker has to be installed on your system as a prerequisite (Note: Commercial usage of Docker is subject to license restrictions). Please refer to the following pages for further information about Docker and for installation guidelines:
 

For Windows: https://docs.docker.com/desktop/install/windows-install/
For macOS: https://docs.docker.com/desktop/install/mac-install/
For Linux: https://docs.docker.com/desktop/install/linux-install/
 

To run the testdata generator you can simply use the Testdata Generator Container Image image provided on GitHub.

Docker

docker run --rm -t --name testdata-generator -p 8080:8080 ghcr.io/openepcis/testdata-generator:latest

Podman

podman run --rm -t --name testdata-generator -p 8080:8080 ghcr.io/openepcis/testdata-generator:latest

Native Testdata Generator Quarkus REST App Artifacts

The service runtime is also available as Quarkus Native Build. You can find the latest version of the native binaries on the GitHub release page https://github.com/openepcis/epcis-testdata-generator/releases/.

Below are the available Quarkus Native build artifacts for our latest release. These have been optimized for different platforms:

Replace ${VERSION} with the version number related to the release.

These artifacts are created using the Quarkus framework showcasing its extraordinary capabilities in building natively light, exceptionally fast, and truly container-native Java applications. Quarkus native builds significantly optimizes memory consumption and start-up time, setting a new coolness-standard for cloud-based microservices where resource efficiency is key!

A generic .jar file is also available: testdata-generator-quarkus-rest-app-.jar

Our multi-architecture native container image, available on GitHub https://github.com/openepcis/epcis-testdata-generator/pkgs/container/testdata-generator-native, effectively optimizes the deployment of our Testdata-Generator application. Utilizing the Quarkus Java framework, it enables faster, more efficient operations.

Usage

Following section provides quick overview of how to generate EPCIS events:

Web application

The web application is beneficial to users who are new to the EPCIS world as it visually provides all the components of the EPCIS which are segregated based on different dimensions. It is very easy to provide values for necessary fields based on the requirement or supply chain and generate the EPCIS events accordingly. The web application can be fround from here.

API endpoint

Software developers wishing to explore the tool’s underlying APIs (e.g. to understand how a request and response body of an API call looks like) might be interested in the Swagger-UI page. Swagger-UI is an open source application that generates a web page which lists all the APIs according to the OpenAPI standard. The page gives a summary and visually illustrates various features of the APIs such as their request and response bodies. If you have installed and executed a Podman container as described in the set-up section, you should be able to access it via the URL http://localhost:8080/q/swagger-ui/. You can start making requests to the Test Data Generator service API by using an input template. Of course, you can also use the Swagger-UI page to generate test data (there is no limit as with the web application), but it is not as performant as using the command line.

Command Line

Especially in cases where you need large amounts of test data (i.e. more than 1,000 events), open your command line or terminal and issue a cURL (Client URL) request. If you followed the instructions for setting up the Podman container as described in the set-up section, you should be able to make a cURL request using your input template to API endpoint: http://localhost:8080/api/generateTestData?pretty=true. An exemplary cURL request with a sample InputTemplate (which you can replace with your own) looks as follows:

curl -X 'POST' \
  'http://localhost:8080/api/generateTestData?pretty=true' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
    "events": [{
        "nodeId": 1,
        "eventType": "ObjectEvent",
        "eventCount": 10,
        "locationPartyIdentifierSyntax": "URN",
        "ordinaryEvent": true,
        "action": "ADD",
        "eventID": false,
        "eventTime": {
            "timeZoneOffset": "+02:00",
            "fromTime": "2022-10-01T10:22:16+02:00",
            "toTime": "2022-10-31T10:22:16+02:00"
        },
        "businessStep": "COMMISSIONING",
        "disposition": "ACTIVE",
        "referencedIdentifier": [{
            "identifierId": 1,
            "epcCount": 10,
            "classCount": 0
        }],
        "parentReferencedIdentifier": {},
        "outputReferencedIdentifier": []
    }],
    "identifiers": [{
        "identifierId": 1,
        "objectIdentifierSyntax": "URN",
        "instanceData": {
            "sgtin": {
                "identifierType": "sgtin",
                "gcpLength": 10,
                "sgtin": "40584954485984",
                "serialType": "random",
                "randomCount": 10,
                "randomType": "NUMERIC",
                "randomMinLength": 2,
                "randomMaxLength": 10
            }
        },
        "classData": null,
        "parentData": null
    }]
}'

Application code

The developed OpenEPCIS Test Data Generator application has been divided into 3 modules testdata-generator-common, testdata-generator-rest-api, and testdata-generator-ui. Following is the overview of the individual module:

testdata-generator-common (Generic java library for generating testdata events)

This is the core of the OpenEPCIS Test Data Generator where the events are actually generated. It includes the logic for generating the events and formatting the values according to the EPCIS standard. To make the application memory efficient and to generate a large number of test events quickly only one event information is stored in memory at a time. It employs the Reactive Stream approach to do the same. This module has been developed primarily using Java with some additional libraries such as Jackson, Lombok, SmallRye Mutiny etc.

testdata-generator-rest-api (REST service wrapper for generating testdata events)

This is the service that acts as a bridge between the front-end and back-end server. When a user provides the input information in the form of InputTemplate to generate the test data events then this service module will capture that information and validates it. If the InputTemplate adheres to all the constraints and rules then it’s sent to the backend server (testdata-generator-common) to generate the events. This service can also be directly accessed via the cURL command or Swagger-UI. This module has been developed using the latest technologies such as Java with the Quarkus framework. However, it also utilizes some libraries such as Jackson, OpenAPI, Hibernate validator, etc.

testdata-generator-ui (user interface for generating testdata events)

This is the user’s view of the application where users can interact with various fields of EPCIS and provide the necessary values. Also, It does the task of converting the user-provided values into the InputTemplate required for the subsequent modules. It has been developed primarily using the technologies such as HTML (HyperText Markup Language), JavaScript library Nuxt.js/Vue.js, and CSS(Cascading Style Sheets). It also includes some important front-end libraries/frameworks such as Drawflow, Bootstrap, Bootstrap icons, CodeMirror, etc.

Direct usage

The EPCISEventGenerator class within testdata-generator-common contains the Java method generate for producing test data events. To generate events, you must have the proper inputTemplate JSON prepared with all necessary data, which you can pass to the following method to obtain the required events:

// Pass the required JSON inputTemplate to generate EPCIS testdata events.
EPCISEventGenerator.generate(inputTemplate).collect().asList().await().indefinitely().forEach(e - > {
    try {
        System.out.println(e.toString());
    } catch (JsonProcessingException ex) {
        throw new CompletionException(ex);
    }
});

Sample inputTemplate:

{
    "events": [{
        "nodeId": 1,
        "eventType": "ObjectEvent",
        "eventCount": 5,
        "locationPartyIdentifierSyntax": "URN",
        "ordinaryEvent": true,
        "action": "ADD",
        "eventID": true,
        "eventTime": {
            "timeZoneOffset": "+02:00",
            "fromTime": "2022-04-01T18:30:04+02:00",
            "toTime": "2022-04-05T18:30:04+02:00"
        },
        "recordTimeType": "CURRENT_TIME",
        "businessStep": "COMMISSIONING",
        "disposition": "ACTIVE",
        "referencedIdentifier": [{
            "identifierId": 1,
            "epcCount": 10,
            "classCount": 5
        }],
        "parentReferencedIdentifier": {},
        "outputReferencedIdentifier": []
    }],
    "identifiers": [{
        "identifierId": 1,
        "objectIdentifierSyntax": "URN",
        "instanceData": {
            "sgtin": {
                "identifierType": "sgtin",
                "gcpLength": 10,
                "sgtin": "89384989388934",
                "serialType": "range",
                "rangeFrom": 100,
                "rangeTo": 110
            }
        },
        "classData": {
            "grai": {
                "identifierType": "grai",
                "gcpLength": 10,
                "quantityType": null,
                "uom": null,
                "serialType": "",
                "grai": "8384783874378",
                "classIdentifiersCount": 5
            }
        },
        "parentData": null
    }]
}