Payload Parser*

*Hang tight — we're still working on it!

Overview

The Payload Parser is a crucial component in IoT device integration, as sensors or external data sources transmit data in various formats such as binary, hexadecimal, JSON, and others. Its primary function is to decode raw data payloads and convert them into a Valeiot Data Point. This ensures that data from any sensor or external component can be easily interpreted, stored, and utilized within the platform as illustrated in the diagram below.

The Payload Parser is a JavaScript script that processes the data from the sensors and converts it into human-readable values. The script follows the protocol specifications provided in the manufacturer's harware documentation, to accurately decode the data fields.

Key Characteristics

• Supported Language: Javascript
• Enrionment variables: Hang tight — we're still working on it!
• Built-in libraries: Hang tight — we're still working on it!

Use Cases

• Data Decoding: The Payload Parser decodes the raw data payloads transmitted by sensors or external data sources. This includes converting binary, hexadecimal data into meaningful values such as latitude, longitude, battery voltage, speed, and other device statistics.
• Valeiot Data Point Normalization: Adjusting customized JSON to conform to the Valeiot Data Point structure, ensuring successful insertion into Data Sources.
• Flexibility: Handling various types of uplink messages and can be extended for custom processing before insertion into Data Sources.
• Error Handling: Handling errors or unexpected data formats, ensuring robust operation even in cases of corrupted or incomplete data.
• Customization: The parser can be customized to meet specific application requirements, such as filtering out certain data points or adding additional metadata to the decoded data.

Payload Parser Structure

The Payload Parser has a mandatory entry function called parse, which is executed whenever a request is sent. It should always follow the structure below, with its parameter being the payload, which contains the message sent in the request. Finally, it must return an array of Valeiot Data Points.

function parse(payload) {
  let dataList = [];

  //your logic goes here

  return dataList;
}

Example

Below is an example of a JavaScript-based Payload Parser, which decodes the payload sent from the device and converts it into Valeiot Data Points:

parserExample.js

function parse(payload) {
  let dataList = [];

  if (typeof payload === "string") {
    const buffer = hexToUint8Array(payload);

    if (buffer[0]) {
      const batteryValue = buffer[0];
      dataList.push({
        variable: "battery",
        value: batteryValue,
      });
    }

    if (buffer[1]) {
      const temperatureValue = buffer[1];
      dataList.push({
        variable: "temperature",
        value: temperatureValue,
      });
    }

    if (buffer[3]) {
      const doorBit = buffer[3] & 0x08;
      let doorState = "";
      if (doorBit > 0x00) {
        doorState = "Closed";
      } else {
        doorState = "Open";
      }
      dataList.push({
        variable: "door_state",
        value: doorState,
      });
    }

    dataList = dataList.concat({ variable: "payload", value: payload });
  } else {
    dataList = dataList.concat(payload);
  }

  const ignoreVars = ["snr", "rssi", "gateway_eui", "fcnt", "fport"];
  dataList = dataList.filter((x) => !ignoreVars.includes(x.variable));

  return dataList;
}

The parser expects a payload as input, using the following example of a payload:

"621b6310";

The expected output is the result of data points:

[
  { variable: "battery", value: 98 },
  { variable: "temperature", value: 27 },
  { variable: "position_state", value: "Open" },
  { variable: "payload", value: "621b6310" },
];

Explanation

The decoding process is free, you can do basically whatever you want with javascript. For the example mentioned, the script decodes the battery, temperature and door state from the payload. It gets the equivalent from hexidacimal to decimal and perform a simple bitmask. It's also trimming a few unwanted variables such as snr, rssi, gateway_eui, fcnt and fport.

function parse(payload) {
  let dataList = [];

  if (typeof payload === "string") {
    const buffer = hexToUint8Array(payload);

    if (buffer[0]) {
      const batteryValue = buffer[0];
      dataList.push({
        variable: "battery",
        value: batteryValue,
      });
    }

    if (buffer[1]) {
      const temperatureValue = buffer[1];
      dataList.push({
        variable: "temperature",
        value: temperatureValue,
      });
    }

    if (buffer[3]) {
      const doorBit = buffer[3] & 0x08;
      let doorState = "";
      if (doorBit > 0x00) {
        doorState = "Closed";
      } else {
        doorState = "Open";
      }
      dataList.push({
        variable: "door_state",
        value: doorState,
      });
    }

    dataList = dataList.concat({ variable: "payload", value: payload });
  } else {
    dataList = dataList.concat(payload);
  }

  const ignoreVars = ["snr", "rssi", "gateway_eui", "fcnt", "fport"];
  dataList = dataList.filter((x) => !ignoreVars.includes(x.variable));

  return dataList;
}

The calculator below shows hexadecimal and binary values.

Battery: 0110 0010 or 0x62 or 98

Temperature: 0001 1011 or 0x1B or 27

Door Status: 0001 0000 which is 0x10 when operating 0x10 & 0x08 the result is 0x00

  00010000   (0x10)
& 00001000   (0x08)
  --------
  00000000   (Result)

This example demonstrates how the Payload Parser can be used to decode data from the device and convert it into Valeiot Data Point format.