Unit Testing JSONata Expressions in Notehub with JavaScript and Jest

Testing. Nobody actually wants to do it. The act of writing tests is tedious, time-consuming, and rarely provides value. Right?

Except of course when you deploy to production on a Friday afternoon, expose a regression that nobody ever thought could possibly happen, and then spend your weekend picking up the pieces! (Not speaking from experience of course...)

Unit testing has become an essential practice for ensuring code quality and robustness, especially in environments where multiple contributors may be editing the same codebase.

This article will help guide you through the process of unit testing JSONata expressions with Jest and VS Code, and even how to test expressions and Events directly from your Notehub project.

A Brief Intro to JSONata and Jest

JSONata is a query and transformation language for JSON, allowing developers to extract, manipulate, and transform JSON data. It is particularly useful in scenarios like tranforming data sent to a Notehub Route (e.g. transforming JSON to reduce the size of the payload, conforming to a remote endpoint's requirements, and/or performing dynamic calculations) before it is then delivered to a cloud application.

JSONata offers powerful capabilities for handling JSON data, but like anything else, its expressions and functions should be tested thoroughly.

note

You can read more about writing JSONata expressions in our guide on Using JSONata to Transform JSON.

Jest is a popular testing framework for JavaScript that provides a rich API for writing unit, integration, and end-to-end tests. It is highly favored in the JavaScript community for its simplicity, speed, and mocking capabilities.

Combining JSONata with Jest allows us to write automated tests to ensure that our JSON data manipulations and transformations work correctly.

Let's look at how we can build some tests of our own with Jest and VS Code.

Setting Up Our Environment

Before we begin writing tests, we need to set up our dev environment:

Installing Node.js and npm

First, ensure that you have Node.js and npm (Node Package Manager) installed. You can download the installer from the official Node.js website.

To check if you already have Node.js and npm installed, run the following commands in your terminal:

node -v
npm -v

If both commands return a version number, you are good to go!

Setting Up a New Project

Navigate to an arbitrary directory and initialize a new Node.js project:

mkdir jsonata-jest-testing
cd jsonata-jest-testing
npm init -y

Next, install Jest as a development dependency:

npm install --save-dev jest

Then, install JSONata:

npm install jsonata

Create a Basic Script for Testing

Let's create a relatively simple script to get an idea of how Jest works. Start by creating a new directory called src and adding a file called transform.js:

mkdir src
touch src/transform.js

In transform.js, write a function that will accept an expression (i.e. a JSONata expression we want to test) and an input (a stringified JSON object we want to test):

async function transformData(expression, input) {

  const parsedExpression = jsonata(expression);
  const parsedInput = JSON.parse(input);

  let result = await parsedExpression.evaluate(parsedInput);
  
  // convert null to [] for Jest
  result = result ?? [];
  
  // if it's an array, remove any custom properties (like 'sequence') that jsonata may add
  if (Array.isArray(result)) {
    result = JSON.parse(JSON.stringify(result));
  }
  
  return result;
}

Writing Unit Tests for JSONata with Jest

Now, let's write some unit tests and supply some static expression and input values.

Create a new directory named __tests__ and add a file called transform.test.js:

mkdir __tests__
touch __tests__/transform.test.js

When writing unit tests, it's important we follow a clear structure:

• Describe Blocks: Group related tests together using describe.
• Test Cases: Use test or it to define individual test cases.
• Assertions: Use Jest's expect function to check if the result matches the expected output.

In transform.test.js, we'll start by creating two tests with hardcoded expression and input values:

const transformData = require('../src/transform');

describe('JSONata Transformation Tests', () => {
  
  // ###################################################################
  // test that validates a specific element always returns a value range
  // ###################################################################

  test('should return true as temperature is always > 20', async () => {

    const expression = "$.body.temperature > 20";

    const input = `
    {
      "body": {
        "temperature": 25
      }
    }`;

    const result = await transformData(expression, input);
    expect(result).toEqual(true);

  });

  // #################################################
  // test that validates the format of the json object
  // #################################################

  test('should correctly return formatted json regardless of values', async () => {
    const expression = "$.body.temperature > 20";

    const input = `
    {
      "body": {
        "temperature": 25
      }
    }`;

    const result = await transformData(expression, input);
    expect(result).toEqual(true);

  });

});

What are these two tests actually testing? They demonstrate how you might want to:

1. Test that one or more specific properties of your JSON contains a certain value or range of values.
2. Test that your JSON is transformed into a format that you expect.

Configuring VS Code

Before we run these tests, there are a few options you have in terms of how and when the tests are run.

To ensure a seamless development experience, I recommend installing the following VS Code extensions:

• Jest: Provides syntax highlighting and automatic test running.
• ESLint: Helps maintain consistent code style.

Option 1: Run on Save

Add the following setting to your settings.json file in VS Code. This can be applied on a user or workspace level, it's up to you.

This configuration will automatically run tests every time you save a file:

{
  "jest.autoRun": {
    "watch": true,
    "onSave": "test-file"
  }
}

Option 2: Manually Initiate Test Runs

Update your package.json to include the following:

"scripts": {
  "test": "jest"
}

Then, manually run the tests by executing:

npm test

Test Results

No matter how you set it up, when you run the tests they should pass with flying colors:

PASS  __tests__/transform.test.js

Test Suites: 2 passed, 2 total

Testing JSONata Expressions in Notehub

Building static tests is one thing, but in the real world we work with dynamic data and JSONata expressions that could break our device-to-cloud dataflow with Notecard and Notehub.

Luckily, using the Notehub API we can:

1. Get a JSONata expression from a Route.
2. Get one or more Events from a Notehub project.
3. Programmatically test the expression against the Event.

Let's extend our existing Node.js project to utilize the Notehub API.

First, install Axios, which will let us make HTTP calls to the Notehub API:

npm install axios

Next, create a new file in the __tests__ directory specific for our Notehub tests:

touch __tests__/notehub.test.js

In this new notehub.test.js file, add some require statements:

const axios = require('axios');
const transformData = require('../src/transform');

Next, create a function that will get a new auth token from Notehub, based on the client_id and client_secret provided in your Notehub project's settings:

async function getNotehubAuthToken() {
  const tokenUrl = 'https://notehub.io/oauth2/token';

  const params = new URLSearchParams({
    grant_type: 'client_credentials',
    client_id: 'your-client-id',
    client_secret: 'your-client-secret',
  });

  const response = await axios.post(tokenUrl, params, {
    headers: {
      'content-type': 'application/x-www-form-urlencoded'
    }
  });

  return response.data.access_token;
}

The next step is to create a function that will return a JSONata expression from a known Route in your Notehub project.

async function getJSONAtaExpression() {
  try {
    const accessToken = await getNotehubAuthToken();
    const projectUID = 'app:9a442885-d04d-4f5d....';
    const routeUID = 'route:214a8c8c25b2647....';
    const url = `https://api.notefile.net/v1/projects/${projectUID}/routes/${routeUID}`;

    const response = await axios.get(url, {
      headers: {
        Authorization: `Bearer ${accessToken}`
      }
    });

    return String(response.data.http.transform.jsonata);

  } catch (error) {
    console.log(error);
  }
}

note

You can get the required projectUID from your Notehub project's settings or by using the Get Projects API.

Likewise, you can get the routeUID from the URL when you navigate to edit your Route in the Notehub UI or by using the Get Routes API.

Next, we'll want to test against one or more Events from a Notehub project. The following function simply gets the most recent Event. This may or may not be what you want, but you can consult the Event API documentation to customize this API call for your needs.

async function getLatestEvent() {
  try {
    const accessToken = await getNotehubAuthToken();
    const projectUID = 'app:9a442885-d04d-4f5d....';
    const url = `https://api.notefile.net/v1/projects/${projectUID}/events`;

    const response = await axios.get(url, {
      headers: {
        Authorization: `Bearer ${accessToken}`
      }
    });

    return JSON.stringify(response.data.events[0].body);

  } catch (error) {
    console.log(error);
  }
}

Lastly, this function runs a test on the specified Event, using the JSONata expression pulled from the Route:

async function testLatestEvent() {

  describe('JSONata Transformation Tests', () => {

    test('should correctly return a voltage regardless of value', async () => {
    
      const expression = await getJSONAtaExpression();
      const input = await getLatestEvent();
      const result = await transformData(expression, input);
  
      expect(result).toEqual(
        expect.objectContaining({
          voltage: expect.anything()
        })
      );
    
    });
  });
}

testLatestEvent();

Save this file and run your tests again! Hopefully they pass...

PASS  __tests__/transform.test.js
PASS  __tests__/notehub.test.js

Test Suites: 2 passed, 2 total
Tests:       3 passed, 3 total
Snapshots:   0 total
Time:        1.676 s, estimated 2 s
Ran all test suites.

Best Practices for Unit Testing JSONata

As your projects scale in Notehub, it's critical to ensure your JSONata expressions are covered with unit tests. Consider the following best practices when testing Notehub Events:

• Cover Edge Cases: Write tests for edge cases, such as missing elements from an Event or data that is outside the range you expect.
• Use Meaningful Test Data: Use test Events from Notehub that reflect real-world scenarios from your devices to ensure your tests are realistic and effective.
• Isolate Tests: Don't just copy my spaghetti code above! Each test you write should be independent, focusing on individual aspects of your JSONata expression and the Event(s) you are testing.

Conclusion

Unit testing JSONata expressions from Notehub with Jest and VS Code provides a reasonably easy way to ensure your data transformations are accurate and reliable. By following the steps outlined in this article, you can set up your VS Code environment, write comprehensive tests, and deploy with confidence.

Happy Hacking! 💙