How to communicate JSON data between Python and Node.js ?

The following article covers how to communicate JSON data between Python and Node.js. Suppose we are working with the Node.js application, and we want to make use of a specific library that is only available in python or vice versa. We should be able to share the results from one language to another and to achieve it, we will use the JSON as it is language independent.

Approach:

1. Set up a server for each language and share data using JSON using the good old GET and POST request.
2. Call a python background process from Node.js or vice versa and listen to the stdout stream of the process in both instances.

Project Structure: All the files used below are present in the same directory as shown below.

1. Using Servers: This is similar to the approach of using a third-party API service wherein we make a GET request to the remote server to get the data and a POST request to send the data. The only difference is we will be running the servers locally (This would also work on the remote servers with the required URL).

Node.js to Python: When we are working in node.js and want to process some data in python.

In the following example, we will set up a server for Python and make requests from node.js. We are using Flask micro framework as this is the simplest way to set up a server in Python and to make requests in Node.js we will need a request package.

Module Installation:

• Install the flask module for the Python using the following command:

  pip install flask

• Install the request module for the NodeJS using the following command:

  npm install request-promise

Example: Calculate the sum of an array containing integers and return the result back to Node.js

from flask import Flask, request
import json 
 
# Setup flask server
app = Flask(__name__) 

# Setup url route which will calculate
# total sum of array.
@app.route('/arraysum', methods = ['POST']) 
def sum_of_array(): 
    data = request.get_json() 
    print(data)

    # Data variable contains the 
    # data from the node server
    ls = data['array'] 
    result = sum(ls) # calculate the sum

    # Return data in json format 
    return json.dumps({"result":result})
 
if __name__ == "__main__": 
    app.run(port=5000)

Run the server using the following command.

python pyserver.py

This will start the server at http://127.0.0.1:5000/. Now we make a POST request from Node.js to http://127.0.0.1:5000/arraysum

var request = require('request-promise');

async function arraysum() {

    // This variable contains the data
    // you want to send 
    var data = {
        array: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
    }

    var options = {
        method: 'POST',

        // http:flaskserverurl:port/route
        uri: 'http://127.0.0.1:5000/arraysum',
        body: data,

        // Automatically stringifies
        // the body to JSON 
        json: true
    };

    var sendrequest = await request(options)

        // The parsedBody contains the data
        // sent back from the Flask server 
        .then(function (parsedBody) {
            console.log(parsedBody);
            
            // You can do something with
            // returned data
            let result;
            result = parsedBody['result'];
            console.log("Sum of Array from Python: ", result);
        })
        .catch(function (err) {
            console.log(err);
        });
}

arraysum();

Run this script by the following command.

node talk.js

Output:

{ result: 55 }
Sum of Array from Python:  55

Python to Node.js: When we are working in python and want to process some data in Node.js.

Here we will reverse the above process and use express to fire up the server in node.js and request package in python. 

Module Installation:

• Install the request module for the Python using the following command:

  pip install requests

• Install the express and body-parser module for the NodeJS using the following command:

  npm install express
  npm install body-parser

var express = require('express');
var bodyParser = require('body-parser');

var app = express();

app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: false }));

app.post("/arraysum", (req, res) => {

    // Retrieve array form post body
    var array = req.body.array;  
    console.log(array);

    // Calculate sum
    var sum = 0;
    for (var i = 0; i < array.length; i++) {
        if (isNaN(array[i])) {
            continue;
        }
        sum += array[i];
    }
    console.log(sum);

    // Return json response
    res.json({ result: sum });
});

// Server listening to PORT 3000
app.listen(3000);

Run the server using the following command.

node nodeserver.js

This starts the server at http://127.0.0.1:3000/. Now we make a POST request from Python to 127.0.0.1:3000/arraysum 

import requests

# Sample array
array = [1,2,3,4,5,6,7,8,9,10]

# Data that we will send in post request.
data = {'array':array}

# The POST request to our node server
res = requests.post('http://127.0.0.1:3000/arraysum', json=data) 

# Convert response data to json
returned_data = res.json() 

print(returned_data)
result = returned_data['result'] 
print("Sum of Array from Node.js:", result)

Run this script by the following command.

python talk.py 

Output:

{'result': 55}
Sum of Array from Node.js: 55

2. Using background processes: In the following example, we will communicate by spawning a Python process from Node.js and vice versa and listen to the stdout stream.

Node.js to Python: Calling the python process from node.js. It involves the following steps:

1. Calling python process and pass JSON data as a command-line argument.
2. Read that data in python, process it, and output it to stdout stream in JSON format.
3. Again from node.js read the output stream and process the JSON data.

import sys, json

# Function to calculate the sum of array
def arraysum(arr):
    return sum(arr)

# Get the command line arguments
# and parse it to json
data = json.loads(sys.argv[1])

# Get the required field from
# the data
array = data['array']

# Calculate the result
result = arraysum(array)

# Print the data in stringified
# json format so that we can
# easily parse it in Node.js
newdata = {'sum':result}
print(json.dumps(newdata))

Now python will process the sum of the array and print it to stdout as shown in the below code.

const spawn = require('child_process').spawn;

// Initialise the data
const data = {
  array: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
}

// We need to stringify the data as 
// python cannot directly read JSON
// as command line argument.
let stringifiedData = JSON.stringify(data);

// Call the python process and pass the
// data as command line argument.
const py = spawn('python', ['arraysum.py', stringifiedData]);

resultString = '';

// As the stdout data stream is chunked,
// we need to concat all the chunks.
py.stdout.on('data', function (stdData) {
  resultString += stdData.toString();
});

py.stdout.on('end', function () {

  // Parse the string as JSON when stdout
  // data stream ends
  let resultData = JSON.parse(resultString);

  let sum = resultData['sum'];
  console.log('Sum of array from Python process =', sum);
});

Run this script by the following command:

node caller.js

Output:

Sum of array from Python process = 55

Python to Node.js: Calling node.js process from python. The steps essentially remain the same as mentioned above with python and node.js interchanging their roles.

// Function to calculate sum of array
function arraysum(arr) {
  let sum = 0;
  for (var i = 0; i < arr.length; i++) {
    if (isNaN(arr[i])) {
      continue;
    }
    sum += arr[i];
  }
  return sum;
}

// Get the command line arguments and
// parse it to json
var data = JSON.parse(process.argv[2]);

// Get the required field form the data.
array = data['array'];

// Calculate the result.
var sum = arraysum(array);

// Print the data in stringified json
// format so that we can easily parse
// it in Python
const newData = { sum }
console.log(JSON.stringify(newData));

Now run this Node.js process from python.

Filename: caller.py

from subprocess import Popen, PIPE
import json

# Initialise the data
array = [1,2,3,4,5,6,7,8,9,10]
data = {'array':array}

# Stringify the data.
stingified_data = json.dumps(data)

# Call the node process and pass the
# data as command line argument
process = Popen(['node', 'arraysum.js', 
        stingified_data], stdout=PIPE)

# This line essentially waits for the 
# node process to complete and then
# read stdout data
stdout = process.communicate()[0]

# The stdout is a bytes string, you can
# convert it to another encoding but
# json.loads() supports bytes string
# so we aren't converting

# Parse the data into json
result_data = json.loads(stdout)
array_sum = result_data['sum']
print('Sum of array from Node.js process =',array_sum)

Run this script by the following command.

python caller.py

Output:

Sum of array from Node.js process = 55