jerni
a framework to build data-driven products from the ground up

Getting Started with jerni

In this tutorial, we will create an app with jerni and MongoDB.

Prerequisites: Before you start, make sure your have node.js (version 12+) and mongodb (version 3.6+) installed on your machine.

Installation

First and foremost, let create a folder for this new project. Open a terminal and run these bash code

mkdir my-first-jerni-project
cd my-first-jerni-project
npm init -y

You need to install jerni and MongoDB projection store packages.

npm i jerni @jerni/store-mongo

For better developer experience, please install jerni-dev as a dev dependency.

npm i -D jerni-dev

To verify if you have installed the packages correctly, open your package.json file, and you should be able to find this snippet (the actual versions may vary).

file: package.json
{
  "dependencies": {
    "@jerni/store-mongo": "^1.0.0",
    "jerni": "^1.0.0"
  },
  "devDependencies": {
    "jerni-dev": "^1.0.0"
  }
}

Set Up API Server

back to table of content

First, we need to create an express app to listen on port 3000. Although express is not required to use jerni, it's the simplest API server to setup.

npm i express

Then create a server.js file at project root directory, with the following content:

file: server.js
const express = require("express");
const app = express();
const port = 3000;
// GET to return the list of all accounts
app.get("/api/accounts", (req, res) => {
  res.send([]);
});
// POST to create a new account
app.post("/api/accounts", express.json(), async (req, res) => {
  const { name, currency } = req.body;
  const id = Math.trunc(Math.random() * 1e11) + 1e12;
  res.send({ name, balance: { [currency]: 0 }, id });
});
app.listen(port, () =>
  console.log(`Example app listening at http://localhost:${port}`),
);

Now start the app by running this bash script in terminal

node server.js

Output

Example app listening at http://localhost:3000

Leaving this process running, in another terminal, let run a quick test to see if it works

GET

curl localhost:3000/api/accounts

Output

[]

POST

curl -XPOST localhost:3000/api/accounts \
     -H"content-type: application/json" \
     -d '{"name":"test", "currency":"USD"}'

Output

{"name":"test","balance":{"USD":0},"id":1014151097134}

Create a journey

back to table of content

We need to declare our journey by specifying where the events store and your mongodb are.

file: journey/index.js
const createJourney = require("jerni");
const { makeStore } = require("@jerni/store-mongo");
const accounts = require("./models/accounts");
module.exports = async function initialize() {
  return createJourney({
    writeTo: process.env.EVENTS_QUEUE_URL,
    stores: [
      await makeStore({
        name: "banks",
        url: process.env.MONGODB_URL,
        dbName: process.env.MONGODB_DBNAME,
        models: [accounts],
      }),
    ],
  });
};
file: journey/models/accounts.js
const { Model } = require("@jerni/store-mongo");
const mapEvents = require("jerni/lib/mapEvents");
module.exports = new Model({
  name: "accounts",
  version: "1",
  transform: mapEvents({
    ACCOUNT_CREATED(event) {
      return {
        insertOne: {
          id: event.payload.id,
          name: event.payload.account_name,
          balance: {
            [event.payload.currency]: 0,
          },
        },
      };
    },
  }),
});

We have declared a journey that will commit events to a server behind the environment key EVENTS_QUEUE_URL and read projected data from a mongodb specified by MONGODB_URL and MONGODB_DBNAME.

Note: we now have 2 different places for data to stay. This is under the effect of CQRS — Command–Query Responsibility Segregation. This architectural pattern optimizes for both write and read operations as each of them can be carried out by a specialized tool. For example, we write data to an append-only queue in-memory (with disk dump) and read from a denormalized mongodb database.

Don't be confused, there is only one source of truth and it's the events queue. The mongodb stores are only projections. Similar to Views in SQL, they are derived from the sequence of events.

To verify if this works, you can run this bash script in your terminal at the project root directory, given you have your local mongodb server listening on its default port

MONGODB_URL=mongodb://localhost:27017 MONGODB_DBNAME=banks npx jerni-dev start ./journey

Output

[ cli ] info: jerni-dev.start({ version: '1.0.0' })
[ cli ] info:  * source file: /path/to/your/my-first-jerni-project/banks/journey
[ cli ] info:  * options { http: 6181, verbose: false, dataPath: 'jerni.db' }
[ cli ] info: checking integrity of data file since last start
[ cli ] info: heq-server is listening on port 6181
[ cli ] info: writing lockfile to .jerni-dev
[ cli ] info: clean start new journey
[ cli ] info: worker ready
[jerni] info: start receiving data
jerni-dev>

You can use Ctrl-C to terminate this process

Note: for the purpose of this tutorial, we don't need to worry about the value of EVENTS_QUEUE_URL because jerni-dev will handle the events queue for us. It will store committed events in a human-readable file (default to jerni.db) and automatically inject the server address into your application.

Commit an Event

back to table of content

Nothing fancy yet, now let's initialize jerni to our server by invoking initialize() function we exported above. That function returns a Promise that will eventually resolve to a journey object.

file: server.js
/* ... */
const initialize = require("./journey");
initialize().then((journey) => {
  console.log("journey is ready!");
  
  app.get("/api/accounts", (req, res) => { /* ... */ });
  
  app.post("/api/accounts", express.json(), (req, res) => { /* ... */ });
  
  app.listen(port, () =>
    console.log(`Example app listening at http://localhost:${port}`),
  );
});

Then let's use that journey object to commit an event.

file: server.js
/* ... */
app.post("/api/accounts", express.json(), async (req, res) => {
  const { name, currency } = req.body;
  const id = Math.trunc(Math.random() * 1e11) + 1e12;
  await journey.commit({
    type: "ACCOUNT_CREATED",
    payload: { id, account_name: name, currency },
  });
  res.send({ name, balance: { [currency]: 0 }, id });
});
/* ... */

Restart express server, this time don't forget to include environment keys

MONGODB_URL=mongodb://localhost:27017 MONGODB_DBNAME=banks node server

Output

journey is ready!
Example app listening at http://localhost:3000

Now try to POST /api/accounts again

curl -XPOST localhost:3000/api/accounts \
     -H"content-type: application/json" \
     -d '{"name":"test", "currency":"USD"}'

Output

curl: (52) Empty reply from server

You will see a new error message

journey is ready!
Example app listening at http://localhost:3000
 jerni-dev  invalid jerni server provided, received: .

  1) if you're in development mode, make sure you ran jerni-dev.
  2) if you're in production mode, please set NODE_ENV=production before starting your app

jerni-dev It prompts you to run jerni-dev. Open another terminal (this is your last terminal to open) and run:

MONGODB_URL=mongodb://localhost:27017 MONGODB_DBNAME=banks npx jerni-dev start ./journey

Output

[ cli ] info: jerni-dev.start({ version: '1.0.0' })
[ cli ] info:  * source file: /path/to/your/my-first-jerni-project/banks/journey
[ cli ] info:  * options { http: 6181, verbose: false, dataPath: 'jerni.db' }
[ cli ] info: checking integrity of data file since last start
[ cli ] info: heq-server is listening on port 6181
[ cli ] info: writing lockfile to .jerni-dev
[ cli ] info: clean start new journey
[ cli ] info: worker ready
[jerni] info: start receiving data
jerni-dev>

Restart express server, and try to POST /api/accounts again

curl -XPOST localhost:3000/api/accounts \
     -H"content-type: application/json" \
     -d '{"name":"test", "currency":"USD"}'

Output

{"name":"test","balance":{"USD":0},"id":1024340236565}

Notice the messages from express server terminal:

 jerni-dev  running in development mode

versions:
  jerni:     1.0.0
  jerni-dev: 1.0.0

heq-server:
  original URL: undefined (not used in development mode)
  dev server URL: http://localhost:6181

stores:
  - mongodb://localhost:27017


 jerni-dev  event #1 [type=ACCOUNT_CREATED] has been committed to dev server at http://localhost:6181

You have successfully committed your first jerni event. To double check, open the file jerni.db that jerni-dev has created. You will see this:

file: jerni.db
### BEGIN checksum: e07846fb1760381ec374b65a9ccc1605 ###
{"type":"ACCOUNT_CREATED","payload":{"id":1024340236565,"account_name":"test","currency":"USD"},"meta":{"occurred_at":1595144407458,"client":"jerni-banks-demo","clientVersion":"1.0.0"}}

Note: whether or not to include this file to source control is up to you.

We recommend to commit this file to source control. However, review the diff before you commit to make sure it contains only useful seeding data for the next developer to begin with.

Read data from MongoDB

back to table of content

Modify the code in app.get(...) handler

file: server.js
app.get("/api/accounts", async (req, res) => {
  const Accounts = await journey.getReader(
    require("./journey/models/accounts"),
  );
  const allAccounts = await Accounts.find({}).toArray();
  res.send(allAccounts);
});

Restart express app and try to GET /api/accounts again

curl localhost:3000/api/accounts

Output

[
  {
    "_id": "5f13fe55239e75e5d26027f9",
    "__op": 0,
    "__v": 1,
    "balance": {
      "USD": 0
    },
    "id": 1024340236565,
    "name": "test"
  }
]

#getReader() will return a promise of a read-only mongodb's Collection object. You don't need to worry about the actual collection name.

jerni also adds __op and __v to all mongodb documents in order to handle optimistic locking. You can leverage that to detect stale data from your API clients. Otherwise, you don't have to send those information back to the aforementioned clients.

Verdict

back to table of content

That's it! These are the minimal code you need to bootstrap a jerni application. This does NOT in any way says that jerni is the best way to build such a simple API.