Tracks in Depth


The common building blocks of a FlowThings.io application are Flows and Drops. However, Tracks provide the power to build complex event streams and applications.

A Track is a construct for:

  • Routing a Drop between Flows
  • Performing arbitrary computation based on Drop data
  • Communicating with external services using HTTP

Whilst Tasks are scheduled opererations, Tracks represent real-time computation, and are triggered whenever a Drop is processed.

Tracks are attached to a source Flow, such that whenever a Drop hits that Flow, it triggers the Track. Tracks can have an optional destination Flow, such that by default all Drops are routed to that destination.

This is the most basic form of Track. To go beyond basic routing and filtering, users can attach custom JavaScript to the Track definition. The Javascript allows users to define any amount of processing on the Drop that they need. This is where the true power of flowthings.io comes into play.

In this guide, we will see examples of different Tracks that can be written.

Sections:

A Basic Track


The most simple Track connects one Flow to another, and forwards all Drops it receives to the destination Flow.

curl https://api.flowthings.io/v0.1/alice/track \
       -H "X-Auth-Token: ZPpj7B2LI66QtCAQgK97Yij68tOgPyfo" \
       -d '{
              "source" : "/alice/sensor",
              "destination" : "/alice/display"
           }'

The source and destination attributes are the minimum required fields for this type of Track - both Flows must have been created first.

This kind of basic Track is not particularly powerful on it's own, other than for routing data. Let's dive a little deaper.

A Track with a Filter


We can add filters to Tracks, which make use of the powerful Flow Filter Language, to selectively route Drops from source to destination:

curl https://api.flowthings.io/v0.1/alice/track \
       -H "X-Auth-Token: ZPpj7B2LI66QtCAQgK97Yij68tOgPyfo" \
       -d '{
                "source" : "/alice/sensor",
                "destination" : "/alice/display",
                "filter" : "elems.temperature > 72 && AGE < 1000"
            }'

This Track will forward only those Drops who pass the filter check

Tracks with embedded JavaScript (FlowJS)


The true power of the flowthings.io is harnessed when embedding JavaScript code into a Track. This allows for custom computation upon receipt of a Drop.

The simplest form of Track with JavaScript would be:

curl https://api.flowthings.io/v0.1/alice/track \
       -H "X-Auth-Token: ZPpj7B2LI66QtCAQgK97Yij68tOgPyfo" \
       -d '{
                "source" : "/alice/sensor",
                "destination" : "/alice/display",
                "js" : "function (input_drop) { return input_drop; }"
            }'

... which just passes the Drop through, as though it were a regular Track. From the above code, we see that the function performs no processing on the Drop, before returning it.

To do something a little more complicated, we could add a field to the Drop as it passes through:

curl https://api.flowthings.io/v0.1/alice/track \
       -H "X-Auth-Token: ZPpj7B2LI66QtCAQgK97Yij68tOgPyfo" \
       -d '{
                "source" : "/alice/sensor",
                "destination" : "/alice/display",
                "js" : "function (input_drop) {
                    if (input_drop.elems.sensor.value == \"thermostat\"){
                        input_drop.elems.myMessage = \"Cooking with Gas!\"
                    }
                    return input_drop;
                }"
            }'

This Track will read the data inside the Drop as it arrives, and conditionally add new data when it sends the Drop to the destination.

Aside - Testing your Tracks through Simulation

To ensure your Track processing works before you commit to using it in a live application, you can use the simulate REST API endpoint. You supply the Javascript code and a sample Drop, and the Track processing will be run and returned:

curl "https://api.flowthings.io/v0.1/alice/track/simulate?pretty=true&hints=false" \
       -H "X-Auth-Token: ZPpj7B2LI66QtCAQgK97Yij68tOgPyfo" \
       -d '{
                "drop" : { "elems" : { "a" : 2 } },
                "js" : "function (input_drop) {
                    var a = input_drop.elems.a.value;
                    input_drop.elems.b = a * 3;
                    return input_drop;
                }"}'

Note the drop attribute, which can be a Drop with any values you like.

The output from this call is this:

{
  "head" : {
    "status" : 200,
    "ok" : true,
    "messages" : [ ],
    "errors" : [ ],
    "references" : { }
  },
  "body" : {
    "drops" : {
      "id" : "d000000000000000000000000",
      "elems" : {
        "a" : 2,
        "b" : 6.0
      }
    },
    "log" : [ {
      "a" : 2,
      "line_no" : 2,
      "input_drop" : {
        "elems" : {
          "a" : {
            "type" : "integer",
            "value" : 2
          },
          "b" : 6.0
        },
        "id" : "d000000000000000000000000"
      }
    } ],
    "errors" : [ ]
  }
}

The result shows how the Drop is altered. In this case, it is what we expect. What happens if we simulate with the elems.a attribute missing:

curl "https://api.flowthings.io/v0.1/alice/track/simulate?pretty=true&hints=false" \
       -H "X-Auth-Token: ZPpj7B2LI66QtCAQgK97Yij68tOgPyfo" \
       -d '{
                "drop" : { "elems" : { "oh-no" : 2 } },
                "js" : "function (input_drop) {
                    var a = input_drop.elems.a.value;
                    input_drop.elems.b = a * 3;
                    return input_drop;
                }"}'

The result is we are shown the error, before we run into it in a Production setting:

{
  "head" : {
    "status" : 200,
    "ok" : true,
    "messages" : [ ],
    "errors" : [ ],
    "references" : { }
  },
  "body" : {
    "drops" : [ ],
    "log" : [ ],
    "errors" : [ "TypeError: Cannot read property \"value\" from undefined" ]
  }
}

Simulating can be extremely helpful for experimenting with the powerful features flowthings.io offers.

Utilizing the Flow API inside a Track


Aside from manipulating Drops as they are processed within a Track, you can also make calls to the Flow API using JavaScript. With this capability you can:

  • Query other Flows for data based on the Drop contents
  • Update and Delete other Drops

The Flow JS api documentation gives full descriptions of what is available.

curl https://api.flowthings.io/v0.1/alice/track \
       -H "X-Auth-Token: ZPpj7B2LI66QtCAQgK97Yij68tOgPyfo" \
       -d '{
                "source" : "/alice/sensor",
                "destination" : "/alice/display",
                "js" : "function (input_drop) {
                    // Lookup other Drops
                    var conversions = Flow.Drop.find(\"f54ad9d8bd4c68117a914ba24\",
                        {\"filter\": \"elems.category == TEMPERATURE\"});

                    // Do some conversions ....
                }"
            }'

Performing HTTP requests inside a Track


You can interact with external services within a Track, by performing HTTP requests:

curl https://api.flowthings.io/v0.1/alice/track \
       -H "X-Auth-Token: ZPpj7B2LI66QtCAQgK97Yij68tOgPyfo" \
       -d '{
                "source" : "/alice/sensor",
                "destination" : "/alice/display",
                "js" : "function (input_drop) {
                    // Perform an HTTP GET request
                    var g = Flow.Http.get(
                        \"http://httpbin.org/get\",
                        {\"header1\":1,\"header2\":2});

                    // Do something with the data ....
                }"
            }'

The response will be a javascript object:

{
  "content": "<raw HTTP response>",
  "status": "<HTTP Status Codes, e.g. 200>",
  "contentType": "<The content type of the response>",
  "message": "<the response message, if any>",
  "headers": "<a map of the headers>",
  "json": "<if the response is JSON, the parsed response will be available here>",
  "xml": "<if the response is XML, the parsed response will be available here>"
}

You can then use the response, synchronously, in the track.

Tracks with Multiple Destinations

Tracks can have multiple destinations. You may want to use this when:

  • You want to route the Drop to a different Flow depending on some processing
  • You want the Drop to go to multiple Flows

When specifying multiple destinations, you must do the following:

  1. Ommit the destination attribute on the Track
  2. Return a JavaScript object within the js, which specifies where the Drop(s) should go

The format for this JavaScript object is simple. See below for an example:

curl https://api.flowthings.io/v0.1/alice/track \
       -H "X-Auth-Token: ZPpj7B2LI66QtCAQgK97Yij68tOgPyfo" \
       -d '{
                "source" : "/alice/sensor",
                "js" : "function (input_drop) {

                    // Return the Drop to multiple places
                    return {
                        \"/alice/alarm\" : [input_drop],
                        \"/alice/display\" : [{\"a\" : 1, \"b\" : 2}],
                        \"/alice/notification\" : [{\"c\" : 1},
                                                  {\"d\" : 2}]
                    }
                }"
            }'

This shows the different options available to the user when returning Drops:

  • The original drop structure can be returned.
  • New Drops can be created using JSON notation.
  • Multiple Flows can be specified.
  • Multiple Drops can be sent to each Flow.