Working with Bytes and Strings

Generally the data we're working with in Pictorus apps is numeric, either as scalars or vectors. However, in some cases it's necessary to represent data as bytes. This is especially useful for receiving data from an external source, and for publishing data outside of an app. In these cases we first need to convert the data to a common format that can be understood by both the publisher and the receiver. For instance, we might run an app that calculates a desired position state, formats it as a bytes representation of a JSON object, and then publishes that data over UDP for a separate process to consume and act upon.

Strings vs. Bytes

In the above example, people often think of the formatted data as a String. In Pictorus, we don't distinguish between strings and bytes, as is common in many programming languages. All data that is received by an Input Block or published by an Output Block is represented as bytes, which is just an vector of 8-bit values. A String, on the otherhand, is a specific interpretation of bytes that conforms to a standard such as UTF-8. These can be represented as bytes in an app, but take on no special meaning.

For example, the string "Hello World" is just the UTF-8 interpretation of the following byte vector:

[72, 101, 108, 108, 111, 32, 87, 111, 114, 108, 100]

It's also common to see byte arrays expressed in hex notation:

[0x48, 0x65, 0x6c, 0x6c, 0x6f, 0x20, 0x57, 0x6f, 0x72, 0x6c, 0x64]

Serializing and Deserializing

The process of converting data into bytes that can be used outside of a program is commonly referred to as serialization. Converesely, interpreting bytes into a known format is often called deserialization. Blocks used for these operations can be found under the "Serialization" tab in the block palette.

Receiving data as bytes

You can receive data from an external source in your apps using any of the supported Input Blocks. This data could be in any number of formats, so you need to tell the app how to interpret it and parse it into useful numeric data that can be handled by other blocks and components. Going back to our JSON example, we might have an app that expects to receive an x/y/z position in meters from an external source, and then command a robot to try and move to that position. One way we could represent this data is with the following JSON object:

{
  pos_x_m: 1.0,
  pos_y_m: 2.0,
  pos_z_m: 3.0,
}

To receive this data in our app, we could do the following:

1. Create a UDP Receive Block that binds to a port that another app can publish to.
2. Connect a JSON Load block to the output of our UDP Receive block.
3. Configure the JSON Load block to extract pos_x_m, pos_y_m, and pos_z_m into 3 different numeric outputs.
4. We can now use our position data for further calculations!

Publishing data as bytes

Publishing data from an app is essentially the opposite process as receiving it. Given some numeric fields we want to send out, we first need to convert our data into a format that an external consumer can understand. If we wanted to implement the publisher app in our UDP example we would do the following:

1. Create a signal for each of our desired values (x, y, and z position).
2. Create a JSON Dump block and connect our 3 position signals as inputs.
3. Update the JSON Dump block settings to have the correct object key names for each corresponding signal.
4. Create a UDP Transmit block that publishes to the receiver address, and, finally, connect the JSON Dump output as an input to this block.

Tips for working with byte data

Working with byte data in Pictorus is very similar to working with numeric data. However, there are a few helpful tips that can simplify things.

Debugging bytes

Plot blocks are commonly used for debugging numeric data in apps. However, they aren't particularly useful for displaying byte data. Instead, you can use the Inspect Block, which allows you to visualize a byte array at different points in time. Inspect Blocks will attempt to render data as a string, an array of bytes, and an array of hex values, so you can quickly understand what your data looks like in a few common formats.

Simulating bytes

It's often helpful to be able to simulate input data, so you can quickly validate your deserialization steps and downstream calculations. Luckily this is easy to do in Pictorus!

1. Simulate your desired data by connecting blocks as usual.
2. Use the desired "Serialization" blocks to convert your sim data into the correct bytes representation.
3. Connect your serialized bytes signal to the "Sim Input" port of your input block

Any blocks that are used for simulation will be removed when compiling for a hardware target, so they won't affect your real builds.

It's common to create multiple apps in Pictorus that communicate with each other by serializing/deserializing data sent across a network/file protocol. If your serialization strategy is complex, you can make things easier by converting the relevant blocks into a reusable component. This component can then be used by your publisher app, and also pulled into your receiver app for serializing sim data. This ensures your apps can communicate with each other successfully using whatever format you choose for your data!

Working with complex protocols

Having to explicitly convert all byte data into numeric data and vice-versa can seem a bit complicated, but this approach allows us to handle a wide variety of serialization strategies. For instance, you might have a JSON object that contains a field of comma-delimited numbers. Because most serialization blocks allow you to output data as bytes, they can be easily chained together to handle nested encodings like this. In this example we would do the following:

1. Connect the raw bytes signal to a JSON Load block
2. Configure the JSON Load block to extract the desired key and output it as bytes
3. Feed this signal into a Bytes Split Block, and configure it to grab data from the desired indices.