For the first several years that I worked in Intel’s Internet of Things Group (IOTG)1, I worked on projects related to Radio Frequency Identification (RFID) products. Initially the product was a cloud SaaS solution, but this later evolved into the RFID Sensor Platform (RSP), an inventory tracking solution using our RFID sensors and edge compute gateway. This image from “Delivering Accurate and Automated Inventory Tracking” (a white paper describing a pilot study with DHL) shows the main ideas behind the system:
RSP in a warehouse environment.
Intel RFID Sensor Platform
Intel RSP provides inventory visibility and near-real-time analytics at the edge in a warehouse or factory environment covering large areas, overcoming challenges associated with separate buildings and numerous sources of signal interference.
- Intel RSP Sensors, placed at Receiving and other areas of the warehouse, register the presence of RFID tags.
- Intel RSP Sensors detect and track items as they are transferred from Staging to Sorting and Quality Control.
- In Storage, Intel RSP Sensors on each shelf provide accurate location and time-stamp information.
- Intel RSP Sensors connect to a central Intel edge compute device via Ethernet. Inventory data associated with the RFID tags is aggregated and analyzed at the edge for near-real-time analytics.
- The Intel edge compute device can connect to an on-premise company inventory system or to the cloud through the Intel RSP Software Toolkit. Outbound orders also are received here and sent to fulfillment.
- Intel RSP Sensors facilitate automated inventory registration during each step of the order Fulfillment process.
Wide Coverage
Each Intel RSP Sensor captures data within a range of up to 15 square meters (160 square feet), tracking multiple items at one time.
Timely event capture
The Intel RSP Sensors record how long an item remained at each station, providing useful data for process optimization or improved service-level agreements.
Automation
Actionable insights from the Intel RSP Sensors decrease throughput time by eliminating manual inventory registration throughout the process flow.
High read accuracy
Intel RSP Sensors are able to capture RFID data with greater than 99% accuracy.
When the hardware platform was discontinued, we open-sourced parts of the work into EdgeX Foundry. As part of that, we were asked to develop a Go implementation of the Low-level Reader Protocol (LLRP), an open standard for communicating with networked RFID readers. I’m particularly proud to have contributed the client library that forms its backbone.
Go Meta
As a primary feature, the library handles serialization and deserialization among LLRP’s binary format, Go structs, and JSON. The actual code for this is generated by a Python script that reads a specification of the LLRP message graph from a YAML file, performs static validation and optimizations, then outputs the relevant Go code, complete with doc-comments and unit tests.
Why bother doing it this way? In short, because other ways weren’t good. LLRP is a large specification with a highly structured message format. Handcrafting the explicit parsing code would have been tedious and error-prone, so the most sensible approach was some form of meta-programming. At the time, Go still did not support type parameterization or generics, so the only meta-programming options were runtime reflection or code generation.
Go uses reflection to handle conversions between its structures and JSON. The standard library parser can fill structs based on field names, and its default behavior can be modified by “struct tags”, which are arbitrary string literals made available via the reflection interface2. I experimented with these initially, but it was too easy to make a mistake, and I spent too much time debugging difficult to spot errors that could only be triggered at runtime. In addition, I had some concerns about the potential performance hit that typically comes with reflection; the library would be spending most of its time on these conversions, so small gains and losses would add up quickly3.
As I was building up a mental model around LLRP,
I built a spreadsheet to reference the memory layouts, types, and rules.
Looking at it, I realized it had all the needed parsing info,
so I converted to it the file messages.yaml
and wrote the script I described above.
When executed, it generates 12,000 lines of compile-time-checkable parsing code,
plus an additional 3,400 lines of unit tests
that validate arbitrary bytes round-trip successfully.
Although this project uses Make as its primary build tool,
Go’s own tooling includes a special go generate
command specifically for this sort of meta-programming.
It searches the code for magic comments in the file header
which specify arbitrary commands it should execute.
Whether Make or go:generate is more appropriate is up for debate,
but I would argue it communicates a level of intent:
using go:generate seems more appropriate when it’s the library author running it
and committing the generated results into the repository.
In any case, the YAML file and companion script had other benefits. For instance, I was able to keep relevant notes on the LLRP structures within my YAML specification and use them to directly generate proper Go documentation. It also meant I could generate DOT files to generate graphs of the LLRP hierarchy, though viewing the full thing all at once is only nominally useful (click here for a zoomable PDF version)4:
Footnotes
IOTG was reorganized and renamed several times over the years, and as of 2024, the relevant organization is named Network and Edge (NEX).
Struct tags are marginally better than magic comments, but their syntax is entirely based on convention, and bizarrely, they influence type identity.
I had similar concerns about the reflection cost associated with the conversion between JSON and Go structs, but there are 3rd-party drop-in replacements available that we could make use of if it became a problem.
This PDF is licensed under the terms of the Apache License 2.0.