Back to Top Mobile Nav

Bridging the Sensor Gap

This articles discusses IoT, software architectures for smart environments, and the PlanIT Urban Operating System™ (“UOS”) from Living PlanIT. As is the way of things, a number of project engagements and other stimuli continue to coalesce our thoughts and we hope that others might find them useful, or at least they might stimulate further thinking or discussion.

The term “Sensor Gap” is most definitely not our invention, but we use it if for no other reason than there is a little irony inherent in the overloading of the term. Some types of sensor, mostly of electromagnetic form, have a gap as part of their construction – as an example, tape heads classically have a ferrite gap. There is also an entire class of sensors for measuring gaps in a medium, particularly air gaps (these are often capacitive). However, the Sensor Gap we are referring to is the relative scarcity of really useful sensors in the context of smart buildings and infrastructure. We’ll explain this a bit more in a moment.

Before we get into that however a quick aside – while researching this article, we were intrigued to find a web page on the University of Cambridge site with exactly the same title as this article ( The article was announcing 'CamBridgeSens' – an initiative to coordinate sensor research across departments, disciplines and cultures. Sounds promising, you might think, and we did. We also thought that someone else likes bad puns almost as much as we do.

To our chagrin though, the article dated from May 2008, while the initiative is still ongoing and doubtless has spawned some interesting research especially into MEMS, microfluidics, fiber and CMOS sensors – neither it, nor commercial endeavors, nor multiple standards committees, have bridged the Sensor Gap in our view.

So what is this Sensor Gap? The principal form in which this is generally felt to exist is the bi-polar distribution of the sensor price-performance spectrum, when applied to scenarios such as urban environments. Or to put it more simply and pragmatically, many commercially available sensors are either relatively low cost and low capability (hobbyist, toy) or expensive (for scaled deployment) precise lab or military spec. devices. In comparison what the mass market needs - a common lament of practitioners in IoT and smart building/ city projects - is a middle ground of capability, resilience, and price.

This has all of the limitations one might expect. Sensors that are produced at scale are limited and narrow in their applicability, usually coming from a specific class of solutions or a vertical, while more sophisticated sensors have limited deployment - for example structural monitoring on large scale infrastructure - and therefore are built to a low volume and specialized application characteristic. Some have tried to “bootstrap” the mass IoT sensor market, but have generally targeted relatively low end capability partly because there is an enthusiast market which can be drawn upon (essentially part of the 'builder movement'), partly because of the desire to drive revenue and establish a foothold, and partly because they do not necessarily have the resources to engineer the solutions that will be required.

This is a point-in-time problem though. As more scaled deployments occur, the market will require more appropriately designed and packaged solutions, and companies working in this sector will figure out what the parameters of these solutions need to be. With appropriate vision and designing for future requirements, this can be accelerated and there are a few examples where this is already occurring. Indeed, while Living PlanIT is not principally a hardware or sensor company, our research and development of under-exploited and leading-edge approaches to biosensing will result in unique solutions which will provide optimized elements in the forthcoming landscape – see Peter van Manen's (Living PlanIT’s EVP Research & Development) recent article 'Gut Feeling' for more details (quick shout out here to Nuno Silva and Ricardo Eiras who lead this effort for us – two of the coolest and smartest people on our team).

Adjacent industries such as automotive, aerospace, and industrial control are also a good source for the right kinds of hardware, as they have already developed volume solutions with strong resilience and rational price/ performance compromise. It should be noted however that in many cases network and connectivity types will need to be adapted to support the built environment, since architectures such as a Controller Area Network (“CAN”) bus would be a poor fit for the larger distances and more open/ variable conditions found in this field. However, signs of adaptation from these verticals are already with us. For example, the integrated video analytics (“IVA”) found in Bosch Security Systems’ cameras - which we use in our deployments - use the same field-programmable gate array (“FPGA”) accelerated algorithms originally developed for automotive purposes such as parking control and lane guidance. Our PlanIT Edge Appliance™ draws upon automotive and aerospace designs and components to benefit from a similar dynamic.

We will see the emergence of volume devices which have an ideal compromise between price, capability, performance, and resilience, and the more that informed designers and operators demand better solutions and are creative about their sourcing, the faster this subclass of devices will be manifested. It's likely that this dynamic - occurring during the growth phase of the market - will result in a more diverse set of device connectivity and capability than is ideal, because we are some way short from having the canonical standards and patterns needed to ensure otherwise. However, providing we are landing in the right part of the volume curve to help control acquisition cost, there are approaches which can help mitigate the impact of this.

There is a secondary Sensor Gap of which the market is less familiar. One can easily argue that there is a gap between IoT and Smart Infrastructure architectures and the sensors themselves. What? Surely - you might say - one of the reasons for solutions like your UOS is to integrate and collect data from sensors, as well as enabling actuation and closed-loop control? So if there is a gap, aren't you and those like you filling it?

Well yes – and arguably effectively too. In an existing built environment - a 'retrofit' scenario - our range of out-of-the-box drivers integrate most sensors with most forms of connectivity. For unique or proprietary devices, a new driver can be implemented by Living PlanIT, our partners or customers. This is usually straightforward. You don't even necessarily need to be a coder, because you can develop drivers graphically using Mathworks Simulink and our library of configurable blocks.

However, the principal goal of sensor integration as generally practiced at the current state-of-the-art is to enable data capture and flow. While this is critically important, it is simply table stakes, and absolutely isn't the whole story.

Before we drill into that further, let's look at the architecture of sensor deployment briefly. Ultimately everything starts at the sensor head itself, which can take a number of forms from MEMS on-silicon designs through discrete devices through assemblies of mechanical/ chemical/ biological/ electrical parts. Eventually though, the physical property being measured has to be expressed as something which can be rendered as a data value. In the vast majority of cases, the stage immediately before the data value is electrical, measuring current/ voltage or variations thereof.

There is a lot of variation however in how the electricity-to-data part of this process chain is organized. In some cases the circuit is completed by an A-to-D converter or I/O port or low voltage side of a relay – fundamental low level connections directly to an I/O board or port on a computing device. At the other end of the spectrum, the sensor head might be integrated with a 'smart sensor' board which performs the conversion to data, smoothing and scaling, control of data rates, power management, encryption/ signing, and the network transport and protocol implementation to relay the data to the rest of the infrastructure.

Referring back to the primary Sensor Gap discussed earlier, currently complete smart sensor assemblies are limited in scope, performance, environmental factors, and relatively expensive. They conform to (many) multiple standards in terms of management and control. This isn't a great surprise given the current state of the IoT market – consolidation and effective standardization around clusters and patterns are still some years off. (See prior posts by John Stenlake, Living PlanIT’s CTO for more on this – ‘Managing Data in the Urban Operating System’ and ‘Why the Urban Operating System Matters’ ).

What does this mean in practice? If you are designing and deploying a smart building or similar environment, you are unlikely to find a homogenous architecture in terms of one brand, one type of connectivity, or set of standards that spans the whole of your sensor requirements. Kudos to the likes of Digi International and EnOcean for starting to build solutions and ecosystems at useful scale, encompassing a wide range of devices with consistent ways to be integrated and addressed, but we are still some way off from having both adequate and sufficiently diverse choices of compatible devices of appropriate performance to fill most solution bills of material.

This diversity leads to complex network/ connectivity architectures, and an incomplete and variable array of protocols and APIs for security and device management even where these capabilities are exposed. Frequently, these capabilities won't be exposed, which means that enabling end-to-end security in a sensor chain, or having scaling or filtering scaled-out to reduce centralized processor load, is a lot harder than it might or should be. We would argue that this also represents a Sensor Gap – between the set of capabilities needed to build a well-managed smart sensor infrastructure, and those practically available.

A further example of this kind of problem are the well-meaning manufacturers of mesh-network sensor families who inevitably have to provide a gateway device to provide the point of presence on the IP network. For starters, the API for every one of these is different requiring its own integration solution. Worse however, quite a few of them only support polling!

This is one of our pet peeves, because when everyday life requires real-time measurement and control, being asked to poll is exactly what you don't want. If you aren't entirely familiar with the theory, to obtain any level of temporal precision you have to poll at a frequency of at least 2/t, which still has a latency as high as t/2. Either way, the polling process imposes a significant and unnecessary burden on both server and client, as opposed to efficient push-based techniques.

There are some pragmatic solutions to mitigate the impact of some of these design limitations such as direct coupling of basic sensors with edge infrastructure - leveraging distributable solutions like UOS Real-Time Control™ (“RTC”) and our forthcoming PlanIT Edge Appliance™ - can minimize the scope for intrusion. Features such as data inspection and heuristics can help detect endpoint tampering in the absence of security infrastructure. And features such as auto-synchronization logic can help reduce latency when polling for events that occur at regular intervals. However, mitigation cannot entirely compensate for the inherent failings of an architecture, and it is better to avoid these limitations in device and infrastructure selection.

Where sensors, actuators, gateways, and aggregators expose management and security features, the next challenge is to manage their inherent heterogeneity. Certain solution providers are focusing on this incipient requirement as separate infrastructure. However, where a modular driver architecture already exists to enable data capture and real-time-control, this can be extended to map diverse forms of management API to a capability meta-model which to some degree reifies to requirements already inherent in a distributed architecture. This means that architectures such as that of the UOS can be extended to manage connected devices in a consistent way.

This isn't a complete solution to the problem of device capability and management, but it is a pragmatic one which will both suffice until consolidation occurs, and help drive the development of capability clusters and ecosystems.

There is also scope to integrate this with existing capabilities in network and IT management which while much better supported by standards (hello SNMP!) are also generally implemented by expensive and complex vertical solution suites. While these are compelling for data center operators and systems integrators looking for a consistent process model, this doesn't necessarily make sense for the new customer who is primarily looking for an IoT solution. The emergence of unified solutions is a further reason to invest in flexible and professional smart environment middleware.

Ultimately what we are seeking, and what truly underlies our concept of 'Edgeless Computing', is distributed intelligence - or 'intelligence everywhere' - fabric, which facilitates the distribution of logic to where it makes most sense. Despite many generations of attempts to realize this ('Oh my God, they killed CORBA') it doesn't naturally fall out of established IT architectures with any degree of utility. But to quote Victor Hugo, “There is nothing as powerful as an idea whose time has come", and so perhaps we will see this evolve in our current era. With such capabilities, however manifested, being able to unify and distribute approaches to smoothing, scaling, change thresholds, management of data rates and events, management, security, etc. would be straightforward, with decisions made and distribution according to what can be reasonably done where, to what degree and in what timeframe (nanosecond, millisecond, seconds etc.) allowing even humble processors to participate in a mesh of capacity and still enabling economies of scale-out.

So while there are at least a couple of dimensions to the Sensor Gap, and it is definitely still with us, effective bridging solutions are coming to fruition, and the level of economic activity and demand in this space will help to ensure that gaps will ultimately close. In the meantime, if you are struggling to find effective solution design or bills of material for a particular business need, let us know – we may be able to help!

Special thanks to John Stenlake for his contributions to this article.

About Living PlanIT

Living PlanIT is a technology company that created the world’s first Urban Operating System (UOS) which, in combination with the products it supports, unlocks the full potential of data to make cities better, safer and more vibrant places to live.

Living PlanIT has built an extensive partner network around the concept of a shared, unified approach to smart urban technology architecture in which machine intelligence moves ever closer to originating sources of data and control. We call this architecture PlanIT Edgeless Computing™ and it is implemented throughout the PlanIT Urban Operating System™, providing a framework for resilient and secure computer and systems architecture for digital and biological sensing, control, analytics, machine learning, applications and visualization techniques.

Living PlanIT asserts its rights to the following Trademarks: Living PlanIT™, PlanIT Edgeless Computing™, Edgeless Computing™, PlanIT Urban Operating System™, PlanIT UOS™, PlanIT OS™, PlanIT Crumbs™, PlanIT PlaceApps™, PlanIT Valley™, PlanIT Assurance™, PlanIT Labs™, PlanIT Retail™, PlanIT Life™. Any reference to these names in this content shall be considered as an assertion of these Trademarks. or follow us on Twitter @Living_PlanIT