The following collaboration between David Beberman, CSO at Aicas and Michael Holdmann, CEO, CYTIoT, INc. discusses the architecture needs of Industrial Internet from a device resource perspective. The reason for the post is due to real world issues that are now coming to surface (with direct involvement from authors) at some very large companies who have only focused on closed system architectures comprised of few devices speaking with few applications up to this point. These same companies have subsequently designed their Industrial Internet, Industrie 4.0, IoT systems thusly and now understand different approaches and architectures are necessary.
The vision of the Industrial Internet is billions of connected devices communicating, sharing data, controlling and becoming “smarter” through big data analytics. The concept of the Industrial Internet depends on network connected sensors, controllers, actuators and other devices. The vision is all the devices getting “smarter” by being connected together and to the Internet.
In the last 10 to 15 years, we have grown accustomed to the Internet just being there. Network bandwidth in the gigabits on demand, websites that can handle millions of users at a time are the norm. Why can’t we just connect all the devices to the Internet too?
There is a subtle, but important difference between humans connecting to the Internet with smartphones, tablets and laptops, and devices connecting to the Internet. The human interaction is client-to-server where the client is the smartphone, tablet or laptop, and the website is the server. The Industrial Internet turns this on its head. The device is the server and the client is an application in the cloud.
Following this model, current Industrial Internet architectures embed a webserver in the device and try to connect to it using a request/response browser-type model. That is, an application on the Internet connects to each device it is interested in, and requests data from the device or posts data to the device. Architectures that use this model include OPC UA, SOA, and/or general embedded webservers.
Although this model seems to work in the lab, and for small rollouts, it faces an insurmountable scaling problem with any large sized deployment. There are two problems with this model: many applications in the cloud connecting to a single device, and many devices connecting to a single application. That is, there is a many-to-one and a one-to-many scaling problem. What’s needed is a scalable connectivity architecture for the many-to-one and one-to-many problems.
In a bit more detail, the many-to-one problem, multiple applications connecting to a single device means that the device has to handle all the requests from all of the applications. Since the applications are independent from each other, such as independent manager applications, all of the applications might make requests to the device simultaneously, and repeatedly.
For example, a PLC device might be managed by any number of SCADA managers using OPC UA. Each manager may want current values for any number of data points. The OPC UA server will receive requests repeatedly for the value of the same datapoints. The amount of redundant data and redundant consumption of resources on the devices is equal to the number of datapoints X the number of managers.
Clearly, there will always be a point where the device will run out of resources to respond to the multiple polling requests from all of the managers. With a direct client-server request/response polling architecture, the only reasonable workarounds are to limit the number of simultaneously connected managers, and the number of requests per period that the device will respond to. Neither session limits nor request limits are a full solution.
In detail the one-to-many problem, many devices directly connected to a single server application, any important change in state that a device may have available to report, has to wait for a subsequent poll by the application. The polling period for any given device is dependent on the number of devices the application is currently managing with periodic polls. Although in the one-to-many problem, no redundant polling of a device’s datapoints occurs per period, if a device is acting as a gateway for additional devices (e.g. sensor, actuators), the polling for each datapoint from each additional device is another instance of CPU resource consumption. The number of devices the gateway has resources to manage is now a function of the polling rate X the number of devices X the number of datapoints.
Solution architecture must be able to scale with a large and growing number of applications, and a large and growing number of devices. Such architecture must eliminate redundant communication of data and minimize the computing resource demands on devices. Such architecture would enable the Industrial Internet to grow organically at a pace to keep up with customer and user demands, and would help foster innovation.
There is one architectural detail that the many-to-one and one-to-many configurations discussed have in common. That is they all are examples of direct client-server connections. If multiple applications need the data from a device, each application must connect to the device and request the data. If a single application needs data from multiple devices, it must connect to each device and request the data from the device. The direct client-server connection fundamentally limits the scalability of the Industrial Internet.
The alternative to the direct client-server connection is “messaging bus” architecture. In contrast to the direct client-server connection, all parties, clients and servers, connect to a messaging server using message-oriented middleware whether a client or server, each is a messaging client to the messaging server. The messaging bus solves the scalability problem using a publish/subscribe communications model implemented in the message server.
The following examples explain why this is true.A many-to-one configuration of multiple applications communicating with a single device requires the device to send a single message to the messaging server. The device does not need to send separate messages, one for each application. This single message approach means that the minimal numbers of resources are consumed by the device. The message server is responsible for forwarding the message to any subscribe applications and any resources associated with performing this task.
A one-to-many configuration of multiple devices communicating with a single application via a message server enables each device to send a message only when its state information has changed. The application does not need to do a separate request/response poll for each device it is managing. The device “pushes” a message to the message server when a change in state triggers an event to send a message. This eliminates the redundancy caused by polling.
The messaging server is separate and independent of the devices and the server applications. The implementation of the messaging server may consist of multiple physical servers with distributed load balancing, failover capabilities and other scaling technologies as needed.
An architecture consisting of client devices, application servers, and message-oriented middleware can be seen as a “separation of concerns” approach to scaling the Industrial Internet. It is the best use of the limited device compute resources, while providing a robust, scalable architecture for the SCADA managers, and big-data analytic applications.