I was talking to Andy Stanford-Clark on Friday, and asked him if he and Arlen ever considered using a payload format in MQTT. I was interested to know this because we are discussing approaches to MQTT metadata in the standardization process.
Many of the industrial protocols that MQTT was intended to replace, enhance or work alongside go to great lengths to specify the details of the payload. Data types, metadata, objects, all of these things may be defined in the data. MQTT does none of these things – the payload is completely undefined.
Andy’s answer follows.
MQTT is intended as a transport, not a protocol. It’s job is to get the data from one machine to another, not to define what that data is. Industrial protocols have often conflated the two jobs, so that they are inseparable. This makes them more complex and difficult to understand than they need to be.
MQTT can be used as a basis for a protocol which also defines the data to be exchanged, but the job of getting the data from one machine to another is already done. Andy gave the example of a company who used MQTT to defined a new protocol for the first time. Because MQTT took care of the connection between client and server and transporting the data between them, the process of defining the new protocol took half a day. Andy asked them how long defining a protocol would have previously taken: the answer was “two weeks”.
Since MQTT was first defined in 1998, data serialization formats have come into and gone out of fashion. XML was the “obvious” choice for a while. JSON and Google Protocol Buffers may be the favourite options today. MQTT is agnostic to them all, it can carry any of them, so it has not outlived its usefulness.
When people first learn about MQTT, they often ask about the lack of payload format, the implicit question being, what use is MQTT then? The answer being, whatever you want to use it for. Combine it with some data formatting and exchange sequences, then you have a protocol. Maybe we will see some standardized protocols in the future which will be based on MQTT, using it for the data transport.