I don’t know of any rigorous comparisons, mainly because up to now MQTT-SN has found only limited use.
I think MQTT-SN could perform better than MQTT under certain circumstances but I wouldn’t say it’s likely as a blanket statement. First of all there are the different characteristics of UDP and TCP. TCP has the reliability and segmentation, so the quality of your connections and payload sizes will be a factor. For instance, if you have an unreliable (satellite) link, you may need to retry UDP messages yourself which could be worse than letting TCP do it for you.
There is at least one scenario where I think MQTT-SN should perform better than MQTT, and I think it’s a good way of thinking about the comparison. In IBM we used to from time to time discuss how to get MQSeries used on financial trading floors. There, TIBCO for one, reigned supreme, and we could not make headway because of performance. The reason the competition performed better, in terms of message latency, was because they used UDP multicast. Where MQ used TCP client-server connections for pub-sub (not MQTT but identical topology), TIBCO publishers would send messages to a multicast group. The filtering for topics interested in was done at the client end – all messages would be received by the client library, but only those subscribed to would be passed on to the application. I think that content was not encrypted (for speed), because the system was limited to the self-contained and isolated trading floor. As soon as you add more connectivity you have to think about security, auth and encryption, which slows everything down from the optimal.
A similar solution can be implemented using MQTT-SN QoS -1, at least over UDP and I think could definitely be faster. But multicast is limited to a LAN or subnet, not available on WANs. QoS -1 multicast is inherently unreliable – although that’s probably just fine on a network that’s not overloaded. Whether an MQTT-SN connection oriented UDP solution using MQTT-SN QoS 0, 1 or 2 would be faster than a similar MQTT one, I’m not sure. The differences could be marginal.
In many cases, I think the fastest solution could be a fat MQTT pipe from the cloud to the MQTT-SN gateway, then MQTT-SN multicast on the LAN. If you want high security then you might need a connection oriented MQTT-SN solution. Going completely MQTT-SN instead of MQTT might be faster but I wouldn’t bet on it. And I expect many solutions will need the extra features of MQTT; they wouldn’t be able to live with the limitations of MQTT-SN.
