Which Paho MQTT C API to use? (And some history)

It may not be obvious if you take a look at the Paho MQTT (http://mqtt.org) C client libraries (http://www.eclipse.org/paho/) , but my intention all along was to avoid proliferation of C APIs.

When I first started writing a C client for MQTT, around 2008, there was one other C MQTT API that I knew about – the now withdrawn IA93 (http://www-01.ibm.com/support/docview.wss?uid=swg24006525).  With all respect to the author, this had an interface which I did not want to copy.  Firstly, there were the variable length option structures, which meant that you had to copy topic strings, for instance, into their correct location in the structure before making the API call.  Secondly, there was the overly complicated method of setting up the API to operate with background threads.

IA93 did have one feature that I felt it was desirable to copy: the ability to work using no background threads.  As MQTT is intended for embedded devices, we would like the library to work in very simple environments.

My other starting point was the MQTT Java API at the time, I saw no reason why the C API should be much more difficult to use.  So I set out to create a C API that was close to the Java API, with the addition that it could be optionally run with no background threads.  This became the MQTTClient API (http://www.eclipse.org/paho/files/mqttdoc/Cclient/index.html).  The parameters are similar to the Java API – no unnecessary variable length structures.  The method I chose to indicate whether you want background threads or not is to simply make a call to MQTTClient_setCallbacks().  Asking for callbacks means that you have to have a background thread, right?

I was developing Really Small Message Broker (RSMB) around the same time, which is single threaded, using multiplexed I/O on the TCP port.  At the time, every MQTT server I knew of was multi-threaded – at least one thread for every connecting MQTT client.  I thought it would be interesting to see how far you could get with a single thread: reminding me of the techniques we had to use in the 1980s to write games on ZX81s and the like.  So, I thought reusing the same approach would be good for the C client – hence only one background thread no matter how many client objects you create (limited to whatever the ‘select’ system call will handle).

Another conscious decision was to use C rather than C++.  I started RSMB in C++, but quickly found out that compiler template support was pretty buggy at the time, so dropped back to ANSI standard C.  That was somewhat disappointing, but several years later does have the advantage of not being incompatible with Objective-C.

The final part of the story so far is the widespread use of graphical environments, or more accurately, environments where the application itself is a series of callbacks.   The ubiquitous example is the JavaScript program running in the web browser.  When the Paho MQTT JavaScript API was devised, it needed to be completely asynchronous.  The previous C API, in the mould of the Java API, had some calls which would block, to make programming easier.  We needed a totally asynchronous C API too, so I followed the model of the JavaScript API – which unfortunately meant a new interface, MQTTAsync (http://www.eclipse.org/paho/files/mqttdoc/Casync/index.html).

So, which to use when?

  1. MQTTClient, single-threaded: when you want to use only one thread for your entire application.
  2. MQTTClient, multi-threaded: when you want ease of use, aren’t bothered about some calls blocking, but still want messages to be sent and received in the background.
  3. MQTTAsync: when you want no calls to block at all.  (Messages will be sent and received on two background threads).

The MQTTAsync API happens to perform better than the MQTTClient API, but that wasn’t the rationale behind it.

Thanks to a Paho contributor, Frank Pagliughi, there is also now a C++ layer over the asynchronous C API.  His intent is to have an API which mirrors the corresponding Java one as much as possible.

Hmm, I think we’ve been here before!

Author: Ian Craggs

I am the project lead of Eclipse Paho, a member of the Eclipse IoT working group and Eclipse IoT PMC, and co-chair of the OASIS MQTT-SN standardization Technical Sub-Committee.

6 thoughts on “Which Paho MQTT C API to use? (And some history)”

  1. Interesting stuff Ian, really appreciate the insight re the C APIs and RSMB. RSMB always impressed me. I think it is a remarkable bit of kit. Very compact, fast and rugged. Put one on the cloud in Europe for demos years ago and it is never down. Now I know who to thank. I’m looking forward RSMB joining your Paho work on Eclipse, especially knowing there’s such much in common between it and the Paho clients.

  2. Thank you for this article, very informative.

    Question regarding the MQTTAsync API:
    If I create more than one client in a single application, will they both share the same two threads or will they both get their own two background threads?

  3. Hi Ian, thanks for your hard work, your knowledge is definitely worth sponsoring! (as we have)

    We are looking to use MQTT-C on our openWRT platform.
    Could you guide me to the correct place to ask questions and get support on implementation?

    Thanks!
    Xarion

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.