New “Embedded” Paho MQTT C Client

Yesterday I added the first phase of a new C client for “embedded” platforms to the Paho project in a new repository. It might seem odd to start another C client library, but I think the logic is sound.

When I started writing the first version of the Paho C client it was to work with IBM MQTT servers like Microbroker, Message Broker (now IBM Integration Bus Advanced) and my own RSMB. In spite of trying to keep it as lightweight as I could, I had to respond to many requirements without which the library would not be ‘complete’. So it grew. The main priority was to make MQTT programming simple – size was not that important.

The end result is a library which builds to about 80k (if you strip the debugging symbols), and is not easily stripped down if you want something smaller. A later addition was the asynchronous client, in the same repository, which changed the programming model to be entirely non-blocking, but didn’t make the library any smaller.

Even though most devices are getting ever more capable, there are still the smallest, most frugal motes that have limited capabilities. MQTT was intended to work on these devices, which is why the protocol is public – you can write a bespoke client library if you want. The existing client libraries are too big and inflexible, so this is the first stage in attempting to fix that.

I didn’t want to destabilize the existing libraries, so decided to start afresh. These small devices might not use the usual libraries for networking, threads, timers or even memory allocation. So I thought I would start from the bottom up, with a basic lower layer that would omit all network or memory allocation calls. This lower layer merely serializes and deserializes MQTT packet data so that it can be written to or read from the data destination or source. The end goal is a totally asynchronous API – perhaps with an external definition akin to the existing asynchronous API.

The best way to see it in action is to look at some samples. The simplest sample qos0pub.c is to send a QoS 0 MQTT message. It connects, publishes and disconnects all in one go, making it one of the shortest MQTT applications you can write. The core of the program looks like:

MQTTPacket_connectData data = MQTTPacket_connectData_initializer;
int rc = 0;
char buf[200];
MQTTString topicString = MQTTString_initializer;
char* payload = "mypayload";
int payloadlen = strlen(payload);int buflen = sizeof(buf);

data.clientID.cstring = "me";
data.keepAliveInterval = 20;
data.cleansession = 1;
len = MQTTSerialize_connect(buf, buflen, &data); /* 1 */

topicString.cstring = "mytopic";
len += MQTTSerialize_publish(buf + len, buflen - len, 0, 0, 0, 0, topicString, payload, payloadlen); /* 2 */

len += MQTTSerialize_disconnect(buf + len, buflen - len); /* 3 */

rc = Socket_new("127.0.0.1", 1883, &mysock);
rc = write(mysock, buf, len);
rc = close(mysock);

We set up the data we want to connect with, and add that to a buffer (1). Then we add the MQTT publish command to the end of the same buffer (2). Lastly we add the disconnect command to the same buffer (3). The disconnect is not strictly necessary – it’s an act of kindness to the server, for the cost of a few bytes.

We can write this buffer directly to a socket which is connected to an MQTT server, and then close the socket immediately, and we’ve published a message. The entire program comes in at about 5k over and above a completely empty one using RedHat and gcc – which is more like it, although I’d prefer smaller.

I’ll talk about receiving messages in a second post.

30 Replies to “New “Embedded” Paho MQTT C Client”

  1. Hi Ian!

    Thanks so much for all you’ve achieved and contribute to the MQTT community. I have a very genuine and long term interest in using MQTT for embedded c applications. I consider myself a competent software engineer but for all I have achieved, I haven’t had the experience of managing the makefile.

    I have successfully download the Embedded C library you’ve uploaded to GitHub and am already familiar with the Code Composer toolchain.

    I’m receiving the following compiler error in a “plug and play” effort at compiling the code and was hoping you could help me resolve the error ( see screenshot below)? If its outside the scope of this blog I understand. I did try posting to the Paho mailing list but with no success.

    Screenshot:
    http://houndandhorsecare.co.uk/Capture.JPG

    Many Thanks,
    Oskar

    1. Hi Oskar,

      sorry for the delay in replying. I haven’t created a full build for the embedded library because I envisaged that it would be picked up piecemeal. That is, people would take the source pieces they need and no more, and compile them into their applications. I will look into creating a build mechanism in due course then, since you’ve asked.

  2. Very good idea. The cleaner separation between the application layer protocol and the transport facilitates a better management of exceptions – such as sudden lack of connectivity 🙂

    Thank you
    Regards
    daniel

  3. I built the MQTTPacket library and the qos0pub.c sample. I tried running it and it says it successfully published data to iot.eclipse.org:1883 – Is there a portal where I can get verification the message was actually received?

      1. Hello,
        I was wondering if there’s a suitable mqtt publish client code for PIC32 board. I tried to use the embedded-sn but since I’m using an mpide compiler (ie. pic32-gcc), it cannot resolve some of the libraries (ie. sys/socket.h). Any suggestion if there’s one that’s is more likely to work.

        Thanks much
        Gerry

        1. Hi Gerry,

          you should be able to use the embedded MQTT or MQTT-SN clients, the very low level layers which I call MQTTPacket and MQTTSNPacket at least. I know of at least one person using the embedded MQTT client on a PIC board. Looking at the code for the embedded MQTT-SN client, transport.c really ought to be in the linux directory. Those functions aren’t part of the core MQTTSNPacket library which contains no network code at all, so you would have to implement the functions to send and receive data on the UDP sockets yourself. Are you intending to use UDP?

          Ian

  4. Hi,
    I have an Arduino clone connected to a GSM board (Adafruit Fona). The GSM board have limited support for TCP/IP and HTTP over AT commands (GPRS).

    In the example code Hello.ino, you use Ethernet and IPStack in setup() as arguments to the MQTT::Client:
    EthernetClient c; // replace by a YunClient if running on a Yun
    IPStack ipstack(c);
    MQTT::Client client = MQTT::Client(ipstack);

    Do you have any guidance how I can use the MQTT:Client without the Ethernet or IPStack to publish QoS0 sensor data (connect, publish, disconnect), and let the GSM board Adafruit Fona library do the GPRS (TCP/IP) connectivity setup/teardown?

    Thanks,
    Karim

  5. Hi Ian,

    Thanks for your library, I think I’ve found a bug where the “Session Present” flag isn’t correctly returned from the de-serialise function. The source of the problem seems to be at just passed line 115 in MQTTconnect.h , in the bits struct. I think you’ve got the two ifdef options the wrong way around (compared to the other “#if defined(REVERSED)” statements), swapping them fixes it.

    I think it should be as follows:

    #if defined(REVERSED)
    struct
    {
    unsigned int : 7; /**< unused */
    unsigned int sessionpresent : 1; /**< session present flag */
    } bits;
    #else
    struct
    {
    unsigned int sessionpresent : 1; /**< session present flag */
    unsigned int : 7; /**< unused */
    } bits;
    #endif

    Best Regards,
    William

  6. Hi Ian

    I have a query re your example up top of publishing and disconnecting. It looks like the library compiles full MQTT strings and fires them out via TCP socket to connect, publish and disconnect.

    Does this library account for checking ACKS/ANCKS between messages or are these ignored?

    Thanks for the tutorial!

    Tony

      1. Hi Lan,
        I have added some code to examine the command.as follow:
        if (MQTTPacket_read(buf, buflen, transport_getdata) == CONNACK)
        {
        unsigned char sessionPresent, connack_rc;
        printf(“receive connack\n”);
        if (MQTTDeserialize_connack(&sessionPresent, &connack_rc, buf, buflen) != 1 || connack_rc != 0)
        {
        printf(“Unable to connect, return code %d\n”, connack_rc);
        goto exit;
        }
        }
        else
        goto exit;
        the code is up publish ang disconnect, but it can’t receive connack, and the program straightly goto exit, so can you give me some suggesstions ,thank you very much. (well, the socket has successsfully established.)

        1. You should check the return code from MQTTDeserialize_connack to see if the packet was read and understood properly. If it was 1, then it was. Then check the connack_rc to see if that indicates why the server rejected the connect (the values are in the MQTT specification). You could try connecting to a local server (like Eclipse Mosquitto) to test your application code, because the Mosquitto logs would provide more feedback.

  7. Hi Ian,
    I would like to ask you about my issue in imported embedded c client library to my project. Could you describe me how can I handle ping responses. I call MQTTPacket_read function for wait until the response income, but it cause infinite wait to already read data. All other functions work without trouble.

    Thank your very much,

    Martin Petřík

    1. Hi Martin,

      there are examples of how to handle ping responses in the MQTTClient C++ and MQTTClient-C sub-projects, which use the MQTTPacket library underneath. Maybe those will help?

      Ian

  8. Hello,
    I would like to Build a mqtt-sn client for an cortex m0. So my Idea was to use an existing Embedded c Implementation and port i to the cortex. My Problem is, I do not have any ethernet. I would like to use the serial port only and later maybe CAN or Rs485. So my idea was to build a bridge for that. I found a serial mqtt sn Bridge but i do not found an client with serial only.

    So i will port the code an replace the UDP part with my serial read and write part.

    1) Do you have any suggestion which client from paho fits best for my goal?

    2) In the specification i read from a forwarder. But what is a forwarder used for?

    Thanks for your help greetings from Austria.
    Mathias

    1. Hi Mathias,

      1) the Paho embedded MQTT-SN client is the best starting point — the MQTTSNPacket project.

      2) a forwarder is used to translate from one transport to another – like MQTT-SN over serial to MQTT-SN over UDP. There is a Java example in the MQTT-SN Paho project, but it hasn’t been touched for a while. It did work the last time I tried it.

      Ian

      Ian

  9. Hi,

    can I use the same Client object to do a subscribe and a publish on two different threads?
    If so, I need to use different MQTTPacket_connectData?

    1. Hi Matteo,

      no, for this client code, you would have to have two separate client objects. The asynchronous version, which I haven’t written yet, would be thread safe. Making the client object thread safe would take more resources, which is why it isn’t.

      One option would be to make the thread safety code conditionally compiled, so it wasn’t included when not needed.

      Ian

  10. Hi Ian! Thanks for your work! I’m a student and I’m trying to learn something about this protocol. I’m having some troubles with the little example above:
    when you declare the buf variable, you give it a char type and when you pass it to the function “MQTTSerialize_connect(buf, buflen, &data)” you give it a char* type, but it wants an unsigned char* type and I receive an error from the compiler. I have the same problem with the payload variable in the function “MQTTSerialize_publish”. Thank you very much for your help!

    1. In between writing this example and the present, I changed the type of the buffer accepted by the MQTTSerialize_connect call from char* to unsigned char*. That seemed the best approach. You can make the example compile by changing the declaration of buf (and probably payload) to unsigned char.

  11. Hi lan,
    I have registered a device in the IBM iot foundation.I have successfully download the Embedded C library you’ve uploaded to and modified the transport.c and transport.h to adapt to my own os. so,how can I ues the samle qos0pub to establish a connect with quickstart.messaging.internetofthings.ibmclound.com?

      1. thank you for your help, there is one more question, in the transport.c,you have used the domain analystic function to get the ip (getaddrinfo), in my library,I don’t have such function, so I just ping the address of quickstart. messaging.internetofthings.ibmclound.com,then I get the ip:184.172.124.189,I directly connect to the ip address without using the dns function, but it seems to be useless, can you give me some suggestions? thanks a lot !

  12. Hi lan,
    I have successfully download the Embedded C library you’ve uploaded and built a project including transport.c and the src folder in linux,I tried to connect to the quickstart(host:quickstart.messaging.internetofthings.ibmclound.com), the code has no problem and it can be implemented successfully,but the quickstart page can’t receive the payload ( payload: “{\”d\”:{\”temp\”:34}}”).so can you give me some suggestions, thanks a lot!

Leave a Reply

Your email address will not be published. Required fields are marked *