Building a LIFX packet

Continuing the discussion from Controlling lights with Bash:

So I’ve had a few questions about how I built the packets for the bash example. I thought it might be a good tutorial for those who havent had much experience with binary protocols before. Hopefully it gets a few more people over that hurdle and lets them talk directly to the bulbs over LAN.

Before I start there are a few concepts that you will need to be familiar with. Here are a few wikipedia articles you may wish to review:

Binary protocols

Binary protocols usually exist as several layers applied one after the other, with each layer describing details of the next. When all combined together these make up what is called a frame or a packet. The LIFX protocol looks like this too. In the LIFX protocol there are two major sections, the headers and the payload. The headers describe the action that is being taken and the way in which it will work and the payload provides the data for that specific action. Sometimes the action requires no data, and in these cases the payload is not included.

In the LIFX protocol there are three headers that are included in every message.

  • The frame header includes details about how to process the frame itself. This includes the protocol version and the size of the frame.
  • The frame address header includes details about the destination and the processing of the frame.
  • The protocol header describes the type of the payload.

These headers are included in the order above, with the payload included at the end if required.

An example packet

Lets build a packet to change all of the lights on our network to green. We will be representing the packet in hexidecimal format, which means every character corresponds to 4 bits, we will group them in groups of two, representing a full byte.

To do this we first need to build the frame header. You can find the description of what is included in the frame header here. The documentation says we start with 16 bits which indicate the size of the message. Since we don’t have the frame yet we cant possibly know its size. But we do know that the size is stored in two bytes. So lets write them as ? for now. So our packet starts out as:

?? ??

A Binary Field

Next up is a byte split up into several fields, so we will have to manually assemble it ourselves. You will need to do this whenever you encounter fields that are partial bytes. So lets start with two zero bytes:

???? ????  ???? ????

The documentation says that the first two bits here are the message origin indicator and must be zero.

00?? ????  ???? ????

The next bit represents the tagged field. We want this packet to be processed by all devices that receive it so we need to set it to one (1).

001? ????  ???? ????

The next bit represents the addressable field. This indicates that the next header will be a frame address header. Since all of our frames require this it will always be set to 1.

0011 ????  ???? ????

The final 12 bits are the protocol field. This must be set to 1024 which is 0100 0000 0000 in binary. Now our two bytes are complete.

0011 0100  0000 0000

Lets look at them in hexidecimal:

34 00

Back to the packet

Before we add these to the packet there is one final thing to note. The protocol is specified to be little endian. We have been working in big endian for ease of explanation, so we need to switch the two bytes around before we add them to the frame. Now the frame looks like this:

?? ?? 00 34

The next 32 bit (4 bytes) are the source, which are unique to the client and used to identify broadcasts that it cares about. Since we are a dumb client and don’t care about the response, lets set this all to zero (0). If you are writing a client program you’ll want to use a unique value here.

?? ?? 00 34 00 00 00 00

That finishes off the frame header, and we move onto the frame address. The frame address starts with 64 bits (8 bytes) of the target field. Since we want this message to be processed by all device we will set it to zero.

?? ?? 00 34 00 00 00 00 00 00 00 00 00 00 00 00

This is followed by a reserved section of 48 bits (6 bytes) that must be all zeros.

?? ?? 00 34 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

The next byte is another field so follow the steps above to build the binary then hex representations. In this example we will be setting the ack_required and res_required fields to zero (0) because our bash script wont be listening for a response. This leads to a byte of zero being added.

?? ?? 00 34 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Since we aren’t processing or creating responses the sequence number is irrelevant so lets also set it to zero (0).

?? ?? 00 34 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Next we include the protocol header. which begins with 64 reserved bits (8 bytes). Set these all to zero.

?? ?? 00 34 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Now we have to indicate the message type. The different message types are listed in the Device Messages and Light Messages pages of the documentation. We are changing the color of our lights, so lets use SetColor, which is message type 0x66 (102 in decimal). Remember to represent this in little endian.

?? ?? 00 34 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 66 00

Finally another reserved field of 16 bits (2 bytes).

?? ?? 00 34 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 66 00 00 00

Now we have finished with the headers. All that’s left now is the payload. Since our packet is getting quite long I’m going to build the payload separately for now. But imagine it is included at the end of the packet we have been building so far.

The payload starts with a reserved field of 8 bits (1 bytes).

00

Next up is the color described in HSBK. The HSBK format is described at the top of the light messages page. It starts with a 16 bit (2 byte) integer representing the Hue. The hue of green is 120 degrees. Our scale however goes from 1-65535 instead of the traditional 1-360 hue scale. To represent this we use a simple formula to find the hue in our range. 120 / 360 * 65535 which yields a result of 21845. This is 0x5555 in hex. In our case it isn’t important, but remember to represent this number in little endian.

00 55 55

We want maximum saturation which in a 16bit (2 byte) value is represented as 0xFFFF.

00 55 55 FF FF

We also want maximum brightness.

00 55 55 FF FF FF FF

Finally we set the Kelvin to mid-range which is 3500 in decimal or 0x0DAC.

00 55 55 FF FF FF FF AC 0D

The final part of our payload is the number of milliseconds over which to perform the transition. Lets set it to 1024ms because its an easy number and this is getting complicated. 1024 is 0x00000400 in hex.

00 55 55 FF FF FF FF AC 0D 00 04 00 00

So there we are, with 36 bytes of header information and 13 bytes of payload we have a frame that is 49 bytes big (or 0x31 in hex). So lets combine the two and fill in the size field.

31 00 00 34 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 66 00 00 00 00 55 55 FF FF FF FF AC 0D 00 04 00 00

And we have a packet! If you add the escaping to it that was on the bash post you get:

allgreen.echo

\x31\x00\x00\x34\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x66\x00\x00\x00\x00\x55\x55\xff\xff\xff\xff\xac\x0d\x00\x04\x00\x00

Of course building packets by hand is time consuming and annoying, best to use a computer. :stuck_out_tongue:

7 Likes

This answer is so perfect, really. thanks ! :heart_eyes:

Totally agree but any script is in fact only building the packet, right ?

Btw, i’ve read that you are a Python developper. Would you maybe write a code example ? like you published for the HTTP api in PHP ?

Exactly, any application that wishes to talk to a LIFX bulb will perform similar operations to build a packet and then send it. Most applications will also have to keep track of where the bulbs are, perform discovery, retransmit dropped packets and more.

Here is an example of building the frame header from the example above using the python bitstruct library.

from bitstruct import pack, byteswap, calcsize
from binascii import hexlify

frame_header_format = 'u16u2u1u1u12u32'
frame_header_byteswap = '224'

# Encode a frame header
def make_frame_header(size, origin, tagged, addressable, protocol, source):
    be_header = pack(frame_header_format, size, origin, tagged, addressable, protocol, source)
    return byteswap(frame_header_byteswap, be_header)

def sizeof_frame_header():
    return calcsize(frame_header_format)

packet = make_frame_header(49, 0, 1, 1, 1024, 0)

print "Frame Header:"
print hexlify(packet)
print "Frame Header Size:"
print sizeof_frame_header() / 8

You should be able to follow a similar process to build the rest of the headers, the payload and all the types you need.

A minor correction…

In the opening post, in the section entitled “A Binary Field”, there are two paragraphs which begin with the text “The next byte represents…” whereas they should say “The next bit represents…”.

Thanks! I’ve updated the post to include your correction.

It might be worth pointing out the significance of:

frame_header_byteswap = ‘224’

As I couldn’t find any documentation about this that made it clear to me.

The ‘224’ tells the byteswap operation how the header/payload is divided up, so that it knows which parts to swap. The first ‘2’ tells it that the first section of the header is in 2 bytes (i.e. 00 31), which need to be swapped around (31 00).

The next ‘2’ tells it that the second section of the header is also in two bytes (34 00) that need to be swapped to (00 34).

The ‘4’ then tells it that the rest of the header is in 4 bytes. In our frame header, these bytes are 00 00 00 00, but let’s pretend that they are 12 34 56 78. In this case, byteswap would swap them to 78 56 34 12.

So: when you are creating your other headers, (I think) you should use:

frame_address_format = ‘u64u48u6u1u1u8’
frame_address_byteswap = ‘8611’

protocol_header_format = ‘u64u16u16’
protocol_header_byteswap = ‘822’

And if your payload is a SetColor message, then:

payload_format = ‘u8u16u16u16u16u32’
payload_byteswap = ‘122224’

(I am feeling very pleased with myself for working this out from first principles… and having said that - I really hope I’ve got it right?).

And here’s my code for a LAN protocol colour-changing script, in case anyone finds it useful. Feedback would be 100% welcome, as I’m sure I’ve done a lot of this in a dumb way:

    import socket
    
    from bitstruct import pack, byteswap, calcsize
    from binascii import hexlify
    
    frame_header_format = 'u16u2u1u1u12u32'
    frame_header_byteswap = '224'
    
    frame_address_format = 'u64u48u6u1u1u8'
    frame_address_byteswap = '8611'
    
    protocol_header_format = 'u64u16u16'
    protocol_header_byteswap = '822'
    
    def send_packet(packet):
        # Broadcast the packet as a UDP message to all bulbs on the network. (Your broadcast IP address may vary?)
        sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
        sock.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)
        sock.sendto(packet, ("192.168.0.255", 56700))
    
    def sizeof(format):
        return calcsize(format)
    
    def make_frame_header(size, origin, tagged, addressable, protocol, source):
        # Make the frame header, specifying the size of the whole packet, the origin, whether it's tagged and addressable,
        # what protocol it uses, and what source it comes from)
        unswapped_header = pack(frame_header_format, size, origin, tagged, addressable, protocol, source)
        #print "\nunswapped frame header: %s" % hexlify(unswapped_header)
        frame_header = byteswap(frame_header_byteswap, unswapped_header)
        #print "frame header: %s" % hexlify(frame_header)
        return frame_header
    
    def make_frame_address(target, ack_required, res_required, sequence):
        # Make the frame address header, specifying the target, whether acknowledgement or response are required, and what
        # number packet in the current sequence this is.
        unswapped_header = pack(frame_address_format, target, 0, 0, ack_required, res_required, sequence)
        #print "\nunswapped frame address: %s" % hexlify(unswapped_header)
        frame_address = byteswap(frame_address_byteswap, unswapped_header)
        #print "frame address: %s" % hexlify(frame_address)
        return frame_address
    
    def make_protocol_header(message_type):
        # Make the protocol header, specifying the message type.
        unswapped_header = pack(protocol_header_format, 0, message_type, 0)
        #print "\nunswapped protocol header: %s" % hexlify(unswapped_header)
        protocol_header = byteswap(protocol_header_byteswap, unswapped_header)
        #print "protocol header: %s" % hexlify(protocol_header)
        return protocol_header
    
    def set_color(hue, saturation, brightness, kelvin, duration):
        # Set the colour of the bulb, based on the input values.
    
        # Set the format of the payload for this type of message
        payload_format = 'u8u16u16u16u16u32'
        payload_byteswap = '122224'
    
        # Use the payload format and the header formats to calculate the total size of the packet, in bytes
        packet_size = (sizeof(frame_header_format + frame_address_format + protocol_header_format + payload_format)) / 8
        #print "\npacket size is %s" % packet_size
    
        # CREATE THE HEADERS
        # 1. Frame header: use packet_size to indicate the length, set origin to 0, set tagged to 1 (because we want all bulbs
        # to respond to it), set addressable to 1, set protocol to 1024, set source to 0 (because we're a dumb client and
        # don't care about responses).
        frame_header = make_frame_header(packet_size, 0, 1, 1, 1024, 0)
        # 2. Frame address: set target to 0 (because we want all bulbs to respond), set ack_required and res_required to 0
        # (because we don't need an acknowledgement or response), and sequence number to 0 (because it doesn't matter what
        # order in the sequence this message is).
        frame_address = make_frame_address(0, 0, 0, 0)
        # 3. Protocol header: set message type to 102, which is a "SetColor" message.
        protocol_header = make_protocol_header(102)
        # 4. Add all the headers together.
        header = frame_header + frame_address + protocol_header
    
        # CREATE THE PAYLOAD
        # 1. Convert the colours into the right format
        hue = int((float(hue) / 360) * 65535)
        saturation = int(float(saturation) * 65535)
        brightness = int(float(brightness) * 65535)
        kelvin = int(kelvin)
        duration = int(duration)
        #print "\nhue %s %s\nsaturation %s %s\nbrightness %s %s\nkelvin %s %s\nduration %s %s" % (hue, hex(hue), saturation, hex(saturation), brightness, hex(brightness), kelvin, hex(kelvin), duration, hex(duration))
        # 2. Pack the payload information
        unswapped_payload = pack (payload_format, 0, hue, saturation, brightness, kelvin, duration)
        #print "\nunswapped payload: %s" % hexlify(unswapped_payload)
        payload = byteswap(payload_byteswap, unswapped_payload)
        #print "payload: %s" % hexlify(payload)
    
        # CREATE THE PACKET AND SEND IT
        packet = header + payload
        send_packet(packet)
    
    def input_validation(prompt,lower,upper):
        # Checks that the input is a number (float), and that it's in the required range.
        validation = False
        while validation == False:
            try:
                variable = float(raw_input(prompt + " (%s to %s) " % (lower, upper)))
                if not lower <= variable <= upper:
                    print "You need to enter a number between %s and %s." % (lower, upper)
                else:
                    validation = True
            except ValueError:
                print "You need to enter a number."
        return variable
    
    hue = input_validation("What hue should the bulbs have?", 0, 360)
    saturation = input_validation("What saturation should the bulbs have?", 0, 1)
    brightness = input_validation("How bright should the bulbs be?", 0, 1)
    kelvin = input_validation("What temperature should the light be?", 2500, 9000)
    duration = input_validation("How many milliseconds should the bulbs take to change?", 0, 2147483647)
    
    set_color(hue, saturation, brightness, kelvin, duration)

Almost worked perfectly on the bulbs in the LIFX office, except for one small thing…

On a Linux system you have to specifically set the socket to allow broadcasts before you send them. So I just had to add a line here:

    sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    sock.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)
    sock.sendto(packet, ("192.168.0.255", 56700))

Ah, cool - thanks @daniel_hall. It seems to work fine without on Windows, but I’ll add this line to my code anyway [edit: done]

Next comes the trickier challenge of receiving and deciphering packets from the bulbs themselves! I am learning a lot this week…

I might be able to save you the trouble there. :smiley: After this thread thread started I noticed that my examples covered most of the SDK and I could extend it slightly and cover all packets. Anyway, the bit that I just finished (like just then) is the packet creation and parsing. Ill put it on PyPI eventually.

You can find it here: GitHub - smarthall/python-lifx-sdk: An SDK for local LAN control of bulbs, using Python

This is cool! I will play around with it as soon as I get the chance - hopefully this weekend. Great work!

After stumbling across the LAN protocol documentation, I wrote the Python package lifxlan for direct control over the LAN and published it to PyPi!

I should have checked here first though, Daniel’s code stubs are elegant. It would have saved me some effort and resulted in more beautiful code. Looking forward to refactoring now, at least!

If any of you have a setup with multiple LIFX bulbs, if you could run examples/rainbow_all.py or examples/blink_all.py and let me know how they work out, that would be awesome!

Thanks. I really appreciate your feedback. I don’t often get told that my code looks nice. :smiley:

I can’t see a license for your repository, but if you are happy with an MIT or a GPL license then feel free to pull my code directly into your repo if it helps. You can never have too many implementations of something. Also my job here is to empower developers like yourself, so feel free to reach out if you have any questions.

I’ll give your code a quick test tomorrow with the lights in the LIFX office and let you know how it goes.

The license is MIT, but that was buried in setup.py. I’ve added an explicit license file to make it clear!

Thanks for the encouragement! This is a super-friendly community. Looking forward to hearing how the test goes. :smile:

Hi mclarkk. Can you spend time and help me. My situation is strange and i can control lifx only from linux pc. Tried to use lifxlan - all works fine. But i need simple script to send on/off/color directly to my 1 bulb. I’am not programmer - here is the problem. If you can help me, please email me at brutevinch@icloud.com

If you have a question about a specific library it might be a good idea to start a new thread about it…

If you need to notify someone in particular you can place an @ in front of their forum name and the system will email them to let them know.

@brutevinch I’d be happy to add some scripts to the examples. If there are simple scripts that would be helpful to you, I’m sure you’re not the only one they would help. I’ll email you to get a better idea of what you have in mind.

im new to networking for lan and stuff, could you show me a an example lifx packet that broadcast to the LAN and recieves a list off lifx ip’s

@rasputin303 @daniel_hall

Hi Guys, I’m new to python and trying to learn it specifically to experiment with the LAN protocol.

Ed and Daniel i really like the work that you have put into the script on this page and have it working with plans to hopefully turn it into something of my own.

I don’t know if anyone else has experienced this but when i run the program and input the values, one of my 2 bulbs changes colour instanly, however it takes about a second or two for the second bulb to change. Do you have any idea why this would be the case.

Cheers,
Darren

I don’t know if there’s a code-related reason why this should happen - I’m a Python noob too, and that script was only the second thing I ever wrote. But in theory, the bulbs should change at the same time. The code is broadcasting to the whole network at once, it’s not sending messages sequentially to one bulb at a time - so there’s no reason that one bulb should change before the other. My guess is that because UDP is a pretty flaky protocol, sometimes messages bounce around the network for a bit before the bulbs pick them up. But I will defer to people like @daniel_hall who actually know what they’re talking about!