Lightsd: a daemon with a JSON-RPC API to control your bulbs

lightsd: a daemon with a JSON-RPC API to control your bulbs

I’m very pleased to announce the first release of about a year of work:

lightsd is a background service (daemon) that will discover LIFX bulbs on your network and expose a JSON-RPC interface over TCP/IP, Unix sockets or a pipe to control them.

lightsd implements the original LIFX LAN protocol and works across all different versions of the bulbs:

  • firmwares tested: 1.1, 1.5, 2.0, 2.1;
  • models tested: A21 (Original), A19 (White 800), GU10 (Color 650).

It supports the main functionalities of the bulbs:

  • power: on/off/toggle;
  • color/label: get/set;
  • set the color according to a function (SAW/SINE/TRIANGLE/SQUARE);
  • tag/untag.

If you are familiar with Mac OS X’s Homebrew or Arch Linux’s AUR you will get lightsd running in a minute, otherwise, it’s fine, everything is included in the documentation:

http://lightsd.readthedocs.org/en/latest/

I started lightsd with some original design goals:

  • no discovery delay ever: that was a major frustration point to me;
  • run my bulbs on a different Wi-Fi network: an effective safety measure;
  • portability & performances: lightsd is written in C and designed to directly run directly on your Wi-Fi access point;
  • automatically retry missed commands: lightsd watches the state of your bulb and make sure it changes (only implemented for power on/off/toggle).

But lightsd also features:

  • a simpler API: since lightsd takes care of all the background tasks (discovery, state sampling, etc.) the API is reduced to a more simple set of actions;
  • interoperability: lightsd can support different models of bulbs behind the same API;
  • an extensible design: the API could be supported over something else than TCP or and could be something else than JSON-RPC.

With this release, lightsd enters a maintenance cycle: I wanna see how the LIFX community welcomes the project, improve my tooling around it, package it for other platforms (OpenWrt and Debian maybe) and take a look at the latest firmware changes (I’d love to directly talk with the devs at LIFX). But here is some ideas I already have:

  • firmware support: some APIs have changed in recent firmwares;
  • instrumentation: I need to measure how well lightsd is working;
  • implement auto-retry across all commands and a packet queue optimizer;
  • json-rpc extensions: subscribing to events is something for which lightsd was also designed for;
  • Android/iOS support: since lightsd is written in C it will run on mobile devices without too many efforts;
  • support another model of bulbs, Phillips hue maybe?

Thanks for reading me this far, let me know if you have any question and/or feedback!

Release history

11 Likes

This looks really cool, and there have been a dozen or so people asking for something like this, so I’m sure you will get users.

On the topic of talking directly to the LIFX devs, you already are! I’m one of the engineers here at LIFX and the rest of us lurk here and occasionally pop up and answer questions.

Nice, I wish I could reach out to those users!

I’m gonna go ahead with some questions I have then:

  • What’s the firmware situation on the White 800? Looks like the latest version for those bulbs is 1.5, what are the differences with the firmware 1.5 for the LIFX Original (if any)?
  • It looks like the TagLabel API to tag/untag bulbs has been deprecated how does its replacement work?
  • For how long the TagLabel API will be maintained? Is it even compatible with the new API?
  • How can I get access to the source code of the bulbs firmwares? (I’d be ready to work out some legalese for that)
  • I’d like to investigate real-time oriented features have you guys experimented with that?
  • You need to power on a bulb before you can change its color this is really annoying to restore a scene from turned off bulbs, is there a way to fix that?

Thanks!

I’m not sure what exactly you meant by this, do you mind expanding on it?

Sure, by real-time I meant being able to control the bulb in a way that the lag between the user input and the bulb reaction becomes imperceptible (very much like a non-smart bulb works).

Well on Raspbian the latest version of make is 2.8.9 but you have a dependency of 2.8.11

Won’t work on the Pi.

Here is a patch that should make it work, i’ll include it in a 1.0.1 release tonight/this week-end. From the root of the clone repository, copy paste the following command:

patch -p1 << 'EOF'
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 9643d1e..31fa2ae 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -1,5 +1,5 @@
-# first version with TARGET_INCLUDE_DIRECTORIES:
-CMAKE_MINIMUM_REQUIRED(VERSION 2.8.11)
+# it probably works with older versions but this the oldest tested one:
+CMAKE_MINIMUM_REQUIRED(VERSION 2.8.9)

 PROJECT(LIGHTSD C)

@@ -87,7 +87,16 @@ INCLUDE_DIRECTORIES(
 ADD_SUBDIRECTORY(compat)
 ADD_SUBDIRECTORY(core)
 ADD_SUBDIRECTORY(lifx)
-ADD_SUBDIRECTORY(tests)
+# 2.8.11 is the first version with TARGET_INCLUDE_DIRECTORIES:
+IF (CMAKE_VERSION VERSION_GREATER 2.8.10)
+    ADD_SUBDIRECTORY(tests)
+ELSE ()
+    MESSAGE(
+        STATUS
+        "The tests suite requires CMake >= 2.8.11 "
+        "but you have ${CMAKE_VERSION}, disabling it"
+    )
+ENDIF ()

 INSTALL(
     FILES COPYING README.rst docs/protocol.rst
diff --git a/lifx/wire_proto.h b/lifx/wire_proto.h
index 1a03bff..3ad5a87 100644
--- a/lifx/wire_proto.h
+++ b/lifx/wire_proto.h
@@ -29,7 +29,7 @@ static inline floatle_t
 lgtd_lifx_wire_htolefloat(float f)
 {
     union { float f; uint32_t i; } u = { .f = f };
-    htole32(u.i);
+    u.i = htole32(u.i);
     return u.f;
 }

@@ -37,7 +37,7 @@ static inline floatle_t
 lgtd_lifx_wire_lefloattoh(float f)
 {
     union { float f; uint32_t i; } u = { .f = f };
-    le32toh(u.i);
+    u.i = le32toh(u.i);
     return u.f;
 }
EOF

The actual patches can be found my patch queue:

Anyway, you should you consider upgrading to the latest version of Raspbian! The latest version of Raspbian (Debian Jessie) has 3.0.2.

edit: I, accidentally, a word.

1 Like

Thanks I will give this a go today. I have not seen much about Jessie yet but know it is out there so that was my next bit of investigation.

Thanks

@boxhead, I just released lightsd 1.0.1 it should solve your build issue with CMake 2.8.9, let me know!

1 Like

worked well thanks, now to find time to use it.

1 Like

Hai there,

This thing looks really promissing, especialy the hue compatibility.

I have at home a mix of lifx/hue/limitless(strip) and i’d love to be able to control them all via one single API (with groups and in sync ?).

I’m using Jeedom at home as home automation system, and i’m sure it will work fine with your daemon.

some ideas :

  • Ability to turn on the bulb with a set color at once
  • Use RGB or HEX codes to set color.
  • Set multiples bulbs to different colors at the same time (arrays ?), or even better : give them a sequence with the ability to loop trough it !

Hi lopter!

This is brilliant work, and has made my HA system radically more responsive. Thank you very much for writing and sharing lightsd!

For HA, I’m using a Vera (lite) and sending lightsd messages with custom lua in startup and scenes. Happy to share what I’ve got, if anyone’s interested. It’s not packaged and pretty casual (read: sloppy)

Yes, the idea would totally be to support groups across different brand of bulbs as well too. Really just control any kind of bulbs at once from the same API. I wont be able to write and maintain all of that by myself though!

You can batch set_light_color_from_hsbk and power_on to turn on a bulb and set its color at once. You can also use batch to set bulbs to different colors at the same time.

You can just write your own small script or program to loop through a sequence, you just need to keep a connection open with lightsd. The Python example shipped with lightsd can be a great starting point. The documentation covers its usage.

I’m open to support other type of color representation, I’ll probably add RGB.

Thank you so much, glad you find it useful!! Responsiveness is definitely one of my main concern and a challenging issue.

Nice! So you run lightsd somewhere on your network? and control it from the VeraLite controller using lua?

Feel free to share your code! I’m always happy to learn about people use cases and see how they use lightsd or the bulbs in general.

In other news, I packaged lightsd for OpenWRT (trunk) and I’ll release that with the next version of lightsd, I’d like to solve open issues first.

Cool, I’m happy to share - if anyone has any suggestions for improving my mess, I won’t take offense :smile:

Notes:

  • I’m using the “MultiString” plugin to create persistent variables I can reference elsewhere and manipulate with the UI - this is essentially legacy now, and I’ll likely write around them shortly and just implement is a multi-dimensional lua table.

  • The MultiString objects I’m using currently are laid out as 1 conf device for a set of lights (interior vs exterior in my setup).
    Example MS conf device (variable1 … variableN):

Enabled: [1|0]
ColorString: "h,s,b,k,t"

Example room device:

LIFXGroupName=Office
LastMovementTime=1444694044
Delay=300
Unoccupied=0
ConfDevice=67
  • My lightsd instance is running at 192.168.1.149, listening on tcp 8080

  • I’m using occupancy sensors throughout my house to turn up lights with lightsd, or trigger light events as alerts (ala “flash blue when porch motion sensed”), but the Lifx cloud to turn them off at the moment, as the cloud API has cool support for a very long duration fade then power off. I can do the fade with lightsd, but I’d have to read state to know when to power off, complicating things and exceeding my meager lua skills (~2 hours’ reading so far)

  • Each sensor has a scene which turns lights on (if the lights are set to auto), and there’s a cleanup process that starts them shutting down after occupancy isn’t detected for a period. I found this to work pretty well, and lights don’t turn off on me.

Here’s my Vera (UI7) startup script

function isempty(s)
  return s == nil or s == ''
end

function lightsd_command (json)
    socket = require "socket"
    address, port = "192.168.1.149", 8080

    tcp = socket.tcp()
    tcp:connect(address, port)

    tcp:send(json)

    tcp:close()
end

function lifx_command (method, params)
    local io = require "io"
    local json = require "akb-json"
    local mime = require "mime"

    local devurandom = io.open("/dev/urandom", "r")
    local urandom = devurandom:read(8)
    devurandom:close()

    json_id = mime.b64(urandom)

    jt={jsonrpc="2.0",
        method=method,
        params=params,
        id=json_id}

    json_obj=json.encode(jt)

    lightsd_command(json_obj)
end

function lifx_multi_command (batch)
    local io = require "io"
    local json = require "akb-json"
    local mime = require "mime"
    local jb = {}
    local pos = 1

    for method, params in pairs(batch) do
      local devurandom = io.open("/dev/urandom", "r")
      local urandom = devurandom:read(8)
      devurandom:close()

      json_id = mime.b64(urandom)

      jt={jsonrpc="2.0",
          method=method,
          params=params,
          id=json_id}
      jb[pos]=jt
      pos = pos + 1
    end

    json_obj=json.encode(jb)

    lightsd_command(json_obj)
end

--example usage
--lifx_multi_command({power_off={"#Office"}, power_on={"OfficeTorch"}})

-- Color array managed in manual: autolights next color scene
lifx_colors = { "0,0,1,3700,1000" }
lifx_color_sel = 1
lifx_ext_colors = { "0,0,1,3700,1000" }
lifx_ext_color_sel = 1


local lifx_groups = {}
lifx_groups["BackYard"] = {"BackPorch"}
lifx_groups["DiningRoom"] = {"DiningRoom1", "DiningRoom2"}
lifx_groups["FrontBedroom"] = {"FrontBedroom1", "FrontBedTorch"}
lifx_groups["FrontYard"] = {"FrontPorchColor"}
lifx_groups["Garage"] = {"GarageNorthRear", "GarageNorthFront", "GarageRear", "GarageSideSouth"}
lifx_groups["Hallway"] = {"HallSconce"}
lifx_groups["Kitchen"] = {"KitchenColor1", "KitchenColor2", "KitchenColor3"}
lifx_groups["LivingRoom"] = {"LureLamp", "LivingRoomSconce"}
lifx_groups["MasterBedroom"] = {"MasterBedAneka", "MasterBedJon"}
lifx_groups["Office"] = {"OfficeTorch", "OfficeFloorLamp"}
lifx_groups["Interior"] = {"DiningRoom1", "DiningRoom2", "FrontBedroom1", "FrontBedTorch", "HallSconce", "KitchenColor1", "KitchenColor2", "KitchenColor3", "LureLamp", "LivingRoomSconce", "MasterBedAneka", "MasterBedJon", "OfficeTorch", "OfficeFloorLamp"}
lifx_groups["Exterior"] = {"BackPorch", "FrontPorchColor"}
lifx_groups["Alerts"] = {"BackPorch", "DiningRoom1", "FrontBedTorch", "GarageRear", "HallSconce", "KitchenColor1", "LivingRoomSconce", "MasterBedJon", "OfficeTorch"}

for k, v in pairs(lifx_groups) do
        local tagname = k
        local bulbs = v
        i=1
        for i=1, #bulbs do
          local bulb = bulbs[i]
          lifx_command("tag", {bulb, tagname})
        end
end

function string:split( inSplitPattern, outResults )
  if not outResults then
    outResults = { }
  end
  local theStart = 1
  local theSplitStart, theSplitEnd = string.find( self, inSplitPattern, theStart )
  while theSplitStart do
    table.insert( outResults, string.sub( self, theStart, theSplitStart-1 ) )
    theStart = theSplitEnd + 1
    theSplitStart, theSplitEnd = string.find( self, inSplitPattern, theStart )
  end
  table.insert( outResults, string.sub( self, theStart ) )
  return outResults
end

function lifxLightGroupOn (device)
  if (luup.is_ready(device) == false) then
        return
    end
  local confdevice = luup.variable_get("urn:upnp-org:serviceId:VContainer1","Variable5", device)
  if isempty(confdevice) then
      return
    end
  if (luup.is_ready(tonumber(confdevice)) == false) then
        return
    end
  local enabled = luup.variable_get("urn:upnp-org:serviceId:VContainer1","Variable1", tonumber(confdevice))
  local room = luup.variable_get("urn:upnp-org:serviceId:VContainer1","Variable1", device)
  local color = luup.variable_get("urn:upnp-org:serviceId:VContainer1","Variable2", tonumber(confdevice))

  if isempty(room) then
      return
    end

  if enabled and enabled ~= '' and enabled ~= "0" then

    if isempty(color) then
      color = "0,0,1,3700,200"
    end

    colortab=color:split(",")
    i=1
    for i=1, #colortab do
      local color = tonumber(colortab[i])
      colortab[i] = color
    end

    local t = os.time()
    luup.variable_set("urn:upnp-org:serviceId:VContainer1", "Variable2", tostring(t), device)
    luup.variable_set("urn:upnp-org:serviceId:VContainer1", "Variable4", "0", device)

    lifx_command("set_light_from_hsbk", {"#" .. room, colortab[1], colortab[2], colortab[3], colortab[4], colortab[5]})
    lifx_command("power_on", {"#" .. room})
    lifx_command("set_light_from_hsbk", {"#" .. room, colortab[1], colortab[2], colortab[3], colortab[4], colortab[5]})
  end
end

function lifxLightGroupOff (device)
  if (luup.is_ready(device) == false) then
        return
    end
  local confdev = luup.variable_get("urn:upnp-org:serviceId:VContainer1","Variable5", device)
  if isempty(confdev) then
      return
    end
  if (luup.is_ready(tonumber(confdev)) == false) then
        return
    end
  local enabled = luup.variable_get("urn:upnp-org:serviceId:VContainer1","Variable1", tonumber(confdev))

  if enabled and enabled ~= '' and enabled ~= "0" then
      local room = luup.variable_get("urn:upnp-org:serviceId:VContainer1","Variable1", device)
      local delay = luup.variable_get("urn:upnp-org:serviceId:VContainer1","Variable3", device)
      local unoccupied, last_t = luup.variable_get("urn:upnp-org:serviceId:VContainer1","Variable4", device)

      if isempty(room) then
          return
        end
      if isempty(delay) then
          return
        end

      local t = os.time()

      if unoccupied and unoccupied ~= "" and unoccupied ~= "0"  and unoccupied ~= "1" and t >= (last_t + tonumber(delay)) then
        color = {0, 0, 0, 3700, 60000}
        --lifx_command("set_light_from_hsbk", {"#" .. room, color[1], color[2], color[3], color[4], color[5]})
        os.execute('curl -ku "token:" -X PUT -d "state=off;duration=30" "https://api.lifx.com/v1beta1/lights/group:' .. room .. '/power"')
        luup.variable_set("urn:upnp-org:serviceId:VContainer1", "Variable4", "1", device)
      end
  end
end

function roomUnoccupied (device)
  local t = os.time()
  luup.variable_set("urn:upnp-org:serviceId:VContainer1", "Variable4", tostring(t), device)
end

The lights on scenes look like:

local device = 72
lifxLightGroupOn(device)

Each coupled with an unoccupied scene for the same sensor:

local device = 72
roomUnoccupied(device)

And this cleanup scene which is scheduled to run every 15 sec across the MultiString devices for all rooms (don’t go more frequent than you need - the UI7 doesn’t protect you from creating queue runaway):

local devices = {68, 69, 70, 71, 72, 73, 75, 76, 77, 86}
local i
local dev

i=1
for i=1, #devices do
  local dev = devices[i]
  lifxLightGroupOff(dev)
end

return true

Here’s an example notification scene - this one triggers when the front porch sensor is tripped (only notifies specific lights, if each light’s on):

local hue = 180
local count = 2
local freq = 200

lifx_command("set_waveform", { "#Alerts", "TRIANGLE", hue, 1, 1, 3700, freq, count, 0.5, true})

return true

I hope someone finds this valuable - feel free to ask for more info if I’ve left anything out!

Some rendering issues with the code above - mind the weird linebreaks, they aren’t intended. :wink:

This is really cool! Yeah, motion sensors is totally an usage that requires responsiveness.

I have a Nest smoke detector that has one, I’d love to use it with lightsd but they don’t have any API: Home - Google Nest Community. :disappointed:

I see how the LIFX API is better here, it’s a good idea, I’ll add it to lightsd: GH-5.

Oh, I also see that you are never reading responses back, if you do that and re-use the same lightsd connection for too many commands you’ll probably run into issues as some socket buffer somewhere will become full.

Thank you for sharing all of that!

edit: For the power duration argument, if you only use lightsd to control your bulbs you can also just set the brightness to 0 with a duration to emulate the power off. This doesn’t collaborate well with other LIFX apps though.

1 Like

In case anyone wondered, lightsd 1.0.1 works with the new Color 1000 bulb!

I just received one and everything is working, including tag/untag even though I’m using their old API for that.

You will need to add it to your Wi-Fi network through the LIFX app first.

2 Likes

Great work, thank you!!

I just got lightsd working on my Raspberry Pi with very little effort. I had previously setup my windows machine to run a ruby script every hour to change the lights on/off/color/brightness based on preferences and also based on that collaboration article that lifx did on circadium rhythm and light. Anyway, updating to windows 10 made ruby very difficult and I have wanted to move the work to Pi anyway.

I was hoping to either use this forum for some more detailed questions or perhaps get you my email so you can add any of the ruby work into your work (probably along the lines of your sh and py scripts). I have 15 lights and ultimately would like to tie in a bit more automation with my nest thermostat, CO detector and then trying to decide on switch/motion sensors (insteon sort of in the running…no real clear winners yet although my UPB switches seem to have lost the battle), …but I’m patient :smile:

Anyway, I really like the daemon element to this project for the reasons you have listed although I have already found that some of my lights are not consistently registering and so hoping to make sense of that and then any sort of generic ruby code to help others would be helpful I hope. I may test PERL as well since ruby and json rpc are surprisingly tricky to get going (I just used json with the 4 elements for rpc…super easy once I gave up on the other clients…it was a quoting issue which I gave up on once I realized how hard I was making it).

Anyway, I know this is hobbyist stuff so don’t mean to blast you with too much of my stuff…great work and thanks again! My goal with Ruby is only to save a bit of work from my previous automation and also there was a really great ruby API which is no longer being supported as I understand so this would be a real simple conversion if I can make time to convert things properly. I honestly have no idea how many people use the old ruby API, but it worked very well for about a year for me!

Winter

1 Like

This is good to hear! I wanna give a shot to a native Windows port; I like writing portable code.

Absolutely, right now, this is the best place to ask questions about lightsd. I hope @daniel_hall doesn’t mind.

Feel free to share your work! The easiest way to do it is to use places like github, bitbucket or gitlab they make it really easy to share code.

Whether it’s for questions or code, I’ll still take emails too, you’ll find my address in lightsd’s source code.

I actually know little about home automation, I think I’m having enough fun and problems to solve with the bulbs right now! But, I’m definitely very interested in hearing other people use cases and help integrate lightsd with other systems.

Very interested in hearing more details about your issues! Which models of bulb are you using? I’m also interested in the firmware version. Both are returned with get_light_state. I’m having issues with non-original bulbs, I’m trying to troubleshoot them in another topic.

Did you check out the example Python client? With 15 bulbs, you’ll need to increase the buffer size here quite a bit if you wanna try it. I’ll make it more official at some point, I definitely hope to see people come with other clients!

It’s all good, getting feedback and learning about what people try to build is really useful.

Thanks, for giving lightsd a chance!

I’m perfectly fine with it. Feel free to support lightsd here or in other threads in these forums. If it gets too noisy we will give you your own subcategory. :smiley:

2 Likes