AI Makes Smart Lighting Even Smarter

Article By : Sally Ward-Foxton

Today’s smart-lighting systems still must be set up manually by the user. The OpenLicht project has developed a prototype for a more intelligent lighting system...

German research project OpenLicht has successfully developed a smart-lighting system based on open-source software and machine-learning libraries, plus inexpensive hardware, that can automatically adjust lighting in a room based on what the user is doing.

Today’s smart-lighting solutions are based on smart bulbs such as the Philips Hue and Osram Lightify. While they offer some smart features, they generally require manual control by the user via a smartphone app. Some can be programmed (for example, to turn on and off at certain times), but the rules still have to be set up manually, so the basic relationship between user and lighting system is not changed by making it smarter.

Today’s smart-lighting systems still must be set up manually by the user. The OpenLicht project has developed a prototype for a more intelligent lighting system (Image: Infineon Technologies/OpenLicht)

The OpenLicht project, which launched in September 2016, set out to change that by adding artificial intelligence and machine learning techniques to smart lighting. “The project had two main aims: on one hand, advancing research and collaboration of AI-based smart-lighting systems further, and on the other hand, making state-of-the-art technology accessible for everyone, including startups and the maker community,” said project coordinator Juan Mena-Carrillo, R&D manager for smart lighting at Infineon Technologies.

The recently concluded project was funded by the German Ministry of Education and Research (BMBF) with team members Infineon, Bernitz Electronics, Deggendorf Institute of Technology, and the Technical University of Dresden. Infineon and TU Dresden developed the machine learning application and algorithms. Deggendorf developed the graphical interface/app for the project, and Bernitz was in charge of the gateway and the communications among the system’s sensors and actuators.

Smart prototype

The project’s main deliverable was a prototype of a smart-lighting system based on AI, including an adaptive software system with a graphical interface (app) and a Raspberry Pi-based central hardware gateway that handles data processing and all control tasks. The prototype system automatically adjusts the lighting in a room based on the user’s position and activity; this might be different settings for watching TV and reading, for example. The system learns the user’s preferences and responds accordingly. To some degree, it can also respond to situations it has not already encountered and learned about.

The open-source hardware gateway is based on a Raspberry Pi with an expansion board. There is also a miniaturized version, based on a microcontroller architecture, but it requires connection to an OpenLicht gateway (Image: Infineon Technologies/OpenLicht)

The prototype’s central component is its open-source smart-home middleware, based on openHAB, a vendor- and technology-agnostic open-source home automation software platform. Project researchers developed openHAB bindings for the various sensors, including pressure and radar sensors, that are placed in a room to detect occupancy and motion. The sensors push their data to the corresponding bindings, which connect the real-world sensors with the openHAB system and deliver the data to “items” — virtual representations of sensors and actuators. When changes occur in the items, those changes are sent to the open-source machine-learning framework Encog.

A neural network that’s been trained on sensor data processes the changes and intuits the user’s current activity. Its prediction is combined with data on the natural light conditions, and the information is fed to a second neural network, which is self-learning and adapts to users’ preferences while in use. Based on the data that is fed to it, the second neural net determines the appropriate configuration for the lamps in the room. That configuration is then transformed into lamp commands, which are sent via actuator items and bindings to the real-world lamps.

The team set up a demo room to evaluate the prototype system and collect sensor data needed for training the neural network (Image: Infineon Technologies/OpenLicht)

“A user can always adapt the color and light intensity of lamps via a user interface, switch, dimmer, or remote control when she or he is not satisfied,” said Mena-Carrillo. “The system recognizes the change and maps the new configuration to the natural light and activity, which are recognized and measured in the moment of the adaptation. This mapping is then used to retrain the neural network by combining it [the new mapping] with the old data. However, the new data gets higher weight than the old data in the retraining process.”

Privacy problem

Key challenges for the project included the linked issues of security and privacy. A hardware-based trusted platform module (TPM) from Infineon was incorporated to guard against attacks from hackers by encrypting and protecting system integrity. TPMs are security chips based on an international standard for secure processors that are used to store critical data such as passwords and encryption keys, as well as to run encryption algorithms.

“After conducting interviews with many end users, we identified that the question of privacy is one of the major hurdles for the acceptance of smart-home systems,” Mena-Carrillo said. It was therefore decided that the OpenLicht system would use AI at the edge; that is, user data is processed within the smart-home system rather than in the cloud. The technique preserves user privacy because sensitive data will only be processed locally. It also generally enables faster response times and reduces or eliminates the need for an internet connection.

Open source

One of the OpenLicht project’s key aims was to make the technology accessible to industry and the wider community. All of the software is based on open-source technologies: The results are implemented as extensions for openHAB, and open-source machine-learning library Encog was adapted for use in the project. The use of inexpensive hardware was also a deliberate decision to make it accessible. The project’s resulting software itself will be open-sourced imminently.

“Everyone can use our software results, and the great thing is that our system can be enhanced with new features and functions,” Mena-Carrillo said. “These results now allow users to add such new AI functions and features to their openHAB systems.”

All of the OpenLicht software will be available on GitHub “very soon,” including the machine-learning software, knowledge base, and openHAB bindings, he added. While the project met both of its key aims, Mena-Carrillo admitted the team also realized that there is a lot more work to do before such a system works reliably in every circumstance. Given the project’s open-source nature, OpenLicht’s developers hope it will evolve once the industry and maker community get access to it.

Leave a comment