Lego uses AI to translate instructions into braille and audio

Children use Lego audio and braille building Instructions

The toy giant has launched a global trial with four building sets and hopes to release more next year.

Lego is building its way to making instructions friendlier for the visually impaired by using artificial intelligence (AI) to translate instructions into braille and audio.

The Danish toy company has launched a global trial with four building sets and hopes to release more next year depending on feedback.

Using the design script used by Lego creators, known as LXFML data, the Austrian Research Institute for Artificial Intelligence developed software that is able to turn the instructions into braille and English audio.

The idea came about after blind entrepreneur and Lego enthusiast Matthew Shifrin previously relied on a friend to translate instructions into braille, which enabled him to build independently for the first time.

After his friend died, Mr Shifrin was introduced to Lego’s Creative Play Lab and decided to push for development of the technology.

Matthew Shifrin plays Lego
Matthew Shifrin, who was born blind, inspired the idea to help children with visual impairments (Lego/PA)

This is extremely important for blind children because there aren’t a lot of places where we can say ‘Look Mum and Dad! I built this on my own… I did this’. For blind children, we don’t have access to what sighted kids are used to. Lego bricks enable us to learn about our environment, to see the world. It is so important because blind kids get left out of a lot of social stuff, especially in elementary school. But Lego building is one of the things we can do.

Mr Shifrin, blind entrepeneur and Lego enthusiast

Lego said there is still “a lot of progress” to be made with the AI software and automating the process, but it hopes to introduce more languages in the future.

As I build a set, I develop a better sense of what a building looks like and how it is laid out and constructed. For blind people, Lego sets act as miniature 3D substitutes for real-life buildings in lieu of two-dimensional photographs. Lego bricks allow me to see things that are impossible to explore by touch, such as the arches of a Middle Eastern palace or the towers of the London Tower Bridge.

Mr Shifrin, blind entrepeneur and Lego enthusiast


Child playing Lego
Lego is providing audio and braille building instructions to four sets initially (Lego/PA)

Lego Group creative director Fenella Blaize Charity said: “His project will help visually impaired children around the world experience the same joy of building and pride of creation that all our fans feel.”

Paul Welham, chief executive of CereProc, the speech synthesis company that provided Lego with the Software Development Kit, said: “This fantastic initiative is a major step forward for people living with visual impairment.

“By adopting Text-to-Speech technology in this way, Lego will not only improve the lives of those using its products, but hopefully also inspire other companies to follow suit with characterful synthetic speech.”

If your business is also involved in software development you could be benefitting from R&D (research and development) Tax Credit Scheme which will help your company grow further and continue to innovate. At MPA, our team of technical specialists have the expertise to uncover the research and development often found in daily activities. These attempts to solve technical problems in the bid to create something new, has the potential to unlock cash for any qualifying company through the R&D tax incentive.

Contact us today and speak to one of our industry experts.