Microsoft creates a physical programming language inclusive of visually impaired children

As written on blogs.microsoft.com

These days, most kids get their first introduction to coding through simplified tools that let them drag and drop blocks of commands, creating programs that can do things like navigate mazes or speed through space.

A team of Microsoft researchers and designers in the company’s Cambridge, UK, lab is taking that concept one step further. The team has created what they are calling a physical programming language. It’s a way for kids to physically create code by connecting pods together to build programs.

The system, called Project Torino, is designed to make sure that kids who have visual impairments or other challenges can participate in coding classes along with all their classmates. But Cecily Morrison, one of the researchers working on the project, is hoping the system also will be appealing and useful for all learners, regardless of whether they have visual impairments or other challenges.

“One of our key design principles was inclusion. We didn’t want to isolate these kids again,” she said. “The idea was to create something that a whole mainstream class could use, and they could use together.”

The ultimate goal is even more ambitious: To get more kids with visual impairments and other challenges, such as dyslexia or autism, on the path to becoming software engineers and computer scientists.

“It’s clear that there’s a huge opportunity in professional computing jobs,” Morrison said. “This is a great career for a lot of kids who might have difficulty accessing other careers.”

A project like this can serve two goals: Technology companies say they are struggling with a “digital skills gap” that is leaving them without enough engineers and coders to meet their needs, and experts say it can be difficult for visually impaired people to find meaningful, accessible career paths.

The World Health Organization estimates that 285 million people worldwide are blind or visually impaired, and the vast majority of those people live in low-income settings. In the United Kingdom alone, the Royal National Institute of Blind People says only one in four working age adults who are blind or partially sighted are doing paid work.

Steve Tyler, head of solutions, strategy and planning for the Royal National Institute of Blind People, which is working with Morrison on the project, said coding has often been thought of as a promising career path for people with visual impairments. In recent years, however, computer science has come to rely much more on pictorial, graphical and conceptual coding methods, making it harder for kids with visual impairments to get exposed to the field.

Tyler said systems like Project Torino could help provide that path.

“This, for us, was a core reason for running with a project like this and supporting it,” Tyler said.

Tyler, who has a background in education, also said there is currently a woeful lack of resources for visually impaired children who have an interest in coding or more generally are ready for an introduction to mathematical and strategic thinking. That’s a huge problem because a child’s first introduction to these concepts can be a make or break moment for whether they end up being interested in pursuing a career in those types of fields.

Traditionally, Tyler said teachers have used chess to teach those kinds of strategic concepts to visually impaired children.

“I see this project a little bit like that,” he said. “It brings to life, in a 21st century way, that kind of ability to teach children these new concepts.”

The Microsoft team has spent the last year or so testing the system with a small group of about a dozen students. Nicolas Villar, a senior researcher in the UK lab who was instrumental in designing Project Torino, said one of the unexpected pleasures of the project is the opportunity to work with kids who have a very different way of experiencing the world.

For example, he said, the team originally made the pods all white, until the kids with limited vision told them that more colors would help them. And although in electronics there’s often a push to make things as small as possible, with this project they found the kids were more engaged when the pods were larger, in part because two kids working together would often both physically hold the pod and touch hands as part of that teamwork.

“We really honestly designed it with them. It was a collaboration,” Villar said of working with the group of kids. “We thought we were going to be doing something for them but we ended up designing with them.”

Now, they are working with RNIB to do an expanded beta trial of about 100 students. The researchers and the RNIB will be recruiting potential participants for the trial in mid-March at the VIEW conference for educators in the United Kingdom who work with visually impaired children.

For now, the beta is focused only on the UK, which has spearheaded a massive effort to get more kids interesting in coding. Eventually, they hope to make it more broadly available to teachers and students outside of the UK.

A lesson in computational thinking

Project Torino is geared toward kids age 7 to 11. Using the coding tools, students can do things like make songs, even incorporating silly noises, poetry and sounds they create themselves.

As they build their code, Morrison said they learn the kind of programming concepts that will lead to careers in computer science or related fields.

“It is very specifically about building up concepts that will enable them to become computer scientists, programmers, software engineers, computational thinkers,” she said. “It gives them that computational base to whatever direction they go, and a shared vocabulary about what computing is.”

Morrison and her colleagues also have created a curriculum for teachers who want to use Project Torino. She said the teachers do not need to have a computer science background to use the curriculum – in fact, they assume that most teachers will not have any expertise in coding.

The system also is designed to grow with kids. Once they have mastered the physical programming language, Morrison said they also have created an app that allows kids to transfer the coding they have done with the physical system into text-based code, and then use other assistive technologies to continue coding.

“We’re mapping a pathway from the physical to something that a professional software engineer could use,” she said.