Take pictures around you and learn English words?

Eight Microsoft interns have developed a new language learning tool. It was developed for the purpose of learning surrounding words using a smartphone camera and promoting adult English literacy.

With this app, called Read My World, you can even take pictures with your smartphone and learn words from a library of over 1,500 words. It can be used as a complement to the class, or for learning purposes by those who do not have the time or money to attend classes for language acquisition. It’s not a class, so you can learn by taking pictures in your daily life.

Real My World combines Microsoft Cognitive Services and computer vision APIs to check what’s in a picture, spell out words, and say it. It saves a picture of a specific word and saves it in its own dictionary inside the app for future reference. It also includes 3 vocabulary games so you can practice new words.

You may feel that the vocabulary of 1,500 words is small, but in fact, it is similar to the number of words that a foreign language learner can learn with conventional learning methods. For example, according to the BBC report, most language learners find it difficult to learn more than 2,000 to 3,000 words even after several years of learning. In a study conducted in Taiwan, even students who had studied a foreign language for 9 years were unable to acquire the most frequently used 1,000 words. This report tells us that learning the vocabulary you use every day is of utmost importance.

As this app focuses on what is visible, formal education has limitations. As a result of collecting opinions from professors and students who tested the initial version, the development team made it possible to search for words in the document as well. Rather than being like Google Lens, which translates words, the app highlights some specific words, listens to their pronunciation, and lets you see what they represent by looking at the pictures. For example, when you run the app over student belongings, words like pencils, notebooks, scissors, and binders are highlighted.

The app offers some testing through the Microsoft Garage program, Microsoft’s in-house child care program. NGOs and non-profit organizations can participate if they are members of the low literacy community. Related information can be found here .