News
News
02 February 2012
In Tunisia, researchers are developing easy-to-use web tools to automatically translate written text into sign language.
In developing countries, the majority of deaf people are illiterate and cannot use mobile phone Short Message Service (SMS) or text messages. Their preferred language is sign language according to a research paper by scientists from the University of Tunis.
The WebSign project aims to break the silence of deaf people. “It’s an initiative to help deaf individuals use Information and Communications Technology (ICT) and hearing people to communicate with deaf ones. The project is really exciting, especially when meeting and getting in touch with the target community,” said Mohamed Jemni, head of the research laboratory at the University of Tunis and director of the Al Khawarizmi Computing Center.
One application they’ve already released is called MMS Sign, a free service for deaf people who want to use mobile phones, but cannot read. “There are no studies in the literature concerning projects that handle deaf and hard of hearing issues,” said Jemni.
The application automatically translates an SMS sent from a mobile phone. It converts it into a Multimedia Messaging Service (MMS) that displays a 3D Computer-Generated-Image (CGI) avatar who acts-out the message in sign language.
This is how it works: if someone sends a message to a deaf person, e.g. “I will be late”, this is sent to a telecom operator on the mobile phone. Once this text message is received by the telecom operator, the MMS Sign application converts the SMS into an MMS which is automatically sent to the deaf person. The MMS received by the deaf person contains the sign message, “I will be late”.
“For many deaf people, this is the only way to receive messages and use mobile technology to communicate. This is very important for emergencies, asking for help, or communicating with parents. This service is already available in Tunisia. More than 200 deaf individuals were involved in the testing phase. More than 1,000 users use our products in Tunisia and some Arabic countries,” said Jemni.
Do you like to eat green apples?
The MMS Sign tool requires access to a whole host of resources to manage millions of words. Jemni said, “WebSign needs huge computational resources and also sign language experts, deaf users, and linguists to enrich sign dictionaries.”
The readable-text-to-sign-language translation is done through a machine learning approach. These are algorithms that compare the collection of words within the English language (also known as a corpus), in machine-readable form, with sign language, creating a parallel corpora of languages.
Image of MMS Sign application 3D avatar character.
The 3D avatar character animation is helping around 1,000 illiterate deaf people in Tunisia and other arabic countries. Image courtesy Mohamed Jemni.
A mathematical algorithm takes a given sentence in English and finds the most appropriate translation in sign language. An example of a word string would be: “Do you like to eat green apples?” with the equivalent sign translation being “APPLE, GREEN, YOU LIKE EAT?”
The data volume is enormous; the English language corpus alone has 52 million words. Jemni and his colleagues are using the scientific computing resources of the EUMEDGrid-Support e-infrastructure to greatly reduce the processing time and improve the quality of translations. EUMEDGrid is an e-infrastructure network that supports 25 sites across 13 countries.
“We opted to use the EUMEDGrid infrastructure in order to generate a very big parallel corpus of sign language - the biggest parallel corpus of sign language in the world - to improve the efficiency of the automatic translation. EUMEDGrid reduces processing times from several months to several minutes,” said Jemni.
The WebSign project was presented by Jemni at the Mediterranean e-Infrastructures (EUMED4) conference in Amman, Jordan, on 13 December 2011, in conjunction with the First International Platform on Integrating Arab e-Infrastructure in a Global Environment.
Now, Jemni and his team are working on integrating the WebSign project with the EUMEDGrid science gateway. A science gateway is a pre-configured set of tools, applications, and data integrated into a single portal and accessible via a web browser. More applications are being developed. These include a screen reader that helps deaf people to sign what’s written on their screen, and a sign language course to teach deaf and hearing people sign language.
Eventually, the project will be transferred to a cloud service. Jemni said, “once deployed as a cloud, anyone could access it and translate a written text into sign language regardless of the device, software, or community they belong to.”
“To my knowledge, it is the only example of its kind in the grid world. We are currently brainstorming about how to integrate WebSign in the EUMEDGrid science gateway,” said Roberto Barbera, a senior researcher at the University of Catania.
Source: iSGTW, 1 February 2012
There is no need for e-Infrastructures without a user community correspondingly exploiting them. The project has an important responsibility to support existing user’s applications and assist new applications that will run on the Mediterranean e-Infrastructure. This activity is also committed to study and deliver a strategy for the long-term sustainability of the support system that will be put in place during the project lifetime.
