• Eshara is an innovative Arabic sign language translation system that uses deep learning and avatar technology to facilitate communication
  • Now in its seventh year in the UAE, the James Dyson Award empowers young engineers to solve global issues with innovative design and engineering
  • This year’s national winners receive a cash prize of £5,000 and advance to the international stage of the competition

DUBAI, UAE – Eshara, an AI-powered Arabic sign language translator using avatars, has won the 2024 UAE James Dyson Award. Developed by three talented engineering students from American University of Sharjah (AUS) – Salma Shaarawi, Mohamed Eid and Aley Eshra – Eshara addresses a critical gap in the Middle East: the lack of Arabic sign language translation services.

In line with the UAE's vision to improve integration for people of determination, Eshara bridges the divide between hard-of-hearing individuals and the public in Arabic-speaking societies. By focusing on Arabic sign language, the platform offers a unique solution tailored to the region's needs, facilitating communication where traditional translation services fall short.

For the past seven years, the James Dyson Award has invited engineering and design undergraduates and recent graduates in the UAE to "design something that solves a problem”. The instructions are intentionally broad and open-ended, encouraging students to use innovative technology to address major global issues.

The Problem

The winning team of three students recognised the significant communication barrier hard-of-hearing individuals face, and identified a crucial gap: Traditional translation services often lack sign language support in Arabic, despite it being one of the most commonly spoken languages locally. Driven by the desire to create an inclusive world where everyone can communicate effortlessly, they developed the innovative Arabic sign language translation system Eshara.

Team Eshara’s inspiration came from a pivotal conversation with the African Sign Languages Resource Center (ASLRC). Observing the everyday struggles of hard-of-hearing individuals in interacting with society, along with the scarcity of accessible solutions, drove them to harness technology to bridge the communication divide. The trio aims to foster an inclusive world where effortless interaction is possible for everyone regardless of hearing ability.

The Invention

The students began conducting vast amounts of research and development to bring the project to life, creating multiple prototypes to improve accuracy and reliability. They filmed and collated a dataset of 62,000 videos of themselves signing to train their deep learning model effectively. Through this, they were able to develop an innovative sign language translation system integrating both sign-to-text and text-to-sign capabilities.

After testing, the students created a web application that allows users to sign gestures via their phone camera, which are then translated into text in real time. This makes the technology widely accessible through standard web browsers without requiring specialised equipment. Likewise, the team focused on the reverse translation process, creating custom avatars for converting text messages back into sign language. These avatars, meticulously animated to resemble the students themselves, add a unique touch of personalisation to the application.

By combining real-time sign language recognition with animated avatar-based sign language output, this comprehensive system bridges the communication gap between sign language users and non-users, potentially revolutionising accessibility in digital communication.

Eshara takes home £5,000 as the James Dyson Award national prize to support its next steps in development and commercialisation. The team plans to use the funds to create a mobile app, expanding their system to support additional Arabic dialects and sign languages beyond Arabic to benefit the global hard-of-hearing community. This expansion will involve collaborating with sign language experts and gathering new datasets for various sign languages.