An automatic gestural language identification system is being developed in the UPV/EHU

2022/11/29 Agirre Ruiz de Arkaute, Aitziber - Elhuyar Zientzia Iturria: Elhuyar aldizkaria

• Teknologia
• Berdintasuna
• Adimen artifiziala

An UPV/EHU computer group works on research that can contribute a lot from the point of view of inclusion: an automatic sign language identification system is being developed using the reference points of the hands extracted from the videos. In fact, 70 million people in the world with hearing impairment use sign language to communicate, but because the rest don't know sign language, they suffer from problems communicating on a daily basis.

A great deal of technological effort has been made in recent years in the field of automatic knowledge of oral languages, but the 300 languages found in the world are excluded. On this occasion, the Professor of Computer Science and Artificial Intelligence at UPV/EHU Basilio Sierra is developing a system for identifying the Argentine language.

Language structures and characteristics of the languages of origin

In the new system, each video frame contains the reference points of the hand, groups of trees are generated, characteristics are identified and classified using mathematical algorithms. Researchers have had to take into account the linguistic structures and characteristics of minority languages. Therefore, in addition to the position and movement of the hand, they should detect the orientation, as a gesture may have a different meaning according to the orientation.

On the other hand, they have also had to take into account the digital spelling, as there are words that are given (if the sign of the word is not known, to indicate proper names, or to highlight a word, among others). According to the origin of sign language, digital spelling can take on great importance: in American gestural language, for example, it can occupy between 12 and 35% of the conversation, but in Italian it is only used to express foreign words.

Researchers admit that to understand gestural language it is essential to collect not only gestures of hands but also gestures of the face. For example, the face is fundamental when it comes to prayers in the area. And they point out that, at the moment, the new system has been based on hand gestures. But they say the results are very good.

Some previous attempts have been made, but most asked for intrusive methods, such as the use of electronic gloves by the speaker. The new system does not need it. The study has been published in the journal Plos One.