The Talking Hand

Can sign language become acoustic speech? The BrainHUB Lab team has taken upon the task to give a positive answer to this question through an innovative development. It will make communication with people who use this wordless language much easier. However, ambitions go even further. In the digital age, the devices we use continue to “lose” buttons. It’s a matter of time to go from sliding our fingers on the screen to gesture control, since the best means for control are our hands. A prerequisite for this is the development of a reliable technology, which can read, interpreted and make them digital.

The Beginning

The idea came by accident – as students at the Faculty of Mathematics and Informatics, the creators of BrainHUB Lab saw people talking through signs and decided to try to “translate” this language into acoustic speech. Three years passed during which the team made eight prototypes. The development has two goals. The social goal is to translate the language of the deaf into acoustic speech in real-time through technology, and live speech to be visually transformed into gestures. The other goal is to make a unified gesture recognition platform. When the database and the platform are available, it will be possible to find a commercial application through the relevant developments.

The Sofia team is not the first in the world to engage in such an idea. So far, there are two approaches with regard to technology. One is with cameras and computer vision, which limits mobility, and the other – with sensors and portable technologies. BrainHUB Lab is creating a development of the second type, and there have been seven other tries outside of Bulgaria, but none has reached the market.

Success Factor

“The sign language has its specifics in each country, and everyone who uses it adds their own signs. The number of signs is huge, but their combination is even greater. In order to be read, artificial intelligence, which is trained in sign language, is used. If we fulfil the idealistic goal, which is as complex as possible, the commercial will be easy to implement,” explains Oleg Kamenscic, founder of BrainHUB Lab. The decisive factor for the success of this technology is neither money, nor just brains, but the combination of both. There is already a foreign team, which has raised funding through crowd funding, but the development did not reach the promised goal and the money was returned. That’s why BrainHUB Lab is carefully focusing its efforts.

Oleg Kamenscic is an inspirer of the idea of making signs “talk”, and Dimitar Peev is the newest addition to the BrainHUB Lab team

Where and Who

Except for the people who use the sign language, the development is suitable for control of digital technology. It will be possible for presentations to be made without the use of remote devices, all happening by using just our hands. The gaming industry is the other favourable niche. More and more games are transferred to the virtual reality and instead of a joy stick, the control can be performed by both hands. Possible application can also be found in the field of rehabilitation – from a distance, the therapist will give instructions and will monitor how the patient is doing.

The team includes six people. Oleg Kamenscic is the inspirer of the idea, Georgi Peev is the technological demiurge, Vlad Timofeev plays the role of impartial and critical mind, Daniel Taskov is the most mysterious participant since nobody has seen him, and Dimitar Peev is the newest addition to the group. Dimitar Bankov is the man with the greatest life and professional experience and is called by the others a “business strategy watchdog”. They are all engaged in the development, simply because they find it interesting. It is a temptation for them to solve such a complex task.

Approach

Programming, hardware, kinematics, communication between embedded systems, mathematics, and artificial intelligence are the areas of knowledge required to solve this complex task. They are developing a portable technology that has to be universal and fit the maximum number of hands without losing its ergonomic properties. In their effort to get to the right form, they design 3D models based on anthropometric and topological human palm studies by making many prototypes. Without a 3D printer and modern hardware prototyping technology, this would have been extremely complicated. The printer allows you to make samples and enhancements while designing the optimal product. BrainHUB Lab programmers say they do not think that there is any skill or knowledge that cannot be acquired, and it’s all about patience and time. The Internet gives access to vast online knowledge resources and they benefit from it. Prior to this project, Oleg hasn’t worked with a soldering tool, and now he already designs hardware boards: “We are not revolutionizing, we are evolutionizing. About 90% of the programming and development of such a project is boring, but there are 10% when, after sleepless nights and unsuccessful attempts, we can see how the hand is folding on the monitor – this is the proof that our idea can be realized and this is the reward for our work. We are driven by the challenge to solve something complicated.”

Opportunities

So far, they make the development with minimal personal costs. It will be an open source, which will allow the project to be expanded by inviting external developers.  For these technological enthusiasts, it is not important to be the first but to be the best. The team includes only people who are always ready to engage in the implementation of any complicated ideas. So, recently, they got an order to make an extravagant business card. A month later, they handed over the finished product – a white cardboard card without any inscription. But inside, there was an integrated (NFC) chip in which all the contact information was stored. The business card had to touch the smart phone and it automatically saved the data.

Money is just a fuel, and we do not want to have a well-paid routine job, the young innovators say. According to them, Bulgaria is a good place for experiments – it is cheaper here both to fail and to succeed.