Ai.vatar: Like ChatGPT, only with head and body

Is the future technology for artificial intelligence (AI) coming from the Ruhr region? Researchers and tech experts at Witten/Herdecke University (UW/H) have created an embodied AI that surpasses the capabilities of ChatGPT: "Our Ai.vatars are artificially intelligent dialogue partners that, like ChatGPT, can impart knowledge and communicate professionally with people. What makes them special, however, is that they can also interact with body language in virtual space - and thus become residents of the metaverse, so to speak," says project manager Dr Jonathan Harth from UW/H. Metaverse (or metaverse) is described as a virtual reality in which people can interact with each other. It is seen as a new era in the digital age and major tech giants are investing heavily in this technology.

From the very beginning, the joint project "Ai.vatar - the virtual intelligent assistant" aimed to develop and research artificial intelligent avatars for virtual and augmented environments: "The project pursued the goal of combining voice-based artificial intelligence and virtual avatars. When we started the project in 2020, there was no well-functioning system that could integrate both worlds. Our goal was to achieve exactly that," Harth continues.

"Our avatar of the latest technology was born. We christened it 'Hudson'."

It was a success and is probably still unique today: according to Harth, until two years ago it was not possible anywhere in the world to build a conversation system that could hold logically coherent and plausible dialogues over a longer period of time. Only the so-called Transformer technology, which is also used in ChatGPT, made this possible. However, Ai.vatar goes one step further: the Ai.vatar system can not only write, but also speak and understand spoken language. It is also fully embodied in the virtual world as a virtual person: "With the connection to GPT technology, our Ai.vatar of the latest technology was born. We named him 'Hudson' and placed him as a conversation partner in a virtual café," says Harth.

According to the scientist, there is still no major player in the tech industry that offers such comprehensive artificial intelligence: "There is still no established and robust system that offers the best of all these areas of technology, i.e. looks so good, moves so well and is both voice-based and capable of coherent and plausible interaction."

The fields of application are diverse: Ai.vatar could be used as a tutor, counsellor or salesperson in the business or health care sector, for example. With the embodied Ai.vatar system, the expertise of a ChatGPT takes on a whole new social dimension: "Whether I write to an AI via a chat window or simply communicate using spoken language makes a big difference," explains Harth. In addition, the fact that the AI has a virtual body leads to further social effects: it feels much more like a real conversation.

Big milestone, big responsibility

This technological milestone on the road to digital reality comes with a great responsibility: complex AI systems can be fascinating and frightening at the same time, as the controversial debate surrounding ChatGPT shows. That is why, in parallel to the development of Ai.vatar, the UW/H set up several studies that dealt with psychological, ethical and social questions: How do humans and artificial intelligences interact? Can there be a culture of interaction? What mutual expectations do humans and machines have? How can Ai.vatar not only be used in the entertainment sector, but also provide support in the medical field, for example?

One of the many studies carried out, for example, looks at how Ai.vatar can help to break down prejudices against people suffering from mental illness. "In her studies, my colleague Alexandra de Carvalho is investigating, for example, whether an avatar that displays schizophrenic characteristics when interacting with people can help people to better understand this clinical picture. Should an encounter with a schizophrenic person occur in real life, it is easier to react because there is less ignorance. Our understanding of each other grows," says Harth.

The task now is to further professionalise Ai.vatar. "We will be presenting Ai.vatar at relevant conferences and trade fairs in the autumn in order to explore possible further collaborations, engage in dialogue with other experts and raise awareness of the project. At the same time, we will be working on further optimising the technology and expanding the link between non-verbal communication and the spoken word. Because this is still a major challenge and highly complex."

The Ai.vatar project was funded by the Ministry of Economic Affairs, Industry, Climate Protection and Energy of the State of North Rhine-Westphalia and the European Union for a period of three years. Together with two project partners from industry, the virtual world specialist HHVision and the Internet-of-Things start-up IOX, the technical combination of speech recognition software and avatar technology in virtual space was designed at Witten/Herdecke University and successfully implemented during the project period.