The SignGuide project

Automatic Guided Tour of a Museum Using Sign Language

About the project

The object of the project is the development of a standard interactive tour system in a museum for the deaf visitors using mobile devices that will be able to answer visitors' questions on the parent sign language (SL) in relation to the exhibits, and provide additional content using avatar or video also in semantics utilizing techniques from the field of computer vision, machine learning and graphics. Final desired result is the increase of the satisfaction of the deaf visitors and in the long run the better social integration through better dissemination of cultural content, as well as increasing museum attendance by members of that group. SignGuide is expected to help museums and cultural organizations harmonize in the long run their services provided by European legislation obliging them to provide equal access to vulnerable social groups.

The SignGuide project is implemented within the RESEARCH - CREATE - INNOVATE Action. The main objective of the Action is to connect research and innovation with entrepreneurship and to strengthen the competitiveness, productivity and extroversion of businesses towards international markets, with the aim of transitioning to innovative entrepreneurship and increasing domestic added value.

Technical goals:

G1: Implementation of a service for mobile devices that will recognize the user's question with SL about the report.

G2: Implementation of a service that will analyze the previous question and will recall in response additional content in SL.

G3: Development of accompanying material in such a way as to support retrieval based on free text questions.

G4: Development of an avatar for the presentation of the additional material through SL.

Financial goals:

G5: The development of a standard tour system for deaf museum visitors, which will be able to penetrate dynamically in the global market of mobile cultural applications, due to its uniqueness in order to increase visitor satisfaction, the increase in traffic and the low cost of installation and operation.

Operational goal:

G6: The development of a museum tour system for the deaf that will take into account their needs and improve their quality of life. Satisfaction of these objectives will be documented during the trial period with a significant increase in satisfaction and duration stay at the museum e.g. by > 50%.

Dissemination goals:

G7: Dissemination of the results of the project with the aim of advertising the benefits and with the ultimate goal of improving the experience of visiting a museum and better integration of deaf people. The target groups are: Museum administration and curators, deaf visitors and deaf associations, general public, researchers, companies in the field of culture.

Objectives will be technically achievable through (a) computer vision and machine learning algorithms for question recognition (b) machine learning to relate the questions to the content and (c) graphical algorithms to implement the virtual guide. The system will be used at the Archaeological Museum of Thessaloniki (AMTH). The integration and commercial operation will be done through BioAssist. The development of the monitoring system and the virtual tour guide will be developed by the Institute of Technology and Research and the Computational Vision laboratories and Robotics (FORTH-EVOR) and Human-Computer Interaction (FORTH-ERAW) respectively. The users will come from the Deaf Education Unit of the University of Patras (PP-MAK). The Signal and Telecommunications Processing Laboratory (PP-EEST) will undertake the machine learning. The Laboratory of Cultural Technology of the University of Patras (PP-EPT) will organize the content and coordinate the project.

Download the SignGuide App




Signal Processing and Communications Laboratory Logo

The Signal Processing & Communications (SPC) Lab is a Laboratory of the Computer Engineering & Informatics Department (CEID), University of Patras. The SPC Lab presents a research activity in a variety of fields, such as: Signal Processing for Communications, Sensor Networks, System Identification etc.

Institute of Computer Science Logo

Institute of Technology and Research - Institute of Informatics (Laboratory of Computational Vision and Robotics, Human-Computer Interaction Lab)

Archaeological Museum of Thessaloniki Logo

The Archaeological Museum of Thessaloniki is one of the largest museums in the country and the central museum of Northern Greece. With unique collections of ancient objects, with rich activity and extroversion, it is always ready to welcome its visitors..

BioAssist Logo

Founded in 2013, BioAssist is the first Greek ICT services company created for seniors and chronic patients. You only need an Internet connection and our tools will save you time and money by keeping your caring circle into the loop. We believe that mobile Health is the most useful chapter within the book of Internet of Things. Access to health monitoring should be simple, smart, and friendly.

Deaf Studies Unit Logo

The Deaf Studies Unit (DSU) belongs to the Department of Primary Education of the University of Patras, and is a leading center for Deaf Studies, Greek Sign language (ENG), and Deaf Education in Greece. Established in 1994, is the only academic institution in Greece that provides research in the area of deafness, and programs for teachers of the Deaf, Sign Language tutors, and interpreters.

Project coordination

The project is coordinated by the University of Patras,
with Associate Professor Dimitrios Kosmopoulos serving as scientific director. (Homepage)

Contact us

Please direct your enquiries and communication using our contact form:
Co-financed by the European Union and Greek national funds through the Operational ProgramCompetitiveness, Entrepreneurship and Innovation, under the call RESEARCH – CREATE - INNOVATE (project code: Τ2ΕΔΚ-00982).