Acoustic Express
An AIoT tool including a smart helmet & application for hearing impaired motorcycle courier
Background Research
CAPPD collaborates with logistics companies to help deaf individuals secure stable, labor-intensive jobs, with food delivery offering the most opportunities and minimal requirements.
Only 5 % of food delivery riders have disabilities, including hearing loss
Families with disabled members earn 60% of non-disabled households’ income.
A survey of 200+ deaf graduates found a 30% lower employment rate than others
Food Delivery Market in China
3 Days Training to work | Easy to attend
Need Electric Bike and Phone Only
Total number of food delivery riders in China
There are 27 Million hearing impaired people in China. About half of them are in middle ages
Attributes of this group:
Acquired ones: Lower level of education / Hard to work in cities / Mostly indoor jobs / Lower income level
Innate ones: Lower communi-cation ability / Better consentrate ability
Hard to achieve “Same work Same salary”
Primary Stakeholder Research
To learn about the stakeholder challenges with regard to the entire hearing-impaired motorcycle courier (HIMC) delivery experience, I conducted two different types of user research (survey and semi-structured user interview) to collect both quantitative data and qualitative data. Below are my findings:
Field Research
To gain more insight with my target users, I opted to conduct a field research to learn more about the overall courier journey and the delivery process. After the research, I mapped out my findings as below and right:
The outdoors environment in which motorcycle couriers work is usually very complex.
Motorcycle couriers need to interact with the environment and communicate with other people as well.
Design Goal
Improve work efficiency
Reducing the burden of typing and improving communication efficiency when users communicate with others.
Enhanced ability to do multiple tasks without hearing things.
Interaction with passerby & customers
Letting customers know that the user is hearing impaired before the user reach out to them.
Disarming the customers’ unnecessary wariness ahead.
Working Safety
Traffic safety, reducing the risk of crashes during delivary.
Allowing users to take calls while on the road, without impeding driving.
Key Project Values
Emphasis on human wellbeing
With the realization that autonomous driving delivery service may be ubiquitous in the future, this design seeks to improve the delivery experience by the couriers (in this case, hearing-impaired people) and to bridge the gap between human’s psychological development and technological involvement. Therefore, hearing-impaired people can equally receive the sense of achievement by food delivery.
Envision the design with futuristic thinking
Since the project’s main purpose is to address the well-being of hearing-impaired motorcycle couriers by reducing the inconveniences tey face while delivering orders to customers, work efficiency and return on invest is not a prior concern in this project. In this respect, more advanced technologies and systems can be leveraged to solve the problem and improve user experience.
Ideation
Evaluation Principle:
- Adapt to Existing Tools – Design solutions should integrate with current equipment to avoid adding burden or restricting movement in varying outdoor conditions.
- Streamline the Experience – Minimize complexity to prevent confusion and disruption in couriers’ fast-paced work environment.
Feasibility Research
Camera
- The logo and the camera set can be a sign to inform others that the user is deaf before they reach out.
- Ultrasonic sensors can detect objects in close proximity to the user, providing vibrating alerts to avoid collisions.
- The 360 view cameras allow users to see a whole view of immediate surroundings, which is particularly useful for users with hearing impairments to know their surrounding environment.
Bluetooth hearing aids
- A convenient & cost effective solution to make some sounds louder by magnifying sound vibrations entering the ear, which helps improve users’ hearing and speech comprehension.
Speech to text: ASR Technology
- Convert voice message or real-time speech into text for hearing-impaired courier’s better comprehension.
The text can also be saved as documentations of the service.
Prototype
Visor screen view
The microphone on the helmet collects real-time speech and convert it into text displayed on the visor screen
Ongoing order page & Chat page
Final User Flow
- The Bluetooth-enabled hearing aids integrated into the helmet enhance the user’s auditory awareness, enabling better interaction with their surroundings.
- An internal vibration band provides haptic feedback, alerting the courier to potential hazards.
- Cameras and ultrasonic sensors positioned on the sides and back of the helmet continuously capture visual and auditory traffic data, transmitting it to the AI processor. If a potential threat, such as a nearby car honking, is detected, a ‘Red Alert’ notification will appear on the visor display to warn the rider.
- The app automatically transcribes incoming voice messages into text and generates context-based reply options tailored to the user’s location.
- If a message is received from a customer or if the user wishes to contact the restaurant, the app suggests appropriate replies based on the user’s current status and activity.
- If the user wants to communicate with passerby, for example, asking for direction, they can use the button on the bottom of the app to convert voice into text
- The visor will display real-time transcriptions of the other person’s speech, but only when the user’s looking at the speaker.
The ASR Technology
- Deep Learning and Neural Networks to process and interpret spoken language. These algorithms are trained on vast datasets to recognize a wide array of speech patterns, accents, and dialects.
- Natural Language Processing (NLP) enables it to handle a wide range of queries, from simple commands to complex questions.
- Noise Reduction and Echo Cancellation features to accurately capture voice commands even in the presence of background noise or when the user is communicating with people at a distance.
My design is specifically tailored to enhance work efficiency, ensure safer outdoor working conditions, and improve interactions for hearing-impaired couriers.
The system integrates AI-powered tools and advanced assistive technologies, including speech-to-text converters, ultrasonic sensors, and Bluetooth-enabled hearing aids, to optimize the user experience. The visor display provides real-time transcriptions of spoken conversations when eye contact is detected, ensuring seamless communication with customers and passersby. Additionally, haptic feedback through a vibration band alerts the courier to potential hazards, enhancing road safety.
By streamlining communication and improving situational awareness, this design empowers hearing-impaired couriers to work more efficiently and safely in dynamic urban environments.