Overview
Jellow Basic Communicator is a freely downloadable friendly Augmentative and Alternative Communication (AAC) system that uses icons/images to aid communication in people learning to speak or with difficulty with speech and language. Jellow Basic helps non-verbal children to communicate and gradually learn to speak - especially those with learning disabilities.
This app was designed by Prof. Ravi Pooavaiha for kids of the age group 4-9 years old. Parents and teachers could also use this as a learning tool.
Problem Area
After evaluating the app among designers and talking to parents whose kids are using the jellow app, we realized the engagement of this app is pretty low, and it’s hard for them to remember the sentences for a longer time. Kids also have a short attention period, making it difficult for parents to make their kids sit and learn.
Research
From the beginning, we wanted to make a game to make the learning process engaging. Therefore, it was essential to understand the mental learning model of the kids, observe kids and identify those interactions which create a feeling of joy and excitement among them. Also, talk to parents about their kids' problems while using the Jellow app. This would help us to get more context and allows us to lay the foundation for the design of the game.
Understanding the learning process of kids
Use of Symbols and Images augments the learning process
Our target audience begins to think symbolically and learn to use words and pictures to represent objects and manipulate those symbols to mean something else.
Multisensory learning enhances the attention span
Multisensory learning includes visual, auditory, and kinesthetic learning. It increases the attention span toward letters and numbers, especially for kids with learning and speaking deficits.
Semantic Encoding helps in remembering the information
It holds the information better than other encodings as in this Encoding. People try to understand the word's meaning or relate to oneself compared to material that has less personal relevance.
Analyzing the problems faced by kids with LDA
INSIGHT-1
Kids generally have a short span of attention period.
INSIGHT-2
The current way of learning is monotonous and has no fun
INSIGHT-3
They have creative mind.
INSIGHT-4
They enjoy varied sounds & wish to hear them repetitively. Funny sounds like beep-bop even exaggerate their movements & act as an excellent medium for engaging.
INSIGHT-5
Almost all kids love to play video games for long hours and have fun due to exciting sounds and interesting tasks; the game asks kids to complete different levels.
INSIGHT-6
In today's society, some schools are not interested in keeping the so-called "slow child" in their classrooms
Talking to parents in online sessions
Jellow app produces sounds when the user clicks on different images. For example: If I click on a picture of Dosa in-app, it would say, "I want to eat Dosa." So, the jellow app would work best if kids click the right images in the proper context with the help of parents. For example: If a kid wants to eat an orange, they could click on the image of an orange, which would translate their feeling into an audio message. It would say, "I want to eat oranges."
The problem is that it requires the parents to continuously identify the situations in which they can plug in Jellow App. This is not possible all the time.
Opportunity Areas
After gathering a lot of data, we analyzed each of them and synthesized them to articulate our insights. These insights would further act as a source of opportunity areas for us.
INSIGHT-1
To establish a way of learning style that provides a gradual increase in attention and sensory perceptions.
INSIGHT-2
A system that helps them in storing more information easily.
INSIGHT-3
Providing fun and engaging way of learning by incorporating various media.
INSIGHT-4
Developing a story in the game such that they could relate to themselves while learning.
INSIGHT-5
Empower their learning process and allow them to express and mold the things on their own.
Ideation
To get higher-level design ideas out quickly without worrying much about the implementation, I created several sketches of the game, keeping the research insights in my mind, and filtered the best three. Following are some of the ideas.
Painting with the help of food materials
Since kids like to do creative activities, we will sketch an interesting character on the food and ask them to fill it using the food materials as their colors. For example, imagine a smiley face with eyes, a small nose, and a small one made of rice. With the help of different vegetables, cooked food (Dal, Rajma), and salads, kids can fill the eye of the smiley with a slice of onion.
Jumping on the name
Kids can select any sentence or write any sentence as input to the game. The letters of those sentences would be arranged on the ground. Now, each letter would act like a block on which the mascot will jump, and after landing on each block, it will spell that letter, and after leaving the last letter of a particular word, it will pronounce the whole word. The task of the game would be to reach the end of the sentence by crossing various hurdles.
Jumping and eating
This game is inspired by the kid's whole day meal, including breakfast, lunch, and dinner. This game is similar to Mario. For each meal, the game will ask for liked and disliked food. Then food materials will hang on the roof, and the game's mascot has to eat the desired number of favorite food materials using the move left, move right and jump actions. After eating the food every time, the game will give instructions like "Yumm!! I need more apples". The moment they will eat the worst food, the game will get over.
Choosing the best one
I evaluated the brainstorming ideas with my professor's help to reach the best possible solution that aligns with our research and the goal we were trying to accomplish.
Painting with food
Give freedom to play around with colors and learn
Pronounced sentences may or may not be related to user’s context
No storyline
Engaging
No learning of emotions
Jumping on the name
Effort to think and write a sentence for the input
Pronounce a complete sentence only after jumping on all the letters.
No storyline
Engaging
No learning of emotions
Jumping and eating
Simulates the real world & pronounces sentences in context to user
Repeat the sentences which won’t look redundant & augments learning
Has a complete storyline
Engaging
Pronounces sentences with emotions
Visual Design System
I evaluated the brainstorming ideas with my professor's help to reach the best possible solution that aligns with our research and the goal we were trying to accomplish.
DESIGNING THE MASCOT
To make a simplistic and playful game, I tried to filter down the actions(state) and focus on three states. Three rest positions are rest, walking, and jumping. Following are the skeletons of all the actions and their corresponding illustrations
DESIGNING THE BACKGROUND
The background elements of the game simulate the native feel of the kitchen so that the kids can relate to themselves, which would help them in their learning process. For the sake of this project, we followed the Indian theme. Since the kitchen atmosphere is a background element, subtle colors were used to not deviate the user's attention from the primary objects, which are the mascot and food. Following are the rough sketch and the final illustrations of the kitchen.
Learning Model built for the game
Above mentioned theories have been incorporated while making this model. Each part of this model has its significance and helps in enhancing the learning style of kids.
Introducing the game
The game starts by asking them to choose one liked and disliked food out of four options. Then two foods will be hanging on the kitchen roof, and the mascot can eat them as per the action taken by the user. Three levels will be divided based on the day's meal: Breakfast, lunch, and dinner.
Verbally stating the goal
After selecting the liked and disliked food above, the above screen appears, the goal of this level appears at the bottom, and the game verbalizes the goal.
Controls of the game
The game has 3 actions: Moving right, Moving left, & Jumping. The user can use combinations of these actions to eat the food of their choice. Instead of arrows, the buttons have images of real actions, making it easier for the kids to comprehend and relate.
FLOW OF GOALS IN GAME
"I want to eat 4 oranges" -> "I want to eat 3 more oranges" -> "I want to eat 2 more oranges" -> "I want to eat one last orange." But during any of the stages, the mascot eats any disliked food, and then the game will get over with a dialogue, "Eww!! I don't want to eat eggs". After eating the desired number of liked foods, the game will move to the next level. For example, the game will move from "Breakfast Level" to "Lunch Level."
This repetition of the sentence in a subtle way would help the consumers to learn these sentences engagingly and won't get bored due to this repetition.
