“We now know that it’s possible for people to build genuine connections with AI. How can we use this to help create more immersive language learning experiences?”

My Sketch Documentation

Sketch in Full screen link: https://editor.p5js.org/conniehu25/full/MH5i-2GLx p5js Editor Link: https://editor.p5js.org/conniehu25/sketches/MH5i-2GLx

Idea Overview :

• This week I wanted to try something a little different from what I’ve worked on before. I was inspired to start creating some prototypes/components to help me visualize an idea I had while doing my reading homework for “Patterns In Language.”
• My Idea is to create an educational tech tool for learning second languages (L2). I also wanted to continue these areas of my previous research:
  • Child-machine coevolution (HCI/CCI) / HCI in early childhood education
  • Using machine learning to understand and/or mirror alternate(in this case child) intelligences.
  • Distributed intelligence in the classroom (wearing the classroom as a skin)
  • The use of machine learning for human self-learning
  • I did not use Arduino this week because I did not have consistent access to physical computing materials. But thinking about physical computers in part inspired me to think about creating educational tools.

I created a simulation using p5.js, p5 speech, and ml5 object detection to help illustrate a research project I hope to do about second language learning tools.

Some pages from my initial Brainstorming:

The Bigger Picture

• The program would have the following functionality:
  • It would be similar to a chat based system (like Replika), where the language model continuously learns and adapts to the user’s speech. However, unlike Replika, a main feature of my proposal is that the model not only draws user inputted/textual information, but also, environmental information.
  • In this way, the machine learning model’s world view would grow alongside the child’s.
  • The chat model would make conversation with the child and ask questions about the child’s environment which it can sense from cameras, sound modules etc. The conversation would not just be textbook material, but rather constructed in an engaging way, drawing from past interactions with the child.
• I’d like to integrate various machine learning processes in order to create a language learning tool that
• I want to research how emotions, and building machine-child connections can facilitate in machine-child coevolution.
  • What benefits can data collected from children have on machine learning models?
  • What benefits can children gain from building connections with chat based/ AI systems? (i.e how can machine interactions in my proposed way help children learn?)
• The machine asks you questions about your environment and when you answer them, it collets this information and includes them into its database so it can create more realistic and immersive dialogues in the future. This is a chat bot focused on language education.