VEX Robotics Educators Conference. Raise your hand if you've ever heard of a brain computer interface. Awesome. Okay. Raise your hand if you've ever used a brain computer interface.

Okay. All right. Awesome. Raise your hand if you've ever built your own brain computer interface application. Okay. Awesome. Okay. So hopefully some of you I can catch you in the hallway. We can fix that. All right. Um, let's start first about kind of give me a little bit about me and this background, kind of who I am. I'm Dr. Chris S. Crawford, an associate professor at the University of Alabama. I love robots. I love brain computer interfaces, music and farming. That kind of sums up my life, right? So it's kind of a weird mix, but that's typically what I'm doing day to day. Um, and I'm doing that because I can be long-winded. Let's see. Stopwatch, timer, let's go 43. All right. So robotics. I started doing robotics during undergraduate research. So we were doing simulation stuff. So this is like my first research paper on multi UAV stuff. Many years ago, a lot of my work, I approach it through the lens of something called human computer interaction. And so one of the first projects we challenged, we had it still kind of working on it. At least my advisory is, is on voting, right? Ooh, everybody loves voting, right? Ooh. So we were wanting, you know, we wanted to do something real risky and do tele voting, right?

And so this was one of the first projects that I did really far away from robotics, but there's a wealth of different projects I kind of designed and did that brought me into that. So I'm leading you up to that. Second thing, tongue detection. All right. So here's, I like to be as interactive as possible. Anyone has any random idea of why a company wanted to detect when someone was sticking out their tongue? Any random guess from the audience? Why, why could this be useful? What in the world would they want to do with this? Any guess? Temperature check. Speech, temperature check. Could be used to control something remotely. Could be controlling something remotely. Yeah. Fingerprint. All, most of those responses were way better than the actual application. Anyone seen this movie, how to train a dragon?

So I was at Intel and DreamWorks and DreamWorks, they had this great idea of like, well, what if we detected when someone was sticking their tongue out and then Toothless would stick his tongue out, right? And so this was kind of the beginning. This was pre-Snapchat, right around when filters were becoming a thing. This is when they were calling it perceptual computing. All right. Anyone still using that emojis? No, probably not.

Somewhere along the way, I got introduced to this area of brain computer interface, which was still kind of focused on the human. But instead of using, you know, the arms or the tongue, you're focusing on ways we can use brain activity in order to control things around us. Right. So I know some of you said you've seen a brain computer interface or used one. How about raise your hand if you've ever seen it in the movies? Right. This is typically where we, so many of you may be familiar with this clip. All right.

One of the first kind of control the world around you to mind type of clips. All right.

So this is quite a few years ago. Not so long ago is this next clip. Let's see how many of you have seen this one before.

And so it's really interesting because in a lot of ways, although many of us don't aren't aware of it, we have these interfaces today. I don't have the micro bits, right? But I have the device, right? That could measure the activity from our brain. Right. So the question is, it's been out, but how long? Any guesses of how long this technology has been around?

How many decades? Yes. Anyone want to give a random year?

1930s. Okay. You really know the list. So Hans Berger, right? 1930s. That's the measurement of the brain activity from the human brain. You're absolutely correct. Last question. What about the measurement of the human brain and interfacing with a computer?

70s. Somebody's done their homework, right?

There you go.

So in many ways, this technology has been around for years. I want you to listen to how he kind of explains and discuss the technology. The light is being turned on and off by this laboratory technician. She throws no switch. She pushes no button. She's doing it strictly with brain waves.

This is Dr. Edmond Dewan, an Air Force physicist who has been working with brain wave ideas. Dr. Dewan, exactly what is a brain wave? Well, apparently the brain in doing whatever it seems to be doing. So he's very confident about what the brain is doing, right? He's doing something. And in a lot of ways, we know a lot more about the brain than we knew in the 70s, right? But that's if you're a neuroscientist or a researcher or someone who's an expert in this field, which is a very small amount of people in the world. And so a few years ago, when I was in graduate school, I started dreaming up this world, right, of what if instead of this technology being confined and closed research labs or high critical medical environments, we brought it to the classroom, right? Maybe, just maybe, if we did this, we would know much more about the brain because the people who are the most curious about the future, this is the environment they're in, okay? Then I went really crazy. What if we just, instead of only focusing on the brain, what if we start to interface it with all these different robots and drones, environments, right? And so that's a lot of what, this is pretty much the gist of this talk.

Like how do we do, how can we get there? How are we doing it now? And kind of my thoughts about the future. Before I get into the deep details of that, let me first kind of go over a brief overview so everyone is kind of on the same level about how this technology works. So typically these interfaces work by measuring the electrical activity from your brain. You can do that with electrical activity. You can do that using blood oxygenation levels, so like infrared information. But typically this information comes over like this. On the x-axis, you have time. On the y-axis, you have microbe. It's electrical information, right? Just a signal, right? Anyone want to take a random guess at what happened here? What did the person do? Anyone guess?

Stress. One says stress. Yeah, yeah. Any other guesses? Blink! Somebody did their homework. This is an eye blink signature, right? So one of the interesting things about my first entry of doing this work was that when we first built our system, this is what you call an artifact. This is noise. So typically, the way you handle this is you filter this. You're not supposed to see this, right? But we were hard-headed and we didn't filter it. And you want to know the first thing that had kids hooked when we brought this in the classroom?

When they blink their eyes, they can see it. And from that moment, they made a connection with this signal in their bodies, right? But it was completely opposite of the way we were taught to do it in school, okay?

Now, the way it actually works is you're supposed to get that signal, process it, pre-process it, process it, extract some features from it, create a model to basically represent what someone's brain looks like when they maybe imagine them squeezing their hand, their left hand or their right hand or both hands and feet or their tongue. And you send this to a command to application. It could be a computer, a wheelchair. Oftentimes, we use robots. But pretty much everything that happens at this point is very similar to what they were doing previously. The only difference is instead of using their brains, they're using a joystick or controller or mouse, right? Now, you might ask, well, how can you actually separate what your brain is doing? This is one of the most basic ways we can do it. And this is typically referred to as a frequency band. And so when you're in different cognitive states, your brain tends to gravitate towards these different patterns. So like at three o 'clock in the morning, well, I was up at three, I couldn't sleep. At one o 'clock in the morning, my brain was probably like a delta wave, right? These low drawn out frequencies. Right now, it might be closer to the beta or gamma wave.

So here, these more volatile signals because I'm a 10, I'm kind of focused on my talk, a little nervous, right? But we can use things like process and fast fourier transform to go from this time domain to frequency domain information. And so what we do is, the next question is, how do you actually represent this in an environment similar to a scratch or Blockly or VEX code so that a student can interpret what those features mean and use those features in a program, right? To build their own brain computer interface. Now, before this work, typically if you see this in the news, the first thing you're typically going to see this represented as is a clinical translational research. So that's all the way to my right side. Okay. All of the work that we do inside of our interfaces or inside of our software is grounded by something in some random medical journal, right? But you can use those same fundamental principles in something that might be used for gaming or something that's used for user state monitoring to detect if someone is losing or losing attention if they're in the don't do this, but this didn't work out so well. But there was research on if you could use this to detect the attention changes.

But one of the things that we found out that was really interesting is because of the novelty of technology and because of the excitement and curiosity that drew in students, if you did things like brain-robot interaction, something that merged well with the interfaces and devices that they're already working with in the classroom, it was a natural marriage with this new input modality.

So really quick, I'll show you the different types of forms. This is what we refer to as the brain-computer interfaces. Anyone in this room even remember this, though? No, this is so long ago. In an active case, you have to calibrate the system. All right. So my friend that you're going to see in a few minutes, he sat down, he calibrated the system over and over, and the computer basically asked, how does your brain look when you're imagining squeezing your right hand? And just to kind of go over that really briefly, there are specific patterns that each of our brains will show or exhibit when we're imagining squeezing our right hand and our left hand. Now, if you did it one time, it would be hard. But if you did it over 100 or 200 trials, the computer gets better and better at distinguishing are you imagining squeezing your right hand or your left hand. Once you get a signal that can be predicted in a somewhat efficient manner, then you just use like any other traditional input modality. In this case, we're only using one trigger, and it's only detecting if he's actually imagining squeezing his right hand. Now, something very interesting happened here. I forgot to uncomment the code that we could use to control it with the keyboard.

So the only way this drone went forward is because of the brain-computer interface. And that was kind of my aha moment, like, oh, well, wait, because I was still skeptical of these systems when I was doing this. I was just testing it out. But this is my first indication that maybe this is doing something interesting.

There's another type of application, which are passive applications. And so in this case, the person is not intentionally sitting there trying to control the robot or the machine or computer application with their brain. They're just doing their natural whatever, right? It may be studying, maybe it's reading a book, maybe it's doing a math problem. And the system is passively detecting their levels of engagement and it's adapting based on that information. So in this case, this is one of my colleagues who's at University of North Carolina now. He did work years ago, and he went in the classroom and he, they looked at the seat. What do educators do when they're trying to get their students' attention, at least the educators that he studied? And so what they came out is that, well, usually they gesture to their students to try to get their attention. So what this robot typically does in this experiment is anytime during this learning experience, if it detected you lose your engagement, it would wave hands, right? And he found that it did show some improvement of getting the individual's attention when it would wave his hand. Another paradigm is called reactive. I'll play this.

And in this case, each of these flashing boxes are flashing at a different frequency. And it just so happened that Don Shin in the 80s discovered that, well, if you're paying attention to a specific box that's flashing at a specific frequency, you can use a specific paradigm called the oddball paradigm to get software to detect which of these boxes you're looking at, right? So it's just another way of using information from the brain that's going through the central nervous system as an input modality, okay?

We've also did this with the Baxter robot. And so in this case, we were trying to see, well, could we adapt the emotions of the robot, right, during a collaborative pick and place test based on the EEG activity, okay? So just really kind of taking you through the path of us being in a research lab, looking at other scholars and trying to replicate what they do. And then we slowly start to make a pivot, which is kind of what brings us here today, right? One of the things that I initially found out is that a lot of the existing systems at that time were not modular. Anyone use robot operating system at all or heard of it? Can I have a few people? Oh, well, Bob, yeah, of course. So there are these systems that you might see in professional robotic settings or like in a grad, undergraduate, graduate setting to teach robots that were using these modular principles, but this didn't exist at all in the world of brain computer interface. And then the other thing that there was very few existing applications, if any, at that time that were using modern web technologies. So there was things like Node.js that was coming out, Electron, this really didn't exist in this world. But we did start to see hardware companies come to market and really start to target the consumer population, not the researcher, not the medical professional, but really the everyday consumer.

Oftentimes with purposes on like mindfulness training and meditation, right? So you can buy the device. Like if you buy this device, it's about $200, $300. You can download an app, right? You can meditate for two weeks. And then when you get tired of meditating, it's just a paperweight, right? And so one of the things we were trying to figure out, well, if this hardware is becoming more affordable and somewhat more accessible, are there things that we can do with our software or is there things that we can do before we even dreamed up what we have now? But are there things that we can do or software that we can build to address the gap of people who may want to learn more about brain computer interface, but just don't know MATLAB? Or maybe don't know Python?

So one of the things that we did, anyone seen Pacific Rim before? I don't know how many. Okay, awesome. All right. So the funny story about this, this is a simulator robot. And we started to explore the idea of cooperative brain computer interface. And because if you have a person that's sitting down and they're trying to control a robot or drone or something in a space with a BCI, fatigue picks up. You sit there, you'll think sitting there for 30 minutes trying to focus on squeezing left from right. You get tired pretty quick. So we had the question of, what happens if you split the cognitive load? Right? And this was something that at the time, NASA was actually really interested in because there was like, well, in outer space, there's gravity. And so we can do robot control using the brain that might can address that issue. And I was at a movie theater with my friend in California during one of our internships. And we were like, oh, let's go see this movie called Pacific Rim. And the movie comes on and they're talking about splitting the cognitive load. We're like, they stole our idea, right? So I play this video. So in one case, condition with the single user, the user is responsible for having a cognitive command that's mapping to making the robot go forward and rotate.

And it's a simple maze environment. The only thing we were interested in is trying to see how their engagement levels would change throughout this experiment. Right? So in one case, one person is responsible for two commands. And the other condition, you had two people. And so one person would be responsible for making the robot go forward. Another person would be responsible for making it rotate. Okay. And this was made possible, at least in that initial design, because we had started building out these applications very modularly. We were using Node, we were using Modern Web Technologies. So we could prototype up these systems fairly quickly.

And we were also using, you know, JavaScript environments. So we kind of built out these experiments in JavaScript pretty quickly, even pre-AI, no cloud, right? We were just jamming it up. Okay. And so one of the things that we did was what happens if we layer this spatially over the environment? And so we started to see these interesting patterns of, well, in the solo participant condition, you can see this red, all of that red you see over the map means that they had high level of engagement, which typically means that the engagement levels were highly sustained throughout the entire course. But in the solo participant, you can clearly see the who was responsible for only activating their brain activity over the corners. And so for us, it was a way of like doing something interesting in the robotics domain, but also it was a confirmation that, okay, this device is really noisy. It's really hard to get very accurate information from it oftentimes, but there is some source of interesting activity that we're able to people compete against it just because everybody loves a good competition. Of course, this would be the use case that actually took off, right? Everybody, we're going to fight.

But there was this ideal competitive brain computer interface. They came out of a group in Germany early 2000s and they called it Brainball.

And Brainball in this context, each of these individuals have a device, EEG device. They're sitting across from each other. Each of these devices are measuring, actually in this case, their level of relaxation, right? And so in this case, the challenge was, can you out relax your person? I know that's an oxymoron. Competitive relaxation is definitely an oxymoron. Oh yeah, there we go. I love it. I'm stealing that one, right? And to that point, you see the ball right here. And the goal is to get this ball in your opponent's gold mouth, right? And so if my relaxation levels is higher than my opponent, then the ball goes to it, right? Very interesting. I think this is from, this video should show it at one of the first early competitions, right?

No movement. You just sit there and use your brain, right? In a competitive context. Really weird stuff.

So I had a somewhat crazy advisor in grad school and he called me in his office one day and he said, hey, I see you got the drones working, right? I want you to build a system to let people race the drones with their brains. And I looked at him and I said, why would we do that? There's no literature on that. There's literally no reason to do it. He says, just do it, right?

And I'm in grad school. I want to graduate. So we just built it, right? And the trick of what he did, he emailed all the press and he kind of said, hey, we're going to host the world's first brain drone race and we're going to be the first ever to do it. And mind you, it wasn't working at the time he was sending those emails out, but you all have been around robotics. You know how that works, right? And it was fun and it was great in the lab environment, right? But the trick of the matter here is now when you go outside of the lab environment, he didn't say bring the people to the lab. He said, come out and we're going to throw this big event. And he invited the community.

He actually became really interested. because now we had to really think about real world constraints. And I don't know how many of you've seen the new VEX AIR platform. We love it at UA. It's nice. It's light. These drones were not nice. They were huge and they were loud, right? And they would cut your hand off, right? If you accidentally touched it. So those real world constraints. It didn't help that the day before the event, I got a call from the head of human resources of the University of Florida saying, you cannot fly drones on campus. This is not going to happen. You have to cancel your event. So I had to write a page long explanation of how the code works, what safety mechanisms we had built in, in some kind of way that was crazy enough to allow us to keep doing it.

And so again, in this case, we're just pairing, like how can you make sure it's safe? Because brain activity isn't very accurate. Okay, so you're not going to get very, very accurate commands from it. But if you have a downward facing camera, and at that time, if you knew how to do computer vision stuff, right, you can write these simple programs to make sure this drone never left the line. Okay, so in that case, your brain activity only or the information from your brain is only responsible for making it do one thing. And let's go for it. Right? You do that times two, you put them against each other, you bring in a DJ, it's a race. Right?

The other thing we have to do is make sure that anytime and so these drones, you'll notice that they're all kind of loud, you don't need to hear the noise twice. You can see that they're both taken off at the same time, and they kind of do this little dance. This is actually accidental. And we just thought it was cool. But because we were using this modular system where we could send commands almost in real time, and read information from the drones almost in real time across a network, we were really able to build a system that we built safe that even if at times when one drone would veer off the path, we had what we call drone referees with mobile apps on their phone. And they could just stop the whole race. And mind you, this was over a decade ago, right? Really early tick.

And so for years now annually, we've held this event called the Brain Drone Race at the University of Alabama. But then something weird happened. It started popping up at other universities. And so to date, I want to say it's like we're at nine universities, most of those who've officially worked with us in some capacity to host the Brain Drone Race. Many of many people, many universities, we also got our first high school signed up, and they're trying to be the first high school to organize a Brain Drone Race. So it's going to be exciting. And so just more media stuff. I'm going to show you that before I go to the next line.

It's said you can do a lot with mind over matter. These students are putting that theory to the test. They're using their brains to control these drones in a unique race at the University of Florida.

So, yeah, I'm in grad school. I wake up the next morning, I have 100 text messages. People are just sending me this stuff. It goes my first and last time kind of going net viral, right? It lasts for like three days and then any regular person again. But there's something that happened that actually stuck with me there. We started to get a lot of emails from high schoolers and middle schoolers, and they wanted to do it. Hey, we saw this on the news. And we want to know, and we want to do one in our school. And it rains from California and New York, Japan, Germany, right? It just kept coming in. And at the time, you know, we didn't have anything. We had a GitHub repo with a thousand lines of instructions how to get started, right? Because that's just how early it was. Right. But now we actually do have a pretty nice packaged kit. And we're actually sharing it. So if anyone is interested in joining this Brain Zone Racing League, you can scan this and you'll get set up.

We again, this is we've been doing it. This map marks actually the 10th year since that race at Florida. And after 10 years, we finally have it packaged up in a way we feel safe of actually sending it to folks.

This in particular will get you set up with learning more about the Brain Zone Racing League and what you would need to kind of organize it at your school or for your camp, whatever you want to do with that.

Oh, the light. OK. And so again, we talk about these brain computer interfaces. And oftentimes like that video from the 1970s, it seems like this world far, far away. You mentioned like, how close are we to, you know, and I'm like, well, it might be at local target. I don't know. But the challenge is if you buy from your local target now, again, you know, outside of meditation applications, there's pretty much nothing out there to support to support additional exploration that's not geared toward a research community. So how can we fix that?

You know, we got started early in the really around 2013, 2014, looking at this problem. And there was other scholars actually also talking about like, why are we only targeting BCI experts? And at the same time, you had these communities, these hackers and tinkers. Neurotech X is one of the largest and maybe still the largest tinkering community focused on this work. And they would just have hackathons and we'll get on calls with the people at Neurotech X. Let's say I don't have any programming experience.

I'm not an engineer. I just want to learn how to, you know, read activity from my brain and build cool projects. Right.

Now, enter the room block based programming, right. And so I just happened to be studying at the University of Florida where it was a lot of scholars looking at how do we do CS education? Right. And many of them served on my graduate committee. And we started talking about the idea of like, well, could we leverage this block based programming or puzzle piece or primitive as puzzle piece paradigm? And could it actually work to get folks who had no experience, complete novices about brain-computer interfaces engaged with the technology? And so there was wealth of literature out here. Alice, MIT, App Inventor, right. There was loads and loads of paper. Raise your hand if you all have used something that looked like this before. Like everybody, right.

And so we said, okay, how could we go about making the introduction of brain-computer interfacing as simple as starting a basic program in Scratch?

And that eventually led to my dissertation work. It was called NeuroBlock. We've had to change the name because I don't want to get sued. But if you Google it for most of the old work, it's going to come up under the name of NeuroBlock. And the design was designed to be very similar to what students had already experienced. You have a block based interface on the right side. You have a pane where you have the visual feedback in the top left corner. And what we did is on the bottom left, right under in this edition, right under that pane where you get it feedback, we're providing you visual feedback about your brain activity. Okay. And you have the toolbox here and everything else is the same. The only thing that we seek to change there is to insert brain activity or physiological activity.

I had some stuff about the other end. It was a whole lot of sockets and stuff underneath the hood. Most recently we've condensed it down to be as simple as connecting on the EEG device directly via Bluetooth into the software. And so we played around with all different types of design. We looked at flow based programming because a lot of the signal based stuff is more equipped for flow based design. But we've also looked at block based programming and a hybrid approach that connects both blocks and flow based programming to try to get the best of both worlds. And so in this case, students will learn about basic variables, logic, loops, but they will also start to learn a little bit about, well, I know you said that there's this Delta wave, but how do we actually construct it? Okay. We've also looked at different ways of taking these brain signals, processing them and providing feedback. So there's a, the summary of this is a lot of people do not for good reasons want their brain activity land on random server, right? We can agree to that, right? And so one of the things that we did is we built the package called BCIJS, which allows us to do most of really all of the processing, albeit simple processing and your browser.

So we could go into a classroom and completely turn the internet off and it would still work because it doesn't depend on the external server. This package is actually open source. So there's companies across the globe. They've used this underneath the hood. This is our service to mankind. I'm providing this here. And for those of you interested in looking at that, there's also multiple articles published on Towards Data Science that can kind of walk you through each step of getting started with this.

Now, up to that point, when you talked about the concept of a student in a classroom environment or student in studying a student during the learning context, it looked like this, right? Very clinical in this context. The goal here is not to engage a student with constructing anything. The goal is just to measure the student state. And this work is still really good and really important for neuroscience and folks who are interested in psychology and the brain and education. This is very important work, right? But this is not what we're doing. So oftentimes we would go to conferences and the first question would be, well, how accurate can it tell if the student is paying attention? And I'm like, I don't know because that's not what we're doing. Right? I have no idea. That's a very hard thing to do. I would probably not do that. And so because this oftentimes, our work oftentimes got confused with this work, we started to use the term physiological computing education. Okay. To really bring home that this is physiological computing, but it's tightly coupled with the concept of what an ideal education, right? And the student learning. Right? And typically we look at this as sitting right at the intersection of electrophysiology, computing, and other education research.

And some quite some interesting things have been happening recently. We talked with a school that had a sports science group and they also had a competitive programming team and a robotics team. And they were like, Oh, we could get the sports science group to get together with the programming and they could build a little out. I was like, yes, you could if you wanted to. Right. And I'll show you some pictures of some kids at science fairs where we're already starting to see trends of students naturally taking that path.

So again, this was the vision a decade ago, right? Student brain activity learning, but let's look at kind of how it looks in reality.

That's okay.

That's my favorite part where he goes from programming, moving blocks to saying, Hey, relax, close your eyes. Raise your hand. If you've ever had a kid relax or go into meditation mode in classroom, you should give this to them. So again, by introducing this novel, this new, this highly unexplored input modality in the classroom, we continue to discover new interaction paradigms in the classroom that we quite honestly haven't seen before. There's Jon Froehlich at the University of Washington.

There's my mentor from Stanford. There's like two people. That's Marcelo at Northwestern, right? That's like three. I can name them on probably one hand, the amount of people that have even like thought about it and close to this, right?

So again, because it was unexplored, we had to start, you know, I hate new terms because we have enough, I think we all can agree. We have enough terms in the world, right? To try to remember, right? But because I would always try to explain what it is and people have no idea. It's like, well, I think that means I can make a new term because it might help us communicate. And so we have a paper where we talked about a couple of things. We know variables, logics, and expression. And if you use or kind of familiar with Mitch Resnick's computational thinking framework, this is basically built directly on his work. But we just add a slice of physiological activity. So physiological variable, basically variable that's holding data related to physiological activity, physiological logic. Okay, we have a conditional statement here, but the decision is directly being mapped to some type of physiological state. Physiological expression. Well, how do students talk about this in the classroom? Right? So one thing I didn't say in that previous slide, those students had three minutes of instructions of like what a brain, maybe two minutes of what brain activity is, how you can modulate it. And they said, okay, got you.

And because they already knew Scratch, they already was familiar with a block-based programming paradigm. They immediately built a simple brain computer interface. So I would send this to like all of the older folks in my field and they'll be like, what are you doing? Stop it. Right? This is crazy. Like, this is not right. Okay. But then when we started to listen to how the students were talking about it, I say, look, girl, you're a beta thinker. Well, they're naturally using these terms in their language as they go through the exercise and adopting it. And we thought that this was really special because before then they had no idea what frequency bands were. Right? Most folks don't because you don't need to. Right?

Usability was good. We checked that they were able to use it. Great.

Okay. So virtual. Check. We got it in the virtual environment. A few years ago, I was fortunate and won an NSF career award. And they said, well, you need to make something. It's a five-year vision of what you want your career to be. So you got to be bold. Similar to what Jason was saying in his keynote. And so I said all this kind of stuff like, oh, we're going to have kids flying drones with their brain and they're going to be able to build it. And I didn't know how hard that would be. Right. But we did see the impact that it would make in the classroom. This was a tough group of kids.

If you look at the expressions in the background, this is also one of those videos that let me know that, okay, we might be onto something. Because when I walked in this classroom, everyone has their heads on a desk. They did not want me to be there. But we brought out the BCI. We brought out the drones. They were very engaged. And this was actually our first group of students that we worked with. And we eventually created a system called Physiobot. Another name scheme. Because Neuroscope was also a name. I was like, well, someone's using that. Let's use another one. Right. And it's a very simple interface. You have your brain signals on the left side. Or you can't see this here. It's four channels from the device that I have here. These are the feature bands represented as a bar chart. So it's easy for them to kind of see things that brain activity modulate. You have the toolbox, block, interface, you know, a few menu options. Okay. So we tried to make it as simple as possible, but still staying true to the idea of designing and being able to tinker with a brain computer interface.

So the next question was, what I think is fun, but let's see if it's actually causing any change in the learning outcomes. Now we had a small sample size. So when we did this study, at least for now, we don't really see any differences between just using a regular keyboard and using physiological activity. When you look at programming confidence or self-efficacy, at least for that initial introduction. Right. So we look, we say, okay, let's look back and analyze the qualitative data. Let's look at and compare how they talked about their experience in interviews after using a keyboard and after using physiological activity. And the thing that we found out, this is jumping off the papers that they're way more curious about STEM and what they can do in the future and what careers they could take off that's related to this work. Right. So we're going back to the drawing board now and trying to find some quantifiable way, right, using Likert scale or some like standardized survey to study this idea of curiosity. Right. Are you becoming more curious because qualitatively it seemed to suggest it. Right. Now, the other thing was that, you know, I showed you this, but I had the opportunity to really find out if this stuff was really ready for game time or not.

Anyone ever been on live news before like live news? So what's the difference between live news and like recorded videos?

Yeah.

Yeah. Yeah. You can't mess up. Right. And lo and behold, CBS said, Hey, show this on TV. That.

Oh, boy. Show it live on TV. And we pulled it off. And, you know, I was really confident on the technology at that point, because we had just come off of like cycle of a lot of demos with this and it was just working and didn't break. It was bulletproof. And I said, OK, in this video, it does not break. I don't know if I have time to this. You could look at this full video on our YouTube video. I mean, our YouTube channel.

But that was the moment that I said, OK, well, this is why I feel comfortable sending it out to groups.

We sent it out to groups. Here's how you get started. There's a getting started tutorial on our YouTube as well. And we started to share it with a select few amount of people. Right. And I have the link for you as well. But actually, you sign up for that previous link, you automatically get announcements about our educational software. And so as the software started to trickle through random people, started to send me emails about, hey, we won first place in our science fair. And I'm like, how? What did you do? And the students were actually going inside of the console, manually looking at the raw output of their brain, charting it down, doing averages and winning science fairs with it. So here are the students taking the software, using it in a totally different way than it was designed for. Now, for the summer, we had to build a whole new data science arm of the tool. But to me, this kind of goes back to why we're continuing to explore this. There's definitely something here. It's still being refined. But if it was one group that sent me pictures of them winning, you know, science fairs would be different. But it's multiple groups. Right.

Last thing I kind of talk about is this idea of brain activity is really hard. So if you're thinking about going into your classroom and using brain activity immediately for activities, don't. Don't do that. All right. Start here first. This is what we call muscle computer interface. These are the partners we work with at OpenBCI. In these leads, you can connect these to your arm and it measures muscle activity.

So some of you may have seen Meta's new glasses. They also were marketing this wearable device, NeuroBand or something, that allowed you to control your wearables using muscle activity. This is why we call this next generation. You're not going to see this really widespread this year, maybe not even next year. But there's a lot of companies that's exploring this new frontier.

So the idea is that we can capture this electroactivity being passed from your brain to your arm through these leads. And we just basically redid the same thing we were doing in our earlier studies. Let's integrate it into a software and explore how students would engage with it. But you know here, this was the virtual environment. Anyone take a guess on where we go next? What was the next step?

You got to add a robot, right? Oh, our nice VEX AIM. And let me tell you, it's nothing like seeing 60 kids running around with VEX AIM robots and moving it with their muscles and so forth. And it's a video clip of a recent video. And you can kind of see there's programming involved, but the interaction is instead of learning, I'm all the way into it too, instead of being focused on using a joystick or keyboard, they're learning about these signals. They're learning about their bodies.

And they're engaged.

Now, this is, you all are the first people to see this. Anyone heard of VEX AIM before? All right. Okay. Yeah. So we have just completed integration. It's not complete, but the finishing touches is of using this Muse device to control the VEX AIM. And so again, if you signed up for that link, you'll get announcements about kind of what this is, how to get started, and resources. I'll get started with that. So there's also some work we did with machine learning and AI, because if you don't do that, apparently you'll get stones thrown at you now, I guess.

So anyway, use Teachable Machine. Okay. Awesome. I love Teachable Machine. I love that idea of collecting the data. Training is really simple. We love simple. We do this in a lot of classrooms. And so we do have a version of this where instead of focusing on capturing image information, we're capturing motion activity during data collection. We take them through a process of learning how to train the model and looking at labeling different groups of data based on the EEG. Model evaluation is where all the basics. And then at the end, they get to play a game using their model and see kind of if it works and they can go back and stuff like that.

So this work would not be possible without all of these great students I've had throughout the years. Like I mentioned, we're at the University of Alabama. This is what I do for my day job, brains and robots. So if this is interesting to you, feel free to send me an email. You can check out some of our other work at my website below. So I guess we have a few minutes for questions. Thank you. Thank you.

This was a great presentation. I'm actually heavily into BCIs, but for a different reason. Really for AI and VR technologies, because I feel like that is going to be the control medium for once those technologies become more mainstream. Have you ever thought about applying your research to VR and AI technologies? Because I know Valve, they have a couple of... I'm going to actually go off script on that one. Let me see. Oh, if I could connect to the internet, I want to show you something. I think it's really important.

The company that makes this device have a headset called the Galileo. Galileo. And it's all about virtual reality, EEG, eye tracking, like all of it. We have some stuff in the works with that headset that I can't really talk about now, but yes. There we go. Awesome. Yes.

I wanted to ask because I told you that my son was at UA for that next competition a couple years ago, but did y'all ever get the... Because y'all were using the Muse, I guess, plus you were doing the muscle. And y'all were flying the drones with the kids using those two concepts. So have y'all fully integrated muscle with brain activity for that drone flight? Like at the same time? Yes. No, we haven't. That's a good idea. There's no reason we couldn't. That would be like multimodal. We could do that. Yeah. Okay. And this is why I love coming to conferences. Someone always says, oh, we didn't think about that. We definitely could have done. That's a good one. All right. Well, if there's no more questions, thank you so much. Thank you so much, Chris. Let's give a round of applause.