(pleasant music)

Hello, and welcome back to the VEX classroom. My name is Audra, and I am so excited to be back here to learn some more about the AI vision sensor with you today.

Previously, we learned how to configure our first color signature and then make that signature a bit more resilient. In this video, we're gonna build on that practice so that we can start to use some of the other data that we see in the AI vision utility in a project, in order to make our robot move based on the location of the object that we're detecting.

We're going to learn how to use data from the sensor in a project to make our robot turn, to center on an object, and to drive towards an object that is detected. One of the best ways to understand how our robot's going to need to move in relation to the objects in front of the sensor is to be able to essentially see what the sensor is seeing, right? We did that previously using the AI vision utility. So that's what we're going to do here again.

We're gonna get started with that AI vision utility, so we can dive a little deeper into the data that's presented within that snapshot window. Here I am in VEXcode EXP. To get to the AI vision utility, I'm going to open up the devices window. I already have this configured. You can see my previously configured color down here. Now, I'm gonna select configure to open up the AI vision utility.

In my snapshot window, I can see essentially what the AI vision sensor is seeing, everything that's within that field of view. You can see here that I have this blue object that is detected. If I move it a little bit closer, we can see that around that blue object is this box. There's a little plus in the center of it. Above it, we're given some data about this object. We can tell that it's blue because it's got the same name as the color that we configured. Then we have some numbers for an X and a Y value, and a W and an H, or a width and a height value.

Where do these values come from? What do these values tell us? To better understand the X and Y and the width and height, it's important for us to understand the resolution of this snapshot window here. The resolution is basically the number of pixels that exist within the snapshot window. All of this data is given to us in a value of pixels. The resolution here is going to help us to determine how to use this data effectively.

You may have heard about the difference between low resolution and high resolution when you're talking about televisions or monitors, right? High resolution means that there are more pixels present within an image. The more pixels you have, the smaller those pixels are, so your image becomes clearer. On the left-hand side here, we can see each individual pixel in this image, and those pixels are pretty large, right? That means that our resolution is low. We can see each of the pixels, and it's going to make the image a bit blurrier. On the right side, we can't see those individual pixels quite the same way, right? There are more pixels packed into each little area of this image, and so the resolution is higher. Thus, the image is clearer.

The resolution of the AI vision sensor snapshot window is 320 pixels wide by 240 pixels tall. Here, we have a rectangle, like the snapshot window, right? 0,0 is up in this top left corner, which means that this is zero on the X axis that goes 320 pixels wide. Our X axis goes from zero on the left to 320 on the right. We also have a zero here for the Y axis. Our Y axis goes from zero at the top to 240 at the bottom. So it's 240 pixels tall by 320 pixels wide. You could imagine this almost like a sheet of graph paper that had 320 squares wide by 240 squares tall.

So if we were to think about locations of objects within that 320 by 240 resolution, we can think about that in terms of the center point of each of these axes, right? So if it's 320 across, the center of that would be at roughly 160. If it's 120 tall, the center point of that would be at roughly 120. The exact center of the field of view of the AI vision sensor is going to be somewhere around 160 for an X value and 120 for a Y value.

If we're looking at the center X value of an object and we see that the center X is 300, that means that the center of the object is somewhere all the way over here, to the right of the robot. If we see that the center X value is 25, we see that that's all the way over here, right? It's the same way for the Y values. If our Y value is 50, that's going to be somewhere up here, towards the top of the field of view. If the Y value is 200, it's going to be down towards the bottom of the field of view. Knowing these landmark numbers can help us determine how to use that X and Y value in a project to make our robot move effectively in relation to the object that is being detected.

So now, if we go back to the AI vision utility, I'm gonna select freeze video here just to get us a freeze frame of this image to make it a little easier to talk about (indistinct).

[Music Cue]

Now we can see that the center X, which is the X value of this little plus in the middle of the object, is at 109 pixels. If we were to think about that zero value here and 320 value here, 109 would be right about over here. We can see that if we were to draw a straight line all the way through. We know that the center point between zero and 320 is 160. So if 160 is in the middle, that 109 is going to put us somewhere on the left side of this field of view. The center X value is telling us right now that this object is to the left. The Y value is at 150, so the center of this object here is at 150. We know that the center point between zero and 240 is 120. That's going to be a little bit above that Y value, right? So we can say that, based on this Y value, this object here is to the left of the frame and towards the bottom of the frame.

We'll go into more detail about the width and the height a little later in this video. But let's take a look at how we can use this information to our advantage. So I'm gonna unfreeze this video, and now you can see, when I move this ball, I moved it all the way over. Now my X value is 284, right? So now it's all the way over on the other side of the field of view. If I move it back over to this side, we'll see that that number can go down really low. If I move it up on the Y axis, that number's gonna go down. And if I move it down on the Y axis, that number is going to go up.

Now that we have an idea of how to use these numbers in relation to the snapshot window, let's take a look at it in relation to our robot. If we look at our robot and where the AI vision sensor is placed, it's not directly in the center of the robot. It's a little bit offset. That means when we're thinking about how to get an object in the center in relation to the robot, it's not going to be exactly that 160 mark. We can see here that the X values are closer to 200 because of that little bit of offset. But we can use that approximate value of 200 to establish a range that will get us roughly centered on this object.

Let's take a look at a project that does just that. Now that we've found out what that center value is, we can establish a range of what we'll consider centered on the robot.

[Music Cue]

Thank you for watching this video. We hope you found it informative and helpful. If you have any questions or feedback, please feel free to reach out.

[Final Message]

Stay tuned for more tutorials and updates. Have a great day!

When I run this project, the robot is going to turn left or right depending on where the blue ball is detected in order to center on it. If I were to move the blue ball, the robot would continue to move to keep it centered within the range we established. Let's take a look at the project to see how we're using this data effectively.

Here's my project, and let's read this code together. This starts off the same way as our other project, where we're setting the turn velocity nice and low to ensure the robot moves slowly enough to get the most accurate data from the sensor. We have a forever loop to continuously repeat these behaviors. Just like our previous project, the first thing we want to do is take a snapshot to see if blue is detected by the sensor. We want to ensure there's a blue object within our frame of view. If there is, we want the robot to turn left or right depending on the value of the center X.

There are two "repeat until" sections of code here. One is to have the robot turn left until the center X value is over 190. The other is to have the robot turn right until the center X is less than 210. We know that 200 will approximately center our bucky ball on our robot. By establishing a range between 190 and 210, we're saying that is an acceptable range of pixels. That's a small enough margin of error to be able to center on it.

Something else you'll notice is that within the "repeat until," not only am I turning left, but I'm repeatedly taking a snapshot and turning left together until that value changes. Why is that? Each time the robot moves, the center X value of the bucky ball changes as well. If I never took another snapshot, we'd only be using the very first one we took, and our robot would never get to where we wanted it to go. We have to continually take a snapshot over and over again until we reach the center X value over 190, then stop driving. Or continually take a snapshot and turn right until we reach the center X value of less than 210, and then stop driving.

Now, if we were to go back to look at what our robot is doing, if the ball moves, the robot then turns along with it. It's repeating these behaviors because of that forever loop, and it's continually taking a snapshot. Each time I move the ball, or each time the robot moves, the center X is changing as well. We want to keep track of those changing values to make this project as accurate as possible.

What if we wanted to drive our robot closer to this ball? What if we wanted the robot to drive until it was approaching the ball? What piece of data could we use to make that happen? Let's go back to our AI vision utility and look at some of the width and height data we get about the objects in that snapshot window.

Here I am back in the AI vision utility. You can see there's a box around this blue object. We have a plus here to tell us where the center X and center Y are, and we have this box around it as well. This box is roughly the width and height of this object within this frame. If I freeze the video here, we'll see that the width and height are reported as roughly 53 pixels wide by 53 pixels tall. If we think about that pixel grid we talked about before, where zero to 320 across the X axis and zero to 240, and we said, okay, 53, if we've measured from here to here on the X axis, yeah, that would probably be about 50 pixels wide. So why are these the same values?

Thank you for following along with this project. I hope you found it informative and helpful. If you have any questions or need further clarification, feel free to reach out.

Thank you once again, and happy coding!

Well, we're looking at a bucky ball, right? And a bucky ball is round. So the width and the height are going to be in about a one-to-one ratio. When we're working with something like a bucky ball, we can kind of use the width and height here a little bit interchangeably. So we're gonna focus just on width for this project.

Now, something that you'll notice is that as the bucky ball moves, that width and height value changes. Right now, the ball isn't changing in size, but the amount of space that it takes within the frame is. If I move the ball backwards, further away from the robot, now my width is 29 and 30, right? Again, the bucky ball hasn't changed size, but it's taking up less space within the frame because it is further away.

As the bucky ball moves closer to the sensor, it gets bigger, right? Now, the bucky ball, again, is not changing in size. But now, if we freeze it, we can see it's 190 by 185, right? It's much, much bigger. It's taking up much more of the frame because it's very, very close to the sensor. We can use that, again, as we're thinking about where we want to move our robot in relation to an object.

If we know that this value is lower than a certain amount, that means that a lower value, right, means the robot is further away from the object. If the value is above a certain amount, it means that the robot is closer to the object. So if I were to put this right on the front of the claw here, and you can even see the very front of the claw in the very bottom of this snapshot window, I can see that my width is about 73. Now, here, the height has changed because the claw is in the way, right? So my width is 73.

I know that when the robot is really close to the ball, that width is 70. So I don't necessarily need it to be right on top of the ball, but I know that when it's basically right in front of it, it's somewhere over 60. Right here, I can move it a little bit further away and say, all right, we're still in the 60s. So I know that the threshold value I wanna use here is 60.

Let's take a look at a project that will use that width value in order to drive our robot towards the ball that's in front of it. Okay, so here we can see that my bucky ball is a bit far away from my robot, but it's directly in front of it. Now, I'm not concerned about the left or right, now I'm concerned with getting my robot closer to that bucky ball. When I start this project, the robot's going to drive forward until the width of that object is greater than 60. We can see that now it's stopping. If I were to move the robot back, it would do that behavior again. If the ball were to move a little bit further, the robot would continue to drive until that value was greater than 60. We can see that this continues to work, and if it gets closer, it'll stop a little bit sooner.

Let's take a look at the project that we're running here, to talk about how we're using that width data. Here's my project. Much like the other projects we've run, it all starts the same, right? I'm slowing that drive velocity way down. I've got a forever loop here to be able to repeat these behaviors over and over again. The first thing is to take that snapshot to ensure that I'm detecting blue somewhere within the field of vision. Then if that object exists, I want to be able to drive the robot forwards until the width of the object is reported as greater than 60.

I want to drive forward and take a snapshot repeatedly, over and over again, until that width data is greater than 60. I'm gonna repeat those behaviors over and over. Once the width is greater than 60, we'll bounce out of that loop and stop driving. As the robot's moving forward, each time it's starting to drive forward, the width data's going to change. I need to continually take a snapshot over and over again in order to get the most up-to-date data.

So that way, my robot can stop driving when necessary. Let's take a look at that project running one more time, just to see this in action again. When the ball moves, the robot can respond now because we're, again, using that threshold of values. I'm not saying I need it to get to exactly 60 pixels wide. But I'm saying as long as it's greater than 60, I know it's close enough for my purposes.

I hope that you now have an idea about how to start building some projects using additional data from the AI vision sensor. We talked about how we can use the center X value to locate an object left or right, or the center Y to determine whether it's up or down within that frame of view of the AI vision sensor. Then, we use the width value or the height value to drive our robot until an object of a certain size is within that frame, to know that we are closer to it.

You should go and try these projects for yourself. Build them on your own, test them out in your setting. See what you learn about how to make the best use of the data for your purposes, and post about your projects in the community. We'd love to hear about how you are using the AI vision sensor yourself or with your students.

If you want to schedule a one-on-one session to talk more about using this sensor and different projects that you can build with it, we'd be happy to do that at any time.

I hope that this video has made you excited about using the AI vision sensor. I look forward to seeing you back here in the classroom sometime soon.

(pleasant music)