(upbeat music)

VEX Robotics Educators Conference. Welcome, everyone. Please join me in welcoming Dr. Stefik from the University of Nevada, Las Vegas. This session is titled "The Wild Side of Accessibility, What Is Possible?"

Stefik invites us to explore how accessibility, in all its complexity, is not just about disability, but about how the full spectrum of being human—from how we see, hear, move, or think to how our abilities change across a lifetime. This talk encourages us to rethink what's possible in technology and education when we design inclusivity from the start. So get ready for a fascinating and thought-provoking session. Let's please welcome Dr. Stefik.

(audience applauding)

Good afternoon. I know I'm competing with lunch, so I'll be sure to make this talk as boring as I can to try to make it worth your while. If you don't know me, I'm a computer scientist by trade. I study accessibility, and what that means is a little funny and complicated. I also study something called human factors, which is essentially a lot of details about evidence related to how humans use things, and then I also study evidence practices, which kind of means how do we know what we think we might know, right?

My daughter is older now, but still has a tongue, it turns out, and I still have hair, so that works out really well. So when people think about accessibility, it's very natural to think that this means one group. This means the kid that doesn't see so well, or hear so well, or something like that, but that's not actually what accessibility is. And part of what I want to do today is I kind of want to talk about what accessibility means, and the reason I want to talk about that is because it kind of is a lot more broad than many people realize.

So one way that we talk about this is through these principles that are sometimes called POUR, and it has four key ideas, and they're pretty simple to understand. One of them is, is it perceivable? This means information should not be invisible to all of the user's senses. Now that sounds a little bit odd, but it just means, if you can't hear, you should be able to see it. If you can't see very well, you should be able to hear it, or touch it, or something else. One of your senses should be in the running, right? That's the idea.

It should be operable, and this one is a little bit strange, but really it's talking about you shouldn't have to use one particular input technology or another, so for example, many applications you only use the mouse, and that isn't allowed, in fact, it's illegal in some cases, but we'll get to that. Things should be understandable, and this one is actually very difficult to understand. Ironically, the requirements around understandability are very hard to understand, which is both ridiculous and ironic.

And then robust, robust means something very specific, but it is kind of tricky to understand, it actually has to do with robots, not specifically effects robots, sorry guys, but robots that can actually connect into a technology, that's a little bit weird and roundabout, but that's roughly what it means, and I'll explain in a little bit later. But here's the thing. Most of accessibility is about good design.

So let's think of an example. On the screen is a picture of a book, it's a pretty famous book if you've never read it, it's also very entertaining, if you've never read it, it's the kind of book you could give to a kid, it's pretty understandable for even young people, it's called "The Design of Everyday Things" by Don Norman. It's not an advertisement for Don Norman's book, I really just put it on there, because there's a teapot on the book, just because it's kind of a relatively smaller crowd, can anybody shout out what's wrong with this teapot? It's backwards. It's backwards, right?

Now, on the one hand, if you think about it from the perspective of perceivable, operable, or understandable, well, you can kind of, I mean, there's no robots that connect to this. I mean, you could, but that's not really the point. Is it perceivable? You can perceive the teapot. Good job, right? Is it understandable? Well, yes, you do it backwards, and kind of you can understand it, it's just ridiculous. But is it operable? Not really, right?

And it's a famous example from Don Norman's book, not because you couldn't design something this way, but because you shouldn't design it that way. Most of the time, when you have applications that are not accessible, this is what they mean. It doesn't mean you've necessarily excluded any one group. What it means is that you've made a backwards teapot.

So let's take an example, and I'm going to show you a bunch of examples, because it impacts you in ways you probably don't even think about. So on the screen is a picture, and it has for, I think everybody in this room can see, for those that can see the picture, how many of you, by show of hands, find this hard to read? That would be 100% of people, or no, 98%. (audience laughing)

Now, how do you know that it's hard to read? Turns out there's actually a mathematical equation, where you can calculate how hard it is to read. I won't go into the details, but it's pretty simple. If you've done basic algebra, you can do this little equation by hand. You multiply the red times a number, and the green by a number. It's a thing.

But here's the thing, interestingly, this is actually ironically Apple's Accessibility Inspector. This is the application that you use to check accessibility on Apple, and by their own definition of accessibility, Apple's Accessibility Inspector is actually not accessible under the legal regulations around accessibility. I just find that very ironic because it's hard to read.

Interestingly, if you are a blind person, this is actually operable and perceivable. You can listen to it with sound, and touch it with Braille. But if you are an average typical person that can see, you can't read it. That's accessibility. It's not about one group, it's about thinking through all the groups from birth to death, all the various limitations that many humans have, which change over time. As you get older, your touch perception goes down, your sight goes down, it gets harder to hear, especially high frequencies. These are really common things. This is what accessibility is, it's just about how humans change over time.

So let's take another example. What about the Python programming language? Is it accessible? This is kind of counterintuitive. Is it perceivable? Well, it's text, right? That's perceivable. You can see it, if you listen to it, you could hear it, if you touched it in Braille or something, you could do that, that'd be fine. Is it operable? Well, if you put text in a text editor, you can use it, right? That's not really debatable. It is definitely operable. You can actually write text editors, I'll give you an example of one later where they broke comparability, but you can invent a text editor that's not operable, but you kind of have to try hard, you have to like almost be really, really bad to get it to break, 'cause it's just text, right? It's a text editor, you can download it, you call it, you press the button, good.

But is it understandable? One of the ways I want you to think about this, and I'll show you some actual evidence on this fact in a minute, but I just want you to listen to the Python code out loud, and then I'm going to ask you, what does this code do. Is it understandable? If it's understandable, you should be able to figure out what it does.

Thank you for your attention and engagement. I hope this discussion has provided some valuable insights into the importance of accessibility in design and technology. Please feel free to reach out with any questions or for further discussion.

Thank you once again, and have a great day!

Import seaborne as SNS, import stats.models.API SSM, from stat.models.formula.API import OLS. Crashes equals SNS.load_dataset("car_crashes"), obviously. Model equals OLS("INS_losses ~ speeding", data=crashes).fit().

Now think about it. If you have trouble seeing, you do have to listen to this. It's not as crazy as you think. AOV_table equals SM.stats.NOVA_LM(model, TYP=2). So any ideas? What does this do? No clue. No clue? Import libraries. Import libraries. Yes, it does.

Well, it turns out, other fields of studies have used a particular standard to try to evaluate whether something is true or false, right? They use standards, we call it science, but it's not that specific, it's more specific than that. My point is, keep this in your head, it's not the fact that you're hearing it in audio that makes it not understandable, it's something else, which we'll get to.

And in science, by far the most common technique is called the randomized controlled trial. This is how we evaluate whether claims are true or false for common evidence things. This is very famous because of this guy, named Austin Bradford-Hill. Just by show of hands, how many of you have heard of Austin Bradford-Hill? Anybody? A few of you? Well, all of you have, even if you don't know it, 'cause this is a guy that figured out that smoking causes cancer. If you don't know, smoking causes cancer.

And the way that you do it is checks and balances. And it's really that simple. You make studies that have a wide variety of checks and balances, not just in the lab, not just in the field, have a control group, make a placebo, do this thing, do that thing, there's all sorts of little rules that they constructed to try to figure out not whether something is correlated, but whether it causes, right? Those are very different things. Very hard to understand too.

Interestingly, when this technique came out, literally the studies around smoking and cancer became so popular that there are formal equations showing how quickly these were adopted across science, to the point where there's, I mean, can you imagine you're a scholar, your impact is so large that people derive equations about your impact? That's wild to me.

But computer science, unfortunately, didn't get the memo. They were like, "Hey, hi, this is Austin Bradford-Hill, I just figured out smoking causes cancer. Maybe you should try some science." And they said, "Nah."

In fact, this is so famous now, there was a dissertation done by Autie Hanaka Yanaho in Finland, it's a little hard to say, but that's his name. And they showed that between 1976 and 2012, there were only 22 experiments, in medicine, it was like 114,000 in 1990 to 2001, somewhere around there. But in computer science, it was only 22 experiments from really about the 50s till 2012 on programming languages.

Now you might say, "That's really esoteric. Why would that matter?" Well, I'll tell you why, 'cause nowadays, those programming languages impact half a trillion dollars per year in funding. And we've done 22 studies before 2012, 22.

On top of that, there was a paper done by Amy Ko in software engineering, and they found evidence that most of these papers in software engineering, not just programming languages, but software engineering, had such poor methodologies that they forgot things like gathering data, really, gathering data was left out accidentally. They tested only themselves, I tried the drug on myself, and it was great. Or a control group, literally control groups started being used in 1834, right? This is not like modern marvels, this is like basic checks and balances.

It came out, 'cause of homeopathy, oddly enough, the homeopathists accidentally disprove themselves. It's a funny story if you've never read it, they were drunk at a bar, it's a thing.

Anyway, the point is, when a lot of us started figuring this out, we said to ourselves, "Well, what if we actually ran a study? What would we find?" And unfortunately, because there hadn't been that many experiments, what we found was actually kind of embarrassing.

One of the first things that was tried was, why don't we just rip out all the symbols from programming languages, and then randomly inject characters? This was called a placebo language. The idea was to generate a programming language so bad that no rational person would think it's reasonable, right? Like if you have a robot, you want to decide whether you go left or right, that might be a conditional or an if, and we're just going to use pound for no reason. Maybe you want something to do something over and over again, and obviously in computer science, we say four left paren anti equal sum equal zero semicolon, and I++, right paren left brace, which rolls right off the tongue, but instead, we'll use right slash for no reason whatsoever.

But it turns out, a lot of the designs in C-style syntax are so bad that they're no better than random characters, that includes in the language Java, and Perl, and the causes are known. You can actually calculate the probability of particular tokens causing the problem. One more little fact on that: many programming languages use words like for, while, and for each in their concepts to tell to do something over and over again. Studies on wide collections of words actually show those are the three worst choices to use for human beings. Not surprisingly, words like repeat make sense to people. Loop.

And if you pull this out for statistics, the situation gets even worse. In fact, if you take Matlab, Python, and R, and there's a bunch of like nerdy stuff on the screen, but if you take those three languages, and you just ask people, "Do these words make any sense?" And then you compare them again to placebo languages, it turns out, 63%, 62.2%, and 30.4% of the words are no better than random. So literally you compare the words in Python to blurft, or Glarg, or narc dark, or whatever random thing that the students came up with, and it turns out it's no better than what was actually chosen by real engineers.

What it tells you isn't that people did a good or bad job, what it means is that it really wasn't vetted, right? It just, someone chose, they put it in the language, and they called it a day, and people sort of said, "It's great" or "It's bad", and kind of argued.

All right, but what about LEGO? Is LEGO accessible? LEGO's not Python, like LEGO's a higher-level language, so truly, it must be more accessible. Let's ask, "Is it perceivable?" Who says "Yeah"? Now don't forget, if you can see it, but if you can't see, can you hear it? If you can't see or hear, can you feel it? That's harder to validate, isn't it? You kind of have to know how to test a little bit. The answer is no, by the way, definitely not.

Is it operable? Interestingly, even if you turn the blocks off, their editors are now taxed, so in three they should work, and be operable, but it turns out, LEGO, as far as I can tell, forgot to allow things like copy-paste in certain regions, so like yes, those impact people that are blind or others, but like I like to copy-paste. These are accessibility rules, but also like copy-paste, it's like it's really common things. I think it's part of their fine features that are buggy.

Is it understandable? Well, interestingly, one of the things that I think them and other robotics groups have done a good job of, this does feel pretty understandable to me if you can see it. I don't think they're alone in this, lots of robotics groups have done a good job about that.

And then finally, robust. The question is, can little robotic agents connect into here for accessibility technologies? And in this case, the answer is likely no. But you don't have to guess.

There is some really interesting work that's been done on this whole kind of blocky or visualization type idea. One of them, which I know some people in here have heard talks from him before, is by David Weintrop. There is some evidence that these kinds of environments do actually help people. The only catch is that the amount that it helps people is really small. Notably, the technical term here is called a Cohen's d, which is not very understandable. But all it means is like a .4 standard deviation's difference. In other words, if you start with blocks, you learn a little bit more, but not very much. What that means is, it does help with understandability, but most of the major block languages aren't accessible for perceptibility or other reasons. So the question is, can you get both or not?

One final example. Can we make 3D modeling accessible? Who says yes? Who says no way? Not easy though, right? What I'm saying is like the kind of the wild side, this is kinda what I mean, like how far can the rabbit hole go, right? So I started a project, this started pretty humbly. I was working with some blind children at the Washington State School for the Blind, and I had them basically trying programming languages, and then being like, "Dude, what does four mean? I don't know what that means, I'm 12 and I'm hearing it through audio. Is it the number four?" Eventually, we started running studies as we learned that the evidence standard was so poor in computer science. It turned out we started to get evidence on the word choices, things like repeat 10 times make more sense to humans, not children, not professionals, humans, than for left paren, et cetera, et cetera.

After some of the work by Weintrop and others started coming out showing evidence on blocks, the question became, well, a lot of the educational products heavily used graphics, but can you make that accessible? So we started trying to figure out, can you actually make graphics accessible? I had a "Gedankenexperiment", which is a thought experiment, if you haven't heard the term, it's German, and it's basically what would be possible if we made the graphics accessible? Is it even possible? So I had an idea, what if what we do is we explore some graphics pipelines, and then we explore the accessibility pipelines, like how does it work under the hood? What do these really do, and can they be somehow told to talk together?

After some time, I determined that there was a fourth step, which was cry and repeat. At this point, I wasn't even sure if you could do it, and there were lots of technical reasons why, but don't worry, we solved it pretty quickly. It turned out seven years later, until we had our eureka moment. By the way, and I should say here, a lot of this stuff is funded by the National Science Foundation. It's an organization to actually give scientists time to think, sort of a hallmark of the US system. But eventually, it looked like we think we cracked it, this part through cry and repeat, that's the part that took seven years, and then eventually we had an epiphany about it, which is a long story. Unfortunately, because of the way that epiphany works, it had an accidental side effect, which is kind of unfortunate, and what it means is you then have to rebuild everything with accessible graphics from the get-go. But it is possible to rebuild everything with accessible graphics at the get-go.

Let's talk about it. Let's take a temporary nerd break, and I know what you're thinking, "Isn't this whole talk a temporary nerd break?" But my point is, let's take an even nerdier break. The question is how does this stuff work under the hood? Now, I'm not expecting you to walk away from this talk knowing the details about accessibility programming, it's super nerdy and really hard.

But different operating systems, it turns out, have different ways of managing accessibility. And this is purely a technical issue, right? If you get this part right, a lot of it becomes accessible on top, right? On Microsoft, they call this user interface automation. It is 100% incompatible with Apple's inaccessibility, which is 100% incompatible with Apple's iPhone accessibility, unfortunately, and that's 100% incompatible with Android's View API, which it turns out is 100% incompatible with the way that you do it on the web. Whether you do it on the web connects down through all these things, and then on top of that, graphics, graphics themselves, when they put stuff on the screen, they render pixels everywhere.

It turns out they have under the hood, there's a way to send stuff to the OS that's called a server. And it turns out, for graphics, these were broken. I won't get into the details of how these work, 'cause it's brain-bending, but the idea is, if you can figure out how to make a server that can somehow connect to the graphics, all of a sudden, you can have blind-accessible 3D games. You can have 3D modeling. You can be a CAD engineer, maybe you can be a visual effects engineer, seriously, as a blind person. And that's fine. Maybe.

So we thought we can't build this into every programming language, we're a research team, but we can build it into one, so we've been already studying how do you make the languages themselves accessible, repeat 10 times, that's, and then maybe adding some blocks, but there's a lot of graphics even in kids projects. So here's a picture on the screen of Scratch, I think some of you may have seen it, a lot of the blocks in here are from an underlying technology called block lead, which is used by all sorts of providers, all sorts of providers. But on the screen, there's a cat that can do things, there's a video, which has requirements around it, there's actual code, the little blocky things, there's menus, tabs, blocks, controls, all sorts of stuff. You have to get all those accessible, and what does it mean for those to be accessible?

And to make matters worse, when you want to make graphics accessible, it turns out there's an extra problem that has a name that we made up, called accessible starvation. Now, I'm going to show you a demo of this, 'cause it's really funky what it does. The basic idea is this, in graphics, you have eight frames of animation, right? Just like in a movie, like a movie might be 24 or 48 frames per second. But interestingly, when you send accessibility events through a server, it does this process called pumping. All it really means is that you're sending down an event, you're getting back a response, and you kind of go back and forth with the systems.

So let's kind of think that through. Imagine I'm drawing a picture on the screen, which is all you do in graphics, you're just drawing a picture on the screen, a robot, a game, whatever it is. And the frames of animation are going by. Well interestingly, the graphic system and the accessibility system, for whatever strange reason, for whatever strange reason, go through the same pipeline, literally, like where a blindness work type goes is being sent in the same place they send the set events for the graphics processing unit. Why is it done that way? I don't know, but it is. And interestingly, this means that they can be interleaved with frames of animation, which is counterintuitive.

So think about it. User has a fake text box that you're drawing, you tell the operating system, it's a text box, and you press the up arrow, and it says, "Great, well done. That's an up arrow. What are you? Oh, you're a text box. Fantastic." Wait a frame of animation. Hey, what was the font? Oh, it's Comic Sands, excellent. Wait a frame of animation. What line were you on? Because it has to speak that line, right? Or put it on Braille.

Great, wait a frame of animation. And this goes on and on and on, and it isn't three events, it's hundreds that it's waiting, the font, the line, the font size, the indexes, all sorts of stuff that you wouldn't imagine.

So let's watch. Here on the screen is a 2D game editor, it's one of the kind of initial experiments that we did as we were crying for seven years, metaphorically, and literally. And what I'm going to do is I'm going to show, I'm going to play a video, and what's going to happen on screen is, and I'm going to walk over here for a second. This little marker right here is blue, you can't read it at all, but it's irrelevant to the conversation, it's going to move from here down one spot to there. That's it. One visual change. But what you're going to notice is that in the process of those frames of animation going, it starves the accessibility system.

So I just want you to watch what accessible starvation sounds and looks like, so you can identify it.

[Music Cue]

So let's go, and I'll say flip when it flips. Flip. Level three grass, one of 495. 12 second delay per character press. 12 seconds. You can't use it at that point. So I guess you're hosed, you can't do 3D modeling, except there are some tricks, and that's part of that seven years of pain. Oops. But there could be limits, so let's kind of talk about those.

So first of all, let's think about the input. So oftentimes with kids' products especially, it's become more popular to use blocks. This actually makes sense. There's evidence that these are more understandable. That's a fact, right? There is evidence, it's not a big effect, but it is an effect, right? But they have to follow perceptibility, they need to be operable, they need to be robust, they need to be understandable, and so things have to change with block languages, in order to do it, one of them, we should not be using spatial layout for block languages, it's not like it's impossible for operability, but the debate is, if I'm at a block right here in the middle up part of the screen, and I press the down arrow, which one do I go to? Well, is it like in the, do you like look at the X coordinates, and then kind of move around? I don't know. You can make choices, but that makes no sense.

And more than that too, that is something really that only exists in kids' products. Adults don't do this. This is not how you code. There's a code, and it goes line by line. And maybe that's bad for some reason, but if I as a professional had to code, where everything was spatially dispersed, I think I would go insane. And it's also an accessibility issue.

But the second thing is blocks probably shouldn't be only for kids. There's actually an authenticity challenge related to blocks, kids often find them inauthentic, this is also well-known in the academic literature, but more than that, it's not clear that it has to be only for kids. And it's also not true that the evidence doesn't necessarily support that a professional couldn't understand something, 'cause it draws a box around it. What does it matter?

So one of the things we've been working to invent is not just we solved accessible graphics, that doesn't mean anything, you have to rebuild everything with those operability requirements inside. So a couple things that we changed, blocks all have to have keyboard controls. No mouse. You can use the mouse, but not just the mouse. You gotta have copy-paste, and find-replace. Like these really basic things that like if you don't have them, as you learn, you will go nuts, right? Like if you have a block language and it has 3,000 lines of code, and you can't do find-replace, whew, that's brutal.

And then finally, you can't break version control. It can't be only imagery, right? If it's only pixels on the screen, not only does it break accessibility, it breaks every version control system in the universe. They don't work with pictures.

Thank you for your attention and for considering these important aspects of accessibility and usability in design. Your commitment to improving these areas is greatly appreciated.

And so if you want people that are not only and exclusively children, remember, accessibility is birth to death, all senses, all whatevers, then you need to be able to handle text. So we invented a technology, I guess I invented this part, that makes it so the text in the box are mathematically identical. It's kind of a funny thing. You don't wanna look at the equations, they're really gross.

So the last thing that I wanna mention is oftentimes, if you really want blocks to be birth to death, all people, all senses, all things, it needs to do all kinds of programming. So here's an example I use in one of my PhD level classes. This, it's hard to translate this into English, but this has to do if you have end dimensional math, shoving it down to less dimensions, that's as close as I can get to English, but it's seven lines of code. Yeah, really nasty math, but because we've thought through, "What words does it say?" We've thought through the visuals, we've actually tested it with people, we've tested people that can see or don't, or have other disabilities, all of a sudden, you can do end dimensional mathematics in seven lines of code. That's really not that bad.

All right, let's talk about the output. What about LEGO? Well, one of the things that bugged me years ago was there was a school out of Maryland, the Maryland School for the Blind, and they really wanted to do LEGO, and they really coordinated with them to try to get their stuff accessible, but for whatever reason was unable to convince them to work on it. So I was kind of bored on a plane, and I figured out what I think was a reasonable magic trick to do some conversions internally, and when you do that, all of a sudden, you can actually use block languages, and you get fully accessible LEGO robotics kits, which is now apparently being used at a bunch of schools, although I haven't followed up, so I don't know what they're actually doing. It's pretty fun though, and it's pretty good for a plane ride. My daughter gave me a look though. She was like, "What are you doing? We're going to Hawaii, dude."

Anyway. So we've also made some patches for Finch Robotics, and there's some discussions with the VEX folks. We're still talking, I'm talking to Raj a lot. Can you do mobile apps? Yep. Seven lines of code including one blank line. You got yourself a mobile app, accessible.

What about data science? So we made accessible charting libraries. Seriously, like you make yourself a chart, and it's totally visual. Doesn't matter, the way it generates, it works with every accessibility technology off the cuff. You don't have to do anything. You don't program anything. It's interesting, because the regulations around accessibility require that you actually have to specify a quote unquote "alternate description." You describe the chart. That is obsoleted with this technology to the Nth degree. That simply makes no sense. You don't generate descriptions of these charts, the computer just makes them accessible. You click the button that says go, and then you can't really break it, you can try to break it, but it's hard.

What about 3D? So this is actually a real tool, 3D game editor technology, that a person can use with or without sight, with or without whatever sense they have. The very first time I used this that we tested this with people that couldn't see, there was one user that was a Braille user, and he is like, "I just wanna make sure I understand this. I'm using an environment, a 3D gaming creator, and I just placed a cup on a table on a medieval floor." And I'm like, "Yep." He's like "Wild." 'Cause like that just isn't a thing.

Like, so here's a rough way of how you can see it, so you can perceive it, you can see it visually, of course. Like if you're a kid, you still wanna be able to see it if you can see, but if you can't see it, there's this little cursor on the side. It looks like kind of a square that is a little bit transparent, and when you're over something, you can feel it, you can hear it. It's operable, you can use keyboard, or mouse, and other things. You can understand it at least, hopefully, although I will tell you, trying to get people, especially the people that can't see, to understand 3D math is pretty rough. So like I feel like there's lots of research that could be done around that, but nonetheless, like the fact that you can do anything at all is good, and also I think it went pretty well, right?

And then finally, accessible graphics is the robustness, right? That's how it interacts with the starvation issues, and all these kind of funny things. So what can you use this for now? Well, pretty much anything high school, most college courses, there's a few maybe that might be still tough, where, but data structures, graphics, databases, networks, Spike and Finch robots, hopefully more in the future, a huge proportion of data science, but not everything, up through PhD level, and all the way through high school. Could a blind person be a visual effects artist? I don't know. But I think it's at least like a maybe now, like maybe not 100% chance, but like maybe. I don't know.

At the end of the day, accessibility isn't about one person, it's not about just one kid. It's about birth to death, all people and all senses. All, all things. A veteran that comes back with a missing limb, a person that doesn't see so well, an average kid that just can't see Apple's little thing, right?

All right. I'll show you a very brief demo, just so you can see that this is real, 'cause sometimes people think I'm not actually showing you, like what, like well, yeah, that sounds great.

If you wanna get in touch with me, you can, [email protected]. This is all freely available at quorumlanguage.com, it works today, a lot of the stuff we're in the process, or have put things online, so you can actually do these things accessibly in a browser too, even with 2D and 3D, stuff like that.

So let me show you just something very briefly.

Voiceover on Java, Quorum Studio 6.5.0 with --

All right, I'm going to show you briefly what a block ender looks like, I'm going to show you intentionally a complicated file. Of course, you can do a simple one too. One of the things that we thought about when we're thinking of birth to death is, well, what if we wanted to make things a little more complicated? And with traditional --

Plus radio button --

With traditional block line --

Java has not respond --

159% Zoom. One of the things that's interesting about blocks is that we've kind of gotten used to the fact that they're only for kids, right? That's kind of a thing. But they don't have to be, and interestingly, the one thing that's really cool about blocks is it gives you this idea of what do I code? Like I have a list right there, it is like an Uber cheat sheet. But we were thinking, if you actually want to do any arbitrary programming, what if you had 100,000 cheat sheets that you could use? So we wrote some software, so that it can actually predict which things you would want based upon what you're doing.

Class name is game. Keyboard.

So for example, if I'm clicked on a block, it might tell me I can add directional lights. If I click on --

Player undefined. Hint, at -- Player,

it'll read the line of code in the block by default, and I can navigate around. Player player value, edit tech. Et cetera. And it'll also tell me what the player thing can do. And it works for all possible cases.

It's mathematically equivalent, it's not an LLM guessing, it's provably mathematically correct. It cannot be wrong, unless there's a bug, which there might be, but I don't think so, 'cause it's built in, in a funny mathematical way. And if I run this little program --

Build successful in 0.6,

Java is not responding. Java is not responding. Application. Ground slide tactics. Window. Impassable cover, solid rock. Zero zero, has keyboard focus. I could have 3D games, and they're accessible. You can see it visually, you can perceive it, and I'll show you in a second how you can hear it, and if you use this in Braille, that'd be fun. It wouldn't matter. You can operate it, you can use the mouse, or the keyboard, you can understand it, I think, but that's maybe an empirical question, and that could probably be improved. This is just a game we made for fun. And then it's robust. So if I wanted to actually do this, I can move around. Landing pad, 42, sand, 45, four moves to reach. If you're building a game, of course it is bespoke, you have to decide what it means when you move around, but that's okay, the graphic system knows how to let you specify that. If I want to move around, I can press keys. One moves remaining. Player. Sand, sand, zero moves remaining, player, sand. I can take turns. Generate sand, generator one sand, three generator, one player sand. Five player, sand. Et cetera, et cetera. I won't play the whole game, but you can shoot monsters, you can attack little bad guys, and stuff like that, but the point is, at this point, I think we have it to the point, where you can do full on 2D, 3D graphics, maybe you could do CAD, maybe you could do all sorts of crazy stuff if you build it on top of an accessible graphics technology, right? And you think through the accessibility issues from birth to death, all people. And that's what you get.

Thanks for your time. Thank you. (audience applauding)

Voiceover off. Okay, are there any questions? Perfect. Thank you for your presentation. I have a couple questions for you. Number one, what is the back, what is the underground underlying language that allows this thing to work? Is it Java, is it Python? Is it C? And number two, I'm kind of curious about how complex the math is, because I have a math background, and I'm teaching statistics in my high school class right now.

Ah, okay. Great questions. So first of all, the language is custom, it is, the programming language is named Quorum. Quorum is a custom language, because we've done a ton of studies over the years trying to find evidence for or against particular design decisions, so the way it works is, well actually this has a name in the letter, it's called an evidence-based language, not object-oriented, not this, not that, it's a whole category, and the reason is because we run studies to prove ourselves wrong, like that's the purpose of them, and then anytime that we find that we're wrong, which is always, because there's so little data in the literature, what happens is the language changes. And because we work with schools, a lot of schools use Quorum for all sorts of stuff, because we work with schools, we do that on a cycle. So like once a year we're like, "Hey, we're allowed to break it for a while. We have some data. You can look at it, and decide for yourself." It's sort of like that, so under the hood, it actually is itself. Quorum is written in Quorum, which is counterintuitive, but then for it to connect everywhere, it connects to all sorts of crazy technologies under the hood to make that happen, something like eight to 10 programming languages across transpiring, and all sorts of funny stuff. So it's complicated, but the language itself is probably 99% of source code.

The second question was the math, that's actually an interesting question.

A number of schools around the country have been moving data science to high school, and the state that's the most advanced right now is Maryland, which does data science in high school. They have a whole curriculum around this actually built on Quorum, believe it or not, because it is accessible. To my knowledge, that's distributed across the state, definitely high school-friendly. You don't need end-dimensional math for most of it, most of the time, that's taught at the PhD level. So I would say, most of the math is the kind of stuff already described in Common Core, so like if you have Algebra One or Two, you're probably okay for most of that. Most of data science.

So ... I have more of a comment that ... So when you were talking about images, my students do still have to give and give alternate text, because we have to deal with some filters. Mm. It depends on where the image came from, if the filter is going to let it in, and so they still learn that, but they'll be happy to know that outside of school, they don't ever have to do alternate text again. Well, I should give a slight caveat. You can automatically generate a way to do accessibility for certain kinds of things, but not others. For example, if I had a picture, a picture, and it was either Keanu Reeves or the Queen of England, it couldn't tell you that. And what would be the structure that it would give you, like the nose or like the ... So there's definitely still a place for alternate description, it's just that it tends to be when it's world knowledge as opposed to structure.

So for example, a chart or a 3D game, we didn't specify the structure of the 3D game. The computer just knows how to do that, 'cause it has linear algebra and stuff. But if we like tossed up a, if the robot became Keanu Reeves, we would have to tell the system that that's, "Hey, that robot is now Keanu Reeves", or whatever. So it's not totally obsoleted, but we can get rid of a huge chunk of it, and that it's useful, because you can do 3D, and data science, and stuff like that. So ... Yeah, thank you.

Yeah, I think actually, just thinking about it, the way I've heard it described as being from others that have claim, the claimant is the most utility, is if you're doing something like data science, and you have a disability, and you can't use a normal chart, you can actually produce the charts yourself, and use them. That's kind of the key facility, in other words, you're not relying on a third party to label for you, well, anything you generate, you can use yourself. So this is really facilitating, in a lot of ways, which I think is interesting, so ...

One more question. I worked for a school that deals with dyslexia kids. How would this help them exactly? Ah, so dyslexia is an interesting condition. Oftentimes, with dyslexia, they'd use different kinds of technologies, the most common one is probably screen readers. So one of the things that's hard to understand about accessibility is that there's a lot of bleed between the conditions, not in terms of the condition itself, but in terms of the technologies that they use, so this server that I'm talking about, if you're dyslexic, and you're using technologies that read, you would get the same effect. So that's the kind of thing that is the most common for dyslexics, I would say, off the top of my head, not exclusively. Sometimes there is complexity with like word choices and letters, there's some evidence for and against fonts in the literature, but the evidence is kind of unclear, I would say, it's statistically in the wash, whether it is effective or not. So ... But in any case, screen reader's the most common, that would work just fine, so ...

Okay. Well, let's give Stefik a very warm round of applause, thank you so much for being here with us today.

(audience applauding)

(electronic music)

VEX Robotics Educators Conference.

(electronic music ending)

Thank you for joining us today. We hope you enjoyed the show and found it both entertaining and informative. Your support means the world to us, and we are grateful for your continued engagement.

If you have any feedback or suggestions for future episodes, please don't hesitate to reach out. We love hearing from our audience and are always looking for ways to improve.

Stay tuned for more exciting content coming your way soon. Until next time, take care and keep exploring the world of music with us!