AWS in Orbit: Generative AI and Space Resiliency.

>> Maria Varmazis: Hi. I'm Maria Varmazis, host of the T-Minus Space Daily podcast. And this is AWS in Orbit, Generative AI and Resiliency. We're bringing you the second installment of the AWS in Orbit podcast series at the 39th Space Symposium. And, in this episode, I'll be speaking to representatives from Rescale and AWS Aerospace and Satellite about improving space resilience and scaling customer success using generative AI.

>> Derek McCoy: My name is Derek McCoy. I lead our Enterprise Public Sector and Channel businesses here at Rescale. I've been with the company for about six years. And I've also recently been helping lead our Tiger Team around AI and generative AI specifically and how we're helping a number of companies out there in the industries to enhance the way that they're doing their physics simulations and get to market and do rapid prototyping a lot faster.

>> Maria Varmazis: Very cool. Thank you, Derek. And, Kathy, over to you for your intro.

>> Kathy O'Donnell: Hi. Yeah, I'm Kathy O'Donnell. I lead the Space Specialist Solutions Architecture Team in Aerospace and Satellite. I also in the past year started leading our Generative AI and Space Initiative at AWS.

>> Maria Varmazis: Fantastic. Thank you both for joining me today, and welcome. So glad to be speaking with you. So, Derek, let's start with you. Tell me a bit about Rescale.

>> Derek McCoy: Yeah. Absolutely. So Rescale is a company that has been around for about 12 years. And where we fit in the space is that we've been supporting companies in their HPC orchestration with partners such as AWS. So the different industries that we support today are across aerospace and defense, space exploration, manufacturing, automotive, life science, and others. And the users that we're supporting are the engineers that are doing modeling simulation, the research and development folks, as well as the scientific researchers out there. Specifically to some of the use cases that we are looking to do for our customers is we're giving them the ability to explore wider design spaces and take the physical testing aspect down and be able to do more prototyping in a digital world in order to get to market a lot faster and to be able to deliver on deadlines in a secure fashion.

>> Maria Varmazis: You mentioned security. And I wanted to ask about that because I imagine, given what Rescale does, you have a lot of government customers. So can you tell me about what your government customers are looking for in security?

>> Derek McCoy: Yeah. Absolutely. So this is something that's really evolved over the last few years. We've seen a major increase in the way that the government and their partners are adopting cloud. And with that has come a lot of regulation and constraints around the security of their data, of the way that they're doing compute. And so there's a number of different accreditations out there that customers of ours, as well as ourselves, have been achieving along the way with great partners like AWS. Those include but are not limited to FedRAMP, ITAR, IL5, IL6, and beyond. This ultimately is an impact level of data security, as well as the ability to make sure that customers are able to scale out as they look at these mission-ready type of initiatives that they're winning and delivering for the government. So it's a area that most customers are struggling with in a number of areas because there's a lot of nuances.

>> Maria Varmazis: Yeah.

>> Derek McCoy: And, on top of that, there's also a lot of different components in the technology stack that need to be taken into consideration. There's data layers. There's compute layers. There are third parties that you're using for metadata and logging and so forth. We're fortunate to have a tightly lined relationship with AWS where we take advantage of their full catalog and expertise in order to make sure we can create a full turnkey solution for the market.

>> Maria Varmazis: Fantastic, fantastic. Kathy, did you want to add anything to that or?

>> Kathy O'Donnell: Oh, no. I'm just really excited to have partners that care so much about security because, you know, a lot of our customers, that is so key. And we don't want people to think that the cloud is less secure when, in fact, you know, that is one of our primary jobs at AWS is ensuring that security. So it's just really great to hear our partners also carrying that, like, tenet with them.

>> Maria Varmazis: Fantastic. So we are here to talk about AI and generative AI these days. So can you tell me a bit more about how generative AI and AI factor into your customers missions and their security concerns?

>> Derek McCoy: Yeah. Absolutely. So there's a number of ways that we could take this. And I think the first thing I would say is that, in the world that Kathy and I, you know, are working within, a lot of the simulations that people are doing are around physics. And physics introduces a different component to generative AI where we've heard a lot in the market about large language models.

>> Maria Varmazis: Yes.

>> Derek McCoy: Where I think that we typically see more is around large physics models. And what comes with that is still the same requirements around compute and infrastructure and orchestration, but it also brings in a lot of nuances around what is stability around the physics? How do we actually look at these problems? How do we orchestrate a foundation to make sure that we can build upon the data over a number of historical years, future years, bringing in real data, as well as synthetic data to ultimately get a verification or validation that makes us comfortable with running against our traditional solvers. So where our customers are leaning in a lot is looking at how do we create these neural networks and so forth to get a lot further in our simulations a lot faster? How do we take the traditional workloads that are scaling up to hundreds of thousands of cores and taking multiple days to run down to just hours or minutes?

>> Maria Varmazis: Yeah.

>> Derek McCoy: I don't know if you have any thoughts, Kathy.

>> Kathy O'Donnell: Yeah, yeah. One of the things that I love about working with the cloud is that ability to scale when you need to scale.

>> Maria Varmazis: Yeah.

>> Kathy O'Donnell: So, traditionally, like, back in the day, back when I was a youngster, you know, we had to provision all of that, and it was very expensive. It took a long time to do. And then, when you were done with it, what would you do with all that compute?

>> Maria Varmazis: Yeah.

>> Kathy O'Donnell: So I -- one of the things that I love about AWS and just the cloud structure in general is that ability to scale up quickly and to bring it down when you've completed your work.

>> Derek McCoy: Yeah. And I would also add to that. You know, one of the great things about the cloud is having the ability to have a fragmented architecture catalog. So, you know, obviously, there's new GPUs and CPUs coming out all the time.

>> Maria Varmazis: All the time. Yep.

>> Derek McCoy: And within the physics and modeling simulation world, there are a lot of dependencies based on the workflow you're doing with the different infrastructure that's optimized for it.

>> Maria Varmazis: Yeah.

>> Derek McCoy: And having the accessibility in a -- in a product like AWS to be able to span across all of those. So you have the heterogeneity to be able to go out there and do small test cases. But then you have a homogeneous cluster to scale when needed. Once you get to that validation stage and you say, Okay. I'm starting to get more comfortable with this. Now I want to run something at scale.

>> Maria Varmazis: Yeah.

>> Derek McCoy: You have the flexibility to really put those things together.

>> Maria Varmazis: Yeah. Can you tell -- tell me a bit more about, like, that customer experience, doing what you just mentioned. I'd love to hear more about that.

>> Derek McCoy: Yeah. Absolutely. So the way that we've approached the problem is that we truly want to mimic the way that AWS builds their business model where they do have flexibility for on-demand infrastructure, as well as other components of the technology stack. And where we really leverage our platform is being able to allow customers to go in there and choose what they want immediately. So the traditional method that customers go through is, when they go into a partnership with us and AWS is we set them up in, you know, two to three weeks time where traditionally that can take months on end from supply chain issues, if they are looking to -- to develop their own infrastructure and set that up, do the different OS layers and so forth. Well, we have that all preconfigured and installed. And we work with our AWS counterparts to make sure that we optimize based on all of the different software vendors available out there with different solvers. We also support a number of government codes, as we've talked about --

>> Maria Varmazis: Yep.

>> Derek McCoy: -- such as NASA FUN3D, Cart3D, the DoD create codes. And what we do is we actually use AI within our own platform to recommend based on their job attributes what they should be using for AWS infrastructure in their workloads.

>> Maria Varmazis: I imagine that also -- I'm sorry, Kathy. I was going to say I imagine it scales well. But go ahead, Kathy. Go ahead.

>> Kathy O'Donnell: I was going to say that's super cool because, I mean, it can be really difficult. We have a lot of different options at AWS because we serve a lot of different industries and customers. And sometimes it can be very difficult to know what do you want to use? Like, which one of these things is going to work best for your workload? And so having a partner to help you and especially using ML to help with that decision-making, I just think that's awesome.

>> Maria Varmazis: I just think that's neat. Yeah. That's a very valid comment, honestly. Yeah. And I was thinking, as you're describing it, that must scale really well for the customer as a repeatable process.

>> Derek McCoy: Absolutely. It does. And we try to build in functionality together as we find customer requirements to make that very repeatable. So scale is one thing on the infrastructure side. But another area within this domain is the ability to walk in and have those different inputs you have but have the job orchestration there for you. So building out templates and, you know, computational workflows and so forth. And those can become dynamic where there's multiple different infrastructures, multiple dependencies where they need results from the previous step of the job. And so we look as a partnership to say what is the next coming thing as people look to kind of elevate the way that they approach, you know, the government, you know, requirements and things that are -- are down the line for us.

>> Maria Varmazis: Fantastic. So we're talking about reducing those complexities, scaling results. So let's talk about results a little bit. So can you talk a little bit about how introducing AI into customer workflows has created great customer success.

>> Derek McCoy: Yeah. Absolutely. So we've had a few different ways that customers have gone about using AI.

>> Maria Varmazis: Sure.

>> Derek McCoy: Primarily, you know, the ultimate goal is that we get to a repeatable process where generative AI is the go to. And what that looks like in the world that we're living in is being able to put meshes and geometries into a generative AI system and be able to get an accurate result out; where we understand, you know, whether it be the drag coefficient on something as it leaves orbit because, you know, the heat and so forth has a different exchange there.

>> Maria Varmazis: Yep.

>> Derek McCoy: We also see customers that are building neural nets quite frequently on more regular workloads like computation fluid dynamics, and where -- what they do there is they take all of the data that they're running; they build a synthetic data if they need it. And we try to unify that data so, that way, you can build a neural net around it and apply that neural net to run inference against the job. And, with that, you're usually able to see 1000 to 10,000x speed-up time with somewhere between 95 and 98% accuracy right now.

>> Maria Varmazis: Wow. Yeah.

>> Derek McCoy: And then, more traditionally, you know, we have customers that are looking for ml optimization. We have benchmark space systems that we've worked together with where their typical workload where they have 20 studies and they're running on a number of different CPUs, we want to reduce their time from 8 to 10 hours per study on dozens if not hundreds if not thousands of CPUs to accelerate that through GPUs or be able to accelerate that from, you know, running inference against some of these neural nets. And so we look for opportunities like that where we can reduce their time down 85% or more.

>> Maria Varmazis: That's huge. It's huge. Yeah.

>> Derek McCoy: It gives engineers the results they're looking for --

>> Maria Varmazis: Yeah.

>> Derek McCoy: -- to be able to actually make the right decisions on the next evolution of their project.

>> Maria Varmazis: Make -- do those tests and forget that data. That's so important. And that -- that time to results, it's massive. Yeah. Go ahead, Kathy.

>> Kathy O'Donnell: Yeah. I think it's really interesting when people using AWS, it -- when it clicks for them that, by using GPUs, which, to be honest, are a little more expensive per hour than a CPU. But it runs so much faster.

>> Maria Varmazis: Yeah.

>> Kathy O'Donnell: You not only have a time saving. You can also realize a cost saving as well.

>> Maria Varmazis: Yeah.

>> Kathy O'Donnell: And that -- it's really neat when someone's like, Oh. Hey.

>> Maria Varmazis: Light bulb moment.

>> Kathy O'Donnell: Yep.

>> Derek McCoy: Yeah. And I think that, you know, I'd be interested on your opinion too. There's always this debate when it comes to high-performance computing and AI, between TCO and ROI. And I do think that is like an interesting thing that our -- that AI, generative AI specifically, it's kind of flipping the tables on this where I think that, you know, we're going to get to a point where it's worth the investment for the results you're getting because not only are you getting the faster time but you're looking at a wider design space. You're going to create better products.

>> Maria Varmazis: Yeah.

>> Kathy O'Donnell: I mean, sometimes it is difficult to measure because I see generative AI as being an augmentative technology.

>> Maria Varmazis: Yes.

>> Kathy O'Donnell: It helps you do your job faster.

>> Maria Varmazis: Yeah.

>> Kathy O'Donnell: But how do you measure that.

>> Derek McCoy: Right.

>> Kathy O'Donnell: I mean, you know, you have measures of FTE hours. But when you do knowledge work, when you do innovative technology, like, you're still working your entire week. You're just doing much more cool stuff.

>> Maria Varmazis: Yeah.

>> Kathy O'Donnell: And so, yeah. It is a big question. How do we measure the actual, like, augmentation and how much better you're doing.

>> Derek McCoy: Right.

>> Maria Varmazis: Sort of like a gut feeling at a certain point. I mean, honestly.

>> Derek McCoy: Yeah.

>> Maria Varmazis: Yeah. I could absolutely see that. Now, Kathy, I wanted to ask you about customers validating AI results to -- and ensuring customer trust. Can you talk a little bit about that?

>> Kathy O'Donnell: Yeah. So we don't have, like, one singular way to evaluate results coming out of generative AI because, if you've ever used like a chatbot, you know that it changes up its answers. You can do a lot with how you set the parameters for that large language model. But we do have customers doing some really interesting things around testing, like coming up with a, you know, 2- or 300 long list of questions with the answers that they want and then using another large language model to see if the one that they're using, the answer, like, matches the answer they expected. You know, and when you're measuring large language models, there is a suite of tests that we use to compare different models in performance.

>> Maria Varmazis: Makes sense. Yep.

>> Kathy O'Donnell: Yeah. So we were really excited because we recently introduced Claude 3 onto the Bedrock platform. So that's from Anthropic, and it is right now top of the leaderboard, like, Claude 3 Opus, in those set of tests so.

>> Maria Varmazis: Pretty cool stuff, honestly. So, Kathy, I'm going to ask you a follow-up question unless, Derek, you have something you want to add to that?

>> Derek McCoy: No. I think it's creative and out of the box. I love hearing these use cases.

>> Maria Varmazis: I was going to ask about use cases. So AWS customers using generative AI, other use cases, anything you want to mention there, Kathy? Yep.

>> Kathy O'Donnell: Well, so what we see a lot of -- and I tend to split up the use cases into a few different buckets. There are the use cases that you have just by virtue of being a company.

>> Maria Varmazis: Yeah.

>> Kathy O'Donnell: So, you know, interacting with your customers, like, having a chatbot to do that; having, you know, interactive websites powered by a foundation model. Then you have your business processes. How can you speed up, like, search across all of your internal documentation? So we see a lot of customers doing that.

>> Maria Varmazis: I so see the value in that. Oh, my goodness. Open there.

>> Kathy O'Donnell: Well, especially, you've got a company stretching back 20, 30, 40 years? And, you know, they've done their thing. They've digitized all of their records.

>> Maria Varmazis: They've actually done documentation. Oh, my goodness.

>> Kathy O'Donnell: But now it's like, well, how do I -- how do I index it? How do I ask questions? And so you have to look through a huge index of -- or tables of content?

>> Maria Varmazis: Yeah.

>> Kathy O'Donnell: But, instead, what you can do is you load it up into a database that you can then use retrieval augmented generation and a foundation model together.

>> Maria Varmazis: Fantastic. Yeah.

>> Kathy O'Donnell: And so you just asked human natural language questions, and it will pull together all the different pieces over those two, three, four decades of work and give you an answer, which is so cool.

>> Maria Varmazis: Instead of asking your most senior person who's probably super busy with that institutional knowledge.

>> Kathy O'Donnell: Oh, yeah. We call him Dave.

>> Maria Varmazis: So bugging Dave about it. Well, actually, David, yeah.

>> Derek McCoy: And, like, there's just a huge value out there for workforce development and upskilling.

>> Maria Varmazis: Yeah.

>> Derek McCoy: You know, I think all of us are employees of the company, and we all desire and, you know, yearn for that. And I think, like, you know, the ability to have that at your fingertips is in and of itself a huge value --

>> Maria Varmazis: Sure.

>> Derek McCoy: -- for the companies to retain employees.

>> Maria Varmazis: Yeah. Absolutely. There's nothing more frustrating. You're looking for those answers. There's that one person who knows. And they're busy, or they're gone. I mean, it's just -- yeah. I can -- huge value there.

>> Kathy O'Donnell: Yeah. And one of the big things we have coming out now is Amazon Q. So we have Amazon Q Business and Amazon Q Developer. And I've got to tell you. So there's a lot of essays here from AWS. You should go talk to them. In the past week, we've all really started using Amazon Q developer. And it is -- it changes the way that you program because it's not just code completion. You can actually, like, highlight sections of your code and say, Okay. What exactly does this function do?

>> Maria Varmazis: Oh. Yeah.

>> Kathy O'Donnell: You can say, I have a bug here. I'm not getting the right result. Can you see what might be wrong?

>> Maria Varmazis: That saves so much time.

>> Kathy O'Donnell: It is.

>> Maria Varmazis: I couldn't even begin. Days, days. Yeah.

>> Kathy O'Donnell: I've had at least three people so far told me they've stopped using Google when they're coding and have just started using Amazon Q so.

>> Maria Varmazis: Stack overflows hits are going to start dropping.

>> Kathy O'Donnell: Yeah. I was telling someone, you know, back in the day, which is like two weeks ago, you -- you know, if you had a problem, you had to Google.

>> Maria Varmazis: Yep.

>> Kathy O'Donnell: And then you'd get all the stack -- stack overflow hits. And, of course, it would be a guy with your same exact question.

>> Maria Varmazis: And who wouldn't answer the question.

>> Kathy O'Donnell: Right. From 14 years before, the top answer being solved.

>> Maria Varmazis: Yeah. And no -- no result. Yeah. You have no idea. What's the answer?

>> Kathy O'Donnell: Like, okay. What happened?

>> Maria Varmazis: Yeah, yeah. I've been there. I've personally -- I'm feeling that frustration. I remember.

All right. So we're coming up on time. Before we go down that whole rabbit hole, I want to make sure we talk about -- wrap up lessons learned. So, Derek, why don't we start with you.

>> Derek McCoy: Yeah. So, you know, I think -- I think, overall, what I've been seeing in the market is building a foundational platform in order to make sure that you can scale in the long-term and that you can go down the exploration stage but also see the immediate results is really important. And so I think, you know, making sure that you're a good steward of your own technology stack and your partner ecosystem is really important here. Obviously, we want to get to the point where, you know, large physics models and generative AI is an out-of-the-box solution for a lot of these companies doing physics work. So I do think, you know, my thoughts on it, at least, from a personal standpoint is that people need to do the fundamentals first before they move up the stack but also open your mind to be able to explore these type of initiatives. Make the investments where you see fit. Work with your partners and advisors and so forth in the industry, and make sure you're understanding what that ties back to from your day-to-day business. So, you know, I'll pass it to you, Kathy.

>> Maria Varmazis: Great points, Derek. Thank you. That was great.

>> Kathy O'Donnell: Yeah. I think what I'd like to end with is innovation is key. We should always be innovating. But you want to make sure that you can do that in a secure manner, in a safe manner because I know -- as, you know, a programmer, as a technologist I want to do the crazy stuff. But I don't want to take down the company. I don't want to ruin our code base.

>> Maria Varmazis: Of course.

>> Kathy O'Donnell: So making sure that we can do that securely. And one of the things that I really like about partners like Rescale and about the AWS platform is that really is one of our key init -- or goals is making sure that we're secure.

>> Maria Varmazis: Wonderful. Derek and Kathy, it was a joy speaking with you both today. Thank you so much for joining me.

>> Derek McCoy: Yeah. Thank you.

>> Kathy O'Donnell: Oh, thank you.

>> Maria Varmazis: This episode was produced by Alice Carruth and Laura Barber for AWS Aerospace and Satellite, mixing by Elliott Peltzman and Tré Hester with original music and sound design by Elliott Peltzman. Our associate producer is Liz Stokes. Our executive producer is Jen Eiben. Our VP is Brandon Karpf. And I'm Maria Varmazis. See you next time.