AI is moving fast, but banking has to move with care. In this episode, Adam Blue talks with Q2 Chief Information Security Officer Beth Anne Bygum about the difference between AI that’s interesting and AI that’s ready for the real world.
Adam Blue
Hey everyone. Welcome to Cut the Context. I'm Adam Blue. We're here talking about AI and really just all the real things about AI here at Q2 and in banking generally. Joining us today, Beth Anne Bygum, Chief Information Security Officer here at Q2. Super excited to have her talk. And so, let's just dive right in. So Beth Anne, when you think about the intersection of AI and banking, especially digital banking, what's different about the way AI works in our industry?
Beth Anne Bygum
Hey, it's great to be here, Adam, so thanks so much for having a time to connect today. I think what's interesting about any advanced technology when we are leveraging it in the banking sector is the fact that we have accountabilities. And our accountability is to constantly be able to demonstrate the confidentiality, the integrity, and the availability of either our data and our systems. And as practitioners and leaders in that space, our objective is to take the new technology and then translate, "Well, how can I accomplish that? How do I maintain it? And how do I demonstrate it back to the regulators at the same time advancing and elevating innovation?" So, it's a careful balance between the two. And the opportunity for us is to constantly figure out, "Well, how do I enable, but at the same time, demonstrate that transparency, demonstrate that CIA triad, as we call it, in the security profession?"
Adam Blue
Sure. I think that's a fantastic way to think about it. The software industry generally, things like repeatability and reliability are really important. But if I think about Amazon, or Google, or Meta, or even something like a line of business workflow application, people expect a lot of reproducibility and traceability in those applications. When it comes to moving money around in people's bank accounts or determining whether they can or can't make a payment, I think there's a lot of really extraordinary demand for not just doing the right thing at the right time, but knowing why you did it and being able to demonstrate after the fact why you did it. I think this is something that's really different about our business. And I think it underscores the difference between maybe what's interesting to work on with AI, and then what's responsible to work on with AI. And so maybe expand on that a little bit and talk about how you and your team think about where ... And it's probably a wide line, but where's the line between interesting and exciting, and responsible and kind of manageable in the AI space right now?
Beth Anne Bygum
Yeah, for sure. I mean, and I think the opening question is a foundational question, because the ability to maintain transparency, evidence, the trust, and evidence what I would consider being able to maintain fit for use. Am I able to maintain and demonstrate to my organization, to my customers, and to the regulators that we have an understanding of how the technology is executing? How did it manage the data? How is it protecting the data? Is something that we have to maintain at all times. Now, the interesting thing is as we continue to watch the technology evolve, security will be an enabler of a frictionless ecosystem. The question is, is how do we enable the security so that it's friction appropriate at any given time? Identity is the new endpoint, and data is the commodity. So the question becomes, how do I balance those two new intersections so that I'm demonstrating confidentiality, integrity as the identities are passing from one system to the other? And in our industry, that's an expectation. So, that's the difference between something that's interesting, versus something that's operationally ready, because we have to be audit ready at all times.
Adam Blue
That's great. I like what you said there about identity becoming the new endpoint, because more and more ... And I feel this when I work with AI tools that we've built internally, I find myself lending my identity to a robot with some surprising frequency. And at first it's like, "I'm not sure how I feel about this." And then a week later when it turns out to be useful, I'm like, "Yeah, yeah, just be me, whatever. Go get all the data out of Salesforce." So, in order for these tools to really have reach and range and power, we have to let them be us in a sense, or at least have access to data that's equivalent to us. And I think that's powerful and terrifying at the same time. And so, where in the conversation about what we're trying to do with AI, would you like to see security get engaged? How do you think about that?
Beth Anne Bygum
Yeah. Hygiene is still a fundamental concept in technology holistically. And the earlier security can embed or be embedded in the process, the less expensive the implementation is, the less expensive the guardrails are. You just right size it. And it goes back to that whole concept of friction appropriate. The earlier we can understand the design, the outcome, the intent, the data flow, the earlier we can right size the controls that will wrap around it. I think it's an interesting thing around identities. We're doing some modeling here to be able to ascertain the difference between was that a human that executed the command versus the identity, or the agent that's executing on behalf of the human. And this is a new paradigm. When I was in undergrad, I used to have a professor that would say to me, "You get once in life to prepare and then you're on." And I think holistically we're in that window. It's time to practice, it's time to design. And companies that are investing in that early will be ready for the next chapter.
Adam Blue
Yeah. So, there's a lot about AI technology that's fascinating that stems from its non-determinism. I think you can look at hallucinations in AI and say, "That's a defect. It's a liability." It's also possible to look at them the way you would look at analog film, which can turn out very differently than digital photography. And sometimes the accident is part of the value, maybe not so much in a security context around say, I don't know, perimeter defense, but I think there are a lot of examples where maybe sometimes the unexpected outcome is part of the value of using an AI tool. And so, what have you guys done or worked on in terms of bringing human oversight to this non-deterministic kind of approach to the tech?
Beth Anne Bygum
Well, it's a very interesting question. I mean, the first thing that we have had to do is actually adjust how we think as humans, security professionals, and how we think as defenders. First, we have to understand is the technology will execute what it's designed to execute. So, if this is a capability, if it's an agent that's designed to ensure 100%, 99% uptime availability, it means it will execute that mission. And so, as defenders and security professionals, we need to be able to interpret the behavior of the technology in light of what it's attempting to do, because it was designed to do it versus are we trying to intercept something from a misinterpretation of what the technology is executing. So, from a human side, because we apply emotion to everything, potentially at the very beginning as humans interact with the technology, we're misinterpreting the cues. So, we're spending time actually rebuilding our telemetry, because what we intercept and what triggers the potential alert is a totallytal different set of telemetry and of signals. And how we as operators interpret that is super important to not misinterpret those cues.
Adam Blue
Yeah, it's funny, because I think probably since somebody took a rounded off rock and rolled it down a hill, as human beings we have been attributing human characteristics to our tools. I hear people say things like, "My car didn't want to start this morning." And I get it, but linguistically, your car doesn't want anything. Your car is an assemblage of components that creates a mechanical construct. And now we have a technology that at least it has the appearance,arance and it has the behavior of wanting something. It has an objective. And so, I find AI really useful for evaluating the quality of my ideas, because as far as it's concerned, I'm either the most brilliant genius that ever walked the earth, not likely, or a complete imbecile, more likely, but still probably not terribly likely. And I think the tech struggles with nuance in a meaningful way.
And I think you see this come up in AI-generated code, you see it come up with in AI-generated video. The video examples I think are a little more amusing than they are, say in a production software context. But in this world where we can now start to think about like an agent, and I know they don't want things because they're not embodied or insole kind of entities, but an agent is given a set of perspectives and a set of objectives to accomplish. And some of the things that we count on when we manage, I think that's probably the right word, human beings and manage agents could be really different. So, when we think about non-determinism, responsible data use, really securing the keys to the business, what does it mean to talk about the intent of what is effectively a neural network, or an algorithm, or a generative AI model?
It's hard enough to understand people sometimes. I feel like the robots maybe are even more confusing. And so, how does that enter into your thinking around the security perimeter or attack surface, or just kind of basic adherence to controls, that notion that an agent might want something that is really adverse to what we're trying to do in the business?
Beth Anne Bygum
I think that's a fantastic question, because it really lays the landscape and the foundation for how humans in particular roles will interact with this technology as it continues to advance. And I think foundationally it starts with, number one, understanding the purpose and the mission of the technology. If I am interpreting the signals that we receive at our security operations center, whether it's through our prompt monitoring capability or through our software that's interpreting the execution of maybe vulnerabilities that are starting to operate in a different way, the first thing that we have to understand is that, is this an inbound compromise, or is this a capability that's trying to execute as it's been configured to execute?
And this takes us back to even a more interesting conversation is just how we are evolving our secure software development practices as we onboard technologies. Because the design of capabilities using AI is going to have a totally different concept than our traditional static approach to developing. But here's the interesting thing, architecture still has to have its rightful spot. And so, as we adopt technologies, we have to remember to look at the architecture, let it lead us, understand that data flow, understand how the data flow diagram will intersect with the technology and with the architecture, and then make sure that as we're interpreting the signals, the telemetry or any of drift poisoning, that gives us a lot more insight as to whether it's a friendly or a foe, because we have to understand it in light of the architecture.
Adam Blue
Yeah, yeah. I think that's a good framing is to return to the architectural view of the way we're trying to solve the problem independently of the agent or how we've driven agentic kind of solution. Yeah, and that brings me to an interesting ... You probably saw it in the news. There was a framework that a lot of people were using to do orchestration and AI tooling. And for about an hour, the framework was compromised. And so someone had injected basically just a credential harvester, right? But it underscored a problem that is not a new problem that we have not yet solved, and is probably actually getting worse in a post-AI world, which is the software supply chain security problem, right?
Beth Anne Bygum
Yeah.
Adam Blue
And so today, as an engineer, I may go looking for, okay, I need a package that does a thing and that package is dependent on a package and that's dependent on another package. I got to be honest, if I download a new version of LXML for XML parsing, I'm not sure I'm going to go validate every single dependency. There's some tooling we have that helps with that. It's part of the SDLC, but ultimately as we let agents do more coding and I think they have zero concern, like there's just kind of an inherent naivete to the way they approach the task. It seems likely that software supply chain could become even more critical than it is today. And so, where do you see that going in the next 12 to 24 months?
Beth Anne Bygum
Let's unpack that. When we look at the most recent executive order that was issued from the administration out of Washington, you could see exactly where it's going. This is the first time in, I think, 15 years where we see that administration or any administration in Washington saying that there's a certain level of increased accountability that they are now placing at the door of the technology firms.
What does that mean from a translation perspective to your point? It means that we have to align on the definition of secure code. It cannot be subject to different interpretation per company, per language. It's just we're past that at this point in time. And so, I think what we can expect to see is more rigor when we have connected partnerships across the supply chain. And for organizations that have a stronger role in maintaining nation level or federal level technology capabilities, you will see a higher level of expectation played out for those companies.
So if I take that concept and I bring it back down, let's bring it back home, what that means is just leveling up or translating what does it mean to be secure by design, I think is maybe 90s. We're now at the place where it's defense by design and we are designing and testing from the perspective of cyber survivability, which includes what does it mean as I ingest open source packages or packages that are designed by agents? How am I testing it? How do I maintain resilience? And then lastly, what that means is that we're moving from traditional periodic scanning to just persistent, right? It's a persistent scan.
Adam Blue
Yeah, that's interesting, because when in other discussions we've had on the podcast, we've talked about the importance of measurement and validation that the agent is doing what you needed to do, or that the solution delivers what you want it to deliver as really even becoming a higher priority than even enabling the agent to do what it does. And so that notion of kind of continuous scanning or continuous validation, I think is getting much, much more popular. And that may be ... It's interesting. That may be the place where human beings have a role that is just kind of irreplaceable by technology.
There's a lot of discussion now about, if you can generate code very rapidly, then application design, workflow design, product management understanding, subject matter expertise, deep domain expertise, these are the things that now become scarce and are super valuable. And so if I map that onto the set of security compliance and regulatory concerns, it may be less a matter of, "I'm really good at building systems that are hard to compromise." And maybe what it turns into is, "I'm really good at measuring whether or not the systems are being compromised." Maybe that's more valuable than this kind of design thing.
Beth Anne Bygum
It's going to be about the hunt, right? My ability to baseline standard operating practices within a product will be managed by the technology. It's going to be the human's role to be able to start to watch and monitor from a hunt perspective, because where we are right now is the gloves are off, right? We're able to weaponize informational vulnerabilities, right? Medium, low. So it's not about being able to cast a broad net in this. It's all about watching for the launch.
Adam Blue
Yeah. Absolutely. Yeah, it's interesting. I love film and movies, and I spend a lot of time watching film and movies about film and movies. And I'm always fascinated to see what doesn't make it into a film, like what do they leave on the cutting room floor? What gets edited out? Which takes don't make it in. And it can make a huge difference in the final product. And so, I have this vision where, as it becomes possible to create more software that does interesting things, maybe the process of securing things a little bit becomes a matter of taste and curation, and we may produce a lot of features or a lot of capabilities. And then restraint for us is around, I have 10 features, six of them, I don't really like the implications of how those features work, so I don't ship them even though I had them available.
And I think that's kind of the exact opposite of the way we work today where it's expensive to build software and features. So we got to ship everything we can build, otherwise the pipeline goes stale, right? So, I could see for leaders, us producing more software than we ship and being very thoughtful about what we ship, instead of expecting to ship every single thing we build, because it's expensive to build it. And so that kind of switch from scarcity to abundance in the manufacturing process, how does that change the way you think about security on your team? And then what new questions does that lead to for leaders in the banking and technology space that they should be asking? Because it feels like a lot of the underlying assumptions have just been inverted with the arrival of this technology.
Beth Anne Bygum
I like how we've gone full cycle or full circle of our conversation because at the end of the day, what I hear you describe is making that decision, the tradeoff between confidentiality and availability, the tradeoff between integrity and confidentiality and availability. We won't have to do that discussion or host that discussion anymore. The technology is nimble and agile enough to balance the perpetual balance between the triad. And so, the opportunity for us, again, as humans, as we continue to interact with the technology is being able to actually think outside of the box.
We are entering a point in time where the storyteller is very important. I was reading an article the other day that one of the top commodity, like top jobs right now in the ad tech martech industry is being able to tell a story, which I find interesting because I sort of feel like that's always been there.
But when we translate that into the banking sector is to be able to translate the needs of our customers and then the bank's customers into a story. And then the technology, because it's so agile and fluid, it will be able to balance the expectations of confidentiality and availability or integrity through the appropriate guardrails. And then our role, to just close out the question, is adjusting. As a security profession, our role will be part of just being able to defend and intercept, which in time will also be automated. But it's a great time now to be part of the design as human beings as we enter into this next chapter.
Adam Blue
Interesting. Yeah. I think that's a great way to look at it. All right. Well, thanks very much-
Beth Anne Bygum
Thank you.
Adam Blue
... for being on the podcast today, Beth Anne. It was really great and your insights were great. One of the things we do here on Cut the Context is I like to reference a piece of art or media for folks to enjoy. And so today's offering is going to be an interesting one. So if you like a noir film, for instance, Ryan Johnson, who directed a Star Wars movie and a couple of those who done it, I think it's like something “Onion.” Anyway, there's a fantastic movie he did early in his career with Joseph Gordon-Levitt, who's just amazing, called “Brick.” And it is like an old school, like Philip Marlow detective thing set in a high school, and it's very stylized because all the kids talk like they're in a 1940s kind of gangster noir film, but of course it's set in the modern day.
And it's like this fascinating intersection of a series of old ideas and tropes kind of filtered through this modern sensibility. And there's something about it that reminds me of this crazy paradox we have today where everyone's capabilities got lifted simultaneously with the arrival of AI. And so, in some sense, in a reductive way it's kind of a race between the good guys and the bad guys to figure out who can get good fast. So “Brick,” directed by Rian Johnson, it's quite a film. It's very interesting. You'll either love it or hate it, but that's our linkage for today to the world of art in our discussion about AI. So, thanks again for being on.
Beth Anne Bygum
Thank you.
Adam Blue
I think this was really fantastic, and look for Cut the Context wherever you would find your top quality podcasts in the future. Thanks everybody.