Why Q2 Pursued Independent Certification for AI Agent Security

Listen

Subscribe

Related Links

Q2's Responsible AI Pledge

[Webpage] AI for Everyone

[Webpage] AIUC

[Webpage] Grant Thornton

Transcript

Cheryl Brown

Hello, and welcome to The Purposeful Banker, the podcast brought to you by Q2, where we discuss the big topics on the minds of today's best bankers. I'm Cheryl Brown. Welcome to the show.

Banks and credit unions are making big decisions around AI right now, and one of the hardest questions they face is how to know whether an AI system operates the way its provider says it does. Today, we're talking about one answer to that question. Q2 Assistant, the unified AI experience layer embedded across Q2's banking platforms, recently achieved AIUC-1 certification, an independent standard for the security, safety, and reliability of AI agents. That makes Q2 the first digital banking provider to meet that standard.

So I have three guests with me today to talk about what that certification is, what it took, and why it matters. Blair Williams is chief risk officer at Q2. Nicole Julius is national managing partner of Enterprise Assurance Services at Grant Thornton, which performed the independent controls review. And Emil Lassen is one of the founders of AIUC, the organization behind the AIUC-1 standard.

So Blair, Nicole, and Emil, welcome to the show.

Blair Williams

Cheryl, thanks for having me.

Emil Lassen

Yeah, great to be here.

Nicole Julius

Really looking forward to the conversation today.

Cheryl Brown

Great. So let's just dig right in. Blair, I wanted to start with you to help us understand why Q2 pursued the certification in the first place. Q2 already goes through plenty of audits and attestations as a digital banking provider in a very regulated industry. So why did Q2 Assistant need its own independent certification?

Blair Williams

That's a great question, Cheryl. So let me step back a little bit away from AIUC for a moment and talk about what we have done in the past. Traditionally as a digital banking provider, we've provided our financial institutions with SOC 1 and SOC 2 reports. SOC 1 principally focused on financial integrity of transactions and the SOC 2 reports focused on the trust services criteria that are outlined by the AICPA. And those criteria are security, confidentiality, availability, and processing integrity.

Well, those reports have been out for over a decade. And as we were looking forward to our customers, our banks—and we felt this because I'm responsible for third party risk management at Q2, when we use vendors, we ask these questions—so one of the things that we were going through is looking at the reports is the absence of technology related standards to the use of AI.

And so we also at that time were going through a review of our current SOC 1 and SOC 2 auditor, and Grant Thornton was one of the auditors that we were talking to. And when we met up with Nicole and team at a meeting, I asked about this like, "Hey, what are you seeing? What are you hearing about in the existing reports, SOC 1 and SOC 2, anything coming out about AI security standards, controls we can put in place that can then be tested to provide some assurance?" And Nicole passed along, and I'm sure she can talk about this, that, "Hey, look, Grant Thornton as a national auditing firm is on many, many boards and committees related to industry standards.” And like with many things, decisions are made by broad-based committee. There wasn't anything that was immediately on the horizon that we, Q2 could go do that could signal to our financial institution customers that we were ahead of the game, that we were thinking beyond the traditional security metrics of a SOC 2 audit, let's say.

So thinking about that, and then on top of that, Q2 is rolling out AI-related products. Q2 Assistant is one of those. We also have another one, Q2 Code. We have a fraud intelligence platform that we're also rolling out, and we're integrating AI in all of these things to provide a better experience and a more effective product for our financial institution customers.

So thinking in the position of the chief risk officer in one of those institutions who naturally would have questions about our use of AI and the standards by which we grade ourselves, Nicole's suggestion was introducing us to AIUC and the standards that they were putting out. And the great thing about the AIUC is, in my position as I was looking at it, is not only did they have a set of, in our case, there were 65 requirements that we had to meet, not only did they follow the classic audit model or certification model, I should say, of there's a population of … you have controls and then there's a population of things related to proving those controls, and then there's a sample and then you get assurance that those things are done or not done.

But the beauty of the AIUC was, and as we all are experiencing this in real time, AI is rapidly moving forward quickly. So instead of a SOC 1 or SOC 2, which is a high-stakes, once-a-year audit, the auditor comes in, you collect all this material, they sample it, they review it, and then they opine on it, and then you pass that official opinion on to your customers. AIUC was that, but beyond that, it was a continuous set of tests across the year that then enable Q2 in a fast-moving technology environment to ensure that the things we're doing are keeping pace with both the technology that's advancing with AI, but then also the threats that come with AI. So those are the real reasons why we did it, and we picked Q2 Assistant as our first candidate product that we wanted to look at.

And we have plans to expand it to other AI products that we're rolling out in the market because now we can provide a certification to an absence of some national-level or some industry-level standard for the enablement of AI within systems that we can provide that to a financial institution and that in turn that the leadership team, and most importantly, the board and shareholders of that institution can have comfort that yes, in fact, Q2 is doing things in a way that is safe and trusted and they now have something, I hate to say they can hang their hat on, that's kind of how we think about it, and it enables Q2 to just deepen the trust that those customers already give us, that we're moving forward in a safe, secure, assured way that financial institutions’ customers can have comfort that the AI that we enable in their environment is done appropriately.

Cheryl Brown

Great, great. Well, Emil, so Blair told us a little bit about what that certification process looked like for Q2. Can you just back up for us a little bit and talk a little bit about how AIUC came to be and the problems that you were trying to solve with the development of the certification?

Emil Lassen

Absolutely, Cheryl. So we see agents becoming more and more capable, and these capabilities obviously drive more and more business value for a lot of organizations, but we also see the incidents that these agents create grow in both magnitude and in the blast radius of them. So when we started putting AIUC-1 together, really what happened was we spent a lot of time talking to enterprise leaders in security and risk, so the chief IT security officers of the Fortune 500 companies, and asked them really, what is it that holds up adoption of agentic AI in your organization? What are the questions your customers are asking? What are the questions your board members are asking and what would you like to see to prove this? And over time, this quite informal group grew into what is now the AIUC-1 consortium, which spans about 300 security and risk leaders across the Fortune 500.

So think CISOs for the largest companies in the world where the stakes are real. Banks, for example, in the case of Q2. And the three things we kept hearing that they would want to see when it came to agentic AI was one, we need standards that cover agentic risk specifically and keeps up with AI. Two, we need trusted institutions who know how to validate that these controls operate effectively. And then three, because AI is probabilistic in nature, it's not deterministic like classic software systems like BSS, we need to also test the agent behavior itself, and we need to do that more than once a year. So those three things were really the founding design principles behind AIUC-1. And the way we tackle each is first of all, we've created a standard with six pillars to cover enterprise risk holistically.

The six pillars are data and privacy. So what happens with the data that goes in and out of the agent who owns that? Is it protected appropriately also when it comes to sensitive data? Security. Can you prompt inject the agent? Can you fool it to doing things it shouldn't be doing like processing a transaction it shouldn't? Safety. Does it respond appropriately to end users also if it's a vulnerable user or a kid or something else where you want to make sure that the guardrails work? Reliability. Can you trust the outputs? Can you trust the tool calls that the agents make so that when it, for example, updates a database or processes a transaction, it's meant to do these things. The fifth category is accountability, basically ensuring that the organization behind the agent is thinking thoughtfully about how to build and deploy the agent and that you have good governance also if something goes wrong.

And then finally, societal risks such as how do we ensure that these agents are not misused for manipulation attacks or cyber misuse and so forth. And we then update the standard every single quarter, which is an insane process, but given the pace of AI capabilities, we just see it as the only way to create a standard that actually keeps up. And we do this in collaboration with the AIUC-1 Consortium members in the machine room with us rolling up their sleeves and setting the priorities for what we need to ensure that the standard covers. So the latest quarterly update we put out last week, it was open weight models, for example, that has become a growing focus point for a lot of organizations. So we now have three new controls dedicated to open weight models. So that's really the first pillar there. The second pillar of this is partnering up with trusted CPA firms who knows how to do these assurance engagements in detail such as Grant Thornton, which we have with us today.

Grant Thornton came in to do the Q2 assessment and what Grant Thornton brings is decades, if not centuries of trust. So they know how to do SOC 2 engagements from a long track record and a bunch of other engagements that go really deep and test both the policy controls that we want to see organizations implement, such as the legal policies around who owns the data going in and out of systems, but also the technical controls that organizations implement. By having these trusted auditors in the room, we ensure that the audit report that comes with an AIUC-1 certification can be trusted.

Cheryl Brown

Wonderful. So Nicole, Grant Thornton's role was to review the controls. So what did an independent review of AI controls look like? How was that different from reviewing a more traditional system?

Nicole Julius

Thanks, Cheryl. This is something, and Blair mentioned at the beginning, that we had been waiting on. We have all of our clients who are moving forward, they're trying to leverage all of the new technology available. And the reality is a typical SOC report is testing point in time controls, a lot of configuration controls, and you're pulling samples, but that doesn't give us a whole lot of comfort around what an AI agent is doing or an agentic workflow is going to accomplish. So when we first read about the AIUC, we were excited. And the first person I thought of was Blair. It's like, "Blair, I think there's something out here we can consider now. Let's run at this and see if it takes off." And the exciting thing is it has. It's even captured the attention of the AICPA who released press release, letting us know that they're in this partnership.

We have deployed members of Grant Thornton's team onto that committee because we're excited and we think this is definitely going to be something that is needed by everyone in the future and it accomplishes two things. So in a traditional world where you're just testing controls, that's not actually giving you comfort around what is happening day-to-day with that agent. So the really exciting part of this assessment is the fact that it is the controls piece where we're looking at things like AI governance and we're looking at IT general controls, but then what it layers in that nothing else has is actually results of the technical evaluations that are not as a point in time. They are done four times throughout the year, and they can keep those evaluations evolving as the technology evolves. These agents get smarter, they start to change behaviors. So we're really excited about the fact that while the controls really set as a point in time here, the real risk and the technical evaluation is going to be done quarterly to ensure that what we believe is happening is happening in that agent relationship.

So we're seeing a lot of interest in this in the market, and I will say Q2 is definitely ahead of the game. They've been really fun to work with because a lot of the organizations pursuing this are really in the tech space, true blown tech, honestly, open AI firms that this is what they do all day every day. But to get to be the firm who supported Q2 in a true fintech model is really exciting for us. We really think that this will help drive the market in this space.

Cheryl Brown

Well, and Blair, I assume that at Q2 we had to do some prep, we had to get ready for this. What did that preparation look like and were there any key learnings that the team got along the way?

Blair Williams

What I think we had to do is the first step was to get our teams to understand what exactly was going to be tested. And the great thing about it is they're very familiar with the SOC 1, SOC 2. When you think about Sarbanes-Oxley, they know what ITax and IT general controls are. They understand those things, but those controls and the things that they traditionally did weren't uniquely positioned to address the AI and the nature of AI in, as Emil said, in this probabilistic versus deterministic world. The good news for us was we felt that even in absence of a certification like AIUC or having an auditor come in and like Grant Thornton to review the controls and ensure that they were operationally effective in our environment, we were comfortable that we had integrated the requisite security controls, software development life cycle controls, access management controls, change management controls, all the different controls you would expect were up and in place for the baseline of this.

One of the other things that we've heard and seen across the industry is this notion of model evals, constantly evaluating a model from a model risk management. Is it operating as intended? Is it doing what it's supposed to be doing? Is it hallucinating? Is it off base? And so we had already built an internal eval tool that we were using to check those things as well, just to make sure that the models themselves were functionally aligned and were operating appropriately. So the good news was that development team and our general manager who led that group already saw the seriousness of that. They already had integrated a lot of those things into just their way of building Q2 Assistant. So that was kind of step one. So as you move on to other groups that are also building software, so we have other products that we are going to be very interested in getting certification on, it's also then ensuring that those teams understand and are educated on this is what we're going to come look at.

You may have 75, 80% of this done, here's the additional 15 to 25% that you're going to have to get in place and you're going to have to have it in place long enough so Grant Thornton can come and test it. There has to be evidence. So you have to let it run out a little bit, but having the knowledge of what's going to be tested is critically important. So those are the two big things from the operational execution of the audit, so the team that's going to be audited. The other thing I would say is getting the senior leadership of the firm involved as well. Like hey, this is materially, we believe, important to Q2's right to sell these products. This to us has become not a nice to-do, but a requirement in our mind that we have to give our customers, and ultimately it's roughly one in 10 Americans use our digital banking product.

We have to give them comfort that when they use our product and if there's AI being used either in the forefront or in the background of that product to service them, that it's safe and that it's trusted. So our senior leaders in the firm, not only the teams that are getting audited, but the senior leaders in the firm, that's just part of our culture, that's part of the mission statement of Q2. And so you really have to tie in why AIUC is important to the company in general, because ultimately you could see one, AI will become embedded into almost everything over time, and now it becomes just the paradigm you live in. It's not some additional add-on certification, it just is in the environment. When it came to the actual execution of the audit, as I mentioned before, we had 65 requirements that were specifically tested.

Some of those controls were what we would call entity-level controls, controls of how does Q2 do things like good access management? How does Q2 manage its identities? What are the identities that are engaging with the system? Is it a human? Is it an agent? Is it an AI system that's leveraging a human token to do things? So you look at those entity-level controls, and then as Emil talked about, the secret sauce of this is the continued testing of the environment for both functional drift and just general, is it operating as intended? And also could it be manipulated externally by a bad actor, whether that bad actor's a human or a human enabled by AI or AI to manipulate the product itself? And what I would say is not only did we have 65 controls, the AIUC also put us under 1,850 behavioral evaluations against our model.

So what came out of that then for us was a set of findings. And if you look at the tiering of findings, critical and high risk are the ones that you would care about the most, we did not have any critical or high risk findings. If we did, we wouldn't have received the certification. And then anything that's not critical or high risk now, I have a nice baseline as the risk officer to go back and do what classic risk management says I should do. Are we going to go remediate those moderate and lows? Are we going to transfer them away? Are we going to risk accept them? So now I get to apply my lens like I do with all other things risk that my team does across Q2 every day of what are we going to do about these things and how are we going to remediate them with the mindset that something that might be a moderate today could be a high tomorrow because we are being evaluated on a recurring basis by the AIUC team.

So I'm highly incented to fix everything that we have because the lows of today can become the highs of tomorrow as we're seeing with vulnerability management, as we're seeing in a whole other bunch of spaces across the industry. So again, I'll come back to it, Cheryl. It's getting the software teams to understand what they're going to be evaluated against. If you've done a good job internal to the company of prepping them, you should be doing many of these things already, that's the other thing, and then being ready to fix what's found. And then lastly, making sure the senior leadership team is fully bought into it and is talking to all of the engineers in the company, all the product management teams in the company. This is just table stakes for us as a trusted provider of digital banking services to our customers. That's the core thing at the very end I'd emphasize.

And if you can mix all those things together, I think you'll have a great engagement. And the AIUC with Grant Thornton, it was a great engagement. With every audit or certification you go through, if you came up totally clean, you kind of suspect, did I really get looked at? But I felt that the Grant Thornton team, as I would say, metaphorically took a good wire brush to us and they were scratching the rust off to make sure that we were clean under the surface when it came to our controls and the evidence we were providing. So those are the things I think that are necessary and the things that we did.

Nicole Julius

And I would just layer on, Blair, that I think what Q2 did a really great job of is understanding that holistically these agent-agent relationships are going to continue and they put a very, very strong entity-level control environment which will allow them to easily expand where they can drive this certification to.

Cheryl Brown

Let's take this a little bit broader. And for those banks and credit unions who may not work with Q2, so let's not talk about Q2 specifically, but any vendor that provides AI-powered capabilities, what should the leaders of these financial institutions be asking before they put this AI into their operations? What should a risk committee or a board expect to see as evidence and what should be a red flag? I'd love to hear from each of you. Let's start with you, Nicole.

Nicole Julius

Hi. Yeah. And so I'll pile onto what I was just speaking to before. I think it's about thinking about this from a programmatic framework. We can't just start thinking about we're going to deploy an agent here, we're going to insert it here. So it's really thinking about where do we know there's additional value we can offer our customers, and then how do we put a programmatic approach in place so that we can actually manage this long-term going forward? Because without that, then you have a bunch of agents running around the organization all under various processes and you've lost all control. And we hear that with clients that we talk to, that there was kind of a free-for-all period, each team was going out and doing their own thing. And so I think if I'm thinking about it from a risk committee, it's what's my programmatic approach to this and do I have it in place?

Because if I don't, this is going to be a really hard thing to accomplish and it's absolutely not going to be manageable long-term.

Cheryl Brown

Emil, how about your perspective?

Emil Lassen

I spoke to a CISO of one of the large companies in America recently, and she said that the way their organization had been managing AI in the beginning had been the approach of “Let a thousand flowers bloom.” Employees wanted access to these tools, so they went building and they built cool agents and they built workflows and they built all kinds of things. The world since that approach has changed, the incidents are growing more dramatic, and the cost of the incidents also growing with lawsuits and with data breaches and other things that you really want to avoid damaging your brand. So what she was doing in her organization now was starting to figure out what are the trees in that forest among the thousand flowers that we want to start standardizing. And drawing on that metaphor, I think if I was a risk leader in an enterprise today, I would ask my vendor first, what are your trees so you actually know what is the main AI workflows in your organization and how do you test those?

To Blair's point earlier, we're seeing AI risk management becoming a license to operate. This is also why we went out to partner with the AICPA to offer now that organizations can do their SOC 2 engagement and their AIUC-1 certification together as part of one engagement with a CPA like Grant Thornton, because it needs to be easier to get started with this certification. What we talked through before might sound overwhelming, and I also don't want to make AIUC-1 sound like a certification that is easy to get. It is not. It is for organizations that want to be on the frontier and want to take that extra step and prove that they take AI risk seriously and that they are open to be submitted to testing and that they know that the testing will find things because we've never certified an organization where we didn't find something in the technical testing.

That is just the simple nature of how these agents behave, particularly when you give them tool access, data access, and make them capable enough to solve real business problems and deliver real business value. So I think tying it back to that is basically what is the AI running in your organization and how are you testing that to prove that you know actually how it behaves, not just what the deterministic controls you have in your framework, how they operate.

Cheryl Brown

And Blair, as a chief risk officer yourself, what's your view?

Blair Williams

As we look upstream in our vendors, I know what we typically look for in our vendors. And if I look downstream at our customers, this is the types of questions we would anticipate they would ask us. So the three areas, and whether it's a customer or not a customer, and this could be any organization, so take financial institutions out of this. This could be any board for any company or organization that's looking to bring in AI from a vendor. It's really impact, assurance, and resilience. So the first question that has to be asked is what can the AI do and what happens when it's wrong? So really as that comes in, it goes to what Emil just said, what's the use case for that AI? What's the level of autonomy it has? What's its decision authority? Where's human oversight?

What are the potential customer financial compliance, reputational, operational impacts? I mean, AIUC even looks at societal impact, which is really interesting. And so it's like what could go wrong? What would be the impact if this particular AI use case has a problem or fails? So that's one, that's impact. Number two is assurance, and this is where AIUC and Grant Thornton's partnership with Q2, in my view, is critical, is how do we know the AI is secure, compliant, and operating as intended? And that's exactly what AIUC did. AIUC, they're asking us for information, they're testing us. And so if you're sitting in a bank risk committee or a board that's asking this, the question should be, what's the evidence for independent testing and ongoing monitoring for model performance, bias, data privacy, security, explainability, regulatory compliance? What about AI-specific risks? You've got prompt injection, you've got data leakage.

So there's this notion of what could go wrong impact, how do I have assurance now that those impacts, the likelihood of those impacts is now less? And then lastly, it comes back to resilience. No system is 100%. So what happens when that AI model or the vendor has a failure? What's the concentration of risk? How resilient are they in responding to it? How quickly can they pick up on those things that go wrong and fix them fast? So those are really the three core things that any organization that's considering AI, particularly at a bank risk committee or a board meeting should ask: impact, assurance, and resilience.

Cheryl Brown

Well, this has been an enlightening conversation. Thank you all for joining us on the podcast and walking us through the AIUC-1 certification.

Emil Lassen

Pleasure to be here, Cheryl.

Nicole Julius

Yeah, Cheryl, really appreciate the time here. It's been fun going through this journey with Q2 and AIUC-1. We absolutely see the incredible value and working with Q2, they make it very easy. They know what they're getting into and they have a very, very good risk framework in place around this. And so it's been a really fun journey and we continue to see a ton of interest in this. So Grant Thornton's really excited to continue to expand the scope of these certifications to additional organizations.

Emil Lassen

And I guess we should say congratulations as well, for getting the certification then, Cheryl and team. It's been a pleasure to work with you.

Blair Williams

Thanks, Emil. I think on the Q2 side, not only was it a good thing for us to go through, but frankly, it also helped us up our game. So these types of exercises you may think you're doing really well in giving yourself a B+ or A- in these areas, but you can always learn more. And AIUC and Grant Thornton just bring a lens that is so broad and I would say relevant to the current times we live in, that engaging with them was a fantastic experience for both myself and my team, and then the broader Q2 team that went through the audit. It only made us better, it only made us stronger, and I'm looking forward to bringing that benefit and leveraging AIUC and Grant Thornton across other parts of Q2 as we pursue AIUC certifications in those particular areas too. So more to follow from Q2 in this area as well.

Cheryl Brown

And that's it for another episode of The Purposeful Banker. You can subscribe to the show wherever you listen to podcasts, including YouTube, Apple, and Spotify, and you can find our full archive of episodes at q2.com/thepurposefulbanker. Until next time, this is Cheryl Brown. Thanks for listening.