Driving Zero Trust at Two Different Companies: Lessons Learned
As we settle into life with COVID, the topic of zero trust security and a remote workforce is top of mind for all enterprise security teams. During this session you’ll hear from security practitioners who were responsible for the zero trust strategy and implementation at two Fortune 500 global enterprises – Adobe & Cisco. They will share their experiences and tips for rolling out zero trust methodologies at scale.
The audience will gain understanding in:
How to get started:
1. Selling the strategy, people, process and technology
2. How to make real progress in months, not years. 3. Demonstrating continued business value
4. Real world examples. Highlighting improvements to user experience and security posture.
5. Tips and tricks, lessons learned and what to avoid
Den Jones (00:00):
So yeah, I’m Den Jones. We have an agenda. So, I know we’re just after lunch and usually at lunchtime, everyone’s a little sleepy. All right. So let’s start with some exercises. Raise your left hand if you’ve heard the term Zero-Trust too much. Okay. Raise your right hand if you can read. Awesome. Because I don’t want to read the slides to you. I’m assuming you can read. So we’re just going to tell stories as we go along. Now, I’ve done a few of the future cons already. And then the last session, apparently my count of using the word shit was higher than my count of using the word Zero-Trust. So I’m going to try and not offend anyone, but we’re going to try and keep the Zero-Trust term down and I’ll work on the other stuff.
Den Jones (00:45):
So, prior to joining Barnum, which I joined in December, I actually led enterprise security at Cisco. And prior to that, I led enterprise security in Adobe. During 2018, we deployed a thing called ZEN, Zero-Trust Enterprise Network. There’s white papers about it. There’s YouTube stuff. So you can read till your heart’s content.
Den Jones (01:09):
But really in seven months we went from nothing to a deployment that we termed ZEN and that covered 40,000 devices. And I’m going to go into that a little bit as we go through this journey today. And then the other thing was, when I joined Cisco, they’d been talking about this thing for so long. And Cisco out of vendor, so they make stuff related to this, but they hadn’t delivered anything. They’d done pilots and stuff. There was a lot of reasons for why they weren’t getting the forward momentum. But then we stepped in and we reduced the team from about 75 people that thought they had something to do with this to five core people.
Den Jones (01:49):
So we’re going to talk a little bit about that journey as well, but that was 110,000 employees. Almost 200,000 devices. We said a 100,000 on the slide, because I didn’t count the devices. So I didn’t know if you’d trust the answer. And then I joined Barnum in December. So I’m a CISO, I’m a CIO because it’s such a small company that I do that with one hand tied behind my back. So the big part of the job is being out here and really engaging with people like you guys and talking about experiences I’ve had over the years because when people talk about this term, they’re not really talking about it in the sense that you can resonate with you. And it’s not a tangible thing.
Den Jones (02:31):
You’re going to need outcomes. You’re going to need business value. You’re not going to need some marketing term that nobody can even explain. So I’d like to explain a little bit about it. So we think of the identity in access management world, like I’ve logged in and I’ve always said that this is me. And then you’ve done multifactor authentication, if you’re good. And ultimately, we let you in, but. But I had five different devices. So which device was I using? I don’t know. Was that device managed by the IT team or was that just a bring your own piece of nonsense?
Den Jones (03:06):
Or maybe you’ve got vendors and you can’t touch their devices because they’ve got their own MDM. If you’re working with the PWCs of the world, their IT team has locked that machine down. They come into your environment. You’ve given them an account, but you don’t know about their device. So we were just saying, look, identity needs to not ignore the device. We need to realize that if one device is compromised, you don’t blast the user and say, okay, we’re going to pause your account, and then four other devices that were fine are not able to be used. And the users unable to do their work, right?
Den Jones (03:41):
The other thing was, this was pre COVID. So 2018 for me in Adobe was, we the workforce, it was more and more remote. We were 100% SaaS First. Every decision that we were making at the executive level was let’s be in the cloud. So any new application we were bringing in, we weren’t looking to build it in our data centers. We were like let’s cloud first. So we had to really think of, if my users are not on my network and they’re traveling more and the apps are not on my network, why do I want to bring people in my network to go out to the app? Doesn’t make any sense. So we have a shifting workforce and then there’s bad people. There are really some gnarly people out there and Adobe is a really juicy target for nation states, and for other people.
Den Jones (04:32):
I was more worried about nation states than I was anything else, because they’re the ones with the good tools. They’re the ones that will go after you knocking your door every day. They were just fishing people, right? We’re not doing brute force any longer. You might have some Internet facing devices and infrastructure that you may have not looked after very well and left the default password as password. And they’ll get in that way. But ideally you’re not. But what does happen is if you’ve got 40,000 people, do you trust them not to click the link because once a year they took your crap training? I don’t know. I think they were watching the soccer while they were doing the training. So they didn’t care.
Den Jones (05:15):
And I used to always say, look, your corporate, network’s not safe. My home network’s safer. Not because I’m the security expert, but because I’m not the target. Companies like Adobe, they’re big targets. So if your company’s a big target, don’t let your users think they’re safe just because they’re inside your network. That email apparently, got the bad actor inside your network. So it didn’t really matter. And that means that your network perimeter or the firewalls and all the money, you spend millions of dollars to protect your network. I just said, yeah, I don’t know about that. Maybe not do that so much. You still want some defense in depth, but you might want to rethink that the bad actors already on the inside, this perimeter thing. It’s not what it used to be in the nineties and earlier on. So I’d love to change the marketing budget for this slide.
Den Jones (06:09):
So I apologize. There’s no magic. The doves coming out of a hat. Maybe I’m meant to say pull a rabbit out of a hat. But ultimately, there was a couple of things that we had done. We wanted to imagine the world. So when I went to our execs in Adobe, I was like, look, imagine a world where users don’t use a username and password. And also we don’t change passwords every 90 days. So do you guys still change your passwords on a regular basis because of PCI, some idiot in the nineties said that was good, right?
Den Jones (06:38):
Yeah. Yeah. Sure. Well at both Adobe and Cisco, my team, we removed that need. No more seeing your password during the first factor of authentication and no more changing your password every 90 days. Self remediation is really important. I built a security intelligence team in both companies and the reason I’d done it is because you gather all this log information. You do nothing with it until there’s a breach. Then you’re like, let’s look at that now. Right? Okay, cool. What I wanted to do is let’s use that stuff up front so we can look at a animalist event. And if we see something bad happening, we’ll contact the user and the user can say, yeah, that was me or no, it wasn’t. Your banks have been doing this for years. Why has the identity industry not done this? So I took four college grads who were doing some other skunk court project and I had them get opensource libraries and do some machine learning. And it was really easy.
Den Jones (07:41):
Oh, it looked easy on a slide, right? I mean, I don’t know how hard it was in person and for them, but they’d done that stuff in a few months. So, for us it was pretty cool to then get this thing up and running and then you can go back to your PCI people and every other auditor and say we’re not changing passwords every 90 days because instead of that, we’re doing no password during login. We’re doing MFA and we’ve got security intelligence that tells us when that account’s compromised. And we got down to like really refined signals on that. The other thing was, I’d said to my boss, well imagine like never VPNing in again, right? You hate VPNing in. I mean it sucks. And especially if you’re using web apps in your company, like you’ve got an internal Wiki or an Internet site and people just try and hit it because they’ve got a link.
Den Jones (08:31):
You’re getting your email now without VPNing in. Right? So you click the link in the email, then all of a sudden the thing doesn’t work and now you have to VPN in. You’re like this sucks. Don’t do it to me. So when I said to my boss, it’s like, let’s imagine I’m now not going to VPN in. And let’s imagine the other stuff too. No passwords. And that really means the user gets to the app quicker because our job in our industries is we need to get users to be productive quicker and be more secure. So this whole speech is all about improving your productivity and improving security. So if I’ve got conversation with a CIO or a CSO, I can talk to both of them in a way that resonates. That makes sense.
Den Jones (09:20):
Let’s say … so actually raise your hands if you would like to finish a project before you leave the current job you’re in. Yeah. I’m not big fan of these 18 month long projects, right? These things are going for years and years and years and you never see business value. I used to lead the service management team in Adobe for about three or four years. I don’t know who is punishing me, but someone really had a grudge for me. Right? So I’m doing service management and every service management project had everybody involved in was 18 months or longer. And I was like, this sucks. So I started doing 12 week sprints even doing ITSM. So I figured if I can do 12 month sprints with ITSM, we can do this thing here too.
Den Jones (10:05):
So there’s a few benefits of this thing that people don’t all agree what the term really means. For me though, there was two things and very rarely in your career do get to say I improved security and I also improved the user experience. Very rare.
Den Jones (10:23):
So for us to say, you’re not going to use your username and password, and then we’re not going to have you change passwords every 90 days, a user knows how bad it is … when you’re changing your passwords every 90 days, what you’re doing is 15 minutes of recovery time. I’m updating four different devices later on. I’m forgetting my password. Service desk tickets. We were always in the top 10 password change related tickets was in the top 10 of our service test ticket count. We removed 80%. It was between 60 and 80% of tickets related to password changes. We got out at that top 10. We weren’t even in the top 20. So that’s really, really important.
Den Jones (11:07):
Now I want talk about lateral movement. So when we were exercising this in Adobe, one of the things that we’d learned was we had an M and A coming around the corner and the M and A means, usually you’re going to join their network with your network.
Den Jones (11:23):
It’s like, well, we don’t want to do that. We haven’t done enough due diligence on this new acquired company. And it’s going to cost us a lot of money to get the infrastructure, both sides, the network teams together, and really join this stuff. So imagine if you can just publish all the apps that you need them to access in your environment and enable their work stations to access that in a secure way, then you don’t need to join the networks. So what we’d done, was we made sure all of our applications that were internal, we made them cloud-like. So we exposed them externally, we have a zero trust platform, and then users could come in and access it without even being on our network, without VPN. And that was a huge thing is part of the M and A.
Den Jones (12:09):
Now the other thing is, so this lateral movement stuff, and the reason to say that is because really now what you’re getting to is your office networks do not need to be built the way they’ve been built for 20 years.
Den Jones (12:21):
If you build your office network like a guest network, and you can get out to the Internet and you can’t see your peers on that network, could you imagine what you can do when one device is compromised? Because I can tell you what the bad actors can’t do when they compromise one device and they can’t see 40,000 other devices. They can’t attack 40,000 other devices. So what we done with our implementation is we enabled it. So our NAC team, our network security team could change the NAC policy and say, this network’s now just like our guest network. If you can do that, you can eliminate lateral movement. It also means every other campus build that you ever do moving forward is cookie cutter approach every single time. It means you’ll build campuses quicker. You’ll onboard quicker. And frankly, you’ll not care what the campus is like from a networking perspective, other than you’re not building data centers as much. So it’s an office network.
Den Jones (13:23):
Now there is common components. When we were in Adobe, we’d done two different implementations. Our first one, it was built using Okta where we had FI. We had Vmware. So we hodgepodged some stuff together. And then later on, I challenged the team, a year and a half later, I challenged the team to go find me a vendor that can eliminate three of the vendors. And that’s actually the first time I met the guys at Baneon. Adobe later became a customer. When I moved to Cisco, Cisco had a company code Duo. So I was politely asked, I should or suggested I should use the Duo product to do zero trust there. And it worked. We got it working. I can certainly share the differences and the things to look out for between different technologies, but the components were very similar.
Den Jones (14:14):
You’ve got an endpoint user. You’ve got access proxy. That access proxy is the thing that gets you inside your network. And then the policy engine, the brain’s behind it. It’s the thing that says, does your device meet the criteria in order to get into the application? So that posture check was really important for us. We wanted to ensure that regardless of who managed the device, it was up to date within an OS, up to date with patching. And it had some endpoint security software.
Den Jones (14:45):
If you had software that our IT team pushed out, maybe you’re going to get a better score. If you had software from another vendor, hey, that might work too. Because I can’t control what endpoint software PWC might deploy to their folks. Right? So at the end of it, really, these components are really, really similar.
Den Jones (15:04):
Now I’m going to jump through in the interest of time here, policy engine, risk scoring. The cool thing for us, and one thing I loved in Adobe, was we actually gave every device a risk score and we had our device management portal and we rolled those scores all the way up to CEO. And that really gamified, when you see how the leaders and executives looking at their team and they suddenly see, wait a minute, why is her score better than mine? Because I don’t want to tell my boss why my score is the lowest in the team, right? So if I’m a leader and I’ve got five organizations, do you want to say your CEO, why yours is lower than your peers? So we gamified this and all we saw was risk scores getting better and better as people were now doing better things on their devices. So rather than removing software, then they were actually asking to get software on their devices to get a better score.
Den Jones (16:02):
When I left at OB, we were still at the point of flipping the switch. There’s a flipping the switch business where I’m going to build this all out. There’s a flipping the switch business where at the end of it, you’re going to deny access if someone doesn’t meet the posture. Depending on your culture and company, that can become a really dicey thing for an executive. So when I left Adobe, we’d enabled the capabilities. We were flipping the switch on some apps for some people, but we hadn’t went through the engineering environment because my boss didn’t want political suicide. S0 that’s a big thing for us now. One thing you don’t want to do … and there’s a practitioner who has now joined a vendor. So joined the dark side I guess, but I need to tell my company and anyone we’ve worked with you don’t want to throw out your existing investments.
Den Jones (16:54):
A lot of vendors will come in and they’ll tell you, hey, we can do this and we’ll do all of it. But you got to get rid of those things over there because that stuff’s shit. It’s like, well, no, you can’t do that. I’ve got a five year contract on this thing. A three year contract here. One year here. What I really want is to leverage those existing investments. The existing investments also include your team. The people who are doing the VPN stuff today … hey, they can take on the proxy responsibility for that part of the components. Or the people who are doing directory and then endpoint. They’re still the same people.
Den Jones (17:27):
So the team that you’re building really is a small team. So for us, the big thing here is selling the vision. That vision was absolutely focused on experience of the end users. The one thing our CIO was under pressure to do was to save the company money and improve the user experience. So when we were talking about this whole program, it’s let’s improve the user experience. We started with a really small cross-functional core team. We had five people in our team that had done zero trust in Adobe. We outreached to some other people every now and again, but it was five people. I incremented my staff by one. And then the first year spent $240,000. And that got us deployed to 40,000 people.
Den Jones (18:19):
Concrete use case. The jam for us was an M and A. We had this M and A around the corner, and we didn’t want to connect their networks. The M and A was a high risk country as far as we were concerned from a cyber perspective. So we didn’t want to join the networks. And one idiot with a Scottish accent said, we can use ZEN for this. And my boss went, oh, okay. So I went back to the team and said, hey, we’re, we’re going to use this for that. And they were like, we’re not ready. I was like, okay, cool. You got four weeks. Let’s go. But it totally legitimized the use case. It legitimized what we were doing and everybody got it.
Den Jones (18:57):
The existing investments I already covered. No one wants to look like a fool. If you guys always leaders, you’ve spent all your money, you’ve got all your budget already in. You don’t want to then suddenly say, wait a minute, we’re going to rip this stuff out over here. That last year I said was the best thing we had to do. So you’re not going to rip that out. Right. And then we used to run the identity platform and authentication space. So I’ve been doing directory services since the mid nineties, right? I’m an XML guy, so for me it was always fun to say, well, I’ve had this thing. Nobody needs to, oh … you don’t ever need to ask anyone permission to improve the authentication experience. I’m just like, would you like to log in less? It’s not a question I was going around asking. Would you like to log in 10 times day or just once?
Den Jones (19:48):
So I didn’t ask that question. But what we do is we integrated into that Sam workflow so we could then do the posture check. And we’ve got white papers and stuff and more technical things there. So five things, really easy. For some reason in the industry, people say, I don’t know how to do zero trust, but the boards are all hearing about it, all the execs hear about it, but nobody all knows what it is. So we didn’t really talk about that. We just talked about the outcomes because we’re all rewarded to deliver business value. We’re not rewarded to deliver someone’s marketing crap.
Den Jones (20:28):
I’m assuming … so raise your hands if you’ve ever led a project before. Awesome. I just saved you four slides because ultimately all this says is we’d done a POC. And by the way, since you’re going to get these slides, you can read all this later, but it’s the usual stuff. We’d done a POC and then we went to the next phase. Cool. So let’s talk about what we learned and things to avoid. So executive support is essential. I think that goes with the last slide. The biggest thing is no vendor can do it all. They’re all going to try and tell you they can do it all. But even me now working for one of those dark vendors over there in that hallway. Nah, we can’t do it all. But can we integrate with the existing investments you have? I think that’s more important.
Den Jones (21:20):
And that was really important for us. And then introducing principles to an organization. Principles. Cool. And you can say, is this an architecture or is this a, who gives a trap? What we wanted to do is say, look, we’re really focused on users access and applications. Some of the other players out there will tell you there’s two other things. There’s work load and there’s workplace. But workplace is kind of like NAC you know, we’ve been doing that stuff. And what happens if you’re not in your workplace? What happens if the users are remote?
Den Jones (21:53):
So we avoided a lot of that. And then the workload to workload. Yeah. Cool. But if you’re looking for visible bang to your buck. If you’re really looking for kudos, then it’s the user to the app, the user to the data, because now I can actually enforce that unless you’ve got the DLP software of choice, maybe you’re not going to get to that really critical app. Maybe that’s one of my policies. That’d be cool. So avoid the term zero trust.
Den Jones (22:22):
Hey, by the way, this deck I’ve been pretty much using the same deck since 2018 when we deployed this in Adobe, I feel like the band that only had one good album and we played the concert for the next 30 years. We didn’t use the term zero trust because back then nobody really knew what it was. Beyond corp was the buzzword at that time. And that’s because Google spent a trillion billion on their thing.
Den Jones (22:48):
Sharing the benefits with leaders like every other thing you ever do is vitally important. As is certain expectations. What we also had done was, we were actively communicating all the time. And the one thing I always love to do, if I’m going to do a sprint, that’s really a vital and quick, I’m going to do a weekly status update and on the top right, I’m putting a number of days to done. So whatever you agreed done is. When I was at Cisco, we’d agreed we will be done before RSA. In order to be done before RSA, we had to be done the month before. We had to then submit the talk for RSA. We had to have all the PR stuff done. So there was this whole time. And then I … because I’m bit cheeky. I wanted to take away the number of days for vacations, number of days for the weekends.
Den Jones (23:37):
You know? So at the end, that number of days are done was starting to look really, really small. So it’s like, okay, you want to argue the point? Well cool because we’re going to lose another day. You got 46 days left. Now, do you feel good about your point you’re arguing? We’re not going to miss this deadline. I’ve got a bit of a disdain for PMs that in order to be green again, they’ll just refactor the timeline and push it out a month. So that helped me enable that not to happen. So yeah, we used that countdown. Users was funny, right? So one day in Adobe, we’re sitting there and our SOC gives me a call and they’re like Den, we get a problem with our network security. And I’m like, what is it? He’s like, well, somebody’s just submitted this report.
Den Jones (24:23):
He’s done some research. And he’s discovered that he can get in to all these internal applications without VPNing in. I’m like, shit, okay. Could be bad, I guess. Maybe. We’ll check this out. So we contacted the user and we discovered that the user was part of our pilot program who … their machine was already set up with our zero trust goodness. He was in a group of users and going to the apps that were published and I’m like, shit, yeah, yeah. Oh yeah. I guess we just didn’t tell 40,000 people that we were going to do some magic like this. So ultimately the guy thinks he’s a genius. I’ve hacked into an internal apps. I’m the security wizard of the year. It’s like, no, dude, you’re just part of the future. So part of the future, no more VPN, you don’t need VPN. Nobody likes that crap anyway.
Den Jones (25:20):
Improving your employee experience. I’d like employees who are happy. That’d be nice. Working in IT for over 20 years, I can tell you my ears are tired of people phoning and complaining all the time. And then this device security and improved posture. It’s awesome when you get to a position where you can say, I know the devices are accessing these applications. At least meet a minimum bar and I’m not meeting the ones that we’ve got IT managed with our MDM. I’m talking about any of them. That’s something that people today still struggle with. How do you deal with vendors without throwing them through some expensive VDI platform? So this is the shameless commerce slide. I apologize. Apparently me talking nonsense for 25 minutes means we have to do this slide. I want to draw your attention to the bottom right hand side. We have a free team edition.
Den Jones (26:18):
Because one thing like you guys, I hate going around the vendor booth and hiding from all the sales people. I hate them calling me. I know the minute they’ve got your number. Like all of you guys have got your contact details. You will now be harassed for the next two months by all the vendors sitting through there. So I didn’t like that either. So one thing that this company’s doing pretty well is we’ve got a free version, like an entry level subscription. You can set it up yourself. 15 minutes or less if you believe the Koolaid. If you want to do the advanced stuff, I think it’ll take a little bit longer, but it really does work.
Den Jones (26:55):
Now the contact does is, this guy here normally would be standing somewhere here, but he’s busy working. So Carlos and I actually were at Adobe together. He was my right hand guy on the zero trust stuff there. He joined me at Cisco and for some unknown dumb reason joined me at Banon as well. So we are available and as we jump into Q and A, I’d just love to leave you with one thing is if you like this accent and the silliness, we do have a podcast where we talk about things, not just related to the topic of zero trust. We talk about all sorts of nonsense. So questions.
Den Jones (27:34):
Did I explain it so well, there’s not a question. And yeah, it’s quite a good picture. I was impressed how they photoshopped that. So I think we’re about out of time, Kim? Right?
Den Jones (28:01):
Excellent. Well thank you very much, everybody. Appreciate your time.
Book Office Hours with Den Jones
If you are interested in chatting with Den Jones in a more informal setting to talk about your challenges, he hosts office hours that you are welcome to schedule with him directly.
Den is a seasoned professional and loves talking about the best ways to get started, how to measure progress and finally how to get things done.