Securing the developer: LastPass breach highlights risks of DevOps itself

Securing the developer: LastPass breach highlights risks of DevOps itself
Stolen credentials

Updated LastPass has published more details about how its systems were compromised via an attack on a home computer used by one of its senior DevOps engineers, showing not only the extent of the attack, but also how developer machines can be exploited by malicious operators.

According to the company’s latest post, “the threat actor was able to leverage valid credentials stolen from a senior DevOps engineer to access a shared cloud-storage environment … this was accomplished by targeting the DevOps engineer’s home computer and exploiting a vulnerable third-party media software package, which enabled remote code execution capability and allowed the threat actor to implant keylogger malware. The threat actor was able to capture the employee’s master password as it was entered, after the employee authenticated with MFA, and gain access to the DevOps engineer’s LastPass corporate vault.”

The serious nature of the breach is underlined by the fact that this engineer was one of only “four DevOps engineers who had access to the decryption keys needed to access the cloud storage service.” Data exfiltrated included access and decryption keys for LastPass production backups stored on AWS S3, including “customer and encrypted vault data.”

The attack on LastPass systems overall is complex and formed of multiple incidents. It began in August 2022 with a separate attack through a “compromised developer account”, according to CEO Karim Toubba, that lasted four days. Then in December Toubba stated that this stolen information was used to obtain further data. It is this second attack that has now been described in more detail.

What was the media software package? A report on Ars Technica claims that it was Plex, which was itself compromised and user credentials stolen shortly after the LastPass attack, though whether the two are related is unknown.*

The DevOps perspective on this is that in both LastPass incidents the point of entry was a compromised developer account. It is easy to pick holes in the policy or practice that allowed it to happen. Why was vulnerable consumer media software running on a home computer, and presumably with some level of remote access (a feature of Plex), when that computer was also used for security-critical functions which form part of the protection for the credentials of millions of customers?

Remote working is popular in the developer community though, and the problem is complex. “LastPass attack chain via home media centre of senior dev. Sssh, can you hear that? That’s the sound of a shitload of threat models being redone,” posted Daniel Cuthbert, co-author of the OWASP Application Security Verification Standard and on the UK government Cyber Security Advisory Board, on Twitter, adding that “the attack chain here is actually very good and raises a lot of concerns surrounding wfh [working from home], network design etc.”

Microsoft’s Clemens Vasters, Principal Architect for Messaging/Eventing, said “this is why you need to have automated management pipelines where critical secrets are only known to the system itself and no human can get at them, ever, even if they try.” But a comment on Hacker News asks: “Are there any reliable ways to secure remote computers from keyloggers _and_ still provide an efficient software development environment for non-trivial projects?”

Possible solutions include isolating developer environments from production as well as having developers work on protected networks.

* Updated to add on 6 March 2023:
Plex told DevClass in a statement: “We have not been contacted by LastPass so we cannot speak to the specifics of their incident. We take security issues very seriously, and frequently work with external parties who report issues big or small using our guidelines and bug bounty program.

“When vulnerabilities are reported following responsible disclosure we address them swiftly and thoroughly, and we’ve never had a critical vulnerability published for which there wasn’t already a patched version released. And when we’ve had incidents of our own, we’ve always chosen to communicate them quickly. We are not aware of any unpatched vulnerabilities, and as always, we invite people to disclose issues to us following the guidelines linked above.

“Given recent articles about the LastPass incident, although we are not aware of any unpatched vulnerabilities, we have reached out to LastPass to be sure.”