We have already failed to secure AI by doing what we did before - repeating the mistakes of the past
A different take on the problem of AI security
Welcome to Venture in Security! Before we begin, do me a favor and make sure you hit the “Subscribe” button. Subscriptions let me know that you care and keep me motivated to write more. Thanks folks!
Since the beginning of this year, I’ve been closely following the AI security space (my most recent overview of the segment can be found here: Securing AI: state of the market at the beginning of Q3 2023). I think companies securing AI and ML workloads are going to be critical for helping us safeguard our digital and physical worlds in the next decade to come.
While all that is true, there is another rather controversial thought that has been sitting in my mind: the sole fact that we have a growing number of AI and ML security companies is a sign that once again, as a tech industry, we have failed miserably.
The story of unsafe houses and the Internet
Construction technology provides a fantastic case study that is quite relevant to what we are seeing in cybersecurity today.
Decades ago, when civil engineering technology was still relatively immature, there were few safety considerations construction companies had to care about. Although the risks of earthquakes aren’t new, the concept of earthquake-resistant or aseismic structures isn’t that old. It’s well understood today that earthquakes cannot be prevented, but what we can do is reduce the magnitude of impact. Many countries and states in the US passed regulations designed to do just that but compliance with these regulations hasn’t been seamless. A case in point is California where seismic safety standards were enacted about 30 years ago, yet hospitals in the state continue to ask for more time and flexibility because not everyone “can afford” the cost of retrofitting the old structures.
Despite all the challenges, structures designed today are expected to conform to the building code which doesn’t just tackle the risks of earthquakes, but also fire, tsunami, flood, and others. Retrofitting old houses to be safe isn’t always easy, in most cases doesn’t result in the same degree of robustness, and can often cost a fortune. To make matters worse, spending money on seismic retrofitting and fireproofing does not make buildings more valuable, so the owners typically skip it. This has real-life consequences: experts agree that many of the 45,000+ deaths during the Turkey-Syria earthquake in 2023 could have been prevented if people retrofitted their buildings as they were advised.
Problems we see today in cybersecurity mirror those in construction. When the web was built, security was seen as an afterthought. When it became clear that the structure was shaky and the Internet infrastructure was failing because of poor privacy and security, we started taking small steps to make it better. The insecure systems and technologies we could replace were replaced with more robust options, and a lot of the underlying infrastructure was “retrofitted”. For instance, we added “S” to HTTP, and evolved SSL into TLS. Not all infrastructure could be changed. For instance, DNS which translates domain names into specific IP addresses cannot be retrofitted in large part because it would require all providers to agree to make the change and do it at the same time. SMTP, the protocol that powers our email communication, is ridden with gaps and security issues as well.
Since we cannot rebuild the whole Internet from the ground up, security professionals are doing what they can to make it safer. Despite all that effort, every new technology we invent has to make use of the components we all know have not been made robust from day one.
How we are repeating the same mistakes with AI
I have heard many security professionals say that “if we had a second chance to build the Internet knowing what we know today, we would have surely made sure that the foundations are strong and secure”. The current trajectory of AI adoption and development illustrates perfectly well that those are nothing but empty words.
Most of the companies building the underlying infrastructure for AI do not see privacy and security as fundamental architectural requirements for their systems. Don’t take me wrong - all of them say security is important, yet actions speak louder than words. We are so excited about the potential of AI that in this gold rush we are willingly repeating the mistakes we made before when designing the Internet, cloud, and other technologies. I would go as far as to say that what we are doing today is much worse: founders of the Internet had some idea of its potential but didn’t predict that users would look for ways to subvert the code into doing what it wasn’t designed to do and attack one another. After decades of witnessing the breakdown of security, the rise of cyberattacks, and the professionalization of cybercrime, it’s crazy to think that tech companies understand very well what will happen if they don’t put privacy and security first from day one, yet we still move forward as if that doesn’t matter.
Having learned nothing, we are taking the same shortcuts, optimizing for shortening time to market and gaining adoption instead of making sure to put privacy and security first, consider every possibility the new technology can be misused, and put solid guardrails where they are needed.
Once AI and ML gain wide adoption, it will be too late to secure them. We have been through this with the Internet: in the 1990s, to update an underlying component of the World Wide Web, one needed to convince a thousand people representing Internet service providers (ISPs) and several other parties to agree and make the change. Today, the number of people, businesses, and organizations of all types that could be affected by the tiniest change to fundamental parts of the internet like email or domain names is so high that no changes are possible. We are still early on the path of AI adoption, but the time is ticking so it’s a now-or-never situation: if we can’t find ways to secure the infrastructure today, we won’t be able to do it later.
“Security by design” is incredibly hard: creative adversaries will always find ways to exploit technology that could not have been predicted by their creators. However, building privacy and security into AI from day one requires us to take a strong stance, stronger than signing aspirational letters to pause innovation or agreeing to unenforceable voluntary commitments.
The way we approach security today is akin to constructing buildings that are not designed to withstand earthquakes and hoping that another firm will convince buyers to retrofit them. People are bad at comprehending the probability of rare events, and they don’t want to spend limited capital on something that “will certainly never happen”. With that, by not designing solutions secure from the ground up, we are making a conscious decision to build them vulnerable instead.
Drawing the line between AI vendors and security vendors
A few weeks ago, I published a brief overview of the current AI security market featuring some of the companies (many more are in stealth) trying to solve this hard problem. Passionate technologies and security entrepreneurs all over the globe understand the importance of securing this critical innovation from day one.
Source: Securing AI: state of the market at the beginning of Q3 2023
I have had an opportunity to meet some of the people working at these companies, and I have high confidence that many of them will play a critical role in securing the technical fabric of the future. Yet, no matter how hard they work, I don’t think it will be enough unless the AI vendors will architect security and privacy in their solutions from day one.
Drawing a boundary between the security responsibilities of the infrastructure providers and those of their customers is not easy. However, as we have learned from our experience securing the cloud, some answers are obviously wrong. It took Amazon many years until it finally added default privacy settings to S3 buckets to stop the epidemic of data leaks caused when AWS customers would accidentally leave their S3 buckets wide open to the internet. It should be the responsibility of infrastructure providers to:
architect and build their products with privacy and security as the underlying principles from day one
invest in developing robust privacy and security capabilities
default settings in their products to the most secure configuration
When all this becomes true and the underlying infrastructure for AI is built, cybersecurity vendors can focus on helping customers protect what they build and assemble on top. We cannot afford to build the shaky foundation and hope that we can add hundreds of disjoint tools on top to make it safer. Security vendors can add the most value in understanding the unique environment, workflows, and needs of every business looking to leverage AI, and help them solve their problems without compromising their security. I would like to think that in the near future, AI security services and professional advice will be much more common than black box “magic tools” that promise to “keep AI safe”.
Why security will slow us down and how we can live with that
Whenever we talk that it’s critical to architect new solutions with privacy and security baked in from day one, someone will inevitably say “We can’t do that because speed to market is critical, and privacy and security don’t make anyone faster”. That’s true, designing with safety in mind will indeed slow us down. Having said that, we have witnessed how these kinds of slowdowns affected other industries and not only accepted this new reality but learned to see it as the right way of doing things.
Let’s go back and have a look at the evolution of the construction space. In the late 1800s, there were no safety measures for construction workers whatsoever, so the number of fatalities was staggering. It may be shocking for some to think about it, but the construction of the Golden Gate Bridge in the 1930s was the first time construction workers in the US were required to wear hard hats. With time, our tolerance for casualties went down, and the projects that could have been seen as great victories centuries ago would not have been tolerated today. About 30,600 people lost their lives during the construction of the Panama Canal and about 60 people - during the construction of the World Trade Center (WTC). Neither of the two projects would have been allowed to continue if this was happening today (the death rate at the WTC was about 17.14 individuals per 1,000 workers - absolutely insane by today’s measures).
To deal with the death and injury rates, we had to implement restrictions and regulations, and consequently slow down. Even though many construction workers would prefer sneakers to hard boots, and the feeling of the wind messing with their hair over hard hats, neither is possible, and the regulation is being actively enforced. When we think about technology and AI in particular, the same approach can prove valuable: slowing down to pay more attention to security might very well be the right way to build robust foundations that will result in fewer casualties over the long term.
Closing thoughts
Despite the ever-growing investment in security tooling, the number of cyber breaches has been growing year by year for the past decade. One of the reasons is the fact that the Internet was built insecure from day one. We have done our best to retrofit it by securing as many components as we can, and by adding external tools around where we couldn’t. But, decades later, it is clear that when the underlying architecture wasn’t designed with privacy and security in the core, sprinkling security products on top can only do so much.
Artificial intelligence isn’t just a new tool - it’s a new stage of technological development that requires Internet-level infrastructure. Naturally, AI companies are in the race to become first, and security, as we’ve discussed, doesn’t make anyone faster. It’s understandable to focus on growth and market expansion and plan to secure it “later”. We must, however, do better: having accumulated a corpus of lessons learned from securing other technologies such as the Internet and the cloud, we need to leverage that knowledge to design AI securely from day one.
It’s worth remembering that we are still early, and it is today that we are establishing the foundation for the future powered by artificial intelligence. If we fail to do it the right way, there is no chance we can later solve the problem of AI security by funding thousands of vendors and expecting that enterprise buyers will get in line for AI/ML-DR and AI/ML-SPM solutions.
Interesting and relevant reading. I really like the construction to internet to AI analogy. That made it a very relatable lesson / story. Thank you