Some of you may be wondering, how prevalent is the use of open-source software in the industry? In 2018, Black Duck Software (now Synopsis) audited over a 1,000 commercial codebases and reported that 96 percent of them had open-source components, with the average application containing 57 percent of open-source code. In 2017, there were a total of 4,700 vulnerabilities reported, with 54 percent of these considered as high-risk. In case you are wondering how bad these vulnerabilities could really be, let me remind you of the infamous Equifax breach of 2017.
Details of the Equifax Breach
Apache Struts is a popular open-source web framework that is used by many Fortune 500 companies, including Equifax. One of the basic functions of the framework is parsing the “Content-Type header” in the HTTP request. If the content type is unknown, the Struts framework throws an exception. In certain versions of the Struts package, the exception handler, while building the error message, executes the content-type value without any checking. An attacker can issue an HTTP request that includes a malicious string in the content-type field and, consequently, have their commands executed under the privilege of the web server. This is a classic remote command execution and, it has been fully exploited in many other websites besides Equifax.
On March 8, 2017, the U.S. Department of Homeland Security’s Computer Emergency Readiness Team (US-CERT) published an advisory disclosing the Apache Struts vulnerability and recommended an upgrade to version 2.3.32 or 2.5.10.1. Despite Equifax’s knowledge of this and instructions to have the servers patched within 48 hours, these updates were not applied to their online disputes portal. Barely two months from the publication of the advisory, on May 13, 2017, attackers exploited the Apache Struts vulnerability in the Equifax disputes portal and started exploring the surrounding environment for information they could extract.
Once they were able to execute commands on the compromised server, the attackers were able to access three locally attached databases. Over the next few days, they hunted for more and found 48 other databases that they were able to access. In total, they ran 9,000 database queries extracting customer names, Social Security numbers, birthdates, and addresses, affecting more than half the U.S. population. The attackers were able to stay undetected for 76 days and exfiltrated the information in small chunks. The picture above graphically depicts the actions of the attacker. You can find the full details of the Equifax breach in the Government Accountability Office’s report to the U.S. Congress.
How can we best protect ourselves?
While developing features, we download the latest stable version of the open-source libraries to incorporate into our application. Once we are done, in most cases we never think about it again. Unfortunately, along with the functionality we also inherit latent bugs and vulnerabilities. Often times, a vulnerability will be fixed by the authors and a patch or new version will be released, like in the Apache Struts example. It is important for us to keep track of the version of the components that we are using and to have a regular process in place to scan the National Vulnerability Databases, project home pages, and community forums to make sure that all of the components we use are at the right patch-level. We will be evaluating tools like BlackDuck and Snyk that are focused on keeping track of these vulnerabilities and automating the tasks to find and fix them.
Keeping track of it is only part of the solution. As was evident in the Equifax case, not updating the version in a timely manner cost the company and a majority of the U.S. population dearly. We need to make it part of our regular CI/CD process to update OSS components as soon as we are aware of any critical fixes.
It is also good hygiene to ensure that we don’t use OSS components in ways that make it difficult for us to upgrade to a new version. Stay clear of branching the open-source component and committing some of your own enhancements and fixes in this branch. Avoid the use of private APIs and only use public APIs that are widely used by others. This will make it easier to update the open-source component in the event that a vulnerability is found and fixed in a later release.
