Actionable Guidance For Log4Shell Remediation

There are already multiple posts on the Log4j vulnerability (also referred to as “Log4Shell” or CVE-2021-44228) from others explaining the technical details of how exploitation occurs. This post will provide actionable advice for organizations that may not be sure if they are impacted.

The vulnerable component here is a library built into thousands of software products. This is important because the normal “patch now” may not be your best advice in dealing with the vulnerability. For instance, although many programs use libXML, when a new vulnerability is discovered in libXML, Windows users must often wait on downstream application maintainers to patch their products first.

The vulnerable versions of Log4j are being reported as 2.0 – 2.14.1. However, the 1.x branch of Log4j is no longer supported and as such was not originally reported as vulnerable. There are other reasons that you shouldn’t be running any software using Log4j 1.x versions. Still, until now, there hasn’t been a widespread, remotely exploitable vulnerability that would prompt the significant expenditure of resources to update (which in this case would likely require refactoring of application code).

This problem is made more difficult because many organizations have log4j installed as a component in their software but don’t realize it. Software Bill of Materials (SBOM) and Software Composition Analysis (SCA) solutions are designed to identify these unknowns, but that’s forward-looking. Organizations are advised to consider how they might use these solutions to find other unknown components to be more prepared in the future for vulnerability management.

Zero Trust Networking: A Must-Have

The good news here is that to gain code execution, the vulnerable server must make an outbound request to an attacker-controlled server in most cases. Most log4j applications are vulnerable to attack run on web servers, making very few outbound calls to remote servers.

Zero-trust networking (deny all, permit by exception) will be key here. The best advice I can offer at this time is to restrict outbound connections from any potentially vulnerable server to only known-good locations. I believe this completely mitigates the proof-of-concept code we’ve seen so far and certainly so for the automated exploits already deploying coin miners.

There may be unknown exploitation vectors, so patching is still ideal. Additionally, threat actors still might be able to execute a Server Side Request Forgery (SSRF) on a vulnerable server to an expected destination. This would be extremely niche in most cases. The point is that simply limiting outbound requests does not entirely remove the risk of exploitation.

Limit Outbound Connections

Even if you are not sure if you are impacted by this vulnerability (or are sure you are not), limiting outbound connections from a web server is always a security best practice. Doing so mitigates all sorts of attack classes, including most SSRF vulnerabilities and Remote File Inclusion (RFI) attacks. While limiting outbound connections requires diligent analysis to prevent impacting operations, it is a high return on security investment activity.

Another possible mitigation is to restrict exploitation by changing the JVM command line in application configurations by adding the following:

-Dlog4j2.formatMsgNoLookups=true

However, I don’t recommend this approach. So many Java programs spawn subprocesses with their own JVM command lines, and many won’t inherit or honor the configuration change. Organizations should consider this a “best bad plan” if they choose to implement it. The best bet by far is to block outbound connections at the firewall for any impacted server. An analysis of firewall logs will help determine legitimate/normal outbound connections that need to be allow-listed.

What to Communicate to Executives

Mitigating the risks posed by Log4Shell will undoubtedly be a marathon, not a sprint. The task of even identifying what’s vulnerable will take time – as will installing patches handed down by myriad vendors.

CISOs, here’s what you should be communicating to your executive team about Log4Shell:

• -- Inventory all software & systems to determine what's impacted.
• -- Track the public statements of each vulnerable vendor.
• -- Block outbound connections at the firewall for any impacted server.
• -- Patch systems when patches become available.
• -- This is a marathon, not a sprint -- it’s not something that we’re going to fix today.
• -- Threat hunt to understand whether or not we were exploited by this.
• -- Shift towards – at a minimum – a limited zero-trust model. Recognize that you have specific risks across specific systems and deploy a zero-trust model here.
  
  

Helpful Resources

TrustedSec’s Log4J Detection and Response Playbook

Huntress’ Log4Shell Vulnerability Tester

NCSC-NL List of Known Vulnerable Software

IANS Faculty Tanya Janca’s Explainer Video