Linux PAM

For nearly a decade, a China-nexus threat group avoided every tool defenders use to catch intruders. They didn't drop suspicious files on laptops or servers. They didn't run unknown processes.

Instead, they hid inside the Linux login system.

Security firm Sygnia, which tracks the group as Velvet Ant, discovered that the attackers backdoored PAM and OpenSSH, the very components that decide who gets to sign in to a Linux machine. Plant their access there, and ordinary cleanup can't reach it.

The earliest traces go back to 2016. That's nine years of hidden access.

The Target: An Isolated Network

The compromised network had no direct internet access. That's a common defense strategy. If attackers can't reach your systems, they can't break into them.

Velvet Ant worked around this by first compromising internet-facing systems. They used those as a bridge, passing commands through to reach the deeper, isolated network segment.

Once inside, they didn't bother with novel exploits or zero-days. They didn't need them. They had a better idea: change the login software itself.

How the Linux PAM Backdoor Worked

On many compromised machines, the attacker replaced the main PAM login module with backdoored copies. Researchers found nine separate versions across the environment.

The backdoored PAM modules did two things:
1. Allowed the attacker in using a secret password that worked on any account
2. Recorded real usernames and passwords as legitimate users logged in
The OpenSSH programs were altered the same way. They logged every credential entered and every command typed. The attackers even built in a hidden switch to turn logging off when they didn't want to leave traces.

Nothing about this looked suspicious to standard security tools. The files were legitimate login programs, just modified versions. No exploit needed. No malware signature to catch.

Why Normal Responses Failed

Here's where the Linux PAM backdoor Velvet Ant deployed becomes a nightmare for incident response.

Password resets? Useless. The backdoor recorded the new passwords the moment someone typed them.

Killed attacker sessions? Pointless. The attackers could just sign in again using their secret password.

Rebuilt the system from a known-good image? That works, but only if you know the backdoor exists in the first place. And on an isolated network with no direct internet access, detecting that backdoor is extremely difficult.

When the thing that checks credentials is working for the attacker, you have lost control of authentication entirely.

Operation Highland: The Same Playbook, Deeper

Sygnia calls this activity Operation Highland. The name reflects the group's pattern: move to the infrastructure defenders watch least, and set up there.
This isn't new for Velvet Ant.

In 2024, Sygnia found the same actor turning internet-exposed F5 BIG-IP appliances into internal command servers. Those load balancers are trusted by default and rarely audited.

Later that year, the group exploited a Cisco NX-OS flaw, CVE-2024-20399, to plant a backdoor on network switches. That bug requires admin access first, so it's a persistence tool, not a remote break-in. But once planted, it gives the attacker a foothold inside switching infrastructure, another blind spot.

Cisco patched the flaw in July 2024. CISA flagged it as exploited the next day.
Operation Highland is the same idea, one level deeper. Load balancers. Network switches. And now, the login software itself. All are trusted by default. All are rarely checked. All are perfect hiding spots for a patient attacker.

Why This Attack Worked for Nine Years

The Linux PAM backdoor Velvet Ant installed went unnoticed for nearly a decade for three reasons:
1. No malware. Traditional antivirus and EDR look for malicious files. Modified PAM and OpenSSH binaries aren't malicious. They're just... different.

2. Network No Beaconing: The isolated network could not access the internet and therefore attackers were able to use existing infrastructure as a bridge – resulting in no unusual outbound traffic to provide alert notification.

3. Normal Admin Work: Making file system modifications is common practice for Linux Administrators; as there was no integrity monitoring in place to capture the changes, no administrator was aware of the alterations made to the file system.

How to Find the Backdoor

The Linux PAM backdoor Velvet Ant installed cannot be detected by waiting for an alert. You have to go hunting.

1. Monitor login files. Watch PAM and OpenSSH binaries and their key configuration files. Alert on any change. But alerting alone is not enough—attackers could disable your monitoring.

2. To verify against known-good copies, obtain a copy of PAM or OpenSSH from a trusted package repository or installation media. You must then calculate the hash value of your newly acquired copy and compare it against the hash value of your known-good copy; when these values are not the same this indicates some form of tampering or possible compromise with respect to the source of your acquired file.

3. Conduct a search for unusual logging methods or locations related to credentials. Backdoors will typically place a log of usernames and passwords in some location within the system so identify these logs and their respective locations.

4. Audit authentication behavior. Look for successful logins using a single password across multiple different user accounts. That's the secret password at work.

How to Clean Up (Carefully)

Removing the Linux PAM backdoor Velvet Ant installed requires precision. A wrong move can lock every administrator out of a live system.

1. Test replacement in a lab first. 
Do not touch production until you have validated the clean binaries.

2. Eliminate Back Door
If you reset the user's password after the fact, the back door will have only recorded the password you've just reset if it hasn't been removed completely.

3. Utilize Trusted Sources for Replacefiles.
All files and programs must be replaced with a genuine version using the installation medium or through a validated package repository. Do not perform copied files from another infected system.

4. Verify Replacement Before Allowing Logins
Before allowing a user to log in, confirm the hash of their current password matches the hash of their original password. This way you know their new password has not been changed any further after you have re-instated them.

F5 and Cisco (CVE-2024-20399)

If you are responsible for F5 BIG-IP appliances or Cisco Nexus Switches, you will want to add these items to your security checklist.

Cisco Nexus Switches (CVE-2024-20399)

1.  The F5 BIGIP will need to update and patch in July 2024.
2. Inspect switches to identify potential unsecure connections.
3. You need to monitor switches for unauthorized access to the network from outside your company.

F5 BIG-IP

To help ensure that the integrity and security of our Company are maintained, the following should be done:
1. For every F5 BIGIP units you have, you should continuously review activity trends on unsecured outbound traffic that is generated from the load balancer.
2. Audit configuration changes
3. Treat these appliances as potential command servers

The Wider Lesson

The Linux PAM backdoor Velvet Ant campaign proves a simple truth: infrastructure that sits outside normal monitoring still needs integrity checks.
Load balancers, network switches, and now the login system itself are all trusted by default. That trust is exactly why a patient attacker hides inside them.

Defenders have expanded coverage to endpoints, servers, and cloud workloads. The next frontier is the infrastructure layer, the components that make everything else work.

If you aren't checking the integrity of your PAM and OpenSSH binaries right now, you are trusting that no one has changed them.
After Operation Highland, that trust is no longer justified.

FAQ Section

What is Velvet Ant's usage on Linux PAM backdoors?

Velvet Ant is a backdoor that has been planted into the PAM (Pluggable Authentication Module) and Open openssh components of linux. This backdoor allowed attackers to gain access using a secret password and subsequently record the credentials of legitimate users who have logged into the system. 

How long has Velvet Ant been under cover?

The earliest indications reveal Velvet Ant began its activity in 2016. The activity continued for approximately nine years from its original inception until it was detected.

What types of systems were targeted by Velvet Ant? 

These were isolated networks that had no direct connection to the Internet and the attackers used systems on the internet to bridge to the isolated area.

What steps can I take to prevent an unauthorized backdoor from being detected?

The only way to prevent an unauthorized backdoor from being detected is by using modified system files that previously existed and were approved by corporations or vendors. So without integrity monitoring, you will not see these modifications as part of normal system/admin.

How will I know if my Linux systems are infected?

You can verify that the PAM or OpenSSH binaries on your machine are good by comparing them to known-good copies from installation media or a trusted repository. If there are any hash mismatches, it is likely the system has been infected.

What steps should I take to clean up an infected backdoor?

You need to replace any backdoored binaries with known-good copies before you can reset any passwords. Once you have a working set of known-good binaries, you can then verify the integrity of the replacement binaries before providing login access.

Source: The Hacker News