SSH keys, which are an access credential for the SSH (secure shell) network protocols, are undoubtedly one of the most powerful yet overlooked access credentials used in enterprise environments today. IT and security teams increasingly rely on SSH protocol to secure automated processes and remote administrative access because of the high level of trusted access it provides. Yet SSH keys tend to get less attention from security teams than more traditional forms of access credentials like usernames and passwords.
As enterprises continue to adopt cloud and infrastructure automation, they increase the number of privileged SSH connections and the volume of SSH keys within the organization. Today, most enterprises average 50-200 keys per server, with upwards of a million keys across the environment. While hardening trusted connections, SSH access has unfortunately produced adverse effects too: key sprawl and a general lack of oversight and governance are eroding efforts to implement zero-trust initiatives and opening the door to attackers looking for covert network access.
The sheer volume of SSH keys and increasing risk of SSH-based attacks is concerning for IT and security teams. The good news is that organizations are focusing on how they can securely manage keys and digital certificates - but many do not know where to start.
Understanding How SSH Works
SSH key usage and its surge in popularity come down to security and ease of use. Traditional authentication methods like usernames and passwords are vulnerable to basic attack techniques like brute force because they require individual users to log in to individual servers. Public key infrastructure (PKI) and SSH keys do not require user interaction for access – SSH protocol provides strong, encrypted authentication and communication between users and machines over unsecured networks.
SSH protocol is built into Linux, Unix and Windows systems. It’s trusted by IT and security teams, as well as developers and database administrators who leverage SSH to remotely log in to multiple systems. The security appeal, flexibility and wide usability of SSH make it well suited to IT process automation, particularly when machines need to connect to one another without user involvement.
SSH authentication is simple and can be done in minutes by generating a key pair on a user’s workstation, then sharing the public key with a server the user needs to connect to (provided the user has the appropriate access privileges). SSH is convenient, but in a situation where administrators prioritize speed over security, SSH key generation can get out of hand, produce thousands of unmanaged keys and lead to excessive levels of privileged access and risk.
Best Practices for SSH Key Management in Multi-Cloud Operations
Cloud-first, distributed IT and hybrid workforces have made SSH more critical, and consequently, SSH keys are commonly used across three primary applications:
1. Linux: Most public cloud workloads run on Linux and SSH has emerged as the de facto credential to secure remote access. In terms of security risks, exposing access or credentials over the open internet can make them vulnerable to attackers.
2. DevOps: SSH has become a tether between IT operations and developers, facilitating collaboration and security for automated build and release processes.
3. Remote work: Fully and hybrid remote work models are here to stay. SSH allows users to configure and manage systems from anywhere. Security and risk professionals within the organization must have complete and regular visibility into how SSH keys are being used, by whom and the level of access they grant.
Establishing a key management framework is the first step to finding and eliminating existing vulnerabilities and defining how SSH keys will be deployed and managed moving forward. The following six steps can help your team build out its base key management framework and serve as a tool to manage and mitigate key sprawl:
1. Discover and map keys: The first step in eliminating SSH key sprawl is to discover existing keys within your network and bring them into a centralized repository. Once you have gathered all your keys, you can start to map key-user relationships to better understand what they grant access to. Using a network-based mechanism to discover keys takes the heavy-lifting out of inventory and mapping trust relationships to associated users (private keys), servers and service accounts.
2. Analyze your risk: Once discovered, SSH keys must be thoroughly analyzed for potential vulnerabilities, either in configuration or usage. For instance, you will want to make sure that keys are only configured for root-level access when it is necessary. This phase is all about finding ways to reduce your potential risk exposure and achieve better ‘crypto-hygiene’ for auditability. Look closely for root-level access permissions, forgotten keys, orphaned public keys with no known private key and weak keys with shorter key lengths.
3. Remediate vulnerabilities: Once you have identified your risk exposure, you can take action to reduce it. This is not a one-time effort though – continual monitoring and reporting are essential in identifying new risks as they surface, such as rogue keys created out-of-band. With a complete and accurate inventory of all SSH key pairs, you can start to rotate or replace weak and outdated keys, remove duplicate keys or keys with unnecessary root access, and clean up unused or orphaned keys. If someone leaves the company, you can remove their keys from your servers to keep things clean and secure.
4. Create fresh key pairs and rotate them regularly: The best practice approach is to delete all untracked keys and replace them with freshly generated key pairs. Establish a streamlined process that allows specific authorized users to easily create and deploy keys through a simple, yet controlled workflow. Once new key pairs have been generated, they should be rotated regularly at pre-defined intervals to maintain compliance with internal or external policies and reduce the additional risk exposure that ‘stale’ keys create. Whether key rotation is triggered by the user via UI/API or automatically by the system (forced rotation), the backend process of provisioning new keys and removing the old keys on remote servers should be automated.
5. Control SSH keys and access: Now that you have deployed fresh SSH key pairs to target systems, it is important to define permissions for each key and control who has SSH-based access to which systems. This can only be achieved with a proper SSH key management tool, which allows you to centrally assign or revoke access to SSH hosts based on specific users and groups while orchestrating keys in the backend to facilitate those controls.
6. Continuously monitor: Unknown SSH keys pose a continuous threat to your organization. Achieving 100% control is not possible, but it is possible to stay ahead of threats by regularly conducting audits and continually monitoring your key inventory. It also helps to maintain an ongoing audit log of important events, such as key rotation, generation and provisioning.
Putting a stop to SSH key sprawl starts with a key management framework. By following these six steps, and adopting the right tools and processes, you can start to take back control of SSH keys and the trusted access they grant within your organization.
About the author: Chris Hickman is the chief security officer at Keyfactor, the leader in cloud-first PKI as-a-Service and crypto-agility solutions. Follow Chris on LinkedIn.
