}

AIX SSH Configuration: Complete Setup Guide [AIX 7.x]

9 min read

AIX SSH Configuration: Complete Setup Guide for AIX 7.x

Setting up and configuring SSH (Secure Shell) on IBM AIX is essential for secure remote administration. This comprehensive guide covers everything from initial SSH setup on AIX 7.x to advanced configuration, key-based authentication, PAM integration, and security hardening.

Table of Contents

  1. SSH on AIX Overview
  2. Enabling SSH on AIX
  3. SSH Configuration File
  4. Key-Based Authentication
  5. PAM Configuration
  6. Security Hardening
  7. Troubleshooting

SSH on AIX Overview

AIX uses OpenSSH, the same SSH implementation found on most Unix-like systems. However, AIX manages services through the System Resource Controller (SRC) rather than systemd or init scripts. Understanding this difference is key to properly managing SSH on AIX.

SSH Package Location

On AIX 7.x, OpenSSH is typically installed as part of the base operating system or as an installable fileset:

# Check if OpenSSH is installed
lslpp -l | grep -i ssh

Expected output:

openssh.base.client      9.0.0.0    COMMITTED  Open Secure Shell Commands
openssh.base.server      9.0.0.0    COMMITTED  Open Secure Shell Server
openssh.license          9.0.0.0    COMMITTED  Open Secure Shell License
openssh.man.en_US        9.0.0.0    COMMITTED  Open SSH Software License

Key SSH Files on AIX

File/Directory Purpose
/etc/ssh/sshd_config SSH daemon configuration
/etc/ssh/ssh_config SSH client configuration
/etc/ssh/ssh_host_* Host key files
/var/log/sshd.log SSH daemon log file
~/.ssh/ User SSH directory
~/.ssh/authorized_keys Authorized public keys
~/.ssh/known_hosts Known host fingerprints
/etc/pam.d/sshd PAM configuration for SSH

Enabling SSH Daemon on AIX

Method 1: Using startsrc (SRC Command)

The recommended way to start SSH on AIX is using the System Resource Controller:

# Start SSH daemon
startsrc -s sshd

# Verify it's running
lssrc -s sshd

Expected output:

Subsystem         Group            PID          Status
 sshd             ssh              2621442      active

Method 2: Direct Execution (Testing Only)

For testing or debugging, you can start sshd directly:

# Start in foreground with debug output
/usr/sbin/sshd -D -d

# Start normally
/usr/sbin/sshd

Creating the SSH Subsystem (If Not Exists)

If the sshd subsystem isn't registered with SRC:

# Check if subsystem exists
lssrc -s sshd

# If you get "0513-085 The sshd Subsystem is not on file", create it:
mkssys -p /usr/sbin/sshd \
       -s sshd \
       -u 0 \
       -a "-D" \
       -e /dev/console \
       -i /dev/console \
       -o /dev/console \
       -R -Q -S \
       -f 9 \
       -n 15 \
       -E 20 \
       -G ssh \
       -d \
       -w 20

Enable SSH at System Boot

Create /etc/rc.d/rc2.d/S70sshd:

#!/bin/ksh
case "$1" in
    start)
        /usr/bin/startsrc -s sshd
        ;;
    stop)
        /usr/bin/stopsrc -s sshd
        ;;
    *)
        echo "Usage: $0 {start|stop}"
        exit 1
        ;;
esac
exit 0

Set permissions:

chmod 755 /etc/rc.d/rc2.d/S70sshd
chown root:system /etc/rc.d/rc2.d/S70sshd

Alternatively, add to /etc/inittab:

mkitab "sshd:2:once:/usr/bin/startsrc -s sshd"

SSH Configuration File Settings

The main configuration file is /etc/ssh/sshd_config. Here's a comprehensive configuration for AIX 7.x:

Basic Configuration

# /etc/ssh/sshd_config - AIX 7.x Optimized Configuration

# Network Settings
Port 22
AddressFamily any
ListenAddress 0.0.0.0
#ListenAddress ::

# Protocol Settings
Protocol 2

# Host Keys
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_key

# Logging
SyslogFacility AUTH
LogLevel INFO

# Authentication
LoginGraceTime 60
PermitRootLogin yes
StrictModes yes
MaxAuthTries 3
MaxSessions 10

# Public Key Authentication
PubkeyAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys

# Password Authentication
PasswordAuthentication yes
PermitEmptyPasswords no

# Keyboard Interactive (for PAM)
KbdInteractiveAuthentication yes

# PAM Integration
UsePAM yes

# Other Authentication Methods
ChallengeResponseAuthentication yes
HostbasedAuthentication no
IgnoreRhosts yes

# Session Settings
X11Forwarding yes
X11DisplayOffset 10
X11UseLocalhost yes
PrintMotd yes
PrintLastLog yes
TCPKeepAlive yes
ClientAliveInterval 300
ClientAliveCountMax 3

# Environment
AcceptEnv LANG LC_*
PermitUserEnvironment no

# Subsystem for SFTP
Subsystem sftp /usr/sbin/sftp-server

# Banner (optional)
#Banner /etc/ssh/banner.txt

Applying Configuration Changes

After modifying sshd_config:

# Test configuration syntax
/usr/sbin/sshd -t

# If no errors, restart SSH
stopsrc -s sshd && startsrc -s sshd

# Or reload without disconnecting sessions
refresh -s sshd

Key-Based Authentication Setup

Key-based authentication is more secure than passwords and enables passwordless login for automation.

Step 1: Generate SSH Key Pair (Client Side)

On your local machine or another server:

# Generate RSA key (4096 bits recommended)
ssh-keygen -t rsa -b 4096 -C "[email protected]"

# Or generate Ed25519 key (modern, secure)
ssh-keygen -t ed25519 -C "[email protected]"

When prompted: - File location: Press Enter for default (~/.ssh/id_rsa or ~/.ssh/id_ed25519) - Passphrase: Enter a strong passphrase (recommended) or leave empty

Step 2: Copy Public Key to AIX Server

Method A: Using ssh-copy-id

ssh-copy-id -i ~/.ssh/id_rsa.pub user@aix-server

Method B: Manual Copy

# Display your public key
cat ~/.ssh/id_rsa.pub

# On the AIX server, as the target user:
mkdir -p ~/.ssh
chmod 700 ~/.ssh
echo "your-public-key-content" >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

Step 3: Configure SSH Server for Key Authentication

Ensure these settings in /etc/ssh/sshd_config:

PubkeyAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys

Step 4: Set Correct Permissions (Critical!)

Incorrect permissions are the #1 cause of key authentication failures on AIX:

# On the AIX server, as the user:
chmod 700 ~
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
chmod 600 ~/.ssh/id_rsa          # Private key (if exists)
chmod 644 ~/.ssh/id_rsa.pub      # Public key (if exists)

# Ownership must be correct
chown -R $USER:$GROUP ~/.ssh

Step 5: Test Key Authentication

# From client, connect with verbose output
ssh -v user@aix-server

# Look for these lines in output:
# debug1: Offering public key: /home/user/.ssh/id_rsa
# debug1: Server accepts key: /home/user/.ssh/id_rsa

Step 6: Disable Password Authentication (Optional)

Once key authentication works, disable passwords for enhanced security:

# In /etc/ssh/sshd_config
PasswordAuthentication no
ChallengeResponseAuthentication no
UsePAM no

Then restart SSH:

stopsrc -s sshd && startsrc -s sshd

Warning: Always keep a console session open when making these changes!

PAM Configuration for SSH

AIX uses PAM (Pluggable Authentication Modules) for authentication. Proper PAM configuration is essential for SSH to work correctly.

PAM Configuration File

The SSH PAM configuration is in /etc/pam.d/sshd:

# /etc/pam.d/sshd - AIX SSH PAM Configuration
#
# Authentication
auth       required     pam_aix.so
auth       required     pam_permission_group.so

# Account management
account    required     pam_aix.so
account    required     pam_permission_group.so

# Password management
password   required     pam_aix.so

# Session management
session    required     pam_aix.so

Common PAM Issues and Solutions

Issue: "Permission denied" even with correct password

Check /etc/security/user for account restrictions:

lsuser -a account_locked login loginretries user_name

Unlock if needed:

chuser account_locked=false user_name
chsec -f /etc/security/lastlog -a unsuccessful_login_count=0 -s user_name

Issue: SSH works but PAM errors in logs

Verify PAM modules exist:

ls -la /usr/lib/security/

Check PAM configuration syntax:

cat /etc/pam.d/sshd

AIX-Specific PAM Settings

In /etc/security/login.cfg:

# Stanza for SSH authentication
sshd:
    auth_type = STD_AUTH
    auth_domain = ALL

SSH Security Hardening

Recommended Security Settings

# /etc/ssh/sshd_config - Security Hardened

# Disable root login (use sudo instead)
PermitRootLogin no

# Limit authentication attempts
MaxAuthTries 3
MaxSessions 5
LoginGraceTime 30

# Use only strong ciphers
Ciphers [email protected],[email protected],aes256-ctr,aes192-ctr,aes128-ctr

# Use only strong MACs
MACs [email protected],[email protected],hmac-sha2-512,hmac-sha2-256

# Use only strong key exchange
KexAlgorithms curve25519-sha256,[email protected],ecdh-sha2-nistp521,ecdh-sha2-nistp384,ecdh-sha2-nistp256,diffie-hellman-group-exchange-sha256

# Disable empty passwords
PermitEmptyPasswords no

# Disable host-based authentication
HostbasedAuthentication no
IgnoreRhosts yes

# Disable unused authentication methods
GSSAPIAuthentication no
KerberosAuthentication no

# Session timeout
ClientAliveInterval 300
ClientAliveCountMax 2

# Logging
LogLevel VERBOSE

Restrict SSH Access by User/Group

# Allow only specific users
AllowUsers admin1 admin2 svc_backup

# Or allow by group
AllowGroups sshusers admins

# Deny specific users
DenyUsers guest test

Restrict SSH Access by IP

# In /etc/ssh/sshd_config
# Allow only from specific networks
Match Address 10.0.0.0/8,192.168.1.0/24
    PasswordAuthentication yes

Match Address *
    PasswordAuthentication no

Or use AIX firewall (ipsec):

# Allow SSH from specific IP
mkfilt -v 4 -a P -s 192.168.1.0 -m 255.255.255.0 -d 0.0.0.0 -M 0.0.0.0 -p 22 -O eq -P tcp -r B -w I -l N -f Y

SSH Banner Warning

Create /etc/ssh/banner.txt:

************************************************************
*                    AUTHORIZED ACCESS ONLY                 *
*                                                          *
* This system is the property of [Organization Name].       *
* Unauthorized access is prohibited and will be prosecuted. *
* All activities are monitored and logged.                  *
************************************************************

Enable in sshd_config:

Banner /etc/ssh/banner.txt

AIX SSH Troubleshooting

Debug Connection Issues

On the server (run sshd in debug mode):

# Stop the normal sshd
stopsrc -s sshd

# Start in debug mode on alternate port
/usr/sbin/sshd -d -p 2222

# Test connection from client
ssh -p 2222 -v user@aix-server

On the client:

# Verbose connection
ssh -v user@aix-server

# Very verbose
ssh -vvv user@aix-server

Common Problems and Solutions

Problem: "Permission denied (publickey)"

Causes and fixes:

  1. Wrong permissions:
# Fix on server
chmod 700 ~
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
chown -R $(whoami) ~/.ssh
  1. SELinux/security context (rare on AIX, but check):
ls -leZ ~/.ssh/authorized_keys
  1. Wrong key format:
# Ensure authorized_keys has one key per line
# Each line should start with: ssh-rsa, ssh-ed25519, ecdsa-sha2-*, etc.
  1. sshd_config issues:
grep -i pubkey /etc/ssh/sshd_config
# Should show: PubkeyAuthentication yes

Problem: "Connection refused"

# Check if sshd is running
lssrc -s sshd

# Check if listening on port 22
netstat -an | grep "\.22 "

# Check firewall
lsfilt -v 4 | grep 22

# Start sshd if not running
startsrc -s sshd

Problem: "Connection timed out"

# Check network connectivity
ping aix-server

# Check if port 22 is blocked
telnet aix-server 22

# Check routing
traceroute aix-server

# Check AIX firewall rules
lsfilt -v 4

Problem: "Host key verification failed"

# Remove old host key on client
ssh-keygen -R aix-server

# Or remove from known_hosts manually
vi ~/.ssh/known_hosts
# Delete the line for aix-server

Problem: SSH hangs during login

Usually DNS-related. Fix in sshd_config:

UseDNS no
GSSAPIAuthentication no

Problem: "Too many authentication failures"

# On client, specify the identity file
ssh -i ~/.ssh/id_rsa -o IdentitiesOnly=yes user@aix-server

# Or configure in ~/.ssh/config
Host aix-server
    IdentityFile ~/.ssh/id_rsa
    IdentitiesOnly yes

Check SSH Logs

# Primary SSH log
tail -100 /var/log/sshd.log

# System log
tail -100 /var/adm/messages | grep ssh

# Authentication log
cat /etc/security/failedlogin | who /etc/security/failedlogin

Regenerate Host Keys

If host keys are corrupted:

# Backup old keys
mkdir /etc/ssh/old_keys
mv /etc/ssh/ssh_host_* /etc/ssh/old_keys/

# Generate new keys
ssh-keygen -t rsa -b 4096 -f /etc/ssh/ssh_host_rsa_key -N ""
ssh-keygen -t ecdsa -b 521 -f /etc/ssh/ssh_host_ecdsa_key -N ""
ssh-keygen -t ed25519 -f /etc/ssh/ssh_host_ed25519_key -N ""

# Set permissions
chmod 600 /etc/ssh/ssh_host_*
chmod 644 /etc/ssh/ssh_host_*.pub

# Restart SSH
stopsrc -s sshd && startsrc -s sshd

Related Guides

Summary

Proper SSH configuration on AIX 7.x requires understanding both the SRC service management and standard OpenSSH configuration. Key points:

  1. Start SSH with startsrc -s sshd
  2. Configure in /etc/ssh/sshd_config
  3. Key authentication requires correct file permissions (700 for .ssh, 600 for authorized_keys)
  4. PAM is managed in /etc/pam.d/sshd and /etc/security/
  5. Debug with ssh -v on client and sshd -d on server
  6. Logs are in /var/log/sshd.log

Always test configuration changes before disconnecting from your current session!

AIX
SSH