New Server Setup

Filesystem and Partitions

The server should be hardened to the CIS level 1 standard security profile. This requires that some partitions be in a separate volume group than /home.

Directory Recommended Size Used For
/boot/efi 512 mib For EFI systems. Not needed for BIOS
/boot 512 mib Boot partition. Required for EFI and BIOS
/var/log/audit 10 gib Logs from Selinux
/var/log 10 gib Logs from rsyslog and journald
/var 20 gib Docker containers, images, and volumes in /var/lib/docker
/var/tmp 5 gib Temporary storage and files
/tmp 5 gib Temporary storage and files
/ 20 gib Root filesystem
/home 50 gib Users home folders (Can be anywhere from 50GB to 1TB)
swap See below swap chart Swap partitions support virtual memory: data is written to a swap partition when there is not enough RAM to store the data your system is processing.

Swap Chart

Amount of RAM in System Recommended Swap space Recommended Swap space if allowing for hibernation
<2gb 2 times the amount of RAM 3 times the amount of RAM
>2gb - 8gb Equal to the amount of RAM 2 times the amount of RAM
>8gb - 64gb At least 4gb 1.5 times the amount of RAM
>64gb At least 4gb Hibernation not recommended

Configure Server

  1. Run initial updates:

    dnf update
  2. Turn on auto updates:

     dnf install dnf-automatic
     systemctl enable --now dnf-automatic.timer
     systemctl enable --now dnf-automatic-install.timer

acme.sh –issue -d example.com -w /home/wwwroot/example.com

  1. Update the hostname:

    vim /etc/hostname
  2. Configure the firewall to only listen on port 22/tcp (ssh), port 9090/tcp (web gui aka cockpit), and port 546/udp (dhcp6). This is easily done via the Web GUI.

  3. Install QEMU quest agents (If installing on Proxmox VE)

    dnf install qemu-guest-agent

User Accounts

  1. Install Fedora Server on the system with the first.last naming convention for usernames and a long/secure password for the root account that is stored in Bitwarden and shared with the IT group.

  2. Once the server is installed, you can access the web GUI at server_ip:9090

  3. Create user accounts. Only IT members should have sudo unless an appropriate business case is provided to grant sudo to general users. It is important that the username follows the naming convention first.last. For example, an account for Rex Linder would be created with the username rex.linder. Adding sudo users: useradd -m -N -G wheel -c "user.name@xentermd.com" user.name Adding non-sudo users: useradd -m -N -c "user.name@xentermd.com" user.name

  4. Generate passwords for the new users. Replace 20 with the length of the password you want.

    < /dev/urandom tr -dc [:graph:] 2>/dev/null | head -c20; echo;
  5. Set the password for the new user accounts and then distribute to corresponding users.

    passwd user.name

    Password Requirements:

    • 20 characters minimum
    • 32 characters max
    • One of each of the following: symbol, number, lower case, and upper case
    • Randomly generated using CLI or Bitwarden

Configure SSH

We use SSSD and Rippling to manage SSH access. Users can upload their public key to Rippling SSH Key Manager and it will push their key out to all of the servers.

  1. Install ssd and other dependencies

    sudo dnf install sssd sssd-tools openldap-clients
  2. Edit /etc/nsswitch.conf and add sss to the following entries. Leave the other existing entries intact.

    passwd:      compat sss
    shadow:      compat sss
    services:    db files sss
    netgroup:    nis sss
    sudoers:     files sss
  3. Now we’ll configure /etc/sssd/sssd.conf:

    sudo touch /etc/sssd/sssd.conf
    sudo chown root:root /etc/sssd/sssd.conf
    sudo chmod 600 /etc/sssd/sssd.conf
    sudo vim /etc/sssd/sssd.conf

    Note: Failing to update the permissions to 600 will result in sssd failing to start with the error:

    SSSD couldn't load the configuration database [1432158318]: Unknown error 1432158318.
  4. Here is a sample SSSD config file you can start with for sssd.conf. The SSSD documentation has more details about each of the sssd.conf configuration options. Note you’ll need to replace Group_Name in the following example with the name of an SSH group you have configured in Rippling.

    [sssd]
    config_file_version = 2
    services = nss, pam, ssh, sudo
    domains = ldap.rippling.com
    
    [nss]
    filter_groups = root
    filter_users = root
    reconnection_retries = 3
    
    [sudo]
    debug_level = 9
    
    [pam]
    debug_level = 9
    
    [domain/ldap.rippling.com]
    debug_level = 9
    enumerate = false
    cache_credentials = true
    min_id = 150
    lookup_family_order = ipv6_first
    id_provider = ldap
    auth_provider = ldap
    access_provider = simple
    chpass_provider = ldap
    ldap_schema = rfc2307
    ldap_user_search_base = ou=users,dc=xentermd,dc=rippling,dc=com
    ldap_user_search_filter = (member=cn=Group_Name,ou=groups,dc=xentermd,dc=rippling,dc=com)
    ldap_group_search_base = ou=groups,dc=xentermd,dc=rippling,dc=com
    ldap_sudo_search_base = ou=users,dc=xentermd,dc=rippling,dc=com
    ldap_user_object_class = user
    ldap_user_principal = userPrincipalName
    ldap_user_name = uid
    ldap_user_gecos = displayName
    ldap_group_object_class = group
    ldap_group_name = cn
    ldap_user_home_directory = unixHomeDirectory
    ldap_user_ssh_public_key = sshPublicKey
    ldap_uri = ldaps://ldap.rippling.com:636
    ldap_default_bind_dn = cn=sshadmin,ou=users,dc=xentermd,dc=rippling,dc=com
    ldap_default_authtok_type = obfuscated_password
    ldap_default_authtok = dummy_password // Will be replaced in the next step
  5. Sudo access to a group of users can be given by adding the following in the sssd.conf. Note that you’ll need to replace Sudo_Group_Name with the name of an SSH group you have configured in the Rippling Groups step for sudo access.

    ldap_sudo_search_base = ou=users,dc=xentermd,dc=rippling,dc=com?base?(member=cn=Sudo_Group_Name,ou=groups,dc=xentermd,dc=rippling,dc=com)
  6. Next we’ll obfuscate the bind dn password by running the command below. When prompted, enter the SSH service account password (the sshadmin rippling password in bitwarden). If you are not sure what this is, please contact your company admin or visit Settings tab to reset the password. This will replace the ldap_default_authtok entry in in /etc/sssd/sssd.conf.

    sudo sss_obfuscate -d ldap.rippling.com -f /etc/sssd/sssd.conf
  7. Clear SSS cache:

    rm -rf /var/lib/sss/db/*
  8. Enable the SSSD service to start at boot:

    systemctl enable --now sssd
  9. You can check that its working by running this command:

    id user.name
  10. Update pam permissions to allow auto generation of users .ssh folder:

    echo "session optional pam_mkhomedir.so" >> /etc/pam.d/common-session
  11. Configure SSH to use SSSD by editing /etc/ssh/sshd_config and adding the following lines:

    AuthorizedKeysCommand /usr/bin/sss_ssh_authorizedkeys
    AuthorizedKeysCommandUser root
  12. Ensure that the AuthorizedKeysFile is not commented out. This will be used when looking for ssh keys if there doesn’t exist a corresponding user in LDAP.

    AuthorizedKeysFile %h/.ssh/authorized_keys

    Then set PermitRootLogin to no.

    PermitRootLogin no

    Save the file.

  13. Restart the SSH and SSSD services

    systemctl restart sshd
    systemctl restart sssd
  14. Now, any user in the selected group can ssh into the server if they have their SSH public key uploaded into Rippling!

Configure 2FA for Cockpit (WebGUI)

  1. Selinux breaks Google Authenticator. You must use this work around so that the 2FA works while selinux is enabled.

  2. Install the dependencies:

    sudo dnf install -y google-authenticator
  3. Login to the user account you want to configure 2FA for:

    su user.name
  4. Run the google authenticator app to run the initial setup:

    google-authenticator
  5. Say yes to time-based tokens and updating the .google_authenticator file:

    Do you want authentication tokens to be time-based (y/n) y  
    Do you want me to update your "/home/user/.google_authenticator" file (y/n)? y
  6. Scan the QR code using the user’s authenticator and enter the 6 digit code to confirm.

  7. Say yes to the rest of the prompts.

  8. Create a new hidden folder named .ga in the user’s home directory and configure its permissions to 700.

    mkdir ~/.ga && chmod 700 ~/.ga
  9. Move the users .google_authenticator file inside the new .ga folder.

    mv ~/.google_authenticator ~/.ga/google_authenticator
  10. Update the permissions on the .google_authenticator file.

    chmod 0400 ~/.ga/google_authenticator
  11. Restore the selinux context for the new folder and authenticator file.

    restorecon -Rv ~/.ga
  12. Configure PAM to use the new location by appending the following to the end of /etc/pam.d/cockpit (all on one line).

    auth       required     pam_google_authenticator.so nullok secret=${HOME}/.ga/google_authenticator
    • Nullok: Allows users to login if they do not have a 2FA configured.
    • Secret: Changes the location used to locate the google_authenticator file.
    • pam_google_authenticator.so: The 2FA module configured with PAM for authentication.
  13. Restart cockpit and try to login. You should now be prompted for the 2FA code.

  14. If you need to regenerate the 2FA code, you can use this command:

    google-authenticator -s ~/.ga/google_authenticator

Configure 2FA for SSH

  1. First, Fedora 33 includes an extra conf file that will cause issues if not updated. Edit /etc/ssh/sshd_config.d/50-redhat.conf and comment out the following lines. These settings will be configured in the main sshd_config file instead.

    #ChallengeResponseAuthentication no
    #UsePAM yes
  2. Configure PAM to use 2FA for ssh. Update /etc/pam.d/sshd and comment out this line:

    #auth       substack     password-auth
  3. Add the following line beneath the one you just commented out. Then save and close the file.

    auth required pam_google_authenticator.so nullok secret=${HOME}/.ga/google_authenticator
  4. Edit /etc/ssh/sshd_config and append ChallengeResponseAuthentication to the bottom and set it to yes.

    ChallengeResponseAuthentication yes

    Then append the following line to the bottom of the file:

    AuthenticationMethods publickey,keyboard-interactive

    And then set PasswordAuthentication to no.

    PasswordAuthentication no

    And then enable pam:

    UsePAM yes
  5. Restart ssh:

    sudo systemctl restart sshd
  6. Once finished, you should be prompted for the 2FA code next time you login via ssh.