Module 1: Linux Fundamentals and Distribution Specifics

Learning Objectives

By the end of this module, you will be able to:

1. Understanding Linux Distributions: Ubuntu vs. Arch Linux

Package Management Systems

Ubuntu and Arch Linux represent two different philosophies in the Linux ecosystem, most notably in how they manage software packages.

Ubuntu's APT (Advanced Package Tool)

Ubuntu uses the Debian package management system with .deb packages and the APT toolset. Under the hood, APT maintains a local database of available packages and their dependencies in /var/lib/apt/. When you run apt update, it fetches the latest package metadata from repositories defined in /etc/apt/sources.list and /etc/apt/sources.list.d/.

# Updating package information
sudo apt update

# Installing software
sudo apt install nginx

APT resolves dependencies automatically, downloading and installing all required packages. The actual package files are temporarily stored in /var/cache/apt/archives/ during installation.

Arch Linux's Pacman

Arch uses the Pacman package manager with .pkg.tar.zst compressed packages. Pacman's database resides in /var/lib/pacman/ and maintains a simpler structure than APT.

# Updating package information and system
sudo pacman -Syu

# Installing software
sudo pacman -S nginx

Pacman works by comparing the versions in its local database against the repositories defined in /etc/pacman.conf and /etc/pacman.d/mirrorlist. It's designed for minimal overhead and maximum simplicity, aligning with Arch's philosophy.

Release Cycles and Philosophy

Ubuntu: Predictable and Stable

Ubuntu follows a time-based release schedule:

Ubuntu's philosophy prioritizes user-friendliness and stability. Canonical (Ubuntu's parent company) extensively tests packages before inclusion in repositories. This makes Ubuntu well-suited for production environments where predictability is valued over cutting-edge features.

Arch Linux: Rolling Release and Cutting Edge

Arch follows a rolling release model with no fixed release dates. Instead, packages are updated continuously as they become available upstream. This means:

Arch's philosophy is encapsulated in the "Arch Way" principles:

This philosophy extends to Arch's minimalist base installation, which provides only what's necessary for a functional system, leaving further customization to the user.

2. How ACME's Infrastructure Leverages Linux

ACME, an open-source business applications suite, relies heavily on Linux for its infrastructure needs. Understanding how ACME uses Linux provides valuable context for system administrators.

Server Deployment

ACME typically runs on Linux servers because of:

Under the hood, ACME deployments on Linux utilize several key components:

  1. PostgreSQL Database: ACME stores all business data in PostgreSQL, which runs as a service (postgresql.service) managed by systemd.
  2. Python Runtime: ACME is primarily written in Python. Linux provides the necessary Python interpreter and libraries, often installed in /usr/lib/python3/ or within virtual environments.
  3. Web Server: Production ACME deployments typically use Nginx or Apache as reverse proxies, handling client connections and forwarding requests to the ACME application server.
  4. Process Management: Systemd service units (/etc/systemd/system/acme.service) ensure ACME starts automatically and restarts if crashed.

A typical ACME service file in /etc/systemd/system/acme.service might look like:

[Unit]
Description=ACME Open Source ERP and CRM
After=network.target postgresql.service

[Service]
Type=simple
User=acme
Group=acme
ExecStart=/opt/acme/acme-bin --config=/etc/acme/acme.conf
Restart=always

[Install]
WantedBy=multi-user.target

Resource Management

Linux provides several mechanisms ACME leverages for efficient operation:

When troubleshooting ACME performance, Linux tools like htop, iotop, and pg_top provide valuable insights into system resource utilization.

3. Linux File System Hierarchy and Directory Structure

Core Directories and Their Purposes

/etc - System Configuration

The /etc directory contains system-wide configuration files. These are plain text files that control the behavior of applications and services.

Under the hood, applications typically look for their configuration files in /etc/[application-name]/. For example, Nginx's configuration resides in /etc/nginx/. When troubleshooting service issues, /etc is often the first place to check.

Key files in /etc:

/var - Variable Data

The /var directory contains files that are expected to change during normal system operation, including:

Under the hood, services write to /var/log/ using the system logging daemon (like rsyslog or journald). Log files use rotation mechanisms defined in /etc/logrotate.d/ to prevent them from growing indefinitely.

/opt - Optional Software

The /opt directory is used for installing self-contained, add-on application software packages. Unlike packages managed by the system's package manager (which spread files across /usr/bin, /etc, etc.), software in /opt keeps all its files in one directory tree.

For example, a typical ACME installation might have:

This self-contained approach makes it easier to manage software that isn't part of the default distribution.

/home - User Directories

The /home directory contains user home directories where personal files, configurations, and data are stored. Each user typically has a directory named after their username (/home/username/).

Under the hood, when a user logs in, their shell sets the $HOME environment variable to point to their home directory. Configuration files in the home directory often start with a dot (like .bashrc) to make them hidden by default.

/usr - User Programs

Despite its name, /usr doesn't contain user files but rather user programs and data. It's where most of the system's executable files, libraries, documentation, and source code are located.

Key subdirectories:

Understanding the distinction between /bin (essential commands needed before /usr is mounted) and /usr/bin (non-essential user commands) helps explain Linux's boot process design.

/boot - Boot Files

The /boot directory contains files needed for system startup, including:

These files are kept in a separate directory because they need to be in a specific partition that the bootloader can read before the operating system starts.

4. Essential Bash Commands and Navigation

Navigation and File Management

Directory Navigation

# Print working directory
pwd

# List files and directories
ls -la

# Change directory
cd /etc

# Navigate to home directory
cd ~

# Go up one level
cd ..

File Operations

# Create empty file
touch file.txt

# Create directory
mkdir new_directory

# Copy file
cp source.txt destination.txt

# Move/rename file
mv old.txt new.txt

# Remove file
rm file.txt

# Remove directory and contents (use with caution!)
rm -rf directory/

File Viewing and Editing

# View entire file
cat file.txt

# View file with paging
less file.txt

# View first 10 lines
head -n 10 file.txt

# View last 10 lines
tail -n 10 file.txt

# Monitor file for changes
tail -f /var/log/syslog

Text Editing

# Edit with Nano (easier for beginners)
nano file.txt

# Edit with Vim (more powerful, steeper learning curve)
vim file.txt

System Information and Process Management

# Display system information
uname -a

# Show disk usage
df -h

# Show directory size
du -sh /var/log

# Display free and used memory
free -m

Process Management

# List running processes
ps aux

# Interactive process viewer
htop

# Kill process by ID
kill 1234

# Kill process by name
pkill firefox

# Run command in background
command &

# Bring background job to foreground
fg

5. Terminal Usage and Command Line Fundamentals

Understanding the Shell Environment

# Display all environment variables
env

# Set variable for current session
export VAR_NAME="value"

# Display value of variable
echo $VAR_NAME

# Add directory to PATH
export PATH=$PATH:/new/directory

Command History and Shortcuts

# View command history
history

# Search history (Ctrl+R)
# Press Ctrl+R and type part of command

# Repeat last command
!!

# Run command from history by number
!42

# Edit command in editor (set by EDITOR variable)
fc

Input/Output Redirection and Pipes

# Redirect output to file (overwrite)
command > output.txt

# Redirect output to file (append)
command >> output.txt

# Redirect error output
command 2> error.txt

# Redirect both output and errors
command > output.txt 2>&1

# Use command output as input for another command
command1 | command2

# Save output to file and display it
command | tee output.txt

Command Chaining and Scripting Basics

# Run second command only if first succeeds
command1 && command2

# Run second command only if first fails
command1 || command2

# Run commands in sequence regardless of outcome
command1 ; command2

# Group commands
(cd /tmp && ls) | grep "file"

A simple Bash script looks like:

#!/bin/bash
# Sample script to backup a directory

SOURCE_DIR="/var/www"
BACKUP_DIR="/backup"
DATE=$(date +%Y%m%d)

# Create backup filename
BACKUP_FILE="${BACKUP_DIR}/backup-${DATE}.tar.gz"

# Create backup
tar -czf "$BACKUP_FILE" "$SOURCE_DIR"

# Check if backup was successful
if [ $? -eq 0 ]; then
    echo "Backup created successfully: $BACKUP_FILE"
else
    echo "Backup failed!"
    exit 1
fi

6. Practical Exercises

Exercise 1: Setting Up Virtual Machines

Objective: Create virtual machines running Ubuntu and Arch Linux to compare their installation and configuration processes.

Requirements:

Steps for Ubuntu VM:

  1. Create a new virtual machine with at least 2GB RAM and 20GB disk space
  2. Mount the Ubuntu Server ISO and start the VM
  3. Follow the installer prompts, selecting default options for simplicity
  4. After installation, log in and update the system: 
    sudo apt update
sudo apt upgrade
  5. Install a basic web server: 
    sudo apt install nginx
sudo systemctl status nginx
  6. Open a web browser on your host machine and navigate to the VM's IP address to verify Nginx is running

Steps for Arch Linux VM:

  1. Create a new virtual machine with at least 2GB RAM and 20GB disk space
  2. Mount the Arch Linux ISO and start the VM
  3. Follow the installation guide: 
    # Connect to the internet
ping -c 3 archlinux.org
# Update the system clock
timedatectl set-ntp true
# Partition the disk (example for single partition)
fdisk /dev/sda
# Create new partition table (g), new partition (n), write (w)
# Format the partition
mkfs.ext4 /dev/sda1
# Mount the file system
mount /dev/sda1 /mnt
# Install essential packages
pacstrap /mnt base linux linux-firmware
# Generate fstab
genfstab -U /mnt >> /mnt/etc/fstab
# Chroot into the new system
arch-chroot /mnt
# Set time zone
ln -sf /usr/share/zoneinfo/Region/City /etc/localtime
hwclock --systohc
# Install additional needed packages
pacman -S nano dhcpcd networkmanager
# Enable network service
systemctl enable NetworkManager
# Set root password
passwd
# Install and configure bootloader
pacman -S grub
grub-install /dev/sda
grub-mkconfig -o /boot/grub/grub.cfg
# Exit chroot, unmount, and reboot
exit
umount -R /mnt
reboot
  4. After installation, log in as root and update the system: 
    pacman -Syu
  5. Install a basic web server: 
    pacman -S nginx
systemctl enable --now nginx

Comparison Task: Document the differences you observe between the two installation processes and initial system configurations.

Exercise 2: Navigating the Linux File System

Objective: Become familiar with the Linux directory structure and practice file operations.

Tasks:

  1. Create a directory structure for a hypothetical web application: 
    # Create project directories
mkdir -p ~/webapp/{public_html,logs,backups,config}

# Create sample files
touch ~/webapp/public_html/index.html
touch ~/webapp/config/app.conf
  2. Practice file operations: 
    # Copy files
cp ~/webapp/config/app.conf ~/webapp/config/app.conf.backup

# Create symbolic links
ln -s ~/webapp/public_html ~/public_html

# Find files
find ~/webapp -type f -name "*.conf"

# Count files in directories
find ~/webapp -type f | wc -l
  3. Examine system directories: 
    # List installed services
ls -la /etc/systemd/system/

# View system logs
sudo tail -n 20 /var/log/syslog

# Examine mounted filesystems
cat /etc/fstab

# Look at network configuration
cat /etc/network/interfaces   # Ubuntu
ls -la /etc/NetworkManager/   # Arch
  4. Monitor system processes: 
    # View running processes
ps aux | grep nginx

# Monitor system resources
top   # or htop if installed

Questions to Answer:

Exercise 3: Practical Bash Scripting

Objective: Create a simple Bash script for system administration tasks.

  1. Create a system health check script: 
    nano ~/system_health.sh
  2. Add the following content: 
    #!/bin/bash
# System Health Check Script

echo "System Health Check - $(date)"
echo "--------------------------"

# Check disk space
echo "Disk Space Usage:"
df -h | grep -v "tmpfs" | grep -v "udev"
echo

# Check memory usage
echo "Memory Usage:"
free -m
echo

# Check load average
echo "System Load:"
uptime
echo

# Check for failed systemd services
echo "Failed Services:"
systemctl --failed
echo

# Check for high CPU processes
echo "Top 5 CPU-consuming processes:"
ps aux --sort=-%cpu | head -6
echo

# Check for recent errors in system log
echo "Recent system errors:"
sudo journalctl -p err --since "1 hour ago" | tail -10
  3. Make the script executable: 
    chmod +x ~/system_health.sh
  4. Run the script: 
    sudo ./system_health.sh
  5. Modify the script to save output to a file: 
    # Add to the top of the script
OUTPUT_FILE="/tmp/health_check_$(date +%Y%m%d_%H%M%S).log"

# Add to the end of the script
echo "Health check completed. Report saved to $OUTPUT_FILE"
  6. Extension Task: Schedule the script to run daily using cron: 
    # Edit crontab
crontab -e

# Add line to run at 8 AM daily
0 8 * * * /home/username/system_health.sh > /tmp/daily_health.log 2>&1

7. Common Pitfalls and Troubleshooting

File System Permissions

New Linux administrators often struggle with permissions. Remember:

Common permission-related errors:

-bash: ./script.sh: Permission denied

Solution: Add execute permission with chmod +x script.sh

ls: cannot access '/dir': Permission denied

Solution: Check the entire path for appropriate permissions with namei -l /dir

Package Management Issues

Ubuntu/Debian:

Broken dependencies:

sudo apt --fix-broken install

Locked dpkg database:

# Find and kill the process
ps aux | grep apt
sudo kill <process_id>

# Or remove the lock file (if no apt process is running)
sudo rm /var/lib/dpkg/lock
sudo rm /var/lib/apt/lists/lock

Arch Linux:

Package conflicts:

sudo pacman -Syu --overwrite path/to/file

Corrupted package database:

sudo rm /var/lib/pacman/db.lck
sudo pacman-key --init
sudo pacman-key --populate archlinux

Systemd Service Management

Many Linux distributions now use systemd for service management. Common issues:

Service won't start:

# View detailed status
systemctl status service_name

# Check logs
journalctl -u service_name

Configuration errors:

# After editing service file
systemctl daemon-reload
systemctl restart service_name

Network Connectivity

When troubleshooting network issues:

  1. Check physical/virtual connection: 
    ip link show
  2. Verify IP configuration: 
    ip addr show
  3. Test DNS resolution: 
    nslookup google.com
  4. Check routing: 
    ip route show
  5. Test connectivity: 
    ping -c 4 8.8.8.8
traceroute google.com

Resource Exhaustion

Systems may become unresponsive due to:

Disk space:

# Find large files
find / -type f -size +100M -exec ls -lh {} \; 2>/dev/null

# Clean package caches
apt clean        # Ubuntu
pacman -Sc       # Arch

Memory issues:

# Find memory-hungry processes
ps aux --sort=-%mem | head

CPU bottlenecks:

# Find CPU-intensive processes
top -o %CPU

8. Quick Reference Summary

Essential Commands

Category Command Description
Navigation pwd Print working directory
ls -la List all files with details
cd directory Change directory
File Operations cp source dest Copy files or directories
mv old new Move or rename files
rm file Remove files
mkdir dir Create directory
File Content cat file Display file contents
less file View file with paging
grep pattern file Search for pattern in file
System Info uname -a System information
df -h Disk usage
free -m Memory usage
top or htop Process viewer
Package Management apt update && apt upgrade Update Ubuntu/Debian
pacman -Syu Update Arch Linux

Key System Directories

File Permissions

chmod permissions file

# Examples:
chmod 755 file  # rwxr-xr-x
chmod 644 file  # rw-r--r--
chmod +x file   # Add execute permission

Understanding numeric permissions:

Service Management

# Start service
systemctl start service_name

# Stop service
systemctl stop service_name

# Restart service
systemctl restart service_name

# Enable at boot
systemctl enable service_name

# Check status
systemctl status service_name

This module has covered the foundational aspects of Linux that system administrators need to understand when working with systems like ACME. By practicing the exercises and exploring the file system, you'll develop the skills needed to efficiently manage Linux servers in production environments.