Here is a comprehensive, blog-style set of notes on Shell Scripting.
These notes cover the fundamentals, syntax, and logic control required to write effective automation scripts in Linux/Unix environments.
The Ultimate Guide to Shell Scripting
Shell scripting is the art of chaining together Linux commands into a reusable file to automate repetitive tasks. It is the backbone of DevOps, System Administration, and backend automation.
1. What is a Shell?
Before scripting, we must understand the Shell. The Shell is a program that takes commands from the keyboard and gives them to the operating system to perform.
The Kernel: The core of the OS that talks to the hardware (CPU, RAM).
The Shell: The interface between You and the Kernel.
Bash (Bourne Again SHell): The most common shell used in Linux and macOS.
2. Anatomy of a Script
Every shell script follows a specific structure.
The Shebang (#!)
The very first line of your script tells the system which interpreter to use to run the code.
#!/bin/bash
Enter fullscreen mode
Exit fullscreen mode
#!: The “Shebang” (Sharp-Bang).
/bin/bash: The path to the interpreter. You might also see #!/bin/sh or #!/usr/bin/python.
Permissions
By default, a new file is just text. You must make it executable.
chmod +x myscript.sh ./myscript.sh
Enter fullscreen mode
Exit fullscreen mode
3. Variables
Variables are containers for storing data. In Bash, they are untyped (everything is treated as a string usually).
Defining and Using Variables
No spaces around the = sign.
Use $ to access the value.
#!/bin/bash # Defining variables NAME="John" AGE=25 # Using variables echo "Hello, my name is $NAME and I am $AGE years old."
Enter fullscreen mode
Exit fullscreen mode
Special Variables (Arguments)
These are reserved for handling inputs passed to the script (e.g., ./script.sh input1 input2).
Variable
Description
$0
The name of the script itself.
$1
The first argument passed to the script.
$2
The second argument passed.
$#
The total number of arguments provided.
$@
All arguments passed (as a list).
$?
The exit status of the last command (0 = Success, Non-zero = Failure).
4. User Input
How to make your script interactive.
#!/bin/bash echo "What is your website?" read WEBSITE echo "Pinging $WEBSITE now..." ping -c 1 $WEBSITE
Enter fullscreen mode
Exit fullscreen mode
5. Conditionals (If/Else)
Logic allows your script to make decisions. Pay close attention to the spacing inside the brackets [ ... ].
Syntax
#!/bin/bash echo "Enter your age:" read AGE if [ $AGE -ge 18 ]; then echo "You are an adult." elif [ $AGE -eq 17 ]; then echo "Almost there." else echo "You are a minor." fi
#!/bin/bash # Loop through a list of names NAMES="Alice Bob Charlie" for PERSON in $NAMES; do echo "Hello $PERSON" done # Loop through a range of numbers for i in {1..5}; do echo "Count: $i" done
Enter fullscreen mode
Exit fullscreen mode
The While Loop
Runs as long as a condition is true.
#!/bin/bash COUNT=1 while [ $COUNT -le 5 ]; do echo "Line $COUNT" ((COUNT++)) # Increment the counter done
Enter fullscreen mode
Exit fullscreen mode
7. Functions
Functions allow you to write code once and reuse it.
#!/bin/bash # Define the function check_status() { if ping -c 1 google.com > /dev/null; then echo "Internet is UP" else echo "Internet is DOWN" fi } # Call the function check_status
Enter fullscreen mode
Exit fullscreen mode
8. Best Practices for Professional Scripts
Always use comments (#): Explain why you are doing something, not just what you are doing.
Exit on Error: Add set -e at the top of your script. This stops the script immediately if any command fails, preventing a snowball effect of errors.
Use meaningful variable names: Use FILENAME instead of F.
Quote your variables: Use "$VAR" instead of $VAR to prevent bugs if the variable contains spaces.
Summary Table
Concept
Command / Syntax
Purpose
Shebang
#!/bin/bash
Defines the interpreter.
Execution
chmod +x script.sh
Makes the file runnable.
Output
echo "Text"
Prints to the screen.
Input
read VAR_NAME
Takes input from user.
Variables
VAR="Value" / $VAR
Stores and retrieves data.
Condition
if [ condition ]; then
Branching logic.
Loop
for i in list; do
Repeating tasks.
To move from a “beginner” to an “intermediate” scripter, you need to master three key concepts: Functions, Arrays, and Case Statements.
These tools allow you to write scripts that are modular, organized, and capable of handling complex lists of data.
1. Functions (Don’t Repeat Yourself)
If you find yourself copy-pasting the same code block twice, you should turn it into a Function. Functions make your script cleaner and easier to debug.
Syntax:
function_name() { # Code goes here echo "Argument 1 is $1" }
Enter fullscreen mode
Exit fullscreen mode
Example: A flexible Logger Instead of typing echo "$(date) ..." every time, create a function.
#!/bin/bash # Define the function log_msg() { local LEVEL=$1 local MESSAGE=$2 TIMESTAMP=$(date +"%Y-%m-%d %H:%M:%S") echo "[$TIMESTAMP] [$LEVEL] $MESSAGE" } # Call the function log_msg "INFO" "Starting the script..." log_msg "ERROR" "Database connection failed!"
Enter fullscreen mode
Exit fullscreen mode
Note: local variables only exist inside the function, keeping your global variables safe.
2. Arrays (Handling Lists)
In DevOps, you often need to loop through a list of servers, packages, or users. Instead of creating $SERVER1, $SERVER2, etc., use an Array.
Syntax:
Create:MY_LIST=("item1" "item2" "item3")
Access All:${MY_LIST[@]}
Access One:${MY_LIST[0]} (Index starts at 0)
Count Items:${#MY_LIST[@]}
Example: Installing Multiple Packages
#!/bin/bash # Define a list of packages to install PACKAGES=("git" "curl" "nginx" "htop") echo "We need to install ${#PACKAGES[@]} packages." # Loop through the array for PKG in "${PACKAGES[@]}"; do echo "Installing $PKG..." sudo apt-get install -y "$PKG" done
Enter fullscreen mode
Exit fullscreen mode
3. Case Statements (The “Menu” Logic)
If you have a script that needs to handle many different options (like start, stop, restart, status), using 10 different if/else statements is messy. Use case instead.
Example: A Service Manager Script Run this script like: ./manage_service.sh start
When a complex script fails, it can be hard to see why. You can turn on “Debug Mode” to print every command before it executes.
Option A: Add -x to the shebang line: #!/bin/bash -x
Option B: Run the script with bash: bash -x myscript.sh
Output Example:
+ ACTION=start + case "$ACTION" in + echo '🟢 Starting application...' 🟢 Starting application...
Enter fullscreen mode
Exit fullscreen mode
The lines starting with + show you exactly what Bash is doing.
5. The “Master Script”: Putting it all together
Here is a professional-grade script that combines Functions, Arrays, Checks, and Logging. This is the level of scripting expected in a DevOps interview.
Scenario: A “System Provisioner” script that sets up a new server.
#!/bin/bash set -e # Exit on error # --- Configuration --- LOG_FILE="/var/log/provision.log" PACKAGES=("vim" "git" "nginx" "jq") # --- Functions --- log() { echo "[$(date +'%T')] $1" | tee -a "$LOG_FILE" } check_root() { if [ "$EUID" -ne 0 ]; then log "❌ Error: Please run as root." exit 1 fi } install_packages() { log "📦 Updating package repositories..." apt-get update -y > /dev/null 2>&1 for PKG in "${PACKAGES[@]}"; do log " -> Installing $PKG..." apt-get install -y "$PKG" > /dev/null 2>&1 done } configure_firewall() { log "shield: Configuring Firewall..." # Check if ufw is installed first if command -v ufw > /dev/null; then ufw allow 22/tcp ufw allow 80/tcp ufw --force enable else log "⚠️ Warning: UFW not found, skipping firewall." fi } # --- Main Execution Flow --- check_root log "🚀 Starting System Provisioning..." install_packages configure_firewall log "✅ Provisioning Complete!"
Enter fullscreen mode
Exit fullscreen mode
Here are 4 real-world shell scripts used frequently in actual DevOps projects. These cover Monitoring, Cleanup, Backups, and CI/CD Automation.
In a real job, these usually run as Cron Jobs (scheduled tasks) or steps in a Jenkins/GitHub Actions pipeline.
1. The “Disk Space Alerter” (Monitoring)
Scenario: Servers often crash because logs fill up the disk. You need a script that runs every hour, checks disk usage, and sends a specific Slack notification if it crosses a dangerous threshold (e.g., 80%).
Key Concepts:df, awk, curl (for API calls).
#!/bin/bash # Configuration THRESHOLD=80 SLACK_WEBHOOK_URL="https://hooks.slack.com/services/T000/B000/XXXX" HOSTNAME=$(hostname) # Get current disk usage percentage (stripping the % sign) # df -h / gives usage of root partition. # awk 'NR==2 {print $5}' gets the percentage column from the second line. # sed 's/%//g' removes the percentage sign so we can do math. CURRENT_USAGE=$(df -h / | awk 'NR==2 {print $5}' | sed 's/%//g') if [ "$CURRENT_USAGE" -gt "$THRESHOLD" ]; then echo "⚠️ Disk space critical: ${CURRENT_USAGE}% detected." # Send Alert to Slack MESSAGE="🚨 *CRITICAL ALERT* \nServer: $HOSTNAME \nDisk Usage: ${CURRENT_USAGE}% \nPlease clean up immediately!" curl -X POST -H 'Content-type: application/json' \ --data "{\"text\": \"$MESSAGE\"}" \ "$SLACK_WEBHOOK_URL" else echo "✅ Disk space normal: ${CURRENT_USAGE}%" fi
Enter fullscreen mode
Exit fullscreen mode
2. The “Docker Garbage Collector” (Maintenance)
Scenario: CI/CD runners (like Jenkins agents) build thousands of Docker images. Eventually, the disk fills up with unused “dangling” images. This script cleans them up safely.
Key Concepts:docker, if/else, Exit Codes.
#!/bin/bash echo "🐳 Starting Docker Cleanup..." # 1. Prune stopped containers (older than 24h) to prevent deleting active work # -f means force (don't ask for confirmation) # --filter "until=24h" creates a safety buffer docker container prune -f --filter "until=24h" # 2. Prune unused images (dangling) docker image prune -f # 3. Prune unused volumes (be careful with this in production!) docker volume prune -f # Check how much space is left FREE_SPACE=$(df -h / | awk 'NR==2 {print $4}') echo "✅ Cleanup Complete. Free Space: $FREE_SPACE"
Enter fullscreen mode
Exit fullscreen mode
3. The “Log Rotator & S3 Uploader” (Cloud Ops)
Scenario: You need to keep application logs for legal reasons (Compliance), but you can’t keep them on the server forever because they are expensive. This script compresses yesterday’s logs and pushes them to AWS S3 (Cheap storage).
Key Concepts:tar (compression), date math, aws cli.
#!/bin/bash set -e # Exit if any command fails # Variables LOG_DIR="/var/log/myapp" ARCHIVE_DIR="/tmp/log_archives" S3_BUCKET="s3://my-company-logs-backup" YESTERDAY=$(date -d "yesterday" +'%Y-%m-%d') ARCHIVE_NAME="logs-$YESTERDAY.tar.gz" mkdir -p $ARCHIVE_DIR echo "📦 Compressing logs for $YESTERDAY..." # Find logs from yesterday and compress them # We assume logs are named like 'app-2024-01-01.log' tar -czf "$ARCHIVE_DIR/$ARCHIVE_NAME" -C "$LOG_DIR" . echo "☁️ Uploading to AWS S3..." aws s3 cp "$ARCHIVE_DIR/$ARCHIVE_NAME" "$S3_BUCKET/$ARCHIVE_NAME" if [ $? -eq 0 ]; then echo "✅ Upload Successful. Deleting local archive..." rm "$ARCHIVE_DIR/$ARCHIVE_NAME" # Optional: Delete the original log files from the server to save space # find "$LOG_DIR" -type f -name "*$YESTERDAY*" -delete else echo "❌ Upload Failed!" exit 1 fi
Enter fullscreen mode
Exit fullscreen mode
4. The “Semantic Version Tag Generator” (CI/CD)
Scenario: In a CI pipeline (GitLab CI or Jenkins), you want to automatically generate a new version number (e.g., v1.2.5) every time a build passes, based on the previous tag.
Key Concepts: String manipulation, git.
#!/bin/bash # Get the latest tag (e.g., v1.2.4). If no tag, default to v1.0.0 LATEST_TAG=$(git describe --tags `git rev-list --tags --max-count=1` 2>/dev/null || echo "v1.0.0") echo "Current Version: $LATEST_TAG" # Split the string by "." # v1.2.4 -> MAJOR=v1, MINOR=2, PATCH=4 MAJOR=$(echo $LATEST_TAG | awk -F. '{print $1}') MINOR=$(echo $LATEST_TAG | awk -F. '{print $2}') PATCH=$(echo $LATEST_TAG | awk -F. '{print $3}') # Increment the Patch version (v1.2.4 -> v1.2.5) NEW_PATCH=$((PATCH + 1)) NEW_TAG="$MAJOR.$MINOR.$NEW_PATCH" echo "🚀 New Version Detected: $NEW_TAG" # Apply the tag (In a real pipeline, you would push this back to git) # git tag $NEW_TAG # git push origin $NEW_TAG
Enter fullscreen mode
Exit fullscreen mode
How to use these in an Interview?
If asked, “What shell scripts have you written?”, pick Script #1 (Monitoring) or Script #3 (Backup).
Say this:
“I wrote a script to automate our log retention policy. It ran as a cron job every night, compressed the application logs, and uploaded them to an S3 bucket for long-term storage, which saved us about 20% on EBS disk costs.”
For further actions, you may consider blocking this person and/or reporting abuse
Залишити відповідь