The AWS Command Line Interface (CLI) is a powerful tool that allows you to interact with AWS services directly from your terminal. Whether you’re managing S3 buckets, deploying applications, or automating cloud operations, the AWS CLI is an essential tool for any developer or DevOps engineer working with AWS.
In this comprehensive guide, we’ll walk through everything you need to know to get started with AWS CLI, from installation to performing real-world operations like backing up databases to S3.
What is AWS CLI?
AWS CLI is a unified tool that enables you to manage AWS services from the command line. It provides direct access to public APIs of AWS services and supports scripting to automate your workflows. With AWS CLI, you can:
- Manage AWS services programmatically
- Automate repetitive tasks
- Integrate AWS operations into scripts and CI/CD pipelines
- Access all AWS services from a single interface
Key Benefits
- Unified Interface: One tool to manage all AWS services
- Scriptable: Perfect for automation and CI/CD pipelines
- Cross-Platform: Works on Linux, macOS, and Windows
- Powerful: Access to all AWS service APIs
- Free: No additional cost beyond AWS service usage
Prerequisites
Before we begin, make sure you have:
- An AWS account (you can create one at aws.amazon.com)
- Basic knowledge of command-line interfaces
- Understanding of your operating system’s terminal/command prompt
- An IAM user with appropriate permissions (we’ll cover this)
Step 1: Installing AWS CLI
AWS CLI installation varies by operating system. Let’s cover the most common platforms.
Installing on Linux
For Linux systems, AWS CLI v2 is the recommended version. Here’s how to install it:
Method 1: Using the Installer (Recommended)
# Download the AWS CLI installer
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
# Install unzip if not already installed
sudo apt-get update && sudo apt-get install -y unzip # For Debian/Ubuntu
# OR
sudo yum install -y unzip # For RHEL/CentOS/Amazon Linux
# Unzip the installer
unzip awscliv2.zip
# Run the installer
sudo ./aws/install
# Verify the installation
aws --version
The installer places AWS CLI in /usr/local/aws-cli and creates a symbolic link in /usr/local/bin.
Method 2: Using Package Managers
For Ubuntu/Debian:
sudo apt-get update
sudo apt-get install -y awscli
For Amazon Linux:
sudo yum install -y aws-cli
For Arch Linux:
sudo pacman -S aws-cli
Installing on macOS
Method 1: Using the Standalone Installer
# Download the installer
curl "https://awscli.amazonaws.com/AWSCLIV2.pkg" -o "AWSCLIV2.pkg"
# Run the installer
sudo installer -pkg AWSCLIV2.pkg -target /
# Verify the installation
aws --version
Method 2: Using Homebrew (Recommended)
# Install using Homebrew
brew install awscli
# Verify the installation
aws --version
Installing on Windows
Method 1: Using the MSI Installer
-
Download the AWS CLI MSI installer for Windows (64-bit) from: https://awscli.amazonaws.com/AWSCLIV2.msi
-
Run the downloaded
.msifile and follow the on-screen instructions -
Open Command Prompt or PowerShell and verify:
aws --version
Method 2: Using PowerShell
# Download the installer
Invoke-WebRequest -Uri "https://awscli.amazonaws.com/AWSCLIV2.msi" -OutFile "AWSCLIV2.msi"
# Run the installer
Start-Process msiexec.exe -ArgumentList '/i AWSCLIV2.msi /quiet' -Wait
# Verify the installation
aws --version
Verifying Your Installation
After installation, verify that AWS CLI is working correctly:
aws --version
You should see output similar to:
aws-cli/2.x.x Python/3.x.x Linux/x86_64
Step 2: Configuring AWS CLI
Once AWS CLI is installed, you need to configure it with your AWS credentials. There are several ways to do this, but we’ll start with the most common method.
Basic Configuration
Run the configuration command:
aws configure
You’ll be prompted for four pieces of information:
- AWS Access Key ID: Your IAM user’s access key
- AWS Secret Access Key: Your IAM user’s secret key
- Default region name: Your preferred AWS region (e.g.,
us-east-1,us-west-2,eu-west-1) - Default output format: Choose
json,yaml,yaml-stream,text, ortable
Example interaction:
$ aws configure
AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE
AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
Default region name [None]: us-east-1
Default output format [None]: json
Creating IAM User and Access Keys
If you don’t have an IAM user yet, here’s how to create one:
-
Log in to AWS Console
- Go to console.aws.amazon.com
- Sign in with your AWS account
-
Navigate to IAM
- Search for “IAM” in the services search bar
- Click on “IAM” service
-
Create a New User
- Click “Users” in the left sidebar
- Click “Add users” or “Create user”
- Enter a username (e.g.,
aws-cli-user) - Select “Provide user access to the AWS Management Console” if needed, or skip for CLI-only access
- Click “Next”
-
Set Permissions
- For now, you can attach the “AdministratorAccess” policy for full access (we’ll cover best practices later)
- Or create a custom policy with specific permissions
- Click “Next”
-
Create Access Keys
- After creating the user, click on the username
- Go to the “Security credentials” tab
- Scroll to “Access keys” section
- Click “Create access key”
- Select “Command Line Interface (CLI)” as the use case
- Click “Next” and then “Create access key”
- Important: Download or copy the Access Key ID and Secret Access Key immediately - you won’t be able to see the secret key again!
Configuration File Locations
AWS CLI stores configuration in two files:
- Credentials:
~/.aws/credentials(Linux/macOS) or%USERPROFILE%\.aws\credentials(Windows) - Config:
~/.aws/config(Linux/macOS) or%USERPROFILE%\.aws\config(Windows)
You can also manually edit these files:
~/.aws/credentials:
[default]
aws_access_key_id = AKIAIOSFODNN7EXAMPLE
aws_secret_access_key = wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
~/.aws/config:
[default]
region = us-east-1
output = json
Using Multiple Profiles
You can configure multiple AWS profiles for different accounts or roles:
# Configure a named profile
aws configure --profile production
# Use a specific profile
aws s3 ls --profile production
# Set a profile as default
export AWS_PROFILE=production # Linux/macOS
# OR
set AWS_PROFILE=production # Windows
Step 3: Setting Up IAM Permissions
Security is crucial when working with AWS. Let’s set up proper IAM permissions following best practices.
Principle of Least Privilege
Only grant the minimum permissions necessary for the task. Instead of using AdministratorAccess, create specific policies.
Creating a Custom IAM Policy for S3 Operations
Let’s create a policy that allows S3 bucket operations:
-
Navigate to IAM → Policies
- Click “Create policy”
- Click the “JSON” tab
-
Add Policy JSON
For basic S3 operations (create bucket, upload, download):
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:CreateBucket",
"s3:DeleteBucket",
"s3:ListBucket",
"s3:GetBucketLocation",
"s3:GetObject",
"s3:PutObject",
"s3:DeleteObject",
"s3:GetObjectVersion",
"s3:PutObjectAcl"
],
"Resource": [
"arn:aws:s3:::your-bucket-name",
"arn:aws:s3:::your-bucket-name/*"
]
},
{
"Effect": "Allow",
"Action": [
"s3:ListAllMyBuckets"
],
"Resource": "*"
}
]
}
For a more flexible policy that works with any bucket:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:CreateBucket",
"s3:ListAllMyBuckets",
"s3:GetBucketLocation"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetBucketLocation"
],
"Resource": "arn:aws:s3:::*"
},
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:DeleteObject",
"s3:GetObjectVersion",
"s3:PutObjectAcl"
],
"Resource": "arn:aws:s3:::*/*"
}
]
}
-
Name and Create the Policy
- Name:
S3BucketOperations - Description: “Allows S3 bucket creation and file operations”
- Click “Create policy”
- Name:
-
Attach Policy to User
- Go to IAM → Users
- Click on your user
- Click “Add permissions” → “Attach policies directly”
- Search for and select your policy
- Click “Add permissions”
Security Best Practices
-
Never Commit Credentials
# ❌ Bad - Never do this git add ~/.aws/credentials # ✅ Good - Add to .gitignore echo ".aws/" >> .gitignore -
Use IAM Roles Instead of Access Keys When Possible
- For EC2 instances, use IAM roles
- For CI/CD pipelines, use OIDC or IAM roles
-
Rotate Access Keys Regularly
- Set a reminder to rotate keys every 90 days
- Create new keys before deleting old ones
-
Enable MFA for Sensitive Operations
# Configure MFA for your profile aws configure set mfa_serial arn:aws:iam::ACCOUNT_ID:mfa/USERNAME -
Use Temporary Credentials
# Assume a role with temporary credentials aws sts assume-role \ --role-arn arn:aws:iam::ACCOUNT_ID:role/ROLE_NAME \ --role-session-name SESSION_NAME
Step 4: Basic AWS CLI Commands
Let’s start with some fundamental commands to verify everything is working:
Testing Your Configuration
# List all S3 buckets (tests basic connectivity)
aws s3 ls
# Get your AWS account ID
aws sts get-caller-identity
# List available regions
aws ec2 describe-regions --output table
Common Command Structure
AWS CLI commands follow this pattern:
aws <service> <operation> [options]
Examples:
aws s3 ls # List S3 buckets
aws ec2 describe-instances # Describe EC2 instances
aws iam list-users # List IAM users
Step 5: Working with S3 Buckets
S3 (Simple Storage Service) is one of the most commonly used AWS services. Let’s learn how to manage S3 buckets with AWS CLI.
Creating an S3 Bucket
S3 bucket names must be globally unique across all AWS accounts. Let’s create one:
# Create a bucket in a specific region
aws s3 mb s3://my-unique-bucket-name-2026 --region us-east-1
# Create a bucket with default region
aws s3 mb s3://my-unique-bucket-name-2026
Important Notes:
- Bucket names must be 3-63 characters long
- Can contain only lowercase letters, numbers, dots, and hyphens
- Must start and end with a letter or number
- Must be globally unique
Listing Buckets
# List all your buckets
aws s3 ls
# List contents of a specific bucket
aws s3 ls s3://my-unique-bucket-name-2026
# List with details (size, date)
aws s3 ls s3://my-unique-bucket-name-2026 --human-readable --summarize
Uploading Files to S3
Upload a Single File
# Upload a file
aws s3 cp /path/to/local/file.txt s3://my-unique-bucket-name-2026/
# Upload with a specific name
aws s3 cp /path/to/local/file.txt s3://my-unique-bucket-name-2026/remote-file.txt
# Upload with metadata
aws s3 cp /path/to/local/file.txt s3://my-unique-bucket-name-2026/ \
--metadata "author=John,project=MyProject"
Upload a Directory (Recursive)
# Upload entire directory
aws s3 cp /path/to/local/directory s3://my-unique-bucket-name-2026/ --recursive
# Upload with sync (only changed files)
aws s3 sync /path/to/local/directory s3://my-unique-bucket-name-2026/
# Exclude certain files
aws s3 sync /path/to/local/directory s3://my-unique-bucket-name-2026/ \
--exclude "*.log" --exclude "*.tmp"
Upload with Specific Options
# Upload with server-side encryption
aws s3 cp file.txt s3://my-unique-bucket-name-2026/ \
--server-side-encryption AES256
# Upload with specific storage class (for cost optimization)
aws s3 cp file.txt s3://my-unique-bucket-name-2026/ \
--storage-class STANDARD_IA # Infrequent Access
# Upload with ACL (Access Control List)
aws s3 cp file.txt s3://my-unique-bucket-name-2026/ \
--acl public-read
Downloading Files from S3
Download a Single File
# Download a file
aws s3 cp s3://my-unique-bucket-name-2026/file.txt /path/to/local/
# Download with a specific name
aws s3 cp s3://my-unique-bucket-name-2026/file.txt /path/to/local/new-name.txt
Download a Directory (Recursive)
# Download entire directory
aws s3 cp s3://my-unique-bucket-name-2026/directory/ /path/to/local/ --recursive
# Sync (download only changed files)
aws s3 sync s3://my-unique-bucket-name-2026/directory/ /path/to/local/
# Download with exclude patterns
aws s3 sync s3://my-unique-bucket-name-2026/ /path/to/local/ \
--exclude "*.log" --exclude "backup/*"
Updating Files in S3
Updating files in S3 is simply a matter of uploading a new version:
# Upload updated file (overwrites existing)
aws s3 cp /path/to/updated-file.txt s3://my-unique-bucket-name-2026/
# Sync updated files
aws s3 sync /path/to/local/directory s3://my-unique-bucket-name-2026/ \
--delete # Also deletes files in S3 that don't exist locally
Deleting Files and Buckets
# Delete a file
aws s3 rm s3://my-unique-bucket-name-2026/file.txt
# Delete a directory
aws s3 rm s3://my-unique-bucket-name-2026/directory/ --recursive
# Delete entire bucket (must be empty first)
aws s3 rb s3://my-unique-bucket-name-2026
# Force delete bucket with all contents
aws s3 rb s3://my-unique-bucket-name-2026 --force
Step 6: Practical Example: Database Backup to S3
Let’s create a complete example of backing up a database to S3. This is a common use case for production systems.
Example 1: MySQL Database Backup
Step 1: Create Backup Script
Create a file called backup-mysql.sh:
#!/bin/bash
# Configuration
DB_HOST="localhost"
DB_USER="your_username"
DB_PASS="your_password"
DB_NAME="your_database"
S3_BUCKET="my-database-backups"
BACKUP_DIR="/tmp/backups"
DATE=$(date +%Y%m%d_%H%M%S)
BACKUP_FILE="${DB_NAME}_${DATE}.sql"
# Create backup directory if it doesn't exist
mkdir -p $BACKUP_DIR
# Create database backup
echo "Creating backup of $DB_NAME..."
mysqldump -h $DB_HOST -u $DB_USER -p$DB_PASS $DB_NAME > $BACKUP_DIR/$BACKUP_FILE
# Compress the backup
echo "Compressing backup..."
gzip $BACKUP_DIR/$BACKUP_FILE
BACKUP_FILE="${BACKUP_FILE}.gz"
# Upload to S3
echo "Uploading to S3..."
aws s3 cp $BACKUP_DIR/$BACKUP_FILE s3://$S3_BUCKET/databases/$BACKUP_FILE
# Verify upload
if [ $? -eq 0 ]; then
echo "Backup uploaded successfully!"
# Delete local backup to save space
rm $BACKUP_DIR/$BACKUP_FILE
else
echo "Upload failed! Keeping local backup."
fi
# Clean up old backups (keep last 7 days locally)
find $BACKUP_DIR -name "*.sql.gz" -mtime +7 -delete
Step 2: Make Script Executable
chmod +x backup-mysql.sh
Step 3: Run the Backup
./backup-mysql.sh
Step 4: Restore from Backup
Create a restore script restore-mysql.sh:
#!/bin/bash
# Configuration
DB_HOST="localhost"
DB_USER="your_username"
DB_PASS="your_password"
DB_NAME="your_database"
S3_BUCKET="my-database-backups"
BACKUP_FILE=$1 # Pass backup filename as argument
BACKUP_DIR="/tmp/restores"
# Create restore directory
mkdir -p $BACKUP_DIR
# Download from S3
echo "Downloading backup from S3..."
aws s3 cp s3://$S3_BUCKET/databases/$BACKUP_FILE $BACKUP_DIR/$BACKUP_FILE
# Decompress
echo "Decompressing backup..."
gunzip $BACKUP_DIR/$BACKUP_FILE
RESTORE_FILE="${BACKUP_FILE%.gz}"
# Restore database
echo "Restoring database..."
mysql -h $DB_HOST -u $DB_USER -p$DB_PASS $DB_NAME < $BACKUP_DIR/$RESTORE_FILE
# Clean up
rm $BACKUP_DIR/$RESTORE_FILE
echo "Restore completed!"
Usage:
./restore-mysql.sh your_database_20260111_120000.sql.gz
Example 2: PostgreSQL Database Backup
Create backup-postgres.sh:
#!/bin/bash
# Configuration
DB_HOST="localhost"
DB_USER="your_username"
DB_NAME="your_database"
S3_BUCKET="my-database-backups"
BACKUP_DIR="/tmp/backups"
DATE=$(date +%Y%m%d_%H%M%S)
BACKUP_FILE="${DB_NAME}_${DATE}.dump"
# Create backup directory
mkdir -p $BACKUP_DIR
# Create database backup
echo "Creating backup of $DB_NAME..."
PGPASSWORD=your_password pg_dump -h $DB_HOST -U $DB_USER -d $DB_NAME -F c -f $BACKUP_DIR/$BACKUP_FILE
# Upload to S3
echo "Uploading to S3..."
aws s3 cp $BACKUP_DIR/$BACKUP_FILE s3://$S3_BUCKET/databases/$BACKUP_FILE
# Verify and clean up
if [ $? -eq 0 ]; then
echo "Backup uploaded successfully!"
rm $BACKUP_DIR/$BACKUP_FILE
else
echo "Upload failed! Keeping local backup."
fi
Example 3: Automated Daily Backups with Cron
Set up automatic daily backups:
# Edit crontab
crontab -e
# Add this line for daily backup at 2 AM
0 2 * * * /path/to/backup-mysql.sh >> /var/log/db-backup.log 2>&1
Example 4: Backup with Retention Policy
Enhanced backup script with S3 lifecycle management:
#!/bin/bash
# Configuration
DB_NAME="your_database"
S3_BUCKET="my-database-backups"
BACKUP_DIR="/tmp/backups"
DATE=$(date +%Y%m%d_%H%M%S)
RETENTION_DAYS=30
# ... (backup creation code) ...
# Upload to S3 with date-based path
S3_PATH="databases/$(date +%Y)/$(date +%m)/${BACKUP_FILE}"
aws s3 cp $BACKUP_DIR/$BACKUP_FILE s3://$S3_BUCKET/$S3_PATH
# Set expiration tag (S3 lifecycle policy handles deletion)
aws s3api put-object-tagging \
--bucket $S3_BUCKET \
--key $S3_PATH \
--tagging "TagSet=[{Key=ExpiresAfter,Value=$RETENTION_DAYS}]"
Step 7: Advanced S3 Operations
Setting Bucket Policies
# Create a bucket policy file (bucket-policy.json)
cat > bucket-policy.json <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "PublicReadGetObject",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::my-bucket/*"
}
]
}
EOF
# Apply the policy
aws s3api put-bucket-policy --bucket my-bucket --policy file://bucket-policy.json
Enabling Versioning
# Enable versioning on a bucket
aws s3api put-bucket-versioning \
--bucket my-unique-bucket-name-2026 \
--versioning-configuration Status=Enabled
Setting Up Lifecycle Rules
Create a lifecycle configuration file:
{
"Rules": [
{
"Id": "DeleteOldBackups",
"Status": "Enabled",
"Prefix": "backups/",
"Expiration": {
"Days": 30
},
"Transitions": [
{
"Days": 7,
"StorageClass": "STANDARD_IA"
},
{
"Days": 30,
"StorageClass": "GLACIER"
}
]
}
]
}
Apply it:
aws s3api put-bucket-lifecycle-configuration \
--bucket my-unique-bucket-name-2026 \
--lifecycle-configuration file://lifecycle.json
Monitoring S3 Operations
# Get bucket size and object count
aws s3 ls s3://my-unique-bucket-name-2026 --recursive --human-readable --summarize
# List objects with details
aws s3api list-objects-v2 \
--bucket my-unique-bucket-name-2026 \
--query 'Contents[*].[Key,Size,LastModified]' \
--output table
Step 8: Troubleshooting Common Issues
Issue 1: “Unable to locate credentials”
Solution:
# Verify credentials are configured
aws configure list
# Reconfigure if needed
aws configure
Issue 2: “Access Denied” Errors
Solution:
- Check IAM permissions
- Verify bucket policies
- Ensure your access key has proper permissions
# Test your permissions
aws s3 ls # Should list buckets
aws sts get-caller-identity # Shows your identity
Issue 3: “Bucket name already exists”
Solution: S3 bucket names are globally unique. Choose a different name:
aws s3 mb s3://my-unique-bucket-name-$(date +%s)
Issue 4: Slow Upload/Download Speeds
Solution:
- Use
--regionto specify the correct region - Use multipart upload for large files (automatic for files > 64MB)
- Consider using
aws s3 syncinstead ofcpfor directories
# Use specific region
aws s3 cp large-file.zip s3://my-bucket/ --region us-east-1
# Increase multipart threshold
aws configure set default.s3.multipart_threshold 64MB
aws configure set default.s3.multipart_chunksize 16MB
Issue 5: “InvalidAccessKeyId” or “SignatureDoesNotMatch”
Solution:
- Verify your access key and secret key are correct
- Check system clock is synchronized
- Regenerate access keys if needed
Step 9: Best Practices
1. Use IAM Roles Instead of Access Keys When Possible
For EC2 instances:
# No credentials needed - uses instance role
aws s3 ls
2. Use Named Profiles for Different Environments
aws configure --profile production
aws configure --profile development
# Use specific profile
aws s3 ls --profile production
3. Enable MFA for Sensitive Operations
aws configure set mfa_serial arn:aws:iam::ACCOUNT_ID:mfa/USERNAME
4. Use S3 Sync for Efficient Transfers
# Only transfers changed files
aws s3 sync /local/directory s3://my-bucket/directory/
5. Implement Proper Error Handling in Scripts
#!/bin/bash
set -e # Exit on error
aws s3 cp file.txt s3://my-bucket/ || {
echo "Upload failed!"
exit 1
}
6. Use S3 Lifecycle Policies for Cost Optimization
Automatically move old files to cheaper storage classes or delete them.
7. Enable Versioning for Important Data
Protect against accidental deletion or modification.
8. Use Server-Side Encryption
aws s3 cp file.txt s3://my-bucket/ --server-side-encryption AES256
Step 10: Integration with CI/CD Pipelines
GitHub Actions Example
name: Deploy to S3
on:
push:
branches: [ main ]
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-east-1
- name: Build application
run: npm run build
- name: Deploy to S3
run: aws s3 sync dist/ s3://my-bucket/ --delete
GitLab CI Example
deploy:
stage: deploy
script:
- apt-get update && apt-get install -y awscli
- aws configure set aws_access_key_id $AWS_ACCESS_KEY_ID
- aws configure set aws_secret_access_key $AWS_SECRET_ACCESS_KEY
- aws configure set default.region us-east-1
- aws s3 sync dist/ s3://my-bucket/ --delete
only:
- main
Conclusion
You’ve now learned how to:
- Install AWS CLI on Linux, macOS, and Windows
- Configure AWS CLI with credentials
- Set up proper IAM permissions following security best practices
- Create and manage S3 buckets
- Upload, download, and update files in S3
- Create automated database backup scripts
- Troubleshoot common issues
- Integrate AWS CLI into CI/CD pipelines
AWS CLI is a powerful tool that becomes essential as you work more with AWS services. The skills you’ve learned here will help you automate cloud operations, manage infrastructure, and build robust backup solutions.
Next Steps
- Explore other AWS services via CLI (EC2, RDS, Lambda, etc.)
- Set up automated backup schedules
- Implement S3 lifecycle policies for cost optimization
- Learn about AWS CLI profiles for multi-account management
- Explore AWS CLI autocomplete for faster command entry
Resources
Happy automating! 🚀