backup-retention-policy

Backup Retention Policy Expert

Expert in data lifecycle management and failure recovery.

3-2-1-1-0 Rule

• 3 copies of important data (1 primary + 2 backup)
• 2 different types of media
• 1 external/cloud backup
• 1 offline/immutable backup
• 0 errors after testing

Storage tiers

Tier	Access	Period	Cost
Hot	Frequent	0-30 days	$$$
Warm	Periodic	30-90 days	$$
Cold	Rare	90 days-7 years	$
Archive	Long-term	7+ years	¢

AWS S3 Lifecycle Policy

{
  "Rules": [
    {
      "ID": "BackupLifecycle",
      "Status": "Enabled",
      "Filter": {
        "Prefix": "backups/"
      },
      "Transitions": [
        {
          "Days": 30,
          "StorageClass": "STANDARD_IA"
        },
        {
          "Days": 90,
          "StorageClass": "GLACIER"
        },
        {
          "Days": 365,
          "StorageClass": "DEEP_ARCHIVE"
        }
      ],
      "Expiration": {
        "Days": 2555
      },
      "NoncurrentVersionTransitions": [
        {
          "NoncurrentDays": 30,
          "StorageClass": "GLACIER"
        }
      ],
      "NoncurrentVersionExpiration": {
        "NoncurrentDays": 365
      }
    }
  ]
}

Azure Blob Lifecycle

{
  "rules": [
    {
      "name": "backupRetention",
      "type": "Lifecycle",
      "definition": {
        "actions": {
          "baseBlob": {
            "tierToCool": {"daysAfterModificationGreaterThan": 30},
            "tierToArchive": {"daysAfterModificationGreaterThan": 90},
            "delete": {"daysAfterModificationGreaterThan": 2555}
          },
          "snapshot": {
            "delete": {"daysAfterCreationGreaterThan": 90}
          }
        },
        "filters": {
          "blobTypes": ["blockBlob"],
          "prefixMatch": ["backups/"]
        }
      }
    }
  ]
}

Backup rotation script

#!/bin/bash
BACKUP_DIR="/var/backups"
DAILY_RETENTION=7
WEEKLY_RETENTION=4
MONTHLY_RETENTION=12

# Удаление ежедневных старше 7 дней
find "$BACKUP_DIR/daily" -type f -mtime +$DAILY_RETENTION -delete

# Удаление еженедельных старше 4 недель
find "$BACKUP_DIR/weekly" -type f -mtime +$((WEEKLY_RETENTION * 7)) -delete

# Удаление ежемесячных старше 12 месяцев
find "$BACKUP_DIR/monthly" -type f -mtime +$((MONTHLY_RETENTION * 30)) -delete

# Логирование
echo "$(date): Rotation completed" >> /var/log/backup_rotation.log

PostgreSQL Backup

#!/bin/bash
DB_NAME="production"
BACKUP_DIR="/var/backups/postgres"
DATE=$(date +%Y%m%d_%H%M%S)

# Полный бэкап
pg_dump -Fc -f "$BACKUP_DIR/full_$DATE.dump" $DB_NAME

# Инкрементальный с WAL
pg_basebackup -D "$BACKUP_DIR/base_$DATE" -Ft -z -P

# Архивация WAL логов
archive_command = 'cp %p /var/backups/wal/%f'

MySQL Backup

#!/bin/bash
MYSQL_USER="backup_user"
MYSQL_PASS="secure_password"
BACKUP_DIR="/var/backups/mysql"
DATE=$(date +%Y%m%d_%H%M%S)

# Полный бэкап
mysqldump --user=$MYSQL_USER --password=$MYSQL_PASS \
    --all-databases --single-transaction \
    --routines --triggers --events \
    | gzip > "$BACKUP_DIR/full_$DATE.sql.gz"

# Инкрементальный с binlog
mysqlbinlog --read-from-remote-server \
    --host=localhost --user=$MYSQL_USER \
    --raw --stop-never mysql-bin.000001

Compliance Requirements

Standard	Requirement
GDPR	Right to erasure, data minimization
SOX	7 years for financial records
HIPAA	6 years for medical data
PCI DSS	1 year minimum for audit

RTO/RPO Planning

Критические системы:
  RPO: 1 час
  RTO: 4 часа
  Стратегия: Синхронная репликация + горячий standby

Бизнес-системы:
  RPO: 4 часа
  RTO: 24 часа
  Стратегия: Асинхронная репликация + теплый standby

Архивные системы:
  RPO: 24 часа
  RTO: 72 часа
  Стратегия: Ежедневные бэкапы + холодное хранение

Policy validation in Python

from datetime import datetime, timedelta
from typing import List, Dict

class BackupRetentionValidator:
    def __init__(self, retention_policy: Dict):
        self.policy = retention_policy

    def validate_backup(self, backup_date: datetime, backup_type: str) -> bool:
        """Проверка соответствия бэкапа политике хранения"""
        retention_days = self.policy.get(backup_type, {}).get('retention_days', 0)
        expiry_date = backup_date + timedelta(days=retention_days)
        return datetime.now() < expiry_date

    def get_expired_backups(self, backups: List[Dict]) -> List[Dict]:
        """Получить список просроченных бэкапов"""
        expired = []
        for backup in backups:
            if not self.validate_backup(backup['date'], backup['type']):
                expired.append(backup)
        return expired

    def calculate_storage_forecast(self, daily_backup_size_gb: float) -> Dict:
        """Прогноз использования хранилища"""
        total_storage = 0
        for backup_type, config in self.policy.items():
            retention = config.get('retention_days', 0)
            frequency = config.get('frequency_days', 1)
            copies = retention // frequency
            total_storage += copies * daily_backup_size_gb

        return {
            'total_gb': total_storage,
            'monthly_cost_estimate': total_storage * 0.023  # S3 Standard pricing
        }

Testing schedule

Test type	Frequency	Scope
Selective recovery	Monthly	10% of data
Full recovery	Quarterly	Critical systems
DR exercise	Annually	Full scenario
Policy audit	Annually	All policies

Monitoring and alerts

Алерты:
  backup_failed:
    severity: critical
    notification: [pagerduty, slack]

  backup_size_anomaly:
    threshold: 20%
    severity: warning

  retention_violation:
    severity: high
    action: auto_remediate

  storage_threshold:
    threshold: 80%
    severity: warning

Best practices

1. Test recovery — a backup without testing = no backup
2. Encrypt data — AES-256 for data at rest
3. Versioning — keep multiple versions
4. Geographic distribution — at least 2 regions
5. Immutable storage — protection against ransomware
6. Document procedures — runbooks for DR