# Never Touch Production Data Directly

Never suggest running commands, queries, or scripts directly against production systems.

## Why This is Critical

Direct production access can cause:
- **Data loss** - Accidental deletes/updates
- **Downtime** - Locked tables, crashed services
- **Corruption** - Inconsistent state
- **Security breaches** - Exposed sensitive data
- **Compliance violations** - Audit trail gaps
- **No rollback** - Changes are immediate and permanent

## NEVER Suggest These for Production

### Direct Database Modifications
```sql
-- NEVER run directly in production
DELETE FROM users WHERE active = false;
UPDATE orders SET status = 'cancelled';
TRUNCATE TABLE sessions;
DROP TABLE old_data;
ALTER TABLE users ADD COLUMN email_verified BOOLEAN;
```

### Mass Data Updates
```javascript
// NEVER run directly in production
User.updateMany({}, { verified: true });
db.collection('users').deleteMany({ lastLogin: { $lt: cutoffDate }});
```

### Schema Changes
```sql
-- NEVER without proper migration process
ALTER TABLE users DROP COLUMN password;
CREATE INDEX on huge_table(column);  -- Can lock table
RENAME TABLE users TO customers;
```

### Direct File System Changes
```bash
# NEVER on production servers
rm -rf /var/log/*
chmod 777 /app/config
mv /app/database.db /backup/
```

## What to Do Instead

### For Data Fixes

#### ❌ Bad
"Run this in production:
`UPDATE users SET email = 'fixed@example.com' WHERE id = 123`"

#### ✅ Good
"To fix this user's email:

**Option 1: Through Your Application**
Use the admin panel or API endpoint designed for this

**Option 2: Safe Script**
1. Write a migration script with:
   - Validation
   - Logging
   - Dry-run mode
   - Rollback capability
2. Test on staging database (full production copy)
3. Review with team
4. Schedule during maintenance window
5. Execute with monitoring
6. Verify results

**Script template:**
```javascript
// fix-user-email.js
const DRY_RUN = true;  // Set false only when ready

async function fixUserEmail(userId, newEmail) {
  // Validate
  const user = await User.findById(userId);
  if (!user) throw new Error('User not found');
  
  console.log(`Would update: ${user.email} -> ${newEmail}`);
  
  if (DRY_RUN) {
    console.log('DRY RUN - no changes made');
    return;
  }
  
  // Log for audit trail
  await AuditLog.create({
    action: 'email_fix',
    userId,
    oldValue: user.email,
    newValue: newEmail,
    timestamp: new Date()
  });
  
  // Make change
  user.email = newEmail;
  await user.save();
  
  console.log('✓ Updated successfully');
}
```"

### For Schema Changes

#### ❌ Bad
"Add this column in production:
`ALTER TABLE users ADD COLUMN avatar_url VARCHAR(255)`"

#### ✅ Good
"To add the avatar column safely:

**1. Create Migration File**
```javascript
// migrations/2024-01-15-add-avatar-url.js
exports.up = async (db) => {
  await db.schema.table('users', (table) => {
    table.string('avatar_url', 255).nullable();
  });
};

exports.down = async (db) => {
  await db.schema.table('users', (table) => {
    table.dropColumn('avatar_url');
  });
};
```

**2. Test Process:**
1. Run on local dev database
2. Run on staging (prod copy)
3. Verify application still works
4. Check performance impact
5. Review with team

**3. Production Deployment:**
1. Backup database first
2. Run during low-traffic window
3. Monitor during execution
4. Verify with test queries
5. Monitor application logs

**4. If Issues:**
- Have rollback plan ready
- Can run down() migration
- Restore from backup if needed"

### For Data Investigation

#### ❌ Bad
"Query production to check:
`SELECT * FROM users WHERE email LIKE '%test%'`"

#### ✅ Good
"To investigate user data:

**Option 1: Use Read Replica** (Safest)
- Query the read-only replica
- No risk to production
- May be slightly behind

**Option 2: Use Monitoring/Analytics Tools**
- Datadog, New Relic, etc.
- Purpose-built for production queries
- Built-in safety limits

**Option 3: Export to Staging**
- Take sanitized production snapshot
- Query staging database
- No production impact

Never query production directly unless:
- Emergency incident investigation
- Read-only query
- With appropriate monitoring
- With timeout limits set"

## Required Safety Measures

### Before Any Production Change

1. **Document the Change**
   - What and why
   - Expected impact
   - Rollback plan

2. **Test First**
   - Local environment
   - Staging (production copy)
   - Verify all edge cases

3. **Get Approval**
   - Code review
   - Team lead approval
   - For critical changes: multi-person sign-off

4. **Prepare Safety Nets**
   - Backup before change
   - Rollback script ready
   - Monitoring alerts configured

5. **Schedule Appropriately**
   - Low-traffic window
   - Team available to respond
   - Not before weekend/vacation

6. **Monitor During/After**
   - Watch error rates
   - Check performance metrics
   - Verify expected outcome

## Emergency Access Guidelines

### When Production Access is Necessary

For critical incidents only:
- System is down
- Data breach in progress
- Critical bug affecting users

### Even Then, Follow Rules:

1. **Get Authorization**
   - On-call manager approval
   - Document in incident report

2. **Use Read-Only First**
   - Investigate with SELECT only
   - Verify problem before fixing

3. **Log Everything**
   - What queries ran
   - What changed
   - Who authorized it

4. **Minimize Blast Radius**
   - Change one thing at a time
   - Test each change
   - Can you fix in code instead?

5. **Create Follow-up Tasks**
   - Proper fix via deployment
   - Prevent recurrence
   - Document what happened

## Safe Patterns

### Use Deployment Pipelines
```yaml
# Changes go through CI/CD
1. Code change → Pull Request
2. Automated tests pass
3. Staging deployment
4. Manual QA
5. Production deployment
6. Automated rollback if issues
```

### Use Feature Flags
```javascript
// Enable features gradually
if (featureFlags.newAvatarSystem) {
  // New code
} else {
  // Old code
}

// Roll out:
// - 1% of users
// - Monitor
// - 10%, 50%, 100%
// - Can rollback instantly
```

### Use Blue-Green Deployments
```
- Deploy to new environment
- Test new environment
- Switch traffic
- Keep old environment for instant rollback
```

## Warning Signs

Reject suggestions that include:

- "Just run this in production..."
- "Quick database fix..."
- "Direct SQL to update..."
- "SSH into prod and..."
- "This will only take a second..."
- "Shortcut: run this command..."

## The Right Approach

```markdown
**Wrong:** "Here's a SQL query to run in production"

**Right:** "Here's a safe migration script to:
1. Test locally
2. Test on staging
3. Review with team
4. Deploy through normal process
5. Monitor for issues"

**Wrong:** "SSH to production and update the config"

**Right:** "Update the config in git:
1. Create PR with change
2. Review
3. Deploy through CI/CD
4. Config auto-updates via deployment"

**Wrong:** "Delete those records from production DB"

**Right:** "Create a data cleanup script:
1. Soft delete first (reversible)
2. Test on staging
3. Review data to be deleted
4. Schedule cleanup job
5. Hard delete after confirmation period"
```

## Exceptions (Rare)

Only touch production directly when:
- Critical security breach (patch immediately)
- System completely down (emergency recovery)
- Data loss in progress (stop it now)

**Even then:**
- Get authorization
- Document everything
- Follow with proper fix
- Post-mortem review

## Summary

Production data is sacred:
- Never suggest direct changes
- Always use safe deployment processes
- Test everything in staging first
- Have rollback plans ready
- Document and monitor all changes

If it's important enough to change in production,  
it's important enough to do safely.