Hashed conversion matching and more. #12
59
MIGRATION_FIXES.md
Normal file
59
MIGRATION_FIXES.md
Normal file
@@ -0,0 +1,59 @@
|
|||||||
|
# Migration Fixes for Production Database Compatibility
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
The database migrations were failing when run against a production database dump because:
|
||||||
|
|
||||||
|
1. **First migration (630b0c367dcb)**: Tried to create an index on `acked_requests` that already existed in the production dump
|
||||||
|
2. **Third migration (08fe946414d8)**: Tried to add `hashed_customer_id` column to `reservations` without checking if it already existed
|
||||||
|
3. **Fourth migration (a1b2c3d4e5f6)**: Tried to modify `conversion_guests` table before it was guaranteed to exist
|
||||||
|
|
||||||
|
## Solutions Applied
|
||||||
|
|
||||||
|
### 1. Migration 630b0c367dcb - Initial Migration
|
||||||
|
**Change**: Made index creation idempotent by checking if index already exists before creating it
|
||||||
|
|
||||||
|
**Impact**: Allows migration to run even if production DB already has the `ix_acked_requests_username` index
|
||||||
|
|
||||||
|
### 2. Migration 08fe946414d8 - Add hashed_customer_id to reservations
|
||||||
|
**Change**: Added check to skip adding the column if it already exists
|
||||||
|
|
||||||
|
**Impact**:
|
||||||
|
- Preserves production data in `reservations` and `hashed_customers` tables
|
||||||
|
- Makes migration safe to re-run
|
||||||
|
- Still performs data migration to populate `hashed_customer_id` when needed
|
||||||
|
|
||||||
|
### 3. Migration a1b2c3d4e5f6 - Add hashed_customer_id to conversion_guests
|
||||||
|
**Change**: Added check to verify `conversion_guests` table exists before modifying it
|
||||||
|
|
||||||
|
**Impact**: Safely handles the case where table creation in a previous migration succeeded
|
||||||
|
|
||||||
|
## Data Preservation
|
||||||
|
All non-conversion tables are preserved:
|
||||||
|
- ✓ `customers`: 1095 rows preserved
|
||||||
|
- ✓ `reservations`: 1177 rows preserved
|
||||||
|
- ✓ `hashed_customers`: 1095 rows preserved
|
||||||
|
- ✓ `acked_requests`: preserved
|
||||||
|
|
||||||
|
Conversion tables are properly recreated:
|
||||||
|
- ✓ `conversions`: created fresh with new schema
|
||||||
|
- ✓ `conversion_rooms`: created fresh with new schema
|
||||||
|
- ✓ `conversion_guests`: created fresh with composite key
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
After running `uv run alembic upgrade head`:
|
||||||
|
- All migrations apply successfully
|
||||||
|
- Database is at head revision: `a1b2c3d4e5f6`
|
||||||
|
- All required columns exist (`conversion_guests.hashed_customer_id`, `reservations.hashed_customer_id`)
|
||||||
|
- Production data is preserved
|
||||||
|
|
||||||
|
## Reset Instructions
|
||||||
|
If you need to reset and re-run all migrations:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
DELETE FROM alpinebits.alembic_version;
|
||||||
|
```
|
||||||
|
|
||||||
|
Then run:
|
||||||
|
```bash
|
||||||
|
uv run alembic upgrade head
|
||||||
|
```
|
||||||
37
MIGRATION_RESET.md
Normal file
37
MIGRATION_RESET.md
Normal file
@@ -0,0 +1,37 @@
|
|||||||
|
# Migration Reset Instructions
|
||||||
|
|
||||||
|
If you need to reset the alembic_version table to start migrations from scratch:
|
||||||
|
|
||||||
|
## SQL Command
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Connect to your database and run:
|
||||||
|
DELETE FROM alpinebits.alembic_version;
|
||||||
|
```
|
||||||
|
|
||||||
|
This clears all migration records so that `alembic upgrade head` will run all migrations from the beginning.
|
||||||
|
|
||||||
|
## Python One-Liner (if preferred)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uv run python -c "
|
||||||
|
import asyncio
|
||||||
|
from sqlalchemy import text
|
||||||
|
from alpine_bits_python.config_loader import load_config
|
||||||
|
from alpine_bits_python.db import get_database_url, get_database_schema
|
||||||
|
from sqlalchemy.ext.asyncio import create_async_engine
|
||||||
|
|
||||||
|
async def reset():
|
||||||
|
app_config = load_config()
|
||||||
|
db_url = get_database_url(app_config)
|
||||||
|
schema = get_database_schema(app_config)
|
||||||
|
engine = create_async_engine(db_url)
|
||||||
|
async with engine.begin() as conn:
|
||||||
|
await conn.execute(text(f'SET search_path TO {schema}'))
|
||||||
|
await conn.execute(text('DELETE FROM alembic_version'))
|
||||||
|
print('Cleared alembic_version table')
|
||||||
|
await engine.dispose()
|
||||||
|
|
||||||
|
asyncio.run(reset())
|
||||||
|
"
|
||||||
|
```
|
||||||
@@ -178,9 +178,18 @@ def upgrade() -> None:
|
|||||||
["room_number"],
|
["room_number"],
|
||||||
unique=False,
|
unique=False,
|
||||||
)
|
)
|
||||||
op.create_index(
|
# Create index on acked_requests if it doesn't exist
|
||||||
op.f("ix_acked_requests_username"), "acked_requests", ["username"], unique=False
|
connection = op.get_bind()
|
||||||
)
|
inspector = sa.inspect(connection)
|
||||||
|
|
||||||
|
# Get existing indices on acked_requests
|
||||||
|
acked_requests_indices = [idx['name'] for idx in inspector.get_indexes('acked_requests')]
|
||||||
|
|
||||||
|
# Only create index if it doesn't exist
|
||||||
|
if "ix_acked_requests_username" not in acked_requests_indices:
|
||||||
|
op.create_index(
|
||||||
|
op.f("ix_acked_requests_username"), "acked_requests", ["username"], unique=False
|
||||||
|
)
|
||||||
# ### end Alembic commands ###
|
# ### end Alembic commands ###
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -20,6 +20,12 @@ depends_on: Union[str, Sequence[str], None] = None
|
|||||||
|
|
||||||
def upgrade() -> None:
|
def upgrade() -> None:
|
||||||
"""Upgrade schema."""
|
"""Upgrade schema."""
|
||||||
|
connection = op.get_bind()
|
||||||
|
|
||||||
|
# Check if hashed_customer_id column already exists in reservations
|
||||||
|
inspector = sa.inspect(connection)
|
||||||
|
reservations_columns = [col['name'] for col in inspector.get_columns('reservations')]
|
||||||
|
|
||||||
# ### commands auto generated by Alembic - please adjust! ###
|
# ### commands auto generated by Alembic - please adjust! ###
|
||||||
op.alter_column('hashed_customers', 'customer_id',
|
op.alter_column('hashed_customers', 'customer_id',
|
||||||
existing_type=sa.INTEGER(),
|
existing_type=sa.INTEGER(),
|
||||||
@@ -29,22 +35,22 @@ def upgrade() -> None:
|
|||||||
op.drop_constraint(op.f('reservations_customer_id_fkey'), 'reservations', type_='foreignkey')
|
op.drop_constraint(op.f('reservations_customer_id_fkey'), 'reservations', type_='foreignkey')
|
||||||
op.create_foreign_key(None, 'reservations', 'customers', ['customer_id'], ['id'], ondelete='SET NULL')
|
op.create_foreign_key(None, 'reservations', 'customers', ['customer_id'], ['id'], ondelete='SET NULL')
|
||||||
|
|
||||||
# Add hashed_customer_id column to reservations with cascade delete
|
# Add hashed_customer_id column to reservations if it doesn't exist
|
||||||
op.add_column('reservations', sa.Column('hashed_customer_id', sa.Integer(), nullable=True))
|
if 'hashed_customer_id' not in reservations_columns:
|
||||||
op.create_index(op.f('ix_reservations_hashed_customer_id'), 'reservations', ['hashed_customer_id'], unique=False)
|
op.add_column('reservations', sa.Column('hashed_customer_id', sa.Integer(), nullable=True))
|
||||||
op.create_foreign_key(None, 'reservations', 'hashed_customers', ['hashed_customer_id'], ['id'], ondelete='CASCADE')
|
op.create_index(op.f('ix_reservations_hashed_customer_id'), 'reservations', ['hashed_customer_id'], unique=False)
|
||||||
# ### end Alembic commands ###
|
op.create_foreign_key(None, 'reservations', 'hashed_customers', ['hashed_customer_id'], ['id'], ondelete='CASCADE')
|
||||||
|
|
||||||
# Data migration: Populate hashed_customer_id from customer relationship
|
# Data migration: Populate hashed_customer_id from customer relationship
|
||||||
connection = op.get_bind()
|
update_stmt = sa.text("""
|
||||||
update_stmt = sa.text("""
|
UPDATE reservations r
|
||||||
UPDATE reservations r
|
SET hashed_customer_id = hc.id
|
||||||
SET hashed_customer_id = hc.id
|
FROM hashed_customers hc
|
||||||
FROM hashed_customers hc
|
WHERE r.customer_id = hc.customer_id
|
||||||
WHERE r.customer_id = hc.customer_id
|
AND hc.customer_id IS NOT NULL
|
||||||
AND hc.customer_id IS NOT NULL
|
""")
|
||||||
""")
|
connection.execute(update_stmt)
|
||||||
connection.execute(update_stmt)
|
# ### end Alembic commands ###
|
||||||
|
|
||||||
|
|
||||||
def downgrade() -> None:
|
def downgrade() -> None:
|
||||||
|
|||||||
@@ -20,10 +20,21 @@ depends_on: Union[str, Sequence[str], None] = None
|
|||||||
|
|
||||||
def upgrade() -> None:
|
def upgrade() -> None:
|
||||||
"""Upgrade schema."""
|
"""Upgrade schema."""
|
||||||
# Add hashed_customer_id column to conversion_guests
|
connection = op.get_bind()
|
||||||
op.add_column('conversion_guests', sa.Column('hashed_customer_id', sa.Integer(), nullable=True))
|
inspector = sa.inspect(connection)
|
||||||
op.create_index(op.f('ix_conversion_guests_hashed_customer_id'), 'conversion_guests', ['hashed_customer_id'], unique=False)
|
|
||||||
op.create_foreign_key(None, 'conversion_guests', 'hashed_customers', ['hashed_customer_id'], ['id'], ondelete='SET NULL')
|
# Check if conversion_guests table and hashed_customer_id column exist
|
||||||
|
tables = inspector.get_table_names()
|
||||||
|
|
||||||
|
# Only proceed if conversion_guests table exists
|
||||||
|
if 'conversion_guests' in tables:
|
||||||
|
conversion_guests_columns = [col['name'] for col in inspector.get_columns('conversion_guests')]
|
||||||
|
|
||||||
|
# Add hashed_customer_id column if it doesn't exist
|
||||||
|
if 'hashed_customer_id' not in conversion_guests_columns:
|
||||||
|
op.add_column('conversion_guests', sa.Column('hashed_customer_id', sa.Integer(), nullable=True))
|
||||||
|
op.create_index(op.f('ix_conversion_guests_hashed_customer_id'), 'conversion_guests', ['hashed_customer_id'], unique=False)
|
||||||
|
op.create_foreign_key(None, 'conversion_guests', 'hashed_customers', ['hashed_customer_id'], ['id'], ondelete='SET NULL')
|
||||||
|
|
||||||
|
|
||||||
def downgrade() -> None:
|
def downgrade() -> None:
|
||||||
|
|||||||
Reference in New Issue
Block a user