Some tasks announce themselves as dangerous. "Rewrite the payment flow" comes with a warning label. Nobody relaxes when they read it.

And then there's this:

Schema::table('payments', function (Blueprint $table) {
    $table->decimal('amount', 12, 2)->change();
    $table->index(['tenant_id', 'due_date']);
});

Five lines. Widen a column, add an index. It runs in under a second locally. What could possibly go wrong?

Reader, quite a lot.

The five stages of a "quick" migration

Stage 1: Confidence. ❌ The migration ran instantly locally, on a table with a few hundred seeded rows. Production has years of history in that table. Same code, very different table.

Stage 2: Mild curiosity. ❌ The deploy pipeline has been on "Running migrations" for a while now. Probably just slow today. You refresh. You refresh again.

Stage 3: Unease. ❌ Someone posts "is the site slow for anyone else?" in the team chat. The payments screen spins. The mobile app shows its friendly "something went wrong" message to everyone trying to pay.

Stage 4: Investigation, at speed. ❌ SHOW PROCESSLIST shows your ALTER TABLE, and behind it a long, growing queue of ordinary queries, all "Waiting for table metadata lock". Your migration hasn't even started its real work yet.

Stage 5: Acceptance. ❌ You can't safely kill it halfway without thinking hard, you can't speed it up, and you definitely can't explain to anyone why "add an index" took the site down. You wait. You make tea. You don't drink the tea.

What was actually going on

Two separate things, both invisible on a small table.

Changing a column's data type rebuilds the table. In MySQL with InnoDB, many changes can happen "online" now: adding a column is often instant, and adding an index can run while reads and writes continue. But changing a column's type, like the precision of a decimal, needs a full table copy. MySQL creates a new table, copies every row across, and blocks writes while it does. On a big table, that's not a second. That's a coffee break you didn't plan.

Even "online" changes need a metadata lock. Briefly, at the start and end, the ALTER needs exclusive access to the table's definition. If any long-running query or open transaction is using the table (a big report, a stuck worker, someone's forgotten console session), the ALTER waits. And here's the nasty part: every new query on that table now queues behind the waiting ALTER. The migration doesn't have to do anything to take the site down. It just has to wait in the wrong place.

What actually makes migrations boring again

✅ Know your table sizes before you write the migration. A quick row count tells you whether this is a five-line job or a planned operation.

✅ Test on a copy with production-sized data (anonymised where needed) and time it. "Fast locally" proves nothing about a table that's thousands of times bigger.

✅ Ask MySQL to refuse instead of surprise you. Spell out the algorithm and lock you expect. If the change can't be done that way, MySQL fails immediately with an error instead of quietly copying the table:

DB::statement('SET SESSION lock_wait_timeout = 5');
DB::statement(
    'ALTER TABLE payments ADD INDEX payments_tenant_due (tenant_id, due_date), ALGORITHM=INPLACE, LOCK=NONE'
);

The short lock_wait_timeout means that if the metadata lock isn't available, the ALTER gives up after a few seconds instead of holding up every query behind it. A failed migration you can retry later beats a queue of frozen requests.

✅ Use an online schema change tool for the heavy stuff. For type changes on big, busy tables, tools like gh-ost or pt-online-schema-change build a copy in the background, keep it in sync, and swap it in at the end. More moving parts, far less downtime.

✅ Expand, then contract. Instead of changing a column in place: add a new column, backfill it in small batches, switch the code to use it, and drop the old one in a later release. Each step is small and reversible.

DB::table('payments')->whereNull('amount_v2')->chunkById(1000, function ($rows) {
    DB::table('payments')
        ->whereIn('id', $rows->pluck('id'))
        ->update(['amount_v2' => DB::raw('amount')]);
});

✅ Keep data backfills out of schema migrations. Run them as a queued job or an Artisan command you can pause, resume and watch, not as part of a deploy that's holding everything else up.

✅ Pick a quiet time and tell people. Off-peak hours, a heads-up to the team, and someone watching the database while it runs.

The happy ending

The table survives. It always does. What changes is the habit: the next time someone writes ->change() on a big table, a reviewer asks "how many rows, and has this been timed on a copy?" That one question is worth more than any tool.

What's the most innocent-looking migration that ever ruined your afternoon? Bonus points if it was "just an index".