fix: use bulk updates for migration 0048 #538

Merged

Erethon merged 1 commit from fix-0048 into main

2025-05-08 07:09:49 +00:00

Erethon commented

2025-05-07 16:51:39 +00:00

(Migrated from github.com)

The shared_nixderivation table in the production instance contains about three million rows (2.5GB). This isn't anything big by Postgres standards, but migration 0048 was accessing and updating each row one by one, which is slow and was running for more than 24 hours in the production instance.

I edited the migration to use bulk_updates instead and batched the processing of rows to make things faster. The thirty thousand rows as a batch number is somewhat arbitrary after some tests I made with 100k that seemed a bit too much.

The `shared_nixderivation` table in the production instance contains about three million rows (2.5GB). This isn't anything big by Postgres standards, but migration 0048 was accessing and updating each row one by one, which is slow and was running for more than 24 hours in the production instance. I edited the migration to use [bulk_updates](https://docs.djangoproject.com/en/5.2/ref/models/querysets/#bulk-update) instead and batched the processing of rows to make things faster. The thirty thousand rows as a batch number is somewhat arbitrary after some tests I made with 100k that seemed a bit too much.

❤️ 1

fricklerhandwerk (Migrated from github.com) approved these changes

2025-05-08 06:38:05 +00:00