729e0f5ac9
* fix: correct typedef for Document in helpers.mjs * add move-migrations codemod * update migration paths to use shared migrations directory * move migrations to shared location * fix: update Dockerfile and docker-compose.ci.yml to include migrations directory * feat: add migrations tool to workspaces in package.json * [monorepo] Fix order of docker ignore rules * [web] remove unused docker ignore file * [monorepo] replace old references to migrations folder * [server-ce] copy migrations from new place * [migrations] Inline web scripts Co-authored-by: Brian Gough <brian.gough@overleaf.com> * [migrations] move three web scripts over Co-authored-by: Brian Gough <brian.gough@overleaf.com> * [migrations] add missing collection Co-authored-by: Brian Gough <brian.gough@overleaf.com> * [migrations] remove lodash dependency Co-authored-by: Brian Gough <brian.gough@overleaf.com> * [migrations] avoid mongodb-legacy dependency Co-authored-by: Brian Gough <brian.gough@overleaf.com> * [monorepo] run migrations from tools/migrations Co-authored-by: Brian Gough <brian.gough@overleaf.com> * [migrations] simplify migration for adding gitBridge feature to users * [monorepo] run migrations from tests in all the services * [migrations] add Jenkins pipeline for linting/formatting * [monorepo] fixup running web migrations everywhere * [monorepo] trigger Jenkins builds on changes to mongo migrations * [migrations] add Jenkins pipeline for linting/formatting * [monorepo] build scripts: update devDependencies before deps scanning * [monorepo] build scripts: formerly depend on tools/migrations * [monorepo] run eslint on .mjs files * [migrations] enable more eslint rules and fix all the errors * [rake] fix migrations:list task --------- Co-authored-by: Jakob Ackermann <jakob.ackermann@overleaf.com> GitOrigin-RevId: 14cf69cc1b9405bbc75adbb9a000e555500e0614
Database migrations
The history service uses knex to manage PostgreSQL migrations.
To create a new migrations, run:
npx knex migrate:make migration_name
To apply migrations, run:
npx knex migrate:latest
For more information, consult the knex migrations guide.
Global blobs
Global blobs are blobs that are shared between projects. The list of global blobs is stored in the projectHistoryGlobalBlobs Mongo collection and is read when the service starts. Changing the list of global blobs needs to be done carefully.
Adding a blob to the global blobs list
If we identify a blob that appears in many projects, we might want to move that blob to the global blobs list.
- Add a record for the blob to the projectHistoryGlobalBlobs collection.
- Restart the history service.
- Delete any corresponding project blobs.
Removing a blob from the global blobs list
Removing a blob from the global blobs list is trickier. As soon as the global blob is made unavailable, every project that needs the blob will have to get its own copy. To avoid disruptions, follow these steps:
-
In the projectHistoryGlobalBlobs collection, set the
demotedproperty tofalseon the global blob to remove. This will make the history system write new instances of this blob to project blobs, but still read from the global blob. -
Restart the history service.
-
Copy the blob to all projects that need it.
-
Remove the blob from the projectHistoryGlobalBlobs collection.
-
Restart the history service.