0e352b11f5
Merge pull request #26547 from overleaf/em-revert-expiry-post
...
Revert "use POST requests for expiring redis buffer from cron"
GitOrigin-RevId: 95e9fd1be7b73699d6fac24035437d467c273d0a
2025-06-19 08:06:58 +00:00
a6ec5e479c
Merge pull request #26539 from overleaf/jpa-post
...
[history-v1] use POST requests for expiring redis buffer from cron
GitOrigin-RevId: 51c9a25b998e581ed20c0e113bd4989537a1e6ef
2025-06-19 08:06:50 +00:00
0d8e6e6175
Merge pull request #26533 from overleaf/jpa-queue
...
[history-v1] use POST requests for flushing history queues
GitOrigin-RevId: ae87a046a7844b25cc123d08ac6c8d1624608394
2025-06-19 08:06:16 +00:00
847db0b056
Merge pull request #26477 from overleaf/em-redis-buffer-resync
...
Handle invalid content hash when persisting changes
GitOrigin-RevId: 5259190396c8c261cad1abcd5de66314c1e871fb
2025-06-18 08:06:17 +00:00
5dfa4d6a46
Merge pull request #26353 from overleaf/bg-history-redis-extend-persist-worker
...
extend persist worker to make parallel requests
GitOrigin-RevId: 8def7d5a8b5c9fcbe5fe45ac8f3ace503d31877a
2025-06-17 08:05:55 +00:00
97169ff991
Merge pull request #26351 from overleaf/bg-history-redis-convert-persist-worker-to-esm
...
history redis convert persist worker to esm
GitOrigin-RevId: edcbac6e3f1d3dde3fa8239378995f3ff3afcfdd
2025-06-13 08:06:32 +00:00
b2dfb88c90
Merge pull request #26293 from overleaf/bg-history-redis-fix-persist-worker
...
add missing load global blobs from persist worker
GitOrigin-RevId: ae9393f2353fb4d5afe349aa7d0a26bab80c7f53
2025-06-11 08:06:59 +00:00
6c23d09fdd
Merge pull request #26273 from overleaf/bg-history-redis-add-persist-worker-to-cron
...
modify existing run-chunk-lifecycle cron job to persist and expire redis queues
GitOrigin-RevId: afb94b3e2fba7368cfec11997dfd5b2bbd6321a9
2025-06-11 08:06:23 +00:00
f54e695367
Merge pull request #26122 from overleaf/bg-history-redis-make-persist-buffer-consistent
...
make persistBuffer export consistent with other methods
GitOrigin-RevId: 24536e521e1d20ef63cc74bd9ba40e095025d512
2025-06-09 08:05:39 +00:00
64f729d838
Merge pull request #26116 from overleaf/bg-history-redis-show-buffer
...
add script to display redis buffer for a given history ID
GitOrigin-RevId: 71c2e79480c0873d30801ed3c13aa9a7fc7873f6
2025-06-09 08:05:35 +00:00
4c697a5659
Merge pull request #25993 from overleaf/bg-history-refactor-persist-buffer-limits
...
refactor persist buffer to add limits
GitOrigin-RevId: 4a40a7a8812acf5bb7f98bfd7b94d81ebe19fc57
2025-06-04 08:06:19 +00:00
2088d8e30a
Merge pull request #25954 from overleaf/bg-history-expire-worker-fix
...
fix expire_redis_chunks to only clear job on error
GitOrigin-RevId: f7ec435edda95958b453fba501686dcfd84426f7
2025-06-04 08:06:14 +00:00
f0da86a18c
Merge pull request #25909 from overleaf/bg-history-persist-worker
...
add history persist worker
GitOrigin-RevId: b9e31e7bdd84570efc0b87b9f5e90b4078551a8c
2025-06-04 08:06:10 +00:00
dfed87723f
[web] remove deledFiles collection ( #25750 )
...
* [history-v1] remove processing of deleted files when back-filling hashes
* [web] remove deledFiles collection
GitOrigin-RevId: 7c080e564f7d7acb33ebe7ebe012f415a847d0df
2025-05-23 08:05:01 +00:00
1fab10a24a
Merge pull request #25482 from overleaf/bg-fix-bull-exporter-errors
...
retain completed and failed jobs for backup queue
GitOrigin-RevId: 7831ce2565dc493e3ce7f55001207daea2140575
2025-05-13 08:05:40 +00:00
5560eeb0ef
Merge pull request #25449 from overleaf/bg-histoy-redis-refactor-expire-worker
...
refactor the expire worker to make it easier to extend
GitOrigin-RevId: 7b277b243ed51ab3b46316d98b7157af95a9e42b
2025-05-12 08:05:19 +00:00
a9cb0cc197
Merge pull request #25361 from overleaf/em-load-latest-raw
...
Rename loadLatestRaw() to getLatestChunkMetadata()
GitOrigin-RevId: e089dcfa57cbbc43df8666b51eca0d81d595a5a7
2025-05-09 08:06:14 +00:00
a61afe06b3
Merge pull request #25306 from overleaf/em-redis-buffer-read-operations
...
Add changes from Redis when reading chunks from the chunk store
GitOrigin-RevId: c0ebf0669b91eb2efc5d1091d025e81efdff9fe4
2025-05-09 08:06:09 +00:00
b2c60ba678
Merge pull request #25241 from overleaf/bg-remove-existing-chunk-buffer
...
remove existing chunk redis backend and chunk buffer
GitOrigin-RevId: 28fb02d1802312de6892e2fb7dd59191e3fc8914
2025-05-08 08:05:09 +00:00
30c7a81361
Merge pull request #25086 from overleaf/bg-history-buffer-use-expire-time
...
add expire time to redis buffer in history-v1
GitOrigin-RevId: 3d74957c341e62e536dc60869a7ca71ac173e380
2025-04-29 08:05:14 +00:00
7fc9e8a483
Merge pull request #24945 from overleaf/bg-redis-buffer-stats-script
...
add script for listing buffer stats from redis
GitOrigin-RevId: 7836563d51a5c6ded264d0e709d5cfcda70596e3
2025-04-24 08:05:49 +00:00
8b3f90b2c1
Merge pull request #24903 from overleaf/bg-fix-backup-scheduler
...
remove history.id check from processPendingProjects function
GitOrigin-RevId: 017ab7c9bf693ed4077d4619574154166af04fe3
2025-04-16 08:05:06 +00:00
5fa69ca474
Merge pull request #24857 from overleaf/bg-backup-scheduler-fix-for-broken-projects
...
handle broken projects in history backup
GitOrigin-RevId: a295d9d7a29715807a8172dae46e0fe3c14ecf12
2025-04-15 08:06:23 +00:00
27bd613580
Merge pull request #24768 from overleaf/bg-history-redis-buffer
...
test redis caching when loading latest chunk in history-v1
GitOrigin-RevId: f0ee09e5e9e1d7605e228913cb8539be4134e1f7
2025-04-15 08:05:03 +00:00
ab4e0e6b71
Merge pull request #24749 from overleaf/bg-history-redis-buffer
...
add history redis to history-v1
GitOrigin-RevId: 70dc1aee809ad17902c93c020f3804c0f1429238
2025-04-10 08:05:35 +00:00
ef491e8b8b
Merge pull request #24636 from overleaf/bg-history-backup-fix-broken-projects
...
add --fix option to backup script
GitOrigin-RevId: 568c9158669bb1cede0f0dd75e7507b10e8ff5a2
2025-04-08 08:05:59 +00:00
4b3c9d5b5c
Merge pull request #24637 from overleaf/bg-history-backup-uninitialised-projects
...
backup uninitialised projects
GitOrigin-RevId: 9310ef9f803decffbd674024a1ffd33d1960a2c4
2025-04-08 08:05:54 +00:00
d66ff5c417
Add a script for verifying a set of blobs have been backed up
...
GitOrigin-RevId: 8f7af647ce0ac86ee9800cdf3d386fb78debed7a
2025-03-24 10:48:39 +00:00
d7d7cd85be
Merge pull request #24335 from overleaf/bg-warn-old-pending-changes
...
add warning for backups outside rpo threshold
GitOrigin-RevId: a8421529ae64693d860b0325961b010a132426da
2025-03-18 09:05:49 +00:00
cc1f7badfa
Extend script to allow multiple blobs to be processed from csv
...
GitOrigin-RevId: ad47eb754436ddc7f56b27ceda627268c3a030a1
2025-03-18 09:05:03 +00:00
39c36babf0
Add a script to backup a single blob from a project
...
GitOrigin-RevId: 464e6d69093b87891497e07d1627cd20e2285380
2025-03-18 09:04:59 +00:00
e96b774278
Add verification looper and handle shutdown signals
...
Shutdown signals become more relevant now that we are looping as we want
to gracefully stop processing records rather than continue looping.
GitOrigin-RevId: dbb499388c86d552d77954988f8fc27d140da3f1
2025-03-18 09:04:54 +00:00
27dff88272
Add new strategy to verify_sampled_projects
...
GitOrigin-RevId: d967da41250bb5945d5b8668b212d4a61b4f9d69
2025-03-18 09:04:50 +00:00
35c71f1da7
Merge pull request #24269 from overleaf/bg-backup-replication-smoothing
...
extend backup scheduler to delay jobs for load levelling
GitOrigin-RevId: be62c458d3be464bf80ca590fcc107c2760aa061
2025-03-13 09:06:28 +00:00
9f81e2d9f1
Merge pull request #24268 from overleaf/bg-backup-replication-lag-metrics
...
add backup replication metrics
GitOrigin-RevId: a734435e3c6ce56350b2286bd218a5e2324d93a9
2025-03-13 09:06:24 +00:00
388a2bdc09
Merge pull request #24267 from overleaf/bg-fix-init-option
...
update backup script to use index when finding uninitialised projects
GitOrigin-RevId: 04ce0654b3d42de1c1a9bb542482c2dd53540628
2025-03-13 09:06:20 +00:00
8fab444cf2
Merge pull request #24256 from overleaf/bg-backup-fix-sample
...
fix backup sample script
GitOrigin-RevId: 4bf4c15324d9b3b7ebd809f7b212e52d75ab216d
2025-03-13 09:06:16 +00:00
ae99cf18c5
Fix script name in usage prompt
...
GitOrigin-RevId: ab4ed7479c96d3d1db82b23ef3c883706c61995e
2025-03-12 09:05:44 +00:00
256c69dadc
Ensure global blobs are loaded before verifying
...
GitOrigin-RevId: 3f78c44e6b4d98fd6c469b0f4fbacc033b0a0868
2025-03-12 09:05:39 +00:00
3600b6c867
Add script for verifying sampled sets of projects
...
GitOrigin-RevId: 23a599ea269a528201c67cda66bd2d00c0d376e4
2025-03-12 09:05:35 +00:00
beddb9bc1c
Merge pull request #24237 from overleaf/bg-fix-backup-worker
...
fixes for backup worker
GitOrigin-RevId: 79c99e5b2d99d1b0298f69d968e0e2cb89281dd6
2025-03-12 09:05:04 +00:00
3e84cb1f0b
Merge pull request #24200 from overleaf/bg-backup-queue-pending-jobs
...
fix backup worker and backup scheduler to handle pending projects
GitOrigin-RevId: a97e011615666b3ae2b8fafe26a96d41b3609edd
2025-03-11 09:06:05 +00:00
061c8d99f8
Merge pull request #24204 from overleaf/ar-jpa-add-chunk-verification
...
[history-v1] add chunk verification
GitOrigin-RevId: 7208ad20872386813bb1c6946283afddb5e8b1cf
2025-03-11 09:05:57 +00:00
6af0371d78
Merge pull request #24142 from overleaf/bg-backup-fix-blob-error-handling
...
fix bug which caused errors to be ignored when backing up blobs
GitOrigin-RevId: 64674a484a6e22493cba8f8e47d4671072dc2e9a
2025-03-07 09:06:26 +00:00
cbf46ae22c
Merge pull request #24080 from overleaf/bg-serialise-backup-initialisation
...
fix backup initialization to avoid incorrect use of batchedUpdate
GitOrigin-RevId: 6984f3510c6b03b3dfda35efea8173f848e58eff
2025-03-07 09:06:22 +00:00
3d2a718f73
Merge pull request #24102 from overleaf/bg-backup-stats-script
...
add backup sampling script
GitOrigin-RevId: cfd546d421cdcfb9f8ad2111703a437a7272bd54
2025-03-07 09:06:17 +00:00
2cde8ded83
Merge pull request #24069 from overleaf/bg-backup-errors
...
more tweaks for backup errors
GitOrigin-RevId: 0f7c7bb5004923c3c22c6e3471bb7152cc3e05e2
2025-03-05 09:05:50 +00:00
f70ef53888
Merge pull request #24053 from overleaf/bg-backup-queue-tweaks-II
...
additional backup queue tweaks
GitOrigin-RevId: b54e0c5f4a23bb9962c5062bd1b80f9f124c3c87
2025-03-05 09:05:15 +00:00
0cb0631b0e
Merge pull request #24037 from overleaf/bg-fix-backup-worker-errors
...
backup worker tweaks
GitOrigin-RevId: 7901bb0257c7c7320afd08ef965f124347a1de30
2025-03-04 09:05:12 +00:00
c5b4e9196b
Merge pull request #24019 from overleaf/bg-backup-add-error-logging
...
add logging of backup errors in batch mode
GitOrigin-RevId: 96ae7a2354f82451b3bbe8c5459c4d4cf47e5459
2025-03-04 09:04:44 +00:00
Eric Mc Sween