d6c2188f2d
Merge pull request #24903 from overleaf/bg-fix-backup-scheduler
...
remove history.id check from processPendingProjects function
GitOrigin-RevId: 017ab7c9bf693ed4077d4619574154166af04fe3
2025-04-16 08:05:06 +00:00
3850e97446
Merge pull request #24857 from overleaf/bg-backup-scheduler-fix-for-broken-projects
...
handle broken projects in history backup
GitOrigin-RevId: a295d9d7a29715807a8172dae46e0fe3c14ecf12
2025-04-15 08:06:23 +00:00
835e14b8b2
Merge pull request #24768 from overleaf/bg-history-redis-buffer
...
test redis caching when loading latest chunk in history-v1
GitOrigin-RevId: f0ee09e5e9e1d7605e228913cb8539be4134e1f7
2025-04-15 08:05:03 +00:00
4ba0e97b95
Merge pull request #24749 from overleaf/bg-history-redis-buffer
...
add history redis to history-v1
GitOrigin-RevId: 70dc1aee809ad17902c93c020f3804c0f1429238
2025-04-10 08:05:35 +00:00
040f70471c
Merge pull request #24636 from overleaf/bg-history-backup-fix-broken-projects
...
add --fix option to backup script
GitOrigin-RevId: 568c9158669bb1cede0f0dd75e7507b10e8ff5a2
2025-04-08 08:05:59 +00:00
f08532dfb0
Merge pull request #24637 from overleaf/bg-history-backup-uninitialised-projects
...
backup uninitialised projects
GitOrigin-RevId: 9310ef9f803decffbd674024a1ffd33d1960a2c4
2025-04-08 08:05:54 +00:00
2779691cd9
Add a script for verifying a set of blobs have been backed up
...
GitOrigin-RevId: 8f7af647ce0ac86ee9800cdf3d386fb78debed7a
2025-03-24 10:48:39 +00:00
0292bc418d
Merge pull request #24335 from overleaf/bg-warn-old-pending-changes
...
add warning for backups outside rpo threshold
GitOrigin-RevId: a8421529ae64693d860b0325961b010a132426da
2025-03-18 09:05:49 +00:00
f134746c9c
Extend script to allow multiple blobs to be processed from csv
...
GitOrigin-RevId: ad47eb754436ddc7f56b27ceda627268c3a030a1
2025-03-18 09:05:03 +00:00
b5d6484991
Add a script to backup a single blob from a project
...
GitOrigin-RevId: 464e6d69093b87891497e07d1627cd20e2285380
2025-03-18 09:04:59 +00:00
78481e010e
Add verification looper and handle shutdown signals
...
Shutdown signals become more relevant now that we are looping as we want
to gracefully stop processing records rather than continue looping.
GitOrigin-RevId: dbb499388c86d552d77954988f8fc27d140da3f1
2025-03-18 09:04:54 +00:00
9d72eeeeac
Add new strategy to verify_sampled_projects
...
GitOrigin-RevId: d967da41250bb5945d5b8668b212d4a61b4f9d69
2025-03-18 09:04:50 +00:00
2d48c86e61
Merge pull request #24269 from overleaf/bg-backup-replication-smoothing
...
extend backup scheduler to delay jobs for load levelling
GitOrigin-RevId: be62c458d3be464bf80ca590fcc107c2760aa061
2025-03-13 09:06:28 +00:00
a178c0f400
Merge pull request #24268 from overleaf/bg-backup-replication-lag-metrics
...
add backup replication metrics
GitOrigin-RevId: a734435e3c6ce56350b2286bd218a5e2324d93a9
2025-03-13 09:06:24 +00:00
cf105cf01d
Merge pull request #24267 from overleaf/bg-fix-init-option
...
update backup script to use index when finding uninitialised projects
GitOrigin-RevId: 04ce0654b3d42de1c1a9bb542482c2dd53540628
2025-03-13 09:06:20 +00:00
3b93efdf5c
Merge pull request #24256 from overleaf/bg-backup-fix-sample
...
fix backup sample script
GitOrigin-RevId: 4bf4c15324d9b3b7ebd809f7b212e52d75ab216d
2025-03-13 09:06:16 +00:00
149b590413
Fix script name in usage prompt
...
GitOrigin-RevId: ab4ed7479c96d3d1db82b23ef3c883706c61995e
2025-03-12 09:05:44 +00:00
282f5f92ff
Ensure global blobs are loaded before verifying
...
GitOrigin-RevId: 3f78c44e6b4d98fd6c469b0f4fbacc033b0a0868
2025-03-12 09:05:39 +00:00
afedce1b0e
Add script for verifying sampled sets of projects
...
GitOrigin-RevId: 23a599ea269a528201c67cda66bd2d00c0d376e4
2025-03-12 09:05:35 +00:00
efd53e567c
Merge pull request #24237 from overleaf/bg-fix-backup-worker
...
fixes for backup worker
GitOrigin-RevId: 79c99e5b2d99d1b0298f69d968e0e2cb89281dd6
2025-03-12 09:05:04 +00:00
c233243948
Merge pull request #24200 from overleaf/bg-backup-queue-pending-jobs
...
fix backup worker and backup scheduler to handle pending projects
GitOrigin-RevId: a97e011615666b3ae2b8fafe26a96d41b3609edd
2025-03-11 09:06:05 +00:00
441c7a89a7
Merge pull request #24204 from overleaf/ar-jpa-add-chunk-verification
...
[history-v1] add chunk verification
GitOrigin-RevId: 7208ad20872386813bb1c6946283afddb5e8b1cf
2025-03-11 09:05:57 +00:00
9ff3a45690
Merge pull request #24142 from overleaf/bg-backup-fix-blob-error-handling
...
fix bug which caused errors to be ignored when backing up blobs
GitOrigin-RevId: 64674a484a6e22493cba8f8e47d4671072dc2e9a
2025-03-07 09:06:26 +00:00
281168fd52
Merge pull request #24080 from overleaf/bg-serialise-backup-initialisation
...
fix backup initialization to avoid incorrect use of batchedUpdate
GitOrigin-RevId: 6984f3510c6b03b3dfda35efea8173f848e58eff
2025-03-07 09:06:22 +00:00
154de1a0f9
Merge pull request #24102 from overleaf/bg-backup-stats-script
...
add backup sampling script
GitOrigin-RevId: cfd546d421cdcfb9f8ad2111703a437a7272bd54
2025-03-07 09:06:17 +00:00
893294e6b8
Merge pull request #24069 from overleaf/bg-backup-errors
...
more tweaks for backup errors
GitOrigin-RevId: 0f7c7bb5004923c3c22c6e3471bb7152cc3e05e2
2025-03-05 09:05:50 +00:00
34b397073f
Merge pull request #24053 from overleaf/bg-backup-queue-tweaks-II
...
additional backup queue tweaks
GitOrigin-RevId: b54e0c5f4a23bb9962c5062bd1b80f9f124c3c87
2025-03-05 09:05:15 +00:00
c815106d62
Merge pull request #24037 from overleaf/bg-fix-backup-worker-errors
...
backup worker tweaks
GitOrigin-RevId: 7901bb0257c7c7320afd08ef965f124347a1de30
2025-03-04 09:05:12 +00:00
1dbf5dca10
Merge pull request #24019 from overleaf/bg-backup-add-error-logging
...
add logging of backup errors in batch mode
GitOrigin-RevId: 96ae7a2354f82451b3bbe8c5459c4d4cf47e5459
2025-03-04 09:04:44 +00:00
4720e8f2c5
Merge pull request #23989 from overleaf/bg-backup-worker-reduce-logging
...
reduce log level from info to debug for backup logging
GitOrigin-RevId: 10f541b0e0068aafd50710226fb8425527fee7c5
2025-03-03 09:05:22 +00:00
8d0fa5be65
Merge pull request #23977 from overleaf/bg-backup-worker-load-global-blobs
...
ensure global blobs are loaded in backup worker
GitOrigin-RevId: b80791ef38e7fa8db364bdd3a9c461757778409d
2025-03-03 09:05:08 +00:00
b37b409994
Merge pull request #23946 from overleaf/bg-backup-queue-implementation
...
additional backup queue implementation
GitOrigin-RevId: 03754e57a6c6798a07dcca6a5248dec61b3cdc7a
2025-02-28 09:05:50 +00:00
763e5ba82c
Merge pull request #23920 from overleaf/bg-backup-queue
...
initial history backup queue worker app
GitOrigin-RevId: e9647a2ec3caeb1fff843cead12164ba89fdb1f8
2025-02-28 09:04:46 +00:00
6e55a9fa81
Merge pull request #23837 from overleaf/bg-redis-redact-config
...
sanitise password in redis config output
GitOrigin-RevId: 45d8bd13f01c3c4f138549c8a9f47dc0db29d0b0
2025-02-25 09:05:22 +00:00
0acc067205
Merge pull request #23836 from overleaf/bg-backup-extend-debugging-script
...
add support for fetching chunks in backup debugging script
GitOrigin-RevId: 641298088d62148693525259748dcddf4d67d14b
2025-02-25 09:05:18 +00:00
9d8f9e5358
Merge pull request #23835 from overleaf/bg-backup-show-errors
...
add error cause logging for missing chunks and blobs in backup comparison
GitOrigin-RevId: 9a1bf84b64621add71fb1d1e0cb013598b226bc8
2025-02-25 09:05:14 +00:00
dd8a683e9e
Merge pull request #23834 from overleaf/bg-backup-graceful-shutdown
...
add graceful shutdown handling to backup script
GitOrigin-RevId: f0fb9a3867b26c7800edefab126fd05fbabdae02
2025-02-25 09:05:10 +00:00
b0ed2efdf9
Merge pull request #23812 from overleaf/ar-allow-debugging-for-remove-backup-blobs-script-errors
...
[history-v1] Minor improvements to AWS cleanup script
GitOrigin-RevId: bd01a72169b9cbc018612173294cfbf8649a9fbd
2025-02-24 09:06:15 +00:00
a0d79d5871
Merge pull request #23742 from overleaf/ar-script-for-removing-blobs-in-wrong-place
...
[history-v1] Script for deleting blobs backed up to wrong location
GitOrigin-RevId: 6abe1dba5164f8fa7d41c4ee3e4a07764e73b7e1
2025-02-24 09:05:53 +00:00
981e91f012
Merge pull request #21763 from overleaf/bg-backup-script
...
initial script for running backups
GitOrigin-RevId: d22c373de30738d8080d40dce10790f0bdcb9f51
2025-02-24 09:04:32 +00:00
32983da29c
Merge pull request #23700 from overleaf/bg-history-redis
...
add redis to history-v1
GitOrigin-RevId: def900c4b560ea0a297b11b6c0ecc8cc8caa5b49
2025-02-21 09:04:06 +00:00
8831fb9a18
Merge pull request #23169 from overleaf/bg-export-global-blobs
...
Add script to export global blobs to CSV
GitOrigin-RevId: 86c885981b150aae4541770a094c822e1fb364b9
2025-02-03 09:05:53 +00:00
d6e89c7338
Merge pull request #23058 from overleaf/bg-export-backed-up-blobs
...
export backed up blobs to csv file
GitOrigin-RevId: 5a176055dc85c56ced64cbf13aa705b56071cdae
2025-01-24 09:05:31 +00:00
9c377b74c1
[history-v1] add flag for back-filling hashes for projectIds from file ( #23005 )
...
* [history-v1] delay process exit to give logging time to flush
* [history-v1] add flag for back-filling hashes for projectIds from file
GitOrigin-RevId: 887a1e1c72d6f5a13bfc8d0e54023afbf5bc671c
2025-01-22 09:05:21 +00:00
6884338a34
[history-v1] fix remaining files with missing hashes ( #22985 )
...
GitOrigin-RevId: df382732954538f9db177180aee0a44c5d5679ca
2025-01-22 09:05:16 +00:00
ab4d1e0986
Merge pull request #22870 from overleaf/jpa-back-fill-fix-up
...
[history-v1] add script for fixing up back-fill errors
GitOrigin-RevId: 118992a32c1f6da4289cd35399ddd07a741da4ee
2025-01-20 09:05:14 +00:00
a92a37bc3c
Merge pull request #22466 from overleaf/ar-backup-files-when-inserting
...
[history-v1] backup files when inserting
GitOrigin-RevId: e636bce178604978c6d41c083bf671795d20b5a1
2024-12-13 09:04:54 +00:00
3d7254b419
Merge pull request #22153 from overleaf/jpa-backup-verifier-minimal
...
[history-v1] backup-verifier-app: initial revision
GitOrigin-RevId: 922c9f94cb7ca7c129e38fd6961d42bdff819cd8
2024-11-27 09:04:55 +00:00
40603e0561
Merge pull request #22134 from overleaf/jpa-broken-file-trees
...
[history-v1] back_fill_file_hash: gracefully handle broken file-trees
GitOrigin-RevId: 463c785e98581364b107f3262951e7fa0fb88a0f
2024-11-26 09:05:51 +00:00
b7d37b434a
Merge pull request #22123 from overleaf/jpa-process-hashed
...
[history-v1] back_fill_file_hash: optionally process hashed files
GitOrigin-RevId: 53ebaa7b03166a6b1aacc0f985bb295c9ec04804
2024-11-26 09:05:47 +00:00
Brian Gough