Commit Graph

168 Commits

Author SHA1 Message Date
Eric Mc Sween 39842c9857 Merge pull request #23250 from overleaf/em-project-history-doc-hash
Propagate the doc hash from project-history to history

GitOrigin-RevId: 341df52df41f7a5f8dbebbce53a47f9d5e1f8175
2025-02-04 09:05:03 +00:00
Jakob Ackermann f727389b2b [project-history] make logging of latency in resync script optional (#23325)
GitOrigin-RevId: ee6863e05be648757ccaa96a0321085fe42f508f
2025-02-04 09:04:59 +00:00
Jakob Ackermann 3a4c5a0d0f [history-v1] add readOnly lookup for raw chunks (#23318)
* [history-v1] add readOnly lookup for raw chunks

Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>

* [history-v1] reduce min poolsize for readOnly pool to 0

Co-authored-by: Brian Gough <brian.gough@overleaf.com>

---------

Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
GitOrigin-RevId: a711c4ee4f3ea3775bd090e620d1ef52689fa1f4
2025-02-04 09:04:52 +00:00
Jakob Ackermann 5404dcb93d [misc] update build scripts (#23316)
* [misc] update build scripts

* [misc] make shellcheck happy

GitOrigin-RevId: 343655fda54aa7e835dd666dcd4218d61c58fd09
2025-02-04 09:04:37 +00:00
Jakob Ackermann c6c623da78 [project-history] script for fixing-up files/metadata with bulk resync (#23184)
* [history-v1] add cheap endpoint for checking time of last history write

The /raw endpoint skips the GCS lookup for the chunk.

* [project-history] script for fixing-up files/metadata with bulk resync

* [project-history] upgrade structure only resync when full sync is needed

* [project-history] start resync and process resync updates under lock

* [project-history] stop retrying during graceful shutdown

GitOrigin-RevId: 73184d5786e1d40f5b7e21f387fc37cf43f0ac2d
2025-02-03 09:05:43 +00:00
Jakob Ackermann b0c9176634 [project-history] add resync for project structure only (#23271)
* [project-history] add resync for project structure only

* [project-history] block resyncProjectStructureOnly that update docs

* [project-history] tweak test description

GitOrigin-RevId: ce2749566c36e04dab21c26dd60dd75d93c0d4c0
2025-02-03 09:05:39 +00:00
Jakob Ackermann ba60f885a4 [project-history] fetch most recent chunk once when processing updates (#23261)
GitOrigin-RevId: 08ddd1f953b0dbae0541a60b45fec0f88e0a1d06
2025-01-31 09:05:14 +00:00
Jakob Ackermann 72b4083318 [project-history] forward all the optional file attributes during resync (#23260)
Notably, 'createdBlob' was missing. And others like metadata should not
be added if not set.

GitOrigin-RevId: c4a6746c4824d637f7c54b4c68a7025b60c15ff2
2025-01-31 09:05:09 +00:00
Jakob Ackermann ae171dbced [project-history] fix name for error in logging statement (#23110)
Only the err/error fields will use the Error serializer and emit all the
OError details into the log.

GitOrigin-RevId: 4b0e16c00be5ea7c0aea747f49b46897ebd100cc
2025-01-27 09:05:22 +00:00
Eric Mc Sween 3e4e9b298f Merge pull request #23089 from overleaf/em-fix-ranges-snapshot
Fix edge cases when restoring comments

GitOrigin-RevId: 6ce2426bfb7233a3c0915bcc8c53bf0588702847
2025-01-27 09:05:06 +00:00
Jakob Ackermann 762266cd87 [project-history] block filestore reads from old queue entries (#23096)
GitOrigin-RevId: 9952cb66e542b17a6a3b5e3b2609d53dc8c371fd
2025-01-27 09:04:54 +00:00
Jakob Ackermann 8d99ad3964 [web] do not send filestore URLs when feature is disabled (#23095)
* [web] do not return createdBlob=true from error path

Defense in-depth, currently the only call-site bails out on error.

* [web] do not send filestore URLs when feature is disabled

GitOrigin-RevId: 7e90cf4c4babffeec337702502234bac73c1f116
2025-01-27 09:04:50 +00:00
Jakob Ackermann 3b602ed93a [project-history] log previous error when retrying createBlob operation (#23103)
GitOrigin-RevId: ff59d15e93c406e17775e5e4acac1f13da9a7788
2025-01-27 09:04:46 +00:00
Antoine Clausse 272108a213 Run bin/update_node 20.18.0 20.18.2 (#23074)
GitOrigin-RevId: 3f1ab14c4c36ea54408e0ce2c404a323bfe8d9a6
2025-01-24 09:06:05 +00:00
Jakob Ackermann 36c9772e0e [project-history] log context of failed history-v1 requests (#23078)
GitOrigin-RevId: a597efc46e00fc07cc6e5b71938c82f513c0b288
2025-01-24 09:05:44 +00:00
Jakob Ackermann 2640d4a566 [project-history] fixes for checking blob exists (#23051)
* [project-history] skip HEAD on blob without file-hash

* [project-history] only 404s indicate a missing blob

* [project-history] error checking for existing blob is fatal

* [project-history] always skip creating blob if already exists

Ignore createdBlob detail after performing check on blob. It might not
 exist on old operations that have been queued.

* [project-history] fix unit tests

* [project-history] migrate test to new fetch call

GitOrigin-RevId: 457963042e9d836647d3c122516329d4dc9b544f
2025-01-24 09:05:09 +00:00
Eric Mc Sween 30e5c95a27 Merge pull request #22871 from overleaf/em-history-flush-metrics
History flush metrics

GitOrigin-RevId: eb8b357427942e9816ad92ccd46c0dd8a65ab939
2025-01-17 09:05:00 +00:00
Andrew Rumble e64e69f539 Merge pull request #22652 from overleaf/ar-handle-filestore-404-when-copying-in-saas
[web] Do not do filestore copy in SAAS

GitOrigin-RevId: 83456ca57f706246a86b997a038968aecfbae4c5
2024-12-24 09:04:48 +00:00
Andrew Rumble e6a2caa088 Merge pull request #22421 from overleaf/ar-pass-created-blob-for-filetree-uploads
[web] Pass createdBlob through for file creation everywhere

GitOrigin-RevId: 880bebd0a55e351a6a61555a46e80faef22db7cb
2024-12-12 09:05:02 +00:00
Andrew Rumble ad8026038c Merge pull request #22397 from overleaf/revert-22320-revert-22213-ar-avoid-duplicate-blob-writes
Revert "Revert "Avoid duplicate blob writes""

GitOrigin-RevId: f6a0083c0edf9ede14dfb52dbf1809a843aa5ff4
2024-12-10 09:05:07 +00:00
Rebeka Dekany 7493936bad Merge pull request #22310 from overleaf/ar-bump-mongoose-8.8.3
[web] Upgrade mongoose to 8.8.3

GitOrigin-RevId: 5540f7384b74daafa01522c23633587fc56a7faa
2024-12-06 09:05:25 +00:00
Andrew Rumble ffa28b1a58 Merge pull request #22320 from overleaf/revert-22213-ar-avoid-duplicate-blob-writes
Revert "Avoid duplicate blob writes"

GitOrigin-RevId: 9f86bcea654cd3fa5f66fbdf42080e7f6e2861a7
2024-12-05 09:04:48 +00:00
Andrew Rumble 40b781eb0a Merge pull request #22213 from overleaf/ar-avoid-duplicate-blob-writes
Avoid duplicate blob writes

GitOrigin-RevId: 5e7a472b235bae5ef84389dbbe791189e951908a
2024-12-04 09:05:08 +00:00
Jakob Ackermann ce0d5fd383 Merge pull request #22177 from overleaf/jpa-file-view-hash-1
[web] migrate file-view to download from history-v1 (via web) 1/2

GitOrigin-RevId: b787e90c57af5e2704b06ba63502aa6fc09ea1df
2024-11-28 09:06:33 +00:00
Brian Gough 6672372828 remove sentry from backend services (#20752)
* remove sentry from backend services - no longer required

* Remove Sentry integration from logging manager

* Remove Sentry from clsi default settings

* Remove `initializeErrorReporting` in libraries/logger

* Remove `@sentry/node` from `libraries/logger`

---------

Co-authored-by: Antoine Clausse <antoine.clausse@overleaf.com>
GitOrigin-RevId: 8149a885f5258804b93ae39cde7b7333e992532a
2024-11-27 09:04:50 +00:00
Andrew Rumble 11213d85fe Merge pull request #21883 from overleaf/ar-disable-keep-alive-remaining-services
Disable keepAlive in all services

GitOrigin-RevId: 9a3bac37e3fb09ee64b05cfda300dfe8d8672aad
2024-11-15 09:05:52 +00:00
Andrew Rumble 73b17dd694 Merge pull request #21627 from overleaf/ar-node-20
[all] node 20 upgrade v2

GitOrigin-RevId: aed13a800894717a796062e3dd4050e045040258
2024-11-15 09:04:32 +00:00
Jakob Ackermann 805c4f8321 Merge pull request #21764 from overleaf/jpa-esm-build-scripts
[misc] prepare linting for ESM files in services

GitOrigin-RevId: 468983ebff40617158bac28514381a6aaf725893
2024-11-12 09:05:50 +00:00
Brian Gough fc84bdf68b Merge pull request #21751 from overleaf/bg-record-background-flush-operations
record background flush operations in project-history

GitOrigin-RevId: 80169b3f93a844d7162322ca21d605fffc64d687
2024-11-11 09:05:35 +00:00
Antoine Clausse 7f48c67512 Add prefer-node-protocol ESLint rule (#21532)
* Add `unicorn/prefer-node-protocol`

* Fix `unicorn/prefer-node-protocol` ESLint errors

* Run `npm run format:fix`

* Add sandboxed-module sourceTransformers in mocha setups

Fix `no such file or directory, open 'node:fs'` in `sandboxed-module`

* Remove `node:` in the SandboxedModule requires

* Fix new linting errors with `node:`

GitOrigin-RevId: 68f6e31e2191fcff4cb8058dd0a6914c14f59926
2024-11-11 09:04:51 +00:00
Andrew Rumble 57d12dfb17 Update to node 18.20.4
GitOrigin-RevId: 4b613ea3973cb7944c3d414ea91bcf367dbf50a6
2024-11-07 09:04:31 +00:00
Jakob Ackermann 5bb90dc6cb Merge pull request #21442 from overleaf/jpa-align-mongo-version
[misc] align all the mongodb dependency versions

GitOrigin-RevId: 1194fe57601af98bb61250a285bfc85b4b8179dd
2024-11-01 09:05:23 +00:00
Andrew Rumble 79d80cc266 Merge pull request #21462 from overleaf/revert-18828-ac-node-20
Revert "Upgrade Node from `18.20.2` to `20.17.0`"

GitOrigin-RevId: cc12c33b4588caa2959817a113547d20d437d6f4
2024-10-30 09:05:07 +00:00
Andrew Rumble a480df8a89 Merge pull request #18828 from overleaf/ac-node-20
Upgrade Node from `18.20.2` to `20.17.0`

GitOrigin-RevId: 0ba9b0921fc246a97d3da0aad7d48635c0e24ae8
2024-10-30 09:04:38 +00:00
Miguel Serrano bafda84420 Merge pull request #21072 from overleaf/msm-cleanup-post-labels
[project-history] Cleanup POST /labels

GitOrigin-RevId: 129f83ab72e4a9cd6f0f5fd7935fafa813ead919
2024-10-16 09:35:48 +00:00
Jakob Ackermann fab69a16c4 Merge pull request #20979 from overleaf/jpa-handle-error
[project-history] gracefully handle errors when loading Changes

GitOrigin-RevId: 214fadb3c779551ea90f38b0fdf77c58ed5df178
2024-10-15 08:05:15 +00:00
Jakob Ackermann 663f6605d7 Merge pull request #20978 from overleaf/jpa-fix-unit-tests
[project-history] fix unit tests for processing updates

GitOrigin-RevId: ce87296b9b97dcfd7598fbf25b3066b7c145fb6f
2024-10-15 08:05:10 +00:00
Domagoj Kriskovic 0998cc04df [project-history] fix diff view on file revert (#20769)
GitOrigin-RevId: 6c95dab9328ecab636272ad507505f5342ec8709
2024-10-14 11:06:36 +00:00
Miguel Serrano cd6631c105 Fix history label creation for anonymous users (#20200)
* Remove decaffeination artifacts in LabelsTests

* Remove decaffeination artifacts in LabelsManagerTests

* Fix label creation for anonymous users

* Update label creation route in MockProjectHistoryApi tests

* Support both endpoints for backwards compatibility

GitOrigin-RevId: 50ce1ba49388e50f147fb620e0425fea83301c9d
2024-10-14 10:57:33 +00:00
Liangjun Song cfd0b39b96 Merge pull request #20557 from overleaf/ls-bump-express
update express and other dependencies

GitOrigin-RevId: 22107eab0fa95d534ab07360548ae2b2123776db
2024-09-27 08:04:18 +00:00
Jakob Ackermann e3feafb7b0 Merge pull request #20332 from overleaf/jpa-small-runner
[misc] CI improvements

GitOrigin-RevId: 819d4d357fcb130f133337668644eedcf5f4a688
2024-09-26 08:04:38 +00:00
Jakob Ackermann 3ba3ffe56d Merge pull request #20277 from overleaf/jpa-ro-mirror-chunk
[misc] use Chunk for read-only mirror in frontend

GitOrigin-RevId: b97e520c7a63015179792542310d18f346752e92
2024-09-24 08:05:42 +00:00
Alf Eaton 2976fd0b44 Fix Dockerfile FromAsCasing warnings (#20388)
GitOrigin-RevId: 6d6be3ba17eec695907dde25da2dbe98818fae10
2024-09-23 08:04:52 +00:00
Eric Mc Sween 250acbf33c Merge pull request #20520 from overleaf/em-ts-import
Use new import JSDoc syntax for Typescript annotations

GitOrigin-RevId: 782456d637fc8f2de6163b5d70fabf06c1d74964
2024-09-23 08:04:47 +00:00
Jakob Ackermann 84be385550 Merge pull request #20331 from overleaf/jpa-build-scripts-sync
[misc] sync build scripts

GitOrigin-RevId: 343fd861b6c7b3d81b21babbd41bbb968f0fd448
2024-09-10 08:05:01 +00:00
Mathias Jakobsen 8245a95b4e Merge pull request #20266 from overleaf/mj-revert-linked-file
[project-history+web] Restore metadata when reverting file

GitOrigin-RevId: dbfa8202a2fe0bf077d8eedb51a2a13b9f1e8a83
2024-09-09 08:04:53 +00:00
Jakob Ackermann 989c48978a Merge pull request #20067 from overleaf/jpa-fix-old-blob-download
[web] project-history expects history-v1 id in blob requests

GitOrigin-RevId: d9170a12fc6070811c188b346dbac32accabbfd7
2024-08-22 14:01:12 +00:00
Jakob Ackermann 577497b655 Merge pull request #19842 from overleaf/jpa-ro-mirror-on-client
[misc] add readonly mirror of full project content on the client

GitOrigin-RevId: 940bd93bfd587f83ca383d10fc44579b38fc3e88
2024-08-22 08:05:13 +00:00
Mathias Jakobsen 654d96ace6 Merge pull request #19743 from overleaf/mj-restore-project
[web+project-history] Add project version reverting

GitOrigin-RevId: 0f77cec730393187d531c0c6561faaa652bebf29
2024-08-21 08:05:08 +00:00
Brian Gough bdbc17c374 Merge pull request #19951 from overleaf/bg-remove-unused-redis-from-project-history
Remove unused modules from project-history

GitOrigin-RevId: 668793c334b7ee1d6c16fccfc686bc758bf8ecb4
2024-08-16 08:04:40 +00:00