agw/overleaf-cep
1
0
Fork 0
mirror of https://github.com/yu-i-i/overleaf-cep.git synced 2026-05-24 09:39:35 +02:00
Code Issues Packages Projects Releases Wiki Activity
26,727 Commits 3 Branches 19 Tags
a313cf0bc7fc1237db173ce96d615bfcd4a6181f
Commit Graph

59 Commits

Author SHA1 Message Date
Miguel Serrano
3a35b8680e Merge pull request #28554 from overleaf/msm-force-s3-lib-storage-uploads 
[object-persistor] Use `@aws-sdk/lib-storage` for all uploads

GitOrigin-RevId: ab8e54a7bae843f9e6b05ed9cf936130a36b8c2f
 2025-10-08 08:05:55 +00:00
Miguel Serrano
176b1a562f Merge pull request #28543 from overleaf/msm-disable-retries-s3 
[object-persistor] fix disable retries on PutObjectCommand

GitOrigin-RevId: 7a812385e70781939f21325572c0eedb856e734f
 2025-09-18 08:06:07 +00:00
Miguel Serrano
f1dc6ec155 Upgrade aws-sdk to v3 (re-attempt) (#28410) 
* Upgrade `aws-sdk` to v3

Also vendored nodemailer-ses-transport

* Moved default region to CE settings

* Ensure timeout is added to `requestHandler` and `region` is populated

* fix unsigned header in signed URL request

The x-amz-acl header is not needed when using the signed request,
as ACL are already defined when creating the signed URL in
PutObjectCommand constructor.

* Add default AWS region for latexqc service

* remove unnecessary region in compose files

* Use AWS_REGION=us-west-2 for history-v1

* prevent retries uploading streams with PutObjectCommand

* Remove AWS SDK JS message suppression

GitOrigin-RevId: 6fda6f02160023ffed76143397bbd965f86a9509
 2025-09-18 08:05:22 +00:00
Miguel Serrano
8948e40b22 Revert "Merge pull request #27679 from overleaf/msm-aws-sdk-upgrade" (#28151) 
This reverts commit 4989ae920d8b7fd9e79623947b7c40bcc2e56d92.

GitOrigin-RevId: 541d95bfeaa7ce820e8af67f646f013fe4fe5d21
 2025-08-28 08:05:52 +00:00
Miguel Serrano
39381d808a Merge pull request #27679 from overleaf/msm-aws-sdk-upgrade 
Upgrade `aws-sdk` to v3

GitOrigin-RevId: 4989ae920d8b7fd9e79623947b7c40bcc2e56d92
 2025-08-28 08:05:36 +00:00
Jakob Ackermann
29045b9085 [web] download binary files in clsi from filestore via new endpoints (#27505) 
* [history-v1] use String.padStart instead of lodash.padStart

* [web] download binary files in clsi from filestore via new endpoints

* [server-ce] tests: Cypress.env() is parsing boolean values

* [server-ce] tests: run history migration as root

GitOrigin-RevId: bdf6c0e542531ccc4b3f13d2ed68ca0d31e580e9
 2025-07-31 08:05:28 +00:00
Andrew Rumble
4157f8ca00 Add a list directory method to the S3 persistor 
GitOrigin-RevId: 6ecff3eb457dc2168ca49ff9409bb09fa932781c
 2025-06-25 08:06:05 +00:00
Antoine Clausse
81941ff335 Update some dependencies so they're compatible with Node 22 (#25317) 
* `"@google-cloud/profiler": "^6.0.3"`

* `bin/npm update pprof`

* `bin/npm update nan`

* `bin/npm update @google-cloud/profiler`

* Ignore false positive of `@typescript-eslint/return-await`

> Returning an awaited value that is not a promise is not allowed
Though the function was promisified

GitOrigin-RevId: 24dbe3e8df2b55c0b9583ac79a61e0956ac3fac0
 2025-05-07 08:06:57 +00:00
Andrew Rumble
d2d556ddf6 Allow cached persistor to get size of a blob 
GitOrigin-RevId: 7e2c14381cf717c21325917ef4f78acb340a50d9
 2025-03-24 10:48:35 +00:00
Jakob Ackermann
26321c5dba [history-v1] block deletion of bucket prefix that is not a valid project prefix (#22889) 
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
GitOrigin-RevId: a3aff76a2e299b2e2ed030e34da0631fce51ad4b
 2025-02-07 09:05:01 +00:00
Jakob Ackermann
3d7254b419 Merge pull request #22153 from overleaf/jpa-backup-verifier-minimal 
[history-v1] backup-verifier-app: initial revision

GitOrigin-RevId: 922c9f94cb7ca7c129e38fd6961d42bdff819cd8
 2024-11-27 09:04:55 +00:00
Jakob Ackermann
0bd8729cc1 Merge pull request #22115 from overleaf/jpa-skip-get 
[history-v1] back_fill_file_hash: optimize obtaining the DEK

GitOrigin-RevId: 6fc5218737551ae4e6152414c32560f545767091
 2024-11-26 09:04:50 +00:00
Jakob Ackermann
ed27af11f8 Merge pull request #22107 from overleaf/jpa-error-handling 
[history-v1] back_fill_file_hash: error handling

GitOrigin-RevId: 2dbf52d015206cf42ca434b1f16221d6e0f2cc2b
 2024-11-26 09:04:45 +00:00
Jakob Ackermann
8e74d3c58c Merge pull request #21947 from overleaf/bg-jpa-back-fill-script-tweaks 
[history-v1] back_fill_file_hash: performance tweaks

GitOrigin-RevId: c3d0c7906707fc902addcde64eaf41c24ceeece7
 2024-11-20 09:04:51 +00:00
Jakob Ackermann
97b39ef98f Merge pull request #21681 from overleaf/jpa-back-fill-file-hash 
[history-v1] add script for back filling hash into project file-tree

GitOrigin-RevId: 70ab8c72f3eea1062708e4471b4ad1c60317ad26
 2024-11-18 09:05:39 +00:00
Jakob Ackermann
d6e94d2586 Merge pull request #21908 from overleaf/jpa-storage-class 
[object-persistor] s3: add support for setting storage class on upload

GitOrigin-RevId: dc5480a95ec7f1551f08848f7fa45f760d97ab22
 2024-11-18 09:05:10 +00:00
Jakob Ackermann
2be894c18a Merge pull request #21892 from overleaf/jpa-auto-gunzip 
[object-persistor] add autoGunzip option to getObjectStream

GitOrigin-RevId: 21cd6c9ab349017ddb28d165238371d967ab9a59
 2024-11-15 09:06:32 +00:00
Jakob Ackermann
bd855044af Merge pull request #21891 from overleaf/jpa-consistent-getObjectSize-type 
[object-persistor] gcs: return a number from getObjectSize

GitOrigin-RevId: f3e0e3269e789b0077c82014c42a4ca63cd84b39
 2024-11-15 09:06:28 +00:00
Jakob Ackermann
38b83d7edf Merge pull request #21881 from overleaf/jpa-missing-break 
[object-persistor] s3SSEC: missing break statement in rootKEK discovery

GitOrigin-RevId: 3116a36006c9c7ae0b15593d8938fd559794a898
 2024-11-15 09:06:14 +00:00
Antoine Clausse
7f48c67512 Add prefer-node-protocol ESLint rule (#21532) 
* Add `unicorn/prefer-node-protocol`

* Fix `unicorn/prefer-node-protocol` ESLint errors

* Run `npm run format:fix`

* Add sandboxed-module sourceTransformers in mocha setups

Fix `no such file or directory, open 'node:fs'` in `sandboxed-module`

* Remove `node:` in the SandboxedModule requires

* Fix new linting errors with `node:`

GitOrigin-RevId: 68f6e31e2191fcff4cb8058dd0a6914c14f59926
 2024-11-11 09:04:51 +00:00
Jakob Ackermann
122d89a831 Merge pull request #21660 from overleaf/jpa-s3-https 
[object-persistor] s3: simplify using a custom CA for HTTPS endpoints

GitOrigin-RevId: 2c6a5312a842582e5e40e917ccc586392087cb7a
 2024-11-08 09:07:00 +00:00
Jakob Ackermann
65dc6bf940 Merge pull request #21631 from overleaf/jpa-live-s3 
[object-persistor] s3SSEC: make compatible with AWS S3

GitOrigin-RevId: 3cd1f5ec43141f15defe081fc916d2ca2d78ca01
 2024-11-08 09:06:55 +00:00
Jakob Ackermann
d5478c11ea Merge pull request #21623 from overleaf/jpa-s3-ssec-static-dek-path 
[object-persistor] s3SSEC: use static path for DEK inside project folder

GitOrigin-RevId: 0c43ef8964c16d3e7638f17ff47b1c4641e439df
 2024-11-08 09:06:51 +00:00
Jakob Ackermann
a67560c26b Merge pull request #21589 from overleaf/jpa-s3-ssec-hkdf 
[object-persistor] s3SSEC: add HKDF layer to KEK

GitOrigin-RevId: 1def9e378e1dfd90f449ad392b0db2101584e17f
 2024-11-08 09:06:46 +00:00
Jakob Ackermann
bec73ddfae Merge pull request #21543 from overleaf/jpa-s3-seec-kek-rotation 
[object-persistor] s3SSEC: add support for (automatic) KEK rotation

GitOrigin-RevId: 315082e894c74e276a8efbc46b41ec7e102f9010
 2024-11-08 09:06:42 +00:00
Jakob Ackermann
859901ac0c Merge pull request #21380 from overleaf/jpa-s3-ssec-backend 
[object-persistor] add backend for SSE-C with S3 using KEK and DEK

GitOrigin-RevId: 9676f5cd5e08107c8c284b68b8d450a1c05bf1b1
 2024-11-08 09:06:37 +00:00
Jakob Ackermann
7b3e39f63f Merge pull request #21375 from overleaf/jpa-if-none-match 
[object-persistor] add support for ifNoneMatch=* in sendStream

GitOrigin-RevId: 268f054ac1b6452105b02757cdec32bad00702fd
 2024-11-01 09:05:11 +00:00
Jakob Ackermann
baaac44172 Merge pull request #20302 from overleaf/jpa-object-persistor-metrics 
[object-persistor] add more metrics to getObjectStream and sendStream

GitOrigin-RevId: 9fe6b9d205de6ad27838f91d92d2b3a3d6c2f129
 2024-09-24 08:05:54 +00:00
Jakob Ackermann
8fa676082e Merge pull request #20299 from overleaf/jpa-object-persistor-metrics-dep 
[object-persistor] depend on @overleaf/metrics directly

GitOrigin-RevId: eb0c07af8101d44def14154abb552bc77254e074
 2024-09-24 08:05:50 +00:00
Andrew Rumble
71187a51ba Merge pull request #18289 from overleaf/ac-ar-eslint-return-await 
Add ESLint rule @typescript-eslint/return-await to backend services

GitOrigin-RevId: 75e3e32597827fcc852e69d479515fc72e8f45e4
 2024-05-27 10:22:49 +00:00
Jakob Ackermann
06c839a230 Merge pull request #17237 from overleaf/jpa-s3-iam 
[object-persistor] add support for IAM credentials in S3Persistor

GitOrigin-RevId: 4ff34082ca557b17d8a7803f700ee704a3d184c6
 2024-02-22 09:04:52 +00:00
Christopher Hoskin
cc35bf4447 Merge pull request #13362 from overleaf/csh-issue-13004-Always-Retry-during-GCS-delete 
Always retry during GCS delete in history-v1 cron

GitOrigin-RevId: 1b64133dd0b9982f18124d9baa6e29bcdf6fa1f3
 2023-07-17 10:28:33 +00:00
Christopher Hoskin
cbaf1c3556 Merge pull request #13256 from overleaf/csh-issue-13004-max-retries 
Increase the maximum retries for the delete-old-history-chunks cron job

GitOrigin-RevId: 8054ac76bbae24d0a3ba032c50c9647c044aef27
 2023-06-02 08:07:17 +00:00
Christopher Hoskin
5fbd5e21a2 Merge pull request #13274 from overleaf/csh-issue-13004-cleanup-custom-code 
Remove apiScheme from GCS

GitOrigin-RevId: c948287cb00c7bc432c797bfdfa92a79b84d1bed
 2023-06-02 08:05:10 +00:00
Jakob Ackermann
2760ad35fc Merge pull request #12899 from overleaf/jpa-object-persistor-pipe 
[object-persistor] move away from manual .pipe()

GitOrigin-RevId: 5407d48fbbb026ba81f43c891499bd3a5ff59518
 2023-05-23 08:05:12 +00:00
Jakob Ackermann
0b82db0abe Merge pull request #11804 from overleaf/jpa-fix-fs-persistor-nested-layout 
[object-persistor] forward settings.useSubdirectories to FSPersistor

GitOrigin-RevId: 92cef51c392cfeaebbcca2597740562c1f3fc95f
 2023-02-13 14:41:08 +00:00
Jakob Ackermann
9cf1373777 Merge pull request #11769 from overleaf/jpa-s3-config 
[server-ce] add support for s3 backend in filestore

GitOrigin-RevId: 20998547326e75acf22562d2964db78ae261b6bf
 2023-02-10 09:12:23 +00:00
Eric Mc Sween
4325f1d947 Merge pull request #11179 from overleaf/em-upgrade-gcs-lib 
Upgrade @google-cloud/storage library to 6.9.0

GitOrigin-RevId: 0e80dcccca4679ffe28d5b16512a691e58694e2b
 2023-01-18 09:05:00 +00:00
Eric Mc Sween
be081856a8 Merge pull request #10261 from overleaf/em-object-persistor-md5 
Compute the md5 hash as we receive the stream

GitOrigin-RevId: 6c0c0a31165c5e1320e7f7a6dbf59a0d3576d3c4
 2022-11-11 09:03:42 +00:00
Eric Mc Sween
3ee794da47 Merge pull request #10259 from overleaf/em-object-persistor-tempfile 
Atomic update of files in FS backend of object-persistor

GitOrigin-RevId: b57c0c1a7d6299affd00e174cb7ae75ae711c5d3
 2022-11-11 09:03:35 +00:00
Eric Mc Sween
a98f752b99 Merge pull request #10249 from overleaf/em-object-persistor-subdirs 
Add useSubdirectories option to object-persistor FS backend

GitOrigin-RevId: dc4f0a57e892ffa5be0c88b8baf7efce683ddfc1
 2022-11-11 09:03:27 +00:00
Eric Mc Sween
8fb3edbecd Merge pull request #10245 from overleaf/em-object-persistor-promises-api 
Use Node 16 promises APIs in object-persistor

GitOrigin-RevId: 8520da0c0678c17b22a9164d15d61c2b57af8f4e
 2022-11-11 09:03:19 +00:00
Jakob Ackermann
81b935e317 Merge pull request #7977 from overleaf/jpa-use-local-libraries 
[misc] use local libraries

GitOrigin-RevId: ec16907c0ce4f9a79689c522f37d52502559e7b2
 2022-05-17 08:05:52 +00:00
Jakob Ackermann
4b308553be Merge pull request #6120 from overleaf/jpa-same-linting-packages 
[misc] move the linting setup to the root of the monorepo

GitOrigin-RevId: 1633e2a58598add0b727738cd3bfba0ab7bae781
 2021-12-17 09:03:06 +00:00
Brian Gough
d630d4a37c Merge pull request #5258 from overleaf/revert-5214-bg-migrate-node-uuid-to-uuid 
Revert "migrate node-uuid to uuid"

GitOrigin-RevId: f512eb1040e2ce6d87774de9c1fc884594480cef
 2021-09-28 08:04:11 +00:00
Brian Gough
9ee1e29286 Merge pull request #5214 from overleaf/bg-migrate-node-uuid-to-uuid 
migrate node-uuid to uuid

GitOrigin-RevId: 742fd4d3f039c6a026b3d35c3b0a7d54c00ca183
 2021-09-27 08:02:54 +00:00
Brian Gough
d4563c8786 Merge pull request #4866 from overleaf/bg-gcs-delete-directory-batch 
make object-persistor delete gcs files in batches

GitOrigin-RevId: 8ebc892c5f6eb30507ec41d5d3a108e650af5cac
 2021-08-27 08:03:25 +00:00
Jakob Ackermann
5679ba894b Merge pull request #4800 from overleaf/jpa-object-persistor-fix-404-handling 
[object-persistor] handle 404s of the entire delete operation in gcs

GitOrigin-RevId: 41c5e70af069361a21adb53a6166081741666887
 2021-08-19 08:02:43 +00:00
Jakob Ackermann
a383dc316c [misc] run format:fix 2021-07-27 16:53:06 +01:00
Eric Mc Sween
0fdfa706f7 Make 404 error handling on deletes more specific 
The GCS persistor error handler ignores 404 errors on deletes. However,
the net is too wide cast, and 404 errors coming from the dual-bucket
lifecycle mechanism are also ignored.
 2020-10-15 10:35:52 -04:00