agw/overleaf-cep
1
0
Fork 0
mirror of https://github.com/yu-i-i/overleaf-cep.git synced 2026-05-24 17:51:51 +02:00
Code Issues Packages Projects Releases Wiki Activity
27,183 Commits 3 Branches 19 Tags
60bb53bbfa2cd0ecfc96ffbee267a8f4823adb8e
Commit Graph

64 Commits

Author SHA1 Message Date
Jakob Ackermann
95f0dac343 [clsi-cache] add sharded disk layout (#30763) 
* [clsi-cache] remove old DiskCrawler

* [clsi-cache] use helpers for populating state during initial crawl

* [clsi-cache] use path helpers everywhere

* [clsi-cache] create project folder as part of the initial crawl

* [clsi-cache] add sharded disk layout

* [k8s] clsi-cache: switch clsi-cache-zone-b-shard-0 to new disk layout

* [clsi-cache] tweak logging context

* [monorepo] add bin/format_project_key util

* [k8s] clsi-cache: use a single nginx config file

GitOrigin-RevId: bb49146502cc6201eef210efae33355321efd77b
 2026-01-15 14:19:28 +00:00
Jakob Ackermann
18cff6e1ac [object-persistor] import ProjectKey helper from history-v1 (#30600) 
GitOrigin-RevId: c72aa4bf91569904a2072c74d6ed2f3c764d97bb
 2026-01-13 09:06:55 +00:00
Brian Gough
7a57ef00cb Merge pull request #30045 from overleaf/bg-object-persistor-make-list-directory-safer 
Improve safety of object persistor

GitOrigin-RevId: bced9814de6613b388ca288a5f72cd42cff6c1d3
 2025-12-05 09:06:42 +00:00
John Lees-Miller
b514ebcc8e Support analytics data in user export bundle 
GitOrigin-RevId: 49739297f40831cf035e2a9d4f3343a8cb2d7fdb
 2025-11-25 09:06:20 +00:00
Andrew Rumble
5a4592b153 Merge pull request #29720 from overleaf/renovate-npm-glob-vulnerability 
[Core] Update dependency glob to v11.1.0 [SECURITY] - abandoned

GitOrigin-RevId: 866612e84e2afd3edc97c244e22faeaff22484cf
 2025-11-20 09:05:44 +00:00
Miguel Serrano
3a35b8680e Merge pull request #28554 from overleaf/msm-force-s3-lib-storage-uploads 
[object-persistor] Use `@aws-sdk/lib-storage` for all uploads

GitOrigin-RevId: ab8e54a7bae843f9e6b05ed9cf936130a36b8c2f
 2025-10-08 08:05:55 +00:00
Miguel Serrano
176b1a562f Merge pull request #28543 from overleaf/msm-disable-retries-s3 
[object-persistor] fix disable retries on PutObjectCommand

GitOrigin-RevId: 7a812385e70781939f21325572c0eedb856e734f
 2025-09-18 08:06:07 +00:00
Miguel Serrano
f1dc6ec155 Upgrade aws-sdk to v3 (re-attempt) (#28410) 
* Upgrade `aws-sdk` to v3

Also vendored nodemailer-ses-transport

* Moved default region to CE settings

* Ensure timeout is added to `requestHandler` and `region` is populated

* fix unsigned header in signed URL request

The x-amz-acl header is not needed when using the signed request,
as ACL are already defined when creating the signed URL in
PutObjectCommand constructor.

* Add default AWS region for latexqc service

* remove unnecessary region in compose files

* Use AWS_REGION=us-west-2 for history-v1

* prevent retries uploading streams with PutObjectCommand

* Remove AWS SDK JS message suppression

GitOrigin-RevId: 6fda6f02160023ffed76143397bbd965f86a9509
 2025-09-18 08:05:22 +00:00
Miguel Serrano
8948e40b22 Revert "Merge pull request #27679 from overleaf/msm-aws-sdk-upgrade" (#28151) 
This reverts commit 4989ae920d8b7fd9e79623947b7c40bcc2e56d92.

GitOrigin-RevId: 541d95bfeaa7ce820e8af67f646f013fe4fe5d21
 2025-08-28 08:05:52 +00:00
Miguel Serrano
39381d808a Merge pull request #27679 from overleaf/msm-aws-sdk-upgrade 
Upgrade `aws-sdk` to v3

GitOrigin-RevId: 4989ae920d8b7fd9e79623947b7c40bcc2e56d92
 2025-08-28 08:05:36 +00:00
Jakob Ackermann
29045b9085 [web] download binary files in clsi from filestore via new endpoints (#27505) 
* [history-v1] use String.padStart instead of lodash.padStart

* [web] download binary files in clsi from filestore via new endpoints

* [server-ce] tests: Cypress.env() is parsing boolean values

* [server-ce] tests: run history migration as root

GitOrigin-RevId: bdf6c0e542531ccc4b3f13d2ed68ca0d31e580e9
 2025-07-31 08:05:28 +00:00
Andrew Rumble
4157f8ca00 Add a list directory method to the S3 persistor 
GitOrigin-RevId: 6ecff3eb457dc2168ca49ff9409bb09fa932781c
 2025-06-25 08:06:05 +00:00
Antoine Clausse
81941ff335 Update some dependencies so they're compatible with Node 22 (#25317) 
* `"@google-cloud/profiler": "^6.0.3"`

* `bin/npm update pprof`

* `bin/npm update nan`

* `bin/npm update @google-cloud/profiler`

* Ignore false positive of `@typescript-eslint/return-await`

> Returning an awaited value that is not a promise is not allowed
Though the function was promisified

GitOrigin-RevId: 24dbe3e8df2b55c0b9583ac79a61e0956ac3fac0
 2025-05-07 08:06:57 +00:00
Andrew Rumble
d2d556ddf6 Allow cached persistor to get size of a blob 
GitOrigin-RevId: 7e2c14381cf717c21325917ef4f78acb340a50d9
 2025-03-24 10:48:35 +00:00
Jakob Ackermann
26321c5dba [history-v1] block deletion of bucket prefix that is not a valid project prefix (#22889) 
Co-authored-by: Brian Gough <brian.gough@overleaf.com>
GitOrigin-RevId: a3aff76a2e299b2e2ed030e34da0631fce51ad4b
 2025-02-07 09:05:01 +00:00
Jakob Ackermann
3d7254b419 Merge pull request #22153 from overleaf/jpa-backup-verifier-minimal 
[history-v1] backup-verifier-app: initial revision

GitOrigin-RevId: 922c9f94cb7ca7c129e38fd6961d42bdff819cd8
 2024-11-27 09:04:55 +00:00
Jakob Ackermann
0bd8729cc1 Merge pull request #22115 from overleaf/jpa-skip-get 
[history-v1] back_fill_file_hash: optimize obtaining the DEK

GitOrigin-RevId: 6fc5218737551ae4e6152414c32560f545767091
 2024-11-26 09:04:50 +00:00
Jakob Ackermann
ed27af11f8 Merge pull request #22107 from overleaf/jpa-error-handling 
[history-v1] back_fill_file_hash: error handling

GitOrigin-RevId: 2dbf52d015206cf42ca434b1f16221d6e0f2cc2b
 2024-11-26 09:04:45 +00:00
Jakob Ackermann
8e74d3c58c Merge pull request #21947 from overleaf/bg-jpa-back-fill-script-tweaks 
[history-v1] back_fill_file_hash: performance tweaks

GitOrigin-RevId: c3d0c7906707fc902addcde64eaf41c24ceeece7
 2024-11-20 09:04:51 +00:00
Jakob Ackermann
97b39ef98f Merge pull request #21681 from overleaf/jpa-back-fill-file-hash 
[history-v1] add script for back filling hash into project file-tree

GitOrigin-RevId: 70ab8c72f3eea1062708e4471b4ad1c60317ad26
 2024-11-18 09:05:39 +00:00
Jakob Ackermann
d6e94d2586 Merge pull request #21908 from overleaf/jpa-storage-class 
[object-persistor] s3: add support for setting storage class on upload

GitOrigin-RevId: dc5480a95ec7f1551f08848f7fa45f760d97ab22
 2024-11-18 09:05:10 +00:00
Jakob Ackermann
2be894c18a Merge pull request #21892 from overleaf/jpa-auto-gunzip 
[object-persistor] add autoGunzip option to getObjectStream

GitOrigin-RevId: 21cd6c9ab349017ddb28d165238371d967ab9a59
 2024-11-15 09:06:32 +00:00
Jakob Ackermann
bd855044af Merge pull request #21891 from overleaf/jpa-consistent-getObjectSize-type 
[object-persistor] gcs: return a number from getObjectSize

GitOrigin-RevId: f3e0e3269e789b0077c82014c42a4ca63cd84b39
 2024-11-15 09:06:28 +00:00
Jakob Ackermann
38b83d7edf Merge pull request #21881 from overleaf/jpa-missing-break 
[object-persistor] s3SSEC: missing break statement in rootKEK discovery

GitOrigin-RevId: 3116a36006c9c7ae0b15593d8938fd559794a898
 2024-11-15 09:06:14 +00:00
Antoine Clausse
7f48c67512 Add prefer-node-protocol ESLint rule (#21532) 
* Add `unicorn/prefer-node-protocol`

* Fix `unicorn/prefer-node-protocol` ESLint errors

* Run `npm run format:fix`

* Add sandboxed-module sourceTransformers in mocha setups

Fix `no such file or directory, open 'node:fs'` in `sandboxed-module`

* Remove `node:` in the SandboxedModule requires

* Fix new linting errors with `node:`

GitOrigin-RevId: 68f6e31e2191fcff4cb8058dd0a6914c14f59926
 2024-11-11 09:04:51 +00:00
Jakob Ackermann
122d89a831 Merge pull request #21660 from overleaf/jpa-s3-https 
[object-persistor] s3: simplify using a custom CA for HTTPS endpoints

GitOrigin-RevId: 2c6a5312a842582e5e40e917ccc586392087cb7a
 2024-11-08 09:07:00 +00:00
Jakob Ackermann
65dc6bf940 Merge pull request #21631 from overleaf/jpa-live-s3 
[object-persistor] s3SSEC: make compatible with AWS S3

GitOrigin-RevId: 3cd1f5ec43141f15defe081fc916d2ca2d78ca01
 2024-11-08 09:06:55 +00:00
Jakob Ackermann
d5478c11ea Merge pull request #21623 from overleaf/jpa-s3-ssec-static-dek-path 
[object-persistor] s3SSEC: use static path for DEK inside project folder

GitOrigin-RevId: 0c43ef8964c16d3e7638f17ff47b1c4641e439df
 2024-11-08 09:06:51 +00:00
Jakob Ackermann
a67560c26b Merge pull request #21589 from overleaf/jpa-s3-ssec-hkdf 
[object-persistor] s3SSEC: add HKDF layer to KEK

GitOrigin-RevId: 1def9e378e1dfd90f449ad392b0db2101584e17f
 2024-11-08 09:06:46 +00:00
Jakob Ackermann
bec73ddfae Merge pull request #21543 from overleaf/jpa-s3-seec-kek-rotation 
[object-persistor] s3SSEC: add support for (automatic) KEK rotation

GitOrigin-RevId: 315082e894c74e276a8efbc46b41ec7e102f9010
 2024-11-08 09:06:42 +00:00
Jakob Ackermann
859901ac0c Merge pull request #21380 from overleaf/jpa-s3-ssec-backend 
[object-persistor] add backend for SSE-C with S3 using KEK and DEK

GitOrigin-RevId: 9676f5cd5e08107c8c284b68b8d450a1c05bf1b1
 2024-11-08 09:06:37 +00:00
Jakob Ackermann
7b3e39f63f Merge pull request #21375 from overleaf/jpa-if-none-match 
[object-persistor] add support for ifNoneMatch=* in sendStream

GitOrigin-RevId: 268f054ac1b6452105b02757cdec32bad00702fd
 2024-11-01 09:05:11 +00:00
Jakob Ackermann
baaac44172 Merge pull request #20302 from overleaf/jpa-object-persistor-metrics 
[object-persistor] add more metrics to getObjectStream and sendStream

GitOrigin-RevId: 9fe6b9d205de6ad27838f91d92d2b3a3d6c2f129
 2024-09-24 08:05:54 +00:00
Jakob Ackermann
8fa676082e Merge pull request #20299 from overleaf/jpa-object-persistor-metrics-dep 
[object-persistor] depend on @overleaf/metrics directly

GitOrigin-RevId: eb0c07af8101d44def14154abb552bc77254e074
 2024-09-24 08:05:50 +00:00
Andrew Rumble
71187a51ba Merge pull request #18289 from overleaf/ac-ar-eslint-return-await 
Add ESLint rule @typescript-eslint/return-await to backend services

GitOrigin-RevId: 75e3e32597827fcc852e69d479515fc72e8f45e4
 2024-05-27 10:22:49 +00:00
Jakob Ackermann
06c839a230 Merge pull request #17237 from overleaf/jpa-s3-iam 
[object-persistor] add support for IAM credentials in S3Persistor

GitOrigin-RevId: 4ff34082ca557b17d8a7803f700ee704a3d184c6
 2024-02-22 09:04:52 +00:00
Christopher Hoskin
cc35bf4447 Merge pull request #13362 from overleaf/csh-issue-13004-Always-Retry-during-GCS-delete 
Always retry during GCS delete in history-v1 cron

GitOrigin-RevId: 1b64133dd0b9982f18124d9baa6e29bcdf6fa1f3
 2023-07-17 10:28:33 +00:00
Christopher Hoskin
cbaf1c3556 Merge pull request #13256 from overleaf/csh-issue-13004-max-retries 
Increase the maximum retries for the delete-old-history-chunks cron job

GitOrigin-RevId: 8054ac76bbae24d0a3ba032c50c9647c044aef27
 2023-06-02 08:07:17 +00:00
Christopher Hoskin
5fbd5e21a2 Merge pull request #13274 from overleaf/csh-issue-13004-cleanup-custom-code 
Remove apiScheme from GCS

GitOrigin-RevId: c948287cb00c7bc432c797bfdfa92a79b84d1bed
 2023-06-02 08:05:10 +00:00
Jakob Ackermann
2760ad35fc Merge pull request #12899 from overleaf/jpa-object-persistor-pipe 
[object-persistor] move away from manual .pipe()

GitOrigin-RevId: 5407d48fbbb026ba81f43c891499bd3a5ff59518
 2023-05-23 08:05:12 +00:00
Jakob Ackermann
0b82db0abe Merge pull request #11804 from overleaf/jpa-fix-fs-persistor-nested-layout 
[object-persistor] forward settings.useSubdirectories to FSPersistor

GitOrigin-RevId: 92cef51c392cfeaebbcca2597740562c1f3fc95f
 2023-02-13 14:41:08 +00:00
Jakob Ackermann
9cf1373777 Merge pull request #11769 from overleaf/jpa-s3-config 
[server-ce] add support for s3 backend in filestore

GitOrigin-RevId: 20998547326e75acf22562d2964db78ae261b6bf
 2023-02-10 09:12:23 +00:00
Eric Mc Sween
4325f1d947 Merge pull request #11179 from overleaf/em-upgrade-gcs-lib 
Upgrade @google-cloud/storage library to 6.9.0

GitOrigin-RevId: 0e80dcccca4679ffe28d5b16512a691e58694e2b
 2023-01-18 09:05:00 +00:00
Eric Mc Sween
be081856a8 Merge pull request #10261 from overleaf/em-object-persistor-md5 
Compute the md5 hash as we receive the stream

GitOrigin-RevId: 6c0c0a31165c5e1320e7f7a6dbf59a0d3576d3c4
 2022-11-11 09:03:42 +00:00
Eric Mc Sween
3ee794da47 Merge pull request #10259 from overleaf/em-object-persistor-tempfile 
Atomic update of files in FS backend of object-persistor

GitOrigin-RevId: b57c0c1a7d6299affd00e174cb7ae75ae711c5d3
 2022-11-11 09:03:35 +00:00
Eric Mc Sween
a98f752b99 Merge pull request #10249 from overleaf/em-object-persistor-subdirs 
Add useSubdirectories option to object-persistor FS backend

GitOrigin-RevId: dc4f0a57e892ffa5be0c88b8baf7efce683ddfc1
 2022-11-11 09:03:27 +00:00
Eric Mc Sween
8fb3edbecd Merge pull request #10245 from overleaf/em-object-persistor-promises-api 
Use Node 16 promises APIs in object-persistor

GitOrigin-RevId: 8520da0c0678c17b22a9164d15d61c2b57af8f4e
 2022-11-11 09:03:19 +00:00
Jakob Ackermann
81b935e317 Merge pull request #7977 from overleaf/jpa-use-local-libraries 
[misc] use local libraries

GitOrigin-RevId: ec16907c0ce4f9a79689c522f37d52502559e7b2
 2022-05-17 08:05:52 +00:00
Jakob Ackermann
4b308553be Merge pull request #6120 from overleaf/jpa-same-linting-packages 
[misc] move the linting setup to the root of the monorepo

GitOrigin-RevId: 1633e2a58598add0b727738cd3bfba0ab7bae781
 2021-12-17 09:03:06 +00:00
Brian Gough
d630d4a37c Merge pull request #5258 from overleaf/revert-5214-bg-migrate-node-uuid-to-uuid 
Revert "migrate node-uuid to uuid"

GitOrigin-RevId: f512eb1040e2ce6d87774de9c1fc884594480cef
 2021-09-28 08:04:11 +00:00