Commit Graph

477 Commits

Author SHA1 Message Date
Antoine Clausse
4cf55bf99c Add prefer-node-protocol ESLint rule (#21532)
* Add `unicorn/prefer-node-protocol`

* Fix `unicorn/prefer-node-protocol` ESLint errors

* Run `npm run format:fix`

* Add sandboxed-module sourceTransformers in mocha setups

Fix `no such file or directory, open 'node:fs'` in `sandboxed-module`

* Remove `node:` in the SandboxedModule requires

* Fix new linting errors with `node:`

GitOrigin-RevId: 68f6e31e2191fcff4cb8058dd0a6914c14f59926
2024-11-11 09:04:51 +00:00
andrew rumble
589b191439 Capture more logging information for output.zip errors
GitOrigin-RevId: a2f15d05100dfdba06778a1a795c8eabefd2666d
2024-09-24 08:05:33 +00:00
andrew rumble
df26ad1cf8 Stop waiting for finalize
Finalize promise will only resolve when the archive is closed. We want
to stream as soon as we have the data so this does not suit us. We want
to log errors that are thrown due to finalize, these should be
propogated by archiver to the response already.

GitOrigin-RevId: 4f9d727a00ead1be3caee62e1e0adba069a545c7
2024-09-24 08:05:28 +00:00
andrew rumble
142185ace3 Get full path only when adding file to archive
This allows us to:
* Get files that are in subdirectories (not known to happen but might as
well support)
* Removes need to sanitise missing_files output

GitOrigin-RevId: 4b27b7f1b506578a6d713ea23979aa9370dfdcb0
2024-09-24 08:05:24 +00:00
andrew rumble
d5b1227b4b Rename files to reflect refactoring elsewhere
GitOrigin-RevId: 335f876913b7868cf69dcd55dd156261edd98945
2024-09-24 08:05:19 +00:00
andrew rumble
f7af8c855c Improve stream error safety
GitOrigin-RevId: de4c512a62d304b3eeb2a1521aac173fa93d8411
2024-09-24 08:05:15 +00:00
andrew rumble
9bbc735de1 Move logging into ArchiveManager
GitOrigin-RevId: 71a28a0215c5f1a6975c9e2fcdcd476513df1cbc
2024-09-24 08:05:10 +00:00
andrew rumble
a9a3c3625a Remove unnecessary symlink check
GitOrigin-RevId: 08d7295403a258818276b9fbd7666a20fbc2e00f
2024-09-24 08:05:06 +00:00
andrew rumble
b102c0f2ed Share ignore logic between web and clsi
GitOrigin-RevId: c4ae306413efb71989d2db410c3859cb69b3d5de
2024-09-24 08:05:01 +00:00
andrew rumble
29abcbba8a Remove files param from output.zip endpoint
GitOrigin-RevId: 300cd870578237ad8e61be4e9e41991f884aa754
2024-09-24 08:04:57 +00:00
andrew rumble
8dd249f786 Allow all files to be in zip (in same directory)
GitOrigin-RevId: 14645a0c3db88faf00e2718b9574b5892ac3efcb
2024-09-24 08:04:53 +00:00
Liangjun Song
3cdeb6e31a remove dry run (#19820)
GitOrigin-RevId: b92e08da6654cdd37314f7c52a6946cc7ec8983a
2024-08-08 08:04:17 +00:00
Liangjun Song
bc0d496fe1 limit the number of concurrent compile requests in clsi (#19717)
GitOrigin-RevId: 17909a4dd0717ea4a75288f734ddef19c7d6592e
2024-08-06 08:04:59 +00:00
Liangjun Song
49524e4212 Revert "Revert "log clsi concurrency (#19651)" (#19655)" (#19656)
This reverts commit 3719f468a6336ff4bf5126f3e5d79dcde172b7ea.

GitOrigin-RevId: 6f4809f3c607c62892fc1e110eeaa2a1e7a124a4
2024-07-31 08:04:28 +00:00
Liangjun Song
663c68f767 Revert "log clsi concurrency (#19651)" (#19655)
This reverts commit 504f6f9cc3d4b7386dbcde86f76c1cf3a13e944f.

GitOrigin-RevId: 3719f468a6336ff4bf5126f3e5d79dcde172b7ea
2024-07-30 08:05:52 +00:00
Liangjun Song
0719638cc4 log clsi concurrency (#19651)
GitOrigin-RevId: 504f6f9cc3d4b7386dbcde86f76c1cf3a13e944f
2024-07-30 08:05:26 +00:00
Jakob Ackermann
38a28561b4 Merge pull request #19296 from overleaf/jpa-issue-19290-3
[clsi] atomic writing of LaTeXMk output

GitOrigin-RevId: d81c497370587b98fc7ad282035cd59b0ae09ec8
2024-07-15 09:01:04 +00:00
Jakob Ackermann
ebbc35536c Merge pull request #19293 from overleaf/jpa-issue-19290-2
[clsi] fix parsing of the requested file in symlink validation

GitOrigin-RevId: 86cfe8d62bb99ed6844faee0ff4af507e571e04d
2024-07-15 09:00:59 +00:00
Andrew Rumble
9cbce4fccc Merge pull request #18474 from overleaf/ar-return-build-id-from-clsi-after-compile
[clsi] Return buildId after compiles

GitOrigin-RevId: 872048f4fea8f5a00b817e29bd26a444d179a45f
2024-05-27 10:24:06 +00:00
Andrew Rumble
83a628c355 Merge pull request #18289 from overleaf/ac-ar-eslint-return-await
Add ESLint rule @typescript-eslint/return-await to backend services

GitOrigin-RevId: 75e3e32597827fcc852e69d479515fc72e8f45e4
2024-05-27 10:22:49 +00:00
Christopher Hoskin
011e6ea7ec Merge pull request #18397 from overleaf/em-revert-download-all-link
Revert "Merge pull request #18190 from overleaf/ar-add-download-all-l…

GitOrigin-RevId: 681eb2734636d76558e682dc85083bfcaa6b7d2d
2024-05-17 08:05:10 +00:00
Brian Gough
9365e08b55 Merge pull request #16775 from overleaf/bg-clsi-timeout-use-fetchutils-connect-timeout
Update UrlFetcher to use custom agent with connect timeout

GitOrigin-RevId: c92dc27a0e682b0b79859b3521bf64cbf8485345
2024-05-16 08:05:18 +00:00
Andrew Rumble
031b1a6d4e Merge pull request #18190 from overleaf/ar-add-download-all-link
[web] Add download all link for output files

GitOrigin-RevId: a275f17b6acf3130250d91462ba01588b93b62f5
2024-05-16 08:05:04 +00:00
Andrew Rumble
7da2d37adb Merge pull request #18148 from overleaf/ar-add-output-zip-endpoint-to-clsi
[clsi] Add endpoints to get zip of output files

GitOrigin-RevId: a1a935e8170ab5a8d40baa6d96f8e42fe22c2e8c
2024-05-02 08:03:44 +00:00
Jakob Ackermann
2fb62a2cc9 Merge pull request #17874 from overleaf/jpa-ol-project-id-in-env
[clsi] expose Overleaf project id to LaTeX compilation via env var

GitOrigin-RevId: cf8be6f8d83351a587d9ac7940f61a046a14035e
2024-04-12 08:04:56 +00:00
Alf Eaton
e0e8a2ffaa Merge pull request #17525 from overleaf/ae-upgrade-prettier
Upgrade Prettier to v3

GitOrigin-RevId: 6f1338f196408f3edb4892d5220ad3665ff1a5bc
2024-03-26 09:04:05 +00:00
Jakob Ackermann
ee9be4364f Merge pull request #16923 from overleaf/jpa-server-pro-fs-rebranding
[server-pro] remove ShareLaTeX branding from filesystem

GitOrigin-RevId: c1d2c63bdc888d81e03bb1721e44b6ed6f64524b
2024-02-12 09:03:39 +00:00
ilkin-overleaf
fe63b4e1b2 Merge pull request #16806 from overleaf/ii-compile-dir-readonly
[clsi] Use readOnly bind-mounts for synctex and wordcount containers

GitOrigin-RevId: 8de094350604544dba4571aa849ad41a4a4d88d8
2024-02-05 09:03:58 +00:00
Brian Gough
26e32a608f Merge pull request #16519 from overleaf/bg-clsi-timeouts-add-metrics
add metrics for clsi cache operations

GitOrigin-RevId: c5ec221afd235434c8b81bafa2f482f11422ac4e
2024-01-18 09:04:19 +00:00
Jakob Ackermann
64ff3d73cc Merge pull request #16471 from overleaf/em-clsi-in-memory-lock
Replace filesystem lock in CLSI with in-memory lock

GitOrigin-RevId: de1ac3beca67bb4e9070806871a1c7b6a59aa77f
2024-01-11 09:05:22 +00:00
Jakob Ackermann
68627a2354 Merge pull request #16440 from overleaf/jpa-em-remove-extra-stat
[clsi] remove unnecessary stat call for checking each output file

GitOrigin-RevId: ba7fe435264596368808552a3da3e36d731eda09
2024-01-11 09:05:17 +00:00
Jakob Ackermann
9808b2406f Merge pull request #16438 from overleaf/jpa-em-replace-find-subprocess
[clsi] replace find subprocess for listing compile dir contents

GitOrigin-RevId: 36c8230ea6d787b1d948407d6473c14af8d6b5f6
2024-01-11 09:04:52 +00:00
Jakob Ackermann
06bcef8e26 Merge pull request #16428 from overleaf/jpa-clsi-replace-fs-extra
[clsi] replace fs-extra to avoid excess syscalls

GitOrigin-RevId: cbc8ec01b8ac9fd973e86a6d763c8f599323db94
2024-01-09 09:03:36 +00:00
Jakob Ackermann
2d86631943 Merge pull request #16426 from overleaf/em-decaf-output-cache-manager
Decaf cleanup OutputCacheManager

GitOrigin-RevId: 59d65930b4a88ab20e330dedac3d00d80c4140fd
2024-01-08 09:06:03 +00:00
Eric Mc Sween
fb5afd1c36 Merge pull request #13778 from overleaf/em-fetch-utils-clsi
Use fetch-utils in clsi

GitOrigin-RevId: c2ba431b2c6b0fb89f684861f34de40d767cc8cd
2023-07-17 10:59:29 +00:00
Tyna William
82c6c9a70e Merge pull request #13475 from overleaf/revert-13473-revert-13439-tw-support-rnw-files
Revert "Revert "support for Rnw files""

GitOrigin-RevId: 2bdaf154592f0b3c4bd3762d266b89a159a0b091
2023-07-17 10:55:08 +00:00
Tyna William
08b753ec03 Merge pull request #13473 from overleaf/revert-13439-tw-support-rnw-files
Revert "support for Rnw files"

GitOrigin-RevId: 1116814958cabed4c741bd0d3cce7670a6ca5ed4
2023-07-17 10:33:54 +00:00
Tyna William
7e7838f127 Merge pull request #13439 from overleaf/tw-support-rnw-files
support for Rnw files

GitOrigin-RevId: 407ab1b6b131408fa0e4256d50781e9f70b0c3b8
2023-07-17 10:33:37 +00:00
Jakob Ackermann
3e25bfba3b Merge pull request #12824 from overleaf/jpa-clsi-pipe
[clsi] rewrite UrlFetcher in async/await and fetch/pipeline

GitOrigin-RevId: a2a90fd886252f06c818f807e85e566f3fc1f841
2023-04-28 08:04:01 +00:00
Jakob Ackermann
710a5eb1e5 Merge pull request #12731 from overleaf/jpa-clsi-log-err
[clsi] log error when writing to log file fails

GitOrigin-RevId: 3f698fd0ecad26fe9782aae6ba9a49c03b8f9923
2023-04-28 08:03:51 +00:00
Eric Mc Sween
e610e07dd1 Merge pull request #12203 from overleaf/em-camel-case-clsi
Camel case variables in clsi

GitOrigin-RevId: 1542a6b7f8886e5fef496ce31acd2bba537befa9
2023-03-23 09:04:46 +00:00
Eric Mc Sween
eafdb05c89 Merge pull request #11661 from overleaf/jpa-cleanup-clsi
[clsi] delete unused DbQueue module and related config option

GitOrigin-RevId: d9f1f84f82149efe1d7cdd75b2b45ba78e4defd2
2023-02-09 14:34:42 +00:00
Jakob Ackermann
71bba54b36 Merge pull request #11296 from overleaf/jpa-pdf-caching-ignore-not-found
[clsi] ignore file not found error when unlinking stale chunk

GitOrigin-RevId: d7b78f7c2773102d7133f710ad7851542417b472
2023-01-18 15:08:20 +00:00
Jakob Ackermann
5078aad83e Merge pull request #11235 from overleaf/jpa-clsi-silence-404
[clsi] reduce noise in sentry by lowering log level of a verbose error

GitOrigin-RevId: faa11e637011167a3b4772ba65dab32a2679c64e
2023-01-17 09:03:58 +00:00
Jakob Ackermann
2641a501a1 Merge pull request #10239 from overleaf/jpa-pdf-caching-backend-tweaks
[clsi] pdf-caching: emit partial set of processed ranges on timeout

GitOrigin-RevId: 0038b5a30ac33fcdcab523d7ddc72fea9f2f5be9
2022-11-02 09:04:43 +00:00
Jakob Ackermann
d1dc798c07 Merge pull request #10139 from overleaf/jpa-split-test-min-chunk-size
[misc] add split test for a per request pdfCachingMinChunkSize

GitOrigin-RevId: 6a8a3c6267501789f2047a67b03db6ac6df427c3
2022-10-26 08:03:39 +00:00
ilkin-overleaf
e865fe4244 Merge pull request #10096 from overleaf/ii-stderr-exceeds-max-size-error
Adjust string output stream settings

GitOrigin-RevId: 02b48e17ad52ea9af44ce330bc8b6bc80c4d4251
2022-10-24 08:02:54 +00:00
Jakob Ackermann
26bfc9551b Merge pull request #9475 from overleaf/jpa-ta-clsi-root-doc-path
[clsi] remove stripping of characters from the root doc path

GitOrigin-RevId: 18a8a5c66eecc1cf21aa202180720707d27671ea
2022-09-02 08:05:02 +00:00
Brian Gough
af454927a4 Merge pull request #9087 from overleaf/jpa-fix-caching
[clsi] fix handling of xref tables with non-continuous ids

GitOrigin-RevId: 02e00d1963f7547304574d61e441240d086bfe7a
2022-08-03 08:04:58 +00:00
Jakob Ackermann
caf7497538 Merge pull request #8950 from overleaf/bg-use-qpdf-xref-table
[clsi] parse xref table from qpdf output

GitOrigin-RevId: cc400c65082c7c8c28c21037b97b4fde66a85835
2022-08-03 08:04:10 +00:00