Commit Graph
21 Commits
Author SHA1 Message Date
Brian Gough 9ff7e66b99 Merge pull request #35504 from overleaf/bg-png2pdf-on-slow-images-only
run png2pdf on slow images only

GitOrigin-RevId: 2d0ecd35bfcfc308773502c28eeb5a722055a5a9
2026-09-17 11:29:12 +00:00
Jakob AckermannandBrian Gough 69ccb3c316 [clsi] integrate png2pdf (#34796)
* Add png to pdf converter for optimising image loading in pdftex

* refactor

* move fixtures

* refactor and add unit tests

* Remove gamma correction from PNG processing and update related tests

* Add crc32fast dependency and enhance PNG processing tests

* Add fixture test cases for PNG processing

* fix alpha channel to match pdftex

pdftex always uses an 8 bit alpha channel in the output even for 16 bit images

* Refactor PNG processing to use Option for alpha stream and update related logic

* Enhance PDF writing by adding DecodeParms for alpha stream and optimize PNG encoding process

* Add pixel dimensions support in PNG processing and PDF writing

* [copybara] move clsi excludes

* [clsi] move gitignore rules

* [png2pdf] move into dockerfiles folder

* [png2pdf] use atomic writes

* [clsi] integrate png2pdf

* [saas-e2e] add coverage for png2pdf

* [png2pdf] make compatible with seccomp profile in clsi

* [copybara] remove exclude following move of png2pdf

* [copybara] remove spurious excludes

* fix broken detection of malformed chunk headers

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* [project-history] add build script flag for depending on doc-updater types

* [png2pdf] run tests as part of clsi pipeline

* [png2pdf] fix calculation of PDF page size

* [web] put png2pdf behind feature flag and make it opt-in

* [web] fix ClsiManager.promises.sendRequest invocation after merge

* extend is_copy_safe function to follow pdftex

see the conditions in pdftex's writepng.c
https://svn.tug.org:8369/texlive/trunk/Build/source/texk/web2c/pdftexdir/writepng.c?revision=78081&view=markup#l575

the criteria are not an exact match because the png crate does not support all the
png chunk types
https://github.com/image-rs/image-png/issues/116

if the criteria result in a png being converted or skipped the final result should be the same.

in one case the conversion will be done by pdftex (at runtime)
in the other case the conversion will be done by png2pdf -

both conversions should give the same result.

* include tests in png2pdf Dockerfile

* [clsi] explicitly disable png2pdf conversion for project conversion

* [clsi] png2pdf: count converted files

* [png2pdf] bump date in docker image tag

* [web] make draft-setting option wider when png2pdf is available

---------

Co-authored-by: Brian Gough <brian.gough@overleaf.com>
GitOrigin-RevId: d0d952ee009bf93c72bba2ffe763ae0160d396e4
2026-09-17 11:17:28 +00:00
Andrew Rumble cd7da983d1 Merge pull request #30232 from overleaf/ar/convert-clsi-to-es-modules
[clsi] convert to ES modules

GitOrigin-RevId: fb7fa52cc8f678ee31be352e62a5dff95e88008b
2026-01-22 09:06:23 +00:00
Eric Mc Sween 74524db293 Merge pull request #28909 from overleaf/em-compile-metrics
Use histograms to track CLSI compile times

GitOrigin-RevId: cf25f1e6d2094186f419acc70748f0c71b6c3240
2025-10-13 08:07:02 +00:00
Jakob Ackermann d19c5e236f Merge pull request #22208 from overleaf/jpa-clsi-hash
[misc] clsi: read files from history-v1 with fallback to filestore

GitOrigin-RevId: c54bb128780198c14e7a63818f39fad62ce65d4e
2024-11-29 09:05:39 +00:00
Antoine Clausse 7f48c67512 Add prefer-node-protocol ESLint rule (#21532)
* Add `unicorn/prefer-node-protocol`

* Fix `unicorn/prefer-node-protocol` ESLint errors

* Run `npm run format:fix`

* Add sandboxed-module sourceTransformers in mocha setups

Fix `no such file or directory, open 'node:fs'` in `sandboxed-module`

* Remove `node:` in the SandboxedModule requires

* Fix new linting errors with `node:`

GitOrigin-RevId: 68f6e31e2191fcff4cb8058dd0a6914c14f59926
2024-11-11 09:04:51 +00:00
Brian Gough e01af0e9c6 Merge pull request #16519 from overleaf/bg-clsi-timeouts-add-metrics
add metrics for clsi cache operations

GitOrigin-RevId: c5ec221afd235434c8b81bafa2f482f11422ac4e
2024-01-18 09:04:19 +00:00
Jakob Ackermann b4b369fea5 Merge pull request #16428 from overleaf/jpa-clsi-replace-fs-extra
[clsi] replace fs-extra to avoid excess syscalls

GitOrigin-RevId: cbc8ec01b8ac9fd973e86a6d763c8f599323db94
2024-01-09 09:03:36 +00:00
Eric Mc Sween 16fee6d7d2 Merge pull request #12203 from overleaf/em-camel-case-clsi
Camel case variables in clsi

GitOrigin-RevId: 1542a6b7f8886e5fef496ce31acd2bba537befa9
2023-03-23 09:04:46 +00:00
Jakob Ackermann 7f9fd00bda Merge pull request #5367 from overleaf/jpa-node-handle-callback-err
[misc] fix eslint violations for node/handle-callback-err

GitOrigin-RevId: 83a4900e8861010df1917bff49382bd9c93375bd
2021-10-28 08:03:26 +00:00
Jakob Ackermann 02918e7483 Merge pull request #4649 from overleaf/jpa-fs-based-caching
[perf] UrlCache: pure fs based cache state for downloads

GitOrigin-RevId: d19afc396324d4c3318b31620c8ad0c04e0544ce
2021-10-07 08:03:18 +00:00
Jakob Ackermann f285e503b4 [misc] run format_fix and lint:fix 2021-07-13 12:04:48 +01:00
Jakob Ackermann 64551f0198 [misc] switch from settings-sharelatex to @overleaf/settings 2021-07-12 17:47:21 +01:00
Jakob Ackermann 8f1ea7f0d1 [misc] track delays of using sqlite for url caching details 2021-06-14 13:03:02 +01:00
Brian Gough 371de76a4a use fs.copyFile instead of fse.copy in UrlCache module 2021-05-17 10:54:11 +01:00
Brian Gough 44e0742aa3 use fse.copy for performance
it uses the native fs.copyFile method
2021-05-14 15:49:20 +01:00
Jakob Ackermann 1d1b9ebebc [misc] bump the dev-env to 3.3.2 2020-08-10 17:01:11 +01:00
Henry Oswald 77e8ba74a7 add pipeUrlToFileWithRetry function to retry file downloads 3 times 2020-05-14 13:24:58 +01:00
mserranom d5bd1790a8 updated sequelize 2020-03-16 16:31:02 +01:00
mserranom 8729acd48c prettier: convert app/js decaffeinated files to Prettier format 2020-02-19 12:14:37 +01:00
mserranom 3af6bdd588 decaffeinate: rename app/coffee dir to app/js 2020-02-19 12:14:34 +01:00