Commit Graph
5375 Commits
Author SHA1 Message Date
nate stemen a019a3776d Merge pull request #60 from overleaf/dependabot-npm_and_yarn-y18n-4.0.1
build(deps): bump y18n from 4.0.0 to 4.0.1
2021-05-17 17:07:22 -04:00
Jakob Ackermann 224ae0c254 [ContentCacheManager] add support for stream detection across chunks
Retain a small part (6 or 9 bytes) of each chunk in memory for providing
 the next iteration with enough context for finding the start/end marker
 of a range.
2021-05-17 17:36:05 +01:00
nate stemen 290c0d90c8 Merge pull request #44 from overleaf/dependabot-npm_and_yarn-hosted-git-info-2.8.9
Bump hosted-git-info from 2.8.8 to 2.8.9
2021-05-17 11:02:06 -04:00
nate stemen 1de6c67910 Merge pull request #43 from overleaf/dependabot-npm_and_yarn-lodash-4.17.21
Bump lodash from 4.17.20 to 4.17.21
2021-05-17 11:01:50 -04:00
nate stemen b076823e7a Merge pull request #42 from overleaf/dependabot-npm_and_yarn-underscore-1.13.1
Bump underscore from 1.9.2 to 1.13.1
2021-05-17 11:01:41 -04:00
nate stemen c33d6e4188 Merge pull request #40 from overleaf/jpa-explicit-dependencies
[misc] add linting for missing explicit dependencies and fix any errors
2021-05-17 11:01:30 -04:00
nate stemen 8d76717ec1 Merge pull request #39 from overleaf/dependabot-npm_and_yarn-y18n-4.0.1
Bump y18n from 4.0.0 to 4.0.1
2021-05-17 11:01:04 -04:00
Jakob Ackermann 6b9c8bced6 [ContentCacheManager] write streams to disk atomically
Use an intermediate file for writing to disk, then rename to the target.
2021-05-17 14:22:37 +01:00
Brian Gough 8c80ddd27a remove unnecessary console.log from hash benchmark 2021-05-17 14:14:00 +01:00
Brian Gough 939ad3af31 include UV_THREADPOOL_SIZE in benchmark logs 2021-05-17 14:14:00 +01:00
Brian Gough cbeba4af00 run the hash benchmark 10 times 2021-05-17 14:14:00 +01:00
Brian Gough 7b6434ef81 add benchmark script for hashing 2021-05-17 14:14:00 +01:00
nate stemen 0b5b8f56cb Merge pull request #73 from overleaf/dependabot-npm_and_yarn-hosted-git-info-2.8.9
Bump hosted-git-info from 2.8.8 to 2.8.9
2021-05-17 08:10:57 -04:00
nate stemen a7c31132fe Merge pull request #72 from overleaf/dependabot-npm_and_yarn-lodash-4.17.21
Bump lodash from 4.17.20 to 4.17.21
2021-05-17 08:10:50 -04:00
nate stemen 007141f545 Merge pull request #71 from overleaf/jpa-explicit-dependencies
[misc] add linting for missing explicit dependencies and fix any errors
2021-05-17 08:10:43 -04:00
nate stemen d1c2dd6cf1 Merge pull request #70 from overleaf/dependabot-npm_and_yarn-y18n-4.0.1
Bump y18n from 4.0.0 to 4.0.1
2021-05-17 08:10:32 -04:00
nate stemen 227b9e9ed8 Merge pull request #66 from overleaf/dependabot-npm_and_yarn-ini-1.3.8
Bump ini from 1.3.5 to 1.3.8
2021-05-17 08:10:17 -04:00
nate stemen 9b234e5946 Merge pull request #65 from overleaf/dependabot-npm_and_yarn-dot-prop-4.2.1
Bump dot-prop from 4.2.0 to 4.2.1
2021-05-17 08:10:08 -04:00
Brian Gough 371de76a4a use fs.copyFile instead of fse.copy in UrlCache module 2021-05-17 10:54:11 +01:00
Jakob Ackermann dde645e770 Merge pull request #176 from overleaf/dependabot-npm_and_yarn-hosted-git-info-2.8.9
Bump hosted-git-info from 2.8.8 to 2.8.9
2021-05-17 10:57:18 +02:00
Jakob Ackermann e92b9fc754 Merge pull request #174 from overleaf/jpa-explicit-dependencies
[misc] add linting for missing explicit dependencies and fix any errors
2021-05-17 10:57:12 +02:00
Jakob Ackermann f97117a6ba Merge pull request #175 from overleaf/jpa-double-down-doc-size-limit
[misc] double down on enforcing doc size limits
2021-05-17 10:56:55 +02:00
Jakob Ackermann 18e943742e [perf] drop useless synchronous syscall on hot path for writing docs 2021-05-17 09:25:29 +01:00
Brian Gough 24ef5786a0 upgrade fs-extra 2021-05-14 15:58:11 +01:00
Brian Gough 44e0742aa3 use fse.copy for performance
it uses the native fs.copyFile method
2021-05-14 15:49:20 +01:00
Brian Gough ff2175e727 add validation for express :content_id parameter 2021-05-13 15:48:17 +01:00
Brian Gough 013d38552a log the expiry timeout value when disk is low 2021-05-13 15:13:29 +01:00
Brian Gough 3044bc4cf5 add transport to get-connected-users metric 2021-05-13 14:51:26 +01:00
b456ea726d [misc] merge pdf caching into main (#226)
* wip generate directory for hash content

* cleanup, remove console logging

* add content caching module

* Return PDF stream ranges with compile response

* Return the PDF file size in the compile response

* PDF range endpoint

* [misc] WIP: pdf caching: preserve the m-time on static content files

* [misc] WIP: pdf caching: improve browser caching, emit caching headers

* [misc] WIP: pdf caching: do not emit very small chunks <1kB

* [misc] keep up with moving output files into a separate directory

* [OutputCacheManager] add global feature flag for enabling pdf caching

* [misc] add contentId into the URL for protecting PDF stream contents

* [misc] support PDF stream caching for anonymous users

* [misc] add per-request feature flag for enabling PDF stream caching

* [misc] enable pdf caching in CI and emit metrics at the end of run

* [misc] expose compile stats and timings to the frontend

* [misc] log an error in case saving output files fails

* [misc] add metrics for pdf bandwidth and pdf caching performance

* [misc] add a dark mode to the pdf caching for computing ranges only

* [misc] move pdf caching metrics into ContentCacheMetrics

* [misc] add a config option for the min chunk size of pdf ranges

Co-authored-by: Brian Gough <brian.gough@overleaf.com>
Co-authored-by: Eric Mc Sween <eric.mcsween@overleaf.com>
2021-05-13 14:07:54 +01:00
Christopher Hoskin 753155db97 Merge pull request #94 from overleaf/csh-issue-3680-template-config
Generate a config file from env vars and template
2021-05-12 12:38:43 +01:00
Christopher Hoskin 725a9e0669 Use CMD as in other apps 2021-05-12 12:32:14 +01:00
Christopher Hoskin 0827795fc7 Consistently use GIT_BRIDGE 2021-05-12 08:42:45 +01:00
dependabot[bot] 8a16c55000 Bump hosted-git-info from 2.8.8 to 2.8.9
Bumps [hosted-git-info](https://github.com/npm/hosted-git-info) from 2.8.8 to 2.8.9.
- [Release notes](https://github.com/npm/hosted-git-info/releases)
- [Changelog](https://github.com/npm/hosted-git-info/blob/v2.8.9/CHANGELOG.md)
- [Commits](https://github.com/npm/hosted-git-info/compare/v2.8.8...v2.8.9)

Signed-off-by: dependabot[bot] <support@github.com>
2021-05-11 20:43:06 +00:00
dependabot[bot] da81a97b82 Bump hosted-git-info from 2.8.8 to 2.8.9
Bumps [hosted-git-info](https://github.com/npm/hosted-git-info) from 2.8.8 to 2.8.9.
- [Release notes](https://github.com/npm/hosted-git-info/releases)
- [Changelog](https://github.com/npm/hosted-git-info/blob/v2.8.9/CHANGELOG.md)
- [Commits](https://github.com/npm/hosted-git-info/compare/v2.8.8...v2.8.9)

Signed-off-by: dependabot[bot] <support@github.com>
2021-05-11 17:18:40 +00:00
dependabot[bot] e66ff7a2ca Bump hosted-git-info from 2.8.5 to 2.8.9
Bumps [hosted-git-info](https://github.com/npm/hosted-git-info) from 2.8.5 to 2.8.9.
- [Release notes](https://github.com/npm/hosted-git-info/releases)
- [Changelog](https://github.com/npm/hosted-git-info/blob/v2.8.9/CHANGELOG.md)
- [Commits](https://github.com/npm/hosted-git-info/compare/v2.8.5...v2.8.9)

Signed-off-by: dependabot[bot] <support@github.com>
2021-05-11 14:32:37 +00:00
dependabot[bot] eb8a3118d8 Bump hosted-git-info from 2.8.5 to 2.8.9
Bumps [hosted-git-info](https://github.com/npm/hosted-git-info) from 2.8.5 to 2.8.9.
- [Release notes](https://github.com/npm/hosted-git-info/releases)
- [Changelog](https://github.com/npm/hosted-git-info/blob/v2.8.9/CHANGELOG.md)
- [Commits](https://github.com/npm/hosted-git-info/compare/v2.8.5...v2.8.9)

Signed-off-by: dependabot[bot] <support@github.com>
2021-05-11 13:04:24 +00:00
dependabot[bot] bc423fa0cd build(deps): bump hosted-git-info from 2.8.5 to 2.8.9
Bumps [hosted-git-info](https://github.com/npm/hosted-git-info) from 2.8.5 to 2.8.9.
- [Release notes](https://github.com/npm/hosted-git-info/releases)
- [Changelog](https://github.com/npm/hosted-git-info/blob/v2.8.9/CHANGELOG.md)
- [Commits](https://github.com/npm/hosted-git-info/compare/v2.8.5...v2.8.9)

Signed-off-by: dependabot[bot] <support@github.com>
2021-05-11 12:49:25 +00:00
dependabot[bot] ce32ca9238 Bump hosted-git-info from 2.8.8 to 2.8.9
Bumps [hosted-git-info](https://github.com/npm/hosted-git-info) from 2.8.8 to 2.8.9.
- [Release notes](https://github.com/npm/hosted-git-info/releases)
- [Changelog](https://github.com/npm/hosted-git-info/blob/v2.8.9/CHANGELOG.md)
- [Commits](https://github.com/npm/hosted-git-info/compare/v2.8.8...v2.8.9)

Signed-off-by: dependabot[bot] <support@github.com>
2021-05-11 12:30:36 +00:00
dependabot[bot] 326c7da1d8 Bump hosted-git-info from 2.8.8 to 2.8.9
Bumps [hosted-git-info](https://github.com/npm/hosted-git-info) from 2.8.8 to 2.8.9.
- [Release notes](https://github.com/npm/hosted-git-info/releases)
- [Changelog](https://github.com/npm/hosted-git-info/blob/v2.8.9/CHANGELOG.md)
- [Commits](https://github.com/npm/hosted-git-info/compare/v2.8.8...v2.8.9)

Signed-off-by: dependabot[bot] <support@github.com>
2021-05-11 12:19:56 +00:00
dependabot[bot] 54bbb3dbbd Bump hosted-git-info from 2.8.8 to 2.8.9
Bumps [hosted-git-info](https://github.com/npm/hosted-git-info) from 2.8.8 to 2.8.9.
- [Release notes](https://github.com/npm/hosted-git-info/releases)
- [Changelog](https://github.com/npm/hosted-git-info/blob/v2.8.9/CHANGELOG.md)
- [Commits](https://github.com/npm/hosted-git-info/compare/v2.8.8...v2.8.9)

Signed-off-by: dependabot[bot] <support@github.com>
2021-05-11 10:38:18 +00:00
dependabot[bot] fc48f8dd98 Bump hosted-git-info from 2.8.8 to 2.8.9
Bumps [hosted-git-info](https://github.com/npm/hosted-git-info) from 2.8.8 to 2.8.9.
- [Release notes](https://github.com/npm/hosted-git-info/releases)
- [Changelog](https://github.com/npm/hosted-git-info/blob/v2.8.9/CHANGELOG.md)
- [Commits](https://github.com/npm/hosted-git-info/compare/v2.8.8...v2.8.9)

Signed-off-by: dependabot[bot] <support@github.com>
2021-05-11 08:45:50 +00:00
dependabot[bot] 847ff87ce3 Bump lodash from 4.17.19 to 4.17.21
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.19 to 4.17.21.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.19...4.17.21)

Signed-off-by: dependabot[bot] <support@github.com>
2021-05-11 04:33:05 +00:00
dependabot[bot] c0fc4c329e build(deps): bump lodash from 4.17.19 to 4.17.21
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.19 to 4.17.21.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.19...4.17.21)

Signed-off-by: dependabot[bot] <support@github.com>
2021-05-11 04:20:08 +00:00
dependabot[bot] 78c0b90eb7 Bump lodash from 4.17.20 to 4.17.21
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.20 to 4.17.21.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.20...4.17.21)

Signed-off-by: dependabot[bot] <support@github.com>
2021-05-11 03:28:23 +00:00
dependabot[bot] 5ab599e163 Bump lodash from 4.17.20 to 4.17.21
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.20 to 4.17.21.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.20...4.17.21)

Signed-off-by: dependabot[bot] <support@github.com>
2021-05-11 03:18:32 +00:00
dependabot[bot] d5f2f4b7e4 Bump lodash from 4.17.20 to 4.17.21
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.20 to 4.17.21.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.20...4.17.21)

Signed-off-by: dependabot[bot] <support@github.com>
2021-05-11 03:16:58 +00:00
dependabot[bot] 39249d0b7e Bump lodash from 4.17.20 to 4.17.21
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.20 to 4.17.21.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.20...4.17.21)

Signed-off-by: dependabot[bot] <support@github.com>
2021-05-11 00:17:10 +00:00
dependabot[bot] 785a69bdb4 Bump lodash from 4.17.20 to 4.17.21
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.20 to 4.17.21.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](https://github.com/lodash/lodash/compare/4.17.20...4.17.21)

Signed-off-by: dependabot[bot] <support@github.com>
2021-05-10 17:09:32 +00:00
Brian Gough 9622b341b7 add transport type to metrics 2021-05-10 10:16:50 +01:00
Shane Kilkelly b47f21a44e Configure from env vars at run time
In both local developent, and production.  Also, vendor the `envsubst`
binary from https://github.com/a8m/envsubst, as it supports default
values, which the gnu envsubst (from gettext-base) does not.
2021-05-07 09:47:49 +01:00