Commit Graph

116 Commits

Author SHA1 Message Date
Eric Mc Sween bb764f76f1 Merge pull request #8132 from overleaf/em-node-fetch-filestore
Replace request with node-fetch in filestore

GitOrigin-RevId: 49487a941b63655920de04fe50fd197f67498e58
2022-06-01 08:04:23 +00:00
Miguel Serrano 90ad1d0c42 [object-persistor] use forked fast-crc32c dependency (#6945)
* [object-persistor] use forked fast-crc32c dependency
* removed amd64 compatibility from history_v1, filestore and docstore

GitOrigin-RevId: 7f1d9242c247f75cbdfde30a87ca25778e825cc1
2022-03-11 09:03:16 +00:00
Jakob Ackermann fbf86af2c5 Merge pull request #6542 from overleaf/jpa-em-code-sharing-workspaces
[misc] npm workspaces

GitOrigin-RevId: 87aa72db6637fb238d7cd35b0a48ac3ed58ab3eb
2022-02-16 11:31:07 +00:00
Jakob Ackermann 2ea1811157 Merge pull request #6120 from overleaf/jpa-same-linting-packages
[misc] move the linting setup to the root of the monorepo

GitOrigin-RevId: 1633e2a58598add0b727738cd3bfba0ab7bae781
2021-12-17 09:03:06 +00:00
Eric Mc Sween b0bf526f81 Merge pull request #6079 from overleaf/em-upgrade-logger
Upgrade logger and metrics in all services

GitOrigin-RevId: 2baf63eeeab77fb3559cf763ddacfbf4b745cd0b
2021-12-15 09:04:25 +00:00
Brian Gough e0bf4217a8 Merge pull request #5258 from overleaf/revert-5214-bg-migrate-node-uuid-to-uuid
Revert "migrate node-uuid to uuid"

GitOrigin-RevId: f512eb1040e2ce6d87774de9c1fc884594480cef
2021-09-28 08:04:11 +00:00
Brian Gough f833780070 Merge pull request #5214 from overleaf/bg-migrate-node-uuid-to-uuid
migrate node-uuid to uuid

GitOrigin-RevId: 742fd4d3f039c6a026b3d35c3b0a7d54c00ca183
2021-09-27 08:02:54 +00:00
Jakob Ackermann dc0aed6604 Merge pull request #4794 from overleaf/jpa-object-persistor-npm
[misc] re-install @overleaf/object-persistor from npm

GitOrigin-RevId: 7ecbc050b9037c7930aa09fe561b99dd36eb59dd
2021-08-20 08:03:01 +00:00
Jakob Ackermann 3003a1dded [misc] upgrade build scripts to version 3.11.0 and cleanup packages
```
npm uninstall prettier-eslint-cli eslint-plugin-standard eslint-plugin-jsx-a11y eslint-plugin-react eslint-config-standard-jsx eslint-config-standard-react babel-eslint
npm dedupe
```
2021-07-13 11:55:17 +01:00
Jakob Ackermann 3152bf361d [misc] switch from settings-sharelatex to @overleaf/settings 2021-07-12 17:47:19 +01:00
Jakob Ackermann 8d6f169c66 [misc] install bunyan as production dependency
```
Error: Cannot find module 'bunyan'
Require stack:
- .../node_modules/@google-cloud/logging-bunyan/build/src/middleware/express.js
- .../node_modules/@google-cloud/logging-bunyan/build/src/index.js
- .../node_modules/logger-sharelatex/logging-manager.js
- .../node_modules/logger-sharelatex/index.js
- .../app.js
```
2021-07-12 17:35:49 +01:00
Jakob Ackermann 5f084f8603 [misc] bump the version of the metrics module to 3.5.1 2021-02-16 15:15:52 +00:00
Jakob Ackermann c55c0d9d25 [misc] install @overleaf/object-persistor from tar-ball 2020-12-02 12:39:01 +00:00
Jakob Ackermann 95f3bcef21 [misc] bump metrics module to 3.4.1
- renamed package from `metrics-sharelatex` to `@overleaf/metrics`
- drop support for statsd backend
- decaffeinate
- compress `/metrics` response using gzip
- bump debugging agents to latest versions
- expose prometheus interfaces for custom metrics (custom tags)
- cleanup of open sockets metrics
- fix deprecation warnings for header access
2020-11-25 11:57:23 +00:00
Christopher Hoskin 6968d5d17b [misc] bump logger-sharelatex to version 2.2.0 2020-08-13 09:02:46 +01:00
Jakob Ackermann f0551307d2 [misc] bump the dev-env to 3.3.2 2020-08-10 17:01:12 +01:00
Simon Detheridge 4bb6509b9c Update object-persistor and make aws a dev dependency 2020-07-23 10:45:27 +01:00
Simon Detheridge 5329cf24b0 Remove unneeded @google-cloud/storage module from main deps 2020-07-22 17:18:17 +01:00
Simon Detheridge 6c853de5d5 Migrate to new object-persistor module (#122)
* Migrate to new object-persistor module

* Support updated persistor module using ES6 classes

* Update object-persistor

* Upgrade new persistor code to OError3

* Update to latest minor release for outdated packages

* Update package hashes to sha512

* Point object-persistor at master branch
2020-07-07 13:49:54 +01:00
Ersun Warncke 83f6eabaf6 upgarde logger-sharelatex 2020-07-06 09:26:59 -04:00
Ersun Warncke e83f6b609b pr fixes 2020-06-25 08:27:14 -04:00
Ersun Warncke 0e4be81406 upgrade to o-error v3 2020-06-24 22:40:04 -04:00
Simon Detheridge 37e3531cc9 Upgrade Node to 12.18.0 2020-06-03 09:59:54 +01:00
Simon Detheridge 256cebf7a7 Add fast-crc32c library 2020-04-03 17:06:41 +01:00
mserranom 70e4dd9fde updated sinon, mocha 2020-03-30 16:15:21 +02:00
mserranom a067198619 updated minor and patch dependencies 2020-03-30 16:05:25 +02:00
Simon Detheridge 82599bce67 Merge pull request #105 from overleaf/jpa-logger-1-9-1
[misc] bump logger-sharelatex to 1.9.1
2020-03-26 10:31:33 +00:00
Simon Detheridge d073fe75ca Use single pipeline when calculating md5 2020-03-25 16:59:51 +00:00
Jakob Ackermann 2ce3c8f97e [misc] bump logger-sharelatex to 1.9.1 2020-03-23 16:18:07 +01:00
Simon Detheridge ce52f8aa60 Update FSPersistor deleteDirectory behaviour to match S3 and GCS 2020-03-16 11:33:51 +00:00
Simon Detheridge 58db14456a Add timestamp to files in deleted bucket 2020-03-14 14:11:17 +00:00
Simon Detheridge edf1ce1f7e Delete files from a directory in parallel 2020-03-14 14:02:58 +00:00
Simon Detheridge 28c3fe4a56 Validate key names when deleting directory with a configurable regex 2020-03-10 17:54:09 +00:00
Simon Detheridge e6cf0687a9 Add gcs client library 2020-03-04 15:09:14 +00:00
Simon Detheridge d06b74b0f8 Bump metrics to 2.5.0 2020-03-04 10:55:01 +00:00
Simon Detheridge b5e4345855 Merge pull request #96 from overleaf/jpa-dependencies-cleanup
[misc] dependencies cleanup
2020-03-04 10:00:56 +00:00
Simon Detheridge 62b2aa8f02 Merge pull request #98 from overleaf/jpa-promisify-filehandler
[misc] promisify FileHandler and remove dependency on async
2020-03-04 09:59:15 +00:00
Jakob Ackermann 2b15729658 [misc] promisify FileHandler and remove dependency on async
Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
2020-03-02 18:23:17 +01:00
Simon Detheridge 0312b00150 Fix prettier task 2020-03-02 16:31:35 +00:00
Jakob Ackermann 4b7c6fafd2 [misc] drop unused dependency pngcrush 2020-03-02 17:11:56 +01:00
Jakob Ackermann 3c61e53918 [misc] move mocha to the devDependencies 2020-03-02 17:11:39 +01:00
Jakob Ackermann 6f27f7a1bf [misc] drop the /heapdump route and related heapdump package 2020-03-02 17:09:35 +01:00
Jakob Ackermann 32557ab1d7 [SafeExec] replace _.once with lodash.once 2020-03-02 17:08:42 +01:00
Jakob Ackermann 0db4a17a14 [HealthCheckController] use fs.copyFile instead of fs-extra.copy
the fs-extra method has a HUGE overhead of JS code and also syscalls for
 no particular benefit in this case: just copy the tiny.pdf file.

Here is an overview of the major operations: paths are relative to
  https://github.com/jprichardson/node-fs-extra/blob/1.0.0

We start in /lib/copy/copy.js
- sys: check that the source file exists
- sys: check that the source file has an existing parent directory?!
Continue in /lib/copy/ncp.js
- sys: more stat calls on both source and dest to determine permissions
- read/write streams to pipe the file content through the process
- sys: chmod on the destination to match the source permissions

What we actually need is a call to the binding and let node/the os
 figure out the best way to copy the contents.

Signed-off-by: Jakob Ackermann <das7pad@outlook.com>
2020-03-02 17:02:35 +01:00
Simon Detheridge c13d7f4197 Bump aws-sdk version in package.json 2020-03-02 11:18:06 +00:00
Simon Detheridge 4e507197fc Revert "Merge pull request #91 from overleaf/revert-78-spd-migration-persistor"
This reverts commit 0cce792604c258538cf012486e7329fa1be7da0f, reversing
changes made to 08b606e22f855121b2d8a2832a5d7e2f45836c3f.
2020-02-17 14:04:42 +00:00
Simon Detheridge df2ddbe0e1 Revert "Add Migration Persistor, to send missing file requests to a fallback persistor" 2020-02-14 14:26:33 +00:00
Simon Detheridge 42a5d168df Remove unused packages 2020-02-12 13:21:52 +00:00
Simon Detheridge b4b7fd226e Add mechanisms to transfer files with md5-based integrity checks
Fix error in settings and tidy up tests

Remove unused variable declaration

Remove .only from tests and update eslint rules to catch it in future

Use  to catch errors more safely getting md5 hash

Avoid unnecessary call to S3 to get md5 response
2020-02-12 13:21:52 +00:00
Simon Detheridge 2625e03a31 Add MigrationPersistor for sending 404 requests to a fallback persistor 2020-02-12 13:19:48 +00:00