Commit Graph

478 Commits

Author SHA1 Message Date
Simon Detheridge 9e0b378948 Remove minipass as dependency and refactor to make things clearer 2020-02-12 13:21:52 +00:00
Simon Detheridge 5d5d325691 Preserve all error information when cleanup of copied file fails 2020-02-12 13:21:52 +00:00
Simon Detheridge b4b7fd226e Add mechanisms to transfer files with md5-based integrity checks
Fix error in settings and tidy up tests

Remove unused variable declaration

Remove .only from tests and update eslint rules to catch it in future

Use  to catch errors more safely getting md5 hash

Avoid unnecessary call to S3 to get md5 response
2020-02-12 13:21:52 +00:00
Simon Detheridge 2625e03a31 Add MigrationPersistor for sending 404 requests to a fallback persistor 2020-02-12 13:19:48 +00:00
Simon Detheridge f877f51775 Rename *PersistorManager to *Persistor 2020-02-12 13:19:48 +00:00
Jakob Ackermann 03246041cd Merge pull request #83 from overleaf/jpa-build-scripts-1-3-5
[misc] update the build scripts to 1.3.5
2020-02-12 13:21:36 +01:00
Jakob Ackermann 1341af04f5 [misc] update the build scripts to 1.3.5 2020-02-11 18:58:55 +01:00
Jakob Ackermann ca94225c19 Merge pull request #80 from overleaf/jpa-cleanup-install-deps
[misc] cleanup install deps
2020-02-10 17:16:44 +01:00
Brian Gough 14ba1ea1c1 update to node 10.19.0 2020-02-07 14:16:31 +00:00
Jakob Ackermann ac4361d560 [misc] dockerignore: hide the local data dirs 2020-01-31 15:50:53 +01:00
Jakob Ackermann c1d81dc1ea [misc] install a maintained ghostscript version and delete package lists
also do not install vim and bail out in case any command exited with a
 non zero exit code.
2020-01-31 15:50:53 +01:00
Simon Detheridge 2f99c220f8 Merge pull request #77 from overleaf/spd-simplified-logging
Simplify logging to only log once per request
2020-01-20 10:52:57 +00:00
Simon Detheridge 997434c930 Remove redundant 'res.logInfo' 2020-01-14 20:55:33 +00:00
Simon Detheridge 035364b62e Fix RequestLogger object lifecycle 2020-01-14 17:15:01 +00:00
Simon Detheridge 28fb998719 Refactor request logger into a class 2020-01-14 12:02:39 +00:00
Simon Detheridge 9615a06e0f Improve logging middleware to add info methods to request object 2020-01-10 17:22:06 +00:00
Simon Detheridge f40fbe77aa Simplify logging to log only once per http request 2020-01-10 15:41:36 +00:00
Simon Detheridge c85d211735 Merge pull request #76 from overleaf/spd-general-errors
Handle AccessDenied and stream-premature-close errors
2020-01-09 14:47:46 +00:00
Simon Detheridge e331a3130c Handle AccessDenied and stream-premature-close errors
These errors tend to occur as part of normal operation and should not generate `logger.err` messages
2020-01-09 14:15:01 +00:00
Simon Detheridge 25f8242d6f Merge pull request #60 from overleaf/spd-integration-node10-decaf
Upgrade to Node 10 and decaffeinate
2020-01-09 11:43:30 +00:00
Simon Detheridge d8470d4a59 Use correct object path for s3BucketCreds 2020-01-08 11:32:05 +00:00
Simon Detheridge 9d52c521c4 Merge pull request #74 from overleaf/spd-decaf-cleanup-11
Restore bucket-specific settings and clean up related endpoint
2020-01-08 09:29:41 +00:00
Simon Detheridge 3bf51cac67 Improve naming on internal '_client' method and use Map over object 2020-01-08 09:17:30 +00:00
Simon Detheridge 384896d70c Merge pull request #73 from overleaf/spd-decaf-cleanup-10
Cleanup and refactor S3PersistorManager to use aws-sdk only
2020-01-07 15:49:06 +00:00
Simon Detheridge 90ab85d2b3 Merge pull request #72 from overleaf/spd-decaf-cleanup-9
Post-decaf cleanup of app.js
2020-01-07 15:48:34 +00:00
Simon Detheridge 0c6fedb9dd Merge pull request #71 from overleaf/spd-decaf-cleanup-8
Post-decaf cleanup of PersistorManager and KeyBuilderTests
2020-01-07 15:47:50 +00:00
Simon Detheridge 70dc58db32 Merge pull request #70 from overleaf/spd-decaf-cleanup-7
Decaf cleanup of FileController
2020-01-07 15:47:20 +00:00
Simon Detheridge 6d8da1ade2 Merge pull request #69 from overleaf/spd-decaf-cleanup-6
Decaf cleanup and promisification for FSPersistorManager
2020-01-07 15:46:51 +00:00
Simon Detheridge a65ea5d034 Merge pull request #68 from overleaf/spd-decaf-cleanup-5
Cleanup and promisify acceptance tests. Run tests for all backends.
2020-01-07 15:46:03 +00:00
Simon Detheridge 7781da15a7 Merge pull request #67 from overleaf/spd-decaf-cleanup-4
Decaf and promisify ImageOptimiser
2020-01-07 15:45:15 +00:00
Simon Detheridge cca5b66313 Merge pull request #66 from overleaf/spd-decaf-cleanup-3
Clean up and promisify health-check controller and KeyBuilder
2020-01-07 15:44:23 +00:00
Simon Detheridge 110aad166a Merge pull request #65 from overleaf/spd-decaf-cleanup-2
Cleanup and promisify FileConverter
2020-01-07 15:43:51 +00:00
Simon Detheridge 66fe01e392 Merge pull request #64 from overleaf/spd-decaf-cleanup-1c
Decaf cleanup for FileHandler and LocalFileWriter
2020-01-07 15:42:51 +00:00
Simon Detheridge 9b64aadcfc Merge pull request #62 from overleaf/spd-decaf-cleanup-1a
Decaf cleanup of SafeExec
2020-01-07 15:41:42 +00:00
Simon Detheridge 80d41cf51b Move bucket-specific file endpoint into FileController 2020-01-07 15:05:51 +00:00
Simon Detheridge 6cc5d94f13 Re-add bucket-specific credentials support for S3 2020-01-07 10:55:00 +00:00
Simon Detheridge 2bb2caf7b3 Clean up settings tests 2020-01-07 09:46:53 +00:00
Simon Detheridge ffd002dc76 Add explanatory comment 2020-01-07 09:30:30 +00:00
Simon Detheridge b642b3cf3a Rename incorrectly-named 'getDirectorySize' method 2020-01-07 09:30:30 +00:00
Simon Detheridge a6d9d34fe3 Configure PersistorManager to use the new S3 manager for 'aws-sdk' 2020-01-07 09:30:30 +00:00
Simon Detheridge 2ca74fdf15 Cleanup and refactor S3PersistorManager to use aws-sdk only 2020-01-07 09:30:30 +00:00
Simon Detheridge 473aea4e60 Remove AWSSDKPersistorManager 2020-01-07 09:30:30 +00:00
Simon Detheridge e27cf4db7b Pass pipeline errors onto 'next' error handler 2020-01-07 09:30:30 +00:00
Simon Detheridge 87b51d0c35 Remove old exception-handling and shutdown-related mechanisms 2020-01-07 09:30:28 +00:00
Simon Detheridge 74b480fc55 Post-decaf cleanup of app.js 2020-01-07 09:29:33 +00:00
Simon Detheridge 85d3c0a852 Remove old exception-handling and shutdown-related mechanisms 2020-01-07 09:27:02 +00:00
Simon Detheridge 6a679023d3 Fix order of .status().send() 2020-01-07 09:27:02 +00:00
Simon Detheridge fce275e1d4 Post-decaf cleanup of app.js 2020-01-07 09:27:02 +00:00
Simon Detheridge 0329c759dc Remove wrapper function in PersistorManager 2020-01-07 09:26:43 +00:00
Simon Detheridge dbfacce988 Post-decaf cleanup of PersistorManager 2020-01-07 09:26:43 +00:00