6589aa6ae0
[misc] test/acceptance: harden the startup check for s3
...
Signed-off-by: Jakob Ackermann <das7pad@outlook.com >
2020-02-28 12:26:24 +01:00
94b8a7f891
[misc] test/unit: KeybuilderTests: use a unique suite label
2020-02-28 12:26:23 +01:00
54e3b57772
[misc] test/unit: add missing globals that are lazy loaded
2020-02-28 12:26:23 +01:00
7920570dd8
[misc] test/unit: add missing require stubs for metrics and settings
2020-02-28 12:26:22 +01:00
492f8abc9f
Update config/settings.defaults.coffee
...
Co-Authored-By: Jakob Ackermann <das7pad@outlook.com >
2020-02-18 10:24:29 +00:00
8acc9ef0ae
Make S3 partSize a setting
2020-02-17 14:06:15 +00:00
7c4bf97a7f
Avoid fetching file a second time when source md5 can't be fetched
2020-02-17 14:06:05 +00:00
9e7eec7a19
Add acceptance test for large files
2020-02-17 14:05:56 +00:00
2f2a819b74
Re-fetch file to calculate md5 if etag is not in correct format
2020-02-17 14:05:43 +00:00
5adfb3e2c0
Use large upload chunks
2020-02-17 14:05:31 +00:00
4e507197fc
Revert "Merge pull request #91 from overleaf/revert-78-spd-migration-persistor"
...
This reverts commit 0cce792604c258538cf012486e7329fa1be7da0f, reversing
changes made to 08b606e22f855121b2d8a2832a5d7e2f45836c3f.
2020-02-17 14:04:42 +00:00
df2ddbe0e1
Revert "Add Migration Persistor, to send missing file requests to a fallback persistor"
2020-02-14 14:26:33 +00:00
e4926e09a7
Move error handler to be the last middleware
2020-02-14 10:58:46 +00:00
31f2572e57
Bump lodash from 4.17.11 to 4.17.15
...
Bumps [lodash](https://github.com/lodash/lodash ) from 4.17.11 to 4.17.15.
- [Release notes](https://github.com/lodash/lodash/releases )
- [Commits](https://github.com/lodash/lodash/compare/4.17.11...4.17.15 )
Signed-off-by: dependabot[bot] <support@github.com >
2020-02-14 09:39:06 +00:00
485cfce6d3
Merge pull request #85 from overleaf/jpa-dependencies-cleanup
...
[misc] rename npm-shrinkwrap.json to package-lock.json and run npm i
2020-02-14 09:24:43 +00:00
16d1674eab
Merge pull request #84 from overleaf/bg-remove-travis-yml
...
Remove unused .travis.yml file
2020-02-14 09:21:03 +00:00
84a80930eb
[misc] rename npm-shrinkwrap.json to package-lock.json and run npm i
2020-02-14 09:19:29 +00:00
44fbc6a733
Merge pull request #82 from overleaf/spd-error-handler-error
...
Fix log info assignment in error handler
2020-02-14 09:18:16 +00:00
3b011258d2
Tidy up md5 hash generation
2020-02-12 13:27:12 +00:00
49ad408b30
Remove unused imports and format correctly
2020-02-12 13:27:12 +00:00
93cd55fb79
Refactor persistors to use a helper for common things
2020-02-12 13:27:12 +00:00
304fdfd35c
Explicitly resume stream after adding listener
2020-02-12 13:21:53 +00:00
6dcf351377
Remove unnecessary 'async'
2020-02-12 13:21:53 +00:00
f4a16cd972
Update tests to properly check for copied files
2020-02-12 13:21:53 +00:00
42a5d168df
Remove unused packages
2020-02-12 13:21:52 +00:00
9e0b378948
Remove minipass as dependency and refactor to make things clearer
2020-02-12 13:21:52 +00:00
5d5d325691
Preserve all error information when cleanup of copied file fails
2020-02-12 13:21:52 +00:00
b4b7fd226e
Add mechanisms to transfer files with md5-based integrity checks
...
Fix error in settings and tidy up tests
Remove unused variable declaration
Remove .only from tests and update eslint rules to catch it in future
Use to catch errors more safely getting md5 hash
Avoid unnecessary call to S3 to get md5 response
2020-02-12 13:21:52 +00:00
2625e03a31
Add MigrationPersistor for sending 404 requests to a fallback persistor
2020-02-12 13:19:48 +00:00
f877f51775
Rename *PersistorManager to *Persistor
2020-02-12 13:19:48 +00:00
a2e1591e93
remove unused .travis.yml file
2020-02-12 12:37:00 +00:00
e3613e52db
Fix log info assignment in error handler
2020-02-12 10:34:56 +00:00
1341af04f5
[misc] update the build scripts to 1.3.5
2020-02-11 18:58:55 +01:00
ca94225c19
Merge pull request #80 from overleaf/jpa-cleanup-install-deps
...
[misc] cleanup install deps
2020-02-10 17:16:44 +01:00
14ba1ea1c1
update to node 10.19.0
2020-02-07 14:16:31 +00:00
ac4361d560
[misc] dockerignore: hide the local data dirs
2020-01-31 15:50:53 +01:00
c1d81dc1ea
[misc] install a maintained ghostscript version and delete package lists
...
also do not install vim and bail out in case any command exited with a
non zero exit code.
2020-01-31 15:50:53 +01:00
997434c930
Remove redundant 'res.logInfo'
2020-01-14 20:55:33 +00:00
035364b62e
Fix RequestLogger object lifecycle
2020-01-14 17:15:01 +00:00
28fb998719
Refactor request logger into a class
2020-01-14 12:02:39 +00:00
9615a06e0f
Improve logging middleware to add info methods to request object
2020-01-10 17:22:06 +00:00
f40fbe77aa
Simplify logging to log only once per http request
2020-01-10 15:41:36 +00:00
e331a3130c
Handle AccessDenied and stream-premature-close errors
...
These errors tend to occur as part of normal operation and should not generate `logger.err` messages
2020-01-09 14:15:01 +00:00
d8470d4a59
Use correct object path for s3BucketCreds
2020-01-08 11:32:05 +00:00
3bf51cac67
Improve naming on internal '_client' method and use Map over object
2020-01-08 09:17:30 +00:00
80d41cf51b
Move bucket-specific file endpoint into FileController
2020-01-07 15:05:51 +00:00
6cc5d94f13
Re-add bucket-specific credentials support for S3
2020-01-07 10:55:00 +00:00
2bb2caf7b3
Clean up settings tests
2020-01-07 09:46:53 +00:00
ffd002dc76
Add explanatory comment
2020-01-07 09:30:30 +00:00
b642b3cf3a
Rename incorrectly-named 'getDirectorySize' method
2020-01-07 09:30:30 +00:00
Jakob Ackermann