mirror of
https://github.com/yu-i-i/overleaf-cep.git
synced 2026-11-12 21:23:45 +01:00
[monorepo] read empty content from its hash, in a shared blob store base (#36831)
The hash of empty content is a property of the content rather than of anything stored: every project's empty file has it, whether or not a blob was ever written under it, and a file may reference it before anything has. Eight blob stores were each paying a lookup -- a database query, an HTTP request, a read of a cache that offline mode may be the only thing standing in for -- to be told what the hash already says, and clsi had already reinvented the answer on its own. overleaf-editor-core gains BlobStoreBase, which answers getString for that hash and delegates the rest to the fetchString its subclasses provide. getObject comes with it: four stores spelled out the same JSON.parse of getString, and a fifth wrapped it in a log line. Adopted by history-v1's BlobStore, BackupBlobStore, HashCheckBlobStore and recover_zip's RecoveryBlobStore, by project-history's BlobStore, by clsi's BlobStore, and by web's HistoryBlobStore, SimpleBlobStore and FrontendBlobStore. history-v1 keeps its own answers in getStream, getBlob and getBlobs, which no other store implements. Writing and copying a blob are untouched, so a project that wrote empty content still has a blob for it and a read answers the same either way. Two paths could not handle a project holding a tracked change or a comment at all. Loading such a file needs the ranges blob its rangesHash points at, and hashing one again needs to write it back, so a store missing getObject or putObject fails outright: - recover_zip died on `blobStore.getObject is not a function`, making every such project unrecoverable. The test covering it asserts a tracked delete is absent from the zip, which the script already asked for and never reached. - GET /projects/:project_id/latest/hashed_content returned 500 from the same cause, then would have hit a missing putObject in snapshot.store. HashCheckBlobStore becomes a class over the base and gains putObject. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> GitOrigin-RevId: c84f62465beff6cf77ffac26aa772984e1b554cb
This commit is contained in:
committed by
Copybot
co-authored by
Claude Opus 5
parent
7608d738a6
commit
6caa5fc3a3
@@ -1,5 +1,11 @@
|
||||
import pLimit from 'p-limit'
|
||||
import { Change, Chunk, Snapshot, File } from 'overleaf-editor-core'
|
||||
import {
|
||||
BlobStoreBase,
|
||||
Change,
|
||||
Chunk,
|
||||
Snapshot,
|
||||
File,
|
||||
} from 'overleaf-editor-core'
|
||||
import type { RawChange, RawChunk } from 'overleaf-editor-core/lib/types'
|
||||
import { FetchError, getJSON, postJSON } from '@/infrastructure/fetch-json'
|
||||
import path from 'path-browserify'
|
||||
@@ -151,11 +157,13 @@ export class ProjectSnapshot {
|
||||
if (byteLength == null) {
|
||||
return null
|
||||
}
|
||||
let blobStoreOptions
|
||||
if (options?.maxSize != null && byteLength > options?.maxSize) {
|
||||
blobStoreOptions = { maxSize: options.maxSize }
|
||||
const blobStoreOptions = { maxSize: options.maxSize }
|
||||
// Straight to fetchString: a binary file large enough to be read with a size
|
||||
// limit is never the empty content the base class answers for.
|
||||
return await this.blobStore.fetchString(hash, blobStoreOptions)
|
||||
}
|
||||
return await this.blobStore.getString(hash, blobStoreOptions)
|
||||
return await this.blobStore.getString(hash)
|
||||
}
|
||||
|
||||
getDocs(): Map<string, File> {
|
||||
@@ -256,24 +264,20 @@ export class ProjectSnapshot {
|
||||
/**
|
||||
* Blob store that fetches blobs from the history service
|
||||
*/
|
||||
export class SimpleBlobStore {
|
||||
export class SimpleBlobStore extends BlobStoreBase {
|
||||
private projectId: string
|
||||
|
||||
constructor(projectId: string) {
|
||||
super()
|
||||
this.projectId = projectId
|
||||
}
|
||||
|
||||
async getString(
|
||||
async fetchString(
|
||||
hash: string,
|
||||
options?: { maxSize?: number }
|
||||
): Promise<string> {
|
||||
return await fetchBlob(this.projectId, hash, options)
|
||||
}
|
||||
|
||||
async getObject(hash: string) {
|
||||
const blob = await this.getString(hash)
|
||||
return JSON.parse(blob)
|
||||
}
|
||||
}
|
||||
|
||||
async function flushHistory(projectId: string) {
|
||||
|
||||
Reference in New Issue
Block a user