The later 1.4.6 -> 1.4.8 bump ran before the module repositories were cloned,
found no lockfiles and patched nothing, so every module then failed on later's
tinycthread vs glibc >= 2.43. The manual build only escaped because its module
checkouts already existed from an earlier run. Move the patch after the clone
loop and fail if there is nothing to patch.
Claude-Session: https://claude.ai/code/session_01GGe7kUhooMFuzY6GaDb2w9
The tarball host serves archives as immutable for a year, but a rebuild of
the same JASP version reused jasp-desktop-<ver>-x86_64.tar.zst with new bytes,
so a cache could hand out the old tarball against the new PKGBUILD's sha256.
The name now carries the first 12 hex digits of its sha256, and the PKGBUILD
keeps that name locally too, so a stale download in the build directory no
longer fails the checksum either.
Claude-Session: https://claude.ai/code/session_01GGe7kUhooMFuzY6GaDb2w9
jaspModuleTools HEAD emits "version": [[0,96,5]]. JASP 0.98.1 checks
root["version"].isMember("Version"), and jsoncpp throws on isMember for an
array. The exception is caught per manifest and only logged to the log file,
so all 36 bundled modules were dropped and JASP started with none, falling
back to downloading Flatpak-built bundles that cannot load on Arch.
Verified headless: with string versions JASP opens every module_libs entry.
Claude-Session: https://claude.ai/code/session_01GGe7kUhooMFuzY6GaDb2w9
A tarball went out with rpath /opt/jasp-desktop/lib only (a stale
bundle-libs.sh on the runner host). libR.so then resolved to the user's system
R 4.6 through ld.so.conf, which lacks utf8clen, so grDevices and every module
failed to load. Check both binaries' rpath before packing.
Claude-Session: https://claude.ai/code/session_01GGe7kUhooMFuzY6GaDb2w9
Upstream's buildModuleBundlesLocally.R sapply()s over the modules, so the first
failure silently abandons every module after it. The resumed build stopped at
26/36 this way: jaspPower failed on V8, and the nine modules behind it were
never attempted in any of the four passes. build/build-modules.R compiles each
module under tryCatch, reports what failed and carries on, and also skips
reinstalling devtools and jaspModuleTools on every call.
V8 needs libv8, which Arch does not package; DOWNLOAD_STATIC_LIBV8=1 makes its
configure download upstream's static build.
Claude-Session: https://claude.ai/code/session_01GGe7kUhooMFuzY6GaDb2w9
The final check counted *.JASPModule files in the staged tree, but those are
deliberately not shipped: modules are installed into module_libs over the
deduplicated binary_pkgs store. The published 0.98.1 tarball has zero bundle
files, so a full CI build would have failed at its very last step. Check that
every module that built is present in module_libs instead.
Claude-Session: https://claude.ai/code/session_01GGe7kUhooMFuzY6GaDb2w9
Every build started from an empty container and recompiled R, JASP and 1000+
R packages, which is days of CPU on the runner. Now:
- /srv/jasp-cache on the host holds ccache and the renv package cache, linked
into the paths jaspModuleTools::compile() hard-codes.
- RENV_PATHS_PREFIX is fixed: renv otherwise keys the cache on the archlinux
image's build date, so any newer image started cold.
- Cached packages with unresolved sonames after an Arch update are purged, so
only those are rebuilt.
- R packages compile without -g (the Stan/Eigen ones spend real time and GBs of
memory on debug info), and retry passes skip modules that already built.
Claude-Session: https://claude.ai/code/session_01GGe7kUhooMFuzY6GaDb2w9
docker run -v ${WORK_DIR}:/work reaches the runner host's docker daemon, so it
mounts the host's /srv/jasp-build, not the job container's. The job prepared
its scripts in a private copy that never reached the build, which has been
running a stale build-jasp.sh left on the host since 2 September (22 modules,
no remotes pre-seed) and failing every night on "The package remotes is
required". The publish steps had the same blind spot for out/.
Claude-Session: https://claude.ai/code/session_01GGe7kUhooMFuzY6GaDb2w9
Running the package surfaced what shipping only the seven common modules
actually costs. Opening Distributions made JASP download jaspDistributions at
runtime, and it failed to load:
binary_pkgs/43a00e0b.../libjpeg.so.62: cannot open shared object file
Those bundles are built against the Flatpak runtime. libjpeg.so.62 is the
Debian-era soname; Arch has carried libjpeg.so.8 for years, and it is only
available here from the AUR. Installing libjpeg6-turbo fixed that one module,
but every downloaded module has its own dependency set built against the same
foreign runtime, so this is not a fixable class - it is found one soname at a
time, by users.
So build all of them here instead. common stays mandatory; extras are
best-effort, because a few need things Arch does not package at all
(jaspProcess wants V8) and losing one of those is better than losing the
build. The blockers that made me scope down to common in the first place -
gsl, OpenMx's memory, rstanarm, the renv ordering race - are all already
solved, so this is removing a restriction rather than starting over.
First run of the actual binary: ldd reported 'libR.so => not found'.
patchelf --set-rpath replaces the existing rpath rather than appending, so
setting it to $PREFIX/lib discarded the -Wl,-rpath the build had given the
JASP binary for the bundled R. Every other bundled library resolved; only R
went missing, because libR.so/libRblas.so/libRlapack.so live under
R/lib/R/lib rather than in the bundled lib directory. The rpath now lists
both.
Two smaller things the same run surfaced:
- makepkg ran its tidying steps on a tree that was already built and
verified - removing libtool and static library files, purging, compressing
man pages. For a -bin package that is a difference between what was tested
and what gets installed. All of them are now disabled.
- /opt/jasp-desktop/bin/JASP was not on PATH; the package now symlinks
/usr/bin/jasp.
The first complete build produced a 4.3 GB tarball. Modules/ alone was 7 GB
staged, because it contained the same packages twice:
Modules/binary_pkgs 4.5 GB content-addressed, deduplicated, placed
there by ninja install
Modules/*.JASPModule 2.5 GB the same packages again, as per-module
compressed archives, copied by this script
binary_pkgs is the installed form and holds 677 package directories, with
Rcpp, ggplot2 and the rest of the shared base appearing once rather than
once per module. The .JASPModule files are the distribution form used for
on-demand downloads and are redundant next to it.
Dropping them, and the build toolchain removed in the previous commit, takes
the tarball from 4.3 GB to 1.86 GB - the same order as the Flatpak's 1.4 GB
rather than three times over.
Still unverified: that JASP runs from binary_pkgs alone. It is upstream's own
install output, so it should, but nothing has been executed yet.
jaspMixedModels stalled at 6/7 for three passes on rstanarm:
g++: fatal error: Killed signal terminated program cc1plus
ERROR: compilation failed for package 'rstanarm'
Stan generates translation units where one cc1plus wants 4-8 GB. At -j2 on a
9 GB container that is an instant kill, and the symptom is simply a module
that never appears - the same shape as the OpenMx failure earlier, and just
as easy to mistake for something else.
Default R_JOBS to 1, and give the runner swap so an over-commit degrades into
a slow compile instead of a dead one. Documented in the runner requirements,
since a fresh runner without swap would hit this too.
ModuleBundleBuildDir/LibraryR was being staged as ModuleLibrary/, 283 MB of
devtools, roxygen2, testthat, pkgdown, rcmdcheck and jaspModuleTools. That is
the toolchain that *builds* module bundles; JASP loads none of it at runtime.
I added it when I still believed the module directories were source trees
missing their library. Once that turned out to be wrong - the bundles carry
their own dependency closure, jaspTTests alone unpacks to 139
content-addressed package directories - the staging should have been removed
and was not.
The seven-module build stalled at 6/7 on all three passes, deterministically,
with a cause unrelated to jaspMixedModels itself:
ERROR: failed to lock directory '.../LibraryR' for modifying
Error in library("devtools") : there is no package called 'devtools'
R leaves 00LOCK-<pkg> directories behind when an install is interrupted and
then refuses to modify the library at all, and the interrupted install had
also taken devtools with it. Every one of my killed containers poisoned the
library for the runs after it.
Clear stale locks and reinstall whatever is missing before building, rather
than assuming a clean library. This matters for the pipeline too: a job that
times out or gets OOM-killed would otherwise leave the runner unable to
build until someone cleaned up by hand.
Modules/modules-settings.json splits modules into 'common' - the seven
bundled with the application - and 'extra', which JASP fetches on demand
into ~/.local/share/JASP. buildAllDefaultJaspModules.R builds 22 regardless:
it is a developer convenience for having everything locally, not a
definition of what a release contains. I had been treating it as the latter.
The cost of that mistake was not just size. Most of the blockers this build
hit came from modules that are never shipped: gsl blocked jaspNetwork and
jaspBFF, OpenMx blocked jaspSem - all 'extra'. Building the right list drops
them entirely.
Measured: the seven common modules come to ~2.3 GB against ~8.3 GB for all
22. Bundles are already zstd-compressed, so the tarball does not shrink them
further; ~2.5-3 GB total is the realistic figure, against the Flatpak's
1.4 GB. The remaining gap is bundleAll=TRUE giving every module its own copy
of its R dependencies.
The guard now names the modules it expects and reports which are missing.
The retry loop converged 6 -> 10 -> 10 bundles and stalled. The cause was
not a race after all:
g++: fatal error: Killed signal terminated program cc1plus
OpenMx instantiates enough Eigen templates that a single cc1plus needs
several GB, and at -j5 on an 8GB runner the kernel killed it every time -
which is why both retries stopped at exactly 10/22 rather than creeping
forward. That in turn blocked jaspFactor, jaspSem, mxsem and semPlot.
R packages now compile at -j2 (R_JOBS) independently of the -j used for
the C++ build, and the runner VM was given 16GB.
The later bump worked - the build reached staging. Three new problems:
- renv restores concurrently and not always in dependency order: forecast
was lazy-loaded while vctrs was still compiling ('there is no package
called vctrs', six lines before 'vctrs 0.7.3 built from source in 52s').
That aborts the whole sapply() loop, so one race cost 20 modules. Retry
the pass up to three times; renv skips what is installed, so each pass
converges.
- The guard only required a non-empty Modules/local, so a build with 2 of
22 modules passed it and went on to stage and package. It now requires
the full expected count and lists what it did get.
- bundle-libs.sh crashed on the dynamic loader: ldd prints it as
'/lib64/ld-linux-x86-64.so.2 => ...', so the first column is a path and
the ld-linux* denylist entry never matched. Basename the soname before
the check.
Two more failures, both found by the guards added in the previous commit
rather than by shipping a broken package:
- install_github() has been deprecated since devtools 2.5.0 and now
delegates to the 'remotes' package, which is no longer a hard dependency
of devtools. jaspModuleTools could therefore never install, regardless of
GitHub tokens - my earlier diagnosis of a rate limit was wrong. remotes is
now pre-seeded into the library the script sets via .libPaths().
- later 1.4.6, pinned in every module renv.lock, bundles a tinycthread whose
'#define once_flag pthread_once_t' collides with the C11 once_flag that
glibc >= 2.43 declares under C23. Fixed upstream in later 1.4.7
(r-lib/later#257); the 2026-04-22 RSPM snapshot the lockfiles already
point at carries 1.4.8.
The lockfile patch is deliberately an explicit per-package list with a
recorded reason, not a blanket 'use latest' - silently changing the analysis
packages users get would be worse than a build failure.
The first full build succeeded and produced a package that would not have
worked. Three separate causes, all silent:
- R was configured --with-blas --with-lapack, so it linked Arch's external
BLAS and never built libRblas.so/libRlapack.so. The JASP module bundles
expect those inside R_HOME, the way a conventional R and the Flatpak ship
them. Now --without-blas, with a hard check after install.
- buildModuleBundlesLocally.R installs jaspModuleTools via install_github,
which failed on anonymous GitHub rate limits. No bundles were produced at
all. GITHUB_PAT is now required rather than merely recommended.
- buildAllDefaultJaspModules.R calls the bundle builder through system2(),
which ignores exit status, so all of the above exited 0. Added explicit
guards: jaspModuleTools must exist, Modules/local must be non-empty, and
the staged tree must contain the bundles and all three R libraries.
Staging was also wrong: it copied ModuleBundleBuildDir/Modules, which is
cloned module *sources* with zero compiled objects, while dropping
LibraryR entirely - 267MB of R packages built against the bundled R.
Cloning a package that does not exist on the AUR yet gives an empty repo
whose initial branch follows the runner's init.defaultBranch, which is not
guaranteed to be master. Verified against the real AUR remote: auth works,
the empty clone yields a .git, and 'git diff --staged --quiet' correctly
reports changes there - so the first publish commits instead of exiting
early with 'nothing to push'.
Serve from /srv/tarballz on CT148 via a 'location /tarballz/' alias, NOT
from under /var/www/asger: the asger-web deploy swaps that whole directory
atomically, so tarballs kept inside it would be deleted on the next site
deploy. CT148's rootfs was 3G with 2.7G free - one tarball would have
filled it - so it is grown to 15G; speedy's thin pool has room even though
vgs reports the VG nearly full.
Publishing uploads to a .part name and renames on arrival so a download can
never see a truncated file, verifies sha256 on the host, then fetches the
public URL and checks it against the sha256 baked into the PKGBUILD. Both
checks run before the AUR push, so the package cannot go out pointing at a
missing file. Old tarballs are pruned to the newest three.
GH_TOKEN is kept but demoted to an optional GITHUB_PAT for R module
downloads; it no longer has anything to do with publishing.
Checked what the AUR actually requires rather than relying on my earlier
assumption. The submission guidelines say only that packages using prebuilt
deliverables must carry the -bin suffix; nothing requires the binary to come
from upstream, so -bin is correct and sufficient here.
There is no formal 'unofficial binary' flag, so disclose it where users look:
an explicit PROVENANCE block in the PKGBUILD comment header naming the tag,
commit, build recipe and release URL, plus '(prebuilt binary)' in pkgdesc.
Corrects the README, which overstated the risk of a maintainer objecting.
Bundling R alone was not enough: R links libicuuc, and Arch moves ICU
regularly. The obvious fix - pinning 'icu>=78 <79' in depends - is worse
than the disease on a rolling distro, because pacman would then refuse to
upgrade ICU while JASP is installed, blocking the user's whole system
upgrade until a rebuild lands.
So bundle-libs.sh copies every non-system shared library into
/opt/jasp-desktop/lib and rewrites rpaths, by denylist rather than
allowlist. depends now carries no version constraints at all.
Qt6 stays system-provided on purpose - Qt is binary compatible across Qt 6
minors, and bundling QtWebEngine would add ~500 MB. That also means the
only rebuild triggers left are a new JASP release and a Qt major bump;
rebuilding for anything else would cost users a 1.7 GB re-download.
Also fixes check-upstream.py querying only repo=Extra, which silently
returned nothing for icu (it lives in core).
Two bugs found by the first real build:
- 'R --version | head -1' aborted the script with exit 141 under
'set -o pipefail': head closes the pipe and R dies of SIGPIPE. Use
'sed -n 1p', which consumes all input.
- R installed to lib64/R, not the lib/R the script assumed. Pin --libdir
and ask 'R RHOME' for the real path instead of guessing.
Also stop exporting R_HOME. R.cmake warns that a set R_HOME makes R emit
'WARNING: ignoring environment value of R_HOME', which corrupts the output
cmake parses; CUSTOM_R_PATH is the supported way to select an R.
Arch ships R 4.6.1, but JASP pins its R packages against a frozen CRAN
snapshot (2026-03-31 for 0.98.1) that targets R 4.5. Configuring against
Arch's R fails at Rcpp 1.1.1, which cannot compile on R 4.6 since
R_NamespaceRegistry was removed; the Rcpp release that fixes it is newer
than the pinned snapshot, so the lockfile cannot reach it either.
Build R 4.5.3 into /opt/jasp-desktop/R and point R_HOME at it, matching
what upstream CI does. This drops 'r' from depends entirely, so an Arch R
upgrade can no longer break an installed JASP, and the rebuild triggers
become Qt6 and ICU - the libraries actually still linked against.
Reuse the established conventions rather than inventing new ones: the
AUR_SSH_PRIVATE_KEY secret name, pushing from an archlinux:base-devel
job container instead of docker-in-docker, and the existing git identity.
The AUR job only needs the PKGBUILD, so the 1.5 GB tarball never crosses
a job boundary - it goes straight to GitHub Releases from the build job.
Also correct the Gitea job-timeout setting name to ENDLESS_TASK_TIMEOUT.
Upstream JASP ships no Linux binary, so the existing AUR package builds
everything from source including several hundred R packages. This builds
that once in CI and publishes the result as jasp-desktop-bin.
JASP links against the system R and Qt6 rather than bundling them, so a
rebuild is triggered by an Arch r/qt6-base bump as well as by a new JASP
release, and the generated PKGBUILD pins depends to the versions actually
built against.
agw