mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-11-06 13:26:40 +00:00
feat(pm): unified package manager — pm store foundation
Introduce the pm store: a unified, hash-verified package store that
replaces lazy_deps and the old installer's ad-hoc tool downloads.
Store tools are provisioned on PATH (ffmpeg, node/npm via pinned uv),
with a resumable 8-way downloader, verify() returning failure reasons,
and adopt() made EPERM-safe. chromium ships in the payload for every
target. The 3600-line install.sh is replaced by a staged bootstrapper
(heavy deps are pm's job after this); setup-hermes.sh, Dockerfile and
nix pin tables are rewired onto the store. Old install-script tests,
lazy_deps/managed_uv/build_info, and the ps1/bash installer test
batteries are removed with the machinery they tested.
Rebuilt from ethie/pm onto upstream/main (ac6c8028e0) after the
utf-8-sig sweep. 16 hot files (main also churned them) hand-merged:
platform adapters, main.py, electron/main.ts, tui_gateway/server.py,
cua_backend, installer-tests workflow, install.sh (full rewrite),
setup-hermes.sh, plugins doc.
This commit is contained in:
1 parent
6e294f543d
commit
3d12e86ef1
272 files changed
+18064
-14720
No files matched your search
@@ -48,6 +48,9 @@ outputs:
|
||||
installer:
|
||||
description: Run the PowerShell installer tests on a Windows runner.
|
||||
value: ${{ steps.classify.outputs.installer }}
|
||||
bootstrap:
|
||||
description: Run the bootstrap installer lane (install.sh sandbox + stamp verification).
|
||||
value: ${{ steps.classify.outputs.bootstrap }}
|
||||
rust:
|
||||
description: Run `cargo test` for the Tauri bootstrap installer.
|
||||
value: ${{ steps.classify.outputs.rust }}
|
||||
|
||||
@@ -0,0 +1,137 @@
|
||||
name: Bootstrap installer
|
||||
|
||||
# Exercises the bootstrap-installer path on PRs that can affect it: the
|
||||
# POSIX shell installer (scripts/install.sh), its generated pin fragments,
|
||||
# and the version stamp the `complete` stage ships. The PowerShell installer
|
||||
# keeps its own Windows-only lane (installer-tests.yml); this lane runs the
|
||||
# cheap cross-checks plus a real sandboxed install against the PR checkout.
|
||||
#
|
||||
# Deliberately minimal: the heavy `python-deps` stage (uv sync of every
|
||||
# extra) is skipped — the pm/uv machinery under it is covered by the Python
|
||||
# lanes, and pulling the whole dependency graph per installer PR would make
|
||||
# this lane slower than everything it protects.
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: bootstrap-installer-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
posix:
|
||||
name: install.sh sandbox install + stamp verification
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 15
|
||||
env:
|
||||
INSTALL_DIR: ${{ runner.temp }}/bootstrap-install
|
||||
HERMES_HOME: ${{ runner.temp }}/hermes-home
|
||||
HERMES_RUNTIME_DIR: ${{ runner.temp }}/hermes-tools
|
||||
steps:
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
|
||||
- name: Stage manifest is well-formed protocol v1
|
||||
shell: bash
|
||||
run: |
|
||||
bash scripts/install.sh --manifest | python3 -c '
|
||||
import json, sys
|
||||
m = json.load(sys.stdin)
|
||||
assert m["protocol_version"] == 1, m
|
||||
names = [s["name"] for s in m["stages"]]
|
||||
expected = ["prerequisites", "repository", "venv", "python-deps",
|
||||
"node-deps", "path", "config", "setup", "gateway", "complete"]
|
||||
assert names == expected, names
|
||||
assert m["stages"][-1]["name"] == "complete"
|
||||
print("stage manifest ok:", " -> ".join(names))
|
||||
'
|
||||
|
||||
- name: Generated bootstrap pins match pm/lock.json
|
||||
shell: bash
|
||||
run: python3 scripts/gen-bootstrap-pins.py --check
|
||||
|
||||
- name: Publish the PR checkout as the install source
|
||||
id: source
|
||||
shell: bash
|
||||
run: |
|
||||
# The installer clones --branch <branch>; a PR checkout is a
|
||||
# detached HEAD, so mirror it onto a named branch first.
|
||||
mirror="$RUNNER_TEMP/source-mirror.git"
|
||||
git init --bare "$mirror"
|
||||
sha="$(git rev-parse HEAD)"
|
||||
git push "$mirror" "HEAD:refs/heads/ci-under-test"
|
||||
echo "mirror=$mirror" >> "$GITHUB_OUTPUT"
|
||||
echo "sha=$sha" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Run the installer stages against the PR checkout
|
||||
shell: bash
|
||||
env:
|
||||
HERMES_REPO_URL: ${{ steps.source.outputs.mirror }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
sha="${{ steps.source.outputs.sha }}"
|
||||
run_stage() {
|
||||
echo "::group::stage $1"
|
||||
bash scripts/install.sh --branch ci-under-test --commit "$sha" \
|
||||
--non-interactive --stage "$1" --json
|
||||
echo "::endgroup::"
|
||||
}
|
||||
run_stage prerequisites
|
||||
run_stage repository
|
||||
run_stage venv
|
||||
# python-deps (uv sync --extra all) is deliberately skipped — the
|
||||
# comment at the top of this file explains why.
|
||||
run_stage node-deps
|
||||
run_stage path
|
||||
run_stage config
|
||||
run_stage complete
|
||||
test -f "$INSTALL_DIR/.hermes-bootstrap-complete"
|
||||
|
||||
- name: Verify the shipped version stamp
|
||||
shell: bash
|
||||
run: |
|
||||
python3 scripts/verify-bootstrap-version-stamp.py \
|
||||
--stamp "$INSTALL_DIR/.hermes-bootstrap-complete" \
|
||||
--repo "$INSTALL_DIR" \
|
||||
--expect-commit "${{ steps.source.outputs.sha }}" \
|
||||
--expect-branch ci-under-test
|
||||
|
||||
- name: The pinned commit is on the branch the installer cloned
|
||||
shell: bash
|
||||
run: |
|
||||
# The stamp must describe the bytes actually checked out: the
|
||||
# installed repo's HEAD is exactly the commit the installer pinned.
|
||||
head="$(git -C "$INSTALL_DIR" rev-parse HEAD)"
|
||||
test "$head" = "${{ steps.source.outputs.sha }}" \
|
||||
|| { echo "::error::installed HEAD $head != pinned ${{ steps.source.outputs.sha }}"; exit 1; }
|
||||
|
||||
windows:
|
||||
name: install.ps1 protocol surface
|
||||
runs-on: windows-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
|
||||
# Windows PowerShell 5.1 is what ships with Windows and what `irm | iex`
|
||||
# lands in for most users — the protocol surface must parse there.
|
||||
- name: Protocol version + stage manifest (Windows PowerShell 5.1)
|
||||
shell: powershell
|
||||
run: |
|
||||
$pv = powershell -NoProfile -ExecutionPolicy Bypass -File scripts/install.ps1 -ProtocolVersion
|
||||
if ("$pv" -ne "1") { Write-Error "protocol version '$pv' != 1"; exit 1 }
|
||||
$json = powershell -NoProfile -ExecutionPolicy Bypass -File scripts/install.ps1 -Manifest | ConvertFrom-Json
|
||||
if ($json.protocol_version -ne 1) { Write-Error "manifest protocol_version != 1"; exit 1 }
|
||||
$names = @($json.stages | ForEach-Object { $_.name })
|
||||
if ($names -notcontains "complete") { Write-Error "manifest missing complete stage"; exit 1 }
|
||||
Write-Host "stage manifest ok: $($names -join ' -> ')"
|
||||
|
||||
- name: Protocol version + stage manifest (pwsh 7)
|
||||
shell: pwsh
|
||||
run: |
|
||||
$pv = pwsh -NoProfile -File scripts/install.ps1 -ProtocolVersion
|
||||
if ("$pv" -ne "1") { Write-Error "protocol version '$pv' != 1"; exit 1 }
|
||||
$json = pwsh -NoProfile -File scripts/install.ps1 -Manifest | ConvertFrom-Json
|
||||
if ($json.protocol_version -ne 1) { Write-Error "manifest protocol_version != 1"; exit 1 }
|
||||
Write-Host "stage manifest ok (pwsh 7)"
|
||||
@@ -49,6 +49,7 @@ jobs:
|
||||
uv_lock: ${{ steps.classify.outputs.uv_lock }}
|
||||
npm_lock: ${{ steps.classify.outputs.npm_lock }}
|
||||
installer: ${{ steps.classify.outputs.installer }}
|
||||
bootstrap: ${{ steps.classify.outputs.bootstrap }}
|
||||
rust: ${{ steps.classify.outputs.rust }}
|
||||
docker_meta: ${{ steps.classify.outputs.docker_meta }}
|
||||
mcp_catalog: ${{ steps.classify.outputs.mcp_catalog }}
|
||||
@@ -114,6 +115,15 @@ jobs:
|
||||
if: needs.detect.outputs.rust == 'true'
|
||||
uses: ./.github/workflows/rust-tests.yml
|
||||
|
||||
bootstrap-installer:
|
||||
name: Bootstrap installer
|
||||
needs: detect
|
||||
# The bootstrap-installer path: install.sh, the pin fragments it embeds,
|
||||
# and the version stamp it ships. The PowerShell installer has its own
|
||||
# Windows-only lane above (installer-tests).
|
||||
if: needs.detect.outputs.bootstrap == 'true'
|
||||
uses: ./.github/workflows/bootstrap-installer.yml
|
||||
|
||||
e2e-desktop:
|
||||
name: Desktop E2E
|
||||
needs: detect
|
||||
@@ -223,6 +233,7 @@ jobs:
|
||||
- js-tests
|
||||
- installer-tests
|
||||
- rust-tests
|
||||
- bootstrap-installer
|
||||
- e2e-desktop
|
||||
- docs-site
|
||||
- history-check
|
||||
|
||||
@@ -0,0 +1,675 @@
|
||||
name: Desktop Bundled Release
|
||||
|
||||
# Builds the bundled desktop installer for each (os, arch) target.
|
||||
#
|
||||
# tag admission (shape + pyproject lockstep + ancestry-on-origin/main,
|
||||
# resolved to an IMMUTABLE full SHA) → 6-target matrix → pm bundle
|
||||
# → stage artifacts to R2 (releases/tag/<tag>/) → finalize job merges
|
||||
# the darwin feeds (releases/darwin/<ch>/<channel>-mac.yml + dmg/zip),
|
||||
# the msixbundle job stages the win32 App Installer feeds
|
||||
# (releases/win32/<ch>/*.appinstaller + *.msixbundle) → builds table.
|
||||
# The GitHub release keeps the notes only — no binaries.
|
||||
#
|
||||
# Feed layout (matches apps/desktop/electron/app-updater.ts's arms):
|
||||
# releases/win32/<stable|nightly>/<ch>.appinstaller App Installer feed
|
||||
# releases/win32/<stable|nightly>/*.msixbundle (msixbundle job)
|
||||
# releases/darwin/<stable|nightly>/*-mac.yml electron-updater (dmg/zip)
|
||||
# The finalize + msixbundle jobs write the feeds ONCE after the whole matrix
|
||||
# is green, so a failed leg can never publish a partial channel.
|
||||
#
|
||||
# Payload staging is `hermes pm bundle` on the native runner. There is
|
||||
# no cross-target staging. Signing and notarization are a later commit;
|
||||
# this file produces unsigned artifacts on forks. When the
|
||||
# release-signing environment has the Apple and Azure identifiers,
|
||||
# the same job signs and notarizes.
|
||||
#
|
||||
# scripts/build-bundled-desktop.mjs is the one driver. Local and CI run
|
||||
# the same command. This workflow adds caching and upload only.
|
||||
#
|
||||
# Triggers: workflow_dispatch with an explicit tag. A tag push does not
|
||||
# start this workflow (a bot-pushed nightly tag would never fire).
|
||||
#
|
||||
# R2 secrets (repo-level or the release-signing environment): the R2
|
||||
# account id + an R2 API token (S3-compatible) with read/write on the
|
||||
# release bucket; CLOUDFLARE_R2_BUCKET and CLOUDFLARE_R2_PUBLIC_URL are
|
||||
# non-secret vars. scripts/r2-release.mjs derives the S3 endpoint from
|
||||
# the account id and needs only node — no npm ci, no extra deps.
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
tag:
|
||||
description: 'Release tag to bundle (vX.Y.Z or vX.Y.0-nightly.YYYYMMDDHHMMSS). Must exist on the repo.'
|
||||
required: true
|
||||
type: string
|
||||
upload_release:
|
||||
description: 'Upload artifacts to the R2 release bucket (staging + feeds)'
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
id-token: write
|
||||
|
||||
concurrency:
|
||||
group: desktop-bundled-release-${{ inputs.tag }}
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
validate:
|
||||
name: Validate the tag
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 5
|
||||
outputs:
|
||||
# The tag's commit, resolved ONCE here and exported as a full SHA.
|
||||
# Every privileged job checks out THIS — never the tag ref, which a
|
||||
# force-push can move between the validate and build jobs.
|
||||
sha: ${{ steps.admission.outputs.sha }}
|
||||
steps:
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
ref: ${{ inputs.tag }}
|
||||
# Full history: the admission gate below runs merge-base against
|
||||
# origin/main, which needs the shared ancestry, not a depth-1 tip.
|
||||
fetch-depth: 0
|
||||
fetch-tags: true
|
||||
|
||||
- name: Validate tag shape, pyproject lockstep, and ancestry on origin/main
|
||||
id: admission
|
||||
env:
|
||||
TAG: ${{ inputs.tag }}
|
||||
run: |
|
||||
case "$TAG" in
|
||||
v[0-9]*.[0-9]*.[0-9]*-nightly.20[0-9][0-9][0-9][0-9][0-9][0-9]*)
|
||||
echo "nightly tag: $TAG"
|
||||
;;
|
||||
v[0-9]*.[0-9]*.[0-9]*)
|
||||
echo "tag: $TAG"
|
||||
PYVER=$(sed -n 's/^version = "\([^"]*\)"/\1/p' pyproject.toml | head -1)
|
||||
if [ "$TAG" != "v$PYVER" ]; then
|
||||
echo "::error::tag $TAG does not match pyproject.toml version $PYVER"
|
||||
exit 1
|
||||
fi
|
||||
;;
|
||||
*)
|
||||
echo "::error::'$TAG' is not a release tag (vX.Y.Z or vX.Y.0-nightly.YYYYMMDDHHMMSS)"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
SHA="$(git rev-parse --verify "$TAG^{commit}")"
|
||||
if [ -z "$SHA" ]; then
|
||||
echo "::error::could not resolve $TAG to a commit"
|
||||
exit 1
|
||||
fi
|
||||
# A correctly-shaped tag on an unreviewed commit is NOT a valid
|
||||
# build input for a release-signing workflow: only a commit that
|
||||
# is already an ancestor of origin/main may be built.
|
||||
git fetch origin main
|
||||
if ! git merge-base --is-ancestor "$SHA" origin/main; then
|
||||
echo "::error::tag $TAG (${SHA:0:12}) is not an ancestor of origin/main — refusing to build from an unreviewed commit"
|
||||
exit 1
|
||||
fi
|
||||
echo "tag $TAG resolves to $SHA (on origin/main)"
|
||||
echo "sha=$SHA" >> "$GITHUB_OUTPUT"
|
||||
|
||||
build:
|
||||
name: bundled ${{ matrix.target.label }}
|
||||
needs: validate
|
||||
runs-on: ${{ matrix.target.runner }}
|
||||
environment: release-signing
|
||||
timeout-minutes: 90
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
target:
|
||||
- label: linux-x64
|
||||
runner: ubuntu-24.04
|
||||
- label: linux-arm64
|
||||
runner: ubuntu-24.04-arm
|
||||
- label: darwin-arm64
|
||||
runner: macos-15
|
||||
- label: darwin-x64
|
||||
runner: macos-15-intel
|
||||
- label: win32-x64
|
||||
runner: windows-2025
|
||||
- label: win32-arm64
|
||||
runner: windows-11-arm
|
||||
env:
|
||||
HERMES_DESKTOP_VARIANT: bundled
|
||||
HERMES_PAYLOAD_TAG: ${{ inputs.tag }}
|
||||
ELECTRON_BUILDER_CACHE: ${{ github.workspace }}/.cache/electron-builder
|
||||
ELECTRON_CACHE: ${{ github.workspace }}/.cache/electron
|
||||
electron_config_cache: ${{ github.workspace }}/.cache/electron
|
||||
CLOUDFLARE_R2_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_R2_ACCOUNT_ID }}
|
||||
CLOUDFLARE_R2_ACCESS_KEY_ID: ${{ secrets.CLOUDFLARE_R2_ACCESS_KEY_ID }}
|
||||
CLOUDFLARE_R2_SECRET_ACCESS_KEY: ${{ secrets.CLOUDFLARE_R2_SECRET_ACCESS_KEY }}
|
||||
CLOUDFLARE_R2_BUCKET: ${{ vars.CLOUDFLARE_R2_BUCKET }}
|
||||
CLOUDFLARE_R2_PUBLIC_URL: ${{ vars.CLOUDFLARE_R2_PUBLIC_URL }}
|
||||
steps:
|
||||
- name: Disable Spotlight indexing and XProtect
|
||||
if: startsWith(matrix.target.label, 'darwin-')
|
||||
shell: bash
|
||||
run: |
|
||||
sudo mdutil -a -i off || true
|
||||
sudo pkill -9 XProtect >/dev/null || true
|
||||
while pgrep XProtect; do sleep 3; done
|
||||
|
||||
- name: Exclude build write paths from Defender scanning
|
||||
if: startsWith(matrix.target.label, 'win32-')
|
||||
shell: powershell
|
||||
run: |
|
||||
$paths = @(
|
||||
$env:GITHUB_WORKSPACE,
|
||||
$env:RUNNER_TEMP,
|
||||
(npm config get cache)
|
||||
)
|
||||
foreach ($p in $paths) {
|
||||
try {
|
||||
Add-MpPreference -ExclusionPath $p
|
||||
Write-Host "Defender exclusion added for $p"
|
||||
} catch {
|
||||
Write-Host "Defender exclusion not applied for $p ($($_.Exception.Message)) - continuing"
|
||||
}
|
||||
}
|
||||
try {
|
||||
Set-MpPreference -DisableRealtimeMonitoring $true
|
||||
} catch {
|
||||
Write-Host "Could not disable windows defender"
|
||||
}
|
||||
|
||||
# Check out the SHA the validate job admitted — never the tag ref,
|
||||
# which a force-push can move between jobs. This is the privileged
|
||||
# (release-signing) build; it must run the reviewed bytes.
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
ref: ${{ needs.validate.outputs.sha }}
|
||||
fetch-tags: true
|
||||
|
||||
- name: Resolve toolchain pins from pm/lock.json
|
||||
id: pins
|
||||
shell: bash
|
||||
# The host toolchain that BUILDS the artifact comes from the same
|
||||
# pin table as the embedded runtimes (pm/lock.json), so gate == pin
|
||||
# by construction in build-bundled-desktop.mjs's toolchain gates.
|
||||
run: |
|
||||
python -c '
|
||||
import json
|
||||
pkgs = json.load(open("pm/lock.json"))["packages"]
|
||||
for tool in ("node", "npm", "uv"):
|
||||
print(tool + "=" + pkgs[tool]["version"])
|
||||
' >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Resolve toolchain cache key
|
||||
id: toolchain
|
||||
shell: bash
|
||||
run: |
|
||||
node -e '
|
||||
const l = require("./package-lock.json")
|
||||
const el = l.packages["apps/desktop/node_modules/electron"].version
|
||||
const eb = l.packages["node_modules/electron-builder"].version
|
||||
if (!el || !eb) process.exit(1)
|
||||
console.log(`electron=${el}`)
|
||||
console.log(`builder=${eb}`)
|
||||
' >> "$GITHUB_OUTPUT"
|
||||
|
||||
- uses: actions/setup-node@v7.0.0 # immutable release. safe to pin.
|
||||
with:
|
||||
node-version: ${{ steps.pins.outputs.node }}
|
||||
cache: npm
|
||||
|
||||
- name: Install pinned npm
|
||||
shell: bash
|
||||
env:
|
||||
NPM_PIN: ${{ steps.pins.outputs.npm }}
|
||||
run: npm --version | grep -qx "$NPM_PIN" || npm i -g "npm@$NPM_PIN"
|
||||
|
||||
- uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # 9.0.0
|
||||
with:
|
||||
version: ${{ steps.pins.outputs.uv }}
|
||||
enable-cache: false
|
||||
|
||||
- name: Cache vcpkg OpenSSL (arm64)
|
||||
if: matrix.target.label == 'win32-arm64'
|
||||
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
||||
with:
|
||||
path: C:\vcpkg\installed\arm64-windows-static-md
|
||||
key: vcpkg-openssl-arm64-windows-static-md-${{ runner.os }}
|
||||
|
||||
- name: Install OpenSSL (arm64)
|
||||
if: matrix.target.label == 'win32-arm64'
|
||||
shell: bash
|
||||
run: |
|
||||
if [ ! -f "/c/vcpkg/installed/arm64-windows-static-md/lib/libcrypto.lib" ]; then
|
||||
"$VCPKG_INSTALLATION_ROOT/vcpkg" install openssl:arm64-windows-static-md
|
||||
fi
|
||||
{
|
||||
printf 'OPENSSL_DIR=C:\\vcpkg\\installed\\arm64-windows-static-md\n'
|
||||
printf 'OPENSSL_STATIC=1\n'
|
||||
} >> "$GITHUB_ENV"
|
||||
|
||||
- name: Cache pm store
|
||||
# Tag-dispatched runs (every nightly) scope actions/cache under the
|
||||
# dispatch ref, which GitHub mangles to refs/heads/refs/tags/<tag> —
|
||||
# a different scope per tag, so an exact key can never be restored
|
||||
# by a later nightly. The content key below is stable across tags
|
||||
# when pm/lock.json + uv.lock are unchanged; the restore-keys prefix
|
||||
# (which ignores the tag entirely) rescues the previous nightly's
|
||||
# store when the locks DID move.
|
||||
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
||||
with:
|
||||
path: apps/desktop/build/agent-payload/tools
|
||||
key: pm-store-${{ matrix.target.label }}-${{ hashFiles('pm/lock.json', 'uv.lock') }}
|
||||
restore-keys: |
|
||||
pm-store-${{ matrix.target.label }}-
|
||||
|
||||
- name: Resolve electron's default download cache path
|
||||
shell: bash
|
||||
run: |
|
||||
# @electron/get does NOT honor ELECTRON_CACHE/electron_config_cache:
|
||||
# the electron-builder build's electron zip download uses the
|
||||
# default env-paths cache root. It must be in the actions/cache
|
||||
# path list or every build re-downloads electron (~115MB).
|
||||
case "$RUNNER_OS" in
|
||||
Windows) echo "ELECTRON_DEFAULT_CACHE=$LOCALAPPDATA/electron/Cache" >> "$GITHUB_ENV" ;;
|
||||
macOS) echo "ELECTRON_DEFAULT_CACHE=$HOME/Library/Caches/electron" >> "$GITHUB_ENV" ;;
|
||||
*) echo "ELECTRON_DEFAULT_CACHE=$HOME/.cache/electron" >> "$GITHUB_ENV" ;;
|
||||
esac
|
||||
|
||||
- name: Cache electron + electron-builder toolchain
|
||||
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
||||
with:
|
||||
path: |
|
||||
${{ github.workspace }}/.cache/electron-builder
|
||||
${{ github.workspace }}/.cache/electron
|
||||
${{ env.ELECTRON_DEFAULT_CACHE }}
|
||||
# eb2: bumped from eb- (2026-08-28) — the old eb- caches never
|
||||
# contained the electron zip (it lives in @electron/get's default
|
||||
# cache root, which wasn't in the path list), so every build
|
||||
# re-downloaded electron. The new key forces a fresh save that
|
||||
# includes the default cache root.
|
||||
key: eb2-${{ runner.os }}-${{ runner.arch }}-electron-${{ steps.toolchain.outputs.electron }}-builder-${{ steps.toolchain.outputs.builder }}
|
||||
# An electron/builder bump misses the exact key, but the previous
|
||||
# dist is still mostly reusable (electron's postinstall skips the
|
||||
# download when dist/ exists) — restore it and let npm ci top up.
|
||||
restore-keys: |
|
||||
eb2-${{ runner.os }}-${{ runner.arch }}-
|
||||
|
||||
- name: Cache node_modules
|
||||
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
||||
with:
|
||||
path: |
|
||||
node_modules
|
||||
apps/*/node_modules
|
||||
ui-tui/node_modules
|
||||
ui-tui/packages/*/node_modules
|
||||
web/node_modules
|
||||
tests-js/node_modules
|
||||
key: node-modules-${{ matrix.target.label }}-node${{ steps.pins.outputs.node }}-npm${{ steps.pins.outputs.npm }}-${{ hashFiles('package-lock.json') }}
|
||||
# npm ci rm -rf's node_modules before installing, so a restore is
|
||||
# never shipped stale — but a restore-key hit still makes the
|
||||
# reinstall incremental (postinstall outputs like node-pty's
|
||||
# prebuilds/ and esbuild's platform binary survive in place).
|
||||
# The build-bundled install-stamp gate (lock sha + node + npm +
|
||||
# target) is the real guard against stale trees shipping.
|
||||
restore-keys: |
|
||||
node-modules-${{ matrix.target.label }}-node${{ steps.pins.outputs.node }}-npm${{ steps.pins.outputs.npm }}-
|
||||
|
||||
- name: Cache node-pty prebuilds (postinstall output, emulated-gyp tax on arm64)
|
||||
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
||||
with:
|
||||
path: |
|
||||
node_modules/node-pty/prebuilds
|
||||
node_modules/node-pty/build
|
||||
key: node-pty-prebuilds-${{ matrix.target.label }}-${{ hashFiles('package-lock.json') }}
|
||||
restore-keys: |
|
||||
node-pty-prebuilds-${{ matrix.target.label }}-
|
||||
|
||||
- name: Azure login (OIDC)
|
||||
if: startsWith(matrix.target.label, 'win32-') && vars.AZURE_CLIENT_ID != ''
|
||||
uses: azure/login@f5d393ae46f8fde4be8b75f32e3fc50e654ad0ca # v3.0.1
|
||||
with:
|
||||
client-id: ${{ vars.AZURE_CLIENT_ID }}
|
||||
tenant-id: ${{ vars.AZURE_TENANT_ID }}
|
||||
allow-no-subscriptions: true
|
||||
|
||||
- name: Mint federated token for the signing dlib
|
||||
if: startsWith(matrix.target.label, 'win32-') && vars.AZURE_CLIENT_ID != ''
|
||||
shell: bash
|
||||
run: |
|
||||
file="$RUNNER_TEMP/azure-federated-token"
|
||||
mint() {
|
||||
curl -sSf -H "Authorization: bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" \
|
||||
"$ACTIONS_ID_TOKEN_REQUEST_URL&audience=api://AzureADTokenExchange" \
|
||||
| jq -r .value > "$file.tmp" && mv -f "$file.tmp" "$file"
|
||||
}
|
||||
mint
|
||||
( while sleep 240; do mint || true; done ) &
|
||||
echo "AZURE_FEDERATED_TOKEN_FILE=$file" >> "$GITHUB_ENV"
|
||||
|
||||
- name: Write App Store Connect key for notarytool
|
||||
if: startsWith(matrix.target.label, 'darwin-')
|
||||
shell: bash
|
||||
env:
|
||||
APPLE_API_KEY_P8: ${{ secrets.APPLE_API_KEY_P8 }}
|
||||
run: |
|
||||
if [ -n "$APPLE_API_KEY_P8" ]; then
|
||||
printf '%s\n' "$APPLE_API_KEY_P8" > "$RUNNER_TEMP/apple-api-key.p8"
|
||||
echo "APPLE_API_KEY=$RUNNER_TEMP/apple-api-key.p8" >> "$GITHUB_ENV"
|
||||
else
|
||||
echo "APPLE_API_KEY_P8 secret not set — notarization will be skipped"
|
||||
fi
|
||||
|
||||
- name: Build and package
|
||||
shell: bash
|
||||
timeout-minutes: 85
|
||||
env:
|
||||
PYTHONUTF8: '1'
|
||||
CC_aarch64_pc_windows_msvc: ${{ matrix.target.label == 'win32-arm64' && 'clang' || '' }}
|
||||
DEBUG: 'electron-osx-sign*,electron-notarize*'
|
||||
APPLE_API_KEY_ID: ${{ secrets.APPLE_API_KEY_ID }}
|
||||
APPLE_API_ISSUER: ${{ secrets.APPLE_API_ISSUER }}
|
||||
CSC_LINK: ${{ secrets.CSC_LINK }}
|
||||
CSC_KEY_PASSWORD: ${{ secrets.CSC_KEY_PASSWORD }}
|
||||
AZURE_SIGN_ENDPOINT: ${{ vars.AZURE_SIGN_ENDPOINT }}
|
||||
AZURE_SIGN_ACCOUNT: ${{ vars.AZURE_SIGN_ACCOUNT }}
|
||||
AZURE_SIGN_PROFILE: ${{ vars.AZURE_SIGN_PROFILE }}
|
||||
AZURE_SIGN_PUBLISHER: ${{ vars.AZURE_SIGN_PUBLISHER }}
|
||||
AZURE_CLIENT_ID: ${{ vars.AZURE_CLIENT_ID }}
|
||||
AZURE_TENANT_ID: ${{ vars.AZURE_TENANT_ID }}
|
||||
AZURE_TOKEN_CREDENTIALS: prod
|
||||
run: |
|
||||
if [ "$RUNNER_OS" = "macOS" ]; then
|
||||
ulimit -n 16384 2>/dev/null || true
|
||||
echo "file descriptor limit: soft=$(ulimit -Sn) hard=$(ulimit -Hn)"
|
||||
fi
|
||||
node scripts/build-bundled-desktop.mjs --tag="$HERMES_PAYLOAD_TAG" --variant=bundled
|
||||
# The Store-submission MSIX is the same bundled payload re-packed
|
||||
# with the Partner Center packaging identity (msixbundle job uploads
|
||||
# it to the tag archive; it never enters a feed dir).
|
||||
if [ "$RUNNER_OS" = "Windows" ]; then
|
||||
node scripts/build-bundled-desktop.mjs --tag="$HERMES_PAYLOAD_TAG" --variant=store
|
||||
fi
|
||||
|
||||
# Smoke the bundled payload on its native linux runner before it is
|
||||
# packed into the desktop artifact: the staged interpreter boots
|
||||
# hermes_cli out of the payload with no network and no PYTHONPATH.
|
||||
# This is the ONE linux build+smoke place — pm-bundle.yml no longer
|
||||
# stages or uploads a duplicate linux payload.
|
||||
- name: Smoke test the payload
|
||||
if: startsWith(matrix.target.label, 'linux-')
|
||||
shell: bash
|
||||
run: bash scripts/smoke-payload.sh apps/desktop/build/agent-payload
|
||||
|
||||
- name: Audit bundle architecture
|
||||
shell: bash
|
||||
run: |
|
||||
MATRIX_LABEL="${{ matrix.target.label }}"
|
||||
node apps/desktop/scripts/audit-bundle-arch.mjs \
|
||||
--arch="${MATRIX_LABEL##*-}" --root=apps/desktop/release
|
||||
|
||||
- name: Upload artifacts
|
||||
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
||||
with:
|
||||
# The mac feed ymls for finalize + (win32 legs) the actual .msix +
|
||||
# Store-.msix binaries for the msixbundle job to bundle and stage.
|
||||
# The mac/linux/AppImage binaries go straight to R2 from each leg.
|
||||
name: hermes-bundled-${{ matrix.target.label }}-${{ inputs.tag }}
|
||||
path: |
|
||||
apps/desktop/release/*-mac.yml
|
||||
apps/desktop/release/*.msix
|
||||
retention-days: 30
|
||||
if-no-files-found: warn
|
||||
|
||||
- name: Stage to Cloudflare R2
|
||||
if: inputs.upload_release == true
|
||||
shell: bash
|
||||
# Every artifact goes to releases/tag/<tag>/ (immutable staging).
|
||||
# The finalize job (below) merges these into the win32/darwin feed
|
||||
# dirs after the whole matrix is green — never from a leg, so a
|
||||
# failed leg cannot publish a partial feed.
|
||||
run: |
|
||||
shopt -s nullglob
|
||||
# Store-*.msix are archived by the dedicated loop below — exclude
|
||||
# them here so they are not uploaded twice.
|
||||
files=(apps/desktop/release/*.dmg apps/desktop/release/*.zip \
|
||||
apps/desktop/release/*.msix \
|
||||
apps/desktop/release/*.AppImage \
|
||||
apps/desktop/release/*.blockmap apps/desktop/release/latest*.yml \
|
||||
apps/desktop/release/nightly*.yml)
|
||||
if [ ${#files[@]} -eq 0 ]; then
|
||||
echo "::error::no release artifacts found"; exit 1
|
||||
fi
|
||||
for f in "${files[@]}"; do
|
||||
case "$f" in
|
||||
*/Store-*) continue ;; # archived by the Store loop below
|
||||
esac
|
||||
node scripts/r2-release.mjs put \
|
||||
--tag "$HERMES_PAYLOAD_TAG" --key "$(basename "$f")" --file "$f"
|
||||
done
|
||||
# The Store-submission MSIX (built by the win legs) goes to the tag
|
||||
# archive too — hidden from the builds table and never a feed dir.
|
||||
for f in apps/desktop/release/Store-*.msix; do
|
||||
[ -f "$f" ] || continue
|
||||
node scripts/r2-release.mjs put \
|
||||
--tag "$HERMES_PAYLOAD_TAG" --key "$(basename "$f")" --file "$f"
|
||||
done
|
||||
|
||||
finalize:
|
||||
name: Publish feeds to R2
|
||||
needs: [validate, build]
|
||||
if: inputs.upload_release == true
|
||||
runs-on: ubuntu-24.04
|
||||
environment: release-signing
|
||||
steps:
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
# Privileged job: pin to the SHA validate admitted, not the tag.
|
||||
ref: ${{ needs.validate.outputs.sha }}
|
||||
# Full checkout: r2-release.mjs imports ./msix-shared.mjs, so a
|
||||
# sparse checkout of scripts/r2-release.mjs alone would die with
|
||||
# ERR_MODULE_NOT_FOUND before the finalize run starts.
|
||||
- name: Download staged artifacts
|
||||
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
|
||||
with:
|
||||
pattern: hermes-bundled-*-${{ inputs.tag }}
|
||||
path: staged
|
||||
merge-multiple: true
|
||||
- name: Finalize feeds
|
||||
env:
|
||||
HERMES_PAYLOAD_TAG: ${{ inputs.tag }}
|
||||
CLOUDFLARE_R2_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_R2_ACCOUNT_ID }}
|
||||
CLOUDFLARE_R2_ACCESS_KEY_ID: ${{ secrets.CLOUDFLARE_R2_ACCESS_KEY_ID }}
|
||||
CLOUDFLARE_R2_SECRET_ACCESS_KEY: ${{ secrets.CLOUDFLARE_R2_SECRET_ACCESS_KEY }}
|
||||
CLOUDFLARE_R2_BUCKET: ${{ vars.CLOUDFLARE_R2_BUCKET }}
|
||||
run: node scripts/r2-release.mjs finalize --tag "$HERMES_PAYLOAD_TAG" --dir staged
|
||||
|
||||
msixbundle:
|
||||
name: Bundle the win32 MSIX feeds
|
||||
needs: [validate, build]
|
||||
if: inputs.upload_release == true
|
||||
runs-on: windows-2025
|
||||
environment: release-signing
|
||||
timeout-minutes: 45
|
||||
env:
|
||||
HERMES_PAYLOAD_TAG: ${{ inputs.tag }}
|
||||
ELECTRON_BUILDER_CACHE: ${{ github.workspace }}/.cache/electron-builder
|
||||
CLOUDFLARE_R2_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_R2_ACCOUNT_ID }}
|
||||
CLOUDFLARE_R2_ACCESS_KEY_ID: ${{ secrets.CLOUDFLARE_R2_ACCESS_KEY_ID }}
|
||||
CLOUDFLARE_R2_SECRET_ACCESS_KEY: ${{ secrets.CLOUDFLARE_R2_SECRET_ACCESS_KEY }}
|
||||
CLOUDFLARE_R2_BUCKET: ${{ vars.CLOUDFLARE_R2_BUCKET }}
|
||||
CLOUDFLARE_R2_PUBLIC_URL: ${{ vars.CLOUDFLARE_R2_PUBLIC_URL }}
|
||||
AZURE_SIGN_ENDPOINT: ${{ vars.AZURE_SIGN_ENDPOINT }}
|
||||
AZURE_SIGN_ACCOUNT: ${{ vars.AZURE_SIGN_ACCOUNT }}
|
||||
AZURE_SIGN_PROFILE: ${{ vars.AZURE_SIGN_PROFILE }}
|
||||
AZURE_SIGN_PUBLISHER: ${{ vars.AZURE_SIGN_PUBLISHER }}
|
||||
steps:
|
||||
# Full checkout: stage-msixbundle.mjs imports ./msix-shared.mjs and the
|
||||
# job runs it with no checkout at all today — ERR_MODULE_NOT_FOUND.
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
# Privileged job: pin to the SHA validate admitted, not the tag.
|
||||
ref: ${{ needs.validate.outputs.sha }}
|
||||
# stage-msixbundle derives the nightly MSIX build number as
|
||||
# minutes-since-the-last-stable from git tags — must see them.
|
||||
fetch-tags: true
|
||||
|
||||
- name: Resolve toolchain pins from pm/lock.json
|
||||
id: pins
|
||||
shell: bash
|
||||
run: |
|
||||
python -c '
|
||||
import json
|
||||
pkgs = json.load(open("pm/lock.json"))["packages"]
|
||||
for tool in ("node", "npm", "uv"):
|
||||
print(tool + "=" + pkgs[tool]["version"])
|
||||
' >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Resolve toolchain cache key
|
||||
id: toolchain
|
||||
shell: bash
|
||||
run: |
|
||||
node -e '
|
||||
const l = require("./package-lock.json")
|
||||
const el = l.packages["apps/desktop/node_modules/electron"].version
|
||||
const eb = l.packages["node_modules/electron-builder"].version
|
||||
if (!el || !eb) process.exit(1)
|
||||
console.log(`electron=${el}`)
|
||||
console.log(`builder=${eb}`)
|
||||
' >> "$GITHUB_OUTPUT"
|
||||
|
||||
# makeappx + signtool live in the winCodeSign toolset that
|
||||
# electron-builder downloads into its cache during the build legs; the
|
||||
# msixbundle job runs on a fresh runner, so restore the same eb2 cache
|
||||
# the win32 legs saved. The path list MUST match the build legs'
|
||||
# byte-for-byte — actions/cache derives the version hash from the paths
|
||||
# input, so a shorter list computes a different version and the restore
|
||||
# misses ("Cache not found") even with the identical key.
|
||||
- name: Resolve electron's default download cache path
|
||||
shell: bash
|
||||
run: |
|
||||
case "$RUNNER_OS" in
|
||||
Windows) echo "ELECTRON_DEFAULT_CACHE=$LOCALAPPDATA/electron/Cache" >> "$GITHUB_ENV" ;;
|
||||
macOS) echo "ELECTRON_DEFAULT_CACHE=$HOME/Library/Caches/electron" >> "$GITHUB_ENV" ;;
|
||||
*) echo "ELECTRON_DEFAULT_CACHE=$HOME/.cache/electron" >> "$GITHUB_ENV" ;;
|
||||
esac
|
||||
|
||||
- name: Cache electron + electron-builder toolchain
|
||||
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
||||
with:
|
||||
path: |
|
||||
${{ github.workspace }}/.cache/electron-builder
|
||||
${{ github.workspace }}/.cache/electron
|
||||
${{ env.ELECTRON_DEFAULT_CACHE }}
|
||||
key: eb2-${{ runner.os }}-${{ runner.arch }}-electron-${{ steps.toolchain.outputs.electron }}-builder-${{ steps.toolchain.outputs.builder }}
|
||||
restore-keys: |
|
||||
eb2-${{ runner.os }}-${{ runner.arch }}-
|
||||
|
||||
- name: Download win legs' MSIX
|
||||
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
|
||||
with:
|
||||
pattern: hermes-bundled-win32-*-${{ inputs.tag }}
|
||||
path: apps/desktop/release
|
||||
merge-multiple: true
|
||||
|
||||
- name: Azure login (OIDC)
|
||||
uses: azure/login@f5d393ae46f8fde4be8b75f32e3fc50e654ad0ca # v3.0.1
|
||||
with:
|
||||
client-id: ${{ vars.AZURE_CLIENT_ID }}
|
||||
tenant-id: ${{ vars.AZURE_TENANT_ID }}
|
||||
allow-no-subscriptions: true
|
||||
|
||||
- name: Mint federated token for the signing dlib
|
||||
shell: bash
|
||||
run: |
|
||||
file="$RUNNER_TEMP/azure-federated-token"
|
||||
mint() {
|
||||
curl -sSf -H "Authorization: bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" \
|
||||
"$ACTIONS_ID_TOKEN_REQUEST_URL&audience=api://AzureADTokenExchange" \
|
||||
| jq -r .value > "$file.tmp" && mv -f "$file.tmp" "$file"
|
||||
}
|
||||
mint
|
||||
( while sleep 240; do mint || true; done ) &
|
||||
echo "AZURE_FEDERATED_TOKEN_FILE=$file" >> "$GITHUB_ENV"
|
||||
|
||||
- name: Bundle + stage the MSIX feeds
|
||||
shell: bash
|
||||
env:
|
||||
AZURE_TOKEN_CREDENTIALS: prod
|
||||
AZURE_CLIENT_ID: ${{ vars.AZURE_CLIENT_ID }}
|
||||
AZURE_TENANT_ID: ${{ vars.AZURE_TENANT_ID }}
|
||||
run: |
|
||||
# Out-of-store feed (bundled variant): universal .msixbundle +
|
||||
# .appinstaller per channel; plus re-upload the Store-submission
|
||||
# .msix to the tag archive (never a feed dir).
|
||||
node scripts/stage-msixbundle.mjs --tag "$HERMES_PAYLOAD_TAG" --variant bundled
|
||||
|
||||
builds-pending:
|
||||
name: Mark the builds table as in progress
|
||||
if: inputs.upload_release == true
|
||||
runs-on: ubuntu-24.04
|
||||
steps:
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
- name: Render the placeholder
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
HERMES_PAYLOAD_TAG: ${{ inputs.tag }}
|
||||
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
|
||||
run: |
|
||||
if ! gh release view "$HERMES_PAYLOAD_TAG" >/dev/null 2>&1; then
|
||||
echo "::error::no release exists for $HERMES_PAYLOAD_TAG — cannot mark the builds table"
|
||||
exit 1
|
||||
fi
|
||||
python3 scripts/render-builds-table.py \
|
||||
--tag "$HERMES_PAYLOAD_TAG" --repo "$GITHUB_REPOSITORY" \
|
||||
--pending-run-url "$RUN_URL"
|
||||
|
||||
builds-table:
|
||||
name: Render the release builds table
|
||||
needs: [validate, build, msixbundle, finalize]
|
||||
if: inputs.upload_release == true
|
||||
runs-on: ubuntu-24.04
|
||||
environment: release-signing
|
||||
steps:
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
# Privileged job: pin to the SHA validate admitted. A bare checkout
|
||||
# here would land on the dispatch branch, not the release bytes.
|
||||
ref: ${{ needs.validate.outputs.sha }}
|
||||
- name: Render
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
HERMES_PAYLOAD_TAG: ${{ inputs.tag }}
|
||||
CLOUDFLARE_R2_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_R2_ACCOUNT_ID }}
|
||||
CLOUDFLARE_R2_ACCESS_KEY_ID: ${{ secrets.CLOUDFLARE_R2_ACCESS_KEY_ID }}
|
||||
CLOUDFLARE_R2_SECRET_ACCESS_KEY: ${{ secrets.CLOUDFLARE_R2_SECRET_ACCESS_KEY }}
|
||||
CLOUDFLARE_R2_BUCKET: ${{ vars.CLOUDFLARE_R2_BUCKET }}
|
||||
CLOUDFLARE_R2_PUBLIC_URL: ${{ vars.CLOUDFLARE_R2_PUBLIC_URL }}
|
||||
run: |
|
||||
if ! gh release view "$HERMES_PAYLOAD_TAG" >/dev/null 2>&1; then
|
||||
echo "::error::no release exists for $HERMES_PAYLOAD_TAG — cannot render the builds table"
|
||||
exit 1
|
||||
fi
|
||||
python3 scripts/render-builds-table.py \
|
||||
--tag "$HERMES_PAYLOAD_TAG" --repo "$GITHUB_REPOSITORY"
|
||||
|
||||
publish-nightly:
|
||||
name: Publish the nightly release
|
||||
needs: [build, builds-table, msixbundle, finalize]
|
||||
if: |
|
||||
always()
|
||||
&& inputs.upload_release == true
|
||||
&& contains(inputs.tag, '-nightly.')
|
||||
&& needs.build.result == 'success'
|
||||
&& needs.builds-table.result == 'success'
|
||||
&& needs.msixbundle.result == 'success'
|
||||
&& needs.finalize.result == 'success'
|
||||
runs-on: ubuntu-24.04
|
||||
steps:
|
||||
- name: Publish
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
TAG: ${{ inputs.tag }}
|
||||
run: |
|
||||
gh release edit "$TAG" --repo "$GITHUB_REPOSITORY" --draft=false
|
||||
echo "Published $TAG"
|
||||
+76
-51
@@ -40,7 +40,6 @@ RUN apt-get -o Acquire::Retries=3 update && \
|
||||
make -j"$(nproc)" && \
|
||||
make install
|
||||
|
||||
FROM ghcr.io/astral-sh/uv:0.11.6-python3.13-trixie@sha256:b3c543b6c4f23a5f2df22866bd7857e5d304b67a564f4feab6ac22044dde719b AS uv_source
|
||||
# Node 26 source stage. Debian trixie's bundled nodejs is pinned to 20.x
|
||||
# which reached EOL in April 2026 — we copy node + npm from the upstream
|
||||
# node:26 image instead (Hermes pins its toolchain to Node 26 everywhere).
|
||||
@@ -57,9 +56,14 @@ FROM debian:13.4
|
||||
ENV PYTHONUNBUFFERED=1
|
||||
ENV PYTHONDONTWRITEBYTECODE=1
|
||||
|
||||
# Store Playwright browsers outside the volume mount so the build-time
|
||||
# install survives the /opt/data volume overlay at runtime.
|
||||
ENV PLAYWRIGHT_BROWSERS_PATH=/opt/hermes/.playwright
|
||||
# The pm-pinned Chromium pair lives in the managed tool store at
|
||||
# /opt/hermes/tools — outside the /opt/data volume mount, so the
|
||||
# build-time install survives the volume overlay at runtime. pm's
|
||||
# chromium package fact exports the same value (PLAYWRIGHT_BROWSERS_PATH
|
||||
# at the store root); the image ENV names the same directory so
|
||||
# Playwright and the browser tool resolve the pinned build even before pm
|
||||
# composes tool env.
|
||||
ENV PLAYWRIGHT_BROWSERS_PATH=/opt/hermes/tools
|
||||
|
||||
# Install system dependencies in one layer, clear APT cache.
|
||||
# tini was previously PID 1 to reap orphaned zombie processes (MCP stdio
|
||||
@@ -68,9 +72,16 @@ ENV PLAYWRIGHT_BROWSERS_PATH=/opt/hermes/.playwright
|
||||
# replaces tini with s6-overlay's /init (PID 1 = s6-svscan), which reaps
|
||||
# zombies non-blockingly on SIGCHLD and additionally supervises the main
|
||||
# hermes process, the dashboard, and per-profile gateways.
|
||||
# The second package list is the shared libraries the pinned Chromium links
|
||||
# against. `npx playwright install --with-deps` used to apt-install them as
|
||||
# a side effect; pm stages the pinned browser instead (below), so the libs
|
||||
# must be declared here. The list is the `ldd ... | grep "not found"` set of
|
||||
# the pinned chrome binary in this base image, mapped to trixie package
|
||||
# names (see .hermes/plans/termux-removal-commit-spec.md).
|
||||
RUN apt-get -o Acquire::Retries=3 update && \
|
||||
apt-get -o Acquire::Retries=3 install -y --no-install-recommends \
|
||||
ca-certificates curl iputils-ping python3 python-is-python3 ripgrep ffmpeg gcc g++ make cmake python3-dev python3-venv libffi-dev libolm-dev libatomic1 procps git openssh-client docker-cli xz-utils && \
|
||||
ca-certificates curl iputils-ping python3 python-is-python3 ripgrep ffmpeg gcc g++ make cmake python3-dev python3-venv libffi-dev libolm-dev libatomic1 procps git openssh-client docker-cli xz-utils \
|
||||
libasound2t64 libatk-bridge2.0-0t64 libatk1.0-0t64 libatspi2.0-0t64 libcairo2 libcups2t64 libdbus-1-3 libgbm1 libglib2.0-0t64 libnspr4 libnss3 libpango-1.0-0 libx11-6 libxcb1 libxcomposite1 libxdamage1 libxext6 libxfixes3 libxkbcommon0 libxrandr2 && \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Prefer the fixed SQLite over Debian's vulnerable libsqlite3.so.0. Keep the
|
||||
@@ -149,8 +160,6 @@ COPY --chmod=0755 docker/tini-shim.sh /usr/bin/tini
|
||||
# Non-root user for runtime; UID can be overridden via HERMES_UID at runtime
|
||||
RUN useradd -u 10000 -m -d /opt/data hermes
|
||||
|
||||
COPY --chmod=0755 --from=uv_source /usr/local/bin/uv /usr/local/bin/uvx /usr/local/bin/
|
||||
|
||||
# Node 26: copy the node binary plus the bundled npm JS install from the
|
||||
# upstream image. npm and npx are recreated as symlinks because they're
|
||||
# symlinks in the source image (and need to live on PATH).
|
||||
@@ -168,9 +177,41 @@ RUN ln -sf /usr/local/lib/node_modules/npm/bin/npm-cli.js /usr/local/bin/npm &&
|
||||
|
||||
WORKDIR /opt/hermes
|
||||
|
||||
# ---------- Pinned toolchain from pm/lock.json (single authority) ----------
|
||||
# The image used to assemble uv from a second authority — an astral image
|
||||
# tag (ghcr.io/astral-sh/uv:0.11.6-python3.13-trixie) that had already
|
||||
# drifted to 0.11.6 while pm/lock.json pinned uv 0.12.3. That is exactly
|
||||
# the two-authorities failure the pm design exists to end. The image is now
|
||||
# a pin consumer: the stdlib-only pm provisioner reads pm/lock.json and
|
||||
# stages the pinned uv + the pinned Chromium pair (sha256-verified at
|
||||
# download — the same code path pm.sh/pm.ps1 and the desktop payload use)
|
||||
# into the image's own runtime dir, a self-contained store baked under
|
||||
# /opt/hermes, outside the /opt/data volume so it survives the overlay.
|
||||
# The pinned uv is linked onto PATH so the `uv sync` / `uv pip install`
|
||||
# build steps below run the lockfile's uv, not a second download.
|
||||
#
|
||||
# The Chromium pair is staged here rather than by `npx playwright install`,
|
||||
# which fetched whatever revision the npm-resolved playwright wanted,
|
||||
# unverified, and recorded no fact. The resolved browser binary path is
|
||||
# baked to /etc/hermes/agent-browser-executable-path for stage2-hook.sh:
|
||||
# the layout differs per arch (chrome-linux64/chrome on amd64,
|
||||
# chromium-linux-arm64/chromium on arm64), so it is resolved at build time
|
||||
# and never hunted at boot.
|
||||
ENV HERMES_RUNTIME_DIR=/opt/hermes/tools
|
||||
COPY pm/ pm/
|
||||
RUN set -eu; \
|
||||
python3 -m pm.cli install uv chromium chromium-headless-shell; \
|
||||
ln -sf /opt/hermes/tools/uv-*/uv /usr/local/bin/uv; \
|
||||
uv --version; \
|
||||
browser_bin="$(find /opt/hermes/tools/chromium-* -type f \( -name chrome -o -name chromium \) -print -quit)"; \
|
||||
test -n "$browser_bin"; \
|
||||
"$browser_bin" --version; \
|
||||
mkdir -p /etc/hermes; \
|
||||
printf '%s' "$browser_bin" > /etc/hermes/agent-browser-executable-path
|
||||
|
||||
# ---------- Layer-cached dependency install ----------
|
||||
# Copy only package manifests first so npm install + Playwright are cached
|
||||
# unless the lockfiles themselves change.
|
||||
# Copy only package manifests first so npm install is cached unless the
|
||||
# lockfiles themselves change.
|
||||
#
|
||||
# ui-tui/packages/hermes-ink/ is copied IN FULL (not just its manifests)
|
||||
# because it is referenced as a `file:` workspace dependency from
|
||||
@@ -197,10 +238,6 @@ COPY apps/shared/ apps/shared/
|
||||
ENV npm_config_install_links=false
|
||||
|
||||
RUN npm install --prefer-offline --no-audit --fetch-retries=5 && \
|
||||
for i in 1 2 3; do \
|
||||
npx playwright install --with-deps chromium --only-shell && break || \
|
||||
{ [ "$i" = 3 ] && exit 1; echo "playwright install failed (attempt $i); retrying in 10s"; sleep 10; }; \
|
||||
done && \
|
||||
npm cache clean --force
|
||||
|
||||
# ---------- Photon iMessage sidecar deps (baked, NS-606) ----------
|
||||
@@ -236,8 +273,8 @@ RUN cd plugins/platforms/photon/sidecar && \
|
||||
# plus gateway messaging adapters that should work in the published image
|
||||
# without a first-boot lazy install. We do NOT use `--all-extras`:
|
||||
# that would pull in `[rl]` (atroposlib + tinker + torch + wandb from
|
||||
# git), `[yc-bench]` (another git dep), and `[termux-all]` (Android
|
||||
# redundancy), none of which belong in the published container.
|
||||
# git) and `[yc-bench]` (another git dep), neither of which belongs in
|
||||
# the published container.
|
||||
#
|
||||
# Provider packages (anthropic, bedrock, azure-identity) are included
|
||||
# so Docker users can use these providers without requiring runtime
|
||||
@@ -311,35 +348,33 @@ RUN mkdir -p /opt/hermes/bin && \
|
||||
# `s6-setuidgid hermes` in its run script. If HERMES_UID is unset, services
|
||||
# run as the default hermes user (UID 10000).
|
||||
|
||||
# ---------- Bake image provenance + build-time git revision ----------
|
||||
# ---------- Image provenance + install stamp ----------
|
||||
# CI (.github/workflows/docker.yml) runs scripts/write_install_stamp.py
|
||||
# before `docker build`, so the bulk `COPY . .` above already placed a
|
||||
# full-provenance /opt/hermes/install-stamp.json next to the code.
|
||||
# .dockerignore excludes .git, so the stamp is the only commit channel the
|
||||
# image carries: hermes_cli/version_info.py reads it at runtime (stamp
|
||||
# first, live git second, unknown third), and both `hermes dump` and
|
||||
# banner.get_git_banner_state() consume it through version_info.
|
||||
#
|
||||
# A local `docker build` without CI gets the minimal all-zero fallback
|
||||
# stamp below; version_info skips the placeholder commit, so dump honestly
|
||||
# reports "(unknown)". updateMechanism is `external`: the image is rebuilt
|
||||
# and re-pulled, it never updates itself.
|
||||
#
|
||||
# The versioned, non-secret provenance marker is the authoritative runtime
|
||||
# signal that this filesystem came from an immutable image. It deliberately
|
||||
# lives outside both /opt/hermes (which operators sometimes bind-mount as a
|
||||
# checkout) and /opt/data (the mutable HERMES_HOME volume).
|
||||
# .dockerignore excludes .git, so `git rev-parse HEAD` from inside the
|
||||
# container always returns nothing — meaning `hermes dump` reports
|
||||
# "(unknown)" and the startup banner drops its `· upstream <sha>` suffix.
|
||||
# That makes support triage from container bug reports impossible:
|
||||
# we can't tell which commit the user is actually running.
|
||||
#
|
||||
# Fix: write the commit SHA passed via the HERMES_GIT_SHA build-arg to
|
||||
# /opt/hermes/.hermes_build_sha at build time, and have
|
||||
# hermes_cli/build_info.py read it at runtime. Both `hermes dump` and
|
||||
# banner.get_git_banner_state() try the baked SHA first, then fall back
|
||||
# to live `git rev-parse` for source installs (unchanged behaviour).
|
||||
#
|
||||
# The arg is optional — local `docker build` without --build-arg omits the
|
||||
# SHA file (and records a null provenance revision), so build-info falls back
|
||||
# to live-git lookup. CI
|
||||
# (.github/workflows/docker.yml) passes ${{ github.sha }} so
|
||||
# every published image has it.
|
||||
ARG HERMES_GIT_SHA=
|
||||
# checkout) and /opt/data (the mutable HERMES_HOME volume). Its `revision`
|
||||
# is read from the install stamp; the fallback stamp's all-zero commit maps
|
||||
# to null.
|
||||
RUN set -eu; \
|
||||
if [ -n "${HERMES_GIT_SHA}" ]; then \
|
||||
printf '%s\n' "${HERMES_GIT_SHA}" > /opt/hermes/.hermes_build_sha; \
|
||||
if [ ! -f /opt/hermes/install-stamp.json ]; then \
|
||||
printf '{"schemaVersion":2,"commit":"0000000000000000000000000000000000000000","distribution":"docker","source":"fallback","updateMechanism":"external"}\n' \
|
||||
> /opt/hermes/install-stamp.json; \
|
||||
fi; \
|
||||
mkdir -p /etc/hermes; \
|
||||
HERMES_GIT_SHA="${HERMES_GIT_SHA}" python3 -c 'import json, os, pathlib, tomllib; project = tomllib.loads(pathlib.Path("/opt/hermes/pyproject.toml").read_text(encoding="utf-8"))["project"]; marker = pathlib.Path("/etc/hermes/image-provenance.json"); marker.write_text(json.dumps({"schema": 1, "deployment_kind": "image", "manager": "docker", "image": "nousresearch/hermes-agent", "version": project["version"], "revision": os.environ.get("HERMES_GIT_SHA") or None}, sort_keys=True, separators=(",", ":")) + "\n", encoding="utf-8"); marker.chmod(0o444)'
|
||||
python3 -c 'import json, pathlib, tomllib; project = tomllib.loads(pathlib.Path("/opt/hermes/pyproject.toml").read_text(encoding="utf-8"))["project"]; stamp = json.loads(pathlib.Path("/opt/hermes/install-stamp.json").read_text(encoding="utf-8")); commit = stamp.get("commit"); revision = commit if commit and set(commit) != {"0"} else None; marker = pathlib.Path("/etc/hermes/image-provenance.json"); marker.write_text(json.dumps({"schema": 1, "deployment_kind": "image", "manager": "docker", "image": "nousresearch/hermes-agent", "version": project["version"], "revision": revision}, sort_keys=True, separators=(",", ":")) + "\n", encoding="utf-8"); marker.chmod(0o444)'
|
||||
|
||||
# ---------- s6-overlay service wiring ----------
|
||||
# Static services declared at build time: main-hermes + dashboard.
|
||||
@@ -386,19 +421,9 @@ ENV HERMES_TUI_DIR=/opt/hermes/ui-tui
|
||||
ENV HERMES_HOME=/opt/data
|
||||
ENV HERMES_WRITE_SAFE_ROOT=/opt/data
|
||||
ENV HERMES_DISABLE_LAZY_INSTALLS=1
|
||||
# The published image seals /opt/hermes (root-owned, read-only) so a runtime
|
||||
# lazy install can't mutate the agent's own venv and brick it. But opt-in
|
||||
# backends (Firecrawl web search, Exa, Feishu, …) keep their SDKs in
|
||||
# tools/lazy_deps.py — deliberately NOT baked into [all] (see pyproject.toml
|
||||
# policy 2026-05-12: one quarantined release must not break every install).
|
||||
# Redirect those lazy installs to a writable dir on the durable data volume.
|
||||
# lazy_deps appends this dir to the END of sys.path, so a package installed
|
||||
# here can only ADD modules — it can never shadow or downgrade a core module,
|
||||
# so the sealed-venv guarantee holds even with installs re-enabled. The dir
|
||||
# is seeded + chowned to the hermes user by docker/stage2-hook.sh and lives
|
||||
# on the /opt/data volume, so it persists across container recreates / image
|
||||
# updates (an ABI stamp invalidates it if a rebuild bumps the interpreter).
|
||||
ENV HERMES_LAZY_INSTALL_TARGET=/opt/data/lazy-packages
|
||||
# Lazy installs are fully disabled in the published image (see
|
||||
# HERMES_DISABLE_LAZY_INSTALLS above): the venv is sealed and opt-in backend
|
||||
# SDKs are not installed at runtime.
|
||||
|
||||
# `docker exec` privilege-drop shim. When operators run
|
||||
# `docker exec <c> hermes ...` they default to root, and any file the
|
||||
|
||||
@@ -0,0 +1,155 @@
|
||||
#!/bin/bash
|
||||
# ============================================================================
|
||||
# venv-style activation for the Hermes dev environment (pm-managed tools).
|
||||
#
|
||||
# source ./activate (bash / zsh, repo root)
|
||||
# .\activate.ps1 (PowerShell)
|
||||
#
|
||||
# Emits the composed pm env (PATH + tool vars) into the CURRENT shell, with
|
||||
# save/restore: `deactivate` undoes exactly what activation changed.
|
||||
#
|
||||
# Requires a completed `./setup-hermes.sh` — it never invokes uv. It runs
|
||||
# the pm store's pinned python (fallback: the repo venv) to emit the env
|
||||
# JSON, then exports it here.
|
||||
# ============================================================================
|
||||
# Guard against double-sourcing: venv activate precedent (re-activating is a
|
||||
# no-op re-save; we keep it idempotent by deactivating first).
|
||||
if [ -n "${__HERMES_ACTIVATED:-}" ]; then
|
||||
deactivate
|
||||
fi
|
||||
|
||||
_HERMES_REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
|
||||
# --- target detection (same rules as pm.sh: msys on Windows must report the
|
||||
# machine's truth from the registry, not the emulated shell's) ---
|
||||
_hermes_os="$(uname -s)"
|
||||
case "$_hermes_os" in
|
||||
Linux) _hermes_os=linux ;;
|
||||
Darwin) _hermes_os=darwin ;;
|
||||
MINGW*|MSYS*|CYGWIN*) _hermes_os=win32 ;;
|
||||
*) echo "activate: unsupported OS $(uname -s)" >&2; return 1 2>/dev/null || exit 1 ;;
|
||||
esac
|
||||
if [ "$_hermes_os" = win32 ]; then
|
||||
_hermes_winarch="$(reg.exe query 'HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Environment' /v PROCESSOR_ARCHITECTURE 2>/dev/null | tr -d '
|
||||
' | awk '/PROCESSOR_ARCHITECTURE/ {print $NF}')"
|
||||
# PROCESSOR_ARCHITECTURE reports the EMULATED arch inside an x64-on-
|
||||
# arm64 shell, and uname -m does too. The registry is the machine's
|
||||
# truth; when reg.exe is unreachable, uname -s's release token (e.g.
|
||||
# MINGW64_NT-10.0-28000-ARM64) still names the real machine arch.
|
||||
if [ -z "$_hermes_winarch" ]; then
|
||||
case "$(uname -s)" in
|
||||
*ARM64) _hermes_winarch=ARM64 ;;
|
||||
*AMD64) _hermes_winarch=AMD64 ;;
|
||||
esac
|
||||
fi
|
||||
case "${_hermes_winarch:-}" in
|
||||
ARM64) _hermes_arch=arm64 ;;
|
||||
AMD64) _hermes_arch=x64 ;;
|
||||
*) : ;; # probe unavailable — the store probe below decides
|
||||
esac
|
||||
fi
|
||||
if [ -z "${_hermes_arch:-}" ] && [ "$_hermes_os" != win32 ]; then
|
||||
case "$(uname -m)" in
|
||||
arm64|aarch64) _hermes_arch=arm64 ;;
|
||||
x86_64|amd64) _hermes_arch=x64 ;;
|
||||
*) : ;; # unknown — the store probe below decides
|
||||
esac
|
||||
fi
|
||||
# Resolve the pm store root here so the arch probe below can consult it.
|
||||
_hermes_store="${HERMES_RUNTIME_DIR:-$HOME/.hermes/tools}"
|
||||
if [ -n "${_hermes_arch:-}" ]; then
|
||||
_hermes_target="$_hermes_os-$_hermes_arch"
|
||||
else
|
||||
# Registry + env both unavailable (blanked PATH): ask the store which
|
||||
# target it actually holds instead of guessing an arch. First
|
||||
# python-<ver>-<os>-<arch> entry wins.
|
||||
_hermes_target=""
|
||||
for _hermes_entry in "$_hermes_store"/python-*; do
|
||||
[ -d "$_hermes_entry" ] || continue
|
||||
case "${_hermes_entry##*/}" in
|
||||
python-*-linux-*|python-*-darwin-*|python-*-win32-*)
|
||||
_hermes_target="${_hermes_entry##*python-}"
|
||||
_hermes_target="${_hermes_target#*-}"
|
||||
break
|
||||
;;
|
||||
esac
|
||||
done
|
||||
if [ -z "$_hermes_target" ]; then
|
||||
echo "activate: cannot determine target arch and the store holds no python entry — run ./setup-hermes.sh first" >&2
|
||||
unset _hermes_repo _hermes_os _hermes_arch _hermes_target _hermes_store _hermes_py
|
||||
return 1 2>/dev/null || exit 1
|
||||
fi
|
||||
fi
|
||||
_hermes_py=""
|
||||
for _hermes_entry in "$_hermes_store"/python-*-"$_hermes_target"; do
|
||||
if [ -x "$_hermes_entry/bin/python" ]; then
|
||||
_hermes_py="$_hermes_entry/bin/python"
|
||||
elif [ -x "$_hermes_entry/bin/python.exe" ]; then
|
||||
_hermes_py="$_hermes_entry/bin/python.exe"
|
||||
fi
|
||||
[ -n "$_hermes_py" ] && break
|
||||
done
|
||||
if [ -z "$_hermes_py" ]; then
|
||||
for _hermes_venvpy in "$_hermes_repo/venv/bin/python" "$_hermes_repo/venv/bin/python.exe" \
|
||||
"$_hermes_repo/venv/Scripts/python.exe"; do
|
||||
[ -x "$_hermes_venvpy" ] && { _hermes_py="$_hermes_venvpy"; break; }
|
||||
done
|
||||
fi
|
||||
if [ -z "$_hermes_py" ]; then
|
||||
echo "activate: no pm python found — run ./setup-hermes.sh first" >&2
|
||||
unset _hermes_repo _hermes_os _hermes_arch _hermes_target _hermes_store _hermes_py
|
||||
return 1 2>/dev/null || exit 1
|
||||
fi
|
||||
|
||||
# --- emit export lines from `pm env` ---
|
||||
_hermes_json="$(PYTHONPATH="$_hermes_repo${PYTHONPATH:+:$PYTHONPATH}" "$_hermes_py" -m pm.cli env 2>/dev/null)"
|
||||
if [ -z "$_hermes_json" ] || [ "${_hermes_json#*{}" = "$_hermes_json" ]; then
|
||||
echo "activate: could not read pm env (run ./setup-hermes.sh first)" >&2
|
||||
unset _hermes_repo _hermes_os _hermes_arch _hermes_target _hermes_store _hermes_py _hermes_json
|
||||
return 1 2>/dev/null || exit 1
|
||||
fi
|
||||
|
||||
_hermes_keys="$(printf '%s' "$_hermes_json" | "$_hermes_py" -c 'import json,sys; print(" ".join(json.load(sys.stdin)))' \
|
||||
| tr ' ' '\n' | grep -E '^[A-Za-z_][A-Za-z0-9_]*$' | tr '\n' ' ')"
|
||||
[ -n "$_hermes_keys" ] || _hermes_keys="PATH"
|
||||
|
||||
# --- snapshot what we are about to change (deactivate restores this) ---
|
||||
__HERMES_ACTIVATED=1
|
||||
__HERMES_SAVED_PATH="$PATH"
|
||||
for _hermes_k in $_hermes_keys; do
|
||||
eval "__HERMES_SAVED_$_hermes_k=\"\${$_hermes_k+set}\""
|
||||
eval "__HERMES_PRIOR_$_hermes_k=\${$_hermes_k-__HERMES_UNSET__}"
|
||||
done
|
||||
|
||||
eval "$(printf '%s' "$_hermes_json" | "$_hermes_py" -c '
|
||||
import json, sys, shlex, re
|
||||
for k, v in json.load(sys.stdin).items():
|
||||
# Windows carries names like ProgramFiles(ARM) that are not valid
|
||||
# bash identifiers — skip them (they pass through untouched).
|
||||
if not re.fullmatch(r"[A-Za-z_][A-Za-z0-9_]*", k):
|
||||
continue
|
||||
print("export %s=%s" % (k, shlex.quote(str(v))))')"
|
||||
|
||||
deactivate() {
|
||||
export PATH="$__HERMES_SAVED_PATH"
|
||||
for _hermes_k in $__HERMES_KEY_LIST; do
|
||||
eval "_hermes_was=\"\$__HERMES_SAVED_$_hermes_k\""
|
||||
eval "_hermes_old=\"\$__HERMES_PRIOR_$_hermes_k\""
|
||||
if [ "$_hermes_was" != "set" ]; then
|
||||
unset "$_hermes_k"
|
||||
else
|
||||
export "$_hermes_k=$_hermes_old"
|
||||
fi
|
||||
unset "__HERMES_SAVED_$_hermes_k" "__HERMES_PRIOR_$_hermes_k"
|
||||
done
|
||||
unset __HERMES_SAVED_PATH __HERMES_KEY_LIST __HERMES_ACTIVATED
|
||||
unset -f deactivate
|
||||
unset _hermes_k _hermes_was _hermes_old
|
||||
}
|
||||
|
||||
# remember the key list for deactivate (kept out of the loop vars above)
|
||||
__HERMES_KEY_LIST="$_hermes_keys"
|
||||
|
||||
unset _hermes_repo _hermes_os _hermes_arch _hermes_target _hermes_store \
|
||||
_hermes_py _hermes_json _hermes_keys _hermes_k _hermes_entry _hermes_venvpy \
|
||||
_hermes_winarch
|
||||
@@ -0,0 +1,75 @@
|
||||
# ============================================================================
|
||||
# venv-style activation for the Hermes dev environment (pm-managed tools).
|
||||
#
|
||||
# .\activate.ps1 (repo root; if script execution is disabled:
|
||||
# powershell -ExecutionPolicy Bypass -File .\activate.ps1, or set
|
||||
# Set-ExecutionPolicy RemoteSigned -Scope CurrentUser once)
|
||||
#
|
||||
# Emits the composed pm env (PATH + tool vars) into the CURRENT session, with
|
||||
# save/restore: `deactivate` undoes exactly what activation changed.
|
||||
#
|
||||
# Requires a completed .\setup-hermes.ps1 — it never invokes uv. It runs the
|
||||
# pm store's pinned python (fallback: the repo venv) to emit the env JSON.
|
||||
# ============================================================================
|
||||
$ErrorActionPreference = 'Stop'
|
||||
|
||||
# Guard against double-sourcing: re-activating deactivates first.
|
||||
if (Test-Path function:deactivate) { deactivate }
|
||||
|
||||
$repo = $PSScriptRoot
|
||||
$machineArch = (Get-ItemProperty 'HKLM:\SYSTEM\CurrentControlSet\Control\Session Manager\Environment').PROCESSOR_ARCHITECTURE
|
||||
$arch = if ($machineArch -eq 'ARM64') { 'arm64' } else { 'x64' }
|
||||
$target = "win32-$arch"
|
||||
|
||||
$store = if ($env:HERMES_RUNTIME_DIR) { $env:HERMES_RUNTIME_DIR } else { Join-Path $HOME '.hermes/tools' }
|
||||
|
||||
$py = $null
|
||||
$entry = Get-ChildItem -Path $store -Directory -Filter "python-*-$target" -ErrorAction SilentlyContinue |
|
||||
Sort-Object Name | Select-Object -Last 1
|
||||
if ($entry -and (Test-Path (Join-Path $entry.FullName 'bin/python.exe'))) {
|
||||
$py = Join-Path $entry.FullName 'bin/python.exe'
|
||||
}
|
||||
if (-not $py) {
|
||||
foreach ($candidate in @(
|
||||
(Join-Path $repo 'venv/Scripts/python.exe'),
|
||||
(Join-Path $repo 'venv/bin/python.exe'))) {
|
||||
if (Test-Path $candidate) { $py = $candidate; break }
|
||||
}
|
||||
}
|
||||
if (-not $py) {
|
||||
Write-Error 'activate: no pm python found - run .\setup-hermes.ps1 first'
|
||||
}
|
||||
|
||||
$envJSON = (& $py -m pm.cli env 2>$null) -join "`n"
|
||||
if (-not $envJSON) {
|
||||
Write-Error 'activate: could not read pm env - run .\setup-hermes.ps1 first'
|
||||
}
|
||||
$composed = $envJSON | ConvertFrom-Json
|
||||
|
||||
# --- snapshot what we are about to change (deactivate restores this) ---
|
||||
$global:_hermesKeys = @($composed.PSObject.Properties.Name)
|
||||
$global:_hermesSaved = @{}
|
||||
foreach ($k in $global:_hermesKeys) {
|
||||
$global:_hermesSaved[$k] = [pscustomobject]@{
|
||||
WasSet = (Test-Path "env:$k")
|
||||
Value = [Environment]::GetEnvironmentVariable($k)
|
||||
}
|
||||
}
|
||||
|
||||
foreach ($prop in $composed.PSObject.Properties) {
|
||||
Set-Item -Path "env:$($prop.Name)" -Value ([string]$prop.Value)
|
||||
}
|
||||
|
||||
function global:deactivate {
|
||||
foreach ($k in $global:_hermesKeys) {
|
||||
$saved = $global:_hermesSaved[$k]
|
||||
if ($saved.WasSet) {
|
||||
Set-Item -Path "env:$k" -Value $saved.Value
|
||||
} else {
|
||||
Remove-Item -Path "env:$k" -ErrorAction SilentlyContinue
|
||||
}
|
||||
}
|
||||
$global:_hermesKeys = $null
|
||||
$global:_hermesSaved = $null
|
||||
Remove-Item function:deactivate
|
||||
}
|
||||
@@ -58,12 +58,13 @@ def _get_anthropic_sdk():
|
||||
global _anthropic_sdk
|
||||
if _anthropic_sdk is ...:
|
||||
try:
|
||||
from tools.lazy_deps import ensure as _lazy_ensure
|
||||
_lazy_ensure("provider.anthropic", prompt=False)
|
||||
from pm import ensure_import as _ensure_import
|
||||
_ensure_import("anthropic")
|
||||
except ImportError:
|
||||
pass
|
||||
except Exception:
|
||||
# FeatureUnavailable — fall through to ImportError handling below
|
||||
# InstallError (lazy install disabled/declined/failed) — fall
|
||||
# through to ImportError handling below
|
||||
pass
|
||||
try:
|
||||
import anthropic as _sdk
|
||||
@@ -1089,7 +1090,7 @@ def _read_claude_code_credentials_from_file() -> Optional[Dict[str, Any]]:
|
||||
if not cred_path.exists():
|
||||
return None
|
||||
try:
|
||||
data = json.loads(cred_path.read_text(encoding="utf-8"))
|
||||
data = json.loads(cred_path.read_text(encoding="utf-8-sig"))
|
||||
except (json.JSONDecodeError, OSError, IOError) as e:
|
||||
logger.debug("Failed to read ~/.claude/.credentials.json: %s", e)
|
||||
return None
|
||||
@@ -1299,7 +1300,7 @@ def _write_claude_code_credentials(
|
||||
# Read existing file to preserve other fields
|
||||
existing = {}
|
||||
if cred_path.exists():
|
||||
existing = json.loads(cred_path.read_text(encoding="utf-8"))
|
||||
existing = json.loads(cred_path.read_text(encoding="utf-8-sig"))
|
||||
|
||||
oauth_data: Dict[str, Any] = {
|
||||
"accessToken": access_token,
|
||||
@@ -1704,7 +1705,7 @@ def read_hermes_oauth_credentials() -> Optional[Dict[str, Any]]:
|
||||
oauth_file = _get_hermes_oauth_file()
|
||||
if oauth_file.exists():
|
||||
try:
|
||||
data = json.loads(oauth_file.read_text(encoding="utf-8"))
|
||||
data = json.loads(oauth_file.read_text(encoding="utf-8-sig"))
|
||||
if data.get("accessToken"):
|
||||
return data
|
||||
except (json.JSONDecodeError, OSError, IOError) as e:
|
||||
|
||||
@@ -54,7 +54,7 @@ SCOPE_AI_AZURE_DEFAULT = "https://ai.azure.com/.default"
|
||||
# Lazy SDK import — only loaded when the Entra path is actually used.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_AZURE_IDENTITY_FEATURE = "provider.azure_identity"
|
||||
_AZURE_IDENTITY_FEATURE = "azure-identity"
|
||||
|
||||
|
||||
def has_azure_identity_installed() -> bool:
|
||||
@@ -80,7 +80,7 @@ def _require_azure_identity():
|
||||
return _ai
|
||||
except ImportError:
|
||||
try:
|
||||
from tools.lazy_deps import ensure, FeatureUnavailable
|
||||
from pm import InstallError, ensure_import
|
||||
except ImportError as exc:
|
||||
raise ImportError(
|
||||
"The 'azure-identity' package is required for Azure AI "
|
||||
@@ -89,8 +89,8 @@ def _require_azure_identity():
|
||||
) from exc
|
||||
|
||||
try:
|
||||
ensure(_AZURE_IDENTITY_FEATURE, prompt=False)
|
||||
except FeatureUnavailable as exc:
|
||||
ensure_import(_AZURE_IDENTITY_FEATURE)
|
||||
except InstallError as exc:
|
||||
raise ImportError(
|
||||
"The 'azure-identity' package is required for Azure AI "
|
||||
"Foundry Entra ID authentication. " + str(exc)
|
||||
|
||||
@@ -41,15 +41,15 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Ensure boto3/botocore are installed before any code in this module runs.
|
||||
# Upstream removed boto3 from [all] extras (PRs #24220, #24515); lazy_deps
|
||||
# Upstream removed boto3 from [all] extras (PRs #24220, #24515); pm
|
||||
# handles on-demand installation so the Bedrock provider still works in the
|
||||
# EKS deployment without baking boto3 into the base image.
|
||||
# ---------------------------------------------------------------------------
|
||||
try:
|
||||
from tools.lazy_deps import ensure
|
||||
ensure("provider.bedrock", prompt=False)
|
||||
from pm import ensure_import
|
||||
ensure_import("bedrock")
|
||||
except Exception:
|
||||
pass # lazy_deps unavailable or install failed — let downstream imports surface the real error
|
||||
pass # pm unavailable or install failed — let downstream imports surface the real error
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
+11
-3
@@ -92,9 +92,17 @@ __all__ = [
|
||||
# ``Thread.join(timeout=...)`` deadlines to an absolute timestamp; very large
|
||||
# relative timeouts overflow ``time_t`` on macOS and raise
|
||||
# ``OverflowError: timestamp out of range for platform time_t`` (#83220).
|
||||
# One year is semantically "unbounded" for every wait in this codebase while
|
||||
# staying far below any platform conversion limit.
|
||||
MAX_SAFE_TIMEOUT_S = 31_536_000.0 # 365 days
|
||||
# On Windows the binding constraint is narrower: the underlying
|
||||
# ``WaitForSingleObject`` takes a DWORD *milliseconds* argument, so any
|
||||
# wait above ~49.7 days (0xFFFFFFFF ms) raises OverflowError rather than
|
||||
# waiting (verified live: 4294967.0s ok, 4294967.3s overflows). One year
|
||||
# is the semantic "unbounded" on platforms that can express it; Windows
|
||||
# takes the DWORD-ms cap, which is still semantically "unbounded" for
|
||||
# every wait in this codebase.
|
||||
_WINDOWS_MAX_WAIT_S = (0xFFFFFFFF - 1000) / 1000.0 # DWORD ms, minus a 1s rounding guard
|
||||
MAX_SAFE_TIMEOUT_S = (
|
||||
_WINDOWS_MAX_WAIT_S if sys.platform == "win32" else 31_536_000.0
|
||||
) # 365 days off-Windows
|
||||
|
||||
# Grace period after a deadline fires before concluding the event loop thread
|
||||
# is blocked in a synchronous call and dumping stacks (family A diagnostics).
|
||||
|
||||
+30
-4
@@ -261,11 +261,37 @@ def _spawn_pyright(root: str, ctx: ServerContext) -> Optional[SpawnSpec]:
|
||||
)
|
||||
|
||||
|
||||
def _pm_store_python() -> Optional[str]:
|
||||
"""The pm-provisioned interpreter (facts + store layout), or None.
|
||||
|
||||
Under no-boot-through-venv the running process is the store python and
|
||||
``VIRTUAL_ENV`` is unset, so the old ambient-env probe had nothing to
|
||||
offer; pm's ``python`` fact names the store entry that owns the
|
||||
runtime. It takes the precedence the ambient ``VIRTUAL_ENV`` probe used
|
||||
to have; project-local candidates follow."""
|
||||
try:
|
||||
from pm import paths
|
||||
from pm.lock import Facts
|
||||
except Exception:
|
||||
return None
|
||||
try:
|
||||
fact = Facts(paths.facts_path()).get("python")
|
||||
if not fact or "entry" not in fact:
|
||||
return None
|
||||
entry = paths.store_root() / fact["entry"]
|
||||
exe = entry / ("python.exe" if os.name == "nt" else "bin/python3")
|
||||
return str(exe) if exe.exists() else None
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def _detect_python(root: str) -> Optional[str]:
|
||||
candidates = []
|
||||
if os.environ.get("VIRTUAL_ENV"):
|
||||
candidates.append(os.environ["VIRTUAL_ENV"])
|
||||
candidates.extend([os.path.join(root, ".venv"), os.path.join(root, "venv")])
|
||||
# pm's store interpreter resolves to a full exe path (its layout is not
|
||||
# a venv), so it is checked directly rather than through the v/sub probe.
|
||||
pm_python = _pm_store_python()
|
||||
if pm_python:
|
||||
return pm_python
|
||||
candidates = [os.path.join(root, ".venv"), os.path.join(root, "venv")]
|
||||
for v in candidates:
|
||||
for sub in ("bin/python", "bin/python3", "Scripts/python.exe"):
|
||||
p = os.path.join(v, sub)
|
||||
|
||||
@@ -278,7 +278,7 @@ def _local_probe_disk_cache_path() -> Path:
|
||||
|
||||
def _load_local_probe_disk_cache() -> Dict[str, Any]:
|
||||
try:
|
||||
with _local_probe_disk_cache_path().open("r", encoding="utf-8") as f:
|
||||
with _local_probe_disk_cache_path().open("r", encoding="utf-8-sig") as f:
|
||||
data = json.load(f)
|
||||
return data if isinstance(data, dict) else {}
|
||||
except Exception:
|
||||
@@ -344,7 +344,7 @@ def _load_model_metadata_disk_cache() -> Dict[str, Dict[str, Any]]:
|
||||
"""Load processed OpenRouter metadata cache from disk."""
|
||||
try:
|
||||
cache_path = _get_model_metadata_cache_path()
|
||||
with cache_path.open("r", encoding="utf-8") as f:
|
||||
with cache_path.open("r", encoding="utf-8-sig") as f:
|
||||
data = json.load(f)
|
||||
if not isinstance(data, dict):
|
||||
return {}
|
||||
@@ -1558,7 +1558,7 @@ def _load_context_cache() -> Dict[str, int]:
|
||||
if not path.exists():
|
||||
return {}
|
||||
try:
|
||||
with open(path, encoding="utf-8") as f:
|
||||
with open(path, encoding="utf-8-sig") as f:
|
||||
data = yaml.safe_load(f) or {}
|
||||
return data.get("context_lengths") or {}
|
||||
except Exception as e:
|
||||
|
||||
@@ -181,11 +181,11 @@ def _enabled(config: Dict[str, Any]) -> bool:
|
||||
return bool(gh.get("enabled") and otlp.get("enabled") and otlp.get("endpoint"))
|
||||
|
||||
|
||||
def _require_metrics_sdk(*, auto_install: bool = True, prompt: bool = False) -> Dict[str, Any]:
|
||||
def _require_metrics_sdk(*, auto_install: bool = True) -> Dict[str, Any]:
|
||||
if auto_install:
|
||||
try:
|
||||
from tools.lazy_deps import ensure as _lazy_ensure
|
||||
_lazy_ensure("export.otlp", prompt=prompt)
|
||||
from pm import ensure_import as _lazy_ensure
|
||||
_lazy_ensure("otlp")
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
@@ -594,7 +594,7 @@ def start_gateway_health_export(config: Dict[str, Any]) -> GatewayHealthExportRu
|
||||
|
||||
if gh.get("metrics_enabled", True) or gh.get("diagnostic_events_enabled", True):
|
||||
try:
|
||||
sdk = _require_metrics_sdk(prompt=False)
|
||||
sdk = _require_metrics_sdk()
|
||||
except Exception:
|
||||
logger.warning(
|
||||
"monitoring.gateway_health_export.enabled but OTLP SDK is unavailable; "
|
||||
|
||||
@@ -277,11 +277,11 @@ def _do_upload(
|
||||
Returns a user-facing status string. Never raises.
|
||||
"""
|
||||
try:
|
||||
from tools import lazy_deps
|
||||
lazy_deps.ensure("tool.trace_upload", prompt=False)
|
||||
import pm
|
||||
pm.ensure_import("trace-upload")
|
||||
except Exception:
|
||||
# lazy-install unavailable/declined — fall through to the import,
|
||||
# which surfaces the install hint below if the package is missing.
|
||||
# lazy-install unavailable — fall through to the import, which
|
||||
# surfaces the install hint below if the package is missing.
|
||||
pass
|
||||
try:
|
||||
from huggingface_hub import HfApi
|
||||
|
||||
@@ -24,14 +24,14 @@ from typing import Optional, Tuple
|
||||
from agent.secret_scope import get_secret as _get_secret, is_multiplex_active
|
||||
|
||||
# Ensure google-auth is installed before importing. The [vertex] extra is no
|
||||
# longer in [all] per the lazy-install policy added 2026-05-12 — lazy_deps
|
||||
# longer in [all] per the lazy-install policy added 2026-05-12 — pm
|
||||
# handles on-demand installation so the Vertex provider still works for users
|
||||
# who installed plain `hermes-agent` and only later selected a Gemini model.
|
||||
try:
|
||||
from tools.lazy_deps import ensure as _lazy_ensure
|
||||
_lazy_ensure("provider.vertex", prompt=False)
|
||||
from pm import ensure_import as _ensure_import
|
||||
_ensure_import("vertex")
|
||||
except Exception:
|
||||
pass # lazy_deps unavailable or install failed — fall through to the real ImportError below
|
||||
pass # pm unavailable or install failed — fall through to the real ImportError below
|
||||
|
||||
try:
|
||||
import google.auth
|
||||
|
||||
@@ -0,0 +1,254 @@
|
||||
// THE electron-builder configuration — the whole thing, one file. There is
|
||||
// no "build" field in package.json: run-electron-builder.mjs always passes
|
||||
// --config for this file, so a stray package.json field would be silently
|
||||
// ignored anyway, and splitting the config across JSON + this overlay is
|
||||
// how the two halves drift.
|
||||
//
|
||||
// A .cjs module (not JSON) so the variant is decided at require time:
|
||||
// HERMES_DESKTOP_VARIANT=light builds "Hermes Light". The whole config
|
||||
// derives from that one flag.
|
||||
// @ts-check
|
||||
'use strict'
|
||||
|
||||
const fs = require('node:fs')
|
||||
const path = require('node:path')
|
||||
|
||||
const {
|
||||
light,
|
||||
store,
|
||||
storeMsix,
|
||||
displayName,
|
||||
appId,
|
||||
appNamePascal,
|
||||
channel,
|
||||
msixAppIdWithOrg
|
||||
} = require('./product-identity.cjs')
|
||||
|
||||
// The out-of-store MSIX publisher (ATS cert subject) — single source, shared
|
||||
// with the .appinstaller generator so the manifest and the App Installer can
|
||||
// never drift (see scripts/msix-shared.mjs).
|
||||
const { OUT_OF_STORE_PUBLISHER } = require('../../scripts/msix-shared.mjs')
|
||||
|
||||
/** @typedef {import("app-builder-lib").Configuration} Configuration */
|
||||
|
||||
const [owner, repo] = (process.env.GITHUB_REPOSITORY || 'NousResearch/hermes-agent').split('/')
|
||||
if (!owner || !repo) {
|
||||
throw new Error(`invalid GITHUB_REPOSITORY ${process.env.GITHUB_REPOSITORY}`)
|
||||
}
|
||||
const electronVersion = require('./package.json').devDependencies.electron
|
||||
if (!/^\d+\.\d+\.\d+$/.test(electronVersion)) {
|
||||
throw new Error(`invalid electron version ${electronVersion} in package.json`)
|
||||
}
|
||||
|
||||
/** @type {Configuration} */
|
||||
module.exports = {
|
||||
electronVersion,
|
||||
appId,
|
||||
productName: displayName,
|
||||
executableName: displayName,
|
||||
protocols: [
|
||||
{
|
||||
name: `${displayName} Protocol`,
|
||||
schemes: ['hermes']
|
||||
}
|
||||
],
|
||||
// A store build is archived, never served to a feed — prefix its artifact
|
||||
// so it can't collide with the out-of-store MSIX of the same tag/arch, and
|
||||
// the release pipeline can keep the two apart.
|
||||
artifactName: `${store ? 'Store-' : ''}${appNamePascal}-\${version}-\${os}-\${arch}.\${ext}`,
|
||||
icon: 'assets/icon',
|
||||
// The electron-updater feed. CI builds set CLOUDFLARE_R2_PUBLIC_URL (the R2
|
||||
// public bucket / custom domain) and publish there — the feed yml, blockmaps
|
||||
// and installers all live in the same flat R2 bucket, and electron-updater
|
||||
// resolves the yml's relative artifact paths against it. Builds without the
|
||||
// var (local, or a fork without the R2 vars) keep the github provider, which
|
||||
// is exactly today's behavior. The store build has no feed at all (the Store
|
||||
// owns its distribution and updates).
|
||||
publish: store
|
||||
? null
|
||||
: [
|
||||
process.env.CLOUDFLARE_R2_PUBLIC_URL
|
||||
? { provider: 'generic', url: process.env.CLOUDFLARE_R2_PUBLIC_URL.replace(/\/+$/, ''), channel }
|
||||
: { provider: 'github', owner, repo, channel }
|
||||
],
|
||||
extraMetadata: {
|
||||
name: appNamePascal,
|
||||
desktopName: appId
|
||||
},
|
||||
directories: {
|
||||
output: 'release'
|
||||
},
|
||||
files: ['dist/**', 'assets/**', 'public/**', 'package.json'],
|
||||
beforeBuild: 'scripts/before-build.mjs',
|
||||
beforePack: 'scripts/before-pack.mjs',
|
||||
afterPack: 'scripts/after-pack.mjs',
|
||||
...(process.platform === 'darwin' ? { afterSign: 'scripts/notarize.mjs' } : {}),
|
||||
extraResources: [
|
||||
{
|
||||
from: 'build/install-stamp.json',
|
||||
to: 'install-stamp.json'
|
||||
},
|
||||
{
|
||||
from: 'build/agent-payload',
|
||||
to: 'agent-payload'
|
||||
},
|
||||
{
|
||||
from: 'assets/icon.ico',
|
||||
to: 'icon.ico'
|
||||
}
|
||||
],
|
||||
asar: {
|
||||
unpack: ['**/*.node', '**/prebuilds/**', 'dist/**']
|
||||
},
|
||||
mac: {
|
||||
category: 'public.app-category.developer-tools',
|
||||
extendInfo: {
|
||||
CFBundleDisplayName: displayName,
|
||||
CFBundleExecutable: displayName,
|
||||
CFBundleName: displayName,
|
||||
LSRequiresNativeExecution: true,
|
||||
NSAudioCaptureUsageDescription: `${displayName} uses audio capture for voice conversations.`,
|
||||
NSCameraUsageDescription: `${displayName} uses the camera when a plugin or feature you enable requests it.`,
|
||||
NSMicrophoneUsageDescription: `${displayName} uses the microphone for voice input and voice conversations.`,
|
||||
NSCalendarsUsageDescription: `${displayName} needs access to Calendar to provide requested meeting and scheduling support.`,
|
||||
NSCalendarsFullAccessUsageDescription: `${displayName} needs full access to Calendar to read and manage events when explicitly requested.`,
|
||||
NSRemindersUsageDescription: `${displayName} needs access to Reminders to provide requested personal-assistant and scheduling support.`,
|
||||
NSRemindersFullAccessUsageDescription: `${displayName} needs full access to Reminders to read and manage reminders when explicitly requested.`,
|
||||
NSScreenCaptureUsageDescription: `${displayName} captures the screen when you ask the agent to screenshot or record it.`,
|
||||
NSLocalNetworkUsageDescription: `${displayName} connects to devices on your local network when a plugin or feature you enable requests it.`,
|
||||
NSAppleMusicUsageDescription: `${displayName} accesses your music library when a plugin or feature you enable requests it.`
|
||||
},
|
||||
target: ['dmg', 'zip'],
|
||||
sign: {
|
||||
entitlements: 'electron/entitlements.mac.plist',
|
||||
entitlementsInherit: 'electron/entitlements.mac.inherit.plist',
|
||||
hardenedRuntime: true,
|
||||
ignore: (/** @type {string} */ file) => {
|
||||
try {
|
||||
if (fs.lstatSync(file).isDirectory()) {
|
||||
return false
|
||||
}
|
||||
return !isMachO(file)
|
||||
} catch {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
dmg: {
|
||||
title: `Install ${displayName}`,
|
||||
backgroundColor: '#f5f5f7',
|
||||
iconSize: 96,
|
||||
window: {
|
||||
width: 560,
|
||||
height: 360
|
||||
},
|
||||
contents: [
|
||||
{
|
||||
x: 160,
|
||||
y: 170,
|
||||
type: 'file'
|
||||
},
|
||||
{
|
||||
x: 400,
|
||||
y: 170,
|
||||
type: 'link',
|
||||
path: '/Applications'
|
||||
}
|
||||
]
|
||||
},
|
||||
win: {
|
||||
legalTrademarks: displayName,
|
||||
target: ['msix'],
|
||||
...windowsSigning()
|
||||
},
|
||||
msix: {
|
||||
// A store build uses the Partner Center packaging identity (the Store
|
||||
// re-signs + rewrites the publisher on submission); everything else uses
|
||||
// the out-of-store ATS-cert identity.
|
||||
identityName: store ? storeMsix.identityName : msixAppIdWithOrg,
|
||||
applicationId: appNamePascal,
|
||||
displayName,
|
||||
publisher: store ? storeMsix.publisher : OUT_OF_STORE_PUBLISHER,
|
||||
publisherDisplayName: store ? storeMsix.publisherDisplayName : 'Nous Research',
|
||||
// Nightly MSIX versions are `X.Y.Z.<minutes-since-stable>` (see
|
||||
// scripts/msix-shared.mjs). setBuildNumber makes getVersionInWeirdWindowsForm
|
||||
// use the BUILD_NUMBER env (4th component) instead of hardcoding ".0" — a
|
||||
// stable build sets no BUILD_NUMBER and stays X.Y.Z.0, a nightly build sets
|
||||
// it via build-bundled-desktop.mjs so App Installer updates over equal
|
||||
// nightly-over-nightly versions instead of refusing them.
|
||||
setBuildNumber: true,
|
||||
// Floor Windows 11 22H2. Below build 18307 the manifest schema caps
|
||||
// AppExtension Name at 39 chars and Microsoft's own
|
||||
// "com.microsoft.windows.copilotkeyprovider" is 40 (makeappx
|
||||
// 0x80080204 — A/B-verified against the 26100 kit; 18307 exactly
|
||||
// still failed on it, 22621 passes), and 22621 is the documented
|
||||
// Copilot hardware key floor anyway.
|
||||
minVersion: '10.0.22621.0',
|
||||
maxVersionTested: '10.0.26100.0',
|
||||
// Static path: the file itself is written by scripts/before-build.mjs at
|
||||
// build time (see the comment on the hook) — never at config require
|
||||
// time, so typecheck/test imports don't touch the filesystem.
|
||||
customExtensionsPath: 'build/msix-extensions.xml',
|
||||
customManifestPath: 'assets/msix-manifest.xml',
|
||||
showNameOnTiles: true
|
||||
},
|
||||
linux: {
|
||||
category: 'Development',
|
||||
maintainer: 'Nous Research <support@nousresearch.com>',
|
||||
synopsis: light
|
||||
? 'Remote-only desktop client for Hermes Agent.'
|
||||
: 'Native desktop shell for Hermes Agent.',
|
||||
target: ['AppImage']
|
||||
}
|
||||
}
|
||||
|
||||
// MSIX build-time staging (build/appx icons + build/msix-extensions.xml)
|
||||
// lives in scripts/before-build.mjs — an electron-builder lifecycle hook —
|
||||
// NOT here at require time, so importing this config for typecheck/tests
|
||||
// never writes to the filesystem.
|
||||
|
||||
// Azure Trusted Signing. The hook signs the .msix package itself and the
|
||||
// product exe (after electron-builder's rcedit — the batch in afterPack runs
|
||||
// too early for the exe); every other payload binary is batch-signed in
|
||||
// afterPack via scripts/batch-sign-binaries.mjs, which this hook acknowledges
|
||||
// with `true` so electron-builder never re-signs one-by-one. The package
|
||||
// signature is all Windows install validation checks; inner Authenticode
|
||||
// covers SmartScreen/WDAC tree scans. See batch-sign-binaries.mjs for the
|
||||
// ordering contract.
|
||||
const MACHO_MAGICS = new Set([
|
||||
0xfeedface,
|
||||
0xcefaedfe,
|
||||
0xfeedfacf,
|
||||
0xcffaedfe,
|
||||
0xcafebabe,
|
||||
0xbebafeca
|
||||
])
|
||||
|
||||
/** @param {string} file */
|
||||
function isMachO(file) {
|
||||
const buf = Buffer.alloc(4)
|
||||
const fd = fs.openSync(file, 'r')
|
||||
try {
|
||||
if (fs.readSync(fd, buf, 0, 4, 0) !== 4) {
|
||||
return false
|
||||
}
|
||||
} finally {
|
||||
fs.closeSync(fd)
|
||||
}
|
||||
return MACHO_MAGICS.has(buf.readUInt32BE(0))
|
||||
}
|
||||
|
||||
function windowsSigning() {
|
||||
if (!process.env.AZURE_SIGN_ENDPOINT || !process.env.AZURE_CLIENT_ID) {
|
||||
return {}
|
||||
}
|
||||
return {
|
||||
sign: {
|
||||
type: 'signtool',
|
||||
sign: './scripts/batch-sign-binaries.mjs',
|
||||
signingHashAlgorithms: ['sha256'],
|
||||
publisherName: process.env.AZURE_SIGN_PUBLISHER
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -6,142 +6,61 @@ import { test } from 'vitest'
|
||||
import {
|
||||
appendUniquePathEntries,
|
||||
buildDesktopBackendEnv,
|
||||
buildDesktopBackendPath,
|
||||
hermesManagedNodePathEntries,
|
||||
normalizeHermesHomeRoot,
|
||||
pathEnvKey,
|
||||
POSIX_SANE_PATH_ENTRIES
|
||||
} from './backend-env'
|
||||
|
||||
test('desktop backend PATH adds Hermes-managed bins and missing POSIX sane entries', () => {
|
||||
const result = buildDesktopBackendPath({
|
||||
hermesHome: '/Users/test/.hermes',
|
||||
venvRoot: '/Users/test/.hermes/hermes-agent/venv',
|
||||
currentPath: '/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin',
|
||||
platform: 'darwin',
|
||||
pathModule: path.posix
|
||||
test('backend env scrubs PYTHONPATH and PYTHONHOME', () => {
|
||||
const env = buildDesktopBackendEnv({
|
||||
currentEnv: {
|
||||
PATH: '/usr/bin:/bin',
|
||||
PYTHONPATH: '/leaked/other/checkout',
|
||||
PYTHONHOME: '/leaked/python'
|
||||
},
|
||||
platform: 'darwin'
|
||||
})
|
||||
|
||||
const entries = result.split(':')
|
||||
// Both managed-Node layouts lead, POSIX-native shape first, then the venv.
|
||||
assert.deepEqual(entries.slice(0, 3), [
|
||||
'/Users/test/.hermes/node/bin',
|
||||
'/Users/test/.hermes/node',
|
||||
'/Users/test/.hermes/hermes-agent/venv/bin'
|
||||
])
|
||||
assert.ok(entries.includes('/opt/homebrew/bin'), 'Apple Silicon Homebrew bin is added')
|
||||
assert.ok(entries.includes('/opt/homebrew/sbin'), 'Apple Silicon Homebrew sbin is added')
|
||||
assert.ok(entries.includes('/usr/local/sbin'), 'missing standard sbin is added')
|
||||
assert.equal(env.PYTHONPATH, '')
|
||||
assert.equal(env.PYTHONHOME, '')
|
||||
})
|
||||
|
||||
test('POSIX backend PATH keeps the inherited PATH first and appends missing sane entries', () => {
|
||||
const env = buildDesktopBackendEnv({
|
||||
currentEnv: { PATH: '/opt/homebrew/bin:/usr/bin:/bin' },
|
||||
platform: 'darwin'
|
||||
})
|
||||
|
||||
const entries = env.PATH.split(':')
|
||||
assert.equal(entries[0], '/opt/homebrew/bin', 'inherited PATH keeps precedence')
|
||||
assert.equal(entries.filter(entry => entry === '/opt/homebrew/bin').length, 1, 'no duplicates')
|
||||
|
||||
for (const expected of POSIX_SANE_PATH_ENTRIES) {
|
||||
assert.ok(entries.includes(expected), `${expected} should be present`)
|
||||
}
|
||||
})
|
||||
|
||||
test('managed Node dirs lead with the platform-native layout but always offer both', () => {
|
||||
const posix = hermesManagedNodePathEntries('/Users/test/.hermes', {
|
||||
platform: 'darwin',
|
||||
pathModule: path.posix
|
||||
})
|
||||
|
||||
const windows = hermesManagedNodePathEntries('C:\\Users\\test\\AppData\\Local\\hermes', {
|
||||
platform: 'win32',
|
||||
pathModule: path.win32
|
||||
})
|
||||
|
||||
// install.sh uses node/bin; install.ps1 unpacks node.exe into node\ itself.
|
||||
// Both shapes are always emitted so migrated installs keep resolving.
|
||||
assert.deepEqual(posix, ['/Users/test/.hermes/node/bin', '/Users/test/.hermes/node'])
|
||||
assert.deepEqual(windows, [
|
||||
'C:\\Users\\test\\AppData\\Local\\hermes\\node',
|
||||
'C:\\Users\\test\\AppData\\Local\\hermes\\node\\bin'
|
||||
])
|
||||
})
|
||||
|
||||
test('managed Node dirs are empty without a Hermes home', () => {
|
||||
assert.deepEqual(hermesManagedNodePathEntries(undefined, { platform: 'darwin', pathModule: path.posix }), [])
|
||||
assert.deepEqual(hermesManagedNodePathEntries('', { platform: 'win32', pathModule: path.win32 }), [])
|
||||
})
|
||||
|
||||
test('every managed Node dir outranks the inherited PATH on both platforms', () => {
|
||||
for (const [platform, pathModule, home, inherited, delimiter] of [
|
||||
['darwin', path.posix, '/Users/test/.hermes', '/usr/local/bin:/usr/bin', ':'],
|
||||
['win32', path.win32, 'C:\\hermes', 'C:\\Program Files\\nodejs;C:\\Windows\\System32', ';']
|
||||
] as const) {
|
||||
const entries = buildDesktopBackendPath({
|
||||
hermesHome: home,
|
||||
venvRoot: null,
|
||||
currentPath: inherited,
|
||||
platform,
|
||||
pathModule
|
||||
}).split(delimiter)
|
||||
|
||||
const managed = hermesManagedNodePathEntries(home, { platform, pathModule })
|
||||
const firstInherited = Math.min(...inherited.split(delimiter).map(entry => entries.indexOf(entry)))
|
||||
|
||||
for (const dir of managed) {
|
||||
assert.ok(
|
||||
entries.indexOf(dir) >= 0 && entries.indexOf(dir) < firstInherited,
|
||||
`${dir} must precede the inherited PATH on ${platform}`
|
||||
)
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
test('desktop backend PATH preserves first occurrence and avoids duplicates', () => {
|
||||
const result = buildDesktopBackendPath({
|
||||
hermesHome: '/Users/test/.hermes',
|
||||
venvRoot: '/Users/test/.hermes/hermes-agent/venv',
|
||||
currentPath: '/opt/homebrew/bin:/usr/bin:/opt/homebrew/bin:/bin',
|
||||
platform: 'darwin',
|
||||
pathModule: path.posix
|
||||
})
|
||||
|
||||
const entries = result.split(':')
|
||||
assert.equal(entries.filter(entry => entry === '/opt/homebrew/bin').length, 1)
|
||||
assert.ok(
|
||||
entries.indexOf('/opt/homebrew/bin') < entries.indexOf('/opt/homebrew/sbin'),
|
||||
'existing Homebrew bin keeps its precedence over appended missing sane entries'
|
||||
)
|
||||
})
|
||||
|
||||
test('buildDesktopBackendEnv extends PYTHONPATH and backend PATH together', () => {
|
||||
test('Windows PATH casing and delimiter are preserved without POSIX sane entries', () => {
|
||||
const env = buildDesktopBackendEnv({
|
||||
hermesHome: '/Users/test/.hermes',
|
||||
pythonPathEntries: ['/repo/hermes-agent'],
|
||||
venvRoot: '/Users/test/.hermes/hermes-agent/venv',
|
||||
currentEnv: {
|
||||
PATH: '/usr/bin:/bin',
|
||||
PYTHONPATH: '/existing/pythonpath'
|
||||
},
|
||||
platform: 'darwin',
|
||||
pathModule: path.posix
|
||||
currentEnv: { Path: 'C:\\Windows\\System32;C:\\Windows' },
|
||||
platform: 'win32'
|
||||
})
|
||||
|
||||
assert.equal(env.PYTHONPATH, '/repo/hermes-agent:/existing/pythonpath')
|
||||
assert.ok(
|
||||
env.PATH.startsWith(
|
||||
'/Users/test/.hermes/node/bin:/Users/test/.hermes/node:/Users/test/.hermes/hermes-agent/venv/bin:'
|
||||
)
|
||||
)
|
||||
assert.ok(env.PATH.includes('/opt/homebrew/bin'))
|
||||
assert.equal(env.Path, 'C:\\Windows\\System32;C:\\Windows')
|
||||
assert.equal(env.PATH, undefined)
|
||||
})
|
||||
|
||||
test('buildDesktopBackendEnv forces PYTHONUTF8 unless the user set it explicitly', () => {
|
||||
const defaulted = buildDesktopBackendEnv({
|
||||
hermesHome: '/Users/test/.hermes',
|
||||
currentEnv: { PATH: '/usr/bin' },
|
||||
platform: 'darwin',
|
||||
pathModule: path.posix
|
||||
platform: 'darwin'
|
||||
})
|
||||
|
||||
assert.equal(defaulted.PYTHONUTF8, '1')
|
||||
|
||||
const optedOut = buildDesktopBackendEnv({
|
||||
hermesHome: '/Users/test/.hermes',
|
||||
currentEnv: { PATH: '/usr/bin', PYTHONUTF8: '0' },
|
||||
platform: 'darwin',
|
||||
pathModule: path.posix
|
||||
platform: 'darwin'
|
||||
})
|
||||
|
||||
assert.equal(optedOut.PYTHONUTF8, '0')
|
||||
@@ -159,33 +78,13 @@ test('normalizeHermesHomeRoot maps profile homes back to the global Hermes root'
|
||||
assert.equal(normalizeHermesHomeRoot('/Users/test/.hermes', { pathModule: path.posix }), '/Users/test/.hermes')
|
||||
})
|
||||
|
||||
test('Windows PATH casing and delimiter are preserved without POSIX sane entries', () => {
|
||||
const env = buildDesktopBackendEnv({
|
||||
hermesHome: 'C:\\Users\\test\\AppData\\Local\\hermes',
|
||||
pythonPathEntries: ['C:\\repo\\hermes-agent'],
|
||||
venvRoot: 'C:\\Users\\test\\AppData\\Local\\hermes\\hermes-agent\\venv',
|
||||
currentEnv: {
|
||||
Path: 'C:\\Windows\\System32;C:\\Windows',
|
||||
PYTHONPATH: 'C:\\existing\\pythonpath'
|
||||
},
|
||||
platform: 'win32',
|
||||
pathModule: path.win32
|
||||
})
|
||||
|
||||
test('pathEnvKey finds the platform-cased PATH key', () => {
|
||||
assert.equal(pathEnvKey({ Path: 'x' }, 'win32'), 'Path')
|
||||
assert.equal(env.PATH, undefined)
|
||||
// Windows leads with the portable layout (install.ps1 unpacks node.exe
|
||||
// straight into node\, no bin\), then the POSIX shape for migrated installs.
|
||||
assert.ok(
|
||||
env.Path.startsWith(
|
||||
'C:\\Users\\test\\AppData\\Local\\hermes\\node;C:\\Users\\test\\AppData\\Local\\hermes\\node\\bin;'
|
||||
)
|
||||
)
|
||||
assert.ok(env.Path.includes('\\venv\\Scripts;'))
|
||||
assert.ok(env.Path.includes(';C:\\Windows\\System32;C:\\Windows'))
|
||||
assert.equal(env.Path.includes('/opt/homebrew/bin'), false)
|
||||
assert.equal(pathEnvKey({ PATH: 'x' }, 'win32'), 'PATH')
|
||||
assert.equal(pathEnvKey({}, 'win32'), 'PATH')
|
||||
assert.equal(pathEnvKey({ Path: 'x' }, 'darwin'), 'PATH')
|
||||
})
|
||||
|
||||
test('appendUniquePathEntries drops empty entries and keeps first occurrence', () => {
|
||||
assert.equal(appendUniquePathEntries([':/a::/b', ['/a', '/c']], { delimiter: ':' }), '/a:/b:/c')
|
||||
test('appendUniquePathEntries flattens, dedupes, and preserves first occurrence', () => {
|
||||
assert.equal(appendUniquePathEntries(['/a:/b', ['/b', '/c'], '', null], { delimiter: ':' }), '/a:/b:/c')
|
||||
})
|
||||
@@ -1,8 +1,9 @@
|
||||
import path from 'node:path'
|
||||
|
||||
// Match the POSIX fallback surface used by the Python terminal environment.
|
||||
// macOS apps launched from Finder/Dock often inherit only /usr/bin:/bin:/usr/sbin:/sbin,
|
||||
// which misses Apple Silicon Homebrew and user-installed CLI tools such as codex.
|
||||
// macOS apps launched from Finder/Dock inherit only /usr/bin:/bin:/usr/sbin:/sbin,
|
||||
// which misses Homebrew and user-installed CLI tools (codex, git credential
|
||||
// helpers). Hermes' own managed tools need no PATH help — the backend composes
|
||||
// their environment in-process via pm — but user tools on PATH do.
|
||||
const POSIX_SANE_PATH_ENTRIES = Object.freeze([
|
||||
'/opt/homebrew/bin',
|
||||
'/opt/homebrew/sbin',
|
||||
@@ -30,12 +31,6 @@ function pathEnvKey(env = process.env, platform = process.platform) {
|
||||
return Object.keys(env || {}).find(key => key.toUpperCase() === 'PATH') || 'PATH'
|
||||
}
|
||||
|
||||
function currentPathValue(env = process.env, platform = process.platform) {
|
||||
const key = pathEnvKey(env, platform)
|
||||
|
||||
return env?.[key] || ''
|
||||
}
|
||||
|
||||
function appendUniquePathEntries(entries, { delimiter = path.delimiter } = {}) {
|
||||
const seen = new Set()
|
||||
const ordered = []
|
||||
@@ -60,49 +55,6 @@ function appendUniquePathEntries(entries, { delimiter = path.delimiter } = {}) {
|
||||
return ordered.join(delimiter)
|
||||
}
|
||||
|
||||
/**
|
||||
* Hermes-managed Node.js directories, in preferred lookup order.
|
||||
*
|
||||
* There are two on-disk layouts. `scripts/install.ps1` unpacks portable Node
|
||||
* straight into `%LOCALAPPDATA%\hermes\node` (node.exe at the root, no `bin\`);
|
||||
* `scripts/install.sh` and the node-bootstrap helper use the POSIX
|
||||
* `$HERMES_HOME/node/bin`. Emit BOTH on every platform so mixed and migrated
|
||||
* installs resolve, leading with the layout native to the current platform.
|
||||
*
|
||||
* This is the single source of truth for the ordering rule on the Node side —
|
||||
* `main.ts` imports it rather than keeping its own copy. Mirrors
|
||||
* `iter_hermes_node_dirs()` in hermes_constants.py, which the Electron main
|
||||
* process cannot import.
|
||||
*/
|
||||
function hermesManagedNodePathEntries(
|
||||
hermesHome,
|
||||
{ platform = process.platform, pathModule = pathModuleForPlatform(platform) }: any = {}
|
||||
) {
|
||||
if (!hermesHome) {
|
||||
return []
|
||||
}
|
||||
|
||||
const root = pathModule.join(hermesHome, 'node')
|
||||
const bin = pathModule.join(root, 'bin')
|
||||
|
||||
return platform === 'win32' ? [root, bin] : [bin, root]
|
||||
}
|
||||
|
||||
function buildDesktopBackendPath({
|
||||
hermesHome,
|
||||
venvRoot,
|
||||
currentPath = '',
|
||||
platform = process.platform,
|
||||
pathModule = pathModuleForPlatform(platform)
|
||||
}: any = {}) {
|
||||
const delimiter = delimiterForPlatform(platform)
|
||||
const hermesNodeDirs = hermesManagedNodePathEntries(hermesHome, { platform, pathModule })
|
||||
const venvBin = venvRoot ? pathModule.join(venvRoot, platform === 'win32' ? 'Scripts' : 'bin') : null
|
||||
const saneEntries = platform === 'win32' ? [] : POSIX_SANE_PATH_ENTRIES
|
||||
|
||||
return appendUniquePathEntries([hermesNodeDirs, venvBin, currentPath, saneEntries], { delimiter })
|
||||
}
|
||||
|
||||
function normalizeHermesHomeRoot(hermesHome, { pathModule = pathModuleForPlatform(process.platform) }: any = {}) {
|
||||
if (!hermesHome) {
|
||||
return hermesHome
|
||||
@@ -118,20 +70,21 @@ function normalizeHermesHomeRoot(hermesHome, { pathModule = pathModuleForPlatfor
|
||||
return resolved
|
||||
}
|
||||
|
||||
function buildDesktopBackendEnv({
|
||||
hermesHome,
|
||||
pythonPathEntries = [],
|
||||
venvRoot,
|
||||
currentEnv = process.env,
|
||||
platform = process.platform,
|
||||
pathModule = pathModuleForPlatform(platform)
|
||||
}: any = {}) {
|
||||
/**
|
||||
* The environment for the spawned Python backend. Electron knows ONE thing:
|
||||
* where the interpreter is (by convention). Everything else — managed tool
|
||||
* PATHs, browser paths, node — is composed in-process by pm when the backend
|
||||
* spawns tools. PYTHONPATH/PYTHONHOME are scrubbed so an inherited value
|
||||
* can't make the backend import modules from another checkout.
|
||||
*/
|
||||
function buildDesktopBackendEnv({ currentEnv = process.env, platform = process.platform }: any = {}) {
|
||||
const delimiter = delimiterForPlatform(platform)
|
||||
const currentPythonPath = currentEnv?.PYTHONPATH || ''
|
||||
const key = pathEnvKey(currentEnv, platform)
|
||||
const saneEntries = platform === 'win32' ? [] : POSIX_SANE_PATH_ENTRIES
|
||||
|
||||
return {
|
||||
PYTHONPATH: appendUniquePathEntries([...pythonPathEntries, currentPythonPath], { delimiter }),
|
||||
PYTHONPATH: '',
|
||||
PYTHONHOME: '',
|
||||
// Force PEP 540 UTF-8 mode in the spawned Python backend so its stdio and
|
||||
// subprocess defaults are UTF-8 even on non-UTF-8 Windows locales (GBK,
|
||||
// cp1252, ...). hermes_bootstrap sets this inside the child too, but only
|
||||
@@ -139,22 +92,14 @@ function buildDesktopBackendEnv({
|
||||
// pre-bootstrap tracebacks) still decodes with the locale default without
|
||||
// this. User's explicit setting wins. Re-port of PR #56499 (echoriver89).
|
||||
PYTHONUTF8: currentEnv?.PYTHONUTF8 ?? '1',
|
||||
[key]: buildDesktopBackendPath({
|
||||
hermesHome,
|
||||
venvRoot,
|
||||
currentPath: currentPathValue(currentEnv, platform),
|
||||
platform,
|
||||
pathModule
|
||||
})
|
||||
[key]: appendUniquePathEntries([currentEnv?.[key] || '', saneEntries], { delimiter })
|
||||
}
|
||||
}
|
||||
|
||||
export {
|
||||
appendUniquePathEntries,
|
||||
buildDesktopBackendEnv,
|
||||
buildDesktopBackendPath,
|
||||
delimiterForPlatform,
|
||||
hermesManagedNodePathEntries,
|
||||
normalizeHermesHomeRoot,
|
||||
pathEnvKey,
|
||||
POSIX_SANE_PATH_ENTRIES
|
||||
|
||||
@@ -141,13 +141,14 @@ function hermesRuntimeImportProbe() {
|
||||
* @param {object} [opts.env] - Additional environment for the probe.
|
||||
* @returns {boolean}
|
||||
*/
|
||||
function canImportHermesCli(pythonPath: string, opts: { env?: Record<string, string> } = {}) {
|
||||
function canImportHermesCli(pythonPath: string, opts: { env?: Record<string, string>; cwd?: string } = {}) {
|
||||
if (!pythonPath) {
|
||||
return false
|
||||
}
|
||||
|
||||
try {
|
||||
execProbeSync(pythonPath, ['-c', hermesRuntimeImportProbe()], {
|
||||
cwd: opts.cwd,
|
||||
env: { ...process.env, ...(opts.env || {}) },
|
||||
stdio: 'ignore',
|
||||
timeout: PROBE_TIMEOUT_MS,
|
||||
|
||||
@@ -1,54 +0,0 @@
|
||||
import assert from 'node:assert/strict'
|
||||
|
||||
import { test } from 'vitest'
|
||||
|
||||
import { findGitBash } from './find-git-bash'
|
||||
|
||||
const yes = () => true
|
||||
const no = () => false
|
||||
|
||||
test('HERMES_GIT_BASH_PATH override takes precedence', () => {
|
||||
const result = findGitBash({
|
||||
isWindows: true,
|
||||
env: { HERMES_GIT_BASH_PATH: 'D:\\CustomGit\\bin\\bash.exe' },
|
||||
fileExists: yes,
|
||||
findOnPath: () => null
|
||||
})
|
||||
|
||||
assert.equal(result, 'D:\\CustomGit\\bin\\bash.exe')
|
||||
})
|
||||
|
||||
test('HERMES_GIT_BASH_PATH invalid path falls through to candidates', () => {
|
||||
const env = {
|
||||
HERMES_GIT_BASH_PATH: 'X:\\Missing\\bash.exe',
|
||||
LOCALAPPDATA: 'C:\\Users\\test\\AppData\\Local',
|
||||
ProgramFiles: 'C:\\Program Files',
|
||||
'ProgramFiles(x86)': 'C:\\Program Files (x86)'
|
||||
}
|
||||
|
||||
const fileExists = (p: string) => p !== 'X:\\Missing\\bash.exe' && p.includes('Program Files\\Git\\bin\\bash.exe')
|
||||
const result = findGitBash({ isWindows: true, env, fileExists, findOnPath: () => null })
|
||||
assert.equal(result, 'C:\\Program Files\\Git\\bin\\bash.exe')
|
||||
})
|
||||
|
||||
test('HERMES_GIT_BASH_PATH empty string is ignored', () => {
|
||||
const result = findGitBash({
|
||||
isWindows: true,
|
||||
env: { HERMES_GIT_BASH_PATH: '', LOCALAPPDATA: '' },
|
||||
fileExists: no,
|
||||
findOnPath: () => 'C:\\msys64\\usr\\bin\\bash.exe'
|
||||
})
|
||||
|
||||
assert.equal(result, 'C:\\msys64\\usr\\bin\\bash.exe')
|
||||
})
|
||||
|
||||
test('non-Windows uses findOnPath', () => {
|
||||
const result = findGitBash({
|
||||
isWindows: false,
|
||||
env: {},
|
||||
fileExists: no,
|
||||
findOnPath: () => '/usr/bin/bash'
|
||||
})
|
||||
|
||||
assert.equal(result, '/usr/bin/bash')
|
||||
})
|
||||
@@ -1,67 +0,0 @@
|
||||
import path from 'node:path'
|
||||
|
||||
export interface GitBashOptions {
|
||||
isWindows: boolean
|
||||
env: Record<string, string | undefined>
|
||||
fileExists: (filePath: string) => boolean
|
||||
findOnPath?: (command: string) => string | null
|
||||
}
|
||||
|
||||
/**
|
||||
* Locate bash.exe on Windows.
|
||||
* Resolution order (first match wins):
|
||||
* 1. HERMES_GIT_BASH_PATH env var override
|
||||
* 2. PortableGit under %LOCALAPPDATA%\hermes\git\ (install.ps1)
|
||||
* 3. Standard Git for Windows install locations
|
||||
* 4. %LOCALAPPDATA%\Programs\Git\ (user-scoped)
|
||||
* 5. bash on PATH
|
||||
*/
|
||||
export function findGitBash(opts: GitBashOptions): string | null {
|
||||
const { isWindows, env, fileExists, findOnPath } = opts
|
||||
|
||||
if (!isWindows) {
|
||||
return findOnPath ? findOnPath('bash') : null
|
||||
}
|
||||
|
||||
// Respect HERMES_GIT_BASH_PATH if set (mirrors tools/environments/local.py:_find_bash).
|
||||
const gitBashPath = env.HERMES_GIT_BASH_PATH
|
||||
|
||||
if (gitBashPath && fileExists(gitBashPath)) {
|
||||
return gitBashPath
|
||||
}
|
||||
|
||||
const localAppData = env.LOCALAPPDATA || ''
|
||||
const candidates: string[] = []
|
||||
|
||||
// Candidate paths are Windows paths regardless of host platform (tests run
|
||||
// on POSIX CI hosts too), so join with win32 semantics explicitly.
|
||||
const joinWin = path.win32.join
|
||||
|
||||
if (localAppData) {
|
||||
candidates.push(joinWin(localAppData, 'hermes', 'git', 'bin', 'bash.exe'))
|
||||
candidates.push(joinWin(localAppData, 'hermes', 'git', 'usr', 'bin', 'bash.exe'))
|
||||
}
|
||||
|
||||
candidates.push(joinWin(env['ProgramFiles'] || 'C:\\Program Files', 'Git', 'bin', 'bash.exe'))
|
||||
candidates.push(joinWin(env['ProgramFiles(x86)'] || 'C:\\Program Files (x86)', 'Git', 'bin', 'bash.exe'))
|
||||
|
||||
if (localAppData) {
|
||||
candidates.push(joinWin(localAppData, 'Programs', 'Git', 'bin', 'bash.exe'))
|
||||
}
|
||||
|
||||
for (const candidate of candidates) {
|
||||
if (fileExists(candidate)) {
|
||||
return candidate
|
||||
}
|
||||
}
|
||||
|
||||
if (findOnPath) {
|
||||
const onPath = findOnPath('bash')
|
||||
|
||||
if (onPath) {
|
||||
return onPath
|
||||
}
|
||||
}
|
||||
|
||||
return null
|
||||
}
|
||||
+737
-276
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,196 @@
|
||||
import assert from 'node:assert/strict'
|
||||
import fs from 'node:fs'
|
||||
import os from 'node:os'
|
||||
import path from 'node:path'
|
||||
|
||||
import { test } from 'vitest'
|
||||
|
||||
import { createHash } from 'node:crypto'
|
||||
|
||||
import { adoptPayloadVenv, installIdForRoot, isBundledInstall, resolvePayload } from './payload-backend'
|
||||
|
||||
function tmpdir() {
|
||||
return fs.mkdtempSync(path.join(os.tmpdir(), 'payload-test-'))
|
||||
}
|
||||
|
||||
function writePayload(
|
||||
root: string,
|
||||
{
|
||||
manifest = { schema: 1, target: 'win32-x64', repo: 'hermes-agent', venv: 'venv', store: 'tools' },
|
||||
isWindows = true
|
||||
}: any = {}
|
||||
) {
|
||||
const dir = path.join(root, 'agent-payload')
|
||||
|
||||
fs.mkdirSync(path.join(dir, 'hermes-agent'), { recursive: true })
|
||||
fs.mkdirSync(path.join(dir, 'tools', 'python-3.11.16-win32-x64'), { recursive: true })
|
||||
|
||||
// Store python (win: python.exe at the entry root; posix: bin/python3).
|
||||
if (isWindows) {
|
||||
fs.writeFileSync(path.join(dir, 'tools', 'python-3.11.16-win32-x64', 'python.exe'), '')
|
||||
} else {
|
||||
fs.mkdirSync(path.join(dir, 'tools', 'python-3.11.16-win32-x64', 'bin'), { recursive: true })
|
||||
fs.writeFileSync(path.join(dir, 'tools', 'python-3.11.16-win32-x64', 'bin', 'python3'), '')
|
||||
}
|
||||
|
||||
// Venv site-packages (where the project deps are installed).
|
||||
const sp = isWindows
|
||||
? path.join(dir, 'venv', 'Lib', 'site-packages')
|
||||
: path.join(dir, 'venv', 'lib', 'python3.11', 'site-packages')
|
||||
fs.mkdirSync(sp, { recursive: true })
|
||||
|
||||
// The self-relative CLI entrypoint (minted launcher exe on win32, bash
|
||||
// trampoline on POSIX) — the bundled entry point.
|
||||
const binDir = path.join(dir, 'bin')
|
||||
fs.mkdirSync(binDir, { recursive: true })
|
||||
fs.writeFileSync(path.join(binDir, isWindows ? 'hermes.exe' : 'hermes'), '')
|
||||
|
||||
fs.writeFileSync(path.join(dir, 'manifest.json'), JSON.stringify(manifest))
|
||||
fs.writeFileSync(
|
||||
path.join(dir, 'tools', 'facts.json'),
|
||||
JSON.stringify({ packages: { python: { entry: 'python-3.11.16-win32-x64', version: '3.11.16' } } })
|
||||
)
|
||||
|
||||
return dir
|
||||
}
|
||||
|
||||
const fsDeps = {
|
||||
fileExists: (p: string) => {
|
||||
try {
|
||||
return fs.statSync(p).isFile()
|
||||
} catch {
|
||||
return false
|
||||
}
|
||||
},
|
||||
directoryExists: (p: string) => {
|
||||
try {
|
||||
return fs.statSync(p).isDirectory()
|
||||
} catch {
|
||||
return false
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
test('resolvePayload finds a complete payload (store python + venv site-packages)', () => {
|
||||
const root = tmpdir()
|
||||
|
||||
writePayload(root)
|
||||
|
||||
const payload = resolvePayload(root, { ...fsDeps, isWindows: true })
|
||||
|
||||
assert.ok(payload)
|
||||
assert.equal(payload.repoDir, path.join(root, 'agent-payload', 'hermes-agent'))
|
||||
assert.ok(payload.storePython.endsWith(path.join('tools', 'python-3.11.16-win32-x64', 'python.exe')))
|
||||
assert.ok(payload.sitePackages.endsWith(path.join('venv', 'Lib', 'site-packages')))
|
||||
assert.ok(payload.shim.endsWith(path.join('bin', 'hermes.exe')))
|
||||
})
|
||||
|
||||
test('resolvePayload resolves the posix store python + site-packages layout', () => {
|
||||
const root = tmpdir()
|
||||
|
||||
writePayload(root, { isWindows: false })
|
||||
|
||||
const payload = resolvePayload(root, { ...fsDeps, isWindows: false })
|
||||
|
||||
assert.ok(payload)
|
||||
assert.ok(payload.storePython.endsWith(path.join('tools', 'python-3.11.16-win32-x64', 'bin', 'python3')))
|
||||
assert.ok(payload.sitePackages.endsWith(path.join('venv', 'lib', 'python3.11', 'site-packages')))
|
||||
assert.ok(payload.shim.endsWith(path.join('bin', 'hermes')))
|
||||
})
|
||||
|
||||
test('resolvePayload returns null without a manifest, for external stubs, and for a broken payload', () => {
|
||||
assert.equal(resolvePayload(tmpdir(), { ...fsDeps, isWindows: true }), null)
|
||||
assert.equal(resolvePayload(undefined, { ...fsDeps, isWindows: true }), null)
|
||||
|
||||
const externalRoot = tmpdir()
|
||||
|
||||
writePayload(externalRoot, { manifest: { schema: 1, external: true } })
|
||||
assert.equal(resolvePayload(externalRoot, { ...fsDeps, isWindows: true }), null)
|
||||
|
||||
const brokenRoot = tmpdir()
|
||||
const dir = writePayload(brokenRoot)
|
||||
|
||||
fs.rmSync(path.join(dir, 'venv'), { recursive: true })
|
||||
assert.equal(resolvePayload(brokenRoot, { ...fsDeps, isWindows: true }), null)
|
||||
})
|
||||
|
||||
test('adoptPayloadVenv verifies store python + site-packages without any cfg write', () => {
|
||||
const root = tmpdir()
|
||||
const dir = writePayload(root)
|
||||
const payload = resolvePayload(root, { ...fsDeps, isWindows: true })
|
||||
|
||||
assert.ok(payload)
|
||||
// Bundled builds run the store python directly — there is NO pyvenv.cfg
|
||||
// write (the payload may be read-only, e.g. MSIX). The cfg, if present,
|
||||
// is left untouched.
|
||||
fs.writeFileSync(path.join(dir, 'venv', 'pyvenv.cfg'), 'home = C:\\ci\\build\\python\nversion_info = 3.11.16\n')
|
||||
assert.equal(adoptPayloadVenv(payload, { isWindows: true }), true)
|
||||
const text = fs.readFileSync(path.join(dir, 'venv', 'pyvenv.cfg'), 'utf8')
|
||||
assert.ok(text.includes('C:\\ci\\build'), 'the shipped cfg is not rewritten')
|
||||
|
||||
// second call: still reports usable
|
||||
assert.equal(adoptPayloadVenv(payload, { isWindows: true }), true)
|
||||
})
|
||||
|
||||
// adoptPayloadVenv is now a pure presence-verify; resolvePayload's own
|
||||
// existence checks already reject a payload missing the store python or
|
||||
// site-packages (covered above), so there is no separate fail-closed path
|
||||
// to assert at this layer.
|
||||
|
||||
test('isBundledInstall is true for a real payload manifest', () => {
|
||||
const root = tmpdir()
|
||||
|
||||
writePayload(root)
|
||||
|
||||
assert.equal(isBundledInstall(root, fsDeps), true)
|
||||
})
|
||||
|
||||
test('isBundledInstall is false for the external stub and for missing manifests', () => {
|
||||
const externalRoot = tmpdir()
|
||||
|
||||
writePayload(externalRoot, { manifest: { schema: 1, external: true } })
|
||||
assert.equal(isBundledInstall(externalRoot, fsDeps), false)
|
||||
|
||||
assert.equal(isBundledInstall(tmpdir(), fsDeps), false)
|
||||
assert.equal(isBundledInstall(undefined, fsDeps), false)
|
||||
})
|
||||
|
||||
test('isBundledInstall is true even when the payload is damaged (never-install guard)', () => {
|
||||
const root = tmpdir()
|
||||
const dir = writePayload(root)
|
||||
|
||||
// A broken payload must still read as bundled: isBundledInstall is the
|
||||
// guard that REFUSES the installer, and a damaged bundle is exactly the
|
||||
// case where the installer must not run.
|
||||
fs.rmSync(path.join(dir, 'venv'), { recursive: true })
|
||||
fs.rmSync(path.join(dir, 'tools', 'python-3.11.16-win32-x64'), { recursive: true })
|
||||
|
||||
assert.equal(resolvePayload(root, { ...fsDeps, isWindows: true }), null)
|
||||
assert.equal(isBundledInstall(root, fsDeps), true)
|
||||
})
|
||||
|
||||
test('isBundledInstall is false for a malformed manifest', () => {
|
||||
const root = tmpdir()
|
||||
const dir = path.join(root, 'agent-payload')
|
||||
|
||||
fs.mkdirSync(dir, { recursive: true })
|
||||
fs.writeFileSync(path.join(dir, 'manifest.json'), 'not json {')
|
||||
|
||||
assert.equal(isBundledInstall(root, fsDeps), false)
|
||||
})
|
||||
|
||||
// ─── update channel helpers ─────────────────────────────────────────
|
||||
|
||||
test('installIdForRoot matches the Python install id (sha16 of the canonical path)', () => {
|
||||
// sha256('/home/u/.hermes/hermes-agent')[:16] — recomputed independently.
|
||||
assert.equal(
|
||||
installIdForRoot('/home/u/.hermes/hermes-agent'),
|
||||
createHash('sha256').update('/home/u/.hermes/hermes-agent', 'utf8').digest('hex').slice(0, 16)
|
||||
)
|
||||
// The canonicalizer output is what gets hashed (symlinked homes).
|
||||
assert.equal(
|
||||
installIdForRoot('/link/hermes-agent', () => '/real/hermes-agent'),
|
||||
installIdForRoot('/real/hermes-agent')
|
||||
)
|
||||
})
|
||||
|
||||
@@ -0,0 +1,172 @@
|
||||
/**
|
||||
* payload-backend.ts
|
||||
*
|
||||
* The bundled-install backend: a pm payload staged by `hermes pm bundle`
|
||||
* and shipped under resources/agent-payload. The payload carries the repo
|
||||
* snapshot, the tool store (with facts.json), and a relocatable venv
|
||||
* built on the staged python-build-standalone interpreter.
|
||||
*
|
||||
* Electron's whole job here is finding the interpreter and verifying the
|
||||
* payload can boot. Bundled builds run the store python directly —
|
||||
* self-relative, no pyvenv.cfg write, so read-only installs (MSIX) work.
|
||||
* Everything else — managed tool PATHs, env composition — happens
|
||||
* in-process via pm when the backend runs.
|
||||
*/
|
||||
|
||||
import { createHash } from 'node:crypto'
|
||||
import fs from 'node:fs'
|
||||
import path from 'node:path'
|
||||
|
||||
export interface PayloadInfo {
|
||||
root: string
|
||||
repoDir: string
|
||||
toolsDir: string
|
||||
/** The payload's own CPython (tools/<python-entry>/python(.exe)). */
|
||||
storePython: string
|
||||
/** The venv's site-packages (Lib/site-packages on win, lib/python3.11/site-packages on posix). */
|
||||
sitePackages: string
|
||||
/** The self-relative CLI trampoline (bin/hermes(.exe)) — the bundled entry point. */
|
||||
shim: string
|
||||
}
|
||||
|
||||
export function resolvePayload(
|
||||
resourcesPath: string | undefined,
|
||||
deps: {
|
||||
fileExists: (p: string) => boolean
|
||||
directoryExists: (p: string) => boolean
|
||||
isWindows: boolean
|
||||
}
|
||||
): PayloadInfo | null {
|
||||
if (!resourcesPath) {
|
||||
return null
|
||||
}
|
||||
|
||||
const root = path.join(resourcesPath, 'agent-payload')
|
||||
const manifestPath = path.join(root, 'manifest.json')
|
||||
|
||||
if (!deps.fileExists(manifestPath)) {
|
||||
return null
|
||||
}
|
||||
|
||||
let manifest: any
|
||||
|
||||
try {
|
||||
manifest = JSON.parse(fs.readFileSync(manifestPath, 'utf8'))
|
||||
} catch {
|
||||
return null
|
||||
}
|
||||
|
||||
if (!manifest || manifest.external === true) {
|
||||
return null
|
||||
}
|
||||
|
||||
// The manifest names the payload layout; a bundle that omits the fields
|
||||
// is malformed — refuse it rather than guess at cmd_bundle's layout.
|
||||
if (typeof manifest.repo !== 'string' || typeof manifest.store !== 'string' || typeof manifest.venv !== 'string') {
|
||||
return null
|
||||
}
|
||||
|
||||
const repoDir = path.join(root, manifest.repo)
|
||||
const toolsDir = path.join(root, manifest.store)
|
||||
const venvDir = path.join(root, manifest.venv)
|
||||
// The CLI trampoline staged into bin/ (hermes/hermes-agent/hermes-acp —
|
||||
// build-bundled-desktop.mjs 5b: distlib-minted launchers on win32,
|
||||
// $0-relative bash trampolines on POSIX). It execs the store python with
|
||||
// the payload's own PYTHONPATH, so it is the single bundled entry point.
|
||||
const shim = path.join(root, 'bin', deps.isWindows ? 'hermes.exe' : 'hermes')
|
||||
|
||||
// The store CPython + the venv's site-packages. Bundled builds run the
|
||||
// STORE python (self-relative, no pyvenv.cfg write — works on read-only
|
||||
// MSIX) with PYTHONPATH pointing at the venv site-packages (where the
|
||||
// project deps are installed). The venv python itself is NOT used in
|
||||
// bundled builds.
|
||||
let storePython = ''
|
||||
let sitePackages = ''
|
||||
try {
|
||||
const facts = JSON.parse(fs.readFileSync(path.join(toolsDir, 'facts.json'), 'utf8'))
|
||||
const entry = facts?.packages?.python?.entry
|
||||
if (typeof entry === 'string') {
|
||||
storePython = path.join(toolsDir, entry, deps.isWindows ? 'python.exe' : 'bin', deps.isWindows ? '' : 'python3')
|
||||
sitePackages = deps.isWindows
|
||||
? path.join(venvDir, 'Lib', 'site-packages')
|
||||
: path.join(venvDir, 'lib', `python${process.env.PYTHON_VER || '3.11'}`, 'site-packages')
|
||||
}
|
||||
} catch {
|
||||
// fall through to the existence checks below
|
||||
}
|
||||
|
||||
if (!deps.directoryExists(repoDir) || !deps.fileExists(storePython) || !deps.directoryExists(sitePackages) || !deps.fileExists(shim)) {
|
||||
return null
|
||||
}
|
||||
|
||||
return { root, repoDir, toolsDir, storePython, sitePackages, shim }
|
||||
}
|
||||
|
||||
/**
|
||||
* "Is this artifact a bundled install?" — the app ships its own Hermes payload.
|
||||
* True whenever resources/agent-payload/manifest.json exists and is not the
|
||||
* external stub (before-build.mjs writes {schema:1, external:true} for
|
||||
* non-bundled builds). Deliberately does NOT verify payload usability: a
|
||||
* damaged bundle still must never install — callers use this to refuse the
|
||||
* installer, not to decide the payload can boot.
|
||||
*/
|
||||
export function isBundledInstall(
|
||||
resourcesPath: string | undefined,
|
||||
deps: { fileExists: (p: string) => boolean }
|
||||
): boolean {
|
||||
if (!resourcesPath) {
|
||||
return false
|
||||
}
|
||||
|
||||
const manifestPath = path.join(resourcesPath, 'agent-payload', 'manifest.json')
|
||||
|
||||
if (!deps.fileExists(manifestPath)) {
|
||||
return false
|
||||
}
|
||||
|
||||
try {
|
||||
const manifest = JSON.parse(fs.readFileSync(manifestPath, 'utf8'))
|
||||
|
||||
return Boolean(manifest) && manifest.external !== true
|
||||
} catch {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Verify the payload is usable — the store python + venv site-packages
|
||||
* resolve. NO pyvenv.cfg write: bundled builds run the store python
|
||||
* directly (self-relative, works on read-only MSIX), so there is nothing
|
||||
* to re-point. Returns true when the payload can boot.
|
||||
*/
|
||||
export function adoptPayloadVenv(
|
||||
payload: PayloadInfo,
|
||||
deps: { isWindows: boolean; log?: (m: string) => void }
|
||||
): boolean {
|
||||
if (!payload.storePython || !payload.sitePackages) {
|
||||
deps.log?.('[payload] missing store python or site-packages')
|
||||
return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// ─── update channel ─────────────────────────────────────────────────────────
|
||||
//
|
||||
// The CLI owns the channel records; Electron only reads the install id for
|
||||
// `update.installs.<sha16>/` bookkeeping. Channel resolution itself lives in
|
||||
// hermes_cli/update_channel.py — main.ts keys nightly/stable off the baked
|
||||
// install stamp tag directly.
|
||||
|
||||
export type UpdateChannel = 'stable' | 'main' | 'nightly'
|
||||
|
||||
/**
|
||||
* The install id of the tree at `root`: sha16 of the canonical PATH,
|
||||
* byte-identical to Python's install id (sha256 of the resolved root,
|
||||
* first 16 hex chars — `boot_bootstrap._install_key` /
|
||||
* `update_channel._install_key_sha16`). Path-derived so it survives
|
||||
* artifact replacement at the same location; the same key names
|
||||
* `installs/<sha16>/`.
|
||||
*/
|
||||
export function installIdForRoot(root: string, canonicalize: (p: string) => string = p => p): string {
|
||||
return createHash('sha256').update(canonicalize(root), 'utf8').digest('hex').slice(0, 16)
|
||||
}
|
||||
@@ -0,0 +1,66 @@
|
||||
/**
|
||||
* Tests for electron/update-root-policy.ts — the pure classifier that decides
|
||||
* whether the desktop's git-based self-update may run against a resolved
|
||||
* update root. A checkout the install contract does not manage (steward-owned
|
||||
* `external`/`electron-updater` mechanisms) must be refused with a user-action
|
||||
* pointer instead of the desktop pulling into it.
|
||||
*/
|
||||
|
||||
import assert from 'node:assert/strict'
|
||||
|
||||
import { test } from 'vitest'
|
||||
|
||||
import { classifyUpdateRoot } from './update-root-policy'
|
||||
|
||||
test('a non-git root is not updatable and carries no user advice', () => {
|
||||
const result = classifyUpdateRoot({ isGitTree: false, updateMechanism: null })
|
||||
|
||||
assert.equal(result.updatable, false)
|
||||
assert.equal(result.verdict, 'not-a-checkout')
|
||||
assert.equal(result.provenance, 'not-a-checkout')
|
||||
assert.equal(result.advice, null)
|
||||
assert.ok(result.message)
|
||||
})
|
||||
|
||||
test('a self-managed checkout is updatable', () => {
|
||||
const result = classifyUpdateRoot({ isGitTree: true, updateMechanism: 'self' })
|
||||
|
||||
assert.equal(result.updatable, true)
|
||||
assert.equal(result.verdict, 'updatable')
|
||||
assert.equal(result.provenance, 'managed-self')
|
||||
assert.equal(result.advice, null)
|
||||
assert.equal(result.message, null)
|
||||
})
|
||||
|
||||
test('a checkout owned by the external steward is refused with a git pull pointer', () => {
|
||||
const result = classifyUpdateRoot({ isGitTree: true, updateMechanism: 'external' })
|
||||
|
||||
assert.equal(result.updatable, false)
|
||||
assert.equal(result.verdict, 'steward-owned-git-tree')
|
||||
assert.equal(result.provenance, 'steward-owned')
|
||||
assert.equal(result.advice, 'git pull')
|
||||
assert.ok(result.message?.includes('external'))
|
||||
})
|
||||
|
||||
test('a checkout owned by electron-updater is refused the same way', () => {
|
||||
const result = classifyUpdateRoot({ isGitTree: true, updateMechanism: 'electron-updater' })
|
||||
|
||||
assert.equal(result.updatable, false)
|
||||
assert.equal(result.verdict, 'steward-owned-git-tree')
|
||||
assert.equal(result.advice, 'git pull')
|
||||
assert.ok(result.message?.includes('electron-updater'))
|
||||
})
|
||||
|
||||
test('an unstamped git tree (dev checkout) stays updatable with unknown provenance', () => {
|
||||
const result = classifyUpdateRoot({ isGitTree: true, updateMechanism: null })
|
||||
|
||||
assert.equal(result.updatable, true)
|
||||
assert.equal(result.verdict, 'updatable')
|
||||
assert.equal(result.provenance, 'unknown')
|
||||
assert.equal(result.advice, null)
|
||||
})
|
||||
|
||||
test('the classification is a pure function of its inputs', () => {
|
||||
const facts = { isGitTree: true, updateMechanism: 'external' as const }
|
||||
assert.deepEqual(classifyUpdateRoot(facts), classifyUpdateRoot({ ...facts }))
|
||||
})
|
||||
@@ -0,0 +1,94 @@
|
||||
// update-root-policy.ts — pure classifier for the desktop self-update root.
|
||||
//
|
||||
// The desktop's git-based self-update arm runs `git fetch/merge` against a
|
||||
// checkout it resolved (resolveUpdateRoot in main.ts). That root is only
|
||||
// legitimate update territory when the install contract says so:
|
||||
//
|
||||
// - Not a `.git` tree → there is nothing to pull; the desktop cannot
|
||||
// self-update here (bundled installs use the OS App Installer arm and
|
||||
// never reach this classifier).
|
||||
// - A `.git` tree whose install stamp says `updateMechanism: "self"` → the
|
||||
// install is a managed self-updating checkout (e.g. the desktop
|
||||
// bootstrap's clone, which writes exactly that stamp) — updatable.
|
||||
// - A `.git` tree with any OTHER mechanism (`external`: the store/steward
|
||||
// owns updates; `electron-updater`: the desktop package does) → the tree
|
||||
// is managed by someone else. Pulling into it from the desktop would
|
||||
// stash-and-move a user's checkout out from under its steward.
|
||||
// - No stamp at all (null mechanism; a dev source checkout) → provenance
|
||||
// unknown. The classifier reports `updatable` with `provenance:
|
||||
// 'unknown'`: a developer's working tree keeps its existing update flow,
|
||||
// and callers log the ambiguity rather than silently widening the refusal.
|
||||
//
|
||||
// Pure and dependency-injected (same shape as update-gate.ts) so the policy
|
||||
// is unit-testable without booting Electron, and every caller gets the same
|
||||
// answer from one authority.
|
||||
|
||||
export type UpdateMechanism = 'self' | 'electron-updater' | 'external'
|
||||
|
||||
export type UpdateRootProvenance = 'managed-self' | 'steward-owned' | 'unknown' | 'not-a-checkout'
|
||||
|
||||
export interface UpdateRootClassification {
|
||||
/** Whether the desktop's git-based self-update may run against this root. */
|
||||
updatable: boolean
|
||||
/** Machine-readable verdict for update-check results and logs. */
|
||||
verdict: 'updatable' | 'not-a-checkout' | 'steward-owned-git-tree' | 'unmanaged-git-tree'
|
||||
/** Why the root is (or is not) updatable, for user-facing messages. */
|
||||
message: string | null
|
||||
/** The command that fixes it, when the user (not the app) must act. */
|
||||
advice: 'git pull' | null
|
||||
provenance: UpdateRootProvenance
|
||||
}
|
||||
|
||||
export interface UpdateRootFacts {
|
||||
/** True when the resolved update root contains a `.git` entry. */
|
||||
isGitTree: boolean
|
||||
/** The install stamp's updateMechanism, or null when there is no stamp. */
|
||||
updateMechanism: UpdateMechanism | null
|
||||
}
|
||||
|
||||
/**
|
||||
* Classify whether the desktop may self-update (git pull semantics) against
|
||||
* the resolved update root.
|
||||
*/
|
||||
export function classifyUpdateRoot(facts: UpdateRootFacts): UpdateRootClassification {
|
||||
if (!facts.isGitTree) {
|
||||
return {
|
||||
updatable: false,
|
||||
verdict: 'not-a-checkout',
|
||||
message: 'This install has no git checkout to update — the app or its steward owns the update loop.',
|
||||
advice: null,
|
||||
provenance: 'not-a-checkout'
|
||||
}
|
||||
}
|
||||
|
||||
if (facts.updateMechanism === 'self') {
|
||||
return {
|
||||
updatable: true,
|
||||
verdict: 'updatable',
|
||||
message: null,
|
||||
advice: null,
|
||||
provenance: 'managed-self'
|
||||
}
|
||||
}
|
||||
|
||||
if (facts.updateMechanism === 'external' || facts.updateMechanism === 'electron-updater') {
|
||||
return {
|
||||
updatable: false,
|
||||
verdict: 'steward-owned-git-tree',
|
||||
message:
|
||||
`This checkout is managed by its install method (${facts.updateMechanism}); ` +
|
||||
'the desktop will not run git updates against it.',
|
||||
advice: 'git pull',
|
||||
provenance: 'steward-owned'
|
||||
}
|
||||
}
|
||||
|
||||
// No stamp: unknown provenance (typically a developer source checkout).
|
||||
return {
|
||||
updatable: true,
|
||||
verdict: 'updatable',
|
||||
message: null,
|
||||
advice: null,
|
||||
provenance: 'unknown'
|
||||
}
|
||||
}
|
||||
@@ -13,16 +13,10 @@
|
||||
// re-selected forever instead of falling through to bootstrap.
|
||||
|
||||
import assert from 'node:assert/strict'
|
||||
import path from 'node:path'
|
||||
|
||||
import { test } from 'vitest'
|
||||
|
||||
import {
|
||||
buildPathExtCandidates,
|
||||
chooseUpdaterArgs,
|
||||
getVenvSitePackagesEntries,
|
||||
resolveVenvHermesCommand
|
||||
} from './windows-hermes-path'
|
||||
import { buildPathExtCandidates, chooseUpdaterArgs, resolveVenvHermesCommand } from './windows-hermes-path'
|
||||
|
||||
test('buildPathExtCandidates: Windows tries PATHEXT extensions before the empty extension', () => {
|
||||
const extensions = buildPathExtCandidates('.COM;.EXE;.BAT;.CMD', true)
|
||||
@@ -92,9 +86,7 @@ function makeDeps(overrides: Partial<Parameters<typeof resolveVenvHermesCommand>
|
||||
directoryExists: () => false,
|
||||
canImportHermesCli: () => true,
|
||||
getVenvPython: (venvRoot: string) => `${venvRoot}/Scripts/python.exe`,
|
||||
getVenvSitePackagesEntries: () => [],
|
||||
buildDesktopBackendEnv: () => ({ FAKE_ENV: '1' }),
|
||||
hermesHome: '/fake/hermes-home',
|
||||
resolvePath: (...segments: string[]) => segments.join('/').replace(/\/+/g, '/'),
|
||||
dirname: (p: string) => p.slice(0, p.lastIndexOf('/')) || '/',
|
||||
basename: (p: string) => p.slice(p.lastIndexOf('/') + 1),
|
||||
@@ -170,71 +162,3 @@ test('resolveVenvHermesCommand: is case-insensitive on hermes.exe and the Script
|
||||
assert.ok(resolveVenvHermesCommand('/root/venv/Scripts/HERMES.EXE', [], deps))
|
||||
assert.ok(resolveVenvHermesCommand('/root/venv/SCRIPTS/hermes.exe', [], deps))
|
||||
})
|
||||
|
||||
// ── getVenvSitePackagesEntries ─────────────────────────────────────────────
|
||||
|
||||
test('getVenvSitePackagesEntries: returns Lib/site-packages on Windows when it exists', () => {
|
||||
const expected = path.join('C:\\venv', 'Lib', 'site-packages')
|
||||
|
||||
const result = getVenvSitePackagesEntries('C:\\venv', {
|
||||
isWindows: true,
|
||||
directoryExists: p => p === expected
|
||||
})
|
||||
|
||||
assert.deepEqual(result, [expected])
|
||||
})
|
||||
|
||||
test('getVenvSitePackagesEntries: returns empty on Windows when site-packages does not exist', () => {
|
||||
const result = getVenvSitePackagesEntries('C:\\venv', {
|
||||
isWindows: true,
|
||||
directoryExists: () => false
|
||||
})
|
||||
|
||||
assert.deepEqual(result, [])
|
||||
})
|
||||
|
||||
test('getVenvSitePackagesEntries: reads pyvenv.cfg version on POSIX and resolves lib/pythonX.Y/site-packages', () => {
|
||||
const result = getVenvSitePackagesEntries('/venv', {
|
||||
isWindows: false,
|
||||
directoryExists: p => p === '/venv/lib/python3.12/site-packages',
|
||||
readFile: () => 'version_info = 3.12.1\n'
|
||||
})
|
||||
|
||||
assert.deepEqual(result, ['/venv/lib/python3.12/site-packages'])
|
||||
})
|
||||
|
||||
test('getVenvSitePackagesEntries: returns empty on POSIX when pyvenv.cfg is missing', () => {
|
||||
const result = getVenvSitePackagesEntries('/venv', {
|
||||
isWindows: false,
|
||||
directoryExists: () => true,
|
||||
readFile: () => undefined
|
||||
})
|
||||
|
||||
assert.deepEqual(result, [])
|
||||
})
|
||||
|
||||
test('getVenvSitePackagesEntries: returns empty on POSIX when pyvenv.cfg has no version_info', () => {
|
||||
const result = getVenvSitePackagesEntries('/venv', {
|
||||
isWindows: false,
|
||||
directoryExists: () => true,
|
||||
readFile: () => 'home = /usr/bin\n'
|
||||
})
|
||||
|
||||
assert.deepEqual(result, [])
|
||||
})
|
||||
|
||||
test('getVenvSitePackagesEntries: returns empty on POSIX when version is present but site-packages dir is absent', () => {
|
||||
const result = getVenvSitePackagesEntries('/venv', {
|
||||
isWindows: false,
|
||||
directoryExists: () => false,
|
||||
readFile: () => 'version_info = 3.11\n'
|
||||
})
|
||||
|
||||
assert.deepEqual(result, [])
|
||||
})
|
||||
|
||||
test('getVenvSitePackagesEntries: returns empty for a falsy venvRoot', () => {
|
||||
assert.deepEqual(getVenvSitePackagesEntries('', { isWindows: true, directoryExists: () => true }), [])
|
||||
assert.deepEqual(getVenvSitePackagesEntries(null, { isWindows: true, directoryExists: () => true }), [])
|
||||
assert.deepEqual(getVenvSitePackagesEntries(undefined, { isWindows: true, directoryExists: () => true }), [])
|
||||
})
|
||||
@@ -29,9 +29,6 @@
|
||||
* backend-command.ts.
|
||||
*/
|
||||
|
||||
import fs from 'node:fs'
|
||||
import path from 'node:path'
|
||||
|
||||
/**
|
||||
* Build the ordered list of extensions findOnPath() should try when
|
||||
* resolving a bare command name off PATH.
|
||||
@@ -82,104 +79,14 @@ export function chooseUpdaterArgs(signals: BootstrapRecoverySignals, branch: str
|
||||
return canRunUpdater ? ['--update', '--branch', branch] : ['--repair', '--branch', branch]
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the site-packages directory entries for a Python venv.
|
||||
*
|
||||
* On Windows, venv layout is `<venvRoot>/Lib/site-packages`.
|
||||
* On POSIX, it's `<venvRoot>/lib/python<version>/site-packages` where
|
||||
* `<version>` (e.g. `3.12`) is read from the venv's `pyvenv.cfg`
|
||||
* `version_info` field.
|
||||
*
|
||||
* Returns only directories that actually exist on disk. Returns an empty
|
||||
* array when `venvRoot` is falsy or no matching site-packages dir is found.
|
||||
*
|
||||
* Extracted from main.ts so the platform branching can be tested without
|
||||
* reading source text. `isWindows` and `directoryExists` are injectable;
|
||||
* `readFile` defaults to `fs.readFileSync` but can be overridden for tests.
|
||||
*/
|
||||
export function getVenvSitePackagesEntries(
|
||||
venvRoot: string | undefined | null,
|
||||
opts: {
|
||||
isWindows?: boolean
|
||||
directoryExists?: (p: string) => boolean
|
||||
readFile?: (p: string) => string | undefined
|
||||
} = {}
|
||||
): string[] {
|
||||
const entries: string[] = []
|
||||
|
||||
if (!venvRoot) {
|
||||
return entries
|
||||
}
|
||||
|
||||
const isWindows = opts.isWindows ?? process.platform === 'win32'
|
||||
|
||||
const directoryExists =
|
||||
opts.directoryExists ??
|
||||
((p: string) => {
|
||||
try {
|
||||
return fs.statSync(p).isDirectory()
|
||||
} catch {
|
||||
return false
|
||||
}
|
||||
})
|
||||
|
||||
const readFile =
|
||||
opts.readFile ??
|
||||
((p: string) => {
|
||||
try {
|
||||
return fs.readFileSync(p, 'utf8')
|
||||
} catch {
|
||||
return undefined
|
||||
}
|
||||
})
|
||||
|
||||
if (isWindows) {
|
||||
const sitePackages = path.join(venvRoot, 'Lib', 'site-packages')
|
||||
|
||||
if (directoryExists(sitePackages)) {
|
||||
entries.push(sitePackages)
|
||||
}
|
||||
|
||||
return entries
|
||||
}
|
||||
|
||||
const cfg = readFile(path.join(venvRoot, 'pyvenv.cfg'))
|
||||
|
||||
const version = (() => {
|
||||
if (!cfg) {
|
||||
return null
|
||||
}
|
||||
|
||||
const match = cfg.match(/^version_info\s*=\s*(\d+\.\d+)/im)
|
||||
|
||||
return match ? match[1].trim() : null
|
||||
})()
|
||||
|
||||
if (version) {
|
||||
const sitePackages = path.join(venvRoot, 'lib', `python${version}`, 'site-packages')
|
||||
|
||||
if (directoryExists(sitePackages)) {
|
||||
entries.push(sitePackages)
|
||||
}
|
||||
}
|
||||
|
||||
return entries
|
||||
}
|
||||
|
||||
export interface ResolveVenvHermesCommandDeps {
|
||||
isWindows: boolean
|
||||
isCommandScript: (command: string) => boolean
|
||||
fileExists: (filePath: string) => boolean
|
||||
directoryExists: (filePath: string) => boolean
|
||||
canImportHermesCli: (python: string, opts?: { env?: Record<string, string> }) => boolean
|
||||
canImportHermesCli: (python: string, opts?: { env?: Record<string, string>; cwd?: string }) => boolean
|
||||
getVenvPython: (venvRoot: string) => string
|
||||
getVenvSitePackagesEntries: (venvRoot: string) => string[]
|
||||
buildDesktopBackendEnv: (opts: {
|
||||
hermesHome: string
|
||||
pythonPathEntries: string[]
|
||||
venvRoot: string
|
||||
}) => Record<string, string>
|
||||
hermesHome: string
|
||||
buildDesktopBackendEnv: () => Record<string, string>
|
||||
resolvePath: (...segments: string[]) => string
|
||||
dirname: (p: string) => string
|
||||
basename: (p: string) => string
|
||||
@@ -226,9 +133,7 @@ export function resolveVenvHermesCommand(
|
||||
directoryExists,
|
||||
canImportHermesCli,
|
||||
getVenvPython,
|
||||
getVenvSitePackagesEntries,
|
||||
buildDesktopBackendEnv,
|
||||
hermesHome,
|
||||
resolvePath,
|
||||
dirname,
|
||||
basename,
|
||||
@@ -260,15 +165,9 @@ export function resolveVenvHermesCommand(
|
||||
|
||||
const root = dirname(venvRoot)
|
||||
|
||||
if (
|
||||
!canImportHermesCli(python, {
|
||||
env: {
|
||||
PYTHONPATH: [...(directoryExists(root) ? [root] : []), process.env.PYTHONPATH]
|
||||
.filter((entry): entry is string => Boolean(entry))
|
||||
.join(path.delimiter)
|
||||
}
|
||||
})
|
||||
) {
|
||||
// Probe with the same semantics the real spawn uses: venv interpreter,
|
||||
// cwd at the checkout root, no PYTHONPATH.
|
||||
if (!canImportHermesCli(python, { cwd: directoryExists(root) ? root : undefined })) {
|
||||
rememberLog?.(
|
||||
`Ignoring venv Hermes at ${python}: runtime import probe failed (broken/partial venv); falling through to bootstrap.`
|
||||
)
|
||||
@@ -281,11 +180,7 @@ export function resolveVenvHermesCommand(
|
||||
command: python,
|
||||
args: ['-m', 'hermes_cli.main', ...backendArgs],
|
||||
bootstrap: false,
|
||||
env: buildDesktopBackendEnv({
|
||||
hermesHome,
|
||||
pythonPathEntries: [...(directoryExists(root) ? [root] : []), ...getVenvSitePackagesEntries(venvRoot)],
|
||||
venvRoot
|
||||
}),
|
||||
env: buildDesktopBackendEnv(),
|
||||
kind: 'python',
|
||||
root,
|
||||
shell: false
|
||||
|
||||
@@ -21,6 +21,10 @@
|
||||
|
||||
import path from 'node:path'
|
||||
|
||||
import { findPackedPayload, relativizePayloadLinks, stripFetchCache } from './materialize-payload-links.mjs'
|
||||
import { batchSignAppTree } from './batch-sign-binaries.mjs'
|
||||
import { resolveSigningIdentity, signNestedChromium } from './sign-nested-chromium.mjs'
|
||||
import { sanitizeTree } from './sanitize-pe-signatures.mjs'
|
||||
import { stampExeIdentity } from './set-exe-identity.mjs'
|
||||
|
||||
export default async function afterPack(context) {
|
||||
@@ -38,4 +42,12 @@ export default async function afterPack(context) {
|
||||
// Never fail the build over a cosmetic stamp.
|
||||
console.warn(`[after-pack] exe identity stamp failed (${err.message}); Hermes.exe keeps the stock Electron icon`)
|
||||
}
|
||||
|
||||
// Batch-sign every payload binary AFTER sanitize (above) and the rcedit
|
||||
// stamp: a dangling certificate table or a subsequent resource edit would
|
||||
// invalidate the signature. The product exe is excluded here and signed
|
||||
// per-file by the customSign hook (scripts/batch-sign-binaries.mjs) after
|
||||
// electron-builder's own rcedit + fuses pass. No-op with a loud warning when
|
||||
// the AZURE_SIGN_* environment is absent (unsigned/fork/nightly lanes).
|
||||
batchSignAppTree(context.appOutDir, exe)
|
||||
}
|
||||
@@ -0,0 +1,414 @@
|
||||
// batch-sign-binaries.mjs — Authenticode-sign every standalone binary inside
|
||||
// the packed Windows app tree (Hermes.exe's sibling DLLs and everything under
|
||||
// resources/agent-payload/tools/: node.exe, ffmpeg.exe, chromium, the minted
|
||||
// CLI launcher exes, pm tool binaries, …).
|
||||
//
|
||||
// Why a batch: Windows validates only the MSIX package signature
|
||||
// (AppxSignature.p7x over AppxBlockMap.xml), so inner binaries have never been
|
||||
// signed. But an MSIX whose payload carries unsigned PEs trips SmartScreen /
|
||||
// enterprise WDAC policies that scan inner files, and signature-verification
|
||||
// tooling reports the app as mixed signed/unsigned. Task 0 (pm-clean fix
|
||||
// plan): sign the whole tree, once, in chunked signtool invocations.
|
||||
//
|
||||
// Ordering (electron-builder win32 pipeline, pinned by app-builder-lib source):
|
||||
// beforePack → pack → afterPack (this module's caller) → electron fuses
|
||||
// → signAndEditResources (rcedit on the product exe) → per-file sign hook.
|
||||
// Consequences:
|
||||
// - The batch runs in afterPack AFTER sanitize-pe-signatures.mjs (a dangling
|
||||
// certificate table makes signtool fail 0x800700C1) and AFTER the rcedit
|
||||
// identity stamp, so neither can invalidate what we sign.
|
||||
// - The product exe (`<productName>.exe`) is EXCLUDED from the batch:
|
||||
// rcedit edits its resources (and the electron fuses flip) after
|
||||
// afterPack, which invalidates any signature. It is signed per-file by the
|
||||
// customSign hook AFTER rcedit, on the exact Azure mechanism sign-msix.mjs
|
||||
// uses for the package itself.
|
||||
// - The customSign hook returns true (does nothing) for every file the batch
|
||||
// already covered, so electron-builder never re-signs one-by-one.
|
||||
//
|
||||
// Sign-nested-chromium.mjs stays as-is: it is macOS-only (codesign --deep over
|
||||
// .app bundles inside the payload for Apple notarization) and is invoked from
|
||||
// the darwin branch of after-pack.mjs. There is no overlap with this Windows
|
||||
// Authenticode batch.
|
||||
//
|
||||
// Gating matches the rest of the pipeline: this module only signs when the
|
||||
// Azure Trusted Signing variables are present (the same AZURE_SIGN_* set the
|
||||
// release-signing workflow arms provide). Without them — local builds, forks,
|
||||
// unsigned nightly lanes — it is a no-op with a loud warning, exactly like
|
||||
// stage-msixbundle.mjs. The dlib + signtool resolution reuses the
|
||||
// electron-builder cache walk from scripts/stage-msixbundle.mjs; nothing is
|
||||
// hardcoded to C:\Tools.
|
||||
|
||||
import { execFileSync } from 'node:child_process'
|
||||
import fs from 'node:fs'
|
||||
import path from 'node:path'
|
||||
import { isMain } from './utils.mjs'
|
||||
|
||||
export const CHUNK_SIZE = 100
|
||||
|
||||
/**
|
||||
* Recursively collect every .exe/.dll under dir. Symbolic links are skipped
|
||||
* (the payload's materialized-link layout can contain them; the target is
|
||||
* collected on its own walk). Sorted for deterministic chunking.
|
||||
*
|
||||
* @param {string} dir
|
||||
* @param {{ skip?: (file: string) => boolean }} [opts]
|
||||
* @returns {string[]}
|
||||
*/
|
||||
export function getBinaries(dir, opts = {}) {
|
||||
const out = []
|
||||
const walk = (current) => {
|
||||
let entries
|
||||
try {
|
||||
entries = fs.readdirSync(current, { withFileTypes: true })
|
||||
} catch {
|
||||
return
|
||||
}
|
||||
for (const entry of entries) {
|
||||
if (entry.isSymbolicLink()) continue
|
||||
const full = path.join(current, entry.name)
|
||||
if (entry.isDirectory()) {
|
||||
walk(full)
|
||||
continue
|
||||
}
|
||||
if (!entry.isFile()) continue
|
||||
const lower = entry.name.toLowerCase()
|
||||
if (lower.endsWith('.exe') || lower.endsWith('.dll')) {
|
||||
if (opts.skip && opts.skip(full)) continue
|
||||
out.push(full)
|
||||
}
|
||||
}
|
||||
}
|
||||
walk(dir)
|
||||
return out.sort()
|
||||
}
|
||||
|
||||
/** Split a file list into signtool-sized batches. */
|
||||
export function chunk(items, size = CHUNK_SIZE) {
|
||||
const chunks = []
|
||||
for (let i = 0; i < items.length; i += size) {
|
||||
chunks.push(items.slice(i, i + size))
|
||||
}
|
||||
return chunks
|
||||
}
|
||||
|
||||
/** True when the Azure Trusted Signing variables are all present. */
|
||||
export function azureSigningConfigured(env = process.env) {
|
||||
return Boolean(env.AZURE_SIGN_ENDPOINT && env.AZURE_SIGN_ACCOUNT && env.AZURE_SIGN_PROFILE)
|
||||
}
|
||||
|
||||
// The electron-builder toolset cache roots, in precedence order — the same
|
||||
// walk scripts/stage-msixbundle.mjs does (configured ELECTRON_BUILDER_CACHE
|
||||
// beats stray defaults).
|
||||
function cacheRoots(env) {
|
||||
return [
|
||||
env.ELECTRON_BUILDER_CACHE || '',
|
||||
path.join(env.LOCALAPPDATA || '', 'electron-builder', 'Cache'),
|
||||
path.join(env.USERPROFILE || '', 'AppData', 'Local', 'electron-builder', 'Cache')
|
||||
].filter(Boolean)
|
||||
}
|
||||
|
||||
/**
|
||||
* Host arch for the signing toolset. The ATS bundle ships the dlib for
|
||||
* x64/x86 only (its arm64 dir has no dlib), and a 32-bit signtool cannot
|
||||
* load a 64-bit dlib — so the signtool + dlib pair must be same-arch. On
|
||||
* arm64 hosts use the x64 pair (x64 signtool runs under Windows-on-ARM
|
||||
* emulation). Mirrors app-builder-lib's WindowsSignAzureManager
|
||||
* (`process.arch === "ia32" ? Arch.ia32 : Arch.x64`).
|
||||
*/
|
||||
export function signingArch() {
|
||||
return process.arch === 'ia32' ? 'x86' : 'x64'
|
||||
}
|
||||
|
||||
/**
|
||||
* True when a resolved path sits in the `<arch>/` dir (e.g. `.../x64/signtool.exe`
|
||||
* or `.../ats-bundle-.../x64/Azure.CodeSigning.Dlib.dll`), matching both the
|
||||
* modern windows-kits-bundle layout and the legacy windows-10/<arch> layout.
|
||||
*/
|
||||
function archMatch(file, arch) {
|
||||
return path.basename(path.dirname(file)).toLowerCase() === arch
|
||||
}
|
||||
|
||||
/**
|
||||
* Find azure.codesigning.dlib.dll under the electron-builder cache, preferring
|
||||
* the one matching the host arch (x64 unless the host is ia32) so signtool can
|
||||
* load it. Returns an absolute path or null (caller decides loud-fail vs warn).
|
||||
*/
|
||||
export function resolveTrustedSigningDlib(env = process.env) {
|
||||
const arch = signingArch()
|
||||
for (const root of cacheRoots(env)) {
|
||||
if (!fs.existsSync(root)) continue
|
||||
const found = []
|
||||
const walk = (p) => {
|
||||
let entries
|
||||
try {
|
||||
entries = fs.readdirSync(p, { withFileTypes: true })
|
||||
} catch {
|
||||
return
|
||||
}
|
||||
for (const entry of entries) {
|
||||
const full = path.join(p, entry.name)
|
||||
if (entry.isDirectory()) walk(full)
|
||||
else if (entry.name.toLowerCase() === 'azure.codesigning.dlib.dll') found.push(full)
|
||||
}
|
||||
}
|
||||
for (const entry of fs.readdirSync(root)) {
|
||||
walk(path.join(root, entry))
|
||||
}
|
||||
if (found.length > 0) {
|
||||
const matched = found.filter(f => archMatch(f, arch))
|
||||
const pool = matched.length > 0 ? matched : found
|
||||
pool.sort()
|
||||
return pool[pool.length - 1]
|
||||
}
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
||||
/**
|
||||
* Find a host signtool.exe under the electron-builder cache (the winCodeSign
|
||||
* toolset bundles the Windows Kits tools for arm64/x64/x86), or honor
|
||||
* SIGNTOOL_PATH. Prefers the signtool matching the host arch — a 32-bit
|
||||
* signtool cannot load the x64 ATS dlib — and among arch-matched candidates
|
||||
* the newest SDK build (sorted paths put the highest version last).
|
||||
* Returns an absolute path or null.
|
||||
*/
|
||||
export function resolveSigntool(env = process.env) {
|
||||
if (env.SIGNTOOL_PATH && fs.existsSync(env.SIGNTOOL_PATH)) return env.SIGNTOOL_PATH
|
||||
const arch = signingArch()
|
||||
for (const root of cacheRoots(env)) {
|
||||
if (!fs.existsSync(root)) continue
|
||||
const found = []
|
||||
for (const entry of fs.readdirSync(root)) {
|
||||
const dir = path.join(root, entry)
|
||||
if (!fs.existsSync(dir) || !fs.statSync(dir).isDirectory()) continue
|
||||
const walk = (p, depth) => {
|
||||
if (depth > 5) return
|
||||
let entries
|
||||
try {
|
||||
entries = fs.readdirSync(p, { withFileTypes: true })
|
||||
} catch {
|
||||
return
|
||||
}
|
||||
for (const sub of entries) {
|
||||
if (sub.isSymbolicLink()) continue
|
||||
const full = path.join(p, sub.name)
|
||||
if (sub.isDirectory()) walk(full, depth + 1)
|
||||
else if (sub.name.toLowerCase() === 'signtool.exe') found.push(full)
|
||||
}
|
||||
}
|
||||
walk(dir, 0)
|
||||
}
|
||||
if (found.length > 0) {
|
||||
const matched = found.filter(f => archMatch(f, arch))
|
||||
const pool = matched.length > 0 ? matched : found
|
||||
pool.sort()
|
||||
return pool[pool.length - 1]
|
||||
}
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
||||
/**
|
||||
* Find the bundled .NET 8 runtime dir (win-codesign@<ver>/dotnet-runtime-*).
|
||||
* The ATS dlib is a .NET assembly loaded via Ijwhost.dll, which locates
|
||||
* hostfxr.dll through DOTNET_ROOT — without it the dlib cannot initialize even
|
||||
* with a same-arch signtool/dlib pair. Returns an absolute path or null.
|
||||
*/
|
||||
export function resolveDotnetRuntimeDir(env = process.env) {
|
||||
for (const root of cacheRoots(env)) {
|
||||
if (!fs.existsSync(root)) continue
|
||||
const found = []
|
||||
const walk = (p, depth) => {
|
||||
if (depth > 3) return
|
||||
let entries
|
||||
try {
|
||||
entries = fs.readdirSync(p, { withFileTypes: true })
|
||||
} catch {
|
||||
return
|
||||
}
|
||||
for (const entry of entries) {
|
||||
if (!entry.isDirectory()) continue
|
||||
const full = path.join(p, entry.name)
|
||||
if (/^dotnet-runtime-/.test(entry.name)) found.push(full)
|
||||
else walk(full, depth + 1)
|
||||
}
|
||||
}
|
||||
for (const entry of fs.readdirSync(root)) {
|
||||
walk(path.join(root, entry), 0)
|
||||
}
|
||||
if (found.length > 0) {
|
||||
found.sort()
|
||||
return found[found.length - 1]
|
||||
}
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
||||
/**
|
||||
* Sign one chunk of binaries with a single signtool invocation (argv array,
|
||||
* never a shell string — the joined list can be long and must not interpolate
|
||||
* through a shell).
|
||||
*
|
||||
* @param {string[]} files
|
||||
* @param {{ signtool: string, dlib: string, metadataPath: string, exec?: typeof execFileSync, env?: NodeJS.ProcessEnv }} opts
|
||||
*/
|
||||
export function signChunk(files, opts) {
|
||||
const exec = opts.exec ?? execFileSync
|
||||
const execOptions = { stdio: 'inherit' }
|
||||
if (opts.env) execOptions.env = opts.env
|
||||
exec(opts.signtool, [
|
||||
'sign',
|
||||
'/fd', 'SHA256',
|
||||
'/td', 'SHA256',
|
||||
'/tr', 'http://timestamp.acs.microsoft.com',
|
||||
'/dlib', opts.dlib,
|
||||
'/dmdf', opts.metadataPath,
|
||||
...files
|
||||
], execOptions)
|
||||
}
|
||||
|
||||
/**
|
||||
* Batch-sign every binary under a tree.
|
||||
*
|
||||
* @param {string[]} binaries file list from getBinaries
|
||||
* @param {{ env?: NodeJS.ProcessEnv, exec?: typeof execFileSync, chunkSize?: number, mkdtemp?: typeof fs.mkdtempSync, signtool?: string, dlib?: string }} [opts]
|
||||
* @returns {{ signed: number, chunks: number, skipped: boolean }}
|
||||
* skipped=true when Azure signing is not configured (caller warns).
|
||||
*/
|
||||
export function batchSignBinaries(binaries, opts = {}) {
|
||||
const env = opts.env ?? process.env
|
||||
if (!azureSigningConfigured(env)) {
|
||||
return { signed: 0, chunks: 0, skipped: true }
|
||||
}
|
||||
if (binaries.length === 0) {
|
||||
return { signed: 0, chunks: 0, skipped: false }
|
||||
}
|
||||
const dlib = opts.dlib ?? resolveTrustedSigningDlib(env)
|
||||
if (!dlib) {
|
||||
throw new Error('batch-sign-binaries: azure.codesigning.dlib.dll not found under the electron-builder cache')
|
||||
}
|
||||
const signtool = opts.signtool ?? resolveSigntool(env)
|
||||
if (!signtool) {
|
||||
throw new Error('batch-sign-binaries: signtool.exe not found under the electron-builder cache (or SIGNTOOL_PATH)')
|
||||
}
|
||||
// The ATS dlib is a .NET assembly; Ijwhost.dll finds hostfxr.dll via
|
||||
// DOTNET_ROOT. Mirror app-builder-lib's WindowsSignAzureManager and point it
|
||||
// at the bundled runtime so the dlib initializes (a missing runtime reads as
|
||||
// "no certificates found"). Only merged when a runtime dir exists.
|
||||
const signEnv = { ...env }
|
||||
const dotnetRoot = opts.dotnetRoot ?? resolveDotnetRuntimeDir(env)
|
||||
if (dotnetRoot) signEnv.DOTNET_ROOT = dotnetRoot
|
||||
const mkdtemp = opts.mkdtemp ?? fs.mkdtempSync
|
||||
const tmpDir = mkdtemp(path.join(env.TEMP || env.TMP || '.', 'batch-sign-'))
|
||||
const metadataPath = path.join(tmpDir, 'batch-sign.json')
|
||||
fs.writeFileSync(metadataPath, JSON.stringify({
|
||||
Endpoint: env.AZURE_SIGN_ENDPOINT,
|
||||
CodeSigningAccountName: env.AZURE_SIGN_ACCOUNT,
|
||||
CertificateProfileName: env.AZURE_SIGN_PROFILE
|
||||
}))
|
||||
try {
|
||||
const batches = chunk(binaries, opts.chunkSize ?? CHUNK_SIZE)
|
||||
for (const batch of batches) {
|
||||
signChunk(batch, { signtool, dlib, metadataPath, exec: opts.exec, env: signEnv })
|
||||
}
|
||||
return { signed: binaries.length, chunks: batches.length, skipped: false }
|
||||
} finally {
|
||||
fs.rmSync(tmpDir, { recursive: true, force: true })
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* afterPack-side entry: batch-sign the packed tree, excluding the product exe
|
||||
* (it is rcedit-ed and signed per-file after this hook — see the header).
|
||||
* Callers must have run sanitize-pe-signatures.mjs first.
|
||||
*
|
||||
* @param {string} appOutDir
|
||||
* @param {string} productExePath absolute path of the main product exe
|
||||
* @param {{ env?: NodeJS.ProcessEnv, exec?: typeof execFileSync }} [opts]
|
||||
*/
|
||||
export function batchSignAppTree(appOutDir, productExePath, opts = {}) {
|
||||
const env = opts.env ?? process.env
|
||||
if (!azureSigningConfigured(env)) {
|
||||
console.warn(
|
||||
'[batch-sign] AZURE_SIGN_* not set — payload binaries will be UNSIGNED ' +
|
||||
'(package block-map still covers them; release lanes must set the signing env)'
|
||||
)
|
||||
return { signed: 0, chunks: 0, skipped: true }
|
||||
}
|
||||
const productExe = productExePath ? path.resolve(productExePath) : null
|
||||
const binaries = getBinaries(appOutDir, {
|
||||
skip: (file) => (productExe ? path.resolve(file) === productExe : false)
|
||||
})
|
||||
if (binaries.length === 0) return { signed: 0, chunks: 0, skipped: false }
|
||||
const result = batchSignBinaries(binaries, opts)
|
||||
console.log(
|
||||
`[batch-sign] signed ${result.signed} payload binaries in ${result.chunks} signtool batch(es)` +
|
||||
` (product exe excluded — signed per-file after rcedit)`
|
||||
)
|
||||
return result
|
||||
}
|
||||
|
||||
// ── electron-builder custom win.sign hook ───────────────────────────────────
|
||||
//
|
||||
// Per app-builder-lib's signtoolBaseSignManager, the custom `sign` hook is
|
||||
// invoked once per signable file (the product exe after rcedit, top-level
|
||||
// exes, asar.unpacked natives, extraResource exes). The hook's return value is
|
||||
// ignored by the manager, but per the Task 0 contract it resolves true for
|
||||
// files the afterPack batch already signed — a no-op — and delegates the two
|
||||
// artifacts that genuinely need per-file signing to the sign-msix.mjs Azure
|
||||
// machinery: the .msix/.msixbundle package and the product exe.
|
||||
|
||||
const SIGNABLE_PACKAGE_EXTENSIONS = ['.msix', '.msixbundle']
|
||||
const STORE_ARTIFACT_PREFIX = 'Store-'
|
||||
|
||||
/**
|
||||
* The electron-builder custom win.sign hook.
|
||||
*
|
||||
* @param {{ path: string }} configuration
|
||||
* @param {any} packager
|
||||
* @param {{ signMsix?: (configuration: any, packager: any) => Promise<void>, azureSignFile?: (file: string, packager: any) => Promise<void> }} [deps]
|
||||
* @returns {Promise<boolean>} true when this hook handled the file
|
||||
* (batch-signed: nothing to do) — electron-builder must not re-sign it.
|
||||
*/
|
||||
export async function customSign(configuration, packager, deps = {}) {
|
||||
const file = configuration.path
|
||||
const base = path.basename(file)
|
||||
// Store-submission packages are Partner Center's to sign (see sign-msix.mjs).
|
||||
if (base.startsWith(STORE_ARTIFACT_PREFIX)) return true
|
||||
const lower = file.toLowerCase()
|
||||
if (SIGNABLE_PACKAGE_EXTENSIONS.some(ext => lower.endsWith(ext))) {
|
||||
const { default: signMsix } = await import('./sign-msix.mjs')
|
||||
await (deps.signMsix ?? signMsix)(configuration, packager)
|
||||
return true
|
||||
}
|
||||
// The product exe was rcedit-ed after the batch ran, so it is signed here,
|
||||
// after its resources are final, on the same Azure manager sign-msix uses.
|
||||
const productName = packager?.appInfo?.productFilename
|
||||
if (productName && base.toLowerCase() === `${productName.toLowerCase()}.exe`) {
|
||||
const { azureSignFile } = await import('./sign-msix.mjs')
|
||||
await (deps.azureSignFile ?? azureSignFile)(file, packager)
|
||||
return true
|
||||
}
|
||||
// Everything else the hook is offered was already batch-signed in afterPack.
|
||||
return true
|
||||
}
|
||||
|
||||
function main() {
|
||||
const root = process.argv[2]
|
||||
if (!root) {
|
||||
console.error('usage: batch-sign-binaries.mjs <dir>')
|
||||
process.exit(2)
|
||||
}
|
||||
const result = batchSignAppTree(root, process.env.HERMES_PRODUCT_EXE || path.join(root, 'Hermes.exe'))
|
||||
if (result.skipped) process.exit(0)
|
||||
}
|
||||
|
||||
if (isMain(import.meta.url)) {
|
||||
main()
|
||||
}
|
||||
|
||||
// electron-builder's resolveFunction prefers a named export matching the hook
|
||||
// name ("sign") and falls back to the module default — provide both so the
|
||||
// config's `sign: './scripts/batch-sign-binaries.mjs'` binds to customSign.
|
||||
export { customSign as sign }
|
||||
export default customSign
|
||||
@@ -0,0 +1,336 @@
|
||||
import assert from 'node:assert/strict'
|
||||
import fs from 'node:fs'
|
||||
import os from 'node:os'
|
||||
import path from 'node:path'
|
||||
|
||||
import { afterEach, test } from 'vitest'
|
||||
|
||||
import {
|
||||
azureSigningConfigured,
|
||||
batchSignAppTree,
|
||||
chunk,
|
||||
customSign,
|
||||
getBinaries,
|
||||
resolveDotnetRuntimeDir,
|
||||
resolveSigntool,
|
||||
resolveTrustedSigningDlib,
|
||||
signingArch
|
||||
} from './batch-sign-binaries.mjs'
|
||||
|
||||
const tmpDirs = []
|
||||
|
||||
afterEach(() => {
|
||||
for (const dir of tmpDirs.splice(0)) {
|
||||
fs.rmSync(dir, { recursive: true, force: true })
|
||||
}
|
||||
})
|
||||
|
||||
function tmpTree() {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'batch-sign-test-'))
|
||||
tmpDirs.push(dir)
|
||||
return dir
|
||||
}
|
||||
|
||||
test('getBinaries collects .exe and .dll recursively, case-insensitive, sorted', () => {
|
||||
const root = tmpTree()
|
||||
fs.mkdirSync(path.join(root, 'resources', 'agent-payload', 'tools', 'python', 'bin'), { recursive: true })
|
||||
fs.mkdirSync(path.join(root, 'resources', 'agent-payload', 'tools', 'chromium-1', 'meep'), { recursive: true })
|
||||
fs.writeFileSync(path.join(root, 'Hermes.exe'), 'x')
|
||||
fs.writeFileSync(path.join(root, 'ffmpeg.dll'), 'x')
|
||||
fs.writeFileSync(path.join(root, 'resources', 'agent-payload', 'tools', 'python', 'bin', 'node.exe'), 'x')
|
||||
fs.writeFileSync(path.join(root, 'resources', 'agent-payload', 'tools', 'chromium-1', 'meep', 'chrome.dll'), 'x')
|
||||
fs.writeFileSync(path.join(root, 'resources', 'README.md'), 'x')
|
||||
|
||||
const files = getBinaries(root).map(f => path.relative(root, f))
|
||||
|
||||
assert.deepEqual(files, [
|
||||
path.join('Hermes.exe'),
|
||||
path.join('ffmpeg.dll'),
|
||||
path.join('resources', 'agent-payload', 'tools', 'chromium-1', 'meep', 'chrome.dll'),
|
||||
path.join('resources', 'agent-payload', 'tools', 'python', 'bin', 'node.exe')
|
||||
])
|
||||
})
|
||||
|
||||
test('getBinaries skips symlinks and honors the skip predicate (product exe)', () => {
|
||||
const root = tmpTree()
|
||||
fs.mkdirSync(path.join(root, 'tools'), { recursive: true })
|
||||
const target = path.join(root, 'tools', 'real.exe')
|
||||
fs.writeFileSync(target, 'x')
|
||||
fs.writeFileSync(path.join(root, 'Hermes.exe'), 'x')
|
||||
const link = path.join(root, 'tools', 'link.exe')
|
||||
try {
|
||||
fs.symlinkSync(target, link)
|
||||
} catch {
|
||||
// Windows without symlink privilege: the collection contract under test
|
||||
// is the skip predicate; symlink skipping is covered on the other OS.
|
||||
}
|
||||
|
||||
const exe = path.join(root, 'Hermes.exe')
|
||||
const files = getBinaries(root, { skip: file => path.resolve(file) === exe })
|
||||
|
||||
assert.equal(files.includes(exe), false)
|
||||
assert.equal(files.includes(target), true)
|
||||
if (fs.existsSync(link)) {
|
||||
assert.equal(files.includes(link), false)
|
||||
}
|
||||
})
|
||||
|
||||
test('getBinaries tolerates a missing directory', () => {
|
||||
assert.deepEqual(getBinaries(path.join(tmpTree(), 'nope')), [])
|
||||
})
|
||||
|
||||
test('chunk splits into ~100-file batches with no leftovers', () => {
|
||||
assert.deepEqual(chunk([], 100), [])
|
||||
const two50 = Array.from({ length: 250 }, (_, i) => `f${i}.exe`)
|
||||
const batches = chunk(two50)
|
||||
assert.equal(batches.length, 3)
|
||||
assert.deepEqual(batches.map(b => b.length), [100, 100, 50])
|
||||
assert.deepEqual(batches.flat(), two50)
|
||||
|
||||
const exact = Array.from({ length: 200 }, (_, i) => `f${i}.exe`)
|
||||
assert.deepEqual(chunk(exact).map(b => b.length), [100, 100])
|
||||
})
|
||||
|
||||
test('azureSigningConfigured requires endpoint, account, and profile together', () => {
|
||||
assert.equal(azureSigningConfigured({}), false)
|
||||
assert.equal(azureSigningConfigured({ AZURE_SIGN_ENDPOINT: 'https://cus.codesigning.azure.net' }), false)
|
||||
assert.equal(
|
||||
azureSigningConfigured({
|
||||
AZURE_SIGN_ENDPOINT: 'https://cus.codesigning.azure.net',
|
||||
AZURE_SIGN_ACCOUNT: 'codesign2'
|
||||
}),
|
||||
false
|
||||
)
|
||||
assert.equal(
|
||||
azureSigningConfigured({
|
||||
AZURE_SIGN_ENDPOINT: 'https://cus.codesigning.azure.net',
|
||||
AZURE_SIGN_ACCOUNT: 'codesign2',
|
||||
AZURE_SIGN_PROFILE: 'hermesagent'
|
||||
}),
|
||||
true
|
||||
)
|
||||
})
|
||||
|
||||
test('customSign returns true for batch-covered payload files without touching Azure', async () => {
|
||||
let azureTouched = 0
|
||||
const result = await customSign(
|
||||
{ path: 'C:/out/win-unpacked/resources/agent-payload/tools/node.exe' },
|
||||
{ appInfo: { productFilename: 'Hermes' } },
|
||||
{ azureSignFile: async () => { azureTouched += 1 } }
|
||||
)
|
||||
assert.equal(result, true)
|
||||
assert.equal(azureTouched, 0)
|
||||
})
|
||||
|
||||
test('customSign skips Store- submission packages (Partner Center signs)', async () => {
|
||||
const result = await customSign(
|
||||
{ path: 'C:/out/Store-HermesBundled-0.28.0-win-x64.msix' },
|
||||
{ appInfo: { productFilename: 'Hermes' } },
|
||||
{ signMsix: async () => { throw new Error('must not be called') } }
|
||||
)
|
||||
assert.equal(result, true)
|
||||
})
|
||||
|
||||
test('customSign delegates the msix package and the product exe to the Azure signer', async () => {
|
||||
const delegated = []
|
||||
const deps = {
|
||||
signMsix: async configuration => delegated.push(['msix', configuration.path]),
|
||||
azureSignFile: async file => delegated.push(['exe', file])
|
||||
}
|
||||
const packager = { appInfo: { productFilename: 'Hermes' } }
|
||||
|
||||
await customSign({ path: 'C:/out/HermesBundled-0.28.0-win-x64.msix' }, packager, deps)
|
||||
await customSign({ path: 'C:/out/win-unpacked/Hermes.exe' }, packager, deps)
|
||||
|
||||
// The hook's contract is `true` (handled) even when it delegated —
|
||||
// electron-builder must not fall back to per-file default signing.
|
||||
assert.deepEqual(delegated, [
|
||||
['msix', 'C:/out/HermesBundled-0.28.0-win-x64.msix'],
|
||||
['exe', 'C:/out/win-unpacked/Hermes.exe']
|
||||
])
|
||||
assert.equal(await customSign({ path: 'C:/out/win-unpacked/Hermes.exe' }, packager, deps), true)
|
||||
})
|
||||
|
||||
test('customSign does not mistake a similarly-named payload exe for the product exe', async () => {
|
||||
const deps = {
|
||||
azureSignFile: async () => { throw new Error('must not be called') }
|
||||
}
|
||||
assert.equal(
|
||||
await customSign(
|
||||
{ path: 'C:/out/win-unpacked/resources/Hermes-helper.exe' },
|
||||
{ appInfo: { productFilename: 'Hermes' } },
|
||||
deps
|
||||
),
|
||||
true
|
||||
)
|
||||
})
|
||||
|
||||
test('batchSignAppTree is a no-op (skipped=true) without the Azure env, and signs via chunked argv-array invocations when set', () => {
|
||||
const root = tmpTree()
|
||||
fs.mkdirSync(path.join(root, 'tools'), { recursive: true })
|
||||
const exe = path.join(root, 'Hermes.exe')
|
||||
fs.writeFileSync(exe, 'x')
|
||||
fs.writeFileSync(path.join(root, 'tools', 'node.exe'), 'x')
|
||||
fs.writeFileSync(path.join(root, 'tools', 'ffmpeg.dll'), 'x')
|
||||
|
||||
// Unsigned lane: loud no-op, nothing invoked.
|
||||
const skipped = batchSignAppTree(root, exe, { env: {} })
|
||||
assert.deepEqual(skipped, { signed: 0, chunks: 0, skipped: true })
|
||||
|
||||
// Signed lane: product exe excluded, chunked execFileSync with argv arrays.
|
||||
const invocations = []
|
||||
const fakeExec = (tool, args) => {
|
||||
invocations.push({ tool, args })
|
||||
return Buffer.from('')
|
||||
}
|
||||
const result = batchSignAppTree(root, exe, {
|
||||
env: {
|
||||
AZURE_SIGN_ENDPOINT: 'https://cus.codesigning.azure.net',
|
||||
AZURE_SIGN_ACCOUNT: 'codesign2',
|
||||
AZURE_SIGN_PROFILE: 'hermesagent'
|
||||
},
|
||||
exec: fakeExec,
|
||||
mkdtemp: () => root,
|
||||
signtool: 'signtool.exe',
|
||||
dlib: 'azure.codesigning.dlib.dll'
|
||||
})
|
||||
|
||||
assert.deepEqual(result, { signed: 2, chunks: 1, skipped: false })
|
||||
assert.equal(invocations.length, 1)
|
||||
assert.equal(invocations[0].tool, 'signtool.exe')
|
||||
assert.ok(Array.isArray(invocations[0].args), 'argv array, never a shell string')
|
||||
const args = invocations[0].args
|
||||
assert.ok(!args.some(arg => typeof arg === 'string' && arg.includes(' ')))
|
||||
assert.equal(args.includes(exe), false, 'product exe excluded — signed per-file after rcedit')
|
||||
assert.equal(args.includes(path.join(root, 'tools', 'node.exe')), true)
|
||||
assert.equal(args.includes(path.join(root, 'tools', 'ffmpeg.dll')), true)
|
||||
assert.equal(args[args.indexOf('/dlib') + 1], 'azure.codesigning.dlib.dll')
|
||||
assert.ok(args[args.indexOf('/dmdf') + 1].endsWith('batch-sign.json'))
|
||||
assert.equal(args[args.indexOf('/fd') + 1], 'SHA256')
|
||||
assert.equal(args[args.indexOf('/td') + 1], 'SHA256')
|
||||
})
|
||||
|
||||
test('batchSignAppTree chunks large trees into ~100-file signtool invocations', () => {
|
||||
const root = tmpTree()
|
||||
fs.mkdirSync(path.join(root, 'tools'), { recursive: true })
|
||||
for (let i = 0; i < 250; i += 1) {
|
||||
fs.writeFileSync(path.join(root, 'tools', `bin${i}.exe`), 'x')
|
||||
}
|
||||
|
||||
const invocations = []
|
||||
const result = batchSignAppTree(root, path.join(root, 'Hermes.exe'), {
|
||||
env: {
|
||||
AZURE_SIGN_ENDPOINT: 'https://cus.codesigning.azure.net',
|
||||
AZURE_SIGN_ACCOUNT: 'codesign2',
|
||||
AZURE_SIGN_PROFILE: 'hermesagent'
|
||||
},
|
||||
exec: (tool, args) => {
|
||||
invocations.push(args)
|
||||
return Buffer.from('')
|
||||
},
|
||||
mkdtemp: () => root,
|
||||
signtool: 'signtool.exe',
|
||||
dlib: 'azure.codesigning.dlib.dll'
|
||||
})
|
||||
|
||||
assert.deepEqual(result, { signed: 250, chunks: 3, skipped: false })
|
||||
assert.equal(invocations.length, 3)
|
||||
assert.deepEqual(
|
||||
invocations.map(batch => batch.filter(arg => arg.endsWith('.exe')).length),
|
||||
[100, 100, 50]
|
||||
)
|
||||
const flat = invocations.flat().filter(arg => arg.endsWith('.exe'))
|
||||
assert.equal(new Set(flat).size, 250, 'every binary signed exactly once')
|
||||
})
|
||||
|
||||
// ── toolset resolution: signtool + dlib must be same-arch ───────────────────
|
||||
// The ATS bundle ships the dlib for x64/x86 only and a 32-bit signtool cannot
|
||||
// load a 64-bit dlib — resolveSigntool/resolveTrustedSigningDlib must pair the
|
||||
// HOST arch, never "whatever sorted last" (which picked x86 signtool + x64
|
||||
// dlib and broke signing with "No certificates were found"). Regression for
|
||||
// the win32-x64 release build failure.
|
||||
|
||||
function fakeToolsetCache() {
|
||||
const cache = path.join(tmpTree(), 'electron-builder', 'Cache')
|
||||
const kits = path.join(cache, 'win-codesign@1.3.0', 'windows-kits-bundle-10_0_26100_0-abc')
|
||||
for (const arch of ['arm64', 'x64', 'x86']) {
|
||||
fs.mkdirSync(path.join(kits, arch), { recursive: true })
|
||||
fs.writeFileSync(path.join(kits, arch, 'signtool.exe'), 'x')
|
||||
}
|
||||
const ats = path.join(cache, 'win-codesign@1.3.0', 'ats-bundle-1_0_95-def')
|
||||
for (const arch of ['x64', 'x86']) {
|
||||
fs.mkdirSync(path.join(ats, arch), { recursive: true })
|
||||
fs.writeFileSync(path.join(ats, arch, 'Azure.CodeSigning.Dlib.dll'), 'x')
|
||||
}
|
||||
fs.mkdirSync(path.join(cache, 'win-codesign@1.3.0', 'dotnet-runtime-win-x64-8_0_28-ghi'), { recursive: true })
|
||||
return { cache, kits, ats }
|
||||
}
|
||||
|
||||
test('signingArch matches the host (x64 unless ia32)', () => {
|
||||
assert.equal(signingArch(), process.arch === 'ia32' ? 'x86' : 'x64')
|
||||
})
|
||||
|
||||
test('resolveSigntool prefers the host-arch kit over x86', () => {
|
||||
const { cache } = fakeToolsetCache()
|
||||
const signtool = resolveSigntool({ ELECTRON_BUILDER_CACHE: cache })
|
||||
const expectedArch = signingArch()
|
||||
assert.ok(signtool, 'a signtool must resolve from the fake cache')
|
||||
assert.ok(
|
||||
signtool.toLowerCase().includes(`\\${expectedArch}\\signtool.exe`) ||
|
||||
signtool.toLowerCase().includes(`/${expectedArch}/signtool.exe`),
|
||||
`expected the ${expectedArch} signtool, got ${signtool}`
|
||||
)
|
||||
})
|
||||
|
||||
test('resolveTrustedSigningDlib prefers the host-arch dlib over x86', () => {
|
||||
const { cache } = fakeToolsetCache()
|
||||
const dlib = resolveTrustedSigningDlib({ ELECTRON_BUILDER_CACHE: cache })
|
||||
const expectedArch = signingArch()
|
||||
assert.ok(dlib, 'a dlib must resolve from the fake cache')
|
||||
assert.ok(
|
||||
dlib.toLowerCase().includes(`\\${expectedArch}\\azure.codesigning.dlib.dll`) ||
|
||||
dlib.toLowerCase().includes(`/${expectedArch}/azure.codesigning.dlib.dll`),
|
||||
`expected the ${expectedArch} dlib, got ${dlib}`
|
||||
)
|
||||
})
|
||||
|
||||
test('resolveDotnetRuntimeDir finds the bundled .NET runtime dir', () => {
|
||||
const { cache } = fakeToolsetCache()
|
||||
const dotnet = resolveDotnetRuntimeDir({ ELECTRON_BUILDER_CACHE: cache })
|
||||
assert.ok(dotnet, 'the dotnet-runtime bundle must resolve')
|
||||
assert.ok(/dotnet-runtime-/.test(path.basename(dotnet)), `unexpected dotnet dir ${dotnet}`)
|
||||
})
|
||||
|
||||
test('batchSignBinaries sets DOTNET_ROOT from the bundled runtime for the signtool child', () => {
|
||||
const { cache } = fakeToolsetCache()
|
||||
const root = tmpTree()
|
||||
fs.mkdirSync(path.join(root, 'tools'), { recursive: true })
|
||||
fs.writeFileSync(path.join(root, 'tools', 'node.exe'), 'x')
|
||||
const exe = path.join(root, 'Hermes.exe')
|
||||
|
||||
const invocations = []
|
||||
const result = batchSignAppTree(root, exe, {
|
||||
env: {
|
||||
ELECTRON_BUILDER_CACHE: cache,
|
||||
AZURE_SIGN_ENDPOINT: 'https://cus.codesigning.azure.net',
|
||||
AZURE_SIGN_ACCOUNT: 'codesign2',
|
||||
AZURE_SIGN_PROFILE: 'hermesagent'
|
||||
},
|
||||
exec: (tool, args, opts) => {
|
||||
invocations.push({ tool, args, opts })
|
||||
return Buffer.from('')
|
||||
},
|
||||
mkdtemp: () => root
|
||||
})
|
||||
|
||||
assert.equal(result.signed, 1)
|
||||
assert.equal(invocations.length, 1)
|
||||
const opts = invocations[0].opts
|
||||
assert.ok(opts && opts.env, 'exec must receive a child env')
|
||||
assert.equal(opts.env.DOTNET_ROOT, path.join(cache, 'win-codesign@1.3.0', 'dotnet-runtime-win-x64-8_0_28-ghi'))
|
||||
// The signtool chosen must be the host-arch one, not x86.
|
||||
assert.match(invocations[0].tool, new RegExp(`[\\\\/]${signingArch()}[\\\\/]signtool\\.exe$`))
|
||||
// dlib must be same arch.
|
||||
const dlibArg = invocations[0].args[invocations[0].args.indexOf('/dlib') + 1]
|
||||
assert.match(dlibArg, new RegExp(`[\\\\/]${signingArch()}[\\\\/]azure\\.codesigning\\.dlib\\.dll$`, 'i'))
|
||||
})
|
||||
@@ -6,6 +6,106 @@
|
||||
* longer bundled; the Electron app fetches it at first launch via
|
||||
* `install.ps1`'s stage protocol (Windows). See `electron/main.ts`.
|
||||
*/
|
||||
import fs from 'node:fs'
|
||||
import path from 'node:path'
|
||||
import { createRequire } from 'node:module'
|
||||
|
||||
import { CLI_LAUNCHER_SPECS } from '../../../scripts/desktop-cli/cli-entrypoints.mjs'
|
||||
|
||||
const require = createRequire(import.meta.url)
|
||||
const {
|
||||
light,
|
||||
displayName,
|
||||
appNamePascal
|
||||
} = require('../product-identity.cjs')
|
||||
|
||||
export default async function beforeBuild() {
|
||||
return false
|
||||
}
|
||||
|
||||
function stageMsixAssets() {
|
||||
const desktop = path.join(import.meta.dirname, '..')
|
||||
const sourceDir = path.join(desktop, 'assets', 'appx')
|
||||
const stageDir = path.join(desktop, 'build', 'appx')
|
||||
const names = [
|
||||
'Square44x44Logo.png',
|
||||
'Square150x150Logo.png',
|
||||
'StoreLogo.png',
|
||||
'Wide310x150Logo.png'
|
||||
]
|
||||
|
||||
fs.mkdirSync(stageDir, { recursive: true })
|
||||
for (const name of names) {
|
||||
const source = path.join(sourceDir, name)
|
||||
if (!fs.existsSync(source)) {
|
||||
throw new Error(`missing MSIX asset ${source}`)
|
||||
}
|
||||
fs.copyFileSync(source, path.join(stageDir, name))
|
||||
}
|
||||
}
|
||||
|
||||
function writeMsixExtensions() {
|
||||
const desktop = path.join(import.meta.dirname, '..')
|
||||
const output = path.join('build', 'msix-extensions.xml')
|
||||
const file = path.join(desktop, output)
|
||||
// One uap5:Extension per payload CLI launcher exe (see
|
||||
// scripts/desktop-cli/cli-entrypoints.mjs): the minted exes are three
|
||||
// REAL executables now — the old rust shim's one-exe-argv[0]-dispatch is
|
||||
// gone — and an AppExecutionAlias's Executable must name the exact exe
|
||||
// that serves the alias, so each alias gets its own Extension block.
|
||||
const aliases = light ? '' : appExecutionAliasExtensions()
|
||||
// The uap3:AppExtension fragment that registers the app as a Windows
|
||||
// Copilot hardware key provider. The press activates hermes://copilot-key/start.
|
||||
//
|
||||
// Content rules (violations are an opaque makeappx 0x80080204):
|
||||
// * xmlns:uap3 rides on the fragment root — the stock manifest template
|
||||
// declares no uap3 prefix. A/B-verified fine.
|
||||
// * children of uap3:Properties are UNPREFIXED (xs:any content, per
|
||||
// Microsoft's copilot-key-state sample).
|
||||
const copilot = light
|
||||
? ''
|
||||
: `<uap3:Extension
|
||||
xmlns:uap3="http://schemas.microsoft.com/appx/manifest/uap/windows10/3"
|
||||
Category="windows.appExtension">
|
||||
<uap3:AppExtension
|
||||
Name="com.microsoft.windows.copilotkeyprovider"
|
||||
Id="${appNamePascal}CopilotKeyProvider"
|
||||
DisplayName="${displayName}"
|
||||
Description="Launch ${displayName} with the Copilot key"
|
||||
PublicFolder="Public">
|
||||
<uap3:Properties>
|
||||
<SingleTap>hermes://copilot-key/start?state=Tap</SingleTap>
|
||||
<PressAndHoldStart>hermes://copilot-key/start?state=Down</PressAndHoldStart>
|
||||
<PressAndHoldStop>hermes://copilot-key/stop?state=Up</PressAndHoldStop>
|
||||
</uap3:Properties>
|
||||
</uap3:AppExtension>
|
||||
</uap3:Extension>
|
||||
${aliases}`
|
||||
|
||||
fs.mkdirSync(path.dirname(file), { recursive: true })
|
||||
fs.writeFileSync(file, copilot)
|
||||
}
|
||||
|
||||
/**
|
||||
* One uap5:Extension block per payload CLI launcher, each naming its own
|
||||
* Executable (the distlib-minted launcher exes under bin/) and the alias
|
||||
* that exe serves. Exported pure for tests.
|
||||
* @param {{ name: string }[]} [launchers] exe stems under bin/
|
||||
*/
|
||||
export function appExecutionAliasExtensions(launchers = CLI_LAUNCHER_SPECS.map((s) => s.name)) {
|
||||
const bs = String.fromCharCode(92)
|
||||
const executable = (name) => ['app', 'resources', 'agent-payload', 'bin', `${name}.exe`].join(bs)
|
||||
return launchers
|
||||
.map(
|
||||
(name) => `<uap5:Extension
|
||||
xmlns:uap5="http://schemas.microsoft.com/appx/manifest/uap/windows10/5"
|
||||
Category="windows.appExecutionAlias"
|
||||
Executable="${executable(name)}"
|
||||
EntryPoint="Windows.FullTrustApplication">
|
||||
<uap5:AppExecutionAlias>
|
||||
<uap5:ExecutionAlias Alias="${name}.exe" />
|
||||
</uap5:AppExecutionAlias>
|
||||
</uap5:Extension>`
|
||||
)
|
||||
.join('\n')
|
||||
}
|
||||
@@ -0,0 +1,115 @@
|
||||
import assert from 'node:assert/strict'
|
||||
import path from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
|
||||
import { test } from 'vitest'
|
||||
|
||||
import { CLI_LAUNCHER_SPECS, posixTrampolineScripts, renderWinWrapper } from '../../../scripts/desktop-cli/cli-entrypoints.mjs'
|
||||
import { appExecutionAliasExtensions } from './before-build.mjs'
|
||||
|
||||
const scriptDir = path.dirname(fileURLToPath(import.meta.url))
|
||||
|
||||
// The payload layout facts every generator receives (bin-relative, forward
|
||||
// slashes — the same composition build-bundled-desktop.mjs step 5b feeds in).
|
||||
const RELS = {
|
||||
relPython: '../tools/cpython-3.11.16+20260814-win32-x64/bin/python3',
|
||||
relSite: '../venv/lib/python3.11/site-packages',
|
||||
relRepo: '../hermes-agent'
|
||||
}
|
||||
|
||||
test('three launcher specs mirror [project.scripts] in pyproject.toml', () => {
|
||||
assert.deepEqual(CLI_LAUNCHER_SPECS.map((s) => s.name), ['hermes', 'hermes-agent', 'hermes-acp'])
|
||||
assert.deepEqual(CLI_LAUNCHER_SPECS.map((s) => `${s.module}:${s.func}`), [
|
||||
'hermes_cli.main:main',
|
||||
'run_agent:main',
|
||||
'acp_adapter.entry:main'
|
||||
])
|
||||
})
|
||||
|
||||
const scripts = posixTrampolineScripts(RELS)
|
||||
|
||||
test('one POSIX trampoline per entry, named plainly (no suffix)', () => {
|
||||
assert.deepEqual(scripts.map((s) => s.name), ['hermes', 'hermes-agent', 'hermes-acp'])
|
||||
})
|
||||
|
||||
test('trampoline is $0-relative: shebang, symlink-chain resolution, own-dir cd', () => {
|
||||
const text = scripts[0].text
|
||||
assert.ok(text.startsWith('#!/usr/bin/env bash\n'))
|
||||
// The whole relocatability contract: nothing in the script may be an
|
||||
// absolute path or a Windows path — everything resolves off $0.
|
||||
for (const line of text.split('\n')) {
|
||||
const code = line.replace(/^#/, '').trim()
|
||||
assert.ok(!/[A-Za-z]:\\/.test(code), `windows path in generated script: ${line}`)
|
||||
assert.ok(!/=\s*\/(?!usr\/bin\/env)/.test(code), `absolute path in generated script: ${line}`)
|
||||
}
|
||||
assert.match(text, /self="\$0"/)
|
||||
assert.match(text, /while \[ -L "\$self" \]/)
|
||||
assert.match(text, /BIN_DIR="\$\(cd -- "\$\(dirname -- "\$self"\)" && pwd\)"/)
|
||||
assert.match(text, /PYTHON="\$BIN_DIR"\/\.\.\/tools\//)
|
||||
})
|
||||
|
||||
test('trampoline composes the payload import roots (repo first) and replaces inherited PYTHONPATH', () => {
|
||||
const text = scripts[0].text
|
||||
assert.match(text, /unset PYTHONPATH PYTHONHOME/)
|
||||
assert.match(text, /export PYTHONPATH="\$REPO:\$SITE"/)
|
||||
assert.match(text, /REPO="\$BIN_DIR"\/\.\.\/hermes-agent/)
|
||||
assert.match(text, /SITE="\$BIN_DIR"\/\.\.\/venv\/lib\/python3\.11\/site-packages/)
|
||||
})
|
||||
|
||||
test('trampoline keeps pycache out of the payload and execs the entry module', () => {
|
||||
const text = scripts[0].text
|
||||
assert.match(text, /PYTHONPYCACHEPREFIX="\$HOME\/\.cache\/hermes-pycache"/)
|
||||
assert.match(text, /if \[ -z "\$\{PYTHONPYCACHEPREFIX:-\}" \]/, 'user-set prefix must win')
|
||||
assert.match(text, /exec "\$PYTHON" -m hermes_cli\.main "\$@"/)
|
||||
})
|
||||
|
||||
test('trampoline fails loudly (exit 2) when the bundled interpreter is missing', () => {
|
||||
const text = scripts[0].text
|
||||
assert.match(text, /bundled interpreter missing at \$PYTHON/)
|
||||
assert.match(text, /exit 2/)
|
||||
})
|
||||
|
||||
test('each trampoline execs its own entry module', () => {
|
||||
const modules = scripts.map((s) => /exec "\$PYTHON" -m (\S+) "\$@"/.exec(s.text)?.[1])
|
||||
assert.deepEqual(modules, ['hermes_cli.main', 'run_agent', 'acp_adapter.entry'])
|
||||
})
|
||||
|
||||
test('win wrapper substitution bakes the entry module and payload layout facts', () => {
|
||||
const text = renderWinWrapper(CLI_LAUNCHER_SPECS[0], RELS.relRepo, RELS.relSite)
|
||||
assert.ok(!text.includes('__HERMES_'), 'placeholders must be fully substituted')
|
||||
assert.match(text, /HERMES_ENTRY_MODULE = "hermes_cli\.main"/)
|
||||
assert.match(text, /HERMES_ENTRY_FUNC = "main"/)
|
||||
assert.match(text, /HERMES_REPO_REL = "\.\.\/hermes-agent"/)
|
||||
assert.match(text, /HERMES_SITE_REL = "\.\.\/venv\/lib\/python3\.11\/site-packages"/)
|
||||
})
|
||||
|
||||
test('win wrapper rejects values that still contain placeholders', () => {
|
||||
assert.throws(() => renderWinWrapper({ module: '__HERMES_REPO_REL__', func: 'main' }, RELS.relRepo, RELS.relSite))
|
||||
})
|
||||
|
||||
test('MSIX: one uap5 alias Extension per launcher exe, Executable naming that exe', () => {
|
||||
const xml = appExecutionAliasExtensions()
|
||||
const blocks = xml.match(/<uap5:Extension\b/g) ?? []
|
||||
assert.equal(blocks.length, 3)
|
||||
for (const spec of CLI_LAUNCHER_SPECS) {
|
||||
const expected = ['app', 'resources', 'agent-payload', 'bin', `${spec.name}.exe`].join('\\')
|
||||
assert.ok(
|
||||
xml.includes(`Executable="${expected}"`),
|
||||
`no alias Extension names ${expected}`
|
||||
)
|
||||
// Backslashes: MSIX Executable must be manifest-backslash form.
|
||||
assert.ok(!xml.includes(`${expected}/`), 'forward slashes are not manifest form')
|
||||
}
|
||||
// The old rust shim's single-exe-argv[0]-dispatch shape is gone: each
|
||||
// block serves exactly ONE alias.
|
||||
for (const block of (xml.match(/<uap5:Extension[\s\S]*?<\/uap5:Extension>/g) ?? [])) {
|
||||
assert.equal((block.match(/<uap5:ExecutionAlias /g) ?? []).length, 1)
|
||||
}
|
||||
})
|
||||
|
||||
test('light variant emits no alias fragments', () => {
|
||||
// The light gating lives in writeMsixExtensions (light ? '' : ...); the
|
||||
// pure builder must stay gating-free, so just pin its shape here.
|
||||
assert.ok(appExecutionAliasExtensions(['hermes']).includes('windows.appExecutionAlias'))
|
||||
assert.ok(scriptDir.length > 0)
|
||||
})
|
||||
@@ -0,0 +1,115 @@
|
||||
// Generate the AppxManifest.xml that MsixTarget.writeManifest produces on
|
||||
// the win32 CI lane, using the REAL app-builder-lib helpers and the real
|
||||
// builder config, so the manifest can be inspected off-Windows — makeappx
|
||||
// reports every manifest problem as a bare 0x80080204, and this is the
|
||||
// XML it is rejecting.
|
||||
//
|
||||
// Usage (from apps/desktop):
|
||||
// node scripts/gen-msix-manifest.mjs [bundled|light] [x64|arm64]
|
||||
//
|
||||
// The output is the substituted manifest on stdout. Imports reach into
|
||||
// app-builder-lib's dist because the exports map hides the internals; the
|
||||
// paths are the same ones MsixTarget itself uses, so a bump that moves
|
||||
// them fails here loudly.
|
||||
import { readFile, readdir } from "node:fs/promises"
|
||||
import { createRequire } from "node:module"
|
||||
import path from "node:path"
|
||||
import { fileURLToPath } from "node:url"
|
||||
|
||||
import {
|
||||
buildCapabilitiesXml,
|
||||
buildExtensionsXml,
|
||||
defaultTileTag,
|
||||
lockScreenTag,
|
||||
resolvePackageApplicationId,
|
||||
resolvePackageIdentityName,
|
||||
resourceLanguageTag,
|
||||
splashScreenTag,
|
||||
substituteManifestMacros,
|
||||
} from "../../../node_modules/app-builder-lib/dist/targets/win/winAppUtil.js"
|
||||
import { appIdentity } from "../../../scripts/msix-shared.mjs"
|
||||
|
||||
const desktop = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..")
|
||||
const repoRoot = path.resolve(desktop, "..", "..")
|
||||
const variant = process.argv[2] || "bundled"
|
||||
const arch = process.argv[3] || "x64"
|
||||
|
||||
if (!["bundled", "light", "store"].includes(variant)) {
|
||||
console.error(`variant must be 'bundled', 'light', or 'store', got '${variant}'`)
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
process.env.HERMES_DESKTOP_VARIANT = variant
|
||||
const require = createRequire(import.meta.url)
|
||||
const config = require(path.join(desktop, "electron-builder.config.cjs"))
|
||||
const pkg = require(path.join(desktop, "package.json"))
|
||||
|
||||
const options = config.msix
|
||||
// The payload CLI launcher (hermes.exe, the minted distlib launcher) is the
|
||||
// MSIX entry point; the alias and AppExtension fragments in the config
|
||||
// reference the same path. The launcher is self-relative and sets the
|
||||
// payload's own PYTHONPATH.
|
||||
const executable = `app\\resources\\agent-payload\\bin\\hermes.exe`
|
||||
const displayName = options.displayName || config.productName
|
||||
// appInfo.name honours extraMetadata.name the way packager merging does.
|
||||
const appInfoName = config.extraMetadata?.name || pkg.name
|
||||
|
||||
// Same asset discovery as computeUserAssets: requiring the config staged
|
||||
// assets/appx/ into build/appx/ (the buildResources "appx" dir MsixTarget
|
||||
// reads), so listing it here sees exactly what the win32 lane packs.
|
||||
const userAssets = (await readdir(path.join(desktop, "build", "appx")))
|
||||
.filter((it) => !it.startsWith(".") && !it.endsWith(".db") && it.includes("."))
|
||||
|
||||
const extensions = await buildExtensionsXml({
|
||||
protocols: config.protocols,
|
||||
fileAssociations: [],
|
||||
addAutoLaunchExtension: options.addAutoLaunchExtension,
|
||||
customExtensionsPath: options.customExtensionsPath,
|
||||
appDir: desktop,
|
||||
executable,
|
||||
displayName,
|
||||
dependencyNames: {},
|
||||
})
|
||||
|
||||
// Same precedence as MsixTarget.writeManifest: the config's custom
|
||||
// template when set (resolved relative to the app dir, as getResource
|
||||
// does), else the stock template.
|
||||
const template = await readFile(
|
||||
options.customManifestPath
|
||||
? path.resolve(desktop, options.customManifestPath)
|
||||
: path.join(repoRoot, "node_modules", "app-builder-lib", "templates", "msix", "appxmanifest.xml"),
|
||||
"utf8"
|
||||
)
|
||||
|
||||
const manifest = substituteManifestMacros(template, (m) => {
|
||||
switch (m) {
|
||||
case "publisher": return options.publisher
|
||||
case "publisherDisplayName": return options.publisherDisplayName
|
||||
// Same 4-part derivation the real build uses (msix-shared::appIdentity) —
|
||||
// a nightly shows its minutes-since-stable component here too, so this
|
||||
// inspection tool never disagrees with what electron-builder shipped.
|
||||
case "version": return appIdentity(desktop, process.env.HERMES_PAYLOAD_TAG).version
|
||||
case "applicationId": return resolvePackageApplicationId(options.applicationId, options.identityName, appInfoName, "MSIX")
|
||||
case "identityName": return resolvePackageIdentityName(options.identityName, appInfoName, "MSIX")
|
||||
case "executable": return executable
|
||||
case "displayName": return displayName
|
||||
case "description": return pkg.description || config.productName
|
||||
case "backgroundColor": return options.backgroundColor || "#464646"
|
||||
case "logo": return "assets\\StoreLogo.png"
|
||||
case "square150x150Logo": return "assets\\Square150x150Logo.png"
|
||||
case "square44x44Logo": return "assets\\Square44x44Logo.png"
|
||||
case "lockScreen": return lockScreenTag(userAssets)
|
||||
case "defaultTile": return defaultTileTag(userAssets, options.showNameOnTiles || false)
|
||||
case "splashScreen": return splashScreenTag(userAssets)
|
||||
case "arch": return arch
|
||||
case "resourceLanguages": return resourceLanguageTag(options.languages)
|
||||
case "capabilities": return `<Capabilities>\n${buildCapabilitiesXml(options.capabilities)}\n</Capabilities>`
|
||||
case "extensions": return extensions
|
||||
case "minVersion": return options.minVersion || "10.0.17763.0"
|
||||
case "maxVersionTested": return options.maxVersionTested || options.minVersion || "10.0.17763.0"
|
||||
case "packageIntegrity": return ""
|
||||
default: throw new Error(`Macro ${m} is not defined`)
|
||||
}
|
||||
})
|
||||
|
||||
console.log(manifest)
|
||||
@@ -0,0 +1,116 @@
|
||||
// MSIX BUILD_NUMBER round-trip (plan item 3.2): scripts/msix-shared.mjs's
|
||||
// appIdentity() derivation must equal the Version= the built manifest ships.
|
||||
//
|
||||
// The "built manifest" here is the REAL one the win32 lane packs: the repo's
|
||||
// custom template (assets/msix-manifest.xml) run through app-builder-lib's
|
||||
// substituteManifestMacros — the same helper MsixTarget.writeManifest uses —
|
||||
// with the version macro fed by appIdentity(), exactly as
|
||||
// scripts/gen-msix-manifest.mjs (the offline inspection twin of the build)
|
||||
// does. The test then reads the Version= back out of the XML and compares.
|
||||
//
|
||||
// The git-backed stable lookup is deterministic here because
|
||||
// node:child_process.execFileSync is mocked; the math it feeds is the
|
||||
// contract App Installer and makeappx compare.
|
||||
import assert from 'node:assert/strict'
|
||||
import fs from 'node:fs'
|
||||
import os from 'node:os'
|
||||
import path from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import { createRequire } from 'node:module'
|
||||
import { beforeEach, test, vi } from 'vitest'
|
||||
|
||||
vi.mock('node:child_process', () => ({
|
||||
execFileSync: vi.fn(),
|
||||
}))
|
||||
|
||||
const { execFileSync } = await import('node:child_process')
|
||||
const msix = await import('../../../scripts/msix-shared.mjs')
|
||||
const require = createRequire(import.meta.url)
|
||||
|
||||
// The real helpers + template the build itself uses.
|
||||
const { substituteManifestMacros } = require('../../../node_modules/app-builder-lib/dist/targets/win/winAppUtil.js')
|
||||
const scriptsDir = path.dirname(fileURLToPath(import.meta.url))
|
||||
const desktopDir = path.resolve(scriptsDir, '..')
|
||||
const template = fs.readFileSync(path.join(desktopDir, 'assets', 'msix-manifest.xml'), 'utf8')
|
||||
|
||||
// A fake app dir with just enough for appIdentity (same shape as
|
||||
// msix-shared.test.mjs's fixture).
|
||||
function makeFakeDesktop(version) {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'msix-roundtrip-'))
|
||||
fs.writeFileSync(
|
||||
path.join(dir, 'product-identity.cjs'),
|
||||
"module.exports = { store: false, light: false, displayName: 'Hermes', appId: 'com.nousresearch.hermes-bundled', channel: 'latest', appNamePascal: 'HermesBundled', msixAppIdWithOrg: 'NousResearch.HermesBundled' }\n"
|
||||
)
|
||||
fs.writeFileSync(path.join(dir, 'package.json'), JSON.stringify({ name: 'hermes-desktop', version }))
|
||||
return dir
|
||||
}
|
||||
|
||||
function gitMock(tags, stableEpoch) {
|
||||
execFileSync.mockImplementation((cmd, args) => {
|
||||
if (args[0] === 'tag') return `${tags.join('\n')}\n`
|
||||
if (args[0] === 'log') return `${stableEpoch}\n`
|
||||
throw new Error(`unexpected git call ${cmd} ${args.join(' ')}`)
|
||||
})
|
||||
}
|
||||
|
||||
// Substitute the template the way the build does: the version macro comes
|
||||
// from appIdentity(); every other macro gets a deterministic placeholder —
|
||||
// they are identity/packaging strings, not the contract under test here.
|
||||
function buildManifest(appDir, tag) {
|
||||
const { version } = msix.appIdentity(appDir, tag)
|
||||
return { version, xml: substituteManifestMacros(template, (m) => (m === 'version' ? version : `test-${m}`)) }
|
||||
}
|
||||
|
||||
function identityVersion(xml) {
|
||||
const identity = /<Identity\b[^>]*>/.exec(xml)
|
||||
assert.ok(identity, 'substituted manifest has no Identity element')
|
||||
const version = /Version="([^"]*)"/.exec(identity[0])
|
||||
assert.ok(version, 'Identity element carries no Version attribute')
|
||||
return version[1]
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
execFileSync.mockReset()
|
||||
})
|
||||
|
||||
test('stable tag: appIdentity derivation round-trips into the manifest Version', () => {
|
||||
const app = makeFakeDesktop('0.27.1')
|
||||
const { version, xml } = buildManifest(app, 'v0.27.1')
|
||||
assert.equal(version, '0.27.1.0')
|
||||
assert.equal(identityVersion(xml), version)
|
||||
})
|
||||
|
||||
test('nightly tag: minutes-since-stable build number round-trips into the manifest Version', () => {
|
||||
const app = makeFakeDesktop('0.27.1')
|
||||
// Stable v0.27.1 committed 2026-08-01T00:00:00Z; nightly cut 2026-08-29T01:02:03Z.
|
||||
const stableEpoch = Math.floor(Date.UTC(2026, 7, 1) / 1000)
|
||||
gitMock(['v0.27.1', 'v0.27.2-nightly.20260829010203'], stableEpoch)
|
||||
const { version, xml } = buildManifest(app, 'v0.27.2-nightly.20260829010203')
|
||||
const expectedMinutes = Math.floor((Date.UTC(2026, 7, 29, 1, 2, 3) - Date.UTC(2026, 7, 1)) / 60000)
|
||||
assert.equal(version, `0.27.2.${expectedMinutes}`)
|
||||
assert.equal(identityVersion(xml), version)
|
||||
})
|
||||
|
||||
test('manifest Version components are 16-bit (makeappx rejects anything larger)', () => {
|
||||
const app = makeFakeDesktop('0.27.1')
|
||||
const stableEpoch = Math.floor(Date.UTC(2026, 7, 1) / 1000)
|
||||
gitMock(['v0.27.1', 'v0.27.2-nightly.20260829010203'], stableEpoch)
|
||||
const { xml } = buildManifest(app, 'v0.27.2-nightly.20260829010203')
|
||||
for (const part of identityVersion(xml).split('.')) {
|
||||
const n = Number(part)
|
||||
assert.ok(Number.isInteger(n) && n >= 0 && n <= 65535, `component ${part} outside 16 bits`)
|
||||
}
|
||||
})
|
||||
|
||||
test('a later nightly stamps a strictly larger BUILD_NUMBER than an earlier one', () => {
|
||||
// The build number exists so Windows can order nightlies on the same
|
||||
// base version; monotonicity across the stamp is the actual contract.
|
||||
const app = makeFakeDesktop('0.27.1')
|
||||
const stableEpoch = Math.floor(Date.UTC(2026, 7, 1) / 1000)
|
||||
const early = 'v0.27.2-nightly.20260815080000'
|
||||
const late = 'v0.27.2-nightly.20260829010203'
|
||||
gitMock([early, late, 'v0.27.1'], stableEpoch)
|
||||
const earlyBuild = Number(buildManifest(app, early).version.split('.')[3])
|
||||
const lateBuild = Number(buildManifest(app, late).version.split('.')[3])
|
||||
assert.ok(lateBuild > earlyBuild, `${lateBuild} must exceed ${earlyBuild}`)
|
||||
})
|
||||
@@ -0,0 +1,151 @@
|
||||
// electron-builder custom win.sign hook: Azure Trusted Signing for the MSIX
|
||||
// package only. Windows install validation checks the package signature
|
||||
// (AppxSignature.p7x over AppxBlockMap.xml); inner files are covered by the
|
||||
// block-map hashes, NOT per-file Authenticode, so Hermes.exe and every
|
||||
// payload binary stay unsigned and this hook signs exactly one artifact per
|
||||
// build.
|
||||
//
|
||||
// Wired from electron-builder.config.cjs as
|
||||
// win.sign = { type: 'signtool', sign: './scripts/sign-msix.mjs', ... }
|
||||
// The Azure endpoint/account/profile do NOT ride through the config: this
|
||||
// module reads the AZURE_SIGN_* environment variables directly
|
||||
// (azureConfigFromEnv) and hands them to app-builder-lib's own
|
||||
// WindowsSignAzureManager, so the actual signing path is byte-for-byte the
|
||||
// one a plain { type: 'azure' } config would run.
|
||||
//
|
||||
// The Store-submission variant (artifactName "Store-" prefixed) is the one
|
||||
// exception: its manifest Publisher is the Partner Center publisher ID
|
||||
// (CN=EE6D86E4-...), which no signable cert subject can match — CA/B CSBR
|
||||
// requires the legal entity's validated name and Artifact Signing cannot
|
||||
// customize CN — so SignerSign rejects the package with ERROR_BAD_FORMAT
|
||||
// (0x8007000B). Partner Center re-signs the package with the Microsoft
|
||||
// Store certificate on ingestion, so the Store- msix ships unsigned.
|
||||
|
||||
import fs from 'node:fs'
|
||||
import { createRequire } from 'node:module'
|
||||
import path from 'node:path'
|
||||
import { pathToFileURL } from 'node:url'
|
||||
|
||||
const require = createRequire(import.meta.url)
|
||||
|
||||
// Artifacts that carry the only signature Windows validates for a packaged
|
||||
// app. A .msixbundle envelope is included for the single-invocation bundle
|
||||
// case (the envelope signature covers the inner .msix packages).
|
||||
const SIGNED_ARTIFACT_EXTENSIONS = ['.msix', '.msixbundle']
|
||||
|
||||
// Store-submission artifacts are not ATS-signable (publisher mismatch, see
|
||||
// the header) and don't need a signature (Partner Center signs on
|
||||
// ingestion). Anything with this basename prefix is left unsigned.
|
||||
const STORE_ARTIFACT_PREFIX = 'Store-'
|
||||
|
||||
/**
|
||||
* Whether `file` is a signing target. Everything except the MSIX package is
|
||||
* covered by the package's block-map hashes and must stay untouched —
|
||||
* signing it after packaging would break the hash. The Store-submission
|
||||
* variant is excluded too: it cannot carry an ATS signature and does not
|
||||
* need one.
|
||||
*/
|
||||
export function shouldSignFile(file) {
|
||||
if (path.basename(file).startsWith(STORE_ARTIFACT_PREFIX)) return false
|
||||
const lower = file.toLowerCase()
|
||||
return SIGNED_ARTIFACT_EXTENSIONS.some(ext => lower.endsWith(ext))
|
||||
}
|
||||
|
||||
/**
|
||||
* @param {NodeJS.ProcessEnv} [env]
|
||||
* @returns {{ type: 'azure' } & Record<string, string | undefined>} the
|
||||
* win.sign azure configuration, composed from the environment.
|
||||
*/
|
||||
export function azureConfigFromEnv(env = process.env) {
|
||||
return {
|
||||
type: 'azure',
|
||||
endpoint: env.AZURE_SIGN_ENDPOINT,
|
||||
codeSigningAccountName: env.AZURE_SIGN_ACCOUNT,
|
||||
certificateProfileName: env.AZURE_SIGN_PROFILE,
|
||||
publisherName: env.AZURE_SIGN_PUBLISHER
|
||||
}
|
||||
}
|
||||
|
||||
// app-builder-lib's exports map exposes only "." and "./internal";
|
||||
// import('app-builder-lib/dist/...') throws ERR_PACKAGE_PATH_NOT_EXPORTED.
|
||||
// Resolve the class the way run-electron-builder.mjs finds the CLI: resolve
|
||||
// the entry module, walk up to the package root, then direct-file import
|
||||
// (file URLs bypass the exports map). The constructibility of this class is
|
||||
// pinned by the tripwire test in sign-msix.test.mjs, so a builder bump
|
||||
// that moves it fails js-tests instead of a release build.
|
||||
async function loadAzureManagerClass() {
|
||||
const entry = require.resolve('app-builder-lib')
|
||||
let root = path.dirname(entry)
|
||||
while (!fs.existsSync(path.join(root, 'package.json'))) {
|
||||
const parent = path.dirname(root)
|
||||
if (parent === root) throw new Error('app-builder-lib package root not found')
|
||||
root = parent
|
||||
}
|
||||
const mod = await import(
|
||||
pathToFileURL(path.join(root, 'dist', 'codeSign', 'win', 'windowsSignAzureManager.js')).href
|
||||
)
|
||||
return mod.WindowsSignAzureManager
|
||||
}
|
||||
|
||||
// One manager per process: electron-builder calls the hook once per file,
|
||||
// and the manager memoizes toolset downloads / dlib metadata behind it.
|
||||
let managerPromise = null
|
||||
|
||||
function azureManager(packager) {
|
||||
if (managerPromise == null) {
|
||||
managerPromise = (async () => {
|
||||
const WindowsSignAzureManager = await loadAzureManagerClass()
|
||||
// The manager's constructor re-derives the signing config from
|
||||
// packager.platformOptions.sign and throws unless type === 'azure' —
|
||||
// but our config's win.sign is the { type: 'signtool' } hook wiring.
|
||||
// Shim the packager with the azure config composed from the
|
||||
// environment; everything else (config.toolsets, buildResourcesDir,
|
||||
// getTempFile) delegates to the real packager via the prototype.
|
||||
const shim = Object.create(packager)
|
||||
Object.defineProperty(shim, 'platformOptions', {
|
||||
value: { ...packager.platformOptions, sign: azureConfigFromEnv() }
|
||||
})
|
||||
const manager = new WindowsSignAzureManager(shim)
|
||||
// No-op on the modern signtool /dlib path (winCodeSign >= 1.3.0);
|
||||
// installs the legacy PowerShell module otherwise.
|
||||
await manager.initialize()
|
||||
return manager
|
||||
})()
|
||||
}
|
||||
return managerPromise
|
||||
}
|
||||
|
||||
/**
|
||||
* Sign one file on app-builder-lib's own WindowsSignAzureManager — the exact
|
||||
* path a plain { type: 'azure' } config runs. Used by the sign-msix hook for
|
||||
* the package itself and by batch-sign-binaries.mjs's customSign hook for the
|
||||
* product exe (which must be signed after rcedit, not in the afterPack batch).
|
||||
*
|
||||
* @param {string} file
|
||||
* @param {any} packager WinPackager
|
||||
*/
|
||||
export async function azureSignFile(file, packager) {
|
||||
const mgr = await azureManager(packager)
|
||||
// signFileWithDlib reads only options.path (plus the manager's own
|
||||
// signing config), so platformOptions is sufficient here.
|
||||
await mgr.signFile({ path: file, options: packager.platformOptions })
|
||||
}
|
||||
|
||||
/**
|
||||
* The electron-builder custom sign hook.
|
||||
*
|
||||
* @param {{ path: string }} configuration CustomWindowsSignTaskConfiguration
|
||||
* @param {any} packager WinPackager
|
||||
*/
|
||||
export default async function sign(configuration, packager) {
|
||||
if (path.basename(configuration.path).startsWith(STORE_ARTIFACT_PREFIX)) {
|
||||
console.log(`[sign-msix] skip Store- submission package (Partner Center signs on ingestion): ${configuration.path}`)
|
||||
return
|
||||
}
|
||||
if (!shouldSignFile(configuration.path)) {
|
||||
console.log(`[sign-msix] skip non-MSIX (covered by package block map): ${configuration.path}`)
|
||||
return
|
||||
}
|
||||
|
||||
await azureSignFile(configuration.path, packager)
|
||||
}
|
||||
@@ -0,0 +1,47 @@
|
||||
/**
|
||||
* Stage the pm payload into build/agent-payload for a BUNDLED desktop
|
||||
* build. Runs `hermes pm bundle` with the repo's own venv interpreter —
|
||||
* the payload carries the repo snapshot (committed state), the tool
|
||||
* store + facts, and a relocatable venv on the pinned interpreter.
|
||||
*
|
||||
* Plain `npm run dist` (no payload) still works: before-build.mjs writes
|
||||
* an external:true stub and the app resolves a runtime at first launch.
|
||||
*/
|
||||
import { execFileSync } from 'node:child_process'
|
||||
import fs from 'node:fs'
|
||||
import path from 'node:path'
|
||||
|
||||
const desktopRoot = path.join(import.meta.dirname, '..')
|
||||
const repoRoot = path.join(desktopRoot, '..', '..')
|
||||
const payloadDir = path.join(desktopRoot, 'build', 'agent-payload')
|
||||
|
||||
function venvPython() {
|
||||
for (const venv of ['.venv', 'venv']) {
|
||||
for (const rel of [
|
||||
['Scripts', 'python.exe'],
|
||||
['bin', 'python']
|
||||
]) {
|
||||
const candidate = path.join(repoRoot, venv, ...rel)
|
||||
|
||||
if (fs.existsSync(candidate)) {
|
||||
return candidate
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return null
|
||||
}
|
||||
|
||||
const python = venvPython()
|
||||
|
||||
if (!python) {
|
||||
console.error('stage-payload: no repo venv found — run `uv sync` in the repo root first')
|
||||
process.exit(1)
|
||||
}
|
||||
|
||||
console.log(`stage-payload: ${python} -m pm.cli bundle --out ${payloadDir}`)
|
||||
execFileSync(python, ['-m', 'pm.cli', 'bundle', '--out', payloadDir], {
|
||||
cwd: repoRoot,
|
||||
stdio: 'inherit',
|
||||
env: { ...process.env, PYTHONUTF8: '1' }
|
||||
})
|
||||
@@ -179,7 +179,11 @@ export function BootFailureOverlay() {
|
||||
|
||||
await desktop?.oauthLogoutConnectionConfig?.(remoteReauth.url)
|
||||
|
||||
let result: { connected?: boolean } | undefined
|
||||
// `error` matters here: the oauth arm (oauthLoginConnectionConfig) and
|
||||
// the cloud cascade can both fail with a reason, and the incomplete
|
||||
// sign-in notice below surfaces it. (DesktopCloudAgentSignInResult has
|
||||
// no error field; oauthLoginConnectionConfig does.)
|
||||
let result: { connected?: boolean; error?: string } | undefined
|
||||
|
||||
if (connectionConfig?.mode === 'cloud' && desktop?.cloud) {
|
||||
const status = await desktop.cloud.status()
|
||||
|
||||
@@ -1,15 +0,0 @@
|
||||
# Termux / Android dependency constraints for Hermes Agent.
|
||||
#
|
||||
# Usage:
|
||||
# python -m pip install -e '.[termux]' -c constraints-termux.txt
|
||||
#
|
||||
# These pins keep the tested Android install path stable when upstream packages
|
||||
# move faster than Termux-compatible wheels / sdists.
|
||||
|
||||
ipython<10
|
||||
jedi>=0.18.1,<0.20
|
||||
parso>=0.8.4,<0.9
|
||||
stack-data>=0.6,<0.7
|
||||
pexpect>4.3,<5
|
||||
matplotlib-inline>=0.1.7,<0.2
|
||||
asttokens>=2.1,<3
|
||||
+88
-41
@@ -4037,20 +4037,61 @@ def _get_session_db_timeout() -> float:
|
||||
return 10.0
|
||||
|
||||
|
||||
def _read_windows_pyvenv_cfg(venv_dir: Path) -> dict[str, str]:
|
||||
cfg_path = venv_dir / "pyvenv.cfg"
|
||||
try:
|
||||
lines = cfg_path.read_text(encoding="utf-8-sig").splitlines()
|
||||
except OSError:
|
||||
return {}
|
||||
def _pm_runtime_venv_dir() -> Optional[Path]:
|
||||
"""The venv pm provisioned for this install, resolved from pm's own
|
||||
records (facts.json + store layout) — never from interpreter state.
|
||||
|
||||
parsed: dict[str, str] = {}
|
||||
for raw in lines:
|
||||
if "=" not in raw:
|
||||
continue
|
||||
key, value = raw.split("=", 1)
|
||||
parsed[key.strip().lower()] = value.strip()
|
||||
return parsed
|
||||
Under no-boot-through-venv the scheduler process is the store python
|
||||
(``sys.prefix == sys.base_prefix``) and ``VIRTUAL_ENV`` is unset in
|
||||
bundled installs, so prefix/env probing has nothing to offer. pm is
|
||||
the authority: a bundled install keeps its relocatable venv beside the
|
||||
manifest (a sibling of the store), a dev install syncs the project
|
||||
venv (``venv``/``.venv`` — the same layout
|
||||
``pm.packages.Venv.venv_dir`` materializes). The venv fact must exist:
|
||||
pm only vouches for what it provisioned.
|
||||
"""
|
||||
try:
|
||||
from pm import paths
|
||||
from pm.lock import Facts
|
||||
except Exception:
|
||||
return None
|
||||
try:
|
||||
if not Facts(paths.facts_path()).get("venv"):
|
||||
return None
|
||||
except Exception:
|
||||
return None
|
||||
store = paths.store_root()
|
||||
bundled = store.parent / "venv"
|
||||
if (store.parent / "manifest.json").is_file():
|
||||
return bundled if bundled.is_dir() else None
|
||||
try:
|
||||
from hermes_constants import project_venv_dir
|
||||
except ImportError:
|
||||
return None
|
||||
return project_venv_dir(paths.repo_root())
|
||||
|
||||
|
||||
def _pm_store_python_exe() -> Optional[Path]:
|
||||
"""The base interpreter pm staged for this install (the ``python``
|
||||
fact's store entry), or None. This is the store python — never a venv
|
||||
launcher — so it is the right interpreter to run cron scripts with the
|
||||
venv overlaid via environment (see
|
||||
``_windows_cron_python_invocation``)."""
|
||||
try:
|
||||
from pm import paths
|
||||
from pm.lock import Facts
|
||||
except Exception:
|
||||
return None
|
||||
try:
|
||||
fact = Facts(paths.facts_path()).get("python")
|
||||
if not fact or "entry" not in fact:
|
||||
return None
|
||||
exe = paths.store_root() / fact["entry"] / (
|
||||
"python.exe" if sys.platform == "win32" else "bin/python3"
|
||||
)
|
||||
return exe if exe.is_file() else None
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def _windows_cron_python_invocation(python_exe: str) -> tuple[str, dict[str, str]]:
|
||||
@@ -4059,15 +4100,16 @@ def _windows_cron_python_invocation(python_exe: str) -> tuple[str, dict[str, str
|
||||
Cron scripts capture stdout/stderr, so using ``pythonw.exe`` directly can
|
||||
lose script output. uv-created venv ``python.exe`` launchers are also a
|
||||
problem: even with CREATE_NO_WINDOW, the launcher can re-exec the base
|
||||
console interpreter and flash a visible window. For uv venvs, bypass the
|
||||
launcher and run the base ``python.exe`` directly with the venv paths
|
||||
overlaid in the environment.
|
||||
console interpreter and flash a visible window. When pm has provisioned
|
||||
a runtime venv (facts/store resolution — never pyvenv.cfg, which is
|
||||
dead config under no-boot-through-venv), bypass the launcher and run
|
||||
the pm store python directly with the venv paths overlaid in the
|
||||
environment.
|
||||
"""
|
||||
if sys.platform != "win32":
|
||||
return python_exe, {}
|
||||
|
||||
interpreter = Path(python_exe)
|
||||
venv_dir = interpreter.parent.parent
|
||||
env_overlay: dict[str, str] = {}
|
||||
|
||||
if interpreter.name.lower() == "pythonw.exe":
|
||||
@@ -4075,22 +4117,20 @@ def _windows_cron_python_invocation(python_exe: str) -> tuple[str, dict[str, str
|
||||
if sibling.exists():
|
||||
interpreter = sibling
|
||||
|
||||
cfg = _read_windows_pyvenv_cfg(venv_dir)
|
||||
home = cfg.get("home", "")
|
||||
site_packages = venv_dir / "Lib" / "site-packages"
|
||||
if "uv" in cfg and home:
|
||||
base_python = Path(home) / "python.exe"
|
||||
if base_python.exists() and site_packages.exists():
|
||||
interpreter = base_python
|
||||
env_overlay["VIRTUAL_ENV"] = str(venv_dir)
|
||||
pythonpath_entries = [
|
||||
str(Path(__file__).resolve().parents[1]),
|
||||
str(site_packages),
|
||||
]
|
||||
existing_pythonpath = os.environ.get("PYTHONPATH", "")
|
||||
if existing_pythonpath:
|
||||
pythonpath_entries.append(existing_pythonpath)
|
||||
env_overlay["PYTHONPATH"] = os.pathsep.join(pythonpath_entries)
|
||||
venv_dir = _pm_runtime_venv_dir()
|
||||
base_python = _pm_store_python_exe()
|
||||
site_packages = venv_dir / "Lib" / "site-packages" if venv_dir else None
|
||||
if base_python is not None and site_packages is not None and site_packages.exists():
|
||||
interpreter = base_python
|
||||
env_overlay["VIRTUAL_ENV"] = str(venv_dir)
|
||||
pythonpath_entries = [
|
||||
str(Path(__file__).resolve().parents[1]),
|
||||
str(site_packages),
|
||||
]
|
||||
existing_pythonpath = os.environ.get("PYTHONPATH", "")
|
||||
if existing_pythonpath:
|
||||
pythonpath_entries.append(existing_pythonpath)
|
||||
env_overlay["PYTHONPATH"] = os.pathsep.join(pythonpath_entries)
|
||||
|
||||
return str(interpreter), env_overlay
|
||||
|
||||
@@ -4248,8 +4288,10 @@ def _windows_cron_bootstrap_argv(
|
||||
the pre-existing PYTHONPATH behaviour is strictly better than failing
|
||||
to run at all.
|
||||
"""
|
||||
site_packages = Path(env_overlay.get("VIRTUAL_ENV", "")) / "Lib" / "site-packages"
|
||||
if not site_packages.is_dir():
|
||||
venv_value = env_overlay.get("VIRTUAL_ENV", "")
|
||||
venv_dir = Path(venv_value) if venv_value else _pm_runtime_venv_dir()
|
||||
site_packages = venv_dir / "Lib" / "site-packages" if venv_dir else None
|
||||
if site_packages is None or not site_packages.is_dir():
|
||||
# Silent here would make the "editable installs invisible" failure
|
||||
# undiagnosable; the pre-existing PYTHONPATH-only behaviour applies.
|
||||
logger.warning(
|
||||
@@ -4364,14 +4406,19 @@ def _run_job_script(
|
||||
# choice explicit here keeps the allowed surface small and auditable.
|
||||
suffix = path.suffix.lower()
|
||||
if suffix in {".sh", ".bash"}:
|
||||
# Resolve bash dynamically so Windows (Git Bash) and Linux/macOS
|
||||
# all work. On native Windows without Git for Windows installed
|
||||
# shutil.which returns None — fall back to a clear error rather
|
||||
# Resolve bash dynamically so Windows (Git for Windows) and
|
||||
# Linux/macOS all work — through pm.shell, the one bash authority
|
||||
# (it prefers conventional installs over MSIX-package aliases,
|
||||
# which only spawn inside their package context). On native
|
||||
# Windows without any bash — fall back to a clear error rather
|
||||
# than a FileNotFoundError with a confusing "[WinError 2]"
|
||||
# traceback.
|
||||
_bash = shutil.which("bash") or (
|
||||
"/bin/bash" if os.path.isfile("/bin/bash") else None
|
||||
)
|
||||
try:
|
||||
from pm.shell import bash as _pm_bash
|
||||
|
||||
_bash = _pm_bash()
|
||||
except Exception:
|
||||
_bash = None
|
||||
if _bash is None:
|
||||
return False, (
|
||||
f"Cannot run .sh/.bash script {path.name!r}: bash not found on PATH. "
|
||||
|
||||
+22
-42
@@ -247,7 +247,7 @@ if [ "$needs_chown" = true ]; then
|
||||
# Hermes-owned subdirs: recursive chown is safe here because these are
|
||||
# created and managed exclusively by hermes (see the s6-setuidgid mkdir
|
||||
# -p block below for the canonical list).
|
||||
for sub in cron sessions logs hooks memories skills skins plans workspace home profiles pairing platforms/pairing lazy-packages; do
|
||||
for sub in cron sessions logs hooks memories skills skins plans workspace home profiles pairing platforms/pairing; do
|
||||
if [ -e "$HERMES_HOME/$sub" ] && tree_has_non_hermes_owner "$HERMES_HOME/$sub"; then
|
||||
chown_hermes_tree "$HERMES_HOME/$sub"
|
||||
fi
|
||||
@@ -262,16 +262,8 @@ fi
|
||||
# non-writable prevents an agent session from self-modifying the installed
|
||||
# source, venv, TUI bundle, or node_modules and bricking the gateway.
|
||||
#
|
||||
# Lazy-installable optional backends (Firecrawl, Exa, Feishu, etc.) cannot
|
||||
# install into the sealed venv, so they are redirected to the writable
|
||||
# $HERMES_HOME/lazy-packages dir on the data volume (Dockerfile sets
|
||||
# HERMES_LAZY_INSTALL_TARGET). That dir is appended to the END of sys.path,
|
||||
# so a package installed there can only ADD modules — it can never shadow or
|
||||
# break a core module, which is what keeps the sealed-venv guarantee intact
|
||||
# even though installs are re-enabled. The dir is seeded + chowned to hermes
|
||||
# in the mkdir/chown blocks above so first-use installs succeed as the
|
||||
# unprivileged runtime user, and it persists across container recreates /
|
||||
# image updates (an ABI stamp wipes it if a rebuild bumps the interpreter).
|
||||
# Lazy installs are fully disabled at runtime (HERMES_DISABLE_LAZY_INSTALLS=1,
|
||||
# see the Dockerfile), so no writable lazy-install target is provisioned here.
|
||||
|
||||
# Always reset ownership of $HERMES_HOME/profiles to hermes on every
|
||||
# boot. Profile dirs and files can land owned by root when commands
|
||||
@@ -392,8 +384,7 @@ as_hermes mkdir -p \
|
||||
"$HERMES_HOME/workspace" \
|
||||
"$HERMES_HOME/home" \
|
||||
"$HERMES_HOME/pairing" \
|
||||
"$HERMES_HOME/platforms/pairing" \
|
||||
"$HERMES_HOME/lazy-packages"
|
||||
"$HERMES_HOME/platforms/pairing"
|
||||
|
||||
# --- Install-method stamp ---
|
||||
# The 'docker' stamp is baked into the immutable install tree at
|
||||
@@ -646,42 +637,31 @@ if [ -d "$INSTALL_DIR/skills" ]; then
|
||||
|| echo "[stage2] Warning: skills_sync.py failed; continuing"
|
||||
fi
|
||||
|
||||
# --- Discover agent-browser's Chromium binary ---
|
||||
# The image's Dockerfile runs `npx playwright install chromium`, which
|
||||
# populates ``$PLAYWRIGHT_BROWSERS_PATH`` (=/opt/hermes/.playwright) with
|
||||
# a ``chromium_headless_shell-<build>/chrome-headless-shell-linux64/``
|
||||
# directory. agent-browser (the runtime CLI Hermes spawns for the
|
||||
# browser tool) doesn't recognise this layout in its own cache scan and
|
||||
# fails with "Auto-launch failed: Chrome not found" — even though the
|
||||
# binary is right there (#15697).
|
||||
# --- Point agent-browser at the pinned Chromium binary ---
|
||||
# The image's Dockerfile pm-provisions the pinned Chromium pair into
|
||||
# $HERMES_RUNTIME_DIR (/opt/hermes/tools) at BUILD time and bakes the
|
||||
# resolved browser binary path into /etc/hermes/agent-browser-executable-path
|
||||
# (the layout differs per arch — chrome-linux64/chrome on amd64,
|
||||
# chromium-linux-arm64/chromium on arm64 — so it is resolved at build time,
|
||||
# not hard-coded). agent-browser (the runtime CLI Hermes spawns for the
|
||||
# browser tool) doesn't recognise Playwright's directory layout in its own
|
||||
# cache scan and fails with "Auto-launch failed: Chrome not found" — even
|
||||
# though the binary is right there (#15697).
|
||||
#
|
||||
# Fix: locate the binary at boot and export ``AGENT_BROWSER_EXECUTABLE_PATH``
|
||||
# Fix: read the baked path and export ``AGENT_BROWSER_EXECUTABLE_PATH``
|
||||
# via /run/s6/container_environment so the `with-contenv` shebang on
|
||||
# main-wrapper.sh propagates it into the supervised ``hermes`` process
|
||||
# and thence to agent-browser subprocesses.
|
||||
#
|
||||
# - Skipped when the user has already set ``AGENT_BROWSER_EXECUTABLE_PATH``
|
||||
# (lets users override with a system Chrome install).
|
||||
# - Filename-matched (not path-matched): the chromium dir contains many
|
||||
# shared libraries (libGLESv2.so, libEGL.so, ...) which inherit the
|
||||
# executable bit from Playwright's tarball but are NOT browser binaries.
|
||||
# We only accept files whose basename is chrome / chromium /
|
||||
# chrome-headless-shell / headless_shell / chromium-browser. Compare
|
||||
# PR #18635's earlier ``find | grep -Ei 'chrome|chromium'`` which would
|
||||
# match the path ``.../chrome-headless-shell-linux64/libGLESv2.so`` and
|
||||
# pick a .so.
|
||||
# - Quietly skipped when $PLAYWRIGHT_BROWSERS_PATH doesn't exist (e.g.
|
||||
# custom builds that strip Playwright).
|
||||
# - Quietly skipped when the baked path file is absent (e.g. custom builds
|
||||
# that strip the pm tool store).
|
||||
if [ -z "${AGENT_BROWSER_EXECUTABLE_PATH:-}" ] && \
|
||||
[ -n "${PLAYWRIGHT_BROWSERS_PATH:-}" ] && \
|
||||
[ -d "$PLAYWRIGHT_BROWSERS_PATH" ]; then
|
||||
browser_bin=$(find "$PLAYWRIGHT_BROWSERS_PATH" -type f -executable \
|
||||
\( -name 'chrome' -o -name 'chromium' \
|
||||
-o -name 'chrome-headless-shell' -o -name 'headless_shell' \
|
||||
-o -name 'chromium-browser' \) \
|
||||
2>/dev/null | head -n 1)
|
||||
if [ -n "$browser_bin" ]; then
|
||||
echo "[stage2] Found agent-browser Chromium binary: $browser_bin"
|
||||
[ -f /etc/hermes/agent-browser-executable-path ]; then
|
||||
browser_bin="$(cat /etc/hermes/agent-browser-executable-path)"
|
||||
if [ -n "$browser_bin" ] && [ -x "$browser_bin" ]; then
|
||||
echo "[stage2] Using pinned agent-browser Chromium binary: $browser_bin"
|
||||
# Write to s6's container_environment so with-contenv picks it
|
||||
# up for all supervised services (main-hermes, dashboard, etc.).
|
||||
# Idempotent: each boot overwrites with the current path.
|
||||
@@ -690,7 +670,7 @@ if [ -z "${AGENT_BROWSER_EXECUTABLE_PATH:-}" ] && \
|
||||
mkdir -p /run/s6/container_environment
|
||||
printf '%s' "$browser_bin" > /run/s6/container_environment/AGENT_BROWSER_EXECUTABLE_PATH
|
||||
else
|
||||
echo "[stage2] Warning: no Chromium binary under $PLAYWRIGHT_BROWSERS_PATH; browser tool may fail"
|
||||
echo "[stage2] Warning: baked Chromium binary is missing (${browser_bin:-<empty>}); browser tool may fail"
|
||||
fi
|
||||
fi
|
||||
|
||||
|
||||
@@ -87,7 +87,7 @@ class PlatformEntry:
|
||||
validate_config: Optional[Callable[[Any], bool]] = None
|
||||
|
||||
# ACTIVE dependency installer: make the platform's dependencies available,
|
||||
# installing them (pip / lazy_deps) if needed. Returns True once deps are
|
||||
# installing them (pm.sync_venv) if needed. Returns True once deps are
|
||||
# importable, False if they could not be installed. Called by
|
||||
# ``create_adapter()`` when ``check_fn`` returns False — i.e. exactly at
|
||||
# the moment the gateway is about to bring the platform up and the user
|
||||
|
||||
@@ -455,7 +455,7 @@ class WebSocketRelayTransport:
|
||||
if not WEBSOCKETS_AVAILABLE:
|
||||
raise RuntimeError(
|
||||
"WebSocketRelayTransport requires the 'websockets' package "
|
||||
"(install the messaging extra)."
|
||||
"(a hermes-agent core dependency)."
|
||||
)
|
||||
self._url = _ws_dial_url(url)
|
||||
self._platform = platform
|
||||
|
||||
+36
-1
@@ -32272,7 +32272,42 @@ def main():
|
||||
with open(args.config, encoding="utf-8") as f:
|
||||
data = yaml.safe_load(f) or {}
|
||||
config = GatewayConfig.from_dict(data)
|
||||
|
||||
|
||||
|
||||
# Post-update boot bootstrap: when this install's code changed since the
|
||||
# last boot (app-updater swap, docker/nix image change, git pull without
|
||||
# `hermes update`), run the idempotent user-state steps (config
|
||||
# migration, skills sync, state.db guard) before platforms connect.
|
||||
# Two file reads when nothing changed; never raises.
|
||||
try:
|
||||
from pathlib import Path as _Path
|
||||
|
||||
from hermes_cli.boot_bootstrap import maybe_run_boot_bootstrap
|
||||
|
||||
maybe_run_boot_bootstrap(_Path(__file__).resolve().parents[1])
|
||||
except Exception as exc:
|
||||
# maybe_run_boot_bootstrap itself never raises and logs internally;
|
||||
# this guard covers the import/lookup path. Don't hide it silently.
|
||||
logger.warning("boot bootstrap setup failed (continuing boot): %s", exc)
|
||||
|
||||
# pm startup: same contract as the CLI dispatch path (hermes_cli/main.py)
|
||||
# — the gateway daemon never passes through CLI dispatch, so without this
|
||||
# it would boot without the store's tools (git/bash/ffmpeg/...) on PATH.
|
||||
# O(1) stamp checks, no network, no installs; warns, never blocks.
|
||||
try:
|
||||
import pm
|
||||
|
||||
pm.adopt()
|
||||
problems = pm.check()
|
||||
if problems:
|
||||
logger.warning(
|
||||
"install out of sync (%s) — run `hermes pm install`",
|
||||
"; ".join(problems),
|
||||
)
|
||||
else:
|
||||
pm.activate()
|
||||
except Exception:
|
||||
logger.debug("pm startup check failed", exc_info=True)
|
||||
# start_gateway() performs the full graceful teardown (adapters
|
||||
# disconnected, sessions saved + flushed, SQLite closed, cron/MCP stopped,
|
||||
# PID file + runtime lock released) before it returns OR raises SystemExit
|
||||
|
||||
@@ -201,39 +201,9 @@ def harden_import_path(src_root: str | None = None) -> None:
|
||||
sys.path.insert(0, root)
|
||||
|
||||
|
||||
def activate_durable_lazy_target() -> None:
|
||||
"""Put the durable lazy-install dir on ``sys.path`` if one is configured.
|
||||
|
||||
On immutable Docker images the agent venv is sealed and lazy installs
|
||||
are redirected to a writable dir on the data volume
|
||||
(``HERMES_LAZY_INSTALL_TARGET``, e.g. ``/opt/data/lazy-packages``).
|
||||
Packages installed there on a previous run must be importable on this
|
||||
run, so we activate the dir here — at the very first import, before any
|
||||
backend module imports its SDK.
|
||||
|
||||
The activation appends to the END of ``sys.path`` so the core venv
|
||||
always wins name collisions (see ``tools.lazy_deps`` for the full
|
||||
security rationale). Never raises; a missing/empty target is a no-op.
|
||||
"""
|
||||
if not os.environ.get("HERMES_LAZY_INSTALL_TARGET", "").strip():
|
||||
return
|
||||
try:
|
||||
from tools import lazy_deps
|
||||
lazy_deps.activate_durable_lazy_target()
|
||||
except Exception:
|
||||
# Bootstrap must never crash an entry point. If activation fails the
|
||||
# backend simply reports itself unavailable, exactly as before.
|
||||
pass
|
||||
|
||||
|
||||
# Apply on import — entry points just need ``import hermes_bootstrap``
|
||||
# (or ``from hermes_bootstrap import apply_windows_utf8_bootstrap``) at
|
||||
# the very top of their module, before importing anything else. The
|
||||
# import side effect does the right thing.
|
||||
apply_windows_utf8_bootstrap()
|
||||
suppress_platform_ver_console()
|
||||
|
||||
# Activate the durable lazy-install target (immutable Docker images) so
|
||||
# packages installed into the data volume on a previous run are importable
|
||||
# this run, before any backend module imports its SDK. No-op when unset.
|
||||
activate_durable_lazy_target()
|
||||
+66
-102
@@ -11,9 +11,9 @@ Both callers need to run the same core ``.[all]`` reinstall:
|
||||
|
||||
This module is deliberately **stdlib-only** so importing it can never fail in
|
||||
the corrupted-venv state it exists to repair. ``hermes_cli.main`` imports
|
||||
``managed_uv``, ``hermes_constants``, and friends only in its late path; the
|
||||
early path must not. Where the late path uses ``managed_uv.ensure_uv`` to
|
||||
bootstrap uv if missing, the early path uses the stdlib
|
||||
``pm``, ``hermes_constants``, and friends only in its late path; the
|
||||
early path must not. Where the late path uses ``pm.uv()`` to
|
||||
realize uv if missing, the early path uses the stdlib
|
||||
:func:`hermes_cli._early_recovery._find_uv_binary` lookup and falls back to
|
||||
plain pip when uv is absent — a degraded but working installer (the late
|
||||
recovery will bootstrap uv on the next launch if it ever matters).
|
||||
@@ -24,7 +24,6 @@ from __future__ import annotations
|
||||
import contextlib
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
@@ -39,19 +38,6 @@ def _is_windows() -> bool:
|
||||
return sys.platform == "win32"
|
||||
|
||||
|
||||
def _is_termux_env(env: dict | None = None) -> bool:
|
||||
"""Stdlib Termux probe (hermes_cli.main's version lives behind imports)."""
|
||||
env = env if env is not None else os.environ
|
||||
try:
|
||||
if env.get("TERMUX_VERSION"):
|
||||
return True
|
||||
prefix = env.get("PREFIX", "")
|
||||
return "com.termux" in prefix
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def _stdout_to_stderr():
|
||||
"""Route fd 1 (and sys.stdout) to stderr for the duration of an install.
|
||||
|
||||
@@ -85,20 +71,14 @@ def _stdout_to_stderr():
|
||||
def _resolve_install_target(root: Path) -> tuple[list[str], dict | None]:
|
||||
"""(install_cmd_prefix, env) for the project venv — stdlib uv lookup.
|
||||
|
||||
Mirrors ``main.py::_default_venv_install_target`` but without
|
||||
``managed_uv``. ``VIRTUAL_ENV`` steers ``uv pip`` at the project venv even
|
||||
Mirrors ``main.py::_default_venv_install_target`` but without ``pm``. ``VIRTUAL_ENV`` steers ``uv pip`` at the project venv even
|
||||
when invoked from the base interpreter (the early-recovery case).
|
||||
Termux strips leaked interpreter-path env vars so uv resolves the venv
|
||||
correctly.
|
||||
"""
|
||||
uv_bin = _er._find_uv_binary()
|
||||
if uv_bin:
|
||||
from hermes_constants import project_venv_dir
|
||||
|
||||
env = {**os.environ, "VIRTUAL_ENV": str(project_venv_dir(root) or root / "venv")}
|
||||
if _is_termux_env(env):
|
||||
env.pop("PYTHONPATH", None)
|
||||
env.pop("PYTHONHOME", None)
|
||||
return [uv_bin, "pip"], env
|
||||
return [sys.executable, "-m", "pip"], None
|
||||
|
||||
@@ -120,36 +100,11 @@ def _venv_scripts_dir(root: Path) -> Path | None:
|
||||
|
||||
#: Launcher command names install.ps1's Set-PathVariable exposes from the
|
||||
#: managed binary dir (the default Hermes root's ``bin``, next to uv.exe)
|
||||
#: on the user PATH. Keep in lockstep with the launcher list in
|
||||
#: scripts/install.ps1.
|
||||
#: on the user PATH. Keep in lockstep with WINDOWS_BIN_LAUNCHERS in
|
||||
#: hermes_cli/_launchers.py and scripts/install.ps1.
|
||||
_WINDOWS_BIN_LAUNCHERS = ("hermes", "hermes-acp")
|
||||
|
||||
|
||||
def _venv_is_relocatable(venv_dir: Path) -> bool:
|
||||
"""True when the venv's pyvenv.cfg declares ``relocatable = true``.
|
||||
|
||||
uv writes the flag; ``hermes_cli.managed_uv`` builds its replacement
|
||||
venvs with ``--relocatable`` (they are constructed aside and swapped
|
||||
into place). A relocatable venv's console-script trampolines embed a
|
||||
RELATIVE interpreter reference, so a COPY of one placed outside
|
||||
``venv\\Scripts`` fails at run time with ``uv trampoline failed to
|
||||
canonicalize script path``. Non-relocatable venvs (fresh installs)
|
||||
embed the absolute interpreter path and their trampolines survive
|
||||
copying. This flag decides which launcher form a PATH dir gets.
|
||||
"""
|
||||
try:
|
||||
cfg = (Path(venv_dir) / "pyvenv.cfg").read_text(
|
||||
encoding="utf-8", errors="replace"
|
||||
)
|
||||
except OSError:
|
||||
return False
|
||||
for line in cfg.splitlines():
|
||||
key, _, value = line.partition("=")
|
||||
if key.strip().lower() == "relocatable" and value.strip().lower() == "true":
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _normalize_windows_path(value) -> str:
|
||||
"""Windows path equality key: backslashes, no trailing separator, lowered.
|
||||
|
||||
@@ -185,11 +140,15 @@ def ensure_windows_bin_launchers(
|
||||
windows: bool | None = None,
|
||||
user_path_entries: list[str] | None = None,
|
||||
) -> list[str]:
|
||||
"""Re-stage the Windows ``hermes`` launchers when they vanish.
|
||||
"""Re-stage the Windows ``hermes`` launchers when they vanish or when
|
||||
they still boot through the venv.
|
||||
|
||||
On Windows, ``hermes`` resolves through launchers derived from the venv
|
||||
console scripts — never ``venv\\Scripts`` itself on PATH, which would
|
||||
shadow the user's ``python`` (#83797). The canonical launcher home is
|
||||
On Windows, ``hermes`` resolves through staged launchers — never
|
||||
``venv\\Scripts`` itself on PATH, which would shadow the user's
|
||||
``python`` (#83797) — and under pm the launchers boot the pm STORE
|
||||
python with ``PYTHONPATH=<repo>;<venv>/site-packages``, never the venv
|
||||
interpreter (no-boot-through-venv; ``pyvenv.cfg`` is inert dead
|
||||
config). The canonical launcher home is
|
||||
the managed binary dir — the default Hermes root's ``bin``
|
||||
(``%LOCALAPPDATA%\\hermes\\bin``, next to the managed uv) — which lives
|
||||
OUTSIDE the git checkout so no git operation can ever touch it. It is
|
||||
@@ -205,15 +164,10 @@ def ensure_windows_bin_launchers(
|
||||
terminal. That legacy location is re-staged too, during the transition,
|
||||
for installs whose user PATH still resolves through it.
|
||||
|
||||
The launcher FORM depends on the venv (see :func:`_venv_is_relocatable`):
|
||||
a normal venv's exe trampoline embeds an absolute interpreter path and
|
||||
survives copying, so it is copied as ``<name>.exe``; a relocatable
|
||||
venv's trampoline resolves relative to its own location and a copy
|
||||
dies with ``uv trampoline failed to canonicalize script path``, so a
|
||||
``<name>.cmd`` delegator invoking the in-venv exe by absolute path is
|
||||
written instead. A name counts as present when EITHER form exists —
|
||||
exe copies staged before a venv rebuild keep working (they embed the
|
||||
swapped-in-place venv's absolute path) and are left alone.
|
||||
A name counts as present when an exe exists that does NOT boot the
|
||||
venv interpreter — legacy copied-venv trampolines (detected by their
|
||||
embedded interpreter path) and placeholder .cmd delegators are replaced
|
||||
with a store-python launcher as soon as one can be minted.
|
||||
|
||||
Two targets, two gates, both failing toward inaction:
|
||||
|
||||
@@ -251,6 +205,36 @@ def ensure_windows_bin_launchers(
|
||||
def _launcher_present(target: Path, name: str) -> bool:
|
||||
return (target / f"{name}.exe").exists() or (target / f"{name}.cmd").exists()
|
||||
|
||||
# Launchers boot the pm STORE python with PYTHONPATH=repo;site-packages
|
||||
# — never the venv interpreter (no boot through the venv; pyvenv.cfg is
|
||||
# inert dead config). mint_launcher prefers a distlib exe trampoline
|
||||
# bound to the store python; a runtime-resolving .cmd is written when
|
||||
# the store has not materialized a python yet, and the repair upgrades
|
||||
# it (and any legacy copied-venv trampoline) to the exe once the store
|
||||
# python exists. See hermes_cli/_launchers.py.
|
||||
from hermes_cli._launchers import (
|
||||
exe_is_venv_bound,
|
||||
mint_launcher,
|
||||
resolve_store_python,
|
||||
stage_launcher,
|
||||
venv_site_packages,
|
||||
)
|
||||
|
||||
from hermes_constants import project_venv_dir
|
||||
|
||||
venv_dir = project_venv_dir(root)
|
||||
site_packages = venv_site_packages(venv_dir) if venv_dir else None
|
||||
|
||||
store_python = resolve_store_python(root)
|
||||
|
||||
def _needs_attention(target: Path, name: str) -> bool:
|
||||
"""Missing, a placeholder .cmd, or a launcher that still boots the
|
||||
venv interpreter — anything the store-python launcher should replace."""
|
||||
exe = target / f"{name}.exe"
|
||||
if not exe.exists():
|
||||
return True
|
||||
return exe_is_venv_bound(exe, venv_dir)
|
||||
|
||||
targets: list[Path] = []
|
||||
|
||||
# Canonical target — gate on the managed-clone shape. This runs at
|
||||
@@ -258,7 +242,9 @@ def ensure_windows_bin_launchers(
|
||||
# override), so the healthy path must stay at a couple of stat calls.
|
||||
if _normalize_windows_path(root.parent) == _normalize_windows_path(home):
|
||||
canonical = home / "bin"
|
||||
if any(not _launcher_present(canonical, name) for name in _WINDOWS_BIN_LAUNCHERS):
|
||||
if any(
|
||||
_needs_attention(canonical, name) for name in _WINDOWS_BIN_LAUNCHERS
|
||||
):
|
||||
targets.append(canonical)
|
||||
|
||||
# Legacy transition target — the pre-migration in-checkout dir. Only
|
||||
@@ -268,7 +254,7 @@ def ensure_windows_bin_launchers(
|
||||
# network shares. An entry stored some other way (8.3 short path,
|
||||
# subst drive) misses the re-stage, which fails safe: no-op.
|
||||
legacy = root / "bin"
|
||||
if any(not _launcher_present(legacy, name) for name in _WINDOWS_BIN_LAUNCHERS):
|
||||
if any(_needs_attention(legacy, name) for name in _WINDOWS_BIN_LAUNCHERS):
|
||||
if user_path_entries is None:
|
||||
user_path_entries = _windows_user_path_entries()
|
||||
configured = {_normalize_windows_path(entry) for entry in user_path_entries}
|
||||
@@ -278,44 +264,23 @@ def ensure_windows_bin_launchers(
|
||||
if not targets:
|
||||
return []
|
||||
|
||||
from hermes_constants import project_venv_dir, venv_bin_dir
|
||||
|
||||
venv_dir = project_venv_dir(root)
|
||||
if venv_dir is None:
|
||||
return []
|
||||
scripts_dir = venv_bin_dir(venv_dir, windows=windows)
|
||||
sources = [
|
||||
(name, scripts_dir / f"{name}.exe")
|
||||
for name in _WINDOWS_BIN_LAUNCHERS
|
||||
if (scripts_dir / f"{name}.exe").is_file()
|
||||
]
|
||||
if not sources:
|
||||
return []
|
||||
relocatable = _venv_is_relocatable(venv_dir)
|
||||
|
||||
restored: list[str] = []
|
||||
for target in targets:
|
||||
try:
|
||||
target.mkdir(parents=True, exist_ok=True)
|
||||
except OSError:
|
||||
continue
|
||||
for name, source in sources:
|
||||
if _launcher_present(target, name):
|
||||
for name in _WINDOWS_BIN_LAUNCHERS:
|
||||
if not _needs_attention(target, name):
|
||||
# Already a store-python launcher (or a form this heal does
|
||||
# not understand but that does not boot the venv): leave it.
|
||||
continue
|
||||
final = target / (f"{name}.cmd" if relocatable else f"{name}.exe")
|
||||
staging = target / f"{final.name}.heal.{os.getpid()}"
|
||||
try:
|
||||
if relocatable:
|
||||
staging.write_text(
|
||||
"@echo off\r\n" f'"{source}" %*\r\n', encoding="ascii"
|
||||
)
|
||||
else:
|
||||
shutil.copy2(source, staging)
|
||||
os.replace(staging, final)
|
||||
if store_python is not None:
|
||||
final = mint_launcher(name, root, target, store_python, site_packages)
|
||||
else:
|
||||
final = stage_launcher(name, root, target)
|
||||
if final is not None:
|
||||
restored.append(str(final))
|
||||
except OSError:
|
||||
with contextlib.suppress(OSError):
|
||||
staging.unlink()
|
||||
if restored:
|
||||
# Guarded like everything else in this never-raises helper: a
|
||||
# closed/broken stderr must not turn a successful heal into a crash.
|
||||
@@ -573,7 +538,7 @@ def _run_install_cmd(cmd: list[str], *, env: dict | None, root: Path) -> None:
|
||||
|
||||
|
||||
def _load_installable_optional_extras(root: Path, group: str) -> list[str]:
|
||||
"""Optional extras referenced by a dependency group (all / termux-all)."""
|
||||
"""Optional extras referenced by a dependency group (all)."""
|
||||
try:
|
||||
import tomllib
|
||||
|
||||
@@ -604,17 +569,16 @@ def run_core_install(root: Path) -> None:
|
||||
pip module at all)
|
||||
- prefer ``uv pip`` with VIRTUAL_ENV pointed at the project venv; fall back
|
||||
to ``python -m pip`` when no uv binary is available
|
||||
- target ``.[all]`` (or ``.[termux-all]`` on Termux) with the per-extra
|
||||
fallback ladder when the combined extras resolve fails
|
||||
- target ``.[all]`` with the per-extra fallback ladder when the combined
|
||||
extras resolve fails
|
||||
- quarantine live ``hermes*.exe`` shims on Windows so they can be replaced
|
||||
- route ALL install output to stderr (acp/JSON-RPC safety)
|
||||
- Termux strips leaked PYTHONPATH/PYTHONHOME from the uv env
|
||||
|
||||
Raises ``subprocess.CalledProcessError`` when even the base install fails;
|
||||
callers own marker lifecycle (clear on success, keep on failure).
|
||||
"""
|
||||
prefix, env = _resolve_install_target(root)
|
||||
group = "termux-all" if _is_termux_env(env) else "all"
|
||||
group = "all"
|
||||
|
||||
with _stdout_to_stderr():
|
||||
try:
|
||||
@@ -676,7 +640,7 @@ def bump_marker_attempts(marker_path: Path) -> int:
|
||||
"""
|
||||
attempts = 0
|
||||
try:
|
||||
raw = marker_path.read_text(encoding="utf-8", errors="replace").strip()
|
||||
raw = marker_path.read_text(encoding="utf-8-sig", errors="replace").strip()
|
||||
if raw:
|
||||
try:
|
||||
attempts = int(json.loads(raw).get("attempts", 0))
|
||||
|
||||
@@ -0,0 +1,325 @@
|
||||
"""Windows launcher staging: `hermes` / `hermes-acp` launchers that boot the
|
||||
pm STORE python with ``PYTHONPATH=<repo>;<venv>/site-packages`` — never
|
||||
``venv\\Scripts\\python.exe`` (pm work item 3, "no boot through the venv";
|
||||
``pyvenv.cfg`` is inert dead config).
|
||||
|
||||
Two consumers:
|
||||
|
||||
- ``scripts/install.ps1`` Stage-Path (bootstrap) calls
|
||||
:func:`ensure_install_launchers` through the venv python — install-time
|
||||
use of the venv interpreter is fine; it is the materializer, not a boot
|
||||
path. On a fresh install the pm store interpreter does not exist yet, so
|
||||
a runtime-resolving ``.cmd`` delegator is staged and
|
||||
``hermes_cli._install_repair.ensure_windows_bin_launchers`` upgrades it
|
||||
to an exe once ``hermes pm install`` lands the store python.
|
||||
- ``hermes_cli/_install_repair.py`` calls the same machinery per-name when
|
||||
a launcher is missing or still boots through the venv.
|
||||
|
||||
Store-root resolution mirrors ``pm/paths.py`` (HERMES_RUNTIME_DIR →
|
||||
install-stamp.json ``runtimeDir`` → the pm default root); it is re-implemented
|
||||
here because this module must stay importable from a bare interpreter
|
||||
(stdlib + distlib only) during repair/bootstrap where ``pm`` imports are not
|
||||
guaranteed. If pm's layout changes, keep the two in lockstep.
|
||||
|
||||
The exe form uses distlib's ScriptMaker with a customized script template
|
||||
that inserts the repo root and the venv's site-packages into ``sys.path``
|
||||
before importing the entry point — distlib is taken from pip's vendored
|
||||
copy when the standalone package is absent (uv-synced venvs carry pip but
|
||||
rarely standalone distlib). When distlib is unavailable, a ``.cmd``
|
||||
delegator carrying the same PYTHONPATH composition is written instead.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
#: Launcher command names — keep in lockstep with scripts/install.ps1
|
||||
#: Stage-Path and hermes_cli/_install_repair.py.
|
||||
WINDOWS_BIN_LAUNCHERS = ("hermes", "hermes-acp")
|
||||
|
||||
#: command name -> (entry module, callable) — mirrors pyproject.toml
|
||||
#: [project.scripts].
|
||||
ENTRY_POINTS = {
|
||||
"hermes": ("hermes_cli.main", "main"),
|
||||
"hermes-acp": ("acp_adapter.entry", "main"),
|
||||
}
|
||||
|
||||
|
||||
def _is_windows() -> bool:
|
||||
return os.name == "nt"
|
||||
|
||||
|
||||
def project_venv_dir(repo_root: Path) -> Path | None:
|
||||
"""The uv-sync target venv of the install (``venv`` preferred, ``.venv``
|
||||
for dev checkouts)."""
|
||||
for name in ("venv", ".venv"):
|
||||
candidate = Path(repo_root) / name
|
||||
if candidate.is_dir():
|
||||
return candidate
|
||||
return None
|
||||
|
||||
|
||||
def store_root(repo_root: Path) -> Path:
|
||||
"""The pm byte store, resolved exactly like pm/paths.store_root()."""
|
||||
env = os.environ.get("HERMES_RUNTIME_DIR")
|
||||
if env:
|
||||
return Path(env)
|
||||
stamp = Path(repo_root) / "install-stamp.json"
|
||||
if stamp.is_file():
|
||||
try:
|
||||
runtime_dir = json.loads(stamp.read_text(encoding="utf-8-sig")).get(
|
||||
"runtimeDir"
|
||||
)
|
||||
except (OSError, ValueError):
|
||||
runtime_dir = None
|
||||
if runtime_dir:
|
||||
return Path(runtime_dir)
|
||||
from hermes_constants import get_default_hermes_root
|
||||
|
||||
return Path(get_default_hermes_root()) / "tools"
|
||||
|
||||
|
||||
def resolve_store_python(repo_root: Path) -> Path | None:
|
||||
"""The interpreter the store installed from the ``python`` package
|
||||
(facts.json entry first, newest ``python-*`` entry as fallback), or
|
||||
None when `hermes pm install` has not materialized one yet."""
|
||||
runtime = store_root(repo_root)
|
||||
rel = "python.exe" if _is_windows() else "bin/python3"
|
||||
|
||||
facts = runtime / "facts.json"
|
||||
if facts.is_file():
|
||||
try:
|
||||
packages = json.loads(facts.read_text(encoding="utf-8-sig")).get(
|
||||
"packages", {}
|
||||
)
|
||||
entry = (packages.get("python") or {}).get("entry")
|
||||
except (OSError, ValueError):
|
||||
entry = None
|
||||
if entry:
|
||||
candidate = runtime / entry / rel
|
||||
if candidate.is_file():
|
||||
return candidate
|
||||
|
||||
for entry_dir in sorted(runtime.glob("python-*"), key=lambda p: p.name):
|
||||
candidate = entry_dir / rel
|
||||
if candidate.is_file():
|
||||
return candidate
|
||||
return None
|
||||
|
||||
|
||||
def venv_site_packages(venv_dir: Path) -> Path | None:
|
||||
"""The venv directory uv sync fills (Windows ``Lib\\site-packages``;
|
||||
POSIX ``lib/python3.X/site-packages``)."""
|
||||
venv_dir = Path(venv_dir)
|
||||
if _is_windows():
|
||||
candidate = venv_dir / "Lib" / "site-packages"
|
||||
return candidate if candidate.is_dir() else None
|
||||
for candidate in sorted(venv_dir.glob("lib/python3.*/site-packages")):
|
||||
if candidate.is_dir():
|
||||
return candidate
|
||||
return None
|
||||
|
||||
|
||||
def _load_script_maker():
|
||||
"""distlib's ScriptMaker — standalone first, then pip's vendored copy."""
|
||||
try:
|
||||
from distlib.scripts import ScriptMaker
|
||||
|
||||
return ScriptMaker
|
||||
except ImportError:
|
||||
pass
|
||||
try:
|
||||
from pip._vendor.distlib.scripts import ScriptMaker
|
||||
|
||||
return ScriptMaker
|
||||
except ImportError:
|
||||
return None
|
||||
|
||||
|
||||
def exe_is_venv_bound(exe: Path, venv_dir: Path | None) -> bool:
|
||||
"""True when an existing launcher exe embeds the venv interpreter —
|
||||
i.e. it is a copied venv console-script trampoline from the pre-pm
|
||||
installer, which boots through ``venv\\Scripts\\python.exe`` and must be
|
||||
replaced. distlib launchers append the interpreter shebang after the zip
|
||||
payload; both encodings are scanned to be safe."""
|
||||
if venv_dir is None:
|
||||
return False
|
||||
needles = set()
|
||||
for interpreter in (
|
||||
venv_dir / "Scripts" / "hermes.exe",
|
||||
venv_dir / "Scripts" / "hermes-acp.exe",
|
||||
venv_dir / "Scripts" / "python.exe",
|
||||
venv_dir / "bin" / "python3",
|
||||
venv_dir / "bin" / "hermes",
|
||||
venv_dir / "bin" / "hermes-acp",
|
||||
):
|
||||
for enc in ("utf-8", "utf-16-le"):
|
||||
try:
|
||||
needles.add(str(interpreter).encode(enc))
|
||||
except Exception:
|
||||
continue
|
||||
try:
|
||||
data = Path(exe).read_bytes()
|
||||
except OSError:
|
||||
return False
|
||||
return any(needle in data for needle in needles)
|
||||
|
||||
|
||||
def _write_atomic(target: Path, write) -> Path | None:
|
||||
"""Stage under a pid-suffixed name then os.replace, so a concurrent
|
||||
process start never sees a torn launcher."""
|
||||
staging = target.with_name(f"{target.name}.stage.{os.getpid()}")
|
||||
try:
|
||||
write(staging)
|
||||
os.replace(staging, target)
|
||||
return target
|
||||
except OSError:
|
||||
try:
|
||||
staging.unlink()
|
||||
except OSError:
|
||||
pass
|
||||
return None
|
||||
|
||||
|
||||
def mint_launcher(
|
||||
name: str,
|
||||
repo_root: Path,
|
||||
out_dir: Path,
|
||||
python_exe: Path,
|
||||
site_packages: Path | None,
|
||||
) -> Path | None:
|
||||
"""Write one launcher for *name* into out_dir. Returns the written path
|
||||
or None. Prefers a distlib exe trampoline whose embedded script inserts
|
||||
the repo root and site-packages before importing the entry point; falls
|
||||
back to a .cmd delegator (same interpreter, same PYTHONPATH) when
|
||||
distlib is unavailable."""
|
||||
module, func = ENTRY_POINTS[name]
|
||||
out_dir = Path(out_dir)
|
||||
|
||||
script_maker_cls = _load_script_maker()
|
||||
if script_maker_cls is not None:
|
||||
# The template is %-formatted by distlib with (module, import_name,
|
||||
# func); the install-time paths are baked in as literals. The repo
|
||||
# root goes FIRST — its packages win over site-packages (same
|
||||
# ordering as the desktop shim's PYTHONPATH composition).
|
||||
esc = lambda p: str(p).replace("%", "%%") # noqa: E731
|
||||
site_line = (
|
||||
f"sys.path.append({esc(site_packages)!r})\n" if site_packages else ""
|
||||
)
|
||||
|
||||
class _PathedScriptMaker(script_maker_cls): # type: ignore[misc,valid-type]
|
||||
script_template = (
|
||||
"# -*- coding: utf-8 -*-\n"
|
||||
"import re\n"
|
||||
"import sys\n"
|
||||
f"sys.path.insert(0, {esc(repo_root)!r})\n"
|
||||
f"{site_line}"
|
||||
"if __name__ == '__main__':\n"
|
||||
" from %(module)s import %(import_name)s\n"
|
||||
" sys.argv[0] = re.sub(r'(-script\\.pyw|\\.exe)?$', '', sys.argv[0])\n"
|
||||
" sys.exit(%(func)s())\n"
|
||||
)
|
||||
|
||||
maker = _PathedScriptMaker(None, str(out_dir), add_launchers=True)
|
||||
maker.executable = str(python_exe)
|
||||
maker.variants = {""}
|
||||
maker.clobber = True
|
||||
try:
|
||||
written = maker.make(f"{name} = {module}:{func}")
|
||||
except Exception:
|
||||
written = []
|
||||
for path in written:
|
||||
if Path(path).suffix.lower() == ".exe":
|
||||
return Path(path)
|
||||
# distlib ran but produced no exe (unexpected) — fall through to cmd.
|
||||
|
||||
site = f";{site_packages}" if site_packages else ""
|
||||
code = f"import sys; from {module} import {func}; sys.exit({func}())"
|
||||
body = (
|
||||
"@echo off\r\n"
|
||||
"chcp 65001 >nul\r\n"
|
||||
f'set "PYTHONPATH={repo_root}{site}"\r\n'
|
||||
'set "PYTHONHOME="\r\n'
|
||||
f'"{python_exe}" -c "{code}" %*\r\n'
|
||||
)
|
||||
return _write_atomic(out_dir / f"{name}.cmd", lambda p: p.write_text(body, encoding="utf-8"))
|
||||
|
||||
|
||||
def stage_launcher(name: str, repo_root: Path, out_dir: Path) -> Path | None:
|
||||
"""Stage/refresh ONE launcher for the install rooted at repo_root,
|
||||
writing what the CURRENT state supports:
|
||||
|
||||
- store interpreter present → exe (or .cmd if distlib is missing) bound
|
||||
to it, with the repo-first PYTHONPATH baked in;
|
||||
- store interpreter absent → a runtime-resolving .cmd that finds the
|
||||
store python at boot and fails with a clear message until
|
||||
`hermes pm install` materializes it.
|
||||
|
||||
Never raises; returns the written path or None."""
|
||||
repo_root = Path(repo_root)
|
||||
venv_dir = project_venv_dir(repo_root)
|
||||
site_packages = venv_site_packages(venv_dir) if venv_dir else None
|
||||
store_python = resolve_store_python(repo_root)
|
||||
if store_python is not None:
|
||||
path = mint_launcher(name, repo_root, out_dir, store_python, site_packages)
|
||||
if path is not None:
|
||||
return path
|
||||
return _write_runtime_cmd(name, repo_root, site_packages, out_dir)
|
||||
|
||||
|
||||
def ensure_install_launchers(repo_root: Path, out_dir: Path) -> list[str]:
|
||||
"""Stage/refresh every launcher in WINDOWS_BIN_LAUNCHERS — see
|
||||
:func:`stage_launcher` for the per-name contract."""
|
||||
repo_root = Path(repo_root)
|
||||
written: list[str] = []
|
||||
for name in WINDOWS_BIN_LAUNCHERS:
|
||||
path = stage_launcher(name, repo_root, Path(out_dir))
|
||||
if path is not None:
|
||||
written.append(str(path))
|
||||
return written
|
||||
|
||||
|
||||
def _write_runtime_cmd(
|
||||
name: str,
|
||||
repo_root: Path,
|
||||
site_packages: Path | None,
|
||||
out_dir: Path,
|
||||
) -> Path | None:
|
||||
"""The boot-time-resolving .cmd fallback (fresh install, no store
|
||||
interpreter yet): glob ``<runtime>\\python-*`` for the store python at
|
||||
boot, compose PYTHONPATH, and fail with a clear message when the store
|
||||
is empty. Never references the venv interpreter. The ``endlocal & set``
|
||||
idiom hoists the boot-resolved interpreter and PYTHONPATH out of the
|
||||
setlocal scope (percent expansion happens while setlocal is still
|
||||
active, before endlocal executes)."""
|
||||
module, func = ENTRY_POINTS[name]
|
||||
site = ""
|
||||
if site_packages is not None:
|
||||
site = (
|
||||
";%HERMES_REPO%\\"
|
||||
+ str(site_packages.relative_to(repo_root)).replace("/", "\\")
|
||||
)
|
||||
body = (
|
||||
"@echo off\r\n"
|
||||
"chcp 65001 >nul\r\n"
|
||||
"setlocal\r\n"
|
||||
f'set "HERMES_REPO={repo_root}"\r\n'
|
||||
'set "PM_RT=%HERMES_RUNTIME_DIR%"\r\n'
|
||||
'if not defined PM_RT set "PM_RT=%LOCALAPPDATA%\\hermes\\tools"\r\n'
|
||||
'set "PM_PY="\r\n'
|
||||
'for /d %%D in ("%PM_RT%\\python-*") do if exist "%%D\\python.exe" set "PM_PY=%%D\\python.exe"\r\n'
|
||||
'if not defined PM_PY (\r\n'
|
||||
" echo hermes: no pm store interpreter under %PM_RT% - run \"hermes pm install\" first 1>&2\r\n"
|
||||
" exit /b 1\r\n"
|
||||
")\r\n"
|
||||
f'endlocal & set "PYTHONPATH=%HERMES_REPO%{site}" & set "PM_PY=%PM_PY%"\r\n'
|
||||
'set "PYTHONHOME="\r\n'
|
||||
f'set "PM_ENTRY=import sys; from {module} import {func}; sys.exit({func}())"\r\n'
|
||||
'"%PM_PY%" -c "%PM_ENTRY%" %*\r\n'
|
||||
)
|
||||
return _write_atomic(
|
||||
Path(out_dir) / f"{name}.cmd",
|
||||
lambda p: p.write_text(body, encoding="utf-8"),
|
||||
)
|
||||
@@ -38,6 +38,7 @@ __all__ = [
|
||||
"IS_WINDOWS",
|
||||
"resolve_node_command",
|
||||
"split_command_line",
|
||||
"restore_ambient_pythonpath",
|
||||
"suppress_platform_ver_console",
|
||||
"windows_detach_flags",
|
||||
"windows_detach_flags_without_breakaway",
|
||||
@@ -93,6 +94,29 @@ def split_command_line(line: str) -> list[str]:
|
||||
# -----------------------------------------------------------------------------
|
||||
|
||||
|
||||
def restore_ambient_pythonpath(env: Mapping[str, str]) -> dict:
|
||||
"""Re-add the ambient ``PYTHONPATH`` to a child environment that a
|
||||
``build_subprocess_env``-style factory already built.
|
||||
|
||||
No-boot-through-venv: the boot interpreter is the pm STORE python, whose
|
||||
imports arrive via ``PYTHONPATH=<repo>;<venv>/site-packages`` (it has no
|
||||
editable install). The subprocess-env factories strip Hermes-owned
|
||||
PYTHONPATH entries so agent-run children on DIFFERENT interpreter
|
||||
versions never load the backend's C extensions — but a child that
|
||||
re-execs THIS interpreter (``sys.executable -m hermes_cli.main``) runs
|
||||
on the same version and needs those entries back. Prepending keeps the
|
||||
launcher's repo-first ordering intact.
|
||||
"""
|
||||
merged = dict(env)
|
||||
ambient = os.environ.get("PYTHONPATH")
|
||||
if ambient:
|
||||
existing = merged.get("PYTHONPATH", "")
|
||||
merged["PYTHONPATH"] = (
|
||||
ambient + os.pathsep + existing if existing else ambient
|
||||
)
|
||||
return merged
|
||||
|
||||
|
||||
def resolve_node_command(name: str, argv: Sequence[str]) -> list[str]:
|
||||
"""Resolve a Node-ecosystem command name to an absolute-path argv.
|
||||
|
||||
|
||||
@@ -1731,7 +1731,7 @@ def list_quick_snapshots(
|
||||
manifest_path = d / "manifest.json"
|
||||
if manifest_path.exists():
|
||||
try:
|
||||
with open(manifest_path, encoding="utf-8") as f:
|
||||
with open(manifest_path, encoding="utf-8-sig") as f:
|
||||
results.append(json.load(f))
|
||||
except (json.JSONDecodeError, OSError):
|
||||
results.append({"id": d.name, "file_count": 0, "total_size": 0})
|
||||
@@ -1775,7 +1775,7 @@ def restore_quick_snapshot(
|
||||
if not manifest_path.exists():
|
||||
return False
|
||||
|
||||
with open(manifest_path, encoding="utf-8") as f:
|
||||
with open(manifest_path, encoding="utf-8-sig") as f:
|
||||
meta = json.load(f)
|
||||
|
||||
restored = 0
|
||||
|
||||
@@ -3253,7 +3253,7 @@ class CLICommandsMixin:
|
||||
# Fall back to a bare invocation (editor value may not be a
|
||||
# simple argv-splittable string on some platforms).
|
||||
subprocess.call(f"{editor} {shlex.quote(path)}", shell=True)
|
||||
with open(path, "r", encoding="utf-8") as fh:
|
||||
with open(path, "r", encoding="utf-8-sig") as fh:
|
||||
raw = fh.read()
|
||||
finally:
|
||||
try:
|
||||
|
||||
+34
-161
@@ -1,185 +1,58 @@
|
||||
"""Lazy dependency bootstrapper for non-Python runtime deps.
|
||||
"""Lazy bootstrapper for non-Python runtime deps, routed through pm.
|
||||
|
||||
Detection and prompting live here in Python — not in install.sh — because:
|
||||
1. shutil.which() works on every platform; install.sh needs bash.
|
||||
2. Detection is instant; spawning bash for a "is node installed?" check is waste.
|
||||
3. Python controls the UX (rich prompts, non-interactive fallback, TTY detection).
|
||||
The old shape spawned install.sh/install.ps1 with an --ensure flag; the
|
||||
rewritten bootstraps no longer install tools ("heavy deps are pm's job"),
|
||||
so this module is now a thin adapter: each dep maps to the pm packages
|
||||
that provide it, pm's lazy-install policy decides whether installing is
|
||||
allowed right now, and pm's InstallError carries the remedy when not.
|
||||
|
||||
install.sh is still the *installation* backend because it has 1900 lines of
|
||||
battle-tested OS detection and package-manager logic (apt/brew/pacman/dnf/
|
||||
zypper/Termux/…). Reimplementing that in Python would be huge duplication.
|
||||
|
||||
Deps that degrade gracefully (ripgrep → grep fallback, ffmpeg → skip conversion)
|
||||
don't need ensure_dependency wired in — only hard-fail sites do (TUI needs node,
|
||||
browser tool needs agent-browser).
|
||||
Deps that degrade gracefully (ffmpeg → skip conversion) are not wired
|
||||
here — only hard-fail sites call ensure_dependency (TUI needs node,
|
||||
browser tools need the browser stack).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import platform
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from hermes_constants import agent_browser_runnable, find_node_executable
|
||||
from tools.environments.local import hermes_subprocess_env
|
||||
from hermes_constants import find_node_executable
|
||||
|
||||
_IS_WINDOWS = platform.system() == "Windows"
|
||||
|
||||
_DEP_CHECKS = {
|
||||
# find_node_executable() rather than a bare which(): $HERMES_HOME/node is
|
||||
# not on PATH, so which() would report Node missing on an install that has
|
||||
# a managed one and trigger a redundant re-install.
|
||||
"node": lambda: find_node_executable("node") is not None,
|
||||
"browser": lambda: (
|
||||
agent_browser_runnable(shutil.which("agent-browser"))
|
||||
or _has_system_browser()
|
||||
or _has_hermes_agent_browser()
|
||||
or _has_npx_agent_browser()
|
||||
),
|
||||
"ripgrep": lambda: shutil.which("rg") is not None,
|
||||
"ffmpeg": lambda: shutil.which("ffmpeg") is not None,
|
||||
}
|
||||
|
||||
_DEP_DESCRIPTIONS = {
|
||||
"node": "Node.js (required for browser tools and TUI)",
|
||||
"browser": "Browser engine (Chromium, for web browsing tools)",
|
||||
"ripgrep": "ripgrep (fast file search)",
|
||||
"ffmpeg": "ffmpeg (TTS voice messages)",
|
||||
# dep name -> (availability check, pm packages that provide it)
|
||||
_DEPS = {
|
||||
"node": (lambda: find_node_executable("node") is not None, ("node",)),
|
||||
"browser": (lambda: _browser_available(), ("agent-browser", "chromium")),
|
||||
"ripgrep": (lambda: shutil.which("rg") is not None, ("ripgrep",)),
|
||||
}
|
||||
|
||||
|
||||
def _has_system_browser() -> bool:
|
||||
if _IS_WINDOWS:
|
||||
names = ("chrome", "msedge", "chromium")
|
||||
else:
|
||||
names = ("google-chrome", "google-chrome-stable", "chromium", "chromium-browser", "chrome")
|
||||
for name in names:
|
||||
if shutil.which(name):
|
||||
return True
|
||||
return False
|
||||
def _browser_available() -> bool:
|
||||
from hermes_constants import agent_browser_runnable
|
||||
|
||||
|
||||
def _has_npx_agent_browser() -> bool:
|
||||
"""agent-browser resolves lazily via npx on the default install (#43564),
|
||||
invisible to the PATH/managed-dir probes above. Mirror
|
||||
tools.browser_tool.check_browser_requirements's Termux carve-out so this
|
||||
check can't diverge from what browser tools actually find."""
|
||||
if agent_browser_runnable(shutil.which("agent-browser")):
|
||||
return True
|
||||
try:
|
||||
from tools.browser_tool import (
|
||||
_find_agent_browser,
|
||||
_is_npx_agent_browser_sentinel,
|
||||
_requires_real_termux_browser_install,
|
||||
)
|
||||
browser_cmd = _find_agent_browser(validate=False)
|
||||
import pm
|
||||
|
||||
return pm.is_installed("agent-browser")
|
||||
except Exception:
|
||||
return False
|
||||
if not _is_npx_agent_browser_sentinel(browser_cmd):
|
||||
return False
|
||||
return not _requires_real_termux_browser_install(browser_cmd)
|
||||
|
||||
|
||||
def _has_hermes_agent_browser() -> bool:
|
||||
from hermes_constants import get_hermes_home
|
||||
home = get_hermes_home()
|
||||
if _IS_WINDOWS:
|
||||
# npm -g --prefix puts .cmd shims directly in the prefix dir on Windows
|
||||
return (home / "node" / "agent-browser.cmd").is_file()
|
||||
# install.sh installs globally into $HERMES_HOME/node/bin/ via npm -g --prefix
|
||||
# Also check legacy node_modules/.bin/ path for git-clone installs.
|
||||
return (
|
||||
(home / "node" / "bin" / "agent-browser").is_file()
|
||||
or (home / "node_modules" / ".bin" / "agent-browser").is_file()
|
||||
)
|
||||
|
||||
|
||||
def _find_install_script(
|
||||
package_dir: Path | None = None,
|
||||
repo_root: Path | None = None,
|
||||
) -> tuple[Path | None, str | None]:
|
||||
"""Locate the install script — bundled in wheel or in git checkout.
|
||||
|
||||
On Windows, prefers install.ps1; on POSIX, prefers install.sh.
|
||||
Returns a (path, shell) tuple, or (None, None) if neither is found.
|
||||
"""
|
||||
if package_dir is None:
|
||||
package_dir = Path(__file__).parent
|
||||
if repo_root is None:
|
||||
repo_root = package_dir.parent
|
||||
|
||||
if _IS_WINDOWS:
|
||||
preferred = ("install.ps1", "powershell")
|
||||
fallback = ("install.sh", "bash")
|
||||
else:
|
||||
preferred = ("install.sh", "bash")
|
||||
fallback = ("install.ps1", "powershell")
|
||||
|
||||
for script_name, shell in (preferred, fallback):
|
||||
bundled = package_dir / "scripts" / script_name
|
||||
if bundled.is_file():
|
||||
return bundled, shell
|
||||
repo = repo_root / "scripts" / script_name
|
||||
if repo.is_file():
|
||||
return repo, shell
|
||||
|
||||
return None, None
|
||||
|
||||
|
||||
def ensure_dependency(
|
||||
dep: str,
|
||||
interactive: bool = True,
|
||||
) -> bool:
|
||||
def ensure_dependency(dep: str, interactive: bool = True) -> bool:
|
||||
"""Ensure a non-Python dependency is available. Returns True if available."""
|
||||
check = _DEP_CHECKS.get(dep)
|
||||
if check is None:
|
||||
# Unknown dep — don't silently forward to install script.
|
||||
entry = _DEPS.get(dep)
|
||||
if entry is None:
|
||||
return False
|
||||
check, packages = entry
|
||||
if check():
|
||||
return True
|
||||
|
||||
script, shell = _find_install_script()
|
||||
if script is None:
|
||||
try:
|
||||
import pm
|
||||
|
||||
for name in packages:
|
||||
pm.ensure(name)
|
||||
except Exception as exc:
|
||||
if interactive:
|
||||
desc = _DEP_DESCRIPTIONS.get(dep, dep)
|
||||
print(f" {desc} is not installed and no install script was found.")
|
||||
print(f" Install {dep} manually and try again.")
|
||||
print(f" {exc}")
|
||||
return False
|
||||
|
||||
if interactive and sys.stdin.isatty():
|
||||
desc = _DEP_DESCRIPTIONS.get(dep, dep)
|
||||
try:
|
||||
reply = input(f"{desc} is not installed. Install now? [Y/n] ").strip().lower()
|
||||
except (EOFError, KeyboardInterrupt):
|
||||
return False
|
||||
if reply not in ("", "y", "yes"):
|
||||
return False
|
||||
|
||||
if shell == "powershell":
|
||||
from hermes_constants import get_hermes_home
|
||||
ps_bin = shutil.which("powershell") or shutil.which("pwsh")
|
||||
if not ps_bin:
|
||||
if interactive:
|
||||
print(" PowerShell not found. Install PowerShell or run install.ps1 manually.")
|
||||
return False
|
||||
cmd = [
|
||||
ps_bin,
|
||||
"-ExecutionPolicy", "Bypass",
|
||||
"-File", str(script),
|
||||
"-Ensure", dep,
|
||||
"-HermesHome", str(get_hermes_home()),
|
||||
]
|
||||
else:
|
||||
cmd = ["bash", str(script), "--ensure", dep]
|
||||
|
||||
run_env = hermes_subprocess_env(inherit_credentials=False)
|
||||
run_env["IS_INTERACTIVE"] = "false"
|
||||
result = subprocess.run(
|
||||
cmd,
|
||||
env=run_env,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return False
|
||||
|
||||
if check:
|
||||
return check()
|
||||
return True
|
||||
return check()
|
||||
+41
-9
@@ -706,7 +706,7 @@ def _read_pyproject_version() -> str | None:
|
||||
"""
|
||||
pyproject = PROJECT_ROOT / "pyproject.toml"
|
||||
try:
|
||||
text = pyproject.read_text(encoding="utf-8")
|
||||
text = pyproject.read_text(encoding="utf-8-sig")
|
||||
except OSError:
|
||||
return None
|
||||
in_project = False
|
||||
@@ -1232,6 +1232,34 @@ def check_macos_full_disk_access() -> None:
|
||||
)
|
||||
|
||||
|
||||
def _pm_venv_active() -> bool:
|
||||
"""Whether pm provisioned a runtime venv for this install.
|
||||
|
||||
Resolved from pm's own records (facts.json + store layout), never from
|
||||
interpreter state: under no-boot-through-venv ``sys.prefix`` always
|
||||
equals ``sys.base_prefix`` and ``VIRTUAL_ENV`` is unset in bundled
|
||||
installs. A bundled install keeps its relocatable venv beside the
|
||||
manifest; a dev install syncs the project venv (``venv``/``.venv``).
|
||||
Falls back to the legacy ``sys.prefix`` probe when pm records nothing
|
||||
here (pre-pm checkouts) or pm itself cannot be read.
|
||||
"""
|
||||
try:
|
||||
from pm import paths
|
||||
from pm.lock import Facts
|
||||
|
||||
if Facts(paths.facts_path()).get("venv"):
|
||||
store = paths.store_root()
|
||||
bundled = store.parent / "venv"
|
||||
if (store.parent / "manifest.json").is_file():
|
||||
return bundled.is_dir()
|
||||
from hermes_constants import project_venv_dir
|
||||
|
||||
return project_venv_dir(paths.repo_root()) is not None
|
||||
except Exception:
|
||||
pass
|
||||
return sys.prefix != sys.base_prefix
|
||||
|
||||
|
||||
def run_doctor(args):
|
||||
"""Run diagnostic checks."""
|
||||
should_fix = getattr(args, 'fix', False)
|
||||
@@ -1395,8 +1423,12 @@ def run_doctor(args):
|
||||
_report_database_journal_modes()
|
||||
except Exception as e:
|
||||
check_warn(f"SQLite version probe failed: {e}")
|
||||
# Check if in virtual environment
|
||||
in_venv = sys.prefix != sys.base_prefix
|
||||
# Check whether pm provisioned a runtime venv for this install.
|
||||
# Under no-boot-through-venv the gateway runs the store python
|
||||
# (sys.prefix == sys.base_prefix always, VIRTUAL_ENV unset in bundled
|
||||
# installs), so prefix sniffing cannot distinguish; pm's facts are the
|
||||
# authority. Falls back to the legacy probe when pm has no venv here.
|
||||
in_venv = _pm_venv_active()
|
||||
if in_venv:
|
||||
check_ok("Virtual environment active")
|
||||
else:
|
||||
@@ -1464,7 +1496,7 @@ def run_doctor(args):
|
||||
# latin-1 for Windows Notepad/cp1252 files that are not valid UTF-8 —
|
||||
# matches hermes_cli.env_loader._load_dotenv_with_fallback.
|
||||
try:
|
||||
content = env_path.read_text(encoding="utf-8")
|
||||
content = env_path.read_text(encoding="utf-8-sig")
|
||||
except UnicodeDecodeError:
|
||||
content = env_path.read_text(encoding="latin-1")
|
||||
if _has_provider_env_config(content):
|
||||
@@ -1995,7 +2027,7 @@ def run_doctor(args):
|
||||
# Check for SOUL.md persona file
|
||||
soul_path = hermes_home / "SOUL.md"
|
||||
if soul_path.exists():
|
||||
content = soul_path.read_text(encoding="utf-8").strip()
|
||||
content = soul_path.read_text(encoding="utf-8-sig").strip()
|
||||
# Check if it's just the template comments (no real content)
|
||||
lines = [l for l in content.splitlines() if l.strip() and not l.strip().startswith(("<!--", "-->", "#"))]
|
||||
if lines:
|
||||
@@ -2022,12 +2054,12 @@ def run_doctor(args):
|
||||
memory_file = memories_dir / "MEMORY.md"
|
||||
user_file = memories_dir / "USER.md"
|
||||
if memory_file.exists():
|
||||
size = len(memory_file.read_text(encoding="utf-8").strip())
|
||||
size = len(memory_file.read_text(encoding="utf-8-sig").strip())
|
||||
check_ok(f"MEMORY.md exists ({size} chars)")
|
||||
else:
|
||||
check_info("MEMORY.md not created yet (will be created when the agent first writes a memory)")
|
||||
if user_file.exists():
|
||||
size = len(user_file.read_text(encoding="utf-8").strip())
|
||||
size = len(user_file.read_text(encoding="utf-8-sig").strip())
|
||||
check_ok(f"USER.md exists ({size} chars)")
|
||||
else:
|
||||
check_info("USER.md not created yet (will be created when the agent first writes a memory)")
|
||||
@@ -3178,7 +3210,7 @@ def run_doctor(args):
|
||||
if lock_file.exists():
|
||||
try:
|
||||
import json
|
||||
lock_data = json.loads(lock_file.read_text(encoding="utf-8"))
|
||||
lock_data = json.loads(lock_file.read_text(encoding="utf-8-sig"))
|
||||
count = len(lock_data.get("installed", {}))
|
||||
check_ok(f"Lock file OK ({count} hub-installed skill(s))")
|
||||
except Exception:
|
||||
@@ -3344,7 +3376,7 @@ def run_doctor(args):
|
||||
if not wrapper.is_file():
|
||||
continue
|
||||
try:
|
||||
content = wrapper.read_text(encoding="utf-8")
|
||||
content = wrapper.read_text(encoding="utf-8-sig")
|
||||
if "hermes -p" in content:
|
||||
_m = _re.search(r"hermes -p (\S+)", content)
|
||||
if _m and not profile_exists(_m.group(1)):
|
||||
|
||||
+56
-6
@@ -3525,15 +3525,57 @@ def launchd_gateway_labels_for_install() -> list[str]:
|
||||
return root_label + sorted(profile_labels)
|
||||
|
||||
|
||||
def _pm_runtime_venv_dir() -> Path | None:
|
||||
"""The venv pm provisioned for this install, resolved from pm's own
|
||||
records (facts.json + store layout) — never from interpreter state.
|
||||
|
||||
Under no-boot-through-venv the gateway runs the store python with the
|
||||
venv's site-packages on PYTHONPATH, so ``sys.prefix`` always equals
|
||||
``sys.base_prefix`` and ``VIRTUAL_ENV`` is unset in bundled installs;
|
||||
prefix/env probing silently degrades. pm is the authority instead: a
|
||||
bundled install keeps its relocatable venv beside the manifest (a
|
||||
sibling of the store), a dev install syncs the project venv
|
||||
(``venv``/``.venv``, per ``hermes_constants.project_venv_dir`` — the
|
||||
same layout ``pm.packages.Venv.venv_dir`` materializes). The venv fact
|
||||
must exist: pm only vouches for what it provisioned.
|
||||
"""
|
||||
try:
|
||||
from pm import paths
|
||||
from pm.lock import Facts
|
||||
except Exception:
|
||||
return None
|
||||
try:
|
||||
if not Facts(paths.facts_path()).get("venv"):
|
||||
return None
|
||||
except Exception:
|
||||
return None
|
||||
store = paths.store_root()
|
||||
bundled = store.parent / "venv"
|
||||
if (store.parent / "manifest.json").is_file():
|
||||
return bundled if bundled.is_dir() else None
|
||||
try:
|
||||
from hermes_constants import project_venv_dir
|
||||
except ImportError:
|
||||
return None
|
||||
return project_venv_dir(paths.repo_root())
|
||||
|
||||
|
||||
def _detect_venv_dir() -> Path | None:
|
||||
"""Detect the active virtualenv directory.
|
||||
|
||||
Checks ``sys.prefix`` first (works regardless of the directory name),
|
||||
then ``VIRTUAL_ENV`` env var (covers uv-managed environments where
|
||||
sys.prefix == sys.base_prefix), then falls back to probing common
|
||||
directory names under PROJECT_ROOT.
|
||||
Resolves the pm-provisioned runtime venv first (facts + store layout —
|
||||
the authority under no-boot-through-venv, where ``sys.prefix`` no
|
||||
longer distinguishes and bundled installs carry no ``VIRTUAL_ENV``).
|
||||
Legacy probes follow for pre-pm environments: ``sys.prefix`` (works
|
||||
regardless of the directory name), then the ``VIRTUAL_ENV`` env var
|
||||
(covers uv-managed environments where sys.prefix ==
|
||||
sys.base_prefix), then common directory names under PROJECT_ROOT.
|
||||
Returns ``None`` when no virtualenv can be found.
|
||||
"""
|
||||
pm_venv = _pm_runtime_venv_dir()
|
||||
if pm_venv is not None and pm_venv.is_dir():
|
||||
return pm_venv
|
||||
|
||||
# If we're running inside a virtualenv, sys.prefix points to it.
|
||||
if sys.prefix != sys.base_prefix:
|
||||
venv = Path(sys.prefix)
|
||||
@@ -3706,11 +3748,19 @@ def _build_service_path_dirs(project_root: Path | None = None) -> list[str]:
|
||||
|
||||
candidates = []
|
||||
|
||||
# The interpreter's own bin dir. Under no-boot-through-venv the
|
||||
# gateway runs the store python (sys.prefix == sys.base_prefix), so
|
||||
# the venv bin dir comes from pm's facts/store resolution, not from
|
||||
# prefix sniffing.
|
||||
venv_bin = project_root / "venv" / "bin"
|
||||
if _is_dir(venv_bin):
|
||||
candidates.append(str(venv_bin))
|
||||
elif sys.prefix != sys.base_prefix:
|
||||
candidates.append(str(Path(sys.prefix) / "bin"))
|
||||
else:
|
||||
pm_venv = _pm_runtime_venv_dir()
|
||||
if pm_venv is not None:
|
||||
pm_venv_bin = pm_venv / "bin"
|
||||
if _is_dir(pm_venv_bin):
|
||||
candidates.append(str(pm_venv_bin))
|
||||
|
||||
node_bin = project_root / "node_modules" / ".bin"
|
||||
if _is_dir(node_bin):
|
||||
|
||||
@@ -57,6 +57,19 @@ def icon_path(project_root: Path) -> Path:
|
||||
return project_root / "apps" / "desktop" / "assets" / "icon.png"
|
||||
|
||||
|
||||
def _repo_pythonpath_entry() -> str | None:
|
||||
"""The repo root as a PYTHONPATH entry — under no-boot-through-venv the
|
||||
boot interpreter is the pm STORE python whose imports arrive via
|
||||
PYTHONPATH (no editable install), and a .desktop launch by the DE
|
||||
inherits NO environment, so the Exec line must carry it explicitly."""
|
||||
try:
|
||||
import hermes_cli
|
||||
|
||||
return str(Path(hermes_cli.__file__).resolve().parent.parent)
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def resolve_exec_command() -> str:
|
||||
"""Build the absolute ``Exec=`` command line for ``hermes desktop``.
|
||||
|
||||
@@ -66,6 +79,8 @@ def resolve_exec_command() -> str:
|
||||
"""
|
||||
from hermes_cli.relaunch import resolve_hermes_bin
|
||||
|
||||
repo = _repo_pythonpath_entry()
|
||||
prefix = f"env PYTHONPATH={_quote_exec_arg(repo)} " if repo else ""
|
||||
bin_path = resolve_hermes_bin()
|
||||
if bin_path:
|
||||
resolved = Path(bin_path).resolve()
|
||||
@@ -79,11 +94,13 @@ def resolve_exec_command() -> str:
|
||||
# sys.executable is the interpreter actually running Hermes (the
|
||||
# venv one), so prefix it explicitly.
|
||||
argv = [str(Path(sys.executable).resolve()), str(resolved), "desktop"]
|
||||
return prefix + " ".join(_quote_exec_arg(a) for a in argv)
|
||||
else:
|
||||
argv = [str(resolved), "desktop"]
|
||||
return " ".join(_quote_exec_arg(a) for a in argv)
|
||||
else:
|
||||
argv = [str(Path(sys.executable).resolve()), "-m", "hermes_cli.main", "desktop"]
|
||||
return " ".join(_quote_exec_arg(a) for a in argv)
|
||||
return prefix + " ".join(_quote_exec_arg(a) for a in argv)
|
||||
|
||||
|
||||
def _needs_interpreter(bin_path: Path) -> bool:
|
||||
|
||||
@@ -147,7 +147,7 @@ def _interpreter_source(venv_dir: Path) -> str | None:
|
||||
return None
|
||||
home = ""
|
||||
try:
|
||||
for line in cfg.read_text(encoding="utf-8").splitlines():
|
||||
for line in cfg.read_text(encoding="utf-8-sig").splitlines():
|
||||
if line.lower().startswith("home"):
|
||||
_, _, home = line.partition("=")
|
||||
home = home.strip()
|
||||
@@ -390,7 +390,7 @@ def ensure_tcc_anchor(project_root: Path | None = None) -> Path | None:
|
||||
if not venv_py.is_symlink():
|
||||
marker = _anchor_marker(venv_py.parent)
|
||||
try:
|
||||
if marker.is_file() and marker.read_text(encoding="utf-8").strip() == (
|
||||
if marker.is_file() and marker.read_text(encoding="utf-8-sig").strip() == (
|
||||
_marker_value(source_file)
|
||||
):
|
||||
_provision_libpython(venv_dir, source_file, refresh=False)
|
||||
@@ -436,7 +436,7 @@ def tcc_anchor_state(project_root: Path | None = None) -> tuple[str, str]:
|
||||
source_file = _interpreter_file(source)
|
||||
expected = _marker_value(source_file) if source_file is not None else source
|
||||
try:
|
||||
if marker.is_file() and marker.read_text(encoding="utf-8").strip() == expected:
|
||||
if marker.is_file() and marker.read_text(encoding="utf-8-sig").strip() == expected:
|
||||
return "active", str(venv_py)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
+264
-753
File diff suppressed because it is too large.
Load diff
+37
-203
@@ -1,4 +1,4 @@
|
||||
"""Recover from npm ``EBADENGINE`` failures by upgrading a managed npm.
|
||||
"""Recover from npm ``EBADENGINE`` failures with the pm-pinned npm.
|
||||
|
||||
The repo's ``.npmrc`` sets ``engine-strict=true`` and the root ``package.json``
|
||||
pins an ``engines.npm`` range, so an npm outside that range aborts every
|
||||
@@ -13,39 +13,25 @@ an ``npm --version`` probe before work that usually succeeds), we react to it:
|
||||
npm states the required range in the error, so the recovery reads the
|
||||
constraint straight out of the output it just produced.
|
||||
|
||||
Scope of the repair is deliberately narrow. Hermes only upgrades an npm that
|
||||
lives inside its **own** managed Node tree (``$HERMES_HOME/node``), installing
|
||||
in place with ``--prefix`` so ``bin/npm`` keeps resolving to the upgraded
|
||||
``lib/node_modules/npm``. A system / nvm / brew / Nix npm belongs to the user
|
||||
and their other projects; Hermes never modifies those. When the failing npm is
|
||||
one of those foreign installs, Hermes instead provisions its own managed Node
|
||||
tree (the same tree a fresh install creates), upgrades *that* npm into range,
|
||||
and hands the caller the managed npm to retry with — leaving the user's
|
||||
toolchain untouched.
|
||||
Scope of the repair is deliberately narrow. A system / nvm / brew / Nix npm
|
||||
belongs to the user and their other projects; Hermes never modifies those.
|
||||
When the failing npm is a foreign install, Hermes ensures its own pm-pinned
|
||||
node/npm packages are installed and hands the caller pm's npm to retry with —
|
||||
leaving the user's toolchain untouched. When the failing npm already *is*
|
||||
pm's npm, the lockfile pin itself is out of range and no runtime action can
|
||||
fix that; the caller gets the manual guidance instead.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
from hermes_constants import (
|
||||
bootstrap_hermes_managed_node,
|
||||
get_hermes_home,
|
||||
managed_node_tree_in_use,
|
||||
with_hermes_node_path,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"is_ebadengine",
|
||||
"required_npm_range",
|
||||
"managed_npm_prefix",
|
||||
"upgrade_managed_npm",
|
||||
"maybe_repair_npm_engine",
|
||||
]
|
||||
|
||||
@@ -54,10 +40,6 @@ __all__ = [
|
||||
_REQUIRED_RE = re.compile(r"Required:\s*(\{.*?\})")
|
||||
_ACTUAL_RE = re.compile(r"Actual:\s*(\{.*?\})")
|
||||
|
||||
# Wall-clock cap for the self-upgrade. The measured in-place upgrade of a
|
||||
# managed tree takes ~1s; this only has to cover a slow registry.
|
||||
_UPGRADE_TIMEOUT = 300
|
||||
|
||||
|
||||
def is_ebadengine(output: str) -> bool:
|
||||
"""Return True when *output* is an npm engine-compatibility failure."""
|
||||
@@ -123,7 +105,7 @@ def _repo_npm_range() -> str | None:
|
||||
"""Return ``engines.npm`` from the checkout's root ``package.json``."""
|
||||
package_json = Path(__file__).resolve().parent.parent / "package.json"
|
||||
try:
|
||||
data = json.loads(package_json.read_text(encoding="utf-8"))
|
||||
data = json.loads(package_json.read_text(encoding="utf-8-sig"))
|
||||
except (OSError, ValueError):
|
||||
return None
|
||||
engines = data.get("engines")
|
||||
@@ -133,130 +115,26 @@ def _repo_npm_range() -> str | None:
|
||||
return str(value).strip() if value else None
|
||||
|
||||
|
||||
def managed_npm_prefix(npm: str | os.PathLike[str] | None) -> Path | None:
|
||||
"""Return the Hermes-managed Node root *npm* lives in, else ``None``.
|
||||
|
||||
Symlinks are resolved first: an install links ``~/.local/bin/npm`` at
|
||||
``$HERMES_HOME/node/bin/npm``, which itself links into
|
||||
``lib/node_modules/npm/bin/npm-cli.js``. Every one of those spellings is
|
||||
the managed npm and must be recognised as such, or the repair silently
|
||||
declines to fix the very install it owns.
|
||||
"""
|
||||
if not npm:
|
||||
return None
|
||||
prefix = get_hermes_home() / "node"
|
||||
try:
|
||||
resolved = Path(npm).resolve()
|
||||
prefix_resolved = prefix.resolve()
|
||||
except OSError:
|
||||
return None
|
||||
if resolved == prefix_resolved or prefix_resolved in resolved.parents:
|
||||
return prefix
|
||||
return None
|
||||
|
||||
|
||||
def _upgrade_env() -> dict[str, str]:
|
||||
env = with_hermes_node_path()
|
||||
# The checkout's .npmrc sets `min-release-age`, which would gate the npm
|
||||
# release we are trying to install. The upgrade runs from a temp cwd so
|
||||
# that file is out of scope; this neutralises a user-level ~/.npmrc too.
|
||||
env["npm_config_min_release_age"] = "0"
|
||||
# `unicode-animations`-style postinstall animations no-op under CI=1.
|
||||
env["CI"] = "1"
|
||||
return env
|
||||
|
||||
|
||||
def upgrade_managed_npm(
|
||||
npm: str,
|
||||
npm_range: str,
|
||||
*,
|
||||
prefix: Path,
|
||||
quiet: bool = False,
|
||||
) -> bool:
|
||||
"""Upgrade the managed npm at *npm* in place to satisfy *npm_range*.
|
||||
|
||||
``--prefix`` targets the managed tree explicitly: a managed install writes
|
||||
``prefix=~/.local`` into ``$HERMES_HOME/node/etc/npmrc`` so that global
|
||||
installs land on PATH, and without the override the "upgrade" would install
|
||||
a second npm somewhere else while the managed one stayed stale.
|
||||
"""
|
||||
def _pm_npm(*, quiet: bool = False) -> str | None:
|
||||
"""Install the pm-pinned node/npm packages and return pm's npm path."""
|
||||
if not quiet:
|
||||
print(
|
||||
f"→ Upgrading Hermes-managed npm to satisfy {npm_range}…",
|
||||
"→ Provisioning the Hermes-pinned Node.js runtime "
|
||||
"(the resolved npm belongs to your system and is left alone)…",
|
||||
flush=True,
|
||||
)
|
||||
# The managed npm lives inside the very tree the desktop app's Node
|
||||
# processes execute from; an in-place upgrade while it is in use fails
|
||||
# with PermissionError: [WinError 5] on npm.cmd (#80926). Defer instead
|
||||
# of forcing the write — the upgrade re-triggers on the next resolution
|
||||
# (e.g. the next update once the app is closed).
|
||||
if managed_node_tree_in_use():
|
||||
if not quiet:
|
||||
print(
|
||||
" ⚠ deferred: the Hermes-managed Node.js tree is in use by a "
|
||||
"running app; the npm upgrade will apply on a later update "
|
||||
"once the app is closed.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return False
|
||||
try:
|
||||
# A temp cwd keeps the checkout's .npmrc (engine-strict, min-release-age)
|
||||
# from applying to the upgrade itself.
|
||||
with tempfile.TemporaryDirectory(prefix="hermes-npm-upgrade-") as tmp:
|
||||
result = subprocess.run(
|
||||
[
|
||||
npm,
|
||||
"install",
|
||||
"--global",
|
||||
"--prefix",
|
||||
str(prefix),
|
||||
f"npm@{npm_range}",
|
||||
"--no-fund",
|
||||
"--no-audit",
|
||||
"--progress=false",
|
||||
],
|
||||
cwd=tmp,
|
||||
env=_upgrade_env(),
|
||||
capture_output=True,
|
||||
text=True,
|
||||
encoding="utf-8",
|
||||
errors="replace",
|
||||
timeout=_UPGRADE_TIMEOUT,
|
||||
check=False,
|
||||
)
|
||||
except (OSError, subprocess.SubprocessError):
|
||||
if not quiet:
|
||||
print(" ✗ npm upgrade could not be started", file=sys.stderr)
|
||||
return False
|
||||
import pm
|
||||
|
||||
if result.returncode != 0:
|
||||
if not quiet:
|
||||
detail = (result.stderr or result.stdout or "").strip().splitlines()
|
||||
print(" ✗ npm upgrade failed", file=sys.stderr)
|
||||
for line in detail[-10:]:
|
||||
print(f" {line}", file=sys.stderr)
|
||||
return False
|
||||
pm.ensure("npm")
|
||||
from hermes_constants import _pm_node_executable
|
||||
|
||||
if not quiet:
|
||||
print(f" ✓ npm upgraded to {_probe_version(npm) or npm_range}", flush=True)
|
||||
return True
|
||||
|
||||
|
||||
def _probe_version(npm: str) -> str | None:
|
||||
try:
|
||||
result = subprocess.run(
|
||||
[npm, "--version"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
encoding="utf-8",
|
||||
errors="replace",
|
||||
timeout=30,
|
||||
env=with_hermes_node_path(),
|
||||
check=False,
|
||||
)
|
||||
except (OSError, subprocess.SubprocessError):
|
||||
return None
|
||||
return (result.stdout or "").strip() or None
|
||||
managed = _pm_node_executable("npm")
|
||||
except Exception:
|
||||
managed = None
|
||||
if not managed and not quiet:
|
||||
print(" ✗ Managed Node.js provisioning failed", file=sys.stderr)
|
||||
return managed
|
||||
|
||||
|
||||
def _print_manual_fix(npm: str, npm_range: str, actual: str | None) -> None:
|
||||
@@ -271,41 +149,6 @@ def _print_manual_fix(npm: str, npm_range: str, actual: str | None) -> None:
|
||||
)
|
||||
|
||||
|
||||
def _provision_managed_npm(npm_range: str | None, *, quiet: bool = False) -> str | None:
|
||||
"""Provision a Hermes-managed Node tree and return a satisfying npm.
|
||||
|
||||
Installs the managed tree under ``$HERMES_HOME/node`` (reusing a healthy
|
||||
one when present), then upgrades its bundled npm to *npm_range* — a fresh
|
||||
Node LTS bundles an npm that may itself be outside the repo's range, so
|
||||
without the upgrade the caller's single retry would fail the same way.
|
||||
Falls back to the checkout's own ``engines.npm`` when npm did not state a
|
||||
range (a Node-only mismatch), so the managed npm ends up in range either
|
||||
way. Returns the managed npm path, or ``None`` when provisioning failed.
|
||||
"""
|
||||
if not quiet:
|
||||
print(
|
||||
"→ Provisioning a Hermes-managed Node.js runtime "
|
||||
"(the resolved npm belongs to your system and is left alone)…",
|
||||
flush=True,
|
||||
)
|
||||
managed_npm = bootstrap_hermes_managed_node()
|
||||
if not managed_npm:
|
||||
if not quiet:
|
||||
print(" ✗ Managed Node.js provisioning failed", file=sys.stderr)
|
||||
return None
|
||||
|
||||
prefix = managed_npm_prefix(managed_npm)
|
||||
if prefix is None: # pragma: no cover - bootstrap returned a foreign path
|
||||
return None
|
||||
|
||||
target_range = npm_range or _repo_npm_range()
|
||||
if target_range and not upgrade_managed_npm(
|
||||
managed_npm, target_range, prefix=prefix, quiet=quiet
|
||||
):
|
||||
return None
|
||||
return managed_npm
|
||||
|
||||
|
||||
def maybe_repair_npm_engine(
|
||||
npm: str | None,
|
||||
output: str,
|
||||
@@ -315,13 +158,13 @@ def maybe_repair_npm_engine(
|
||||
"""Repair an ``EBADENGINE`` failure, never touching a foreign toolchain.
|
||||
|
||||
*output* is the combined stdout/stderr of the npm command that just failed.
|
||||
Returns the npm executable the caller should retry its command with —
|
||||
the same *npm* after an in-place upgrade of a Hermes-managed install, or
|
||||
a freshly provisioned managed npm when the failing npm belongs to the
|
||||
user (system / nvm / brew / Nix installs are never modified). Returns
|
||||
``None`` when no repair happened — not an engine failure, a Node mismatch
|
||||
a managed npm upgrade cannot fix, or a failed upgrade/bootstrap — leaving
|
||||
the original failure to stand.
|
||||
Returns the npm executable the caller should retry its command with — the
|
||||
pm-pinned npm, freshly ensured, when the failing npm was a foreign install
|
||||
(system / nvm / brew / Nix installs are never modified). Returns ``None``
|
||||
when no repair happened — not an engine failure, the failing npm already
|
||||
was pm's own (the lockfile pin is out of range; a runtime install cannot
|
||||
fix that), or the pm install failed — leaving the original failure to
|
||||
stand.
|
||||
|
||||
The returned value is truthy exactly when the caller should retry once,
|
||||
so ``if maybe_repair_npm_engine(...)`` call sites keep working; they just
|
||||
@@ -330,25 +173,16 @@ def maybe_repair_npm_engine(
|
||||
if not npm or not is_ebadengine(output):
|
||||
return None
|
||||
|
||||
npm_range = required_npm_range(output)
|
||||
prefix = managed_npm_prefix(npm)
|
||||
|
||||
if prefix is not None:
|
||||
# Hermes owns this npm — upgrade it in place. Only an npm-range
|
||||
# failure is fixable this way; a Node mismatch needs a Node upgrade.
|
||||
if not npm_range:
|
||||
return None
|
||||
if upgrade_managed_npm(npm, npm_range, prefix=prefix, quiet=quiet):
|
||||
return npm
|
||||
return None
|
||||
|
||||
# Foreign npm (system / nvm / brew / Nix): provision our own runtime
|
||||
# instead. This also covers Node-version mismatches — the managed tree
|
||||
# ships a Node the repo supports.
|
||||
managed = _provision_managed_npm(npm_range, quiet=quiet)
|
||||
managed = _pm_npm(quiet=quiet)
|
||||
if managed:
|
||||
return managed
|
||||
try:
|
||||
already_managed = Path(managed).resolve() == Path(npm).resolve()
|
||||
except OSError:
|
||||
already_managed = False
|
||||
if not already_managed:
|
||||
return managed
|
||||
|
||||
npm_range = required_npm_range(output)
|
||||
if not quiet and npm_range:
|
||||
_print_manual_fix(npm, npm_range, actual_npm_version(output))
|
||||
return None
|
||||
@@ -353,7 +353,7 @@ def _read_manifest(plugin_dir: Path) -> dict:
|
||||
try:
|
||||
import yaml
|
||||
|
||||
with open(manifest_file, encoding="utf-8") as f:
|
||||
with open(manifest_file, encoding="utf-8-sig") as f:
|
||||
return yaml.safe_load(f) or {}
|
||||
except Exception as e:
|
||||
logger.warning("Failed to read plugin.yaml in %s: %s", plugin_dir, e)
|
||||
@@ -510,7 +510,7 @@ def _display_after_install(plugin_dir: Path, identifier: str) -> None:
|
||||
after_install = plugin_dir / "after-install.md"
|
||||
|
||||
if after_install.exists():
|
||||
content = after_install.read_text(encoding="utf-8")
|
||||
content = after_install.read_text(encoding="utf-8-sig")
|
||||
md = Markdown(content)
|
||||
console.print()
|
||||
console.print(Panel(md, border_style="green", expand=False))
|
||||
@@ -570,7 +570,7 @@ def _read_install_metadata() -> dict[str, dict[str, object]]:
|
||||
if not path.exists():
|
||||
return {}
|
||||
try:
|
||||
value = json.loads(path.read_text(encoding="utf-8"))
|
||||
value = json.loads(path.read_text(encoding="utf-8-sig"))
|
||||
except (OSError, json.JSONDecodeError) as exc:
|
||||
raise PluginOperationError(f"Could not read plugin install metadata: {exc}") from exc
|
||||
if not isinstance(value, dict):
|
||||
@@ -1779,7 +1779,7 @@ def _read_manifest_info(d: Path, prefix: str):
|
||||
description = ""
|
||||
if yaml:
|
||||
try:
|
||||
with open(manifest_file, encoding="utf-8") as f:
|
||||
with open(manifest_file, encoding="utf-8-sig") as f:
|
||||
manifest = yaml.safe_load(f) or {}
|
||||
name = manifest.get("name", d.name)
|
||||
version = manifest.get("version", "")
|
||||
@@ -1823,7 +1823,7 @@ def _bundled_default_on(dir_path) -> bool:
|
||||
try:
|
||||
import yaml
|
||||
|
||||
with open(manifest_file, encoding="utf-8") as f:
|
||||
with open(manifest_file, encoding="utf-8-sig") as f:
|
||||
manifest = yaml.safe_load(f) or {}
|
||||
kind = str(manifest.get("kind", "standalone")).strip().lower()
|
||||
return kind in _BUNDLED_DEFAULT_ON_KINDS
|
||||
|
||||
@@ -95,26 +95,18 @@ DEFAULT_DIST_OWNED: Tuple[str, ...] = (
|
||||
)
|
||||
|
||||
# Paths that are NEVER part of a distribution. These are user-owned and are
|
||||
# protected on update. Must stay consistent with
|
||||
# ``profiles.py::_DEFAULT_EXPORT_EXCLUDE_ROOT`` plus the ``local/``
|
||||
# convention for user customizations.
|
||||
USER_OWNED_EXCLUDE: frozenset = frozenset({
|
||||
# Credentials & runtime secrets
|
||||
"auth.json", ".env",
|
||||
# Databases & runtime state
|
||||
"state.db", "state.db-shm", "state.db-wal",
|
||||
"hermes_state.db", "response_store.db",
|
||||
"response_store.db-shm", "response_store.db-wal",
|
||||
"gateway.pid", "gateway_state.json", "processes.json",
|
||||
"auth.lock", "active_profile", ".update_check",
|
||||
"errors.log", ".hermes_history",
|
||||
# User data
|
||||
"memories", "sessions", "logs", "plans", "workspace", "home",
|
||||
"image_cache", "audio_cache", "document_cache",
|
||||
"browser_screenshots", "checkpoints", "sandboxes",
|
||||
# protected on update. This is NOT a third copy of the exclude list: it is
|
||||
# the export-side authority (``profiles.py::DEFAULT_EXPORT_EXCLUDE_ROOT``)
|
||||
# plus the distribution-specific additions — user data dirs an export archive
|
||||
# may carry (memories, sessions, ...) but a distribution payload must never
|
||||
# shadow, and the ``local/`` convention for user customizations.
|
||||
from hermes_cli.profiles import DEFAULT_EXPORT_EXCLUDE_ROOT
|
||||
|
||||
USER_OWNED_EXCLUDE: frozenset = DEFAULT_EXPORT_EXCLUDE_ROOT | frozenset({
|
||||
# User data a distribution payload must never shadow (an export archive
|
||||
# intentionally carries these; an install/update must not overwrite)
|
||||
"memories", "sessions", "plans", "workspace", "home",
|
||||
"backups", "cache",
|
||||
# Infrastructure
|
||||
"hermes-agent", ".worktrees", "profiles", "bin", "node_modules",
|
||||
# User customization namespace
|
||||
"local",
|
||||
})
|
||||
|
||||
@@ -209,7 +209,12 @@ def _clone_all_copytree_ignore(source_dir: Path):
|
||||
# The default profile contains infrastructure (repo checkout, worktrees, DBs,
|
||||
# caches, binaries) that named profiles don't have. We exclude those so the
|
||||
# export is a portable, reasonable-size archive of actual profile data.
|
||||
_DEFAULT_EXPORT_EXCLUDE_ROOT = frozenset({
|
||||
# The ONE exclude-list authority for "infrastructure / runtime state that is
|
||||
# not profile data". The export path (``_default_export_ignore``) consumes it
|
||||
# directly; the distribution path (``profile_distribution.USER_OWNED_EXCLUDE``)
|
||||
# layers its distribution-specific additions on top of it. Keep new root
|
||||
# artifacts here — not in a downstream copy.
|
||||
_DEFAULT_EXPORT_EXCLUDE_ROOT = DEFAULT_EXPORT_EXCLUDE_ROOT = frozenset({
|
||||
# Infrastructure
|
||||
"hermes-agent", # repo checkout (multi-GB)
|
||||
".worktrees", # git worktrees
|
||||
|
||||
+6
-5
@@ -922,7 +922,7 @@ def _read_nearest_vercel_project(start: Path | None = None) -> dict[str, str]:
|
||||
if not project_file.exists():
|
||||
continue
|
||||
try:
|
||||
data = json.loads(project_file.read_text(encoding="utf-8"))
|
||||
data = json.loads(project_file.read_text(encoding="utf-8-sig"))
|
||||
except (OSError, json.JSONDecodeError):
|
||||
return {}
|
||||
if not isinstance(data, dict):
|
||||
@@ -1661,19 +1661,20 @@ def setup_terminal_backend(config: dict):
|
||||
print_info("Installing vercel SDK...")
|
||||
import subprocess
|
||||
|
||||
# Managed uv first: $HERMES_HOME/bin is never on PATH, so a bare
|
||||
# which() misses the uv Hermes installed. Bootstrapping one is
|
||||
# Managed uv first: the store is never on PATH, so a bare
|
||||
# which() misses the uv Hermes installed. Realizing one is
|
||||
# welcome here — this is the interactive setup wizard, already
|
||||
# mid-install, and the alternative tier is a pip that a `uv venv`
|
||||
# venv may not even have.
|
||||
from hermes_cli.managed_uv import ensure_uv
|
||||
import pm
|
||||
|
||||
uv_bin = ensure_uv()
|
||||
uv_bin, uv_env = pm.uv()
|
||||
if uv_bin:
|
||||
result = subprocess.run(
|
||||
[uv_bin, "pip", "install", "--python", sys.executable, "vercel"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
env=uv_env,
|
||||
)
|
||||
else:
|
||||
result = subprocess.run(
|
||||
|
||||
@@ -875,17 +875,16 @@ def _pip_install(
|
||||
(or the last failure for the caller to inspect).
|
||||
"""
|
||||
venv_root = Path(sys.executable).parent.parent
|
||||
uv_env = {**os.environ, "VIRTUAL_ENV": str(venv_root)}
|
||||
|
||||
# Managed uv first: $HERMES_HOME/bin is never on PATH, so a bare which()
|
||||
# Managed uv first: the store is never on PATH, so a bare which()
|
||||
# misses the uv Hermes installed and prefers a system one when both exist.
|
||||
# ensure_uv() rather than a pure lookup because this runs during setup,
|
||||
# pm realizes uv rather than a pure lookup because this runs during setup,
|
||||
# where installing uv is in scope — and tier 2 is a pip that the Windows
|
||||
# installer's `uv venv` does not seed, so failing to find uv here is the
|
||||
# difference between a working post-setup hook and "No module named pip".
|
||||
from hermes_cli.managed_uv import ensure_uv
|
||||
import pm
|
||||
|
||||
uv_bin = ensure_uv()
|
||||
uv_bin, uv_env = pm.uv(venv=venv_root)
|
||||
if uv_bin:
|
||||
try:
|
||||
result = subprocess.run(
|
||||
@@ -1434,7 +1433,7 @@ def _clear_stale_cua_install_lock() -> None:
|
||||
holder_pid = None
|
||||
info = lock_dir / "info"
|
||||
try:
|
||||
for line in info.read_text(encoding="utf-8", errors="replace").splitlines():
|
||||
for line in info.read_text(encoding="utf-8-sig", errors="replace").splitlines():
|
||||
if line.startswith("pid="):
|
||||
holder_pid = int(line.split("=", 1)[1].strip())
|
||||
break
|
||||
|
||||
@@ -0,0 +1,272 @@
|
||||
"""Truthful derived build-version metadata for user-facing Hermes displays.
|
||||
|
||||
``__version__`` remains the package/API version. This module adds a display
|
||||
suffix only when it can prove the number of commits since that release.
|
||||
|
||||
Resolution order:
|
||||
1. Install stamp (``install-stamp.json``) — written at build time by
|
||||
``scripts/write_install_stamp.py`` for every packager (Docker, Nix, and
|
||||
the desktop app). The stamp is authoritative
|
||||
for packaged builds.
|
||||
2. Live git — for source/dev installs with a ``.git`` directory.
|
||||
3. Unknown — no stamp and no git. The provenance is unknown.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Literal, cast
|
||||
|
||||
from hermes_cli import __release_date__, __version__
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class VersionInfo:
|
||||
base_version: str
|
||||
derived_version: str
|
||||
distance: int | None
|
||||
commit: str | None
|
||||
branch: str | None
|
||||
source: Literal["build", "ci", "docker", "fallback", "git", "local", "nix", "unknown"]
|
||||
dirty: bool = False
|
||||
commit_date: int | None = None
|
||||
distribution: Literal["docker", "nix", "desktop-app"] | None = None
|
||||
|
||||
|
||||
def _derived_version(base_version: str, distance: int | None, dirty: bool = False) -> str:
|
||||
if distance and distance > 0:
|
||||
return f"{base_version}+{distance}"
|
||||
if dirty and distance is None:
|
||||
return f"{base_version}+?"
|
||||
return base_version
|
||||
|
||||
|
||||
def _run_git(repo_dir: Path, *args: str) -> str | None:
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["git", *args], capture_output=True, text=True, timeout=3, cwd=str(repo_dir)
|
||||
)
|
||||
except (OSError, subprocess.SubprocessError):
|
||||
return None
|
||||
value = (result.stdout or "").strip()
|
||||
return value if result.returncode == 0 and value else None
|
||||
|
||||
|
||||
def _resolve_repo_dir() -> Path | None:
|
||||
"""Use the executing checkout before a profile's optional clone."""
|
||||
repo_dir = Path(__file__).parent.parent.resolve()
|
||||
if (repo_dir / ".git").exists():
|
||||
return repo_dir
|
||||
try:
|
||||
from hermes_constants import get_hermes_home
|
||||
|
||||
candidate = get_hermes_home() / "hermes-agent"
|
||||
if (candidate / ".git").exists():
|
||||
return candidate
|
||||
except Exception:
|
||||
pass
|
||||
return None
|
||||
|
||||
|
||||
def _parse_nonnegative(value: str | None) -> int | None:
|
||||
try:
|
||||
parsed = int(value or "")
|
||||
except ValueError:
|
||||
return None
|
||||
return parsed if parsed >= 0 else None
|
||||
|
||||
|
||||
# --- Install stamp reader ---------------------------------------------------
|
||||
|
||||
# The stamp file lives at the install root: beside the code in source
|
||||
# checkouts and Docker (which writes it to the project root), and in the
|
||||
# artifact's resources dir for sealed installs — whose processes carry
|
||||
# HERMES_INSTALL_ROOT (the Nix wrapper points it at the store path's
|
||||
# share/hermes-agent, where the stamp is baked). One resolution path for
|
||||
# every steward; no stamp-specific env override.
|
||||
_CODE_ROOT = Path(__file__).parent.parent
|
||||
|
||||
|
||||
def _resolve_stamp_file() -> Path | None:
|
||||
root = os.environ.get("HERMES_INSTALL_ROOT")
|
||||
base = Path(root) if root else _CODE_ROOT
|
||||
p = base / "install-stamp.json"
|
||||
return p if p.is_file() else None
|
||||
|
||||
|
||||
def _stamp_version_info() -> VersionInfo | None:
|
||||
"""Read provenance from a build-time install stamp."""
|
||||
stamp_file = _resolve_stamp_file()
|
||||
if stamp_file is None:
|
||||
return None
|
||||
try:
|
||||
raw = stamp_file.read_text(encoding="utf-8-sig")
|
||||
data = json.loads(raw)
|
||||
except (OSError, json.JSONDecodeError):
|
||||
return None
|
||||
|
||||
if not isinstance(data, dict) or "commit" not in data:
|
||||
return None
|
||||
|
||||
# updateMechanism is required in every stamp. A stamp without it means
|
||||
# the writing build lane must be fixed, not tolerated.
|
||||
if data.get("updateMechanism") not in ("self", "electron-updater", "external"):
|
||||
raise RuntimeError(
|
||||
f"install-stamp.json at {stamp_file} is missing a valid "
|
||||
"'updateMechanism' (one of self, electron-updater, external). The "
|
||||
"build lane that wrote this stamp must pass --update-mechanism to "
|
||||
"scripts/write_install_stamp.py (or bake the field directly)."
|
||||
)
|
||||
# A light artifact ships no Python runtime. Reading a light stamp from
|
||||
# a Python process means the artifact was mispackaged.
|
||||
if data.get("payload") == "light":
|
||||
raise RuntimeError(
|
||||
f"install-stamp.json at {stamp_file} marks this artifact as 'light' "
|
||||
"(no agent runtime). No Python process can legitimately run from a "
|
||||
"light artifact — this build is mispackaged."
|
||||
)
|
||||
|
||||
commit = data.get("commit") or None
|
||||
if not commit or set(commit) == {"0"}:
|
||||
# All-zero placeholder = fallback stamp, not real provenance.
|
||||
return None
|
||||
|
||||
base_version = data.get("baseVersion") or __version__
|
||||
display_version = data.get("displayVersion") or base_version
|
||||
distance = data.get("distance")
|
||||
if isinstance(distance, str):
|
||||
distance = _parse_nonnegative(distance)
|
||||
|
||||
# ``source`` describes build provenance, while ``distribution`` identifies
|
||||
# the package form users installed. Keep both facts intact for support.
|
||||
stamp_source = str(data.get("source") or "")
|
||||
source = (
|
||||
cast(Literal["build", "ci", "docker", "fallback", "git", "local", "nix", "unknown"], stamp_source)
|
||||
if stamp_source in {"ci", "docker", "fallback", "local", "nix"}
|
||||
else "build"
|
||||
)
|
||||
distribution = data.get("distribution")
|
||||
if distribution not in {"docker", "nix", "desktop-app"}:
|
||||
distribution = None
|
||||
|
||||
commit_date = data.get("commitDate")
|
||||
if not isinstance(commit_date, int):
|
||||
commit_date = None
|
||||
|
||||
return VersionInfo(
|
||||
base_version,
|
||||
display_version,
|
||||
distance if isinstance(distance, int) else None,
|
||||
commit,
|
||||
data.get("branch") or None,
|
||||
source,
|
||||
bool(data.get("dirty")),
|
||||
commit_date,
|
||||
distribution,
|
||||
)
|
||||
|
||||
|
||||
# --- Git provenance (source/dev installs) -----------------------------------
|
||||
|
||||
|
||||
def _git_version_info(repo_dir: Path) -> VersionInfo:
|
||||
commit = _run_git(repo_dir, "rev-parse", "HEAD")
|
||||
# A detached HEAD has no branch. Leave the field None: every formatter
|
||||
# already prints the commit separately and handles a missing branch.
|
||||
branch = _run_git(repo_dir, "branch", "--show-current")
|
||||
commit_date_raw = _run_git(repo_dir, "log", "-1", "--format=%ct", "HEAD")
|
||||
commit_date: int | None = None
|
||||
if commit_date_raw and commit_date_raw.isdigit():
|
||||
commit_date = int(commit_date_raw)
|
||||
try:
|
||||
# -uno: skip the untracked-file scan. This runs on the startup-banner
|
||||
# path, and a full working-tree walk costs real time on large or cold
|
||||
# checkouts. Same semantics as write_install_stamp.py.
|
||||
dirty_result = subprocess.run(
|
||||
["git", "status", "--porcelain", "-uno"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=3,
|
||||
cwd=str(repo_dir),
|
||||
)
|
||||
dirty = dirty_result.returncode == 0 and bool((dirty_result.stdout or "").strip())
|
||||
except (OSError, subprocess.SubprocessError):
|
||||
dirty = False
|
||||
|
||||
# New releases are SemVer tags. The release-date fallback lets existing
|
||||
# CalVer-tagged releases display a correct distance during the transition.
|
||||
distance = None
|
||||
for tag in (f"v{__version__}", f"v{__release_date__}"):
|
||||
raw_distance = _run_git(repo_dir, "rev-list", "--count", f"{tag}..HEAD")
|
||||
parsed_distance = _parse_nonnegative(raw_distance)
|
||||
if parsed_distance is not None:
|
||||
distance = parsed_distance
|
||||
break
|
||||
|
||||
return VersionInfo(
|
||||
__version__, _derived_version(__version__, distance, dirty), distance, commit, branch, "git", dirty, commit_date
|
||||
)
|
||||
|
||||
|
||||
# --- Cache + public API -----------------------------------------------------
|
||||
|
||||
_cached_version_info: VersionInfo | None = None
|
||||
|
||||
|
||||
def _reset_version_info_cache() -> None:
|
||||
"""Test-only cache reset."""
|
||||
global _cached_version_info
|
||||
_cached_version_info = None
|
||||
|
||||
|
||||
def get_version_info() -> VersionInfo:
|
||||
"""Return cached provenance from install stamp, git, or unknown."""
|
||||
global _cached_version_info
|
||||
if _cached_version_info is not None:
|
||||
return _cached_version_info
|
||||
|
||||
# 1. Install stamp (packaged builds: Docker, Nix)
|
||||
info = _stamp_version_info()
|
||||
|
||||
# 2. Live git (source/dev installs)
|
||||
if info is None:
|
||||
repo_dir = _resolve_repo_dir()
|
||||
if repo_dir is not None:
|
||||
info = _git_version_info(repo_dir)
|
||||
|
||||
# 3. Unknown — no stamp, no git
|
||||
if info is None:
|
||||
info = VersionInfo(__version__, __version__, None, None, None, "unknown")
|
||||
|
||||
_cached_version_info = info
|
||||
return info
|
||||
|
||||
|
||||
def get_code_identity(refresh: bool = False) -> dict:
|
||||
"""Return the running install's code identity as a flat dict.
|
||||
|
||||
Shape: ``{"sha": full sha | None, "short_sha": str | None, "version":
|
||||
base package version | None, "source": str}`` — what the update
|
||||
receipt, runtime inventory, and gateway status stamping consume.
|
||||
Backed by :func:`get_version_info` (install stamp first, live git
|
||||
second, unknown third), so every consumer shares one resolution
|
||||
policy. Returned dicts are fresh copies, never the shared cache.
|
||||
|
||||
``refresh=True`` drops the per-process cache first — the updater uses
|
||||
it to re-read identity after swapping the checkout out from under the
|
||||
running process.
|
||||
"""
|
||||
global _cached_version_info
|
||||
if refresh:
|
||||
_cached_version_info = None
|
||||
info = get_version_info()
|
||||
return {
|
||||
"sha": info.commit,
|
||||
"short_sha": info.commit[:8] if info.commit else None,
|
||||
"version": info.base_version,
|
||||
"source": info.source,
|
||||
}
|
||||
+15
-15
@@ -151,7 +151,7 @@ def _process_start_marker(pid: int) -> str:
|
||||
"""
|
||||
if sys.platform == "linux":
|
||||
try:
|
||||
stat_line = Path(f"/proc/{pid}/stat").read_text(encoding="utf-8")
|
||||
stat_line = Path(f"/proc/{pid}/stat").read_text(encoding="utf-8-sig")
|
||||
except FileNotFoundError as exc:
|
||||
raise ProcessLookupError(pid) from exc
|
||||
|
||||
@@ -3543,7 +3543,7 @@ def _read_or_create_install_id() -> Optional[str]:
|
||||
root = get_default_hermes_root()
|
||||
path = root / _INSTALL_ID_FILENAME
|
||||
try:
|
||||
existing = path.read_text(encoding="utf-8").strip().lower()
|
||||
existing = path.read_text(encoding="utf-8-sig").strip().lower()
|
||||
if _INSTALL_ID_RE.match(existing):
|
||||
return existing
|
||||
except FileNotFoundError:
|
||||
@@ -6091,7 +6091,7 @@ def _read_flat_json(provider: ProviderConfigSchema) -> Dict[str, Any]:
|
||||
if not path.exists():
|
||||
return {}
|
||||
try:
|
||||
data = json.loads(path.read_text(encoding="utf-8"))
|
||||
data = json.loads(path.read_text(encoding="utf-8-sig"))
|
||||
except Exception:
|
||||
_log.warning("Failed to read memory provider config from %s", path, exc_info=True)
|
||||
return {}
|
||||
@@ -6143,7 +6143,7 @@ def _honcho_read_sources() -> tuple[Dict[str, Any], str, Dict[str, Any]]:
|
||||
raw: Dict[str, Any] = {}
|
||||
if path.exists():
|
||||
try:
|
||||
loaded = json.loads(path.read_text(encoding="utf-8"))
|
||||
loaded = json.loads(path.read_text(encoding="utf-8-sig"))
|
||||
raw = loaded if isinstance(loaded, dict) else {}
|
||||
except Exception:
|
||||
_log.warning("Failed to read Honcho config from %s", path, exc_info=True)
|
||||
@@ -6253,7 +6253,7 @@ def _write_provider_honcho(provider: ProviderConfigSchema, values: Dict[str, str
|
||||
cfg: Dict[str, Any] = {}
|
||||
if path.exists():
|
||||
try:
|
||||
loaded = json.loads(path.read_text(encoding="utf-8"))
|
||||
loaded = json.loads(path.read_text(encoding="utf-8-sig"))
|
||||
cfg = loaded if isinstance(loaded, dict) else {}
|
||||
except Exception:
|
||||
_log.warning("Failed to read Honcho config from %s", path, exc_info=True)
|
||||
@@ -6786,7 +6786,7 @@ def _read_json_file(path: Path) -> Dict[str, Any]:
|
||||
if not path.exists():
|
||||
return {}
|
||||
try:
|
||||
data = json.loads(path.read_text(encoding="utf-8"))
|
||||
data = json.loads(path.read_text(encoding="utf-8-sig"))
|
||||
except Exception:
|
||||
_log.debug("Failed to read JSON config from %s", path, exc_info=True)
|
||||
return {}
|
||||
@@ -9749,7 +9749,7 @@ def _whatsapp_phone_from_identifier(value: Any) -> str | None:
|
||||
def _whatsapp_linked_account_from_session(session_path: Path) -> tuple[str | None, str | None, str | None]:
|
||||
creds_path = session_path / "creds.json"
|
||||
try:
|
||||
payload = json.loads(creds_path.read_text(encoding="utf-8"))
|
||||
payload = json.loads(creds_path.read_text(encoding="utf-8-sig"))
|
||||
except Exception:
|
||||
return None, None, None
|
||||
|
||||
@@ -13464,7 +13464,7 @@ def _profile_env_value(home: Path, key: str) -> str:
|
||||
env_path = home / ".env"
|
||||
if not env_path.is_file():
|
||||
return ""
|
||||
for line in env_path.read_text(encoding="utf-8").splitlines():
|
||||
for line in env_path.read_text(encoding="utf-8-sig").splitlines():
|
||||
line = line.strip()
|
||||
if not line or line.startswith("#") or "=" not in line:
|
||||
continue
|
||||
@@ -13628,7 +13628,7 @@ def _gateway_intentionally_stopped(profile: Optional[str]) -> bool:
|
||||
state_file = home / "gateway_state.json"
|
||||
if not state_file.exists():
|
||||
return False
|
||||
data = _json.loads(state_file.read_text(encoding="utf-8"))
|
||||
data = _json.loads(state_file.read_text(encoding="utf-8-sig"))
|
||||
if not isinstance(data, dict):
|
||||
return False
|
||||
return data.get("desired_state") == "stopped"
|
||||
@@ -15740,7 +15740,7 @@ async def get_config_raw(profile: Optional[str] = None):
|
||||
path = get_config_path()
|
||||
if not path.exists():
|
||||
return {"yaml": "", "path": str(path)}
|
||||
return {"yaml": path.read_text(encoding="utf-8"), "path": str(path)}
|
||||
return {"yaml": path.read_text(encoding="utf-8-sig"), "path": str(path)}
|
||||
|
||||
return await asyncio.to_thread(_run)
|
||||
|
||||
@@ -16901,7 +16901,7 @@ def _active_session_file_for_channel(app: "FastAPI", channel: str) -> Path:
|
||||
|
||||
def _read_active_session_file(path: Path) -> Optional[str]:
|
||||
try:
|
||||
data = json.loads(path.read_text(encoding="utf-8"))
|
||||
data = json.loads(path.read_text(encoding="utf-8-sig"))
|
||||
except (OSError, json.JSONDecodeError):
|
||||
return None
|
||||
|
||||
@@ -17974,7 +17974,7 @@ def mount_spa(application: FastAPI):
|
||||
auth scheme for /api/pty and /api/ws (ticket vs token).
|
||||
"""
|
||||
try:
|
||||
html = _index_path.read_text(encoding="utf-8")
|
||||
html = _index_path.read_text(encoding="utf-8-sig")
|
||||
except OSError:
|
||||
# The dist dir existed at mount time but index.html is missing or
|
||||
# unreadable now (partial build, wiped dist, permissions). Without
|
||||
@@ -18044,7 +18044,7 @@ def mount_spa(application: FastAPI):
|
||||
):
|
||||
return JSONResponse({"error": "not found"}, status_code=404)
|
||||
prefix = _normalise_prefix(request.headers.get("x-forwarded-prefix"))
|
||||
css = css_path.read_text(encoding="utf-8")
|
||||
css = css_path.read_text(encoding="utf-8-sig")
|
||||
if prefix:
|
||||
for asset_dir in ("/fonts/", "/fonts-terminal/", "/ds-assets/", "/assets/"):
|
||||
css = css.replace(f"url({asset_dir}", f"url({prefix}{asset_dir}")
|
||||
@@ -18357,7 +18357,7 @@ def _discover_user_themes() -> list:
|
||||
result = []
|
||||
for f in sorted(themes_dir.glob("*.yaml")):
|
||||
try:
|
||||
data = yaml.safe_load(f.read_text(encoding="utf-8"))
|
||||
data = yaml.safe_load(f.read_text(encoding="utf-8-sig"))
|
||||
except Exception:
|
||||
continue
|
||||
normalised = _normalise_theme_definition(data)
|
||||
@@ -18569,7 +18569,7 @@ def _discover_dashboard_plugins() -> list:
|
||||
if not manifest_file.exists():
|
||||
continue
|
||||
try:
|
||||
data = json.loads(manifest_file.read_text(encoding="utf-8"))
|
||||
data = json.loads(manifest_file.read_text(encoding="utf-8-sig"))
|
||||
name = data.get("name", child.name)
|
||||
if name in seen_names:
|
||||
continue
|
||||
|
||||
+1
-1
@@ -32,7 +32,7 @@
|
||||
let
|
||||
repoRoot = ./..;
|
||||
|
||||
npm12 = callPackage ./npm-12-0-2.nix { };
|
||||
npm12 = callPackage ./npm-pinned.nix { };
|
||||
node_gyp_11_4_0 = callPackage ./node-gyp-11-4-0.nix { };
|
||||
nodejs_26_npm_12 = symlinkJoin {
|
||||
name = "nodejs-26-npm-12";
|
||||
|
||||
@@ -1,29 +0,0 @@
|
||||
{
|
||||
stdenv,
|
||||
makeWrapper,
|
||||
fetchurl,
|
||||
nodejs_26,
|
||||
}:
|
||||
stdenv.mkDerivation rec {
|
||||
pname = "npm";
|
||||
version = "12.0.2";
|
||||
|
||||
src = fetchurl {
|
||||
url = "https://registry.npmjs.org/npm/-/npm-${version}.tgz";
|
||||
hash = "sha256-XbuGxx0HoZV/LpBzQJLdali9zZ68LY1ByhxuaiHTZOE=";
|
||||
};
|
||||
|
||||
nativeBuildInputs = [ makeWrapper ];
|
||||
dontBuild = true;
|
||||
|
||||
installPhase = ''
|
||||
mkdir -p $out/lib/npm12
|
||||
cp -r . $out/lib/npm12/
|
||||
mkdir -p $out/bin
|
||||
|
||||
makeWrapper ${nodejs_26}/bin/node $out/bin/npm \
|
||||
--add-flags "$out/lib/npm12/bin/npm-cli.js"
|
||||
makeWrapper ${nodejs_26}/bin/node $out/bin/npx \
|
||||
--add-flags "$out/lib/npm12/bin/npx-cli.js"
|
||||
'';
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
{
|
||||
stdenv,
|
||||
makeWrapper,
|
||||
fetchurl,
|
||||
nodejs_26,
|
||||
}:
|
||||
let
|
||||
# pm/lock.json is the one pin table. Every artifact carries its resolved
|
||||
# url + sha256, so nix consumes it as pure data — no version or url
|
||||
# knowledge lives on the nix side.
|
||||
lock = builtins.fromJSON (builtins.readFile ../pm/lock.json);
|
||||
pin = lock.packages.npm;
|
||||
in
|
||||
stdenv.mkDerivation {
|
||||
pname = "npm";
|
||||
version = pin.version;
|
||||
|
||||
src = fetchurl {
|
||||
url = pin.artifacts.any.url;
|
||||
sha256 = pin.artifacts.any.sha256;
|
||||
};
|
||||
|
||||
nativeBuildInputs = [ makeWrapper ];
|
||||
dontBuild = true;
|
||||
|
||||
installPhase = ''
|
||||
mkdir -p $out/lib/npm
|
||||
cp -r . $out/lib/npm/
|
||||
mkdir -p $out/bin
|
||||
|
||||
makeWrapper ${nodejs_26}/bin/node $out/bin/npm \
|
||||
--add-flags "$out/lib/npm/bin/npm-cli.js"
|
||||
makeWrapper ${nodejs_26}/bin/node $out/bin/npx \
|
||||
--add-flags "$out/lib/npm/bin/npx-cli.js"
|
||||
'';
|
||||
}
|
||||
@@ -0,0 +1,69 @@
|
||||
{
|
||||
lib,
|
||||
stdenv,
|
||||
fetchurl,
|
||||
unzip,
|
||||
}:
|
||||
# The thin translation layer: pm/lock.json -> one fetched, unpacked
|
||||
# derivation per package for this nix system. Nothing here knows versions,
|
||||
# urls, or hashes — the lockfile is the complete machine interface, the
|
||||
# same one `pm install` and `pm bundle` consume.
|
||||
#
|
||||
# Consumers pick what they need: (callPackage ./pm-packages.nix { }).ripgrep
|
||||
# Packages with no artifact for this system are simply absent.
|
||||
let
|
||||
lock = builtins.fromJSON (builtins.readFile ../pm/lock.json);
|
||||
|
||||
target =
|
||||
let
|
||||
arch = if stdenv.hostPlatform.isAarch64 then "arm64" else "x64";
|
||||
os =
|
||||
if stdenv.hostPlatform.isDarwin then "darwin"
|
||||
else if stdenv.hostPlatform.isLinux then "linux"
|
||||
else "win32";
|
||||
in
|
||||
"${os}-${arch}";
|
||||
|
||||
# A target pins one artifact or a list of them (a runtime split across
|
||||
# archives that must land in one directory). Normalize to a list.
|
||||
artifactsFor =
|
||||
pin:
|
||||
let
|
||||
found = pin.artifacts.${target} or pin.artifacts.any or null;
|
||||
in
|
||||
if found == null then null else if builtins.isList found then found else [ found ];
|
||||
|
||||
derive = name: pin: artifacts:
|
||||
stdenv.mkDerivation {
|
||||
pname = name;
|
||||
version = pin.version;
|
||||
|
||||
srcs = map (artifact: fetchurl {
|
||||
url = artifact.url;
|
||||
sha256 = artifact.sha256;
|
||||
}) artifacts;
|
||||
|
||||
# pm's store publishes the unpacked tree; mirror that shape. Several
|
||||
# archives unpack over one another into the same root, exactly as
|
||||
# pm merges them into one store entry.
|
||||
sourceRoot = ".";
|
||||
nativeBuildInputs =
|
||||
lib.optional (lib.any (a: lib.hasSuffix ".zip" a.url) artifacts) unzip;
|
||||
dontBuild = true;
|
||||
dontConfigure = true;
|
||||
|
||||
installPhase = ''
|
||||
mkdir -p $out
|
||||
cp -r . $out/
|
||||
'';
|
||||
};
|
||||
in
|
||||
lib.filterAttrs (_: v: v != null) (
|
||||
lib.mapAttrs (
|
||||
name: pin:
|
||||
let
|
||||
artifacts = artifactsFor pin;
|
||||
in
|
||||
if artifacts == null then null else derive name pin artifacts
|
||||
) lock.packages
|
||||
)
|
||||
@@ -71,7 +71,7 @@ class _RecallResult:
|
||||
|
||||
_DEFAULT_API_URL = "https://api.hindsight.vectorize.io"
|
||||
_DEFAULT_LOCAL_URL = "http://localhost:8888"
|
||||
# Keep in sync with tools/lazy_deps.py ("memory.hindsight") and plugin.yaml.
|
||||
# Keep in sync with the pyproject "hindsight" extra and plugin.yaml.
|
||||
_MIN_CLIENT_VERSION = "0.6.1"
|
||||
_DEFAULT_TIMEOUT = 120 # seconds — cloud API can take 30-40s per request
|
||||
_DEFAULT_IDLE_TIMEOUT = 300 # seconds — Hindsight embedded daemon default
|
||||
@@ -204,8 +204,8 @@ def _local_runtime_hint(reason: str | None) -> str:
|
||||
def _ensure_cloud_client_dependency() -> None:
|
||||
"""Install the Hindsight cloud client lazily before importing it."""
|
||||
try:
|
||||
from tools.lazy_deps import ensure as _lazy_ensure
|
||||
_lazy_ensure("memory.hindsight", prompt=False)
|
||||
from pm import ensure_import as _lazy_ensure
|
||||
_lazy_ensure("hindsight")
|
||||
except ImportError:
|
||||
pass
|
||||
except Exception as exc:
|
||||
@@ -431,7 +431,7 @@ def _load_config() -> dict:
|
||||
profile_path = get_hermes_home() / "hindsight" / "config.json"
|
||||
if profile_path.exists():
|
||||
try:
|
||||
return json.loads(profile_path.read_text(encoding="utf-8"))
|
||||
return json.loads(profile_path.read_text(encoding="utf-8-sig"))
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
@@ -439,7 +439,7 @@ def _load_config() -> dict:
|
||||
legacy_path = Path.home() / ".hindsight" / "config.json"
|
||||
if legacy_path.exists():
|
||||
try:
|
||||
return json.loads(legacy_path.read_text(encoding="utf-8"))
|
||||
return json.loads(legacy_path.read_text(encoding="utf-8-sig"))
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
@@ -937,7 +937,7 @@ class HindsightMemoryProvider(MemoryProvider):
|
||||
existing = {}
|
||||
if config_path.exists():
|
||||
try:
|
||||
existing = json.loads(config_path.read_text(encoding="utf-8"))
|
||||
existing = json.loads(config_path.read_text(encoding="utf-8-sig"))
|
||||
except Exception:
|
||||
pass
|
||||
existing.update(values)
|
||||
@@ -981,15 +981,10 @@ class HindsightMemoryProvider(MemoryProvider):
|
||||
provider_config["mode"] = mode
|
||||
env_writes: dict = {}
|
||||
|
||||
# Step 2: Install/upgrade deps for selected mode
|
||||
cloud_dep = f"hindsight-client>={_MIN_CLIENT_VERSION}"
|
||||
local_dep = "hindsight-all"
|
||||
if mode == "local_embedded":
|
||||
deps_to_install = [local_dep]
|
||||
elif mode == "local_external":
|
||||
deps_to_install = [cloud_dep]
|
||||
else:
|
||||
deps_to_install = [cloud_dep]
|
||||
# Step 2: Install/upgrade deps for selected mode. local_embedded
|
||||
# installs hindsight-all itself (protobuf conflict bars it from the
|
||||
# venv extras; side-venv treatment is planned).
|
||||
extras_needed = [] if mode == "local_embedded" else ["hindsight"]
|
||||
|
||||
llm_provider = ""
|
||||
if mode == "local_embedded":
|
||||
@@ -1013,18 +1008,41 @@ class HindsightMemoryProvider(MemoryProvider):
|
||||
provider_config["llm_provider"] = llm_provider
|
||||
|
||||
print("\n Checking dependencies...")
|
||||
# Environment-aware install: sealed hosted venvs redirect to the durable
|
||||
# data-volume target instead of writing to /opt/hermes (NS-605).
|
||||
from tools.lazy_deps import install_specs
|
||||
if mode == "local_embedded":
|
||||
# hindsight-all cannot live in the hermes venv (its protobuf
|
||||
# floor conflicts with mem0/modal; side-venv treatment is
|
||||
# planned). Keep the direct install for this mode only.
|
||||
import subprocess as _sp
|
||||
|
||||
outcome = install_specs(deps_to_install, timeout=120)
|
||||
if outcome.ok:
|
||||
print(" ✓ Dependencies up to date")
|
||||
elif outcome.blocked:
|
||||
print(f" ⚠ Cannot install dependencies: {outcome.reason}")
|
||||
import pm as _pm
|
||||
|
||||
uv_bin, uv_env = _pm.uv(venv=Path(sys.prefix))
|
||||
cmd = (
|
||||
[uv_bin, "pip", "install"]
|
||||
if uv_bin
|
||||
else [sys.executable, "-m", "pip", "install"]
|
||||
)
|
||||
proc = _sp.run(
|
||||
[*cmd, "hindsight-all"],
|
||||
env=uv_env if uv_bin else None,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=600,
|
||||
)
|
||||
if proc.returncode == 0:
|
||||
print(" ✓ Dependencies up to date")
|
||||
else:
|
||||
print(f" ⚠ Install failed:\n{(proc.stderr or '').strip()[:400]}")
|
||||
print(" Run manually: uv pip install hindsight-all")
|
||||
else:
|
||||
print(f" ⚠ Install failed:\n{(outcome.stderr or '').strip()}")
|
||||
print(f" Run manually: uv pip install --python {sys.executable} {' '.join(deps_to_install)}")
|
||||
import pm
|
||||
|
||||
try:
|
||||
pm.sync_venv(extras_needed, explicit=True)
|
||||
print(" ✓ Dependencies up to date")
|
||||
except Exception as exc:
|
||||
print(f" ⚠ Install failed: {exc}")
|
||||
print(" Run manually: hermes pm install")
|
||||
|
||||
# Step 3: Mode-specific config
|
||||
if mode == "cloud":
|
||||
@@ -1142,7 +1160,7 @@ class HindsightMemoryProvider(MemoryProvider):
|
||||
materialized_config = dict(provider_config)
|
||||
config_path = Path(hermes_home) / "hindsight" / "config.json"
|
||||
try:
|
||||
materialized_config = json.loads(config_path.read_text(encoding="utf-8"))
|
||||
materialized_config = json.loads(config_path.read_text(encoding="utf-8-sig"))
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
@@ -1241,8 +1259,8 @@ class HindsightMemoryProvider(MemoryProvider):
|
||||
+ (f": {reason}" if reason else "")
|
||||
)
|
||||
try:
|
||||
from tools.lazy_deps import ensure as _lazy_ensure
|
||||
_lazy_ensure("memory.hindsight", prompt=False)
|
||||
from pm import ensure_import as _lazy_ensure
|
||||
_lazy_ensure("hindsight")
|
||||
except ImportError:
|
||||
pass
|
||||
except Exception as _e:
|
||||
@@ -1608,18 +1626,14 @@ class HindsightMemoryProvider(MemoryProvider):
|
||||
if Version(installed) < Version(_MIN_CLIENT_VERSION):
|
||||
logger.warning("hindsight-client %s is outdated (need >=%s), attempting upgrade...",
|
||||
installed, _MIN_CLIENT_VERSION)
|
||||
# Environment-aware install: sealed hosted venvs redirect to the
|
||||
# durable data-volume target instead of /opt/hermes (NS-605).
|
||||
from tools.lazy_deps import install_specs
|
||||
outcome = install_specs([f"hindsight-client>={_MIN_CLIENT_VERSION}"], timeout=120)
|
||||
if outcome.ok:
|
||||
logger.info("hindsight-client upgraded to >=%s", _MIN_CLIENT_VERSION)
|
||||
elif outcome.blocked:
|
||||
logger.warning("Auto-upgrade unavailable: %s. Run: uv pip install 'hindsight-client>=%s'",
|
||||
outcome.reason, _MIN_CLIENT_VERSION)
|
||||
else:
|
||||
logger.warning("Auto-upgrade failed: %s. Run: uv pip install 'hindsight-client>=%s'",
|
||||
(outcome.stderr or "").strip() or "install error", _MIN_CLIENT_VERSION)
|
||||
# uv.lock owns the pin; an upgrade is a resync of the extra.
|
||||
import pm
|
||||
|
||||
try:
|
||||
pm.sync_venv(["hindsight"])
|
||||
logger.info("hindsight-client resynced against uv.lock")
|
||||
except Exception as exc:
|
||||
logger.warning("Auto-upgrade unavailable: %s. Run: hermes pm install", exc)
|
||||
except Exception:
|
||||
pass # packaging not available or other issue — proceed anyway
|
||||
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
name: hindsight
|
||||
version: 1.0.0
|
||||
description: "Hindsight — long-term memory with knowledge graph, entity resolution, and multi-strategy retrieval."
|
||||
pip_dependencies:
|
||||
- "hindsight-client>=0.6.1"
|
||||
extra: hindsight
|
||||
requires_env: []
|
||||
hooks:
|
||||
- on_session_end
|
||||
@@ -264,7 +264,7 @@ def _read_config() -> dict:
|
||||
path = _config_path()
|
||||
if path.exists():
|
||||
try:
|
||||
return json.loads(path.read_text(encoding="utf-8"))
|
||||
return json.loads(path.read_text(encoding="utf-8-sig"))
|
||||
except Exception:
|
||||
pass
|
||||
return {}
|
||||
@@ -516,20 +516,15 @@ def _ensure_sdk_installed() -> bool:
|
||||
return False
|
||||
|
||||
print(" Installing honcho-ai...", flush=True)
|
||||
# Environment-aware install: sealed hosted venvs redirect to the durable
|
||||
# data-volume target instead of writing to /opt/hermes (NS-605).
|
||||
from tools.lazy_deps import install_specs
|
||||
import pm
|
||||
|
||||
result = install_specs(["honcho-ai==2.2.0"])
|
||||
if result.ok:
|
||||
try:
|
||||
pm.sync_venv(["honcho"], explicit=True)
|
||||
print(" Installed.\n")
|
||||
return True
|
||||
elif result.blocked:
|
||||
print(f" Cannot install: {result.reason}\n")
|
||||
return False
|
||||
else:
|
||||
print(f" Install failed:\n{(result.stderr or '').strip()}")
|
||||
print(" Run manually: uv pip install 'honcho-ai==2.2.0'\n")
|
||||
except Exception as exc:
|
||||
print(f" Install failed: {exc}")
|
||||
print(" Run manually: hermes pm install\n")
|
||||
return False
|
||||
|
||||
|
||||
@@ -1587,7 +1582,7 @@ def cmd_identity(args) -> None:
|
||||
print(f" File not found: {p}\n")
|
||||
return
|
||||
|
||||
content = p.read_text(encoding="utf-8").strip()
|
||||
content = p.read_text(encoding="utf-8-sig").strip()
|
||||
if not content:
|
||||
print(f" File is empty: {p}\n")
|
||||
return
|
||||
@@ -1767,7 +1762,7 @@ def cmd_migrate(args) -> None:
|
||||
session_key = hcfg.resolve_session_name()
|
||||
mgr.get_or_create(session_key)
|
||||
for f in agent_files:
|
||||
content = f.read_text(encoding="utf-8").strip()
|
||||
content = f.read_text(encoding="utf-8-sig").strip()
|
||||
if content:
|
||||
ok = mgr.seed_ai_identity(session_key, content, source=f.name)
|
||||
status = "seeded" if ok else "failed"
|
||||
|
||||
@@ -102,7 +102,7 @@ def resolve_active_host() -> str:
|
||||
try:
|
||||
path = resolve_config_path()
|
||||
if path.exists():
|
||||
raw = json.loads(path.read_text(encoding="utf-8"))
|
||||
raw = json.loads(path.read_text(encoding="utf-8-sig"))
|
||||
default_host = str(raw.get("defaultHost", "")).strip()
|
||||
if default_host:
|
||||
return default_host
|
||||
@@ -552,7 +552,7 @@ class HonchoClientConfig:
|
||||
return cls.from_env(host=resolved_host)
|
||||
|
||||
try:
|
||||
raw = json.loads(path.read_text(encoding="utf-8"))
|
||||
raw = json.loads(path.read_text(encoding="utf-8-sig"))
|
||||
except (json.JSONDecodeError, OSError) as e:
|
||||
logger.warning("Failed to read %s: %s, falling back to env", path, e)
|
||||
return cls.from_env(host=resolved_host)
|
||||
@@ -1004,7 +1004,7 @@ def _credential_fingerprint(config: HonchoClientConfig | None) -> str:
|
||||
# the supported path for background threads).
|
||||
path = resolve_config_path()
|
||||
if path.exists():
|
||||
raw = json.loads(path.read_text(encoding="utf-8"))
|
||||
raw = json.loads(path.read_text(encoding="utf-8-sig"))
|
||||
block = _host_block(raw, resolve_active_host())
|
||||
oauth_block = block.get("oauth")
|
||||
if isinstance(oauth_block, dict) and oauth_block.get("refreshToken"):
|
||||
@@ -1124,7 +1124,7 @@ def _honcho_json_timeout() -> float | None:
|
||||
|
||||
timeout = None
|
||||
if mtime_ns != -1:
|
||||
raw = json.loads(path.read_text(encoding="utf-8"))
|
||||
raw = json.loads(path.read_text(encoding="utf-8-sig"))
|
||||
host_block = _host_block(raw, resolve_active_host())
|
||||
timeout = _resolve_optional_float(
|
||||
host_block.get("timeout"),
|
||||
@@ -1255,14 +1255,14 @@ def get_honcho_client(config: HonchoClientConfig | None = None) -> Honcho:
|
||||
def _build() -> "Honcho":
|
||||
# Lazy dependency failures fall through to the canonical import error.
|
||||
try:
|
||||
from tools.lazy_deps import FeatureUnavailable, ensure as _lazy_ensure
|
||||
_lazy_ensure("memory.honcho", prompt=False)
|
||||
from pm import ensure_import as _lazy_ensure
|
||||
_lazy_ensure("honcho")
|
||||
except ImportError:
|
||||
# lazy_deps module missing — fall through to the raw import below.
|
||||
# pm module missing — fall through to the raw import below.
|
||||
pass
|
||||
except Exception:
|
||||
# FeatureUnavailable or unexpected error. Don't crash here; let the
|
||||
# actual import attempt produce the canonical error message.
|
||||
# Lazy-install failure or unexpected error. Don't crash here; let
|
||||
# the actual import attempt produce the canonical error message.
|
||||
pass
|
||||
|
||||
try:
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
name: honcho
|
||||
version: 1.0.0
|
||||
description: "Honcho AI-native memory — cross-session user modeling with dialectic Q&A, semantic search, and persistent conclusions."
|
||||
pip_dependencies:
|
||||
- honcho-ai
|
||||
extra: honcho
|
||||
hooks:
|
||||
- on_session_end
|
||||
@@ -101,7 +101,7 @@ def _load_config() -> dict:
|
||||
config_path = get_hermes_home() / "mem0.json"
|
||||
if config_path.exists():
|
||||
try:
|
||||
file_cfg = json.loads(config_path.read_text(encoding="utf-8"))
|
||||
file_cfg = json.loads(config_path.read_text(encoding="utf-8-sig"))
|
||||
config.update({k: v for k, v in file_cfg.items()
|
||||
if v is not None and v != ""})
|
||||
except Exception:
|
||||
@@ -241,7 +241,7 @@ class Mem0MemoryProvider(MemoryProvider):
|
||||
existing = {}
|
||||
if config_path.exists():
|
||||
try:
|
||||
existing = json.loads(config_path.read_text(encoding="utf-8"))
|
||||
existing = json.loads(config_path.read_text(encoding="utf-8-sig"))
|
||||
except Exception:
|
||||
pass
|
||||
existing.update(values)
|
||||
@@ -271,8 +271,8 @@ class Mem0MemoryProvider(MemoryProvider):
|
||||
# target. On failure we fall through so the import inside the backend
|
||||
# produces the canonical error, captured below.
|
||||
try:
|
||||
from tools.lazy_deps import ensure as _lazy_ensure
|
||||
_lazy_ensure("memory.mem0", prompt=False)
|
||||
from pm import ensure_import as _lazy_ensure
|
||||
_lazy_ensure("mem0")
|
||||
except ImportError:
|
||||
pass
|
||||
except Exception:
|
||||
|
||||
@@ -224,7 +224,7 @@ def _save_mem0_json(hermes_home: str, data: dict) -> None:
|
||||
existing = {}
|
||||
if config_path.exists():
|
||||
try:
|
||||
existing = json.loads(config_path.read_text(encoding="utf-8"))
|
||||
existing = json.loads(config_path.read_text(encoding="utf-8-sig"))
|
||||
except Exception:
|
||||
pass
|
||||
existing.update(data)
|
||||
@@ -248,7 +248,7 @@ def _setup_platform(hermes_home: str, config: dict, flags: dict[str, str]) -> No
|
||||
config_path = Path(hermes_home) / "mem0.json"
|
||||
if config_path.exists():
|
||||
try:
|
||||
existing_config = json.loads(config_path.read_text(encoding="utf-8"))
|
||||
existing_config = json.loads(config_path.read_text(encoding="utf-8-sig"))
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
@@ -369,7 +369,7 @@ def _setup_selfhosted(hermes_home: str, config: dict, flags: dict[str, str]) ->
|
||||
config_path = Path(hermes_home) / "mem0.json"
|
||||
if config_path.exists():
|
||||
try:
|
||||
existing_config = json.loads(config_path.read_text(encoding="utf-8"))
|
||||
existing_config = json.loads(config_path.read_text(encoding="utf-8-sig"))
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
@@ -853,32 +853,27 @@ def _setup_oss_interactive(hermes_home: str, config: dict) -> None:
|
||||
|
||||
|
||||
def _install_provider_deps(llm_id: str, embedder_id: str, vector_id: str) -> None:
|
||||
"""Install all optional pip deps for selected providers."""
|
||||
"""Point at the pip deps the selected OSS backends need.
|
||||
|
||||
These are third-party backend SDKs (ollama, qdrant-client, ...), not
|
||||
hermes dependencies — pm does not install arbitrary specs into the
|
||||
hermes venv. Print the exact command instead."""
|
||||
deps: set[str] = set()
|
||||
for registry, pid in [(LLM_PROVIDERS, llm_id), (EMBEDDER_PROVIDERS, embedder_id),
|
||||
(VECTOR_PROVIDERS, vector_id)]:
|
||||
dep = registry.get(pid, {}).get("pip_dep")
|
||||
if dep:
|
||||
deps.add(dep)
|
||||
missing = []
|
||||
for dep in sorted(deps):
|
||||
try:
|
||||
print(f" Installing {dep}...")
|
||||
# Environment-aware install: sealed hosted venvs redirect to the
|
||||
# durable data-volume target instead of /opt/hermes (NS-605).
|
||||
from tools.lazy_deps import install_specs
|
||||
import importlib.util
|
||||
|
||||
outcome = install_specs([dep], timeout=60)
|
||||
if outcome.ok:
|
||||
print(f" ✓ Installed {dep}")
|
||||
elif outcome.blocked:
|
||||
print(f" Warning: cannot install {dep}: {outcome.reason}")
|
||||
else:
|
||||
print(f" Warning: Could not install {dep}. Install manually: uv pip install {dep}")
|
||||
except Exception:
|
||||
print(f" Warning: Could not install {dep}. Install manually: uv pip install {dep}")
|
||||
if deps:
|
||||
import importlib
|
||||
importlib.invalidate_caches()
|
||||
if importlib.util.find_spec(dep.replace("-", "_").split("[")[0]) is None:
|
||||
missing.append(dep)
|
||||
if missing:
|
||||
print("\n The selected backends need extra packages:")
|
||||
print(f" uv pip install {' '.join(missing)}")
|
||||
print(" Run that inside the hermes venv, then re-run setup.")
|
||||
|
||||
|
||||
def _check_qdrant_path(path: str) -> tuple[bool, str]:
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
name: mem0
|
||||
version: 1.3.0
|
||||
description: "Mem0 — server-side LLM fact extraction with semantic search, automatic deduplication, and opt-in reranking (platform mode)."
|
||||
pip_dependencies:
|
||||
- mem0ai>=2.0.10,<3
|
||||
extra: mem0
|
||||
@@ -1,8 +1,6 @@
|
||||
name: openviking
|
||||
version: 2.0.0
|
||||
description: "OpenViking context database — session-managed memory with automatic extraction, tiered retrieval, and filesystem-style knowledge browsing."
|
||||
pip_dependencies:
|
||||
- httpx
|
||||
requires_env: []
|
||||
hooks:
|
||||
- on_session_end
|
||||
@@ -1,7 +1,5 @@
|
||||
name: retaindb
|
||||
version: 1.0.0
|
||||
description: "RetainDB — cloud memory API with hybrid search and 7 memory types."
|
||||
pip_dependencies:
|
||||
- requests
|
||||
requires_env:
|
||||
- RETAINDB_API_KEY
|
||||
@@ -115,7 +115,7 @@ def _load_supermemory_config(hermes_home: str) -> dict:
|
||||
config_path = Path(hermes_home) / "supermemory.json"
|
||||
if config_path.exists():
|
||||
try:
|
||||
raw = json.loads(config_path.read_text(encoding="utf-8"))
|
||||
raw = json.loads(config_path.read_text(encoding="utf-8-sig"))
|
||||
if isinstance(raw, dict):
|
||||
config.update({k: v for k, v in raw.items() if v is not None})
|
||||
except Exception:
|
||||
@@ -162,7 +162,7 @@ def _save_supermemory_config(values: dict, hermes_home: str) -> None:
|
||||
existing = {}
|
||||
if config_path.exists():
|
||||
try:
|
||||
raw = json.loads(config_path.read_text(encoding="utf-8"))
|
||||
raw = json.loads(config_path.read_text(encoding="utf-8-sig"))
|
||||
if isinstance(raw, dict):
|
||||
existing = raw
|
||||
except Exception:
|
||||
@@ -286,8 +286,8 @@ class _SupermemoryClient:
|
||||
# fall through so the raw import below produces the canonical
|
||||
# ImportError message.
|
||||
try:
|
||||
from tools.lazy_deps import ensure as _lazy_ensure
|
||||
_lazy_ensure("memory.supermemory", prompt=False)
|
||||
from pm import ensure_import as _lazy_ensure
|
||||
_lazy_ensure("supermemory")
|
||||
except ImportError:
|
||||
pass
|
||||
except Exception:
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
name: supermemory
|
||||
version: 1.0.1
|
||||
description: "Supermemory semantic long-term memory with profile recall, semantic search, explicit memory tools, and session ingest."
|
||||
pip_dependencies:
|
||||
- supermemory
|
||||
extra: supermemory
|
||||
@@ -191,8 +191,8 @@ def ensure_dingtalk_deps() -> bool:
|
||||
if DINGTALK_STREAM_AVAILABLE and HTTPX_AVAILABLE:
|
||||
return True
|
||||
try:
|
||||
from tools.lazy_deps import ensure as _lazy_ensure
|
||||
_lazy_ensure("platform.dingtalk", prompt=False)
|
||||
from pm import ensure_import as _lazy_ensure
|
||||
_lazy_ensure("dingtalk")
|
||||
except Exception:
|
||||
return False
|
||||
try:
|
||||
|
||||
@@ -356,7 +356,7 @@ class _DiscordNonConversationalMessageTracker:
|
||||
if not path.exists():
|
||||
return []
|
||||
try:
|
||||
data = json.loads(path.read_text(encoding="utf-8"))
|
||||
data = json.loads(path.read_text(encoding="utf-8-sig"))
|
||||
if isinstance(data, list):
|
||||
return [str(message_id) for message_id in data if str(message_id).strip()]
|
||||
except Exception:
|
||||
@@ -493,7 +493,7 @@ def discord_deps_present() -> bool:
|
||||
def check_discord_requirements() -> bool:
|
||||
"""Check if Discord dependencies are available.
|
||||
|
||||
Lazy-installs discord.py via ``tools.lazy_deps.ensure("platform.discord")``
|
||||
Lazy-installs discord.py via ``pm.ensure_import("discord")``
|
||||
on first call if not present. After successful install, re-binds module
|
||||
globals so ``DISCORD_AVAILABLE`` becomes True.
|
||||
"""
|
||||
@@ -501,8 +501,8 @@ def check_discord_requirements() -> bool:
|
||||
if DISCORD_AVAILABLE:
|
||||
return True
|
||||
try:
|
||||
from tools.lazy_deps import ensure as _lazy_ensure
|
||||
_lazy_ensure("platform.discord", prompt=False)
|
||||
from pm import ensure_import as _lazy_ensure
|
||||
_lazy_ensure("discord")
|
||||
except Exception:
|
||||
return False
|
||||
try:
|
||||
@@ -2228,7 +2228,7 @@ class DiscordAdapter(BasePlatformAdapter):
|
||||
path = self._command_sync_state_path()
|
||||
if not path.exists():
|
||||
return {}
|
||||
data = json.loads(path.read_text(encoding="utf-8"))
|
||||
data = json.loads(path.read_text(encoding="utf-8-sig"))
|
||||
except Exception:
|
||||
return {}
|
||||
return data if isinstance(data, dict) else {}
|
||||
|
||||
@@ -1000,8 +1000,8 @@ def matrix_deps_present() -> bool:
|
||||
and runs from ``create_adapter()`` when this returns False (#79812).
|
||||
"""
|
||||
try:
|
||||
from tools.lazy_deps import is_available
|
||||
return is_available("platform.matrix")
|
||||
from pm import available as is_available
|
||||
return is_available("matrix")
|
||||
except Exception: # pragma: no cover — defensive
|
||||
return False
|
||||
|
||||
@@ -1034,22 +1034,20 @@ def ensure_matrix_deps() -> bool:
|
||||
"""ACTIVE deps-only installer (registry ``ensure_deps_fn``).
|
||||
|
||||
Lazy-installs the full ``platform.matrix`` feature group via
|
||||
``tools.lazy_deps.ensure_and_bind`` whenever any of the declared
|
||||
``pm.extras.ensure_and_bind`` whenever any of the declared
|
||||
packages (mautrix, Markdown, aiosqlite, asyncpg, aiohttp-socks) is
|
||||
missing — not just mautrix itself. Previously this short-circuited on
|
||||
``import mautrix``, which left the other four packages uninstalled
|
||||
forever and broke E2EE connect with ``No module named 'asyncpg'``
|
||||
(#31116). Rebinds module-level type globals on success.
|
||||
"""
|
||||
# Check whether any package in the platform.matrix feature group is
|
||||
# missing. ``feature_missing`` is cheap (per-spec importlib.metadata
|
||||
# lookups) and correctly handles ``mautrix[encryption]`` by stripping
|
||||
# the extras marker before checking the bare package.
|
||||
# Check every anchor of the matrix extra (mautrix alone is not enough:
|
||||
# a partial install left asyncpg missing forever, #31116).
|
||||
try:
|
||||
from tools.lazy_deps import feature_missing, ensure_and_bind
|
||||
missing = feature_missing("platform.matrix")
|
||||
from pm.extras import missing as _extra_missing, ensure_and_bind
|
||||
missing = _extra_missing("matrix")
|
||||
except Exception as exc: # pragma: no cover — defensive
|
||||
logger.debug("Matrix: lazy_deps lookup failed: %s", exc)
|
||||
logger.debug("Matrix: extras lookup failed: %s", exc)
|
||||
missing = ()
|
||||
ensure_and_bind = None # type: ignore[assignment]
|
||||
|
||||
@@ -1075,12 +1073,12 @@ def ensure_matrix_deps() -> bool:
|
||||
|
||||
if ensure_and_bind is None:
|
||||
return False
|
||||
if not ensure_and_bind("platform.matrix", _import, globals(), prompt=False):
|
||||
if not ensure_and_bind("matrix", _import, globals()):
|
||||
logger.warning(
|
||||
"Matrix: required packages not installed (%s). "
|
||||
"Run: pip install 'mautrix[encryption]' asyncpg aiosqlite "
|
||||
"Markdown aiohttp-socks",
|
||||
", ".join(missing) if missing else "platform.matrix",
|
||||
", ".join(missing) if missing else "matrix",
|
||||
)
|
||||
return False
|
||||
|
||||
@@ -5323,12 +5321,13 @@ def interactive_setup() -> None:
|
||||
|
||||
matrix_pkg = "mautrix[encryption]" if want_e2ee else "mautrix"
|
||||
try:
|
||||
from tools.lazy_deps import ensure as _lazy_ensure, feature_missing
|
||||
_missing_before = feature_missing("platform.matrix")
|
||||
from pm import ensure_import as _lazy_ensure
|
||||
from pm.extras import missing as _extra_missing
|
||||
_missing_before = _extra_missing("matrix")
|
||||
if _missing_before:
|
||||
print_info(f"Installing {matrix_pkg} (+ {len(_missing_before)} runtime deps)...")
|
||||
try:
|
||||
_lazy_ensure("platform.matrix", prompt=False)
|
||||
_lazy_ensure("matrix")
|
||||
print_success(f"{matrix_pkg} installed")
|
||||
except Exception as exc:
|
||||
print_warning(
|
||||
|
||||
@@ -158,7 +158,7 @@ def _write_runtime_record(port: int, token: str, pid: int) -> None:
|
||||
|
||||
def _read_runtime_record() -> Optional[Dict[str, Any]]:
|
||||
try:
|
||||
raw = json.loads(_runtime_record_path().read_text(encoding="utf-8"))
|
||||
raw = json.loads(_runtime_record_path().read_text(encoding="utf-8-sig"))
|
||||
except (OSError, ValueError):
|
||||
return None
|
||||
return raw if isinstance(raw, dict) else None
|
||||
@@ -418,16 +418,50 @@ def _is_timeout_error(exc: BaseException) -> bool:
|
||||
return "timeout" in type(exc).__name__.lower()
|
||||
|
||||
|
||||
def _node_command(command: str) -> Optional[str]:
|
||||
"""Resolve a Node toolchain binary (node/npm), pm store first.
|
||||
|
||||
Photon's sidecar is Node/TypeScript, so the gateway runs node/npm on
|
||||
the host. Resolution goes through ``hermes_constants.find_node_executable``
|
||||
— the pm store's pinned node/npm wins whenever it is installed, with
|
||||
PATH (Windows shim ordering) as the fallback — never a bare PATH
|
||||
probe, which would miss the managed install (or resolve a system copy
|
||||
Hermes does not own). Returns None when the binary is nowhere.
|
||||
"""
|
||||
try:
|
||||
from hermes_constants import find_node_executable
|
||||
|
||||
resolved = find_node_executable(command)
|
||||
if resolved:
|
||||
return resolved
|
||||
except Exception: # pragma: no cover — hermes_constants is always present
|
||||
pass
|
||||
# pm.ensure wiring: the store's pinned node/npm is the first choice, but
|
||||
# when it has never been installed, lazily provision it (respecting the
|
||||
# lazy-install policy — InstallError is swallowed and PATH is tried) so
|
||||
# the sidecar works on a fresh install without a manual `hermes pm
|
||||
# install node`.
|
||||
try:
|
||||
import pm
|
||||
|
||||
pm.ensure("node")
|
||||
from hermes_constants import find_node_executable as _fne
|
||||
|
||||
resolved = _fne(command)
|
||||
if resolved:
|
||||
return resolved
|
||||
except Exception:
|
||||
pass
|
||||
return shutil.which(command)
|
||||
|
||||
|
||||
def check_requirements() -> bool:
|
||||
"""Return True when both Python deps and the Node sidecar are available."""
|
||||
if not HTTPX_AVAILABLE:
|
||||
logger.warning("photon: httpx not installed — pip install httpx")
|
||||
return False
|
||||
if not shutil.which(os.getenv("PHOTON_NODE_BIN") or "node"):
|
||||
logger.warning(
|
||||
"photon: node binary '%s' not found on PATH",
|
||||
os.getenv("PHOTON_NODE_BIN") or "node",
|
||||
)
|
||||
if not _node_command("node"):
|
||||
logger.warning("photon: node binary not found on PATH or in the pm store")
|
||||
return False
|
||||
if not sidecar_deps_installed():
|
||||
# spectrum-ts not installed yet, or node_modules/ was partially created
|
||||
@@ -444,7 +478,7 @@ def check_requirements() -> bool:
|
||||
# user has no CLI to run `hermes photon setup`, so the connect path
|
||||
# must self-heal). Otherwise keep returning False so
|
||||
# `hermes setup` / status surface the missing-deps state.
|
||||
if bool(shutil.which("npm")) and _dir_writable(_sidecar_dir()):
|
||||
if bool(_node_command("npm")) and _dir_writable(_sidecar_dir()):
|
||||
return True
|
||||
# DEBUG (not WARNING): this is the normal pre-setup state.
|
||||
# check_fn() is called from multiple hot paths in the core
|
||||
@@ -453,7 +487,7 @@ def check_requirements() -> bool:
|
||||
npm_error = ""
|
||||
try:
|
||||
if _npm_error_log().exists():
|
||||
npm_error = _npm_error_log().read_text(encoding="utf-8").strip()[:_NPM_ERROR_LOG_MAX_CHARS]
|
||||
npm_error = _npm_error_log().read_text(encoding="utf-8-sig").strip()[:_NPM_ERROR_LOG_MAX_CHARS]
|
||||
except OSError:
|
||||
pass
|
||||
if npm_error:
|
||||
@@ -499,9 +533,9 @@ def _reinstall_sidecar_deps() -> None:
|
||||
Best-effort — a failure here just leaves the (stale) deps in place and the
|
||||
normal ``_start_sidecar`` readiness check reports the real error.
|
||||
"""
|
||||
npm = shutil.which("npm")
|
||||
npm = _node_command("npm")
|
||||
if not npm:
|
||||
logger.warning("[photon] cannot reinstall stale sidecar deps: npm not on PATH")
|
||||
logger.warning("[photon] cannot reinstall stale sidecar deps: npm not available (pm store or PATH)")
|
||||
return
|
||||
# Windows: suppress the console flash these short-lived npm runs would
|
||||
# otherwise pop (0 elsewhere). Same helper as the sidecar spawn below.
|
||||
@@ -753,7 +787,7 @@ class PhotonAdapter(BasePlatformAdapter):
|
||||
self._autostart_sidecar = str(
|
||||
os.getenv("PHOTON_SIDECAR_AUTOSTART", "true")
|
||||
).lower() not in ("0", "false", "no")
|
||||
self._node_bin = os.getenv("PHOTON_NODE_BIN") or shutil.which("node") or "node"
|
||||
self._node_bin = _node_command("node") or "node"
|
||||
|
||||
# Presence watchdog. spectrum-ts only reconnects when its inbound
|
||||
# iterator throws or ends; a half-open ("zombie") gRPC socket makes the
|
||||
@@ -1692,11 +1726,11 @@ class PhotonAdapter(BasePlatformAdapter):
|
||||
creationflags=windows_hide_flags(),
|
||||
)
|
||||
except FileNotFoundError as exc:
|
||||
# Deterministic: node isn't on PATH / PHOTON_NODE_BIN points at
|
||||
# nothing. Retrying can never fix a missing binary (OOF-153).
|
||||
# Deterministic: node isn't resolvable (pm store or PATH).
|
||||
# Retrying can never fix a missing binary (OOF-153).
|
||||
raise PhotonSidecarStartupError(
|
||||
f"node binary not found ({self._node_bin!r}) — install Node.js "
|
||||
f"or set PHOTON_NODE_BIN: {exc}",
|
||||
f"18+ or run `hermes pm install`: {exc}",
|
||||
code="SIDECAR_NODE_MISSING",
|
||||
retryable=False,
|
||||
) from exc
|
||||
|
||||
@@ -21,7 +21,6 @@ from __future__ import annotations
|
||||
import argparse
|
||||
import getpass
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
@@ -29,7 +28,7 @@ from pathlib import Path
|
||||
from hermes_cli.colors import Colors, color
|
||||
|
||||
from . import auth as photon_auth
|
||||
from .adapter import _NPM_ERROR_LOG_MAX_CHARS, sidecar_deps_installed
|
||||
from .adapter import _NPM_ERROR_LOG_MAX_CHARS, _node_command, sidecar_deps_installed
|
||||
from .sidecar_paths import resolve_sidecar_dir
|
||||
|
||||
# Writable sidecar runtime dir (mirrors to HERMES_HOME on immutable
|
||||
@@ -385,7 +384,7 @@ def _cmd_status(_args: argparse.Namespace) -> int:
|
||||
# callback is the only sink that sees credential-derived strings, so
|
||||
# cli.py keeps zero taint flow according to CodeQL.
|
||||
photon_auth.print_credential_summary(print)
|
||||
node_bin = os.getenv("PHOTON_NODE_BIN") or shutil.which("node")
|
||||
node_bin = _node_command("node")
|
||||
sidecar_installed = sidecar_deps_installed()
|
||||
print(f" node binary : {node_bin or '✗ missing (install Node 18+)'}")
|
||||
print(f" sidecar deps : {'✓ installed' if sidecar_installed else '✗ run `hermes photon install-sidecar`'}")
|
||||
@@ -442,8 +441,8 @@ def _cmd_telemetry(args: argparse.Namespace) -> int:
|
||||
|
||||
|
||||
def _install_sidecar() -> int:
|
||||
npm = shutil.which("npm") or "npm"
|
||||
if not shutil.which(npm):
|
||||
npm = _node_command("npm")
|
||||
if not npm:
|
||||
print(
|
||||
"npm is not on PATH. Install Node.js 18+ (https://nodejs.org/) "
|
||||
"and re-run.",
|
||||
|
||||
@@ -38,10 +38,6 @@ optional_env:
|
||||
description: "Spawn the Node sidecar on connect (true/false, default true)"
|
||||
prompt: "Auto-start the sidecar?"
|
||||
password: false
|
||||
- name: PHOTON_NODE_BIN
|
||||
description: "Path to the node binary (default: shutil.which('node'))"
|
||||
prompt: "Node executable path"
|
||||
password: false
|
||||
- name: PHOTON_DASHBOARD_HOST
|
||||
description: "Photon Dashboard API host (default https://app.photon.codes)"
|
||||
prompt: "Dashboard host"
|
||||
|
||||
@@ -702,7 +702,7 @@ def check_teams_requirements() -> bool:
|
||||
"""Ensure the Teams SDK is importable, lazy-installing it on first use.
|
||||
|
||||
Lazy-installs ``microsoft-teams-apps`` via
|
||||
``tools.lazy_deps.ensure("platform.teams")`` if not present, then rebinds
|
||||
``pm.ensure_import("teams")`` if not present, then rebinds
|
||||
all module-level SDK globals on success. Returns True once the SDK (and
|
||||
aiohttp) are importable, False if they couldn't be installed/imported.
|
||||
|
||||
@@ -763,9 +763,9 @@ def check_teams_requirements() -> bool:
|
||||
"TEAMS_SDK_AVAILABLE": True,
|
||||
}
|
||||
|
||||
from tools.lazy_deps import ensure_and_bind
|
||||
from pm.extras import ensure_and_bind
|
||||
|
||||
return ensure_and_bind("platform.teams", _import, globals(), prompt=False)
|
||||
return ensure_and_bind("teams", _import, globals())
|
||||
|
||||
|
||||
class TeamsAdapter(BasePlatformAdapter):
|
||||
@@ -1500,17 +1500,14 @@ def interactive_setup() -> None:
|
||||
# ── Plugin entry point ────────────────────────────────────────────────────────
|
||||
|
||||
def _install_hint() -> str:
|
||||
"""Build the Teams install hint from the canonical LAZY_DEPS pins.
|
||||
"""Build the Teams install hint string.
|
||||
|
||||
Derived (not hardcoded) so a pin bump in ``tools/lazy_deps.py`` — aiohttp
|
||||
is CVE-pinned, so bumps happen — never leaves this string stale.
|
||||
``feature_install_command(venv_pip=True)`` targets the actual Hermes
|
||||
venv in every layout and sidesteps Ubuntu 24.04's PEP 668 failure that
|
||||
a bare ``pip install`` hint invites.
|
||||
Prefers ``uv sync --frozen --extra teams`` (respects pyproject pinning);
|
||||
falls back to a plain pip install of the two packages if that is
|
||||
unavailable. Restarting the gateway also auto-installs via pm.
|
||||
"""
|
||||
try:
|
||||
from tools.lazy_deps import feature_install_command
|
||||
cmd = feature_install_command("platform.teams", venv_pip=True)
|
||||
cmd = "uv sync --frozen --extra teams"
|
||||
except Exception: # pragma: no cover — defensive
|
||||
cmd = None
|
||||
if not cmd:
|
||||
|
||||
@@ -336,7 +336,7 @@ def check_telegram_requirements() -> bool:
|
||||
"""Check if Telegram dependencies are available.
|
||||
|
||||
If python-telegram-bot is missing, attempts to lazy-install it via
|
||||
``tools.lazy_deps.ensure("platform.telegram")``. After a successful
|
||||
``pm.ensure_import("telegram")``. After a successful
|
||||
install, re-imports the SDK and flips ``TELEGRAM_AVAILABLE`` to True
|
||||
so the adapter's class-level type aliases get rebound.
|
||||
"""
|
||||
@@ -347,8 +347,8 @@ def check_telegram_requirements() -> bool:
|
||||
if TELEGRAM_AVAILABLE:
|
||||
return True
|
||||
try:
|
||||
from tools.lazy_deps import ensure as _lazy_ensure
|
||||
_lazy_ensure("platform.telegram", prompt=False)
|
||||
from pm import ensure_import as _lazy_ensure
|
||||
_lazy_ensure("telegram")
|
||||
except Exception:
|
||||
return False
|
||||
try:
|
||||
@@ -3896,7 +3896,7 @@ class TelegramAdapter(BasePlatformAdapter):
|
||||
return
|
||||
|
||||
import yaml as _yaml
|
||||
with open(config_path, "r", encoding="utf-8") as f:
|
||||
with open(config_path, "r", encoding="utf-8-sig") as f:
|
||||
config = _yaml.safe_load(f) or {}
|
||||
|
||||
# Navigate to platforms.telegram.extra.dm_topics, creating the path
|
||||
|
||||
@@ -97,10 +97,10 @@ def ensure_wecom_callback_requirements() -> bool:
|
||||
return {"ET": _ET, "DEFUSEDXML_AVAILABLE": True}
|
||||
|
||||
try:
|
||||
from tools.lazy_deps import ensure_and_bind
|
||||
from pm.extras import ensure_and_bind
|
||||
except Exception: # pragma: no cover — defensive
|
||||
return False
|
||||
if not ensure_and_bind("platform.wecom_callback", _import, globals(), prompt=False):
|
||||
if not ensure_and_bind("wecom", _import, globals()):
|
||||
return False
|
||||
return check_wecom_callback_requirements()
|
||||
|
||||
|
||||
@@ -355,15 +355,34 @@ def _file_content_hash(path: Path) -> str:
|
||||
return ""
|
||||
|
||||
|
||||
def _pm_ensure_node(command: str) -> str | None:
|
||||
"""Lazily provision node/npm through the pm store, then re-resolve.
|
||||
|
||||
``find_node_executable`` prefers the pm store's pinned Node but never
|
||||
installs it. Node/npm ship as pm packages, so when the store is empty
|
||||
this is the pm.ensure wiring the pm-unified-toolchain plan asks for:
|
||||
install (respecting the lazy-install policy — raises InstallError and
|
||||
returns None when lazy installs are refused) and re-resolve.
|
||||
"""
|
||||
try:
|
||||
import pm
|
||||
|
||||
pm.ensure("node" if command == "npm" else command)
|
||||
return find_node_executable(command)
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def check_whatsapp_requirements() -> bool:
|
||||
"""
|
||||
Check if WhatsApp dependencies are available.
|
||||
|
||||
|
||||
WhatsApp requires a Node.js bridge for most implementations.
|
||||
"""
|
||||
# Prefer Hermes-managed Node/npm so Windows installs are not broken by a
|
||||
# bad or elevation-triggering system Node on PATH.
|
||||
_node = find_node_executable("node")
|
||||
# bad or elevation-triggering system Node on PATH. When the pm store has
|
||||
# no Node yet, pm.ensure lazily provisions it (policy permitting).
|
||||
_node = find_node_executable("node") or _pm_ensure_node("node")
|
||||
if not _node:
|
||||
return False
|
||||
try:
|
||||
@@ -584,8 +603,9 @@ class WhatsAppAdapter(WhatsAppBehaviorMixin, BasePlatformAdapter):
|
||||
if not _deps_fresh:
|
||||
print(f"[{self.name}] Installing WhatsApp bridge dependencies...")
|
||||
# Resolve npm path so Windows uses npm.cmd from the
|
||||
# Hermes-managed portable Node before falling back to PATH.
|
||||
_npm_bin = find_node_executable("npm") or "npm"
|
||||
# Hermes-managed portable Node before falling back to PATH;
|
||||
# pm.ensure provisions npm when the store has none.
|
||||
_npm_bin = find_node_executable("npm") or _pm_ensure_node("npm") or "npm"
|
||||
try:
|
||||
# Read timeout from environment variable, default to 300 seconds (5 minutes)
|
||||
# to accommodate slower systems like Unraid NAS
|
||||
|
||||
@@ -551,7 +551,7 @@ def _load_fal_client() -> Any:
|
||||
"""Lazy-load the ``fal_client`` SDK and cache it on this module.
|
||||
|
||||
Delegates the actual import to :func:`tools.fal_common.import_fal_client`
|
||||
so the ``lazy_deps`` ensure-install handling stays in one place.
|
||||
so the ``pm.ensure_import`` handling stays in one place.
|
||||
|
||||
Thread-safe via double-checked locking: concurrent first calls import
|
||||
the SDK exactly once instead of each racing thread re-running the import.
|
||||
|
||||
@@ -159,6 +159,17 @@ def _run_ddgs_search_bounded(query: str, safe_limit: int) -> list[dict[str, Any]
|
||||
from tools.environments.local import _sanitize_subprocess_env
|
||||
|
||||
env = _sanitize_subprocess_env(dict(os.environ))
|
||||
# No-boot-through-venv: the worker runs under the STORE python, whose
|
||||
# third-party imports (ddgs/primp) arrive via the launcher-composed
|
||||
# PYTHONPATH. The sanitizer strips Hermes-owned entries (cross-version
|
||||
# protection); this worker is the SAME interpreter, so merge the
|
||||
# ambient PYTHONPATH back in before prepending the plugins entry.
|
||||
_ambient_pp = os.environ.get("PYTHONPATH")
|
||||
if _ambient_pp:
|
||||
_current = env.get("PYTHONPATH", "")
|
||||
env["PYTHONPATH"] = (
|
||||
_ambient_pp + os.pathsep + _current if _current else _ambient_pp
|
||||
)
|
||||
if _test_hook:
|
||||
env["HERMES_DDGS_ALLOW_TEST_HOOKS"] = "1"
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
Subclasses :class:`agent.web_search_provider.WebSearchProvider`. Uses the
|
||||
official Exa SDK (``exa-py``) which is lazy-loaded via
|
||||
:func:`tools.lazy_deps.ensure` so that cold-start CLI users don't pay the
|
||||
:func:`pm.ensure_import` so that cold-start CLI users don't pay the
|
||||
SDK import cost when Exa isn't configured.
|
||||
|
||||
Config keys this provider responds to::
|
||||
@@ -61,12 +61,12 @@ def _get_exa_client() -> Any:
|
||||
)
|
||||
|
||||
try:
|
||||
from tools.lazy_deps import ensure as _lazy_ensure
|
||||
from pm import ensure_import as _lazy_ensure
|
||||
|
||||
_lazy_ensure("search.exa", prompt=False)
|
||||
_lazy_ensure("exa")
|
||||
except ImportError:
|
||||
pass
|
||||
except Exception as exc: # noqa: BLE001 — lazy_deps surfaces install hints
|
||||
except Exception as exc: # noqa: BLE001 — pm surfaces install hints
|
||||
raise ImportError(str(exc))
|
||||
|
||||
from exa_py import Exa # noqa: WPS433 — deliberately lazy
|
||||
|
||||
@@ -83,9 +83,9 @@ def _load_firecrawl_cls() -> type:
|
||||
global _FIRECRAWL_CLS_CACHE
|
||||
if _FIRECRAWL_CLS_CACHE is None:
|
||||
try:
|
||||
from tools.lazy_deps import ensure as _lazy_ensure
|
||||
from pm import ensure_import as _lazy_ensure
|
||||
|
||||
_lazy_ensure("search.firecrawl", prompt=False)
|
||||
_lazy_ensure("firecrawl")
|
||||
except ImportError:
|
||||
pass
|
||||
except Exception as exc: # noqa: BLE001 — surface install hint
|
||||
|
||||
@@ -47,14 +47,14 @@ def _ensure_parallel_sdk_installed() -> None:
|
||||
"""Trigger lazy install of the parallel SDK if it isn't present.
|
||||
|
||||
Mirrors the lazy-deps pattern used by the legacy implementation.
|
||||
Swallows benign ImportError from the lazy_deps helper itself; if the
|
||||
Swallows benign ImportError from the pm helper itself; if the
|
||||
SDK is genuinely missing the subsequent ``from parallel import ...``
|
||||
raises ImportError that the caller can handle.
|
||||
"""
|
||||
try:
|
||||
from tools.lazy_deps import ensure as _lazy_ensure
|
||||
from pm import ensure_import as _lazy_ensure
|
||||
|
||||
_lazy_ensure("search.parallel", prompt=False)
|
||||
_lazy_ensure("parallel-web")
|
||||
except ImportError:
|
||||
pass
|
||||
except Exception as exc: # noqa: BLE001 — surface install hint as ImportError
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
"""pm: the hermes package system.
|
||||
|
||||
Everything hermes depends on — tool binaries, the python venv, node_modules
|
||||
dirs, plugins — is a package in one dependency tree. Package definitions
|
||||
(pm/packages.py) say what a package IS. The lockfile (pm/lock.json,
|
||||
machine-written) says exactly which versions and hashes. The installed-state
|
||||
file (facts.json, per install) says what is actually on this machine.
|
||||
|
||||
ensure(name) makes the installed state match the lockfile and returns a
|
||||
Runner with the composed environment. env_for(*names) composes already-installed
|
||||
packages' env without installing anything.
|
||||
"""
|
||||
|
||||
from pm.ensure import (
|
||||
activate,
|
||||
adopt,
|
||||
check,
|
||||
enabled_extras,
|
||||
ensure,
|
||||
env_for,
|
||||
is_installed,
|
||||
lazy_installs_allowed,
|
||||
sync_venv,
|
||||
uv,
|
||||
)
|
||||
from pm.extras import available, ensure_import
|
||||
from pm.lock import Facts, Lockfile
|
||||
from pm.package import InstallError, Package, Runner, compose_env
|
||||
from pm.registry import all_packages, get_package, register, walk
|
||||
from pm.store import Store, current_target
|
||||
|
||||
__all__ = [
|
||||
"ensure",
|
||||
"env_for",
|
||||
"is_installed",
|
||||
"adopt",
|
||||
"check",
|
||||
"activate",
|
||||
"sync_venv",
|
||||
"available",
|
||||
"ensure_import",
|
||||
"enabled_extras",
|
||||
"lazy_installs_allowed",
|
||||
# NOTE: pm.uv is importable but deliberately not in __all__ — it is the
|
||||
# marked transitional bridge for update/repair and must not spread.
|
||||
"Facts",
|
||||
"Lockfile",
|
||||
"InstallError",
|
||||
"Package",
|
||||
"Runner",
|
||||
"compose_env",
|
||||
"all_packages",
|
||||
"get_package",
|
||||
"register",
|
||||
"walk",
|
||||
"Store",
|
||||
"current_target",
|
||||
]
|
||||
|
||||
import pm.packages # noqa: E402,F401 (registers the built-in definitions)
|
||||
@@ -0,0 +1,558 @@
|
||||
"""hermes pm: lock / install / env / doctor / gc / bundle."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
from pm.ensure import _facts, _lockfile, _store, ensure
|
||||
from pm.ensure import uv as pm_uv
|
||||
from pm.package import InstallError
|
||||
from pm.registry import get_package
|
||||
from pm.store import ALL_TARGETS, current_target, hash_url
|
||||
|
||||
|
||||
def cmd_lock(args) -> int:
|
||||
"""--bump <name> <version>: resolve every target's archives, hash them,
|
||||
write. A target with one archive pins the object; several pin a list.
|
||||
Target-independent urls collapse to one "any" artifact."""
|
||||
lockfile = _lockfile()
|
||||
package = get_package(args.name)
|
||||
artifacts: dict[str, object] = {}
|
||||
|
||||
def pin(url: str) -> dict:
|
||||
print(f" {url}")
|
||||
digest = package.known_sha256(args.version, url) or hash_url(url)
|
||||
print(f" sha256 {digest}")
|
||||
return {"url": url, "sha256": digest}
|
||||
|
||||
urls = {
|
||||
target: package.fetch_urls(args.version, target)
|
||||
for target in ALL_TARGETS
|
||||
if package.missing_reason(target) is None
|
||||
}
|
||||
distinct = {tuple(u) for u in urls.values()}
|
||||
if len(distinct) == 1:
|
||||
print(" any:")
|
||||
pinned = [pin(url) for url in next(iter(urls.values()))]
|
||||
artifacts["any"] = pinned[0] if len(pinned) == 1 else pinned
|
||||
else:
|
||||
for target, target_urls in urls.items():
|
||||
print(f" {target}:")
|
||||
pinned = [pin(url) for url in target_urls]
|
||||
artifacts[target] = pinned[0] if len(pinned) == 1 else pinned
|
||||
lockfile.set_pin(args.name, args.version, artifacts)
|
||||
lockfile.save()
|
||||
print(f"pinned {args.name} {args.version} ({len(artifacts)} targets)")
|
||||
return 0
|
||||
|
||||
|
||||
def _fmt_bytes(n: int) -> str:
|
||||
return f"{n / (1024 * 1024):.1f} MiB"
|
||||
|
||||
|
||||
def _live_progress(name: str):
|
||||
"""Per-package progress for ensure(): download as % + MiB, unpack as a
|
||||
phase line. Throttled to ~4 MiB steps — a slow line proves it's moving
|
||||
in a piped (CI) log without flooding it (a 1 MiB tick on a 1.5 GiB
|
||||
model would be ~1,500 lines)."""
|
||||
last = 0
|
||||
|
||||
def report(stage: str, done: int, total: int, label: str) -> None:
|
||||
nonlocal last
|
||||
if stage == "unpack":
|
||||
last = 0
|
||||
print(f" {name}: unpacking{(' ' + label) if label else ''}", flush=True)
|
||||
return
|
||||
if total <= 0:
|
||||
return
|
||||
if done >= total or done - last >= 4 * 1024 * 1024:
|
||||
last = done
|
||||
print(
|
||||
f" {name}: {done / total * 100:5.1f}% {_fmt_bytes(done)} / {_fmt_bytes(total)}",
|
||||
flush=True,
|
||||
)
|
||||
|
||||
return report
|
||||
|
||||
|
||||
def _install_names(names: list[str]) -> int:
|
||||
failed = 0
|
||||
for name in names:
|
||||
try:
|
||||
ensure(name, explicit=True, progress=_live_progress(name))
|
||||
print(f"✓ {name}", flush=True)
|
||||
except InstallError as e:
|
||||
print(f"✗ {e}", flush=True)
|
||||
failed += 1
|
||||
return failed
|
||||
|
||||
|
||||
|
||||
def _bundle_package_names() -> list[str]:
|
||||
names = [
|
||||
n
|
||||
for n in _lockfile().names()
|
||||
if not get_package(n).internal or n == "uv"
|
||||
]
|
||||
if "python" not in names:
|
||||
names.append("python")
|
||||
return names
|
||||
|
||||
|
||||
def _drop_unloadable_runtime_files(store_dir: Path) -> None:
|
||||
"""Drop the x64 VC runtime that python-build-standalone ships beside ARM64 Python."""
|
||||
if current_target() != "win32-arm64":
|
||||
return
|
||||
facts = _facts()
|
||||
facts.reload()
|
||||
python = facts.get("python")
|
||||
if python and "entry" in python:
|
||||
(store_dir / python["entry"] / "vcruntime140_1.dll").unlink(missing_ok=True)
|
||||
|
||||
|
||||
def cmd_install(args) -> int:
|
||||
names = args.names or [
|
||||
n for n in _lockfile().names() if not get_package(n).optional
|
||||
]
|
||||
failed = _install_names(names)
|
||||
if not args.names:
|
||||
from pm.ensure import sync_venv
|
||||
|
||||
try:
|
||||
sync_venv(explicit=True)
|
||||
print("✓ venv")
|
||||
except InstallError as e:
|
||||
print(f"✗ {e}")
|
||||
failed += 1
|
||||
return 1 if failed else 0
|
||||
|
||||
|
||||
def cmd_env(args) -> int:
|
||||
from pm.ensure import env_for
|
||||
|
||||
names = args.names or _lockfile().names()
|
||||
print(json.dumps(env_for(*names), indent=2, sort_keys=True))
|
||||
return 0
|
||||
|
||||
|
||||
def cmd_doctor(args) -> int:
|
||||
from pm.ensure import _identity
|
||||
from pm.store import tree_digest
|
||||
|
||||
lockfile = _lockfile()
|
||||
facts = _facts()
|
||||
store = _store()
|
||||
target = current_target()
|
||||
bad = 0
|
||||
for name in lockfile.names():
|
||||
package = get_package(name)
|
||||
reason = package.missing_reason(target)
|
||||
if reason is not None:
|
||||
print(f"- {name}: n/a on {target} ({reason})")
|
||||
continue
|
||||
fact = facts.get(name)
|
||||
soft = package.optional or package.internal
|
||||
identity = _identity(lockfile, name, target)
|
||||
if (
|
||||
fact is not None
|
||||
and identity is not None
|
||||
and ("target" not in fact or "artifacts" not in fact)
|
||||
):
|
||||
# Legacy fact: pre-dates digest-bound identity; installed()
|
||||
# treats it as not installed and forces one reinstall.
|
||||
print(f"{'?' if soft else '✗'} {name}: legacy fact: no recorded identity, run `hermes pm install`")
|
||||
bad += 0 if soft else 1
|
||||
continue
|
||||
if not facts.installed(name, lockfile.version(name), store.root, identity):
|
||||
state = "not installed" if fact is None else "outdated"
|
||||
print(f"{'?' if soft else '✗'} {name}: {state}")
|
||||
bad += 0 if soft else 1
|
||||
continue
|
||||
entry = store.entry(fact["entry"])
|
||||
reason = package.verify(entry, target)
|
||||
if reason:
|
||||
print(f"✗ {name}: installed but failed verification: {reason}")
|
||||
bad += 1
|
||||
continue
|
||||
recorded = fact.get("digest")
|
||||
if recorded is not None and tree_digest(entry) != recorded:
|
||||
# Doctor is the expensive-path tool: re-hash the realized
|
||||
# bytes. Boot checks stay O(1) json compares.
|
||||
print(f"✗ {name}: realized bytes do not match recorded digest")
|
||||
bad += 1
|
||||
continue
|
||||
print(f"✓ {name} {fact['version']}")
|
||||
return 1 if bad else 0
|
||||
|
||||
|
||||
def _venv_site_packages(venv_dir: Path, win: bool) -> Optional[Path]:
|
||||
"""The site-packages dir uv sync fills inside the venv (the venv stays
|
||||
a dependency target; it is never where `python` resolves)."""
|
||||
if win:
|
||||
candidate = venv_dir / "Lib" / "site-packages"
|
||||
return candidate if candidate.is_dir() else None
|
||||
for candidate in sorted(venv_dir.glob("lib/python3.*/site-packages")):
|
||||
if candidate.is_dir():
|
||||
return candidate
|
||||
return None
|
||||
|
||||
|
||||
def _develop_env(names: list[str]) -> Optional[dict]:
|
||||
"""The `pm develop` subshell environment. `python` resolves to the pm
|
||||
STORE python (its bin dir first on PATH); imports come from
|
||||
PYTHONPATH=<repo>;<venv>/site-packages (repo first — the venv stays a
|
||||
dependency target only). VIRTUAL_ENV and the venv bin dir are
|
||||
deliberately absent: nothing in the devshell boots through the venv
|
||||
(pm work item 3; pyvenv.cfg is inert dead config). Returns None when
|
||||
the store interpreter has not been materialized (`hermes pm install`)."""
|
||||
import os
|
||||
|
||||
from pm import paths
|
||||
from pm.ensure import env_for
|
||||
|
||||
facts = _facts()
|
||||
python_fact = facts.get("python")
|
||||
target = current_target()
|
||||
if not python_fact or "entry" not in python_fact:
|
||||
return None
|
||||
python_bin = get_package("python").binary(
|
||||
_store().entry(python_fact["entry"]), target
|
||||
)
|
||||
if python_bin is None or not python_bin.is_file():
|
||||
return None
|
||||
|
||||
env = env_for(*names, base_env=dict(os.environ))
|
||||
repo = paths.repo_root()
|
||||
venv_dir = repo / (".venv" if (repo / ".venv").is_dir() else "venv")
|
||||
win = target.startswith("win32")
|
||||
venv_bin = venv_dir / ("Scripts" if win else "bin")
|
||||
env.pop("VIRTUAL_ENV", None)
|
||||
env.pop("PYTHONHOME", None)
|
||||
# The venv bin dir must never be where `python` resolves.
|
||||
path_entries = [
|
||||
p for p in env.get("PATH", "").split(os.pathsep) if p and Path(p) != venv_bin
|
||||
]
|
||||
env["PATH"] = os.pathsep.join([str(python_bin.parent), *path_entries])
|
||||
site = _venv_site_packages(venv_dir, win)
|
||||
ours = str(repo) + (os.pathsep + str(site) if site else "")
|
||||
existing = env.get("PYTHONPATH", "")
|
||||
env["PYTHONPATH"] = ours + (os.pathsep + existing if existing else "")
|
||||
return env
|
||||
|
||||
|
||||
def cmd_develop(args) -> int:
|
||||
"""Install everything, sync the venv, then activate: spawn a subshell
|
||||
with every tool's env composed in and the pm STORE python resolving
|
||||
`python` (PYTHONPATH=repo;venv-site-packages for imports — never the
|
||||
venv interpreter). The devshell equivalent of nix develop. --print
|
||||
emits eval-able exports for the current shell."""
|
||||
import os
|
||||
import subprocess
|
||||
|
||||
from pm.ensure import sync_venv
|
||||
|
||||
install_names = [
|
||||
n for n in _lockfile().names()
|
||||
if not get_package(n).optional
|
||||
and get_package(n).missing_reason(current_target()) is None
|
||||
]
|
||||
# The devshell boots the STORE python — make sure it is materialized.
|
||||
if (
|
||||
"python" not in install_names
|
||||
and get_package("python").missing_reason(current_target()) is None
|
||||
):
|
||||
install_names.append("python")
|
||||
failed = _install_names(install_names)
|
||||
try:
|
||||
sync_venv(explicit=True)
|
||||
print("✓ venv")
|
||||
except InstallError as e:
|
||||
print(f"✗ {e}")
|
||||
failed += 1
|
||||
if failed:
|
||||
return 1
|
||||
|
||||
env = _develop_env(_lockfile().names())
|
||||
if env is None:
|
||||
print("✗ develop: no pm store interpreter — run 'hermes pm install' first")
|
||||
return 1
|
||||
|
||||
from pm import paths
|
||||
|
||||
win = current_target().startswith("win32")
|
||||
if args.print_env:
|
||||
changed = {k: v for k, v in env.items() if os.environ.get(k) != v}
|
||||
for key, value in sorted(changed.items()):
|
||||
if win and os.environ.get("SHELL") is None:
|
||||
print(f'$env:{key} = "{value}"')
|
||||
else:
|
||||
escaped = value.replace("'", "'\\''")
|
||||
print(f"export {key}='{escaped}'")
|
||||
return 0
|
||||
|
||||
shell = os.environ.get("SHELL") or os.environ.get("COMSPEC") or (
|
||||
"cmd.exe" if win else "/bin/sh"
|
||||
)
|
||||
print(f"pm develop: entering {shell} (exit to leave)")
|
||||
return subprocess.call([shell], env=env, cwd=paths.repo_root())
|
||||
|
||||
|
||||
def cmd_gc(args) -> int:
|
||||
from pm.downloader import gc_protected_names
|
||||
|
||||
facts = _facts()
|
||||
store = _store()
|
||||
if not store.root.is_dir():
|
||||
return 0
|
||||
removed = 0
|
||||
with store.install_lock():
|
||||
facts.reload()
|
||||
keep = facts.entries_in_use()
|
||||
# Partials an in-flight (or recently interrupted) download still
|
||||
# owns must survive the sweep.
|
||||
protected_partials = gc_protected_names(store.root / "partials")
|
||||
for item in sorted(store.root.iterdir()):
|
||||
if not item.is_dir() or item.name.startswith("."):
|
||||
continue
|
||||
if item.name == "partials" and protected_partials:
|
||||
for child in sorted(item.iterdir()):
|
||||
if child.name in protected_partials:
|
||||
continue
|
||||
print(f"removing {item.name}/{child.name}")
|
||||
if child.is_dir():
|
||||
shutil.rmtree(child, ignore_errors=True)
|
||||
else:
|
||||
try:
|
||||
child.unlink()
|
||||
except OSError:
|
||||
pass
|
||||
continue
|
||||
if item.name in keep:
|
||||
continue
|
||||
print(f"removing {item.name}")
|
||||
shutil.rmtree(item, ignore_errors=True)
|
||||
removed += 1
|
||||
print(f"gc: removed {removed}, kept {len(keep)}")
|
||||
return 0
|
||||
|
||||
|
||||
def _run_live(cmd: list[str], *, cwd, env, timeout: int = 3600) -> tuple[int, str]:
|
||||
"""Run cmd with its output streamed through our stdout — a long uv
|
||||
venv build must prove liveness in a piped (CI) log, not vanish until
|
||||
exit — while still capturing the tail for the failure message. A
|
||||
reader thread drains output so proc.wait(timeout) keeps the wall-clock
|
||||
kill the old subprocess.run(timeout=) had. Returns (returncode, last
|
||||
~2k chars of combined output)."""
|
||||
proc = subprocess.Popen(
|
||||
cmd, cwd=cwd, env=env, stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT, text=True, bufsize=1, errors="replace",
|
||||
)
|
||||
tail = ""
|
||||
lock = threading.Lock()
|
||||
|
||||
def drain() -> None:
|
||||
nonlocal tail
|
||||
for line in proc.stdout:
|
||||
print(line, end="", flush=True)
|
||||
with lock:
|
||||
tail = (tail + line)[-2000:]
|
||||
|
||||
thread = threading.Thread(target=drain, daemon=True)
|
||||
thread.start()
|
||||
try:
|
||||
code = proc.wait(timeout=timeout)
|
||||
except subprocess.TimeoutExpired:
|
||||
proc.kill()
|
||||
proc.wait()
|
||||
raise RuntimeError(f"{cmd[0]} timed out after {timeout}s")
|
||||
thread.join()
|
||||
with lock:
|
||||
return code, tail
|
||||
|
||||
|
||||
def cmd_bundle(args) -> int:
|
||||
"""Stage a complete payload for THIS machine's target into --out:
|
||||
repo snapshot + store + facts (via the normal install path, redirected)
|
||||
+ a relocatable venv built on the staged interpreter and synced from
|
||||
uv.lock. Built natively per (os, arch); there is no cross-target
|
||||
staging."""
|
||||
import os
|
||||
|
||||
from pm import paths
|
||||
|
||||
out = Path(args.out).resolve()
|
||||
store_dir = out / "tools"
|
||||
store_dir.mkdir(parents=True, exist_ok=True)
|
||||
# A manifest from a previous run would make this payload look sealed
|
||||
# and refuse its own staging; it is rewritten at the end.
|
||||
(out / "manifest.json").unlink(missing_ok=True)
|
||||
|
||||
repo_dir = out / "hermes-agent"
|
||||
ref = args.ref or "HEAD"
|
||||
if repo_dir.exists():
|
||||
shutil.rmtree(repo_dir)
|
||||
repo_dir.mkdir(parents=True)
|
||||
print(f"staging repo snapshot ({ref})…", flush=True)
|
||||
archive = subprocess.run(
|
||||
["git", "archive", "--format=tar", ref],
|
||||
cwd=paths.repo_root(), capture_output=True, timeout=600,
|
||||
)
|
||||
if archive.returncode != 0:
|
||||
print(f"✗ repo: git archive {ref} failed: {archive.stderr.decode()[-500:]}")
|
||||
return 1
|
||||
import io
|
||||
import tarfile
|
||||
|
||||
with tarfile.open(fileobj=io.BytesIO(archive.stdout)) as tar:
|
||||
tar.extractall(repo_dir, filter="data")
|
||||
print(f"✓ repo ({ref})")
|
||||
|
||||
os.environ["HERMES_RUNTIME_DIR"] = str(store_dir)
|
||||
paths._stamp.cache_clear()
|
||||
|
||||
failed = 0
|
||||
names = _bundle_package_names()
|
||||
failed += _install_names(
|
||||
[n for n in names if get_package(n).missing_reason(current_target()) is None]
|
||||
)
|
||||
_drop_unloadable_runtime_files(store_dir)
|
||||
|
||||
uv_bin, env = pm_uv()
|
||||
if uv_bin is None:
|
||||
print("✗ venv: uv did not stage")
|
||||
return 1
|
||||
|
||||
python_fact = _facts().get("python")
|
||||
if python_fact is None:
|
||||
print("✗ venv: no staged interpreter to build on")
|
||||
return 1
|
||||
python_bin = get_package("python").binary(
|
||||
_store().entry(python_fact["entry"]), current_target()
|
||||
)
|
||||
|
||||
# Build + sync INSIDE the staged repo: the editable project install
|
||||
# must point at the payload's own tree, not this checkout.
|
||||
venv_dir = out / "venv"
|
||||
if venv_dir.exists():
|
||||
shutil.rmtree(venv_dir)
|
||||
env["VIRTUAL_ENV"] = str(venv_dir)
|
||||
env.pop("UV_NO_CONFIG", None)
|
||||
if current_target().startswith("darwin"):
|
||||
# python-build-standalone bakes phantom toolchain paths (its build
|
||||
# dir's llvm-ar) into sysconfig; sdist builds then fail with
|
||||
# "No such file or directory: .../tools/llvm/bin/llvm-ar". Point
|
||||
# sdist builds at the machine's real toolchain.
|
||||
env.setdefault("AR", "/usr/bin/ar")
|
||||
env.setdefault("CC", "clang")
|
||||
for cmd in (
|
||||
[uv_bin, "venv", "--relocatable", "--python", str(python_bin), str(venv_dir)],
|
||||
[uv_bin, "sync", "--frozen", "--all-extras", "--active"],
|
||||
):
|
||||
print(f" venv: $ {' '.join(cmd)}", flush=True)
|
||||
code, tail = _run_live(cmd, cwd=repo_dir, env=env)
|
||||
if code != 0:
|
||||
print(f"✗ venv: {' '.join(cmd[1:3])} failed:\n{tail}")
|
||||
return 1
|
||||
print("✓ venv (relocatable, all extras, on the staged interpreter)")
|
||||
|
||||
bad = _arch_guard(store_dir)
|
||||
for line in bad:
|
||||
print(f"✗ arch: {line}")
|
||||
failed += 1
|
||||
|
||||
manifest = {
|
||||
"schema": 1,
|
||||
"target": current_target(),
|
||||
"ref": ref,
|
||||
"repo": "hermes-agent",
|
||||
"venv": "venv",
|
||||
"store": "tools",
|
||||
}
|
||||
(out / "manifest.json").write_text(
|
||||
json.dumps(manifest, indent=2, sort_keys=True) + "\n", encoding="utf-8"
|
||||
)
|
||||
print(f"✓ manifest ({out / 'manifest.json'})")
|
||||
return 1 if failed else 0
|
||||
|
||||
|
||||
def _arch_guard(store_dir: Path) -> list[str]:
|
||||
"""Every staged binary must be built for this machine's target — a
|
||||
payload staged with a mismatched interpreter or PATH tool ships an
|
||||
artifact that cannot run. Reads facts, probes each entry binary."""
|
||||
from pm.lock import Facts
|
||||
from pm.package import machine_matches_binary
|
||||
|
||||
facts = Facts(store_dir / "facts.json")
|
||||
problems = []
|
||||
target = current_target()
|
||||
for name in _lockfile().names():
|
||||
package = get_package(name)
|
||||
fact = facts.get(name)
|
||||
if fact is None or "entry" not in fact:
|
||||
continue
|
||||
binary = package.binary(store_dir / fact["entry"], target)
|
||||
if binary is None or not binary.is_file():
|
||||
continue
|
||||
verdict = machine_matches_binary(binary, target)
|
||||
# A package that declares this target as emulated (x64 binary run
|
||||
# under Windows ARM64 built-in emulation) is fine with the x64 PE.
|
||||
if verdict is False and target not in package.emulated_arch_targets:
|
||||
problems.append(f"{name}: {binary.name} is not a {target} binary")
|
||||
return problems
|
||||
|
||||
|
||||
def main(argv=None) -> int:
|
||||
# Windows consoles default to cp1252; pm prints ✓/✗. Never let the
|
||||
# status glyphs crash the command reporting them. line_buffering:
|
||||
# pm output must stream live in a piped (CI) log, not sit in a block
|
||||
# buffer and flush only at exit.
|
||||
for stream in (sys.stdout, sys.stderr):
|
||||
try:
|
||||
stream.reconfigure(errors="replace", line_buffering=True)
|
||||
except (AttributeError, OSError):
|
||||
pass
|
||||
parser = argparse.ArgumentParser(prog="hermes pm")
|
||||
sub = parser.add_subparsers(dest="cmd", required=True)
|
||||
|
||||
p = sub.add_parser("lock", help="write versions+hashes into pm/lock.json")
|
||||
p.add_argument("--bump", dest="name", required=True)
|
||||
p.add_argument("version")
|
||||
p.set_defaults(func=cmd_lock)
|
||||
|
||||
p = sub.add_parser("install", help="install packages (default: all required)")
|
||||
p.add_argument("names", nargs="*")
|
||||
p.set_defaults(func=cmd_install)
|
||||
|
||||
p = sub.add_parser("env", help="print composed env of installed packages")
|
||||
p.add_argument("names", nargs="*")
|
||||
p.set_defaults(func=cmd_env)
|
||||
|
||||
p = sub.add_parser("doctor", help="check installed state against the lockfile")
|
||||
p.set_defaults(func=cmd_doctor)
|
||||
|
||||
p = sub.add_parser("develop", help="install + sync, then activate a devshell with the composed env")
|
||||
p.add_argument("--print", dest="print_env", action="store_true",
|
||||
help="print eval-able exports instead of spawning a shell")
|
||||
p.set_defaults(func=cmd_develop)
|
||||
|
||||
p = sub.add_parser("gc", help="remove store entries nothing references")
|
||||
p.set_defaults(func=cmd_gc)
|
||||
|
||||
p = sub.add_parser("bundle", help="stage a payload (repo+store+facts+relocatable venv) into --out")
|
||||
p.add_argument("--out", required=True)
|
||||
p.add_argument("--ref", help="git ref for the repo snapshot (default HEAD)")
|
||||
p.set_defaults(func=cmd_bundle)
|
||||
|
||||
args = parser.parse_args(argv)
|
||||
return args.func(args)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,427 @@
|
||||
"""Resumable, hash-verified, multi-connection downloads.
|
||||
|
||||
Every large fetch in Hermes goes through this one downloader: pm
|
||||
packages (pinned sha256 from lock.json) and local models (deliberately
|
||||
unverified — catalog sizes may lag an upstream re-upload, so sha256 is
|
||||
optional per source).
|
||||
|
||||
Partial state is the downloader's own. Files land in a managed
|
||||
partials area keyed by sha256(url), and a finished file is MOVED into
|
||||
its real destination, so a caller only ever sees a complete file or
|
||||
nothing. An interrupted download leaves its partial + a .ranges sidecar
|
||||
(the durable byte-range bitmap) behind, and the next run re-fetches
|
||||
exactly the ranges the bitmap says are missing.
|
||||
|
||||
The progress callback reports the whole job AND the per-dest bitmap:
|
||||
``progress(overall_done, overall_total, ranges)`` where ``ranges`` maps
|
||||
``dest.name`` to half-open [start, end) runs. ``done_bytes`` is the sum;
|
||||
``ranges`` is the shape — one datum, two resolutions.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import shutil
|
||||
import threading
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Callable, Optional, Sequence
|
||||
|
||||
_UA = {"User-Agent": "hermes-pm"}
|
||||
_LOOPBACK = ("http://127.0.0.1:", "http://localhost:", "http://[::1]:")
|
||||
_CHUNK = 1 << 20 # read/write block, also the minimum range size
|
||||
|
||||
|
||||
class _HttpsRedirectHandler(urllib.request.HTTPRedirectHandler):
|
||||
"""The https-only gate must hold across redirects, not just the first
|
||||
hop — an https URL could otherwise bounce to http mid-download and
|
||||
carry the payload in the clear."""
|
||||
|
||||
def redirect_request(self, req, fp, code, msg, headers, newurl):
|
||||
if not (newurl.startswith("https://") or newurl.startswith(_LOOPBACK)):
|
||||
raise DownloadError(f"refusing redirect to non-https url: {newurl}")
|
||||
return super().redirect_request(req, fp, code, msg, headers, newurl)
|
||||
|
||||
|
||||
_OPENER = urllib.request.build_opener(_HttpsRedirectHandler())
|
||||
|
||||
|
||||
class DownloadError(RuntimeError):
|
||||
"""Base class for downloader failures."""
|
||||
|
||||
|
||||
class HashError(DownloadError):
|
||||
"""The downloaded bytes did not match the pinned sha256."""
|
||||
|
||||
|
||||
class DownloadPaused(DownloadError):
|
||||
"""pause() was called mid-download; partials were left intact."""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Source:
|
||||
url: str
|
||||
dest: Path
|
||||
sha256: str = "" # "" = no integrity check (model catalog policy)
|
||||
|
||||
|
||||
_Ranges = list[tuple[int, int]]
|
||||
ProgressFn = Callable[[int, int, dict[str, _Ranges]], None]
|
||||
|
||||
|
||||
def _coalesce(ranges: _Ranges) -> _Ranges:
|
||||
"""Merge half-open [start, end) ranges into sorted, disjoint runs."""
|
||||
runs = sorted((a, b) for a, b in ranges if b > a)
|
||||
if not runs:
|
||||
return []
|
||||
out: _Ranges = []
|
||||
a, b = runs[0]
|
||||
for x, y in runs[1:]:
|
||||
if x <= b:
|
||||
b = max(b, y)
|
||||
else:
|
||||
out.append((a, b))
|
||||
a, b = x, y
|
||||
out.append((a, b))
|
||||
return out
|
||||
|
||||
|
||||
def _missing(total: int, covered: _Ranges) -> _Ranges:
|
||||
"""The gaps in [0, total) not covered by the coalesced bitmap."""
|
||||
missing: _Ranges = []
|
||||
cursor = 0
|
||||
for a, b in _coalesce(covered):
|
||||
if a > cursor:
|
||||
missing.append((cursor, a))
|
||||
cursor = max(cursor, b)
|
||||
if cursor < total:
|
||||
missing.append((cursor, total))
|
||||
return missing
|
||||
|
||||
|
||||
def _sha256_file(path: Path) -> str:
|
||||
digest = hashlib.sha256()
|
||||
with open(path, "rb") as f:
|
||||
for block in iter(lambda: f.read(_CHUNK), b""):
|
||||
digest.update(block)
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def _existing_dest_ok(source: "Source") -> bool:
|
||||
"""A dest already on disk counts as done only when it is a genuine
|
||||
complete download. No pinned hash (model-catalog policy): a present
|
||||
file is accepted — size is the tripwire, same as a fresh fetch. With a
|
||||
pinned sha256 the existing file must match, or it is re-fetched."""
|
||||
if not source.dest.exists():
|
||||
return False
|
||||
if not source.sha256:
|
||||
return True
|
||||
try:
|
||||
return _sha256_file(source.dest) == source.sha256
|
||||
except OSError:
|
||||
return False
|
||||
|
||||
|
||||
def _default_partials() -> Path:
|
||||
from pm import paths
|
||||
|
||||
return paths.store_root() / "partials"
|
||||
|
||||
|
||||
# `pm gc` must not delete partials an in-flight download is still writing.
|
||||
# A sidecar's mtime is the heartbeat: it is (re)written whenever a download
|
||||
# pauses or errors, and the .part grows on every chunk. Anything touched
|
||||
# within the grace window is live; everything older is fair game.
|
||||
GC_GRACE_SECONDS = 6 * 60 * 60
|
||||
|
||||
|
||||
def gc_protected_names(
|
||||
partials_dir: Path, *, grace_seconds: float = GC_GRACE_SECONDS
|
||||
) -> set[str]:
|
||||
"""Entry names under ``partials_dir`` an in-flight (or recently
|
||||
interrupted) download still owns — ``<key>.part`` / ``<key>.ranges``
|
||||
pairs whose newest mtime is within ``grace_seconds`` of now.
|
||||
|
||||
gc sweeps call this BEFORE rmtree-ing store entries: partials live in
|
||||
the store root, and the old name-prefix skip (``startswith(".")``)
|
||||
does not cover the "partials" directory. A caller deletes only the
|
||||
names absent from the returned set.
|
||||
"""
|
||||
import time
|
||||
|
||||
partials_dir = Path(partials_dir)
|
||||
now = time.time()
|
||||
protected: set[str] = set()
|
||||
if not partials_dir.is_dir():
|
||||
return protected
|
||||
for entry in partials_dir.iterdir():
|
||||
try:
|
||||
# Strict < : grace_seconds=0 protects nothing (the test
|
||||
# contract for "sweep everything"). mtime can tick a hair
|
||||
# NEWER than `now` on some filesystems, which <= would
|
||||
# wrongly treat as inside a zero window.
|
||||
if now - entry.stat().st_mtime < grace_seconds:
|
||||
protected.add(entry.name)
|
||||
except OSError:
|
||||
protected.add(entry.name) # unreadable → assume live
|
||||
return protected
|
||||
|
||||
|
||||
class Download:
|
||||
"""One resumable download job: a plan of sources, run in parallel.
|
||||
|
||||
Per source: probe Range support, preallocate a flat .part in the
|
||||
managed partials area, split into at most ``connections`` byte
|
||||
ranges, and stream each with a ``Range:`` header from a worker
|
||||
thread. A lock-protected bitmap records which ranges are actually
|
||||
durable; on resume only the missing ranges are re-fetched.
|
||||
|
||||
A finished source is MOVED into ``Source.dest``. ``pause()`` stops
|
||||
between chunks and ``run()`` raises :class:`DownloadPaused`,
|
||||
leaving every partial + sidecar intact for a later resume.
|
||||
"""
|
||||
|
||||
CONNECTIONS = 8
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
sources: Sequence[Source],
|
||||
*,
|
||||
resume: bool = True,
|
||||
connections: int = CONNECTIONS,
|
||||
partials_dir: Optional[Path] = None,
|
||||
):
|
||||
self.sources = [Source(s.url, Path(s.dest), s.sha256) for s in sources]
|
||||
self.resume = resume
|
||||
self.connections = max(1, int(connections))
|
||||
self.partials_dir = Path(partials_dir) if partials_dir else _default_partials()
|
||||
self._paused = threading.Event()
|
||||
|
||||
def pause(self) -> None:
|
||||
"""Request a stop between chunks; run() raises DownloadPaused."""
|
||||
self._paused.set()
|
||||
|
||||
def run(self, progress: Optional[ProgressFn] = None) -> list[Path]:
|
||||
"""Fetch every source; return the moved destination paths."""
|
||||
self._paused.clear()
|
||||
self.partials_dir.mkdir(parents=True, exist_ok=True)
|
||||
for source in self.sources:
|
||||
if not (source.url.startswith("https://")
|
||||
or source.url.startswith(_LOOPBACK)):
|
||||
raise ValueError(f"refusing non-https url: {source.url}")
|
||||
|
||||
# Probe every source up front so overall_total is the whole job.
|
||||
probed = [] # (source, total, range_supported)
|
||||
for source in self.sources:
|
||||
if _existing_dest_ok(source):
|
||||
probed.append((source, source.dest.stat().st_size, True))
|
||||
continue
|
||||
# A pre-existing dest that fails its pinned hash is stale — clear
|
||||
# it so the fetch below replaces it rather than trusting it.
|
||||
if source.dest.exists():
|
||||
source.dest.unlink(missing_ok=True)
|
||||
total, supported = self._probe(source.url)
|
||||
probed.append((source, total, supported))
|
||||
overall_total = sum(t for _, t, _ in probed)
|
||||
|
||||
moved: list[Path] = []
|
||||
done_base = 0
|
||||
completed: dict = {}
|
||||
for source, total, supported in probed:
|
||||
if _existing_dest_ok(source):
|
||||
size = source.dest.stat().st_size
|
||||
completed[source.dest.name] = [(0, size)]
|
||||
done_base += size
|
||||
moved.append(source.dest)
|
||||
continue
|
||||
|
||||
def tick(written: _Ranges) -> None:
|
||||
if progress is not None:
|
||||
ranges = dict(completed)
|
||||
ranges[source.dest.name] = written
|
||||
progress(done_base + sum(b - a for a, b in written),
|
||||
overall_total, ranges)
|
||||
|
||||
if total and supported:
|
||||
self._fetch_ranged(source, total, tick)
|
||||
else:
|
||||
self._fetch_single(source, total, tick)
|
||||
done_base += total
|
||||
completed[source.dest.name] = [(0, total)]
|
||||
moved.append(source.dest)
|
||||
return moved
|
||||
|
||||
# ── internals ─────────────────────────────────────────────
|
||||
|
||||
@staticmethod
|
||||
def _probe(url: str) -> tuple[int, bool]:
|
||||
"""(total bytes, range_supported). A server that ignores Range
|
||||
reports its Content-Length instead; 0 means unknown (the
|
||||
single-stream fallback judges completeness by the body)."""
|
||||
req = urllib.request.Request(url, headers={**_UA, "Range": "bytes=0-0"})
|
||||
try:
|
||||
with _OPENER.open(req, timeout=60) as r:
|
||||
if r.status == 206:
|
||||
content_range = r.headers.get("Content-Range", "")
|
||||
if "/" in content_range:
|
||||
return int(content_range.rsplit("/", 1)[1]), True
|
||||
return 0, False
|
||||
length = int(r.headers.get("Content-Length") or 0)
|
||||
return length, False
|
||||
except urllib.error.HTTPError as exc:
|
||||
if exc.code in (401, 403):
|
||||
raise DownloadError(
|
||||
"the host refused the download (gated or moved). "
|
||||
"This is a catalog problem, not yours — please report it."
|
||||
) from exc
|
||||
raise
|
||||
except Exception: # noqa: BLE001 - unknown length, fall back
|
||||
return 0, False
|
||||
|
||||
def _key(self, url: str) -> str:
|
||||
return hashlib.sha256(url.encode("utf-8")).hexdigest()
|
||||
|
||||
def _load_sidecar(self, side: Path) -> _Ranges:
|
||||
if not self.resume or not side.is_file():
|
||||
return []
|
||||
try:
|
||||
return [tuple(r) for r in json.loads(side.read_text(encoding="utf-8"))]
|
||||
except (OSError, ValueError):
|
||||
return []
|
||||
|
||||
@staticmethod
|
||||
def _write_sidecar(side: Path, covered: _Ranges) -> None:
|
||||
side.parent.mkdir(parents=True, exist_ok=True)
|
||||
side.write_text(json.dumps(covered), encoding="utf-8")
|
||||
|
||||
def _fetch_ranged(self, source: Source, total: int, tick) -> None:
|
||||
key = self._key(source.url)
|
||||
part = self.partials_dir / f"{key}.part"
|
||||
side = self.partials_dir / f"{key}.ranges"
|
||||
covered = self._load_sidecar(side)
|
||||
if any(b > total for _, b in covered):
|
||||
covered = [] # server now serves a smaller file: stale partial
|
||||
# Create the file WITHOUT truncating an existing partial (resume),
|
||||
# then size it to total (extends with zeros / shrinks stale tails).
|
||||
open(part, "ab").close()
|
||||
with open(part, "r+b") as f:
|
||||
f.truncate(total)
|
||||
if covered:
|
||||
ranges = _missing(total, covered)
|
||||
else:
|
||||
n = max(1, min(self.connections, (total + _CHUNK - 1) // _CHUNK))
|
||||
ranges = [(i * total // n, (i + 1) * total // n) for i in range(n)]
|
||||
# NOT coalesced: the fresh split is the parallel fan-out, and
|
||||
# adjacent chunks must stay separate workers. (_missing already
|
||||
# returns disjoint gaps, so resume needs no merge either.)
|
||||
ranges = [r for r in ranges if r[1] > r[0]]
|
||||
|
||||
lock = threading.Lock()
|
||||
errors: list[Exception] = []
|
||||
stop = threading.Event()
|
||||
written = [sum(b - a for a, b in covered)]
|
||||
|
||||
def worker(start: int, end: int) -> None:
|
||||
try:
|
||||
req = urllib.request.Request(
|
||||
source.url,
|
||||
headers={**_UA, "Range": f"bytes={start}-{end - 1}"})
|
||||
with _OPENER.open(req, timeout=120) as r, \
|
||||
open(part, "r+b") as f:
|
||||
f.seek(start)
|
||||
pos = start
|
||||
while pos < end:
|
||||
if self._paused.is_set() or stop.is_set():
|
||||
return
|
||||
chunk = r.read(min(_CHUNK, end - pos))
|
||||
if not chunk:
|
||||
break
|
||||
f.write(chunk)
|
||||
pos += len(chunk)
|
||||
with lock:
|
||||
covered[:] = _coalesce(covered + [(start, pos)])
|
||||
written[0] += len(chunk)
|
||||
tick(list(covered))
|
||||
except Exception as exc: # noqa: BLE001
|
||||
with lock:
|
||||
errors.append(exc)
|
||||
stop.set()
|
||||
|
||||
threads = [
|
||||
threading.Thread(target=worker, args=b, daemon=True,
|
||||
name=f"dl-{key[:8]}-{i}")
|
||||
for i, b in enumerate(ranges)
|
||||
]
|
||||
for t in threads:
|
||||
t.start()
|
||||
for t in threads:
|
||||
t.join()
|
||||
|
||||
if self._paused.is_set():
|
||||
self._write_sidecar(side, covered)
|
||||
raise DownloadPaused(source.url)
|
||||
if errors:
|
||||
self._write_sidecar(side, covered)
|
||||
raise errors[0]
|
||||
if written[0] != total:
|
||||
self._write_sidecar(side, covered)
|
||||
raise DownloadError(
|
||||
f"download incomplete ({written[0]} of {total} bytes)")
|
||||
self._finalize(source, part, side)
|
||||
|
||||
def _fetch_single(self, source: Source, total: int, tick) -> None:
|
||||
"""No-Range fallback: one stream. A server without Range support
|
||||
cannot resume, so each run restarts the file."""
|
||||
key = self._key(source.url)
|
||||
part = self.partials_dir / f"{key}.part"
|
||||
side = self.partials_dir / f"{key}.ranges"
|
||||
covered: _Ranges = []
|
||||
req = urllib.request.Request(source.url, headers=_UA)
|
||||
try:
|
||||
with _OPENER.open(req, timeout=120) as r, open(part, "wb") as f:
|
||||
declared = int(r.headers.get("Content-Length") or 0)
|
||||
pos = 0
|
||||
while True:
|
||||
if self._paused.is_set():
|
||||
self._write_sidecar(side, covered)
|
||||
raise DownloadPaused(source.url)
|
||||
chunk = r.read(_CHUNK)
|
||||
if not chunk:
|
||||
break
|
||||
f.write(chunk)
|
||||
pos += len(chunk)
|
||||
covered = [(0, pos)]
|
||||
tick(list(covered))
|
||||
if self._paused.is_set():
|
||||
self._write_sidecar(side, covered)
|
||||
raise DownloadPaused(source.url)
|
||||
if declared and pos != declared:
|
||||
raise DownloadError(
|
||||
f"download ended at {pos:,} bytes but the server "
|
||||
f"said {declared:,} — connection dropped?")
|
||||
if total and pos != total:
|
||||
raise DownloadError(
|
||||
f"download incomplete ({pos} of {total} bytes)")
|
||||
except DownloadError:
|
||||
self._write_sidecar(side, covered)
|
||||
raise
|
||||
except Exception:
|
||||
self._write_sidecar(side, covered)
|
||||
raise
|
||||
self._finalize(source, part, side)
|
||||
|
||||
def _finalize(self, source: Source, part: Path, side: Path) -> None:
|
||||
if source.sha256:
|
||||
actual = _sha256_file(part)
|
||||
if actual != source.sha256:
|
||||
part.unlink(missing_ok=True)
|
||||
side.unlink(missing_ok=True)
|
||||
raise HashError(
|
||||
f"sha256 mismatch for {source.url}: pinned "
|
||||
f"{source.sha256}, got {actual}")
|
||||
source.dest.parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.move(str(part), str(source.dest))
|
||||
side.unlink(missing_ok=True)
|
||||
+538
@@ -0,0 +1,538 @@
|
||||
"""ensure(): make the installed state match the lockfile for a package,
|
||||
and hand back its composed environment."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import shutil
|
||||
from typing import Optional
|
||||
|
||||
from pm import paths
|
||||
from pm.lock import Facts, Lockfile
|
||||
from pm.package import InstallError, Package, Runner, StatePackage, compose_env
|
||||
from pm.registry import get_package, walk
|
||||
from pm.store import Store, current_target, merge_tree, tree_digest
|
||||
|
||||
LOG = logging.getLogger(__name__)
|
||||
|
||||
# ``progress(stage, done, total, label)`` — stage is "download" | "unpack",
|
||||
# label is the archive counter ("1/2") when a package has several. Slow
|
||||
# lines sit in one stage for minutes, so the byte counters are what prove
|
||||
# liveness to a UI.
|
||||
|
||||
|
||||
def _artifact_progress(progress, index: int, count: int):
|
||||
if progress is None:
|
||||
return None
|
||||
label = f"{index + 1}/{count}" if count > 1 else ""
|
||||
return lambda done, total: progress("download", done, total, label)
|
||||
|
||||
|
||||
def _lockfile() -> Lockfile:
|
||||
return Lockfile(paths.lockfile_path())
|
||||
|
||||
|
||||
def _facts() -> Facts:
|
||||
return Facts(paths.facts_path())
|
||||
|
||||
|
||||
def _store() -> Store:
|
||||
return Store(paths.store_root())
|
||||
|
||||
|
||||
def _identity(lockfile: Lockfile, name: str, target: str):
|
||||
"""The identity the lock currently pins for `name` on `target`:
|
||||
(target, tuple(artifact sha256s)) — or None when the lock pins no
|
||||
artifacts (nothing digest-bound to compare)."""
|
||||
artifacts = lockfile.artifacts(name, target)
|
||||
if not artifacts:
|
||||
return None
|
||||
return (target, tuple(a["sha256"] for a in artifacts))
|
||||
|
||||
|
||||
def lazy_installs_allowed() -> bool:
|
||||
"""Policy: may pm install things on demand right now?
|
||||
|
||||
HERMES_DISABLE_LAZY_INSTALLS is an internal bridge var set by the
|
||||
official Docker image and the hermetic test harness. The user-facing
|
||||
setting is security.allow_lazy_installs in config.yaml; a config
|
||||
system that fails to load counts as ALLOWED only when hermes_cli is
|
||||
genuinely absent (bootstrap) — config errors fail closed.
|
||||
"""
|
||||
import os
|
||||
|
||||
if os.environ.get("HERMES_DISABLE_LAZY_INSTALLS", "").strip().lower() in (
|
||||
"1",
|
||||
"true",
|
||||
"yes",
|
||||
):
|
||||
return False
|
||||
try:
|
||||
from hermes_cli.config import get_config_value
|
||||
except ImportError:
|
||||
return True
|
||||
try:
|
||||
return bool(get_config_value("security.allow_lazy_installs", True))
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def enabled_extras() -> list[str]:
|
||||
"""The venv extras recorded in the installed state."""
|
||||
return list((_facts().get("venv") or {}).get("extras", []))
|
||||
|
||||
|
||||
def is_installed(name: str) -> bool:
|
||||
lockfile = _lockfile()
|
||||
facts = _facts()
|
||||
return facts.installed(
|
||||
name,
|
||||
lockfile.version(name),
|
||||
_store().root,
|
||||
_identity(lockfile, name, current_target()),
|
||||
)
|
||||
|
||||
|
||||
def sealed() -> bool:
|
||||
"""A bundled payload is read-only: its store sits beside the bundle
|
||||
manifest. Asking a sealed install for MORE than it shipped is a
|
||||
Sealed = bundled LAYOUT only; adoption verification is adopt()'s job."""
|
||||
return (paths.store_root().parent / "manifest.json").is_file()
|
||||
|
||||
|
||||
def _refuse_lazy(name: str, what: str) -> InstallError:
|
||||
if sealed():
|
||||
return InstallError(
|
||||
name,
|
||||
f"this install is sealed and does not ship: {what}",
|
||||
"the bundle bakes its entire tree at build time; rebuild the bundle",
|
||||
)
|
||||
return InstallError(
|
||||
name,
|
||||
f"not installed and lazy installs are disabled: {what}",
|
||||
"enable security.allow_lazy_installs or run `hermes pm install`",
|
||||
)
|
||||
|
||||
|
||||
def _remove_entry(store: Store, entry_name: str) -> None:
|
||||
"""Remove a published store entry before re-realizing it. NOT
|
||||
fire-and-forget: a silent failure here would leave the stale entry in
|
||||
place and publish() would then keep it (a concurrent-winner guard),
|
||||
re-recording facts over bytes the new lock never produced. Defender /
|
||||
indexer handles on Windows make removal transiently fail, so retry —
|
||||
then fail loudly."""
|
||||
import time
|
||||
|
||||
entry = store.entry(entry_name)
|
||||
for attempt in range(5):
|
||||
try:
|
||||
shutil.rmtree(entry)
|
||||
return
|
||||
except FileNotFoundError:
|
||||
return
|
||||
except OSError as e:
|
||||
if attempt == 4:
|
||||
raise
|
||||
time.sleep(0.2 * (attempt + 1))
|
||||
|
||||
|
||||
def _install(
|
||||
package: Package,
|
||||
lockfile: Lockfile,
|
||||
facts: Facts,
|
||||
store: Store,
|
||||
target: str,
|
||||
progress=None,
|
||||
) -> None:
|
||||
version = lockfile.version(package.name)
|
||||
if version is None:
|
||||
raise InstallError(
|
||||
package.name, "not in the lockfile", "add it with `hermes pm lock --bump`"
|
||||
)
|
||||
|
||||
reason = package.missing_reason(target)
|
||||
if reason is not None:
|
||||
raise InstallError(package.name, f"unavailable on {target}: {reason}", "none")
|
||||
|
||||
artifacts = lockfile.artifacts(package.name, target)
|
||||
entry_name = package.store_entry(version, target)
|
||||
|
||||
with store.install_lock():
|
||||
facts.reload()
|
||||
if facts.installed(
|
||||
package.name, version, store.root, _identity(lockfile, package.name, target)
|
||||
):
|
||||
return
|
||||
entry = store.entry(entry_name)
|
||||
_cond = store.published(entry_name) and package.verify(entry, target) == ""
|
||||
if _cond:
|
||||
_remove_entry(store, entry_name)
|
||||
if not store.published(entry_name):
|
||||
if not artifacts:
|
||||
raise InstallError(
|
||||
package.name,
|
||||
f"no artifact for {target} in the lockfile",
|
||||
"run `hermes pm lock --bump` for this package",
|
||||
)
|
||||
with store.scratch() as scratch:
|
||||
staged = scratch / "tree"
|
||||
try:
|
||||
for index, artifact in enumerate(artifacts):
|
||||
label = f"{index + 1}/{len(artifacts)}" if len(artifacts) > 1 else ""
|
||||
archive = store.fetch(
|
||||
artifact["url"], artifact["sha256"], scratch,
|
||||
progress=_artifact_progress(
|
||||
progress, index, len(artifacts)),
|
||||
)
|
||||
if progress is not None:
|
||||
progress("unpack", 0, 0, label)
|
||||
if index == 0:
|
||||
package.unpack(archive, staged, target)
|
||||
continue
|
||||
# unpack() empties its destination by contract, so
|
||||
# a second archive must be unpacked apart and moved
|
||||
# in — extracting over `staged` would delete the
|
||||
# first archive's files.
|
||||
extra = scratch / f"extra-{index}"
|
||||
package.unpack(archive, extra, target)
|
||||
merge_tree(extra, staged)
|
||||
package.stage(store, staged, version, target)
|
||||
store.publish(staged, entry_name)
|
||||
except InstallError:
|
||||
raise
|
||||
except Exception as e:
|
||||
raise InstallError(package.name, f"install failed: {e}") from e
|
||||
|
||||
reason = package.verify(entry, target)
|
||||
if reason:
|
||||
raise InstallError(package.name, f"published entry failed verification: {reason}")
|
||||
|
||||
previous = facts.get(package.name)
|
||||
if previous and "entry" in previous:
|
||||
# Work item 6: replacing an ESTABLISHED fact is a repair —
|
||||
# log it, no transaction system, no receipt file.
|
||||
old_artifact = (previous.get("artifacts") or ["?"])[0]
|
||||
old = f"{previous.get('version', '?')}/{str(old_artifact)[:12]}"
|
||||
new = (
|
||||
f"{version}/{artifacts[0]['sha256'][:12]}"
|
||||
if artifacts
|
||||
else version
|
||||
)
|
||||
LOG.info("repair: %s re-realized %s -> %s", package.name, old, new)
|
||||
env = package.env(entry, target)
|
||||
facts.record(
|
||||
package.name,
|
||||
version,
|
||||
entry_name,
|
||||
env,
|
||||
store.root,
|
||||
target=target,
|
||||
artifacts=[a["sha256"] for a in artifacts],
|
||||
digest=tree_digest(entry),
|
||||
)
|
||||
if previous and previous.get("version") != version:
|
||||
package.migrate(previous["version"], version)
|
||||
|
||||
|
||||
def ensure(
|
||||
name: str,
|
||||
*,
|
||||
base_env: Optional[dict] = None,
|
||||
explicit: bool = False,
|
||||
progress=None,
|
||||
) -> Runner:
|
||||
"""``explicit`` marks a deliberate install command (`hermes pm
|
||||
install`, `hermes pm bundle`) — those ARE the remedy the lazy-install
|
||||
policy names, so the policy does not apply to them.
|
||||
|
||||
``progress(stage, done, total, label)`` reports the slow parts of an
|
||||
install to a UI; see _artifact_progress.
|
||||
"""
|
||||
if isinstance(get_package(name), StatePackage):
|
||||
sync_venv(explicit=explicit)
|
||||
return Runner(name, compose_env([], base=base_env))
|
||||
|
||||
lockfile = _lockfile()
|
||||
facts = _facts()
|
||||
store = _store()
|
||||
target = current_target()
|
||||
|
||||
chain = walk([name])
|
||||
missing = [
|
||||
p
|
||||
for p in chain
|
||||
if not facts.installed(
|
||||
p.name, lockfile.version(p.name), store.root, _identity(lockfile, p.name, target)
|
||||
)
|
||||
]
|
||||
|
||||
if missing and (sealed() or (not explicit and not lazy_installs_allowed())):
|
||||
raise _refuse_lazy(name, ", ".join(p.name for p in missing))
|
||||
|
||||
for package in missing:
|
||||
_install(package, lockfile, facts, store, target, progress=progress)
|
||||
if missing:
|
||||
facts.reload()
|
||||
|
||||
diffs = [facts.env_for(p.name, store.root) for p in chain]
|
||||
return Runner(name, compose_env(diffs, base=base_env))
|
||||
|
||||
|
||||
def env_for(*names: str, base_env: Optional[dict] = None) -> dict[str, str]:
|
||||
"""Composed env of already-installed packages only. Never installs,
|
||||
never raises on missing packages — they contribute nothing."""
|
||||
facts = _facts()
|
||||
store = _store()
|
||||
lockfile = _lockfile()
|
||||
target = current_target()
|
||||
diffs: list[dict] = []
|
||||
for name in names:
|
||||
try:
|
||||
chain = walk([name])
|
||||
except KeyError:
|
||||
continue
|
||||
for package in chain:
|
||||
if facts.installed(
|
||||
package.name,
|
||||
lockfile.version(package.name),
|
||||
store.root,
|
||||
_identity(lockfile, package.name, target),
|
||||
):
|
||||
diffs.append(facts.env_for(package.name, store.root))
|
||||
return compose_env(diffs, base=base_env)
|
||||
|
||||
|
||||
def sync_venv(extras: Optional[list[str]] = None, *, explicit: bool = False) -> None:
|
||||
"""Make the venv match uv.lock + the enabled extras. Extras union into
|
||||
the installed state (one ledger); no-op when the stamp already matches.
|
||||
``explicit`` marks a deliberate install command (`hermes pm install`,
|
||||
`hermes update`) — those are the remedy the lazy-install policy points
|
||||
at, so the policy does not apply to them."""
|
||||
package = get_package("venv")
|
||||
facts = _facts()
|
||||
fact = facts.get("venv") or {}
|
||||
enabled = sorted(set(fact.get("extras", [])) | set(extras or []))
|
||||
stamp = package.expected_stamp(enabled)
|
||||
if fact.get("stamp") == stamp:
|
||||
return
|
||||
if sealed() or (not explicit and not lazy_installs_allowed()):
|
||||
raise _refuse_lazy("venv", str(extras) if extras else "venv out of sync")
|
||||
with _store().install_lock():
|
||||
facts.reload()
|
||||
fact = facts.get("venv") or {}
|
||||
enabled = sorted(set(fact.get("extras", [])) | set(extras or []))
|
||||
stamp = package.expected_stamp(enabled)
|
||||
if fact.get("stamp") == stamp:
|
||||
return
|
||||
package.apply(enabled)
|
||||
facts.record_state("venv", stamp, enabled)
|
||||
|
||||
|
||||
def adopt() -> bool:
|
||||
"""First boot of a bundled install: verify the shipped payload, then
|
||||
make it THIS machine's installed state. The payload's own shipped
|
||||
``pm/lock.json`` (inside the repo snapshot) is the offline authority:
|
||||
for every package it pins, the shipped fact must record the shipped
|
||||
identity, package.verify() must pass, and the recorded realized digest
|
||||
must match a fresh tree_digest() over the actual bytes. Any failure:
|
||||
log the offending package, do NOT write `.adopted`, return False —
|
||||
adopt() refuses to vouch for bytes it could not prove.
|
||||
|
||||
Idempotent and cheap-ish: returns False when there is nothing to
|
||||
adopt (no shipped facts, or already adopted)."""
|
||||
store = _store()
|
||||
facts = _facts()
|
||||
if not paths.facts_path().is_file():
|
||||
return False
|
||||
|
||||
marker = store.root.parent / ".adopted"
|
||||
if marker.is_file():
|
||||
return False
|
||||
|
||||
shipped_lock = paths.repo_root() / "pm" / "lock.json"
|
||||
if shipped_lock.is_file():
|
||||
lockfile = Lockfile(shipped_lock)
|
||||
target = current_target()
|
||||
for name in lockfile.names():
|
||||
try:
|
||||
package = get_package(name)
|
||||
except KeyError:
|
||||
continue
|
||||
if isinstance(package, StatePackage):
|
||||
continue
|
||||
fact = facts.get(name)
|
||||
if not facts.installed(
|
||||
name, lockfile.version(name), store.root, _identity(lockfile, name, target)
|
||||
):
|
||||
LOG.warning(
|
||||
"pm adopt: refusing %s: fact missing, legacy (no recorded "
|
||||
"identity), or does not match the shipped lock",
|
||||
name,
|
||||
)
|
||||
return False
|
||||
entry = store.entry(fact["entry"])
|
||||
reason = package.verify(entry, target)
|
||||
if reason:
|
||||
LOG.warning(
|
||||
"pm adopt: refusing %s: staged entry failed verification: %s",
|
||||
name, reason,
|
||||
)
|
||||
return False
|
||||
if fact.get("digest") != tree_digest(entry):
|
||||
LOG.warning(
|
||||
"pm adopt: refusing %s: realized bytes do not match the "
|
||||
"recorded digest",
|
||||
name,
|
||||
)
|
||||
return False
|
||||
|
||||
try:
|
||||
marker.write_text("", encoding="utf-8")
|
||||
except OSError:
|
||||
pass
|
||||
return True
|
||||
|
||||
|
||||
def check() -> list[str]:
|
||||
"""The startup check: cheap stamp comparisons of the installed state
|
||||
against the lockfile. Returns problems; empty means healthy. Never
|
||||
installs, never touches the network. An install pm has never touched
|
||||
(no installed-state file) reports nothing — pm only vouches for what
|
||||
it installed. Lockfile packages this build doesn't know (version skew
|
||||
during a partial update) are skipped, not fatal."""
|
||||
if not paths.facts_path().is_file():
|
||||
return []
|
||||
|
||||
problems: list[str] = []
|
||||
lockfile = _lockfile()
|
||||
facts = _facts()
|
||||
store = _store()
|
||||
target = current_target()
|
||||
for name in lockfile.names():
|
||||
try:
|
||||
package = get_package(name)
|
||||
except KeyError:
|
||||
continue
|
||||
if package.optional or package.internal:
|
||||
continue
|
||||
if package.missing_reason(target) is not None:
|
||||
continue
|
||||
if not facts.installed(
|
||||
name, lockfile.version(name), store.root, _identity(lockfile, name, target)
|
||||
):
|
||||
problems.append(f"{name}: not installed or outdated")
|
||||
try:
|
||||
venv = get_package("venv")
|
||||
except KeyError:
|
||||
venv = None
|
||||
fact = facts.get("venv")
|
||||
if venv is not None and fact is not None:
|
||||
expected = venv.expected_stamp(fact.get("extras", []))
|
||||
if fact.get("stamp") != expected:
|
||||
problems.append("venv: out of sync with uv.lock")
|
||||
return problems
|
||||
|
||||
|
||||
def _store_path_dirs() -> list[str]:
|
||||
"""Composed PATH dirs of all installed (non-internal, on_path) store
|
||||
packages, deps-first, deduped. Includes optional packages that are
|
||||
*installed* (facts say so) — an installed git/gh must be on PATH even
|
||||
though it's not in the root closure. Never installs."""
|
||||
import os
|
||||
|
||||
if not paths.facts_path().is_file():
|
||||
return []
|
||||
facts = _facts()
|
||||
store = _store()
|
||||
lockfile = _lockfile()
|
||||
target = current_target()
|
||||
dirs: list[str] = []
|
||||
for name in lockfile.names():
|
||||
try:
|
||||
package = get_package(name)
|
||||
except KeyError:
|
||||
continue
|
||||
if package.internal:
|
||||
continue
|
||||
if not getattr(package, "on_path", True):
|
||||
continue
|
||||
if package.missing_reason(target) is not None:
|
||||
continue
|
||||
if not facts.installed(
|
||||
name, lockfile.version(name), store.root, _identity(lockfile, name, target)
|
||||
):
|
||||
continue
|
||||
env = facts.env_for(name, store.root)
|
||||
path_dirs = env.get("PATH") or []
|
||||
if isinstance(path_dirs, str):
|
||||
path_dirs = [path_dirs]
|
||||
for directory in path_dirs:
|
||||
if directory and directory not in dirs:
|
||||
dirs.append(str(directory))
|
||||
return dirs
|
||||
|
||||
|
||||
def activate() -> None:
|
||||
"""Make the installed store usable: prepend its tool dirs to
|
||||
os.environ['PATH'] so reactive `shutil.which('git'|'bash'|'ffmpeg'|...)`
|
||||
resolves the bundled binaries. The gate is `check()` — if the store is
|
||||
broken, refuse to inject (fail fast rather than serving a partial PATH).
|
||||
|
||||
This is the ONE sanctioned global PATH write: PATH is the discovery
|
||||
contract every `which` reads, not a tool-specific env leak. Store-first
|
||||
unconditionally — pinned bundled versions win on dev machines too.
|
||||
"""
|
||||
import os
|
||||
|
||||
if check():
|
||||
return # broken store → do not provision; callers surface `hermes pm install`
|
||||
dirs = _store_path_dirs()
|
||||
if not dirs:
|
||||
return
|
||||
existing = os.environ.get("PATH", "")
|
||||
prefix = os.pathsep.join(dirs)
|
||||
existing_lower = {p.lower() for p in existing.split(os.pathsep) if p}
|
||||
missing = [d for d in dirs if d.lower() not in existing_lower]
|
||||
if missing:
|
||||
os.environ["PATH"] = os.pathsep.join([*missing, existing]) if existing else os.pathsep.join(missing)
|
||||
|
||||
|
||||
|
||||
def uv(*, venv=None, realize: bool = True):
|
||||
"""TRANSITIONAL: (uv path, sanitized env) for call sites that still
|
||||
drive uv themselves. Two classes remain: update/repair sites (die with
|
||||
the update collapse, plan step 4) and side-venv installs — browser-use
|
||||
tool venvs (tools_config, browser_use_cli) and hindsight's
|
||||
local_embedded daemon — which survive until pm grows the side-venv
|
||||
package kind (plan step 5's remaining half). Must not spread."""
|
||||
from pm.packages import uv_env
|
||||
|
||||
env = uv_env()
|
||||
if venv is not None:
|
||||
env["VIRTUAL_ENV"] = str(venv)
|
||||
env.pop("UV_NO_CONFIG", None)
|
||||
|
||||
lockfile = _lockfile()
|
||||
facts = _facts()
|
||||
store = _store()
|
||||
package = get_package("uv")
|
||||
if not facts.installed(
|
||||
"uv", lockfile.version("uv"), store.root, _identity(lockfile, "uv", current_target())
|
||||
):
|
||||
if not realize or not lazy_installs_allowed():
|
||||
return None, env
|
||||
try:
|
||||
_install(package, lockfile, facts, store, current_target())
|
||||
facts.reload()
|
||||
except Exception:
|
||||
import logging
|
||||
|
||||
logging.getLogger(__name__).debug("pm.uv: install failed", exc_info=True)
|
||||
return None, env
|
||||
fact = facts.get("uv")
|
||||
if fact is None:
|
||||
return None, env
|
||||
binary = package.binary(store.entry(fact["entry"]), current_target())
|
||||
if binary is None or not binary.is_file():
|
||||
return None, env
|
||||
return str(binary), env
|
||||
Loaded 100 of 272 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user