feat(pm): unified package manager — pm store foundation

Introduce the pm store: a unified, hash-verified package store that
replaces lazy_deps and the old installer's ad-hoc tool downloads.
Store tools are provisioned on PATH (ffmpeg, node/npm via pinned uv),
with a resumable 8-way downloader, verify() returning failure reasons,
and adopt() made EPERM-safe. chromium ships in the payload for every
target. The 3600-line install.sh is replaced by a staged bootstrapper
(heavy deps are pm's job after this); setup-hermes.sh, Dockerfile and
nix pin tables are rewired onto the store. Old install-script tests,
lazy_deps/managed_uv/build_info, and the ps1/bash installer test
batteries are removed with the machinery they tested.

Rebuilt from ethie/pm onto upstream/main (ac6c8028e0) after the
utf-8-sig sweep. 16 hot files (main also churned them) hand-merged:
platform adapters, main.py, electron/main.ts, tui_gateway/server.py,
cua_backend, installer-tests workflow, install.sh (full rewrite),
setup-hermes.sh, plugins doc.
This commit is contained in:
ethernet committed 2026-08-31 18:00:48 -04:00
1 parent 6e294f543d
commit 3d12e86ef1
272 files changed
+18064 -14720

No files matched your search

@@ -48,6 +48,9 @@ outputs:
installer:
description: Run the PowerShell installer tests on a Windows runner.
value: ${{ steps.classify.outputs.installer }}
bootstrap:
description: Run the bootstrap installer lane (install.sh sandbox + stamp verification).
value: ${{ steps.classify.outputs.bootstrap }}
rust:
description: Run `cargo test` for the Tauri bootstrap installer.
value: ${{ steps.classify.outputs.rust }}
+137
View File
@@ -0,0 +1,137 @@
name: Bootstrap installer
# Exercises the bootstrap-installer path on PRs that can affect it: the
# POSIX shell installer (scripts/install.sh), its generated pin fragments,
# and the version stamp the `complete` stage ships. The PowerShell installer
# keeps its own Windows-only lane (installer-tests.yml); this lane runs the
# cheap cross-checks plus a real sandboxed install against the PR checkout.
#
# Deliberately minimal: the heavy `python-deps` stage (uv sync of every
# extra) is skipped — the pm/uv machinery under it is covered by the Python
# lanes, and pulling the whole dependency graph per installer PR would make
# this lane slower than everything it protects.
on:
workflow_call:
permissions:
contents: read
concurrency:
group: bootstrap-installer-${{ github.ref }}
cancel-in-progress: true
jobs:
posix:
name: install.sh sandbox install + stamp verification
runs-on: ubuntu-24.04
timeout-minutes: 15
env:
INSTALL_DIR: ${{ runner.temp }}/bootstrap-install
HERMES_HOME: ${{ runner.temp }}/hermes-home
HERMES_RUNTIME_DIR: ${{ runner.temp }}/hermes-tools
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Stage manifest is well-formed protocol v1
shell: bash
run: |
bash scripts/install.sh --manifest | python3 -c '
import json, sys
m = json.load(sys.stdin)
assert m["protocol_version"] == 1, m
names = [s["name"] for s in m["stages"]]
expected = ["prerequisites", "repository", "venv", "python-deps",
"node-deps", "path", "config", "setup", "gateway", "complete"]
assert names == expected, names
assert m["stages"][-1]["name"] == "complete"
print("stage manifest ok:", " -> ".join(names))
'
- name: Generated bootstrap pins match pm/lock.json
shell: bash
run: python3 scripts/gen-bootstrap-pins.py --check
- name: Publish the PR checkout as the install source
id: source
shell: bash
run: |
# The installer clones --branch <branch>; a PR checkout is a
# detached HEAD, so mirror it onto a named branch first.
mirror="$RUNNER_TEMP/source-mirror.git"
git init --bare "$mirror"
sha="$(git rev-parse HEAD)"
git push "$mirror" "HEAD:refs/heads/ci-under-test"
echo "mirror=$mirror" >> "$GITHUB_OUTPUT"
echo "sha=$sha" >> "$GITHUB_OUTPUT"
- name: Run the installer stages against the PR checkout
shell: bash
env:
HERMES_REPO_URL: ${{ steps.source.outputs.mirror }}
run: |
set -euo pipefail
sha="${{ steps.source.outputs.sha }}"
run_stage() {
echo "::group::stage $1"
bash scripts/install.sh --branch ci-under-test --commit "$sha" \
--non-interactive --stage "$1" --json
echo "::endgroup::"
}
run_stage prerequisites
run_stage repository
run_stage venv
# python-deps (uv sync --extra all) is deliberately skipped — the
# comment at the top of this file explains why.
run_stage node-deps
run_stage path
run_stage config
run_stage complete
test -f "$INSTALL_DIR/.hermes-bootstrap-complete"
- name: Verify the shipped version stamp
shell: bash
run: |
python3 scripts/verify-bootstrap-version-stamp.py \
--stamp "$INSTALL_DIR/.hermes-bootstrap-complete" \
--repo "$INSTALL_DIR" \
--expect-commit "${{ steps.source.outputs.sha }}" \
--expect-branch ci-under-test
- name: The pinned commit is on the branch the installer cloned
shell: bash
run: |
# The stamp must describe the bytes actually checked out: the
# installed repo's HEAD is exactly the commit the installer pinned.
head="$(git -C "$INSTALL_DIR" rev-parse HEAD)"
test "$head" = "${{ steps.source.outputs.sha }}" \
|| { echo "::error::installed HEAD $head != pinned ${{ steps.source.outputs.sha }}"; exit 1; }
windows:
name: install.ps1 protocol surface
runs-on: windows-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
# Windows PowerShell 5.1 is what ships with Windows and what `irm | iex`
# lands in for most users — the protocol surface must parse there.
- name: Protocol version + stage manifest (Windows PowerShell 5.1)
shell: powershell
run: |
$pv = powershell -NoProfile -ExecutionPolicy Bypass -File scripts/install.ps1 -ProtocolVersion
if ("$pv" -ne "1") { Write-Error "protocol version '$pv' != 1"; exit 1 }
$json = powershell -NoProfile -ExecutionPolicy Bypass -File scripts/install.ps1 -Manifest | ConvertFrom-Json
if ($json.protocol_version -ne 1) { Write-Error "manifest protocol_version != 1"; exit 1 }
$names = @($json.stages | ForEach-Object { $_.name })
if ($names -notcontains "complete") { Write-Error "manifest missing complete stage"; exit 1 }
Write-Host "stage manifest ok: $($names -join ' -> ')"
- name: Protocol version + stage manifest (pwsh 7)
shell: pwsh
run: |
$pv = pwsh -NoProfile -File scripts/install.ps1 -ProtocolVersion
if ("$pv" -ne "1") { Write-Error "protocol version '$pv' != 1"; exit 1 }
$json = pwsh -NoProfile -File scripts/install.ps1 -Manifest | ConvertFrom-Json
if ($json.protocol_version -ne 1) { Write-Error "manifest protocol_version != 1"; exit 1 }
Write-Host "stage manifest ok (pwsh 7)"
+11
View File
@@ -49,6 +49,7 @@ jobs:
uv_lock: ${{ steps.classify.outputs.uv_lock }}
npm_lock: ${{ steps.classify.outputs.npm_lock }}
installer: ${{ steps.classify.outputs.installer }}
bootstrap: ${{ steps.classify.outputs.bootstrap }}
rust: ${{ steps.classify.outputs.rust }}
docker_meta: ${{ steps.classify.outputs.docker_meta }}
mcp_catalog: ${{ steps.classify.outputs.mcp_catalog }}
@@ -114,6 +115,15 @@ jobs:
if: needs.detect.outputs.rust == 'true'
uses: ./.github/workflows/rust-tests.yml
bootstrap-installer:
name: Bootstrap installer
needs: detect
# The bootstrap-installer path: install.sh, the pin fragments it embeds,
# and the version stamp it ships. The PowerShell installer has its own
# Windows-only lane above (installer-tests).
if: needs.detect.outputs.bootstrap == 'true'
uses: ./.github/workflows/bootstrap-installer.yml
e2e-desktop:
name: Desktop E2E
needs: detect
@@ -223,6 +233,7 @@ jobs:
- js-tests
- installer-tests
- rust-tests
- bootstrap-installer
- e2e-desktop
- docs-site
- history-check
@@ -0,0 +1,675 @@
name: Desktop Bundled Release
# Builds the bundled desktop installer for each (os, arch) target.
#
# tag admission (shape + pyproject lockstep + ancestry-on-origin/main,
# resolved to an IMMUTABLE full SHA) → 6-target matrix → pm bundle
# → stage artifacts to R2 (releases/tag/<tag>/) → finalize job merges
# the darwin feeds (releases/darwin/<ch>/<channel>-mac.yml + dmg/zip),
# the msixbundle job stages the win32 App Installer feeds
# (releases/win32/<ch>/*.appinstaller + *.msixbundle) → builds table.
# The GitHub release keeps the notes only — no binaries.
#
# Feed layout (matches apps/desktop/electron/app-updater.ts's arms):
# releases/win32/<stable|nightly>/<ch>.appinstaller App Installer feed
# releases/win32/<stable|nightly>/*.msixbundle (msixbundle job)
# releases/darwin/<stable|nightly>/*-mac.yml electron-updater (dmg/zip)
# The finalize + msixbundle jobs write the feeds ONCE after the whole matrix
# is green, so a failed leg can never publish a partial channel.
#
# Payload staging is `hermes pm bundle` on the native runner. There is
# no cross-target staging. Signing and notarization are a later commit;
# this file produces unsigned artifacts on forks. When the
# release-signing environment has the Apple and Azure identifiers,
# the same job signs and notarizes.
#
# scripts/build-bundled-desktop.mjs is the one driver. Local and CI run
# the same command. This workflow adds caching and upload only.
#
# Triggers: workflow_dispatch with an explicit tag. A tag push does not
# start this workflow (a bot-pushed nightly tag would never fire).
#
# R2 secrets (repo-level or the release-signing environment): the R2
# account id + an R2 API token (S3-compatible) with read/write on the
# release bucket; CLOUDFLARE_R2_BUCKET and CLOUDFLARE_R2_PUBLIC_URL are
# non-secret vars. scripts/r2-release.mjs derives the S3 endpoint from
# the account id and needs only node — no npm ci, no extra deps.
on:
workflow_dispatch:
inputs:
tag:
description: 'Release tag to bundle (vX.Y.Z or vX.Y.0-nightly.YYYYMMDDHHMMSS). Must exist on the repo.'
required: true
type: string
upload_release:
description: 'Upload artifacts to the R2 release bucket (staging + feeds)'
required: false
type: boolean
default: false
permissions:
contents: write
id-token: write
concurrency:
group: desktop-bundled-release-${{ inputs.tag }}
cancel-in-progress: false
jobs:
validate:
name: Validate the tag
runs-on: ubuntu-24.04
timeout-minutes: 5
outputs:
# The tag's commit, resolved ONCE here and exported as a full SHA.
# Every privileged job checks out THIS — never the tag ref, which a
# force-push can move between the validate and build jobs.
sha: ${{ steps.admission.outputs.sha }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: ${{ inputs.tag }}
# Full history: the admission gate below runs merge-base against
# origin/main, which needs the shared ancestry, not a depth-1 tip.
fetch-depth: 0
fetch-tags: true
- name: Validate tag shape, pyproject lockstep, and ancestry on origin/main
id: admission
env:
TAG: ${{ inputs.tag }}
run: |
case "$TAG" in
v[0-9]*.[0-9]*.[0-9]*-nightly.20[0-9][0-9][0-9][0-9][0-9][0-9]*)
echo "nightly tag: $TAG"
;;
v[0-9]*.[0-9]*.[0-9]*)
echo "tag: $TAG"
PYVER=$(sed -n 's/^version = "\([^"]*\)"/\1/p' pyproject.toml | head -1)
if [ "$TAG" != "v$PYVER" ]; then
echo "::error::tag $TAG does not match pyproject.toml version $PYVER"
exit 1
fi
;;
*)
echo "::error::'$TAG' is not a release tag (vX.Y.Z or vX.Y.0-nightly.YYYYMMDDHHMMSS)"
exit 1
;;
esac
SHA="$(git rev-parse --verify "$TAG^{commit}")"
if [ -z "$SHA" ]; then
echo "::error::could not resolve $TAG to a commit"
exit 1
fi
# A correctly-shaped tag on an unreviewed commit is NOT a valid
# build input for a release-signing workflow: only a commit that
# is already an ancestor of origin/main may be built.
git fetch origin main
if ! git merge-base --is-ancestor "$SHA" origin/main; then
echo "::error::tag $TAG (${SHA:0:12}) is not an ancestor of origin/main — refusing to build from an unreviewed commit"
exit 1
fi
echo "tag $TAG resolves to $SHA (on origin/main)"
echo "sha=$SHA" >> "$GITHUB_OUTPUT"
build:
name: bundled ${{ matrix.target.label }}
needs: validate
runs-on: ${{ matrix.target.runner }}
environment: release-signing
timeout-minutes: 90
strategy:
fail-fast: false
matrix:
target:
- label: linux-x64
runner: ubuntu-24.04
- label: linux-arm64
runner: ubuntu-24.04-arm
- label: darwin-arm64
runner: macos-15
- label: darwin-x64
runner: macos-15-intel
- label: win32-x64
runner: windows-2025
- label: win32-arm64
runner: windows-11-arm
env:
HERMES_DESKTOP_VARIANT: bundled
HERMES_PAYLOAD_TAG: ${{ inputs.tag }}
ELECTRON_BUILDER_CACHE: ${{ github.workspace }}/.cache/electron-builder
ELECTRON_CACHE: ${{ github.workspace }}/.cache/electron
electron_config_cache: ${{ github.workspace }}/.cache/electron
CLOUDFLARE_R2_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_R2_ACCOUNT_ID }}
CLOUDFLARE_R2_ACCESS_KEY_ID: ${{ secrets.CLOUDFLARE_R2_ACCESS_KEY_ID }}
CLOUDFLARE_R2_SECRET_ACCESS_KEY: ${{ secrets.CLOUDFLARE_R2_SECRET_ACCESS_KEY }}
CLOUDFLARE_R2_BUCKET: ${{ vars.CLOUDFLARE_R2_BUCKET }}
CLOUDFLARE_R2_PUBLIC_URL: ${{ vars.CLOUDFLARE_R2_PUBLIC_URL }}
steps:
- name: Disable Spotlight indexing and XProtect
if: startsWith(matrix.target.label, 'darwin-')
shell: bash
run: |
sudo mdutil -a -i off || true
sudo pkill -9 XProtect >/dev/null || true
while pgrep XProtect; do sleep 3; done
- name: Exclude build write paths from Defender scanning
if: startsWith(matrix.target.label, 'win32-')
shell: powershell
run: |
$paths = @(
$env:GITHUB_WORKSPACE,
$env:RUNNER_TEMP,
(npm config get cache)
)
foreach ($p in $paths) {
try {
Add-MpPreference -ExclusionPath $p
Write-Host "Defender exclusion added for $p"
} catch {
Write-Host "Defender exclusion not applied for $p ($($_.Exception.Message)) - continuing"
}
}
try {
Set-MpPreference -DisableRealtimeMonitoring $true
} catch {
Write-Host "Could not disable windows defender"
}
# Check out the SHA the validate job admitted — never the tag ref,
# which a force-push can move between jobs. This is the privileged
# (release-signing) build; it must run the reviewed bytes.
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: ${{ needs.validate.outputs.sha }}
fetch-tags: true
- name: Resolve toolchain pins from pm/lock.json
id: pins
shell: bash
# The host toolchain that BUILDS the artifact comes from the same
# pin table as the embedded runtimes (pm/lock.json), so gate == pin
# by construction in build-bundled-desktop.mjs's toolchain gates.
run: |
python -c '
import json
pkgs = json.load(open("pm/lock.json"))["packages"]
for tool in ("node", "npm", "uv"):
print(tool + "=" + pkgs[tool]["version"])
' >> "$GITHUB_OUTPUT"
- name: Resolve toolchain cache key
id: toolchain
shell: bash
run: |
node -e '
const l = require("./package-lock.json")
const el = l.packages["apps/desktop/node_modules/electron"].version
const eb = l.packages["node_modules/electron-builder"].version
if (!el || !eb) process.exit(1)
console.log(`electron=${el}`)
console.log(`builder=${eb}`)
' >> "$GITHUB_OUTPUT"
- uses: actions/setup-node@v7.0.0 # immutable release. safe to pin.
with:
node-version: ${{ steps.pins.outputs.node }}
cache: npm
- name: Install pinned npm
shell: bash
env:
NPM_PIN: ${{ steps.pins.outputs.npm }}
run: npm --version | grep -qx "$NPM_PIN" || npm i -g "npm@$NPM_PIN"
- uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # 9.0.0
with:
version: ${{ steps.pins.outputs.uv }}
enable-cache: false
- name: Cache vcpkg OpenSSL (arm64)
if: matrix.target.label == 'win32-arm64'
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: C:\vcpkg\installed\arm64-windows-static-md
key: vcpkg-openssl-arm64-windows-static-md-${{ runner.os }}
- name: Install OpenSSL (arm64)
if: matrix.target.label == 'win32-arm64'
shell: bash
run: |
if [ ! -f "/c/vcpkg/installed/arm64-windows-static-md/lib/libcrypto.lib" ]; then
"$VCPKG_INSTALLATION_ROOT/vcpkg" install openssl:arm64-windows-static-md
fi
{
printf 'OPENSSL_DIR=C:\\vcpkg\\installed\\arm64-windows-static-md\n'
printf 'OPENSSL_STATIC=1\n'
} >> "$GITHUB_ENV"
- name: Cache pm store
# Tag-dispatched runs (every nightly) scope actions/cache under the
# dispatch ref, which GitHub mangles to refs/heads/refs/tags/<tag> —
# a different scope per tag, so an exact key can never be restored
# by a later nightly. The content key below is stable across tags
# when pm/lock.json + uv.lock are unchanged; the restore-keys prefix
# (which ignores the tag entirely) rescues the previous nightly's
# store when the locks DID move.
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: apps/desktop/build/agent-payload/tools
key: pm-store-${{ matrix.target.label }}-${{ hashFiles('pm/lock.json', 'uv.lock') }}
restore-keys: |
pm-store-${{ matrix.target.label }}-
- name: Resolve electron's default download cache path
shell: bash
run: |
# @electron/get does NOT honor ELECTRON_CACHE/electron_config_cache:
# the electron-builder build's electron zip download uses the
# default env-paths cache root. It must be in the actions/cache
# path list or every build re-downloads electron (~115MB).
case "$RUNNER_OS" in
Windows) echo "ELECTRON_DEFAULT_CACHE=$LOCALAPPDATA/electron/Cache" >> "$GITHUB_ENV" ;;
macOS) echo "ELECTRON_DEFAULT_CACHE=$HOME/Library/Caches/electron" >> "$GITHUB_ENV" ;;
*) echo "ELECTRON_DEFAULT_CACHE=$HOME/.cache/electron" >> "$GITHUB_ENV" ;;
esac
- name: Cache electron + electron-builder toolchain
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
${{ github.workspace }}/.cache/electron-builder
${{ github.workspace }}/.cache/electron
${{ env.ELECTRON_DEFAULT_CACHE }}
# eb2: bumped from eb- (2026-08-28) — the old eb- caches never
# contained the electron zip (it lives in @electron/get's default
# cache root, which wasn't in the path list), so every build
# re-downloaded electron. The new key forces a fresh save that
# includes the default cache root.
key: eb2-${{ runner.os }}-${{ runner.arch }}-electron-${{ steps.toolchain.outputs.electron }}-builder-${{ steps.toolchain.outputs.builder }}
# An electron/builder bump misses the exact key, but the previous
# dist is still mostly reusable (electron's postinstall skips the
# download when dist/ exists) — restore it and let npm ci top up.
restore-keys: |
eb2-${{ runner.os }}-${{ runner.arch }}-
- name: Cache node_modules
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
node_modules
apps/*/node_modules
ui-tui/node_modules
ui-tui/packages/*/node_modules
web/node_modules
tests-js/node_modules
key: node-modules-${{ matrix.target.label }}-node${{ steps.pins.outputs.node }}-npm${{ steps.pins.outputs.npm }}-${{ hashFiles('package-lock.json') }}
# npm ci rm -rf's node_modules before installing, so a restore is
# never shipped stale — but a restore-key hit still makes the
# reinstall incremental (postinstall outputs like node-pty's
# prebuilds/ and esbuild's platform binary survive in place).
# The build-bundled install-stamp gate (lock sha + node + npm +
# target) is the real guard against stale trees shipping.
restore-keys: |
node-modules-${{ matrix.target.label }}-node${{ steps.pins.outputs.node }}-npm${{ steps.pins.outputs.npm }}-
- name: Cache node-pty prebuilds (postinstall output, emulated-gyp tax on arm64)
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
node_modules/node-pty/prebuilds
node_modules/node-pty/build
key: node-pty-prebuilds-${{ matrix.target.label }}-${{ hashFiles('package-lock.json') }}
restore-keys: |
node-pty-prebuilds-${{ matrix.target.label }}-
- name: Azure login (OIDC)
if: startsWith(matrix.target.label, 'win32-') && vars.AZURE_CLIENT_ID != ''
uses: azure/login@f5d393ae46f8fde4be8b75f32e3fc50e654ad0ca # v3.0.1
with:
client-id: ${{ vars.AZURE_CLIENT_ID }}
tenant-id: ${{ vars.AZURE_TENANT_ID }}
allow-no-subscriptions: true
- name: Mint federated token for the signing dlib
if: startsWith(matrix.target.label, 'win32-') && vars.AZURE_CLIENT_ID != ''
shell: bash
run: |
file="$RUNNER_TEMP/azure-federated-token"
mint() {
curl -sSf -H "Authorization: bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" \
"$ACTIONS_ID_TOKEN_REQUEST_URL&audience=api://AzureADTokenExchange" \
| jq -r .value > "$file.tmp" && mv -f "$file.tmp" "$file"
}
mint
( while sleep 240; do mint || true; done ) &
echo "AZURE_FEDERATED_TOKEN_FILE=$file" >> "$GITHUB_ENV"
- name: Write App Store Connect key for notarytool
if: startsWith(matrix.target.label, 'darwin-')
shell: bash
env:
APPLE_API_KEY_P8: ${{ secrets.APPLE_API_KEY_P8 }}
run: |
if [ -n "$APPLE_API_KEY_P8" ]; then
printf '%s\n' "$APPLE_API_KEY_P8" > "$RUNNER_TEMP/apple-api-key.p8"
echo "APPLE_API_KEY=$RUNNER_TEMP/apple-api-key.p8" >> "$GITHUB_ENV"
else
echo "APPLE_API_KEY_P8 secret not set — notarization will be skipped"
fi
- name: Build and package
shell: bash
timeout-minutes: 85
env:
PYTHONUTF8: '1'
CC_aarch64_pc_windows_msvc: ${{ matrix.target.label == 'win32-arm64' && 'clang' || '' }}
DEBUG: 'electron-osx-sign*,electron-notarize*'
APPLE_API_KEY_ID: ${{ secrets.APPLE_API_KEY_ID }}
APPLE_API_ISSUER: ${{ secrets.APPLE_API_ISSUER }}
CSC_LINK: ${{ secrets.CSC_LINK }}
CSC_KEY_PASSWORD: ${{ secrets.CSC_KEY_PASSWORD }}
AZURE_SIGN_ENDPOINT: ${{ vars.AZURE_SIGN_ENDPOINT }}
AZURE_SIGN_ACCOUNT: ${{ vars.AZURE_SIGN_ACCOUNT }}
AZURE_SIGN_PROFILE: ${{ vars.AZURE_SIGN_PROFILE }}
AZURE_SIGN_PUBLISHER: ${{ vars.AZURE_SIGN_PUBLISHER }}
AZURE_CLIENT_ID: ${{ vars.AZURE_CLIENT_ID }}
AZURE_TENANT_ID: ${{ vars.AZURE_TENANT_ID }}
AZURE_TOKEN_CREDENTIALS: prod
run: |
if [ "$RUNNER_OS" = "macOS" ]; then
ulimit -n 16384 2>/dev/null || true
echo "file descriptor limit: soft=$(ulimit -Sn) hard=$(ulimit -Hn)"
fi
node scripts/build-bundled-desktop.mjs --tag="$HERMES_PAYLOAD_TAG" --variant=bundled
# The Store-submission MSIX is the same bundled payload re-packed
# with the Partner Center packaging identity (msixbundle job uploads
# it to the tag archive; it never enters a feed dir).
if [ "$RUNNER_OS" = "Windows" ]; then
node scripts/build-bundled-desktop.mjs --tag="$HERMES_PAYLOAD_TAG" --variant=store
fi
# Smoke the bundled payload on its native linux runner before it is
# packed into the desktop artifact: the staged interpreter boots
# hermes_cli out of the payload with no network and no PYTHONPATH.
# This is the ONE linux build+smoke place — pm-bundle.yml no longer
# stages or uploads a duplicate linux payload.
- name: Smoke test the payload
if: startsWith(matrix.target.label, 'linux-')
shell: bash
run: bash scripts/smoke-payload.sh apps/desktop/build/agent-payload
- name: Audit bundle architecture
shell: bash
run: |
MATRIX_LABEL="${{ matrix.target.label }}"
node apps/desktop/scripts/audit-bundle-arch.mjs \
--arch="${MATRIX_LABEL##*-}" --root=apps/desktop/release
- name: Upload artifacts
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
# The mac feed ymls for finalize + (win32 legs) the actual .msix +
# Store-.msix binaries for the msixbundle job to bundle and stage.
# The mac/linux/AppImage binaries go straight to R2 from each leg.
name: hermes-bundled-${{ matrix.target.label }}-${{ inputs.tag }}
path: |
apps/desktop/release/*-mac.yml
apps/desktop/release/*.msix
retention-days: 30
if-no-files-found: warn
- name: Stage to Cloudflare R2
if: inputs.upload_release == true
shell: bash
# Every artifact goes to releases/tag/<tag>/ (immutable staging).
# The finalize job (below) merges these into the win32/darwin feed
# dirs after the whole matrix is green — never from a leg, so a
# failed leg cannot publish a partial feed.
run: |
shopt -s nullglob
# Store-*.msix are archived by the dedicated loop below — exclude
# them here so they are not uploaded twice.
files=(apps/desktop/release/*.dmg apps/desktop/release/*.zip \
apps/desktop/release/*.msix \
apps/desktop/release/*.AppImage \
apps/desktop/release/*.blockmap apps/desktop/release/latest*.yml \
apps/desktop/release/nightly*.yml)
if [ ${#files[@]} -eq 0 ]; then
echo "::error::no release artifacts found"; exit 1
fi
for f in "${files[@]}"; do
case "$f" in
*/Store-*) continue ;; # archived by the Store loop below
esac
node scripts/r2-release.mjs put \
--tag "$HERMES_PAYLOAD_TAG" --key "$(basename "$f")" --file "$f"
done
# The Store-submission MSIX (built by the win legs) goes to the tag
# archive too — hidden from the builds table and never a feed dir.
for f in apps/desktop/release/Store-*.msix; do
[ -f "$f" ] || continue
node scripts/r2-release.mjs put \
--tag "$HERMES_PAYLOAD_TAG" --key "$(basename "$f")" --file "$f"
done
finalize:
name: Publish feeds to R2
needs: [validate, build]
if: inputs.upload_release == true
runs-on: ubuntu-24.04
environment: release-signing
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
# Privileged job: pin to the SHA validate admitted, not the tag.
ref: ${{ needs.validate.outputs.sha }}
# Full checkout: r2-release.mjs imports ./msix-shared.mjs, so a
# sparse checkout of scripts/r2-release.mjs alone would die with
# ERR_MODULE_NOT_FOUND before the finalize run starts.
- name: Download staged artifacts
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
pattern: hermes-bundled-*-${{ inputs.tag }}
path: staged
merge-multiple: true
- name: Finalize feeds
env:
HERMES_PAYLOAD_TAG: ${{ inputs.tag }}
CLOUDFLARE_R2_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_R2_ACCOUNT_ID }}
CLOUDFLARE_R2_ACCESS_KEY_ID: ${{ secrets.CLOUDFLARE_R2_ACCESS_KEY_ID }}
CLOUDFLARE_R2_SECRET_ACCESS_KEY: ${{ secrets.CLOUDFLARE_R2_SECRET_ACCESS_KEY }}
CLOUDFLARE_R2_BUCKET: ${{ vars.CLOUDFLARE_R2_BUCKET }}
run: node scripts/r2-release.mjs finalize --tag "$HERMES_PAYLOAD_TAG" --dir staged
msixbundle:
name: Bundle the win32 MSIX feeds
needs: [validate, build]
if: inputs.upload_release == true
runs-on: windows-2025
environment: release-signing
timeout-minutes: 45
env:
HERMES_PAYLOAD_TAG: ${{ inputs.tag }}
ELECTRON_BUILDER_CACHE: ${{ github.workspace }}/.cache/electron-builder
CLOUDFLARE_R2_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_R2_ACCOUNT_ID }}
CLOUDFLARE_R2_ACCESS_KEY_ID: ${{ secrets.CLOUDFLARE_R2_ACCESS_KEY_ID }}
CLOUDFLARE_R2_SECRET_ACCESS_KEY: ${{ secrets.CLOUDFLARE_R2_SECRET_ACCESS_KEY }}
CLOUDFLARE_R2_BUCKET: ${{ vars.CLOUDFLARE_R2_BUCKET }}
CLOUDFLARE_R2_PUBLIC_URL: ${{ vars.CLOUDFLARE_R2_PUBLIC_URL }}
AZURE_SIGN_ENDPOINT: ${{ vars.AZURE_SIGN_ENDPOINT }}
AZURE_SIGN_ACCOUNT: ${{ vars.AZURE_SIGN_ACCOUNT }}
AZURE_SIGN_PROFILE: ${{ vars.AZURE_SIGN_PROFILE }}
AZURE_SIGN_PUBLISHER: ${{ vars.AZURE_SIGN_PUBLISHER }}
steps:
# Full checkout: stage-msixbundle.mjs imports ./msix-shared.mjs and the
# job runs it with no checkout at all today — ERR_MODULE_NOT_FOUND.
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
# Privileged job: pin to the SHA validate admitted, not the tag.
ref: ${{ needs.validate.outputs.sha }}
# stage-msixbundle derives the nightly MSIX build number as
# minutes-since-the-last-stable from git tags — must see them.
fetch-tags: true
- name: Resolve toolchain pins from pm/lock.json
id: pins
shell: bash
run: |
python -c '
import json
pkgs = json.load(open("pm/lock.json"))["packages"]
for tool in ("node", "npm", "uv"):
print(tool + "=" + pkgs[tool]["version"])
' >> "$GITHUB_OUTPUT"
- name: Resolve toolchain cache key
id: toolchain
shell: bash
run: |
node -e '
const l = require("./package-lock.json")
const el = l.packages["apps/desktop/node_modules/electron"].version
const eb = l.packages["node_modules/electron-builder"].version
if (!el || !eb) process.exit(1)
console.log(`electron=${el}`)
console.log(`builder=${eb}`)
' >> "$GITHUB_OUTPUT"
# makeappx + signtool live in the winCodeSign toolset that
# electron-builder downloads into its cache during the build legs; the
# msixbundle job runs on a fresh runner, so restore the same eb2 cache
# the win32 legs saved. The path list MUST match the build legs'
# byte-for-byte — actions/cache derives the version hash from the paths
# input, so a shorter list computes a different version and the restore
# misses ("Cache not found") even with the identical key.
- name: Resolve electron's default download cache path
shell: bash
run: |
case "$RUNNER_OS" in
Windows) echo "ELECTRON_DEFAULT_CACHE=$LOCALAPPDATA/electron/Cache" >> "$GITHUB_ENV" ;;
macOS) echo "ELECTRON_DEFAULT_CACHE=$HOME/Library/Caches/electron" >> "$GITHUB_ENV" ;;
*) echo "ELECTRON_DEFAULT_CACHE=$HOME/.cache/electron" >> "$GITHUB_ENV" ;;
esac
- name: Cache electron + electron-builder toolchain
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: |
${{ github.workspace }}/.cache/electron-builder
${{ github.workspace }}/.cache/electron
${{ env.ELECTRON_DEFAULT_CACHE }}
key: eb2-${{ runner.os }}-${{ runner.arch }}-electron-${{ steps.toolchain.outputs.electron }}-builder-${{ steps.toolchain.outputs.builder }}
restore-keys: |
eb2-${{ runner.os }}-${{ runner.arch }}-
- name: Download win legs' MSIX
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
pattern: hermes-bundled-win32-*-${{ inputs.tag }}
path: apps/desktop/release
merge-multiple: true
- name: Azure login (OIDC)
uses: azure/login@f5d393ae46f8fde4be8b75f32e3fc50e654ad0ca # v3.0.1
with:
client-id: ${{ vars.AZURE_CLIENT_ID }}
tenant-id: ${{ vars.AZURE_TENANT_ID }}
allow-no-subscriptions: true
- name: Mint federated token for the signing dlib
shell: bash
run: |
file="$RUNNER_TEMP/azure-federated-token"
mint() {
curl -sSf -H "Authorization: bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" \
"$ACTIONS_ID_TOKEN_REQUEST_URL&audience=api://AzureADTokenExchange" \
| jq -r .value > "$file.tmp" && mv -f "$file.tmp" "$file"
}
mint
( while sleep 240; do mint || true; done ) &
echo "AZURE_FEDERATED_TOKEN_FILE=$file" >> "$GITHUB_ENV"
- name: Bundle + stage the MSIX feeds
shell: bash
env:
AZURE_TOKEN_CREDENTIALS: prod
AZURE_CLIENT_ID: ${{ vars.AZURE_CLIENT_ID }}
AZURE_TENANT_ID: ${{ vars.AZURE_TENANT_ID }}
run: |
# Out-of-store feed (bundled variant): universal .msixbundle +
# .appinstaller per channel; plus re-upload the Store-submission
# .msix to the tag archive (never a feed dir).
node scripts/stage-msixbundle.mjs --tag "$HERMES_PAYLOAD_TAG" --variant bundled
builds-pending:
name: Mark the builds table as in progress
if: inputs.upload_release == true
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Render the placeholder
env:
GH_TOKEN: ${{ github.token }}
HERMES_PAYLOAD_TAG: ${{ inputs.tag }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
run: |
if ! gh release view "$HERMES_PAYLOAD_TAG" >/dev/null 2>&1; then
echo "::error::no release exists for $HERMES_PAYLOAD_TAG — cannot mark the builds table"
exit 1
fi
python3 scripts/render-builds-table.py \
--tag "$HERMES_PAYLOAD_TAG" --repo "$GITHUB_REPOSITORY" \
--pending-run-url "$RUN_URL"
builds-table:
name: Render the release builds table
needs: [validate, build, msixbundle, finalize]
if: inputs.upload_release == true
runs-on: ubuntu-24.04
environment: release-signing
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
# Privileged job: pin to the SHA validate admitted. A bare checkout
# here would land on the dispatch branch, not the release bytes.
ref: ${{ needs.validate.outputs.sha }}
- name: Render
env:
GH_TOKEN: ${{ github.token }}
HERMES_PAYLOAD_TAG: ${{ inputs.tag }}
CLOUDFLARE_R2_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_R2_ACCOUNT_ID }}
CLOUDFLARE_R2_ACCESS_KEY_ID: ${{ secrets.CLOUDFLARE_R2_ACCESS_KEY_ID }}
CLOUDFLARE_R2_SECRET_ACCESS_KEY: ${{ secrets.CLOUDFLARE_R2_SECRET_ACCESS_KEY }}
CLOUDFLARE_R2_BUCKET: ${{ vars.CLOUDFLARE_R2_BUCKET }}
CLOUDFLARE_R2_PUBLIC_URL: ${{ vars.CLOUDFLARE_R2_PUBLIC_URL }}
run: |
if ! gh release view "$HERMES_PAYLOAD_TAG" >/dev/null 2>&1; then
echo "::error::no release exists for $HERMES_PAYLOAD_TAG — cannot render the builds table"
exit 1
fi
python3 scripts/render-builds-table.py \
--tag "$HERMES_PAYLOAD_TAG" --repo "$GITHUB_REPOSITORY"
publish-nightly:
name: Publish the nightly release
needs: [build, builds-table, msixbundle, finalize]
if: |
always()
&& inputs.upload_release == true
&& contains(inputs.tag, '-nightly.')
&& needs.build.result == 'success'
&& needs.builds-table.result == 'success'
&& needs.msixbundle.result == 'success'
&& needs.finalize.result == 'success'
runs-on: ubuntu-24.04
steps:
- name: Publish
env:
GH_TOKEN: ${{ github.token }}
TAG: ${{ inputs.tag }}
run: |
gh release edit "$TAG" --repo "$GITHUB_REPOSITORY" --draft=false
echo "Published $TAG"
+76 -51
View File
@@ -40,7 +40,6 @@ RUN apt-get -o Acquire::Retries=3 update && \
make -j"$(nproc)" && \
make install
FROM ghcr.io/astral-sh/uv:0.11.6-python3.13-trixie@sha256:b3c543b6c4f23a5f2df22866bd7857e5d304b67a564f4feab6ac22044dde719b AS uv_source
# Node 26 source stage. Debian trixie's bundled nodejs is pinned to 20.x
# which reached EOL in April 2026 — we copy node + npm from the upstream
# node:26 image instead (Hermes pins its toolchain to Node 26 everywhere).
@@ -57,9 +56,14 @@ FROM debian:13.4
ENV PYTHONUNBUFFERED=1
ENV PYTHONDONTWRITEBYTECODE=1
# Store Playwright browsers outside the volume mount so the build-time
# install survives the /opt/data volume overlay at runtime.
ENV PLAYWRIGHT_BROWSERS_PATH=/opt/hermes/.playwright
# The pm-pinned Chromium pair lives in the managed tool store at
# /opt/hermes/tools — outside the /opt/data volume mount, so the
# build-time install survives the volume overlay at runtime. pm's
# chromium package fact exports the same value (PLAYWRIGHT_BROWSERS_PATH
# at the store root); the image ENV names the same directory so
# Playwright and the browser tool resolve the pinned build even before pm
# composes tool env.
ENV PLAYWRIGHT_BROWSERS_PATH=/opt/hermes/tools
# Install system dependencies in one layer, clear APT cache.
# tini was previously PID 1 to reap orphaned zombie processes (MCP stdio
@@ -68,9 +72,16 @@ ENV PLAYWRIGHT_BROWSERS_PATH=/opt/hermes/.playwright
# replaces tini with s6-overlay's /init (PID 1 = s6-svscan), which reaps
# zombies non-blockingly on SIGCHLD and additionally supervises the main
# hermes process, the dashboard, and per-profile gateways.
# The second package list is the shared libraries the pinned Chromium links
# against. `npx playwright install --with-deps` used to apt-install them as
# a side effect; pm stages the pinned browser instead (below), so the libs
# must be declared here. The list is the `ldd ... | grep "not found"` set of
# the pinned chrome binary in this base image, mapped to trixie package
# names (see .hermes/plans/termux-removal-commit-spec.md).
RUN apt-get -o Acquire::Retries=3 update && \
apt-get -o Acquire::Retries=3 install -y --no-install-recommends \
ca-certificates curl iputils-ping python3 python-is-python3 ripgrep ffmpeg gcc g++ make cmake python3-dev python3-venv libffi-dev libolm-dev libatomic1 procps git openssh-client docker-cli xz-utils && \
ca-certificates curl iputils-ping python3 python-is-python3 ripgrep ffmpeg gcc g++ make cmake python3-dev python3-venv libffi-dev libolm-dev libatomic1 procps git openssh-client docker-cli xz-utils \
libasound2t64 libatk-bridge2.0-0t64 libatk1.0-0t64 libatspi2.0-0t64 libcairo2 libcups2t64 libdbus-1-3 libgbm1 libglib2.0-0t64 libnspr4 libnss3 libpango-1.0-0 libx11-6 libxcb1 libxcomposite1 libxdamage1 libxext6 libxfixes3 libxkbcommon0 libxrandr2 && \
rm -rf /var/lib/apt/lists/*
# Prefer the fixed SQLite over Debian's vulnerable libsqlite3.so.0. Keep the
@@ -149,8 +160,6 @@ COPY --chmod=0755 docker/tini-shim.sh /usr/bin/tini
# Non-root user for runtime; UID can be overridden via HERMES_UID at runtime
RUN useradd -u 10000 -m -d /opt/data hermes
COPY --chmod=0755 --from=uv_source /usr/local/bin/uv /usr/local/bin/uvx /usr/local/bin/
# Node 26: copy the node binary plus the bundled npm JS install from the
# upstream image. npm and npx are recreated as symlinks because they're
# symlinks in the source image (and need to live on PATH).
@@ -168,9 +177,41 @@ RUN ln -sf /usr/local/lib/node_modules/npm/bin/npm-cli.js /usr/local/bin/npm &&
WORKDIR /opt/hermes
# ---------- Pinned toolchain from pm/lock.json (single authority) ----------
# The image used to assemble uv from a second authority — an astral image
# tag (ghcr.io/astral-sh/uv:0.11.6-python3.13-trixie) that had already
# drifted to 0.11.6 while pm/lock.json pinned uv 0.12.3. That is exactly
# the two-authorities failure the pm design exists to end. The image is now
# a pin consumer: the stdlib-only pm provisioner reads pm/lock.json and
# stages the pinned uv + the pinned Chromium pair (sha256-verified at
# download — the same code path pm.sh/pm.ps1 and the desktop payload use)
# into the image's own runtime dir, a self-contained store baked under
# /opt/hermes, outside the /opt/data volume so it survives the overlay.
# The pinned uv is linked onto PATH so the `uv sync` / `uv pip install`
# build steps below run the lockfile's uv, not a second download.
#
# The Chromium pair is staged here rather than by `npx playwright install`,
# which fetched whatever revision the npm-resolved playwright wanted,
# unverified, and recorded no fact. The resolved browser binary path is
# baked to /etc/hermes/agent-browser-executable-path for stage2-hook.sh:
# the layout differs per arch (chrome-linux64/chrome on amd64,
# chromium-linux-arm64/chromium on arm64), so it is resolved at build time
# and never hunted at boot.
ENV HERMES_RUNTIME_DIR=/opt/hermes/tools
COPY pm/ pm/
RUN set -eu; \
python3 -m pm.cli install uv chromium chromium-headless-shell; \
ln -sf /opt/hermes/tools/uv-*/uv /usr/local/bin/uv; \
uv --version; \
browser_bin="$(find /opt/hermes/tools/chromium-* -type f \( -name chrome -o -name chromium \) -print -quit)"; \
test -n "$browser_bin"; \
"$browser_bin" --version; \
mkdir -p /etc/hermes; \
printf '%s' "$browser_bin" > /etc/hermes/agent-browser-executable-path
# ---------- Layer-cached dependency install ----------
# Copy only package manifests first so npm install + Playwright are cached
# unless the lockfiles themselves change.
# Copy only package manifests first so npm install is cached unless the
# lockfiles themselves change.
#
# ui-tui/packages/hermes-ink/ is copied IN FULL (not just its manifests)
# because it is referenced as a `file:` workspace dependency from
@@ -197,10 +238,6 @@ COPY apps/shared/ apps/shared/
ENV npm_config_install_links=false
RUN npm install --prefer-offline --no-audit --fetch-retries=5 && \
for i in 1 2 3; do \
npx playwright install --with-deps chromium --only-shell && break || \
{ [ "$i" = 3 ] && exit 1; echo "playwright install failed (attempt $i); retrying in 10s"; sleep 10; }; \
done && \
npm cache clean --force
# ---------- Photon iMessage sidecar deps (baked, NS-606) ----------
@@ -236,8 +273,8 @@ RUN cd plugins/platforms/photon/sidecar && \
# plus gateway messaging adapters that should work in the published image
# without a first-boot lazy install. We do NOT use `--all-extras`:
# that would pull in `[rl]` (atroposlib + tinker + torch + wandb from
# git), `[yc-bench]` (another git dep), and `[termux-all]` (Android
# redundancy), none of which belong in the published container.
# git) and `[yc-bench]` (another git dep), neither of which belongs in
# the published container.
#
# Provider packages (anthropic, bedrock, azure-identity) are included
# so Docker users can use these providers without requiring runtime
@@ -311,35 +348,33 @@ RUN mkdir -p /opt/hermes/bin && \
# `s6-setuidgid hermes` in its run script. If HERMES_UID is unset, services
# run as the default hermes user (UID 10000).
# ---------- Bake image provenance + build-time git revision ----------
# ---------- Image provenance + install stamp ----------
# CI (.github/workflows/docker.yml) runs scripts/write_install_stamp.py
# before `docker build`, so the bulk `COPY . .` above already placed a
# full-provenance /opt/hermes/install-stamp.json next to the code.
# .dockerignore excludes .git, so the stamp is the only commit channel the
# image carries: hermes_cli/version_info.py reads it at runtime (stamp
# first, live git second, unknown third), and both `hermes dump` and
# banner.get_git_banner_state() consume it through version_info.
#
# A local `docker build` without CI gets the minimal all-zero fallback
# stamp below; version_info skips the placeholder commit, so dump honestly
# reports "(unknown)". updateMechanism is `external`: the image is rebuilt
# and re-pulled, it never updates itself.
#
# The versioned, non-secret provenance marker is the authoritative runtime
# signal that this filesystem came from an immutable image. It deliberately
# lives outside both /opt/hermes (which operators sometimes bind-mount as a
# checkout) and /opt/data (the mutable HERMES_HOME volume).
# .dockerignore excludes .git, so `git rev-parse HEAD` from inside the
# container always returns nothing — meaning `hermes dump` reports
# "(unknown)" and the startup banner drops its `· upstream <sha>` suffix.
# That makes support triage from container bug reports impossible:
# we can't tell which commit the user is actually running.
#
# Fix: write the commit SHA passed via the HERMES_GIT_SHA build-arg to
# /opt/hermes/.hermes_build_sha at build time, and have
# hermes_cli/build_info.py read it at runtime. Both `hermes dump` and
# banner.get_git_banner_state() try the baked SHA first, then fall back
# to live `git rev-parse` for source installs (unchanged behaviour).
#
# The arg is optional — local `docker build` without --build-arg omits the
# SHA file (and records a null provenance revision), so build-info falls back
# to live-git lookup. CI
# (.github/workflows/docker.yml) passes ${{ github.sha }} so
# every published image has it.
ARG HERMES_GIT_SHA=
# checkout) and /opt/data (the mutable HERMES_HOME volume). Its `revision`
# is read from the install stamp; the fallback stamp's all-zero commit maps
# to null.
RUN set -eu; \
if [ -n "${HERMES_GIT_SHA}" ]; then \
printf '%s\n' "${HERMES_GIT_SHA}" > /opt/hermes/.hermes_build_sha; \
if [ ! -f /opt/hermes/install-stamp.json ]; then \
printf '{"schemaVersion":2,"commit":"0000000000000000000000000000000000000000","distribution":"docker","source":"fallback","updateMechanism":"external"}\n' \
> /opt/hermes/install-stamp.json; \
fi; \
mkdir -p /etc/hermes; \
HERMES_GIT_SHA="${HERMES_GIT_SHA}" python3 -c 'import json, os, pathlib, tomllib; project = tomllib.loads(pathlib.Path("/opt/hermes/pyproject.toml").read_text(encoding="utf-8"))["project"]; marker = pathlib.Path("/etc/hermes/image-provenance.json"); marker.write_text(json.dumps({"schema": 1, "deployment_kind": "image", "manager": "docker", "image": "nousresearch/hermes-agent", "version": project["version"], "revision": os.environ.get("HERMES_GIT_SHA") or None}, sort_keys=True, separators=(",", ":")) + "\n", encoding="utf-8"); marker.chmod(0o444)'
python3 -c 'import json, pathlib, tomllib; project = tomllib.loads(pathlib.Path("/opt/hermes/pyproject.toml").read_text(encoding="utf-8"))["project"]; stamp = json.loads(pathlib.Path("/opt/hermes/install-stamp.json").read_text(encoding="utf-8")); commit = stamp.get("commit"); revision = commit if commit and set(commit) != {"0"} else None; marker = pathlib.Path("/etc/hermes/image-provenance.json"); marker.write_text(json.dumps({"schema": 1, "deployment_kind": "image", "manager": "docker", "image": "nousresearch/hermes-agent", "version": project["version"], "revision": revision}, sort_keys=True, separators=(",", ":")) + "\n", encoding="utf-8"); marker.chmod(0o444)'
# ---------- s6-overlay service wiring ----------
# Static services declared at build time: main-hermes + dashboard.
@@ -386,19 +421,9 @@ ENV HERMES_TUI_DIR=/opt/hermes/ui-tui
ENV HERMES_HOME=/opt/data
ENV HERMES_WRITE_SAFE_ROOT=/opt/data
ENV HERMES_DISABLE_LAZY_INSTALLS=1
# The published image seals /opt/hermes (root-owned, read-only) so a runtime
# lazy install can't mutate the agent's own venv and brick it. But opt-in
# backends (Firecrawl web search, Exa, Feishu, …) keep their SDKs in
# tools/lazy_deps.py — deliberately NOT baked into [all] (see pyproject.toml
# policy 2026-05-12: one quarantined release must not break every install).
# Redirect those lazy installs to a writable dir on the durable data volume.
# lazy_deps appends this dir to the END of sys.path, so a package installed
# here can only ADD modules — it can never shadow or downgrade a core module,
# so the sealed-venv guarantee holds even with installs re-enabled. The dir
# is seeded + chowned to the hermes user by docker/stage2-hook.sh and lives
# on the /opt/data volume, so it persists across container recreates / image
# updates (an ABI stamp invalidates it if a rebuild bumps the interpreter).
ENV HERMES_LAZY_INSTALL_TARGET=/opt/data/lazy-packages
# Lazy installs are fully disabled in the published image (see
# HERMES_DISABLE_LAZY_INSTALLS above): the venv is sealed and opt-in backend
# SDKs are not installed at runtime.
# `docker exec` privilege-drop shim. When operators run
# `docker exec <c> hermes ...` they default to root, and any file the
+155
View File
@@ -0,0 +1,155 @@
#!/bin/bash
# ============================================================================
# venv-style activation for the Hermes dev environment (pm-managed tools).
#
# source ./activate (bash / zsh, repo root)
# .\activate.ps1 (PowerShell)
#
# Emits the composed pm env (PATH + tool vars) into the CURRENT shell, with
# save/restore: `deactivate` undoes exactly what activation changed.
#
# Requires a completed `./setup-hermes.sh` — it never invokes uv. It runs
# the pm store's pinned python (fallback: the repo venv) to emit the env
# JSON, then exports it here.
# ============================================================================
# Guard against double-sourcing: venv activate precedent (re-activating is a
# no-op re-save; we keep it idempotent by deactivating first).
if [ -n "${__HERMES_ACTIVATED:-}" ]; then
deactivate
fi
_HERMES_REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# --- target detection (same rules as pm.sh: msys on Windows must report the
# machine's truth from the registry, not the emulated shell's) ---
_hermes_os="$(uname -s)"
case "$_hermes_os" in
Linux) _hermes_os=linux ;;
Darwin) _hermes_os=darwin ;;
MINGW*|MSYS*|CYGWIN*) _hermes_os=win32 ;;
*) echo "activate: unsupported OS $(uname -s)" >&2; return 1 2>/dev/null || exit 1 ;;
esac
if [ "$_hermes_os" = win32 ]; then
_hermes_winarch="$(reg.exe query 'HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Environment' /v PROCESSOR_ARCHITECTURE 2>/dev/null | tr -d '
' | awk '/PROCESSOR_ARCHITECTURE/ {print $NF}')"
# PROCESSOR_ARCHITECTURE reports the EMULATED arch inside an x64-on-
# arm64 shell, and uname -m does too. The registry is the machine's
# truth; when reg.exe is unreachable, uname -s's release token (e.g.
# MINGW64_NT-10.0-28000-ARM64) still names the real machine arch.
if [ -z "$_hermes_winarch" ]; then
case "$(uname -s)" in
*ARM64) _hermes_winarch=ARM64 ;;
*AMD64) _hermes_winarch=AMD64 ;;
esac
fi
case "${_hermes_winarch:-}" in
ARM64) _hermes_arch=arm64 ;;
AMD64) _hermes_arch=x64 ;;
*) : ;; # probe unavailable — the store probe below decides
esac
fi
if [ -z "${_hermes_arch:-}" ] && [ "$_hermes_os" != win32 ]; then
case "$(uname -m)" in
arm64|aarch64) _hermes_arch=arm64 ;;
x86_64|amd64) _hermes_arch=x64 ;;
*) : ;; # unknown — the store probe below decides
esac
fi
# Resolve the pm store root here so the arch probe below can consult it.
_hermes_store="${HERMES_RUNTIME_DIR:-$HOME/.hermes/tools}"
if [ -n "${_hermes_arch:-}" ]; then
_hermes_target="$_hermes_os-$_hermes_arch"
else
# Registry + env both unavailable (blanked PATH): ask the store which
# target it actually holds instead of guessing an arch. First
# python-<ver>-<os>-<arch> entry wins.
_hermes_target=""
for _hermes_entry in "$_hermes_store"/python-*; do
[ -d "$_hermes_entry" ] || continue
case "${_hermes_entry##*/}" in
python-*-linux-*|python-*-darwin-*|python-*-win32-*)
_hermes_target="${_hermes_entry##*python-}"
_hermes_target="${_hermes_target#*-}"
break
;;
esac
done
if [ -z "$_hermes_target" ]; then
echo "activate: cannot determine target arch and the store holds no python entry — run ./setup-hermes.sh first" >&2
unset _hermes_repo _hermes_os _hermes_arch _hermes_target _hermes_store _hermes_py
return 1 2>/dev/null || exit 1
fi
fi
_hermes_py=""
for _hermes_entry in "$_hermes_store"/python-*-"$_hermes_target"; do
if [ -x "$_hermes_entry/bin/python" ]; then
_hermes_py="$_hermes_entry/bin/python"
elif [ -x "$_hermes_entry/bin/python.exe" ]; then
_hermes_py="$_hermes_entry/bin/python.exe"
fi
[ -n "$_hermes_py" ] && break
done
if [ -z "$_hermes_py" ]; then
for _hermes_venvpy in "$_hermes_repo/venv/bin/python" "$_hermes_repo/venv/bin/python.exe" \
"$_hermes_repo/venv/Scripts/python.exe"; do
[ -x "$_hermes_venvpy" ] && { _hermes_py="$_hermes_venvpy"; break; }
done
fi
if [ -z "$_hermes_py" ]; then
echo "activate: no pm python found — run ./setup-hermes.sh first" >&2
unset _hermes_repo _hermes_os _hermes_arch _hermes_target _hermes_store _hermes_py
return 1 2>/dev/null || exit 1
fi
# --- emit export lines from `pm env` ---
_hermes_json="$(PYTHONPATH="$_hermes_repo${PYTHONPATH:+:$PYTHONPATH}" "$_hermes_py" -m pm.cli env 2>/dev/null)"
if [ -z "$_hermes_json" ] || [ "${_hermes_json#*{}" = "$_hermes_json" ]; then
echo "activate: could not read pm env (run ./setup-hermes.sh first)" >&2
unset _hermes_repo _hermes_os _hermes_arch _hermes_target _hermes_store _hermes_py _hermes_json
return 1 2>/dev/null || exit 1
fi
_hermes_keys="$(printf '%s' "$_hermes_json" | "$_hermes_py" -c 'import json,sys; print(" ".join(json.load(sys.stdin)))' \
| tr ' ' '\n' | grep -E '^[A-Za-z_][A-Za-z0-9_]*$' | tr '\n' ' ')"
[ -n "$_hermes_keys" ] || _hermes_keys="PATH"
# --- snapshot what we are about to change (deactivate restores this) ---
__HERMES_ACTIVATED=1
__HERMES_SAVED_PATH="$PATH"
for _hermes_k in $_hermes_keys; do
eval "__HERMES_SAVED_$_hermes_k=\"\${$_hermes_k+set}\""
eval "__HERMES_PRIOR_$_hermes_k=\${$_hermes_k-__HERMES_UNSET__}"
done
eval "$(printf '%s' "$_hermes_json" | "$_hermes_py" -c '
import json, sys, shlex, re
for k, v in json.load(sys.stdin).items():
# Windows carries names like ProgramFiles(ARM) that are not valid
# bash identifiers — skip them (they pass through untouched).
if not re.fullmatch(r"[A-Za-z_][A-Za-z0-9_]*", k):
continue
print("export %s=%s" % (k, shlex.quote(str(v))))')"
deactivate() {
export PATH="$__HERMES_SAVED_PATH"
for _hermes_k in $__HERMES_KEY_LIST; do
eval "_hermes_was=\"\$__HERMES_SAVED_$_hermes_k\""
eval "_hermes_old=\"\$__HERMES_PRIOR_$_hermes_k\""
if [ "$_hermes_was" != "set" ]; then
unset "$_hermes_k"
else
export "$_hermes_k=$_hermes_old"
fi
unset "__HERMES_SAVED_$_hermes_k" "__HERMES_PRIOR_$_hermes_k"
done
unset __HERMES_SAVED_PATH __HERMES_KEY_LIST __HERMES_ACTIVATED
unset -f deactivate
unset _hermes_k _hermes_was _hermes_old
}
# remember the key list for deactivate (kept out of the loop vars above)
__HERMES_KEY_LIST="$_hermes_keys"
unset _hermes_repo _hermes_os _hermes_arch _hermes_target _hermes_store \
_hermes_py _hermes_json _hermes_keys _hermes_k _hermes_entry _hermes_venvpy \
_hermes_winarch
+75
View File
@@ -0,0 +1,75 @@
# ============================================================================
# venv-style activation for the Hermes dev environment (pm-managed tools).
#
# .\activate.ps1 (repo root; if script execution is disabled:
# powershell -ExecutionPolicy Bypass -File .\activate.ps1, or set
# Set-ExecutionPolicy RemoteSigned -Scope CurrentUser once)
#
# Emits the composed pm env (PATH + tool vars) into the CURRENT session, with
# save/restore: `deactivate` undoes exactly what activation changed.
#
# Requires a completed .\setup-hermes.ps1 — it never invokes uv. It runs the
# pm store's pinned python (fallback: the repo venv) to emit the env JSON.
# ============================================================================
$ErrorActionPreference = 'Stop'
# Guard against double-sourcing: re-activating deactivates first.
if (Test-Path function:deactivate) { deactivate }
$repo = $PSScriptRoot
$machineArch = (Get-ItemProperty 'HKLM:\SYSTEM\CurrentControlSet\Control\Session Manager\Environment').PROCESSOR_ARCHITECTURE
$arch = if ($machineArch -eq 'ARM64') { 'arm64' } else { 'x64' }
$target = "win32-$arch"
$store = if ($env:HERMES_RUNTIME_DIR) { $env:HERMES_RUNTIME_DIR } else { Join-Path $HOME '.hermes/tools' }
$py = $null
$entry = Get-ChildItem -Path $store -Directory -Filter "python-*-$target" -ErrorAction SilentlyContinue |
Sort-Object Name | Select-Object -Last 1
if ($entry -and (Test-Path (Join-Path $entry.FullName 'bin/python.exe'))) {
$py = Join-Path $entry.FullName 'bin/python.exe'
}
if (-not $py) {
foreach ($candidate in @(
(Join-Path $repo 'venv/Scripts/python.exe'),
(Join-Path $repo 'venv/bin/python.exe'))) {
if (Test-Path $candidate) { $py = $candidate; break }
}
}
if (-not $py) {
Write-Error 'activate: no pm python found - run .\setup-hermes.ps1 first'
}
$envJSON = (& $py -m pm.cli env 2>$null) -join "`n"
if (-not $envJSON) {
Write-Error 'activate: could not read pm env - run .\setup-hermes.ps1 first'
}
$composed = $envJSON | ConvertFrom-Json
# --- snapshot what we are about to change (deactivate restores this) ---
$global:_hermesKeys = @($composed.PSObject.Properties.Name)
$global:_hermesSaved = @{}
foreach ($k in $global:_hermesKeys) {
$global:_hermesSaved[$k] = [pscustomobject]@{
WasSet = (Test-Path "env:$k")
Value = [Environment]::GetEnvironmentVariable($k)
}
}
foreach ($prop in $composed.PSObject.Properties) {
Set-Item -Path "env:$($prop.Name)" -Value ([string]$prop.Value)
}
function global:deactivate {
foreach ($k in $global:_hermesKeys) {
$saved = $global:_hermesSaved[$k]
if ($saved.WasSet) {
Set-Item -Path "env:$k" -Value $saved.Value
} else {
Remove-Item -Path "env:$k" -ErrorAction SilentlyContinue
}
}
$global:_hermesKeys = $null
$global:_hermesSaved = $null
Remove-Item function:deactivate
}
+7 -6
View File
@@ -58,12 +58,13 @@ def _get_anthropic_sdk():
global _anthropic_sdk
if _anthropic_sdk is ...:
try:
from tools.lazy_deps import ensure as _lazy_ensure
_lazy_ensure("provider.anthropic", prompt=False)
from pm import ensure_import as _ensure_import
_ensure_import("anthropic")
except ImportError:
pass
except Exception:
# FeatureUnavailable — fall through to ImportError handling below
# InstallError (lazy install disabled/declined/failed) — fall
# through to ImportError handling below
pass
try:
import anthropic as _sdk
@@ -1089,7 +1090,7 @@ def _read_claude_code_credentials_from_file() -> Optional[Dict[str, Any]]:
if not cred_path.exists():
return None
try:
data = json.loads(cred_path.read_text(encoding="utf-8"))
data = json.loads(cred_path.read_text(encoding="utf-8-sig"))
except (json.JSONDecodeError, OSError, IOError) as e:
logger.debug("Failed to read ~/.claude/.credentials.json: %s", e)
return None
@@ -1299,7 +1300,7 @@ def _write_claude_code_credentials(
# Read existing file to preserve other fields
existing = {}
if cred_path.exists():
existing = json.loads(cred_path.read_text(encoding="utf-8"))
existing = json.loads(cred_path.read_text(encoding="utf-8-sig"))
oauth_data: Dict[str, Any] = {
"accessToken": access_token,
@@ -1704,7 +1705,7 @@ def read_hermes_oauth_credentials() -> Optional[Dict[str, Any]]:
oauth_file = _get_hermes_oauth_file()
if oauth_file.exists():
try:
data = json.loads(oauth_file.read_text(encoding="utf-8"))
data = json.loads(oauth_file.read_text(encoding="utf-8-sig"))
if data.get("accessToken"):
return data
except (json.JSONDecodeError, OSError, IOError) as e:
+4 -4
View File
@@ -54,7 +54,7 @@ SCOPE_AI_AZURE_DEFAULT = "https://ai.azure.com/.default"
# Lazy SDK import — only loaded when the Entra path is actually used.
# ---------------------------------------------------------------------------
_AZURE_IDENTITY_FEATURE = "provider.azure_identity"
_AZURE_IDENTITY_FEATURE = "azure-identity"
def has_azure_identity_installed() -> bool:
@@ -80,7 +80,7 @@ def _require_azure_identity():
return _ai
except ImportError:
try:
from tools.lazy_deps import ensure, FeatureUnavailable
from pm import InstallError, ensure_import
except ImportError as exc:
raise ImportError(
"The 'azure-identity' package is required for Azure AI "
@@ -89,8 +89,8 @@ def _require_azure_identity():
) from exc
try:
ensure(_AZURE_IDENTITY_FEATURE, prompt=False)
except FeatureUnavailable as exc:
ensure_import(_AZURE_IDENTITY_FEATURE)
except InstallError as exc:
raise ImportError(
"The 'azure-identity' package is required for Azure AI "
"Foundry Entra ID authentication. " + str(exc)
+4 -4
View File
@@ -41,15 +41,15 @@ logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Ensure boto3/botocore are installed before any code in this module runs.
# Upstream removed boto3 from [all] extras (PRs #24220, #24515); lazy_deps
# Upstream removed boto3 from [all] extras (PRs #24220, #24515); pm
# handles on-demand installation so the Bedrock provider still works in the
# EKS deployment without baking boto3 into the base image.
# ---------------------------------------------------------------------------
try:
from tools.lazy_deps import ensure
ensure("provider.bedrock", prompt=False)
from pm import ensure_import
ensure_import("bedrock")
except Exception:
pass # lazy_deps unavailable or install failed — let downstream imports surface the real error
pass # pm unavailable or install failed — let downstream imports surface the real error
# ---------------------------------------------------------------------------
+11 -3
View File
@@ -92,9 +92,17 @@ __all__ = [
# ``Thread.join(timeout=...)`` deadlines to an absolute timestamp; very large
# relative timeouts overflow ``time_t`` on macOS and raise
# ``OverflowError: timestamp out of range for platform time_t`` (#83220).
# One year is semantically "unbounded" for every wait in this codebase while
# staying far below any platform conversion limit.
MAX_SAFE_TIMEOUT_S = 31_536_000.0 # 365 days
# On Windows the binding constraint is narrower: the underlying
# ``WaitForSingleObject`` takes a DWORD *milliseconds* argument, so any
# wait above ~49.7 days (0xFFFFFFFF ms) raises OverflowError rather than
# waiting (verified live: 4294967.0s ok, 4294967.3s overflows). One year
# is the semantic "unbounded" on platforms that can express it; Windows
# takes the DWORD-ms cap, which is still semantically "unbounded" for
# every wait in this codebase.
_WINDOWS_MAX_WAIT_S = (0xFFFFFFFF - 1000) / 1000.0 # DWORD ms, minus a 1s rounding guard
MAX_SAFE_TIMEOUT_S = (
_WINDOWS_MAX_WAIT_S if sys.platform == "win32" else 31_536_000.0
) # 365 days off-Windows
# Grace period after a deadline fires before concluding the event loop thread
# is blocked in a synchronous call and dumping stacks (family A diagnostics).
+30 -4
View File
@@ -261,11 +261,37 @@ def _spawn_pyright(root: str, ctx: ServerContext) -> Optional[SpawnSpec]:
)
def _pm_store_python() -> Optional[str]:
"""The pm-provisioned interpreter (facts + store layout), or None.
Under no-boot-through-venv the running process is the store python and
``VIRTUAL_ENV`` is unset, so the old ambient-env probe had nothing to
offer; pm's ``python`` fact names the store entry that owns the
runtime. It takes the precedence the ambient ``VIRTUAL_ENV`` probe used
to have; project-local candidates follow."""
try:
from pm import paths
from pm.lock import Facts
except Exception:
return None
try:
fact = Facts(paths.facts_path()).get("python")
if not fact or "entry" not in fact:
return None
entry = paths.store_root() / fact["entry"]
exe = entry / ("python.exe" if os.name == "nt" else "bin/python3")
return str(exe) if exe.exists() else None
except Exception:
return None
def _detect_python(root: str) -> Optional[str]:
candidates = []
if os.environ.get("VIRTUAL_ENV"):
candidates.append(os.environ["VIRTUAL_ENV"])
candidates.extend([os.path.join(root, ".venv"), os.path.join(root, "venv")])
# pm's store interpreter resolves to a full exe path (its layout is not
# a venv), so it is checked directly rather than through the v/sub probe.
pm_python = _pm_store_python()
if pm_python:
return pm_python
candidates = [os.path.join(root, ".venv"), os.path.join(root, "venv")]
for v in candidates:
for sub in ("bin/python", "bin/python3", "Scripts/python.exe"):
p = os.path.join(v, sub)
+3 -3
View File
@@ -278,7 +278,7 @@ def _local_probe_disk_cache_path() -> Path:
def _load_local_probe_disk_cache() -> Dict[str, Any]:
try:
with _local_probe_disk_cache_path().open("r", encoding="utf-8") as f:
with _local_probe_disk_cache_path().open("r", encoding="utf-8-sig") as f:
data = json.load(f)
return data if isinstance(data, dict) else {}
except Exception:
@@ -344,7 +344,7 @@ def _load_model_metadata_disk_cache() -> Dict[str, Dict[str, Any]]:
"""Load processed OpenRouter metadata cache from disk."""
try:
cache_path = _get_model_metadata_cache_path()
with cache_path.open("r", encoding="utf-8") as f:
with cache_path.open("r", encoding="utf-8-sig") as f:
data = json.load(f)
if not isinstance(data, dict):
return {}
@@ -1558,7 +1558,7 @@ def _load_context_cache() -> Dict[str, int]:
if not path.exists():
return {}
try:
with open(path, encoding="utf-8") as f:
with open(path, encoding="utf-8-sig") as f:
data = yaml.safe_load(f) or {}
return data.get("context_lengths") or {}
except Exception as e:
+4 -4
View File
@@ -181,11 +181,11 @@ def _enabled(config: Dict[str, Any]) -> bool:
return bool(gh.get("enabled") and otlp.get("enabled") and otlp.get("endpoint"))
def _require_metrics_sdk(*, auto_install: bool = True, prompt: bool = False) -> Dict[str, Any]:
def _require_metrics_sdk(*, auto_install: bool = True) -> Dict[str, Any]:
if auto_install:
try:
from tools.lazy_deps import ensure as _lazy_ensure
_lazy_ensure("export.otlp", prompt=prompt)
from pm import ensure_import as _lazy_ensure
_lazy_ensure("otlp")
except Exception:
pass
try:
@@ -594,7 +594,7 @@ def start_gateway_health_export(config: Dict[str, Any]) -> GatewayHealthExportRu
if gh.get("metrics_enabled", True) or gh.get("diagnostic_events_enabled", True):
try:
sdk = _require_metrics_sdk(prompt=False)
sdk = _require_metrics_sdk()
except Exception:
logger.warning(
"monitoring.gateway_health_export.enabled but OTLP SDK is unavailable; "
+4 -4
View File
@@ -277,11 +277,11 @@ def _do_upload(
Returns a user-facing status string. Never raises.
"""
try:
from tools import lazy_deps
lazy_deps.ensure("tool.trace_upload", prompt=False)
import pm
pm.ensure_import("trace-upload")
except Exception:
# lazy-install unavailable/declined — fall through to the import,
# which surfaces the install hint below if the package is missing.
# lazy-install unavailable — fall through to the import, which
# surfaces the install hint below if the package is missing.
pass
try:
from huggingface_hub import HfApi
+4 -4
View File
@@ -24,14 +24,14 @@ from typing import Optional, Tuple
from agent.secret_scope import get_secret as _get_secret, is_multiplex_active
# Ensure google-auth is installed before importing. The [vertex] extra is no
# longer in [all] per the lazy-install policy added 2026-05-12 — lazy_deps
# longer in [all] per the lazy-install policy added 2026-05-12 — pm
# handles on-demand installation so the Vertex provider still works for users
# who installed plain `hermes-agent` and only later selected a Gemini model.
try:
from tools.lazy_deps import ensure as _lazy_ensure
_lazy_ensure("provider.vertex", prompt=False)
from pm import ensure_import as _ensure_import
_ensure_import("vertex")
except Exception:
pass # lazy_deps unavailable or install failed — fall through to the real ImportError below
pass # pm unavailable or install failed — fall through to the real ImportError below
try:
import google.auth
+254
View File
@@ -0,0 +1,254 @@
// THE electron-builder configuration — the whole thing, one file. There is
// no "build" field in package.json: run-electron-builder.mjs always passes
// --config for this file, so a stray package.json field would be silently
// ignored anyway, and splitting the config across JSON + this overlay is
// how the two halves drift.
//
// A .cjs module (not JSON) so the variant is decided at require time:
// HERMES_DESKTOP_VARIANT=light builds "Hermes Light". The whole config
// derives from that one flag.
// @ts-check
'use strict'
const fs = require('node:fs')
const path = require('node:path')
const {
light,
store,
storeMsix,
displayName,
appId,
appNamePascal,
channel,
msixAppIdWithOrg
} = require('./product-identity.cjs')
// The out-of-store MSIX publisher (ATS cert subject) — single source, shared
// with the .appinstaller generator so the manifest and the App Installer can
// never drift (see scripts/msix-shared.mjs).
const { OUT_OF_STORE_PUBLISHER } = require('../../scripts/msix-shared.mjs')
/** @typedef {import("app-builder-lib").Configuration} Configuration */
const [owner, repo] = (process.env.GITHUB_REPOSITORY || 'NousResearch/hermes-agent').split('/')
if (!owner || !repo) {
throw new Error(`invalid GITHUB_REPOSITORY ${process.env.GITHUB_REPOSITORY}`)
}
const electronVersion = require('./package.json').devDependencies.electron
if (!/^\d+\.\d+\.\d+$/.test(electronVersion)) {
throw new Error(`invalid electron version ${electronVersion} in package.json`)
}
/** @type {Configuration} */
module.exports = {
electronVersion,
appId,
productName: displayName,
executableName: displayName,
protocols: [
{
name: `${displayName} Protocol`,
schemes: ['hermes']
}
],
// A store build is archived, never served to a feed — prefix its artifact
// so it can't collide with the out-of-store MSIX of the same tag/arch, and
// the release pipeline can keep the two apart.
artifactName: `${store ? 'Store-' : ''}${appNamePascal}-\${version}-\${os}-\${arch}.\${ext}`,
icon: 'assets/icon',
// The electron-updater feed. CI builds set CLOUDFLARE_R2_PUBLIC_URL (the R2
// public bucket / custom domain) and publish there — the feed yml, blockmaps
// and installers all live in the same flat R2 bucket, and electron-updater
// resolves the yml's relative artifact paths against it. Builds without the
// var (local, or a fork without the R2 vars) keep the github provider, which
// is exactly today's behavior. The store build has no feed at all (the Store
// owns its distribution and updates).
publish: store
? null
: [
process.env.CLOUDFLARE_R2_PUBLIC_URL
? { provider: 'generic', url: process.env.CLOUDFLARE_R2_PUBLIC_URL.replace(/\/+$/, ''), channel }
: { provider: 'github', owner, repo, channel }
],
extraMetadata: {
name: appNamePascal,
desktopName: appId
},
directories: {
output: 'release'
},
files: ['dist/**', 'assets/**', 'public/**', 'package.json'],
beforeBuild: 'scripts/before-build.mjs',
beforePack: 'scripts/before-pack.mjs',
afterPack: 'scripts/after-pack.mjs',
...(process.platform === 'darwin' ? { afterSign: 'scripts/notarize.mjs' } : {}),
extraResources: [
{
from: 'build/install-stamp.json',
to: 'install-stamp.json'
},
{
from: 'build/agent-payload',
to: 'agent-payload'
},
{
from: 'assets/icon.ico',
to: 'icon.ico'
}
],
asar: {
unpack: ['**/*.node', '**/prebuilds/**', 'dist/**']
},
mac: {
category: 'public.app-category.developer-tools',
extendInfo: {
CFBundleDisplayName: displayName,
CFBundleExecutable: displayName,
CFBundleName: displayName,
LSRequiresNativeExecution: true,
NSAudioCaptureUsageDescription: `${displayName} uses audio capture for voice conversations.`,
NSCameraUsageDescription: `${displayName} uses the camera when a plugin or feature you enable requests it.`,
NSMicrophoneUsageDescription: `${displayName} uses the microphone for voice input and voice conversations.`,
NSCalendarsUsageDescription: `${displayName} needs access to Calendar to provide requested meeting and scheduling support.`,
NSCalendarsFullAccessUsageDescription: `${displayName} needs full access to Calendar to read and manage events when explicitly requested.`,
NSRemindersUsageDescription: `${displayName} needs access to Reminders to provide requested personal-assistant and scheduling support.`,
NSRemindersFullAccessUsageDescription: `${displayName} needs full access to Reminders to read and manage reminders when explicitly requested.`,
NSScreenCaptureUsageDescription: `${displayName} captures the screen when you ask the agent to screenshot or record it.`,
NSLocalNetworkUsageDescription: `${displayName} connects to devices on your local network when a plugin or feature you enable requests it.`,
NSAppleMusicUsageDescription: `${displayName} accesses your music library when a plugin or feature you enable requests it.`
},
target: ['dmg', 'zip'],
sign: {
entitlements: 'electron/entitlements.mac.plist',
entitlementsInherit: 'electron/entitlements.mac.inherit.plist',
hardenedRuntime: true,
ignore: (/** @type {string} */ file) => {
try {
if (fs.lstatSync(file).isDirectory()) {
return false
}
return !isMachO(file)
} catch {
return true
}
}
}
},
dmg: {
title: `Install ${displayName}`,
backgroundColor: '#f5f5f7',
iconSize: 96,
window: {
width: 560,
height: 360
},
contents: [
{
x: 160,
y: 170,
type: 'file'
},
{
x: 400,
y: 170,
type: 'link',
path: '/Applications'
}
]
},
win: {
legalTrademarks: displayName,
target: ['msix'],
...windowsSigning()
},
msix: {
// A store build uses the Partner Center packaging identity (the Store
// re-signs + rewrites the publisher on submission); everything else uses
// the out-of-store ATS-cert identity.
identityName: store ? storeMsix.identityName : msixAppIdWithOrg,
applicationId: appNamePascal,
displayName,
publisher: store ? storeMsix.publisher : OUT_OF_STORE_PUBLISHER,
publisherDisplayName: store ? storeMsix.publisherDisplayName : 'Nous Research',
// Nightly MSIX versions are `X.Y.Z.<minutes-since-stable>` (see
// scripts/msix-shared.mjs). setBuildNumber makes getVersionInWeirdWindowsForm
// use the BUILD_NUMBER env (4th component) instead of hardcoding ".0" — a
// stable build sets no BUILD_NUMBER and stays X.Y.Z.0, a nightly build sets
// it via build-bundled-desktop.mjs so App Installer updates over equal
// nightly-over-nightly versions instead of refusing them.
setBuildNumber: true,
// Floor Windows 11 22H2. Below build 18307 the manifest schema caps
// AppExtension Name at 39 chars and Microsoft's own
// "com.microsoft.windows.copilotkeyprovider" is 40 (makeappx
// 0x80080204 — A/B-verified against the 26100 kit; 18307 exactly
// still failed on it, 22621 passes), and 22621 is the documented
// Copilot hardware key floor anyway.
minVersion: '10.0.22621.0',
maxVersionTested: '10.0.26100.0',
// Static path: the file itself is written by scripts/before-build.mjs at
// build time (see the comment on the hook) — never at config require
// time, so typecheck/test imports don't touch the filesystem.
customExtensionsPath: 'build/msix-extensions.xml',
customManifestPath: 'assets/msix-manifest.xml',
showNameOnTiles: true
},
linux: {
category: 'Development',
maintainer: 'Nous Research <support@nousresearch.com>',
synopsis: light
? 'Remote-only desktop client for Hermes Agent.'
: 'Native desktop shell for Hermes Agent.',
target: ['AppImage']
}
}
// MSIX build-time staging (build/appx icons + build/msix-extensions.xml)
// lives in scripts/before-build.mjs — an electron-builder lifecycle hook —
// NOT here at require time, so importing this config for typecheck/tests
// never writes to the filesystem.
// Azure Trusted Signing. The hook signs the .msix package itself and the
// product exe (after electron-builder's rcedit — the batch in afterPack runs
// too early for the exe); every other payload binary is batch-signed in
// afterPack via scripts/batch-sign-binaries.mjs, which this hook acknowledges
// with `true` so electron-builder never re-signs one-by-one. The package
// signature is all Windows install validation checks; inner Authenticode
// covers SmartScreen/WDAC tree scans. See batch-sign-binaries.mjs for the
// ordering contract.
const MACHO_MAGICS = new Set([
0xfeedface,
0xcefaedfe,
0xfeedfacf,
0xcffaedfe,
0xcafebabe,
0xbebafeca
])
/** @param {string} file */
function isMachO(file) {
const buf = Buffer.alloc(4)
const fd = fs.openSync(file, 'r')
try {
if (fs.readSync(fd, buf, 0, 4, 0) !== 4) {
return false
}
} finally {
fs.closeSync(fd)
}
return MACHO_MAGICS.has(buf.readUInt32BE(0))
}
function windowsSigning() {
if (!process.env.AZURE_SIGN_ENDPOINT || !process.env.AZURE_CLIENT_ID) {
return {}
}
return {
sign: {
type: 'signtool',
sign: './scripts/batch-sign-binaries.mjs',
signingHashAlgorithms: ['sha256'],
publisherName: process.env.AZURE_SIGN_PUBLISHER
}
}
}
+34 -135
View File
@@ -6,142 +6,61 @@ import { test } from 'vitest'
import {
appendUniquePathEntries,
buildDesktopBackendEnv,
buildDesktopBackendPath,
hermesManagedNodePathEntries,
normalizeHermesHomeRoot,
pathEnvKey,
POSIX_SANE_PATH_ENTRIES
} from './backend-env'
test('desktop backend PATH adds Hermes-managed bins and missing POSIX sane entries', () => {
const result = buildDesktopBackendPath({
hermesHome: '/Users/test/.hermes',
venvRoot: '/Users/test/.hermes/hermes-agent/venv',
currentPath: '/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin',
platform: 'darwin',
pathModule: path.posix
test('backend env scrubs PYTHONPATH and PYTHONHOME', () => {
const env = buildDesktopBackendEnv({
currentEnv: {
PATH: '/usr/bin:/bin',
PYTHONPATH: '/leaked/other/checkout',
PYTHONHOME: '/leaked/python'
},
platform: 'darwin'
})
const entries = result.split(':')
// Both managed-Node layouts lead, POSIX-native shape first, then the venv.
assert.deepEqual(entries.slice(0, 3), [
'/Users/test/.hermes/node/bin',
'/Users/test/.hermes/node',
'/Users/test/.hermes/hermes-agent/venv/bin'
])
assert.ok(entries.includes('/opt/homebrew/bin'), 'Apple Silicon Homebrew bin is added')
assert.ok(entries.includes('/opt/homebrew/sbin'), 'Apple Silicon Homebrew sbin is added')
assert.ok(entries.includes('/usr/local/sbin'), 'missing standard sbin is added')
assert.equal(env.PYTHONPATH, '')
assert.equal(env.PYTHONHOME, '')
})
test('POSIX backend PATH keeps the inherited PATH first and appends missing sane entries', () => {
const env = buildDesktopBackendEnv({
currentEnv: { PATH: '/opt/homebrew/bin:/usr/bin:/bin' },
platform: 'darwin'
})
const entries = env.PATH.split(':')
assert.equal(entries[0], '/opt/homebrew/bin', 'inherited PATH keeps precedence')
assert.equal(entries.filter(entry => entry === '/opt/homebrew/bin').length, 1, 'no duplicates')
for (const expected of POSIX_SANE_PATH_ENTRIES) {
assert.ok(entries.includes(expected), `${expected} should be present`)
}
})
test('managed Node dirs lead with the platform-native layout but always offer both', () => {
const posix = hermesManagedNodePathEntries('/Users/test/.hermes', {
platform: 'darwin',
pathModule: path.posix
})
const windows = hermesManagedNodePathEntries('C:\\Users\\test\\AppData\\Local\\hermes', {
platform: 'win32',
pathModule: path.win32
})
// install.sh uses node/bin; install.ps1 unpacks node.exe into node\ itself.
// Both shapes are always emitted so migrated installs keep resolving.
assert.deepEqual(posix, ['/Users/test/.hermes/node/bin', '/Users/test/.hermes/node'])
assert.deepEqual(windows, [
'C:\\Users\\test\\AppData\\Local\\hermes\\node',
'C:\\Users\\test\\AppData\\Local\\hermes\\node\\bin'
])
})
test('managed Node dirs are empty without a Hermes home', () => {
assert.deepEqual(hermesManagedNodePathEntries(undefined, { platform: 'darwin', pathModule: path.posix }), [])
assert.deepEqual(hermesManagedNodePathEntries('', { platform: 'win32', pathModule: path.win32 }), [])
})
test('every managed Node dir outranks the inherited PATH on both platforms', () => {
for (const [platform, pathModule, home, inherited, delimiter] of [
['darwin', path.posix, '/Users/test/.hermes', '/usr/local/bin:/usr/bin', ':'],
['win32', path.win32, 'C:\\hermes', 'C:\\Program Files\\nodejs;C:\\Windows\\System32', ';']
] as const) {
const entries = buildDesktopBackendPath({
hermesHome: home,
venvRoot: null,
currentPath: inherited,
platform,
pathModule
}).split(delimiter)
const managed = hermesManagedNodePathEntries(home, { platform, pathModule })
const firstInherited = Math.min(...inherited.split(delimiter).map(entry => entries.indexOf(entry)))
for (const dir of managed) {
assert.ok(
entries.indexOf(dir) >= 0 && entries.indexOf(dir) < firstInherited,
`${dir} must precede the inherited PATH on ${platform}`
)
}
}
})
test('desktop backend PATH preserves first occurrence and avoids duplicates', () => {
const result = buildDesktopBackendPath({
hermesHome: '/Users/test/.hermes',
venvRoot: '/Users/test/.hermes/hermes-agent/venv',
currentPath: '/opt/homebrew/bin:/usr/bin:/opt/homebrew/bin:/bin',
platform: 'darwin',
pathModule: path.posix
})
const entries = result.split(':')
assert.equal(entries.filter(entry => entry === '/opt/homebrew/bin').length, 1)
assert.ok(
entries.indexOf('/opt/homebrew/bin') < entries.indexOf('/opt/homebrew/sbin'),
'existing Homebrew bin keeps its precedence over appended missing sane entries'
)
})
test('buildDesktopBackendEnv extends PYTHONPATH and backend PATH together', () => {
test('Windows PATH casing and delimiter are preserved without POSIX sane entries', () => {
const env = buildDesktopBackendEnv({
hermesHome: '/Users/test/.hermes',
pythonPathEntries: ['/repo/hermes-agent'],
venvRoot: '/Users/test/.hermes/hermes-agent/venv',
currentEnv: {
PATH: '/usr/bin:/bin',
PYTHONPATH: '/existing/pythonpath'
},
platform: 'darwin',
pathModule: path.posix
currentEnv: { Path: 'C:\\Windows\\System32;C:\\Windows' },
platform: 'win32'
})
assert.equal(env.PYTHONPATH, '/repo/hermes-agent:/existing/pythonpath')
assert.ok(
env.PATH.startsWith(
'/Users/test/.hermes/node/bin:/Users/test/.hermes/node:/Users/test/.hermes/hermes-agent/venv/bin:'
)
)
assert.ok(env.PATH.includes('/opt/homebrew/bin'))
assert.equal(env.Path, 'C:\\Windows\\System32;C:\\Windows')
assert.equal(env.PATH, undefined)
})
test('buildDesktopBackendEnv forces PYTHONUTF8 unless the user set it explicitly', () => {
const defaulted = buildDesktopBackendEnv({
hermesHome: '/Users/test/.hermes',
currentEnv: { PATH: '/usr/bin' },
platform: 'darwin',
pathModule: path.posix
platform: 'darwin'
})
assert.equal(defaulted.PYTHONUTF8, '1')
const optedOut = buildDesktopBackendEnv({
hermesHome: '/Users/test/.hermes',
currentEnv: { PATH: '/usr/bin', PYTHONUTF8: '0' },
platform: 'darwin',
pathModule: path.posix
platform: 'darwin'
})
assert.equal(optedOut.PYTHONUTF8, '0')
@@ -159,33 +78,13 @@ test('normalizeHermesHomeRoot maps profile homes back to the global Hermes root'
assert.equal(normalizeHermesHomeRoot('/Users/test/.hermes', { pathModule: path.posix }), '/Users/test/.hermes')
})
test('Windows PATH casing and delimiter are preserved without POSIX sane entries', () => {
const env = buildDesktopBackendEnv({
hermesHome: 'C:\\Users\\test\\AppData\\Local\\hermes',
pythonPathEntries: ['C:\\repo\\hermes-agent'],
venvRoot: 'C:\\Users\\test\\AppData\\Local\\hermes\\hermes-agent\\venv',
currentEnv: {
Path: 'C:\\Windows\\System32;C:\\Windows',
PYTHONPATH: 'C:\\existing\\pythonpath'
},
platform: 'win32',
pathModule: path.win32
})
test('pathEnvKey finds the platform-cased PATH key', () => {
assert.equal(pathEnvKey({ Path: 'x' }, 'win32'), 'Path')
assert.equal(env.PATH, undefined)
// Windows leads with the portable layout (install.ps1 unpacks node.exe
// straight into node\, no bin\), then the POSIX shape for migrated installs.
assert.ok(
env.Path.startsWith(
'C:\\Users\\test\\AppData\\Local\\hermes\\node;C:\\Users\\test\\AppData\\Local\\hermes\\node\\bin;'
)
)
assert.ok(env.Path.includes('\\venv\\Scripts;'))
assert.ok(env.Path.includes(';C:\\Windows\\System32;C:\\Windows'))
assert.equal(env.Path.includes('/opt/homebrew/bin'), false)
assert.equal(pathEnvKey({ PATH: 'x' }, 'win32'), 'PATH')
assert.equal(pathEnvKey({}, 'win32'), 'PATH')
assert.equal(pathEnvKey({ Path: 'x' }, 'darwin'), 'PATH')
})
test('appendUniquePathEntries drops empty entries and keeps first occurrence', () => {
assert.equal(appendUniquePathEntries([':/a::/b', ['/a', '/c']], { delimiter: ':' }), '/a:/b:/c')
test('appendUniquePathEntries flattens, dedupes, and preserves first occurrence', () => {
assert.equal(appendUniquePathEntries(['/a:/b', ['/b', '/c'], '', null], { delimiter: ':' }), '/a:/b:/c')
})
+16 -71
View File
@@ -1,8 +1,9 @@
import path from 'node:path'
// Match the POSIX fallback surface used by the Python terminal environment.
// macOS apps launched from Finder/Dock often inherit only /usr/bin:/bin:/usr/sbin:/sbin,
// which misses Apple Silicon Homebrew and user-installed CLI tools such as codex.
// macOS apps launched from Finder/Dock inherit only /usr/bin:/bin:/usr/sbin:/sbin,
// which misses Homebrew and user-installed CLI tools (codex, git credential
// helpers). Hermes' own managed tools need no PATH help — the backend composes
// their environment in-process via pm — but user tools on PATH do.
const POSIX_SANE_PATH_ENTRIES = Object.freeze([
'/opt/homebrew/bin',
'/opt/homebrew/sbin',
@@ -30,12 +31,6 @@ function pathEnvKey(env = process.env, platform = process.platform) {
return Object.keys(env || {}).find(key => key.toUpperCase() === 'PATH') || 'PATH'
}
function currentPathValue(env = process.env, platform = process.platform) {
const key = pathEnvKey(env, platform)
return env?.[key] || ''
}
function appendUniquePathEntries(entries, { delimiter = path.delimiter } = {}) {
const seen = new Set()
const ordered = []
@@ -60,49 +55,6 @@ function appendUniquePathEntries(entries, { delimiter = path.delimiter } = {}) {
return ordered.join(delimiter)
}
/**
* Hermes-managed Node.js directories, in preferred lookup order.
*
* There are two on-disk layouts. `scripts/install.ps1` unpacks portable Node
* straight into `%LOCALAPPDATA%\hermes\node` (node.exe at the root, no `bin\`);
* `scripts/install.sh` and the node-bootstrap helper use the POSIX
* `$HERMES_HOME/node/bin`. Emit BOTH on every platform so mixed and migrated
* installs resolve, leading with the layout native to the current platform.
*
* This is the single source of truth for the ordering rule on the Node side —
* `main.ts` imports it rather than keeping its own copy. Mirrors
* `iter_hermes_node_dirs()` in hermes_constants.py, which the Electron main
* process cannot import.
*/
function hermesManagedNodePathEntries(
hermesHome,
{ platform = process.platform, pathModule = pathModuleForPlatform(platform) }: any = {}
) {
if (!hermesHome) {
return []
}
const root = pathModule.join(hermesHome, 'node')
const bin = pathModule.join(root, 'bin')
return platform === 'win32' ? [root, bin] : [bin, root]
}
function buildDesktopBackendPath({
hermesHome,
venvRoot,
currentPath = '',
platform = process.platform,
pathModule = pathModuleForPlatform(platform)
}: any = {}) {
const delimiter = delimiterForPlatform(platform)
const hermesNodeDirs = hermesManagedNodePathEntries(hermesHome, { platform, pathModule })
const venvBin = venvRoot ? pathModule.join(venvRoot, platform === 'win32' ? 'Scripts' : 'bin') : null
const saneEntries = platform === 'win32' ? [] : POSIX_SANE_PATH_ENTRIES
return appendUniquePathEntries([hermesNodeDirs, venvBin, currentPath, saneEntries], { delimiter })
}
function normalizeHermesHomeRoot(hermesHome, { pathModule = pathModuleForPlatform(process.platform) }: any = {}) {
if (!hermesHome) {
return hermesHome
@@ -118,20 +70,21 @@ function normalizeHermesHomeRoot(hermesHome, { pathModule = pathModuleForPlatfor
return resolved
}
function buildDesktopBackendEnv({
hermesHome,
pythonPathEntries = [],
venvRoot,
currentEnv = process.env,
platform = process.platform,
pathModule = pathModuleForPlatform(platform)
}: any = {}) {
/**
* The environment for the spawned Python backend. Electron knows ONE thing:
* where the interpreter is (by convention). Everything else — managed tool
* PATHs, browser paths, node — is composed in-process by pm when the backend
* spawns tools. PYTHONPATH/PYTHONHOME are scrubbed so an inherited value
* can't make the backend import modules from another checkout.
*/
function buildDesktopBackendEnv({ currentEnv = process.env, platform = process.platform }: any = {}) {
const delimiter = delimiterForPlatform(platform)
const currentPythonPath = currentEnv?.PYTHONPATH || ''
const key = pathEnvKey(currentEnv, platform)
const saneEntries = platform === 'win32' ? [] : POSIX_SANE_PATH_ENTRIES
return {
PYTHONPATH: appendUniquePathEntries([...pythonPathEntries, currentPythonPath], { delimiter }),
PYTHONPATH: '',
PYTHONHOME: '',
// Force PEP 540 UTF-8 mode in the spawned Python backend so its stdio and
// subprocess defaults are UTF-8 even on non-UTF-8 Windows locales (GBK,
// cp1252, ...). hermes_bootstrap sets this inside the child too, but only
@@ -139,22 +92,14 @@ function buildDesktopBackendEnv({
// pre-bootstrap tracebacks) still decodes with the locale default without
// this. User's explicit setting wins. Re-port of PR #56499 (echoriver89).
PYTHONUTF8: currentEnv?.PYTHONUTF8 ?? '1',
[key]: buildDesktopBackendPath({
hermesHome,
venvRoot,
currentPath: currentPathValue(currentEnv, platform),
platform,
pathModule
})
[key]: appendUniquePathEntries([currentEnv?.[key] || '', saneEntries], { delimiter })
}
}
export {
appendUniquePathEntries,
buildDesktopBackendEnv,
buildDesktopBackendPath,
delimiterForPlatform,
hermesManagedNodePathEntries,
normalizeHermesHomeRoot,
pathEnvKey,
POSIX_SANE_PATH_ENTRIES
+2 -1
View File
@@ -141,13 +141,14 @@ function hermesRuntimeImportProbe() {
* @param {object} [opts.env] - Additional environment for the probe.
* @returns {boolean}
*/
function canImportHermesCli(pythonPath: string, opts: { env?: Record<string, string> } = {}) {
function canImportHermesCli(pythonPath: string, opts: { env?: Record<string, string>; cwd?: string } = {}) {
if (!pythonPath) {
return false
}
try {
execProbeSync(pythonPath, ['-c', hermesRuntimeImportProbe()], {
cwd: opts.cwd,
env: { ...process.env, ...(opts.env || {}) },
stdio: 'ignore',
timeout: PROBE_TIMEOUT_MS,
@@ -1,54 +0,0 @@
import assert from 'node:assert/strict'
import { test } from 'vitest'
import { findGitBash } from './find-git-bash'
const yes = () => true
const no = () => false
test('HERMES_GIT_BASH_PATH override takes precedence', () => {
const result = findGitBash({
isWindows: true,
env: { HERMES_GIT_BASH_PATH: 'D:\\CustomGit\\bin\\bash.exe' },
fileExists: yes,
findOnPath: () => null
})
assert.equal(result, 'D:\\CustomGit\\bin\\bash.exe')
})
test('HERMES_GIT_BASH_PATH invalid path falls through to candidates', () => {
const env = {
HERMES_GIT_BASH_PATH: 'X:\\Missing\\bash.exe',
LOCALAPPDATA: 'C:\\Users\\test\\AppData\\Local',
ProgramFiles: 'C:\\Program Files',
'ProgramFiles(x86)': 'C:\\Program Files (x86)'
}
const fileExists = (p: string) => p !== 'X:\\Missing\\bash.exe' && p.includes('Program Files\\Git\\bin\\bash.exe')
const result = findGitBash({ isWindows: true, env, fileExists, findOnPath: () => null })
assert.equal(result, 'C:\\Program Files\\Git\\bin\\bash.exe')
})
test('HERMES_GIT_BASH_PATH empty string is ignored', () => {
const result = findGitBash({
isWindows: true,
env: { HERMES_GIT_BASH_PATH: '', LOCALAPPDATA: '' },
fileExists: no,
findOnPath: () => 'C:\\msys64\\usr\\bin\\bash.exe'
})
assert.equal(result, 'C:\\msys64\\usr\\bin\\bash.exe')
})
test('non-Windows uses findOnPath', () => {
const result = findGitBash({
isWindows: false,
env: {},
fileExists: no,
findOnPath: () => '/usr/bin/bash'
})
assert.equal(result, '/usr/bin/bash')
})
-67
View File
@@ -1,67 +0,0 @@
import path from 'node:path'
export interface GitBashOptions {
isWindows: boolean
env: Record<string, string | undefined>
fileExists: (filePath: string) => boolean
findOnPath?: (command: string) => string | null
}
/**
* Locate bash.exe on Windows.
* Resolution order (first match wins):
* 1. HERMES_GIT_BASH_PATH env var override
* 2. PortableGit under %LOCALAPPDATA%\hermes\git\ (install.ps1)
* 3. Standard Git for Windows install locations
* 4. %LOCALAPPDATA%\Programs\Git\ (user-scoped)
* 5. bash on PATH
*/
export function findGitBash(opts: GitBashOptions): string | null {
const { isWindows, env, fileExists, findOnPath } = opts
if (!isWindows) {
return findOnPath ? findOnPath('bash') : null
}
// Respect HERMES_GIT_BASH_PATH if set (mirrors tools/environments/local.py:_find_bash).
const gitBashPath = env.HERMES_GIT_BASH_PATH
if (gitBashPath && fileExists(gitBashPath)) {
return gitBashPath
}
const localAppData = env.LOCALAPPDATA || ''
const candidates: string[] = []
// Candidate paths are Windows paths regardless of host platform (tests run
// on POSIX CI hosts too), so join with win32 semantics explicitly.
const joinWin = path.win32.join
if (localAppData) {
candidates.push(joinWin(localAppData, 'hermes', 'git', 'bin', 'bash.exe'))
candidates.push(joinWin(localAppData, 'hermes', 'git', 'usr', 'bin', 'bash.exe'))
}
candidates.push(joinWin(env['ProgramFiles'] || 'C:\\Program Files', 'Git', 'bin', 'bash.exe'))
candidates.push(joinWin(env['ProgramFiles(x86)'] || 'C:\\Program Files (x86)', 'Git', 'bin', 'bash.exe'))
if (localAppData) {
candidates.push(joinWin(localAppData, 'Programs', 'Git', 'bin', 'bash.exe'))
}
for (const candidate of candidates) {
if (fileExists(candidate)) {
return candidate
}
}
if (findOnPath) {
const onPath = findOnPath('bash')
if (onPath) {
return onPath
}
}
return null
}
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,196 @@
import assert from 'node:assert/strict'
import fs from 'node:fs'
import os from 'node:os'
import path from 'node:path'
import { test } from 'vitest'
import { createHash } from 'node:crypto'
import { adoptPayloadVenv, installIdForRoot, isBundledInstall, resolvePayload } from './payload-backend'
function tmpdir() {
return fs.mkdtempSync(path.join(os.tmpdir(), 'payload-test-'))
}
function writePayload(
root: string,
{
manifest = { schema: 1, target: 'win32-x64', repo: 'hermes-agent', venv: 'venv', store: 'tools' },
isWindows = true
}: any = {}
) {
const dir = path.join(root, 'agent-payload')
fs.mkdirSync(path.join(dir, 'hermes-agent'), { recursive: true })
fs.mkdirSync(path.join(dir, 'tools', 'python-3.11.16-win32-x64'), { recursive: true })
// Store python (win: python.exe at the entry root; posix: bin/python3).
if (isWindows) {
fs.writeFileSync(path.join(dir, 'tools', 'python-3.11.16-win32-x64', 'python.exe'), '')
} else {
fs.mkdirSync(path.join(dir, 'tools', 'python-3.11.16-win32-x64', 'bin'), { recursive: true })
fs.writeFileSync(path.join(dir, 'tools', 'python-3.11.16-win32-x64', 'bin', 'python3'), '')
}
// Venv site-packages (where the project deps are installed).
const sp = isWindows
? path.join(dir, 'venv', 'Lib', 'site-packages')
: path.join(dir, 'venv', 'lib', 'python3.11', 'site-packages')
fs.mkdirSync(sp, { recursive: true })
// The self-relative CLI entrypoint (minted launcher exe on win32, bash
// trampoline on POSIX) — the bundled entry point.
const binDir = path.join(dir, 'bin')
fs.mkdirSync(binDir, { recursive: true })
fs.writeFileSync(path.join(binDir, isWindows ? 'hermes.exe' : 'hermes'), '')
fs.writeFileSync(path.join(dir, 'manifest.json'), JSON.stringify(manifest))
fs.writeFileSync(
path.join(dir, 'tools', 'facts.json'),
JSON.stringify({ packages: { python: { entry: 'python-3.11.16-win32-x64', version: '3.11.16' } } })
)
return dir
}
const fsDeps = {
fileExists: (p: string) => {
try {
return fs.statSync(p).isFile()
} catch {
return false
}
},
directoryExists: (p: string) => {
try {
return fs.statSync(p).isDirectory()
} catch {
return false
}
}
}
test('resolvePayload finds a complete payload (store python + venv site-packages)', () => {
const root = tmpdir()
writePayload(root)
const payload = resolvePayload(root, { ...fsDeps, isWindows: true })
assert.ok(payload)
assert.equal(payload.repoDir, path.join(root, 'agent-payload', 'hermes-agent'))
assert.ok(payload.storePython.endsWith(path.join('tools', 'python-3.11.16-win32-x64', 'python.exe')))
assert.ok(payload.sitePackages.endsWith(path.join('venv', 'Lib', 'site-packages')))
assert.ok(payload.shim.endsWith(path.join('bin', 'hermes.exe')))
})
test('resolvePayload resolves the posix store python + site-packages layout', () => {
const root = tmpdir()
writePayload(root, { isWindows: false })
const payload = resolvePayload(root, { ...fsDeps, isWindows: false })
assert.ok(payload)
assert.ok(payload.storePython.endsWith(path.join('tools', 'python-3.11.16-win32-x64', 'bin', 'python3')))
assert.ok(payload.sitePackages.endsWith(path.join('venv', 'lib', 'python3.11', 'site-packages')))
assert.ok(payload.shim.endsWith(path.join('bin', 'hermes')))
})
test('resolvePayload returns null without a manifest, for external stubs, and for a broken payload', () => {
assert.equal(resolvePayload(tmpdir(), { ...fsDeps, isWindows: true }), null)
assert.equal(resolvePayload(undefined, { ...fsDeps, isWindows: true }), null)
const externalRoot = tmpdir()
writePayload(externalRoot, { manifest: { schema: 1, external: true } })
assert.equal(resolvePayload(externalRoot, { ...fsDeps, isWindows: true }), null)
const brokenRoot = tmpdir()
const dir = writePayload(brokenRoot)
fs.rmSync(path.join(dir, 'venv'), { recursive: true })
assert.equal(resolvePayload(brokenRoot, { ...fsDeps, isWindows: true }), null)
})
test('adoptPayloadVenv verifies store python + site-packages without any cfg write', () => {
const root = tmpdir()
const dir = writePayload(root)
const payload = resolvePayload(root, { ...fsDeps, isWindows: true })
assert.ok(payload)
// Bundled builds run the store python directly — there is NO pyvenv.cfg
// write (the payload may be read-only, e.g. MSIX). The cfg, if present,
// is left untouched.
fs.writeFileSync(path.join(dir, 'venv', 'pyvenv.cfg'), 'home = C:\\ci\\build\\python\nversion_info = 3.11.16\n')
assert.equal(adoptPayloadVenv(payload, { isWindows: true }), true)
const text = fs.readFileSync(path.join(dir, 'venv', 'pyvenv.cfg'), 'utf8')
assert.ok(text.includes('C:\\ci\\build'), 'the shipped cfg is not rewritten')
// second call: still reports usable
assert.equal(adoptPayloadVenv(payload, { isWindows: true }), true)
})
// adoptPayloadVenv is now a pure presence-verify; resolvePayload's own
// existence checks already reject a payload missing the store python or
// site-packages (covered above), so there is no separate fail-closed path
// to assert at this layer.
test('isBundledInstall is true for a real payload manifest', () => {
const root = tmpdir()
writePayload(root)
assert.equal(isBundledInstall(root, fsDeps), true)
})
test('isBundledInstall is false for the external stub and for missing manifests', () => {
const externalRoot = tmpdir()
writePayload(externalRoot, { manifest: { schema: 1, external: true } })
assert.equal(isBundledInstall(externalRoot, fsDeps), false)
assert.equal(isBundledInstall(tmpdir(), fsDeps), false)
assert.equal(isBundledInstall(undefined, fsDeps), false)
})
test('isBundledInstall is true even when the payload is damaged (never-install guard)', () => {
const root = tmpdir()
const dir = writePayload(root)
// A broken payload must still read as bundled: isBundledInstall is the
// guard that REFUSES the installer, and a damaged bundle is exactly the
// case where the installer must not run.
fs.rmSync(path.join(dir, 'venv'), { recursive: true })
fs.rmSync(path.join(dir, 'tools', 'python-3.11.16-win32-x64'), { recursive: true })
assert.equal(resolvePayload(root, { ...fsDeps, isWindows: true }), null)
assert.equal(isBundledInstall(root, fsDeps), true)
})
test('isBundledInstall is false for a malformed manifest', () => {
const root = tmpdir()
const dir = path.join(root, 'agent-payload')
fs.mkdirSync(dir, { recursive: true })
fs.writeFileSync(path.join(dir, 'manifest.json'), 'not json {')
assert.equal(isBundledInstall(root, fsDeps), false)
})
// ─── update channel helpers ─────────────────────────────────────────
test('installIdForRoot matches the Python install id (sha16 of the canonical path)', () => {
// sha256('/home/u/.hermes/hermes-agent')[:16] — recomputed independently.
assert.equal(
installIdForRoot('/home/u/.hermes/hermes-agent'),
createHash('sha256').update('/home/u/.hermes/hermes-agent', 'utf8').digest('hex').slice(0, 16)
)
// The canonicalizer output is what gets hashed (symlinked homes).
assert.equal(
installIdForRoot('/link/hermes-agent', () => '/real/hermes-agent'),
installIdForRoot('/real/hermes-agent')
)
})
+172
View File
@@ -0,0 +1,172 @@
/**
* payload-backend.ts
*
* The bundled-install backend: a pm payload staged by `hermes pm bundle`
* and shipped under resources/agent-payload. The payload carries the repo
* snapshot, the tool store (with facts.json), and a relocatable venv
* built on the staged python-build-standalone interpreter.
*
* Electron's whole job here is finding the interpreter and verifying the
* payload can boot. Bundled builds run the store python directly —
* self-relative, no pyvenv.cfg write, so read-only installs (MSIX) work.
* Everything else — managed tool PATHs, env composition — happens
* in-process via pm when the backend runs.
*/
import { createHash } from 'node:crypto'
import fs from 'node:fs'
import path from 'node:path'
export interface PayloadInfo {
root: string
repoDir: string
toolsDir: string
/** The payload's own CPython (tools/<python-entry>/python(.exe)). */
storePython: string
/** The venv's site-packages (Lib/site-packages on win, lib/python3.11/site-packages on posix). */
sitePackages: string
/** The self-relative CLI trampoline (bin/hermes(.exe)) — the bundled entry point. */
shim: string
}
export function resolvePayload(
resourcesPath: string | undefined,
deps: {
fileExists: (p: string) => boolean
directoryExists: (p: string) => boolean
isWindows: boolean
}
): PayloadInfo | null {
if (!resourcesPath) {
return null
}
const root = path.join(resourcesPath, 'agent-payload')
const manifestPath = path.join(root, 'manifest.json')
if (!deps.fileExists(manifestPath)) {
return null
}
let manifest: any
try {
manifest = JSON.parse(fs.readFileSync(manifestPath, 'utf8'))
} catch {
return null
}
if (!manifest || manifest.external === true) {
return null
}
// The manifest names the payload layout; a bundle that omits the fields
// is malformed — refuse it rather than guess at cmd_bundle's layout.
if (typeof manifest.repo !== 'string' || typeof manifest.store !== 'string' || typeof manifest.venv !== 'string') {
return null
}
const repoDir = path.join(root, manifest.repo)
const toolsDir = path.join(root, manifest.store)
const venvDir = path.join(root, manifest.venv)
// The CLI trampoline staged into bin/ (hermes/hermes-agent/hermes-acp —
// build-bundled-desktop.mjs 5b: distlib-minted launchers on win32,
// $0-relative bash trampolines on POSIX). It execs the store python with
// the payload's own PYTHONPATH, so it is the single bundled entry point.
const shim = path.join(root, 'bin', deps.isWindows ? 'hermes.exe' : 'hermes')
// The store CPython + the venv's site-packages. Bundled builds run the
// STORE python (self-relative, no pyvenv.cfg write — works on read-only
// MSIX) with PYTHONPATH pointing at the venv site-packages (where the
// project deps are installed). The venv python itself is NOT used in
// bundled builds.
let storePython = ''
let sitePackages = ''
try {
const facts = JSON.parse(fs.readFileSync(path.join(toolsDir, 'facts.json'), 'utf8'))
const entry = facts?.packages?.python?.entry
if (typeof entry === 'string') {
storePython = path.join(toolsDir, entry, deps.isWindows ? 'python.exe' : 'bin', deps.isWindows ? '' : 'python3')
sitePackages = deps.isWindows
? path.join(venvDir, 'Lib', 'site-packages')
: path.join(venvDir, 'lib', `python${process.env.PYTHON_VER || '3.11'}`, 'site-packages')
}
} catch {
// fall through to the existence checks below
}
if (!deps.directoryExists(repoDir) || !deps.fileExists(storePython) || !deps.directoryExists(sitePackages) || !deps.fileExists(shim)) {
return null
}
return { root, repoDir, toolsDir, storePython, sitePackages, shim }
}
/**
* "Is this artifact a bundled install?" — the app ships its own Hermes payload.
* True whenever resources/agent-payload/manifest.json exists and is not the
* external stub (before-build.mjs writes {schema:1, external:true} for
* non-bundled builds). Deliberately does NOT verify payload usability: a
* damaged bundle still must never install — callers use this to refuse the
* installer, not to decide the payload can boot.
*/
export function isBundledInstall(
resourcesPath: string | undefined,
deps: { fileExists: (p: string) => boolean }
): boolean {
if (!resourcesPath) {
return false
}
const manifestPath = path.join(resourcesPath, 'agent-payload', 'manifest.json')
if (!deps.fileExists(manifestPath)) {
return false
}
try {
const manifest = JSON.parse(fs.readFileSync(manifestPath, 'utf8'))
return Boolean(manifest) && manifest.external !== true
} catch {
return false
}
}
/**
* Verify the payload is usable — the store python + venv site-packages
* resolve. NO pyvenv.cfg write: bundled builds run the store python
* directly (self-relative, works on read-only MSIX), so there is nothing
* to re-point. Returns true when the payload can boot.
*/
export function adoptPayloadVenv(
payload: PayloadInfo,
deps: { isWindows: boolean; log?: (m: string) => void }
): boolean {
if (!payload.storePython || !payload.sitePackages) {
deps.log?.('[payload] missing store python or site-packages')
return false
}
return true
}
// ─── update channel ─────────────────────────────────────────────────────────
//
// The CLI owns the channel records; Electron only reads the install id for
// `update.installs.<sha16>/` bookkeeping. Channel resolution itself lives in
// hermes_cli/update_channel.py — main.ts keys nightly/stable off the baked
// install stamp tag directly.
export type UpdateChannel = 'stable' | 'main' | 'nightly'
/**
* The install id of the tree at `root`: sha16 of the canonical PATH,
* byte-identical to Python's install id (sha256 of the resolved root,
* first 16 hex chars — `boot_bootstrap._install_key` /
* `update_channel._install_key_sha16`). Path-derived so it survives
* artifact replacement at the same location; the same key names
* `installs/<sha16>/`.
*/
export function installIdForRoot(root: string, canonicalize: (p: string) => string = p => p): string {
return createHash('sha256').update(canonicalize(root), 'utf8').digest('hex').slice(0, 16)
}
@@ -0,0 +1,66 @@
/**
* Tests for electron/update-root-policy.ts — the pure classifier that decides
* whether the desktop's git-based self-update may run against a resolved
* update root. A checkout the install contract does not manage (steward-owned
* `external`/`electron-updater` mechanisms) must be refused with a user-action
* pointer instead of the desktop pulling into it.
*/
import assert from 'node:assert/strict'
import { test } from 'vitest'
import { classifyUpdateRoot } from './update-root-policy'
test('a non-git root is not updatable and carries no user advice', () => {
const result = classifyUpdateRoot({ isGitTree: false, updateMechanism: null })
assert.equal(result.updatable, false)
assert.equal(result.verdict, 'not-a-checkout')
assert.equal(result.provenance, 'not-a-checkout')
assert.equal(result.advice, null)
assert.ok(result.message)
})
test('a self-managed checkout is updatable', () => {
const result = classifyUpdateRoot({ isGitTree: true, updateMechanism: 'self' })
assert.equal(result.updatable, true)
assert.equal(result.verdict, 'updatable')
assert.equal(result.provenance, 'managed-self')
assert.equal(result.advice, null)
assert.equal(result.message, null)
})
test('a checkout owned by the external steward is refused with a git pull pointer', () => {
const result = classifyUpdateRoot({ isGitTree: true, updateMechanism: 'external' })
assert.equal(result.updatable, false)
assert.equal(result.verdict, 'steward-owned-git-tree')
assert.equal(result.provenance, 'steward-owned')
assert.equal(result.advice, 'git pull')
assert.ok(result.message?.includes('external'))
})
test('a checkout owned by electron-updater is refused the same way', () => {
const result = classifyUpdateRoot({ isGitTree: true, updateMechanism: 'electron-updater' })
assert.equal(result.updatable, false)
assert.equal(result.verdict, 'steward-owned-git-tree')
assert.equal(result.advice, 'git pull')
assert.ok(result.message?.includes('electron-updater'))
})
test('an unstamped git tree (dev checkout) stays updatable with unknown provenance', () => {
const result = classifyUpdateRoot({ isGitTree: true, updateMechanism: null })
assert.equal(result.updatable, true)
assert.equal(result.verdict, 'updatable')
assert.equal(result.provenance, 'unknown')
assert.equal(result.advice, null)
})
test('the classification is a pure function of its inputs', () => {
const facts = { isGitTree: true, updateMechanism: 'external' as const }
assert.deepEqual(classifyUpdateRoot(facts), classifyUpdateRoot({ ...facts }))
})
@@ -0,0 +1,94 @@
// update-root-policy.ts — pure classifier for the desktop self-update root.
//
// The desktop's git-based self-update arm runs `git fetch/merge` against a
// checkout it resolved (resolveUpdateRoot in main.ts). That root is only
// legitimate update territory when the install contract says so:
//
// - Not a `.git` tree → there is nothing to pull; the desktop cannot
// self-update here (bundled installs use the OS App Installer arm and
// never reach this classifier).
// - A `.git` tree whose install stamp says `updateMechanism: "self"` → the
// install is a managed self-updating checkout (e.g. the desktop
// bootstrap's clone, which writes exactly that stamp) — updatable.
// - A `.git` tree with any OTHER mechanism (`external`: the store/steward
// owns updates; `electron-updater`: the desktop package does) → the tree
// is managed by someone else. Pulling into it from the desktop would
// stash-and-move a user's checkout out from under its steward.
// - No stamp at all (null mechanism; a dev source checkout) → provenance
// unknown. The classifier reports `updatable` with `provenance:
// 'unknown'`: a developer's working tree keeps its existing update flow,
// and callers log the ambiguity rather than silently widening the refusal.
//
// Pure and dependency-injected (same shape as update-gate.ts) so the policy
// is unit-testable without booting Electron, and every caller gets the same
// answer from one authority.
export type UpdateMechanism = 'self' | 'electron-updater' | 'external'
export type UpdateRootProvenance = 'managed-self' | 'steward-owned' | 'unknown' | 'not-a-checkout'
export interface UpdateRootClassification {
/** Whether the desktop's git-based self-update may run against this root. */
updatable: boolean
/** Machine-readable verdict for update-check results and logs. */
verdict: 'updatable' | 'not-a-checkout' | 'steward-owned-git-tree' | 'unmanaged-git-tree'
/** Why the root is (or is not) updatable, for user-facing messages. */
message: string | null
/** The command that fixes it, when the user (not the app) must act. */
advice: 'git pull' | null
provenance: UpdateRootProvenance
}
export interface UpdateRootFacts {
/** True when the resolved update root contains a `.git` entry. */
isGitTree: boolean
/** The install stamp's updateMechanism, or null when there is no stamp. */
updateMechanism: UpdateMechanism | null
}
/**
* Classify whether the desktop may self-update (git pull semantics) against
* the resolved update root.
*/
export function classifyUpdateRoot(facts: UpdateRootFacts): UpdateRootClassification {
if (!facts.isGitTree) {
return {
updatable: false,
verdict: 'not-a-checkout',
message: 'This install has no git checkout to update — the app or its steward owns the update loop.',
advice: null,
provenance: 'not-a-checkout'
}
}
if (facts.updateMechanism === 'self') {
return {
updatable: true,
verdict: 'updatable',
message: null,
advice: null,
provenance: 'managed-self'
}
}
if (facts.updateMechanism === 'external' || facts.updateMechanism === 'electron-updater') {
return {
updatable: false,
verdict: 'steward-owned-git-tree',
message:
`This checkout is managed by its install method (${facts.updateMechanism}); ` +
'the desktop will not run git updates against it.',
advice: 'git pull',
provenance: 'steward-owned'
}
}
// No stamp: unknown provenance (typically a developer source checkout).
return {
updatable: true,
verdict: 'updatable',
message: null,
advice: null,
provenance: 'unknown'
}
}
@@ -13,16 +13,10 @@
// re-selected forever instead of falling through to bootstrap.
import assert from 'node:assert/strict'
import path from 'node:path'
import { test } from 'vitest'
import {
buildPathExtCandidates,
chooseUpdaterArgs,
getVenvSitePackagesEntries,
resolveVenvHermesCommand
} from './windows-hermes-path'
import { buildPathExtCandidates, chooseUpdaterArgs, resolveVenvHermesCommand } from './windows-hermes-path'
test('buildPathExtCandidates: Windows tries PATHEXT extensions before the empty extension', () => {
const extensions = buildPathExtCandidates('.COM;.EXE;.BAT;.CMD', true)
@@ -92,9 +86,7 @@ function makeDeps(overrides: Partial<Parameters<typeof resolveVenvHermesCommand>
directoryExists: () => false,
canImportHermesCli: () => true,
getVenvPython: (venvRoot: string) => `${venvRoot}/Scripts/python.exe`,
getVenvSitePackagesEntries: () => [],
buildDesktopBackendEnv: () => ({ FAKE_ENV: '1' }),
hermesHome: '/fake/hermes-home',
resolvePath: (...segments: string[]) => segments.join('/').replace(/\/+/g, '/'),
dirname: (p: string) => p.slice(0, p.lastIndexOf('/')) || '/',
basename: (p: string) => p.slice(p.lastIndexOf('/') + 1),
@@ -170,71 +162,3 @@ test('resolveVenvHermesCommand: is case-insensitive on hermes.exe and the Script
assert.ok(resolveVenvHermesCommand('/root/venv/Scripts/HERMES.EXE', [], deps))
assert.ok(resolveVenvHermesCommand('/root/venv/SCRIPTS/hermes.exe', [], deps))
})
// ── getVenvSitePackagesEntries ─────────────────────────────────────────────
test('getVenvSitePackagesEntries: returns Lib/site-packages on Windows when it exists', () => {
const expected = path.join('C:\\venv', 'Lib', 'site-packages')
const result = getVenvSitePackagesEntries('C:\\venv', {
isWindows: true,
directoryExists: p => p === expected
})
assert.deepEqual(result, [expected])
})
test('getVenvSitePackagesEntries: returns empty on Windows when site-packages does not exist', () => {
const result = getVenvSitePackagesEntries('C:\\venv', {
isWindows: true,
directoryExists: () => false
})
assert.deepEqual(result, [])
})
test('getVenvSitePackagesEntries: reads pyvenv.cfg version on POSIX and resolves lib/pythonX.Y/site-packages', () => {
const result = getVenvSitePackagesEntries('/venv', {
isWindows: false,
directoryExists: p => p === '/venv/lib/python3.12/site-packages',
readFile: () => 'version_info = 3.12.1\n'
})
assert.deepEqual(result, ['/venv/lib/python3.12/site-packages'])
})
test('getVenvSitePackagesEntries: returns empty on POSIX when pyvenv.cfg is missing', () => {
const result = getVenvSitePackagesEntries('/venv', {
isWindows: false,
directoryExists: () => true,
readFile: () => undefined
})
assert.deepEqual(result, [])
})
test('getVenvSitePackagesEntries: returns empty on POSIX when pyvenv.cfg has no version_info', () => {
const result = getVenvSitePackagesEntries('/venv', {
isWindows: false,
directoryExists: () => true,
readFile: () => 'home = /usr/bin\n'
})
assert.deepEqual(result, [])
})
test('getVenvSitePackagesEntries: returns empty on POSIX when version is present but site-packages dir is absent', () => {
const result = getVenvSitePackagesEntries('/venv', {
isWindows: false,
directoryExists: () => false,
readFile: () => 'version_info = 3.11\n'
})
assert.deepEqual(result, [])
})
test('getVenvSitePackagesEntries: returns empty for a falsy venvRoot', () => {
assert.deepEqual(getVenvSitePackagesEntries('', { isWindows: true, directoryExists: () => true }), [])
assert.deepEqual(getVenvSitePackagesEntries(null, { isWindows: true, directoryExists: () => true }), [])
assert.deepEqual(getVenvSitePackagesEntries(undefined, { isWindows: true, directoryExists: () => true }), [])
})
+6 -111
View File
@@ -29,9 +29,6 @@
* backend-command.ts.
*/
import fs from 'node:fs'
import path from 'node:path'
/**
* Build the ordered list of extensions findOnPath() should try when
* resolving a bare command name off PATH.
@@ -82,104 +79,14 @@ export function chooseUpdaterArgs(signals: BootstrapRecoverySignals, branch: str
return canRunUpdater ? ['--update', '--branch', branch] : ['--repair', '--branch', branch]
}
/**
* Resolve the site-packages directory entries for a Python venv.
*
* On Windows, venv layout is `<venvRoot>/Lib/site-packages`.
* On POSIX, it's `<venvRoot>/lib/python<version>/site-packages` where
* `<version>` (e.g. `3.12`) is read from the venv's `pyvenv.cfg`
* `version_info` field.
*
* Returns only directories that actually exist on disk. Returns an empty
* array when `venvRoot` is falsy or no matching site-packages dir is found.
*
* Extracted from main.ts so the platform branching can be tested without
* reading source text. `isWindows` and `directoryExists` are injectable;
* `readFile` defaults to `fs.readFileSync` but can be overridden for tests.
*/
export function getVenvSitePackagesEntries(
venvRoot: string | undefined | null,
opts: {
isWindows?: boolean
directoryExists?: (p: string) => boolean
readFile?: (p: string) => string | undefined
} = {}
): string[] {
const entries: string[] = []
if (!venvRoot) {
return entries
}
const isWindows = opts.isWindows ?? process.platform === 'win32'
const directoryExists =
opts.directoryExists ??
((p: string) => {
try {
return fs.statSync(p).isDirectory()
} catch {
return false
}
})
const readFile =
opts.readFile ??
((p: string) => {
try {
return fs.readFileSync(p, 'utf8')
} catch {
return undefined
}
})
if (isWindows) {
const sitePackages = path.join(venvRoot, 'Lib', 'site-packages')
if (directoryExists(sitePackages)) {
entries.push(sitePackages)
}
return entries
}
const cfg = readFile(path.join(venvRoot, 'pyvenv.cfg'))
const version = (() => {
if (!cfg) {
return null
}
const match = cfg.match(/^version_info\s*=\s*(\d+\.\d+)/im)
return match ? match[1].trim() : null
})()
if (version) {
const sitePackages = path.join(venvRoot, 'lib', `python${version}`, 'site-packages')
if (directoryExists(sitePackages)) {
entries.push(sitePackages)
}
}
return entries
}
export interface ResolveVenvHermesCommandDeps {
isWindows: boolean
isCommandScript: (command: string) => boolean
fileExists: (filePath: string) => boolean
directoryExists: (filePath: string) => boolean
canImportHermesCli: (python: string, opts?: { env?: Record<string, string> }) => boolean
canImportHermesCli: (python: string, opts?: { env?: Record<string, string>; cwd?: string }) => boolean
getVenvPython: (venvRoot: string) => string
getVenvSitePackagesEntries: (venvRoot: string) => string[]
buildDesktopBackendEnv: (opts: {
hermesHome: string
pythonPathEntries: string[]
venvRoot: string
}) => Record<string, string>
hermesHome: string
buildDesktopBackendEnv: () => Record<string, string>
resolvePath: (...segments: string[]) => string
dirname: (p: string) => string
basename: (p: string) => string
@@ -226,9 +133,7 @@ export function resolveVenvHermesCommand(
directoryExists,
canImportHermesCli,
getVenvPython,
getVenvSitePackagesEntries,
buildDesktopBackendEnv,
hermesHome,
resolvePath,
dirname,
basename,
@@ -260,15 +165,9 @@ export function resolveVenvHermesCommand(
const root = dirname(venvRoot)
if (
!canImportHermesCli(python, {
env: {
PYTHONPATH: [...(directoryExists(root) ? [root] : []), process.env.PYTHONPATH]
.filter((entry): entry is string => Boolean(entry))
.join(path.delimiter)
}
})
) {
// Probe with the same semantics the real spawn uses: venv interpreter,
// cwd at the checkout root, no PYTHONPATH.
if (!canImportHermesCli(python, { cwd: directoryExists(root) ? root : undefined })) {
rememberLog?.(
`Ignoring venv Hermes at ${python}: runtime import probe failed (broken/partial venv); falling through to bootstrap.`
)
@@ -281,11 +180,7 @@ export function resolveVenvHermesCommand(
command: python,
args: ['-m', 'hermes_cli.main', ...backendArgs],
bootstrap: false,
env: buildDesktopBackendEnv({
hermesHome,
pythonPathEntries: [...(directoryExists(root) ? [root] : []), ...getVenvSitePackagesEntries(venvRoot)],
venvRoot
}),
env: buildDesktopBackendEnv(),
kind: 'python',
root,
shell: false
+12
View File
@@ -21,6 +21,10 @@
import path from 'node:path'
import { findPackedPayload, relativizePayloadLinks, stripFetchCache } from './materialize-payload-links.mjs'
import { batchSignAppTree } from './batch-sign-binaries.mjs'
import { resolveSigningIdentity, signNestedChromium } from './sign-nested-chromium.mjs'
import { sanitizeTree } from './sanitize-pe-signatures.mjs'
import { stampExeIdentity } from './set-exe-identity.mjs'
export default async function afterPack(context) {
@@ -38,4 +42,12 @@ export default async function afterPack(context) {
// Never fail the build over a cosmetic stamp.
console.warn(`[after-pack] exe identity stamp failed (${err.message}); Hermes.exe keeps the stock Electron icon`)
}
// Batch-sign every payload binary AFTER sanitize (above) and the rcedit
// stamp: a dangling certificate table or a subsequent resource edit would
// invalidate the signature. The product exe is excluded here and signed
// per-file by the customSign hook (scripts/batch-sign-binaries.mjs) after
// electron-builder's own rcedit + fuses pass. No-op with a loud warning when
// the AZURE_SIGN_* environment is absent (unsigned/fork/nightly lanes).
batchSignAppTree(context.appOutDir, exe)
}
@@ -0,0 +1,414 @@
// batch-sign-binaries.mjs — Authenticode-sign every standalone binary inside
// the packed Windows app tree (Hermes.exe's sibling DLLs and everything under
// resources/agent-payload/tools/: node.exe, ffmpeg.exe, chromium, the minted
// CLI launcher exes, pm tool binaries, …).
//
// Why a batch: Windows validates only the MSIX package signature
// (AppxSignature.p7x over AppxBlockMap.xml), so inner binaries have never been
// signed. But an MSIX whose payload carries unsigned PEs trips SmartScreen /
// enterprise WDAC policies that scan inner files, and signature-verification
// tooling reports the app as mixed signed/unsigned. Task 0 (pm-clean fix
// plan): sign the whole tree, once, in chunked signtool invocations.
//
// Ordering (electron-builder win32 pipeline, pinned by app-builder-lib source):
// beforePack → pack → afterPack (this module's caller) → electron fuses
// → signAndEditResources (rcedit on the product exe) → per-file sign hook.
// Consequences:
// - The batch runs in afterPack AFTER sanitize-pe-signatures.mjs (a dangling
// certificate table makes signtool fail 0x800700C1) and AFTER the rcedit
// identity stamp, so neither can invalidate what we sign.
// - The product exe (`<productName>.exe`) is EXCLUDED from the batch:
// rcedit edits its resources (and the electron fuses flip) after
// afterPack, which invalidates any signature. It is signed per-file by the
// customSign hook AFTER rcedit, on the exact Azure mechanism sign-msix.mjs
// uses for the package itself.
// - The customSign hook returns true (does nothing) for every file the batch
// already covered, so electron-builder never re-signs one-by-one.
//
// Sign-nested-chromium.mjs stays as-is: it is macOS-only (codesign --deep over
// .app bundles inside the payload for Apple notarization) and is invoked from
// the darwin branch of after-pack.mjs. There is no overlap with this Windows
// Authenticode batch.
//
// Gating matches the rest of the pipeline: this module only signs when the
// Azure Trusted Signing variables are present (the same AZURE_SIGN_* set the
// release-signing workflow arms provide). Without them — local builds, forks,
// unsigned nightly lanes — it is a no-op with a loud warning, exactly like
// stage-msixbundle.mjs. The dlib + signtool resolution reuses the
// electron-builder cache walk from scripts/stage-msixbundle.mjs; nothing is
// hardcoded to C:\Tools.
import { execFileSync } from 'node:child_process'
import fs from 'node:fs'
import path from 'node:path'
import { isMain } from './utils.mjs'
export const CHUNK_SIZE = 100
/**
* Recursively collect every .exe/.dll under dir. Symbolic links are skipped
* (the payload's materialized-link layout can contain them; the target is
* collected on its own walk). Sorted for deterministic chunking.
*
* @param {string} dir
* @param {{ skip?: (file: string) => boolean }} [opts]
* @returns {string[]}
*/
export function getBinaries(dir, opts = {}) {
const out = []
const walk = (current) => {
let entries
try {
entries = fs.readdirSync(current, { withFileTypes: true })
} catch {
return
}
for (const entry of entries) {
if (entry.isSymbolicLink()) continue
const full = path.join(current, entry.name)
if (entry.isDirectory()) {
walk(full)
continue
}
if (!entry.isFile()) continue
const lower = entry.name.toLowerCase()
if (lower.endsWith('.exe') || lower.endsWith('.dll')) {
if (opts.skip && opts.skip(full)) continue
out.push(full)
}
}
}
walk(dir)
return out.sort()
}
/** Split a file list into signtool-sized batches. */
export function chunk(items, size = CHUNK_SIZE) {
const chunks = []
for (let i = 0; i < items.length; i += size) {
chunks.push(items.slice(i, i + size))
}
return chunks
}
/** True when the Azure Trusted Signing variables are all present. */
export function azureSigningConfigured(env = process.env) {
return Boolean(env.AZURE_SIGN_ENDPOINT && env.AZURE_SIGN_ACCOUNT && env.AZURE_SIGN_PROFILE)
}
// The electron-builder toolset cache roots, in precedence order — the same
// walk scripts/stage-msixbundle.mjs does (configured ELECTRON_BUILDER_CACHE
// beats stray defaults).
function cacheRoots(env) {
return [
env.ELECTRON_BUILDER_CACHE || '',
path.join(env.LOCALAPPDATA || '', 'electron-builder', 'Cache'),
path.join(env.USERPROFILE || '', 'AppData', 'Local', 'electron-builder', 'Cache')
].filter(Boolean)
}
/**
* Host arch for the signing toolset. The ATS bundle ships the dlib for
* x64/x86 only (its arm64 dir has no dlib), and a 32-bit signtool cannot
* load a 64-bit dlib — so the signtool + dlib pair must be same-arch. On
* arm64 hosts use the x64 pair (x64 signtool runs under Windows-on-ARM
* emulation). Mirrors app-builder-lib's WindowsSignAzureManager
* (`process.arch === "ia32" ? Arch.ia32 : Arch.x64`).
*/
export function signingArch() {
return process.arch === 'ia32' ? 'x86' : 'x64'
}
/**
* True when a resolved path sits in the `<arch>/` dir (e.g. `.../x64/signtool.exe`
* or `.../ats-bundle-.../x64/Azure.CodeSigning.Dlib.dll`), matching both the
* modern windows-kits-bundle layout and the legacy windows-10/<arch> layout.
*/
function archMatch(file, arch) {
return path.basename(path.dirname(file)).toLowerCase() === arch
}
/**
* Find azure.codesigning.dlib.dll under the electron-builder cache, preferring
* the one matching the host arch (x64 unless the host is ia32) so signtool can
* load it. Returns an absolute path or null (caller decides loud-fail vs warn).
*/
export function resolveTrustedSigningDlib(env = process.env) {
const arch = signingArch()
for (const root of cacheRoots(env)) {
if (!fs.existsSync(root)) continue
const found = []
const walk = (p) => {
let entries
try {
entries = fs.readdirSync(p, { withFileTypes: true })
} catch {
return
}
for (const entry of entries) {
const full = path.join(p, entry.name)
if (entry.isDirectory()) walk(full)
else if (entry.name.toLowerCase() === 'azure.codesigning.dlib.dll') found.push(full)
}
}
for (const entry of fs.readdirSync(root)) {
walk(path.join(root, entry))
}
if (found.length > 0) {
const matched = found.filter(f => archMatch(f, arch))
const pool = matched.length > 0 ? matched : found
pool.sort()
return pool[pool.length - 1]
}
}
return null
}
/**
* Find a host signtool.exe under the electron-builder cache (the winCodeSign
* toolset bundles the Windows Kits tools for arm64/x64/x86), or honor
* SIGNTOOL_PATH. Prefers the signtool matching the host arch — a 32-bit
* signtool cannot load the x64 ATS dlib — and among arch-matched candidates
* the newest SDK build (sorted paths put the highest version last).
* Returns an absolute path or null.
*/
export function resolveSigntool(env = process.env) {
if (env.SIGNTOOL_PATH && fs.existsSync(env.SIGNTOOL_PATH)) return env.SIGNTOOL_PATH
const arch = signingArch()
for (const root of cacheRoots(env)) {
if (!fs.existsSync(root)) continue
const found = []
for (const entry of fs.readdirSync(root)) {
const dir = path.join(root, entry)
if (!fs.existsSync(dir) || !fs.statSync(dir).isDirectory()) continue
const walk = (p, depth) => {
if (depth > 5) return
let entries
try {
entries = fs.readdirSync(p, { withFileTypes: true })
} catch {
return
}
for (const sub of entries) {
if (sub.isSymbolicLink()) continue
const full = path.join(p, sub.name)
if (sub.isDirectory()) walk(full, depth + 1)
else if (sub.name.toLowerCase() === 'signtool.exe') found.push(full)
}
}
walk(dir, 0)
}
if (found.length > 0) {
const matched = found.filter(f => archMatch(f, arch))
const pool = matched.length > 0 ? matched : found
pool.sort()
return pool[pool.length - 1]
}
}
return null
}
/**
* Find the bundled .NET 8 runtime dir (win-codesign@&lt;ver&gt;/dotnet-runtime-*).
* The ATS dlib is a .NET assembly loaded via Ijwhost.dll, which locates
* hostfxr.dll through DOTNET_ROOT — without it the dlib cannot initialize even
* with a same-arch signtool/dlib pair. Returns an absolute path or null.
*/
export function resolveDotnetRuntimeDir(env = process.env) {
for (const root of cacheRoots(env)) {
if (!fs.existsSync(root)) continue
const found = []
const walk = (p, depth) => {
if (depth > 3) return
let entries
try {
entries = fs.readdirSync(p, { withFileTypes: true })
} catch {
return
}
for (const entry of entries) {
if (!entry.isDirectory()) continue
const full = path.join(p, entry.name)
if (/^dotnet-runtime-/.test(entry.name)) found.push(full)
else walk(full, depth + 1)
}
}
for (const entry of fs.readdirSync(root)) {
walk(path.join(root, entry), 0)
}
if (found.length > 0) {
found.sort()
return found[found.length - 1]
}
}
return null
}
/**
* Sign one chunk of binaries with a single signtool invocation (argv array,
* never a shell string — the joined list can be long and must not interpolate
* through a shell).
*
* @param {string[]} files
* @param {{ signtool: string, dlib: string, metadataPath: string, exec?: typeof execFileSync, env?: NodeJS.ProcessEnv }} opts
*/
export function signChunk(files, opts) {
const exec = opts.exec ?? execFileSync
const execOptions = { stdio: 'inherit' }
if (opts.env) execOptions.env = opts.env
exec(opts.signtool, [
'sign',
'/fd', 'SHA256',
'/td', 'SHA256',
'/tr', 'http://timestamp.acs.microsoft.com',
'/dlib', opts.dlib,
'/dmdf', opts.metadataPath,
...files
], execOptions)
}
/**
* Batch-sign every binary under a tree.
*
* @param {string[]} binaries file list from getBinaries
* @param {{ env?: NodeJS.ProcessEnv, exec?: typeof execFileSync, chunkSize?: number, mkdtemp?: typeof fs.mkdtempSync, signtool?: string, dlib?: string }} [opts]
* @returns {{ signed: number, chunks: number, skipped: boolean }}
* skipped=true when Azure signing is not configured (caller warns).
*/
export function batchSignBinaries(binaries, opts = {}) {
const env = opts.env ?? process.env
if (!azureSigningConfigured(env)) {
return { signed: 0, chunks: 0, skipped: true }
}
if (binaries.length === 0) {
return { signed: 0, chunks: 0, skipped: false }
}
const dlib = opts.dlib ?? resolveTrustedSigningDlib(env)
if (!dlib) {
throw new Error('batch-sign-binaries: azure.codesigning.dlib.dll not found under the electron-builder cache')
}
const signtool = opts.signtool ?? resolveSigntool(env)
if (!signtool) {
throw new Error('batch-sign-binaries: signtool.exe not found under the electron-builder cache (or SIGNTOOL_PATH)')
}
// The ATS dlib is a .NET assembly; Ijwhost.dll finds hostfxr.dll via
// DOTNET_ROOT. Mirror app-builder-lib's WindowsSignAzureManager and point it
// at the bundled runtime so the dlib initializes (a missing runtime reads as
// "no certificates found"). Only merged when a runtime dir exists.
const signEnv = { ...env }
const dotnetRoot = opts.dotnetRoot ?? resolveDotnetRuntimeDir(env)
if (dotnetRoot) signEnv.DOTNET_ROOT = dotnetRoot
const mkdtemp = opts.mkdtemp ?? fs.mkdtempSync
const tmpDir = mkdtemp(path.join(env.TEMP || env.TMP || '.', 'batch-sign-'))
const metadataPath = path.join(tmpDir, 'batch-sign.json')
fs.writeFileSync(metadataPath, JSON.stringify({
Endpoint: env.AZURE_SIGN_ENDPOINT,
CodeSigningAccountName: env.AZURE_SIGN_ACCOUNT,
CertificateProfileName: env.AZURE_SIGN_PROFILE
}))
try {
const batches = chunk(binaries, opts.chunkSize ?? CHUNK_SIZE)
for (const batch of batches) {
signChunk(batch, { signtool, dlib, metadataPath, exec: opts.exec, env: signEnv })
}
return { signed: binaries.length, chunks: batches.length, skipped: false }
} finally {
fs.rmSync(tmpDir, { recursive: true, force: true })
}
}
/**
* afterPack-side entry: batch-sign the packed tree, excluding the product exe
* (it is rcedit-ed and signed per-file after this hook — see the header).
* Callers must have run sanitize-pe-signatures.mjs first.
*
* @param {string} appOutDir
* @param {string} productExePath absolute path of the main product exe
* @param {{ env?: NodeJS.ProcessEnv, exec?: typeof execFileSync }} [opts]
*/
export function batchSignAppTree(appOutDir, productExePath, opts = {}) {
const env = opts.env ?? process.env
if (!azureSigningConfigured(env)) {
console.warn(
'[batch-sign] AZURE_SIGN_* not set — payload binaries will be UNSIGNED ' +
'(package block-map still covers them; release lanes must set the signing env)'
)
return { signed: 0, chunks: 0, skipped: true }
}
const productExe = productExePath ? path.resolve(productExePath) : null
const binaries = getBinaries(appOutDir, {
skip: (file) => (productExe ? path.resolve(file) === productExe : false)
})
if (binaries.length === 0) return { signed: 0, chunks: 0, skipped: false }
const result = batchSignBinaries(binaries, opts)
console.log(
`[batch-sign] signed ${result.signed} payload binaries in ${result.chunks} signtool batch(es)` +
` (product exe excluded — signed per-file after rcedit)`
)
return result
}
// ── electron-builder custom win.sign hook ───────────────────────────────────
//
// Per app-builder-lib's signtoolBaseSignManager, the custom `sign` hook is
// invoked once per signable file (the product exe after rcedit, top-level
// exes, asar.unpacked natives, extraResource exes). The hook's return value is
// ignored by the manager, but per the Task 0 contract it resolves true for
// files the afterPack batch already signed — a no-op — and delegates the two
// artifacts that genuinely need per-file signing to the sign-msix.mjs Azure
// machinery: the .msix/.msixbundle package and the product exe.
const SIGNABLE_PACKAGE_EXTENSIONS = ['.msix', '.msixbundle']
const STORE_ARTIFACT_PREFIX = 'Store-'
/**
* The electron-builder custom win.sign hook.
*
* @param {{ path: string }} configuration
* @param {any} packager
* @param {{ signMsix?: (configuration: any, packager: any) => Promise<void>, azureSignFile?: (file: string, packager: any) => Promise<void> }} [deps]
* @returns {Promise<boolean>} true when this hook handled the file
* (batch-signed: nothing to do) — electron-builder must not re-sign it.
*/
export async function customSign(configuration, packager, deps = {}) {
const file = configuration.path
const base = path.basename(file)
// Store-submission packages are Partner Center's to sign (see sign-msix.mjs).
if (base.startsWith(STORE_ARTIFACT_PREFIX)) return true
const lower = file.toLowerCase()
if (SIGNABLE_PACKAGE_EXTENSIONS.some(ext => lower.endsWith(ext))) {
const { default: signMsix } = await import('./sign-msix.mjs')
await (deps.signMsix ?? signMsix)(configuration, packager)
return true
}
// The product exe was rcedit-ed after the batch ran, so it is signed here,
// after its resources are final, on the same Azure manager sign-msix uses.
const productName = packager?.appInfo?.productFilename
if (productName && base.toLowerCase() === `${productName.toLowerCase()}.exe`) {
const { azureSignFile } = await import('./sign-msix.mjs')
await (deps.azureSignFile ?? azureSignFile)(file, packager)
return true
}
// Everything else the hook is offered was already batch-signed in afterPack.
return true
}
function main() {
const root = process.argv[2]
if (!root) {
console.error('usage: batch-sign-binaries.mjs <dir>')
process.exit(2)
}
const result = batchSignAppTree(root, process.env.HERMES_PRODUCT_EXE || path.join(root, 'Hermes.exe'))
if (result.skipped) process.exit(0)
}
if (isMain(import.meta.url)) {
main()
}
// electron-builder's resolveFunction prefers a named export matching the hook
// name ("sign") and falls back to the module default — provide both so the
// config's `sign: './scripts/batch-sign-binaries.mjs'` binds to customSign.
export { customSign as sign }
export default customSign
@@ -0,0 +1,336 @@
import assert from 'node:assert/strict'
import fs from 'node:fs'
import os from 'node:os'
import path from 'node:path'
import { afterEach, test } from 'vitest'
import {
azureSigningConfigured,
batchSignAppTree,
chunk,
customSign,
getBinaries,
resolveDotnetRuntimeDir,
resolveSigntool,
resolveTrustedSigningDlib,
signingArch
} from './batch-sign-binaries.mjs'
const tmpDirs = []
afterEach(() => {
for (const dir of tmpDirs.splice(0)) {
fs.rmSync(dir, { recursive: true, force: true })
}
})
function tmpTree() {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'batch-sign-test-'))
tmpDirs.push(dir)
return dir
}
test('getBinaries collects .exe and .dll recursively, case-insensitive, sorted', () => {
const root = tmpTree()
fs.mkdirSync(path.join(root, 'resources', 'agent-payload', 'tools', 'python', 'bin'), { recursive: true })
fs.mkdirSync(path.join(root, 'resources', 'agent-payload', 'tools', 'chromium-1', 'meep'), { recursive: true })
fs.writeFileSync(path.join(root, 'Hermes.exe'), 'x')
fs.writeFileSync(path.join(root, 'ffmpeg.dll'), 'x')
fs.writeFileSync(path.join(root, 'resources', 'agent-payload', 'tools', 'python', 'bin', 'node.exe'), 'x')
fs.writeFileSync(path.join(root, 'resources', 'agent-payload', 'tools', 'chromium-1', 'meep', 'chrome.dll'), 'x')
fs.writeFileSync(path.join(root, 'resources', 'README.md'), 'x')
const files = getBinaries(root).map(f => path.relative(root, f))
assert.deepEqual(files, [
path.join('Hermes.exe'),
path.join('ffmpeg.dll'),
path.join('resources', 'agent-payload', 'tools', 'chromium-1', 'meep', 'chrome.dll'),
path.join('resources', 'agent-payload', 'tools', 'python', 'bin', 'node.exe')
])
})
test('getBinaries skips symlinks and honors the skip predicate (product exe)', () => {
const root = tmpTree()
fs.mkdirSync(path.join(root, 'tools'), { recursive: true })
const target = path.join(root, 'tools', 'real.exe')
fs.writeFileSync(target, 'x')
fs.writeFileSync(path.join(root, 'Hermes.exe'), 'x')
const link = path.join(root, 'tools', 'link.exe')
try {
fs.symlinkSync(target, link)
} catch {
// Windows without symlink privilege: the collection contract under test
// is the skip predicate; symlink skipping is covered on the other OS.
}
const exe = path.join(root, 'Hermes.exe')
const files = getBinaries(root, { skip: file => path.resolve(file) === exe })
assert.equal(files.includes(exe), false)
assert.equal(files.includes(target), true)
if (fs.existsSync(link)) {
assert.equal(files.includes(link), false)
}
})
test('getBinaries tolerates a missing directory', () => {
assert.deepEqual(getBinaries(path.join(tmpTree(), 'nope')), [])
})
test('chunk splits into ~100-file batches with no leftovers', () => {
assert.deepEqual(chunk([], 100), [])
const two50 = Array.from({ length: 250 }, (_, i) => `f${i}.exe`)
const batches = chunk(two50)
assert.equal(batches.length, 3)
assert.deepEqual(batches.map(b => b.length), [100, 100, 50])
assert.deepEqual(batches.flat(), two50)
const exact = Array.from({ length: 200 }, (_, i) => `f${i}.exe`)
assert.deepEqual(chunk(exact).map(b => b.length), [100, 100])
})
test('azureSigningConfigured requires endpoint, account, and profile together', () => {
assert.equal(azureSigningConfigured({}), false)
assert.equal(azureSigningConfigured({ AZURE_SIGN_ENDPOINT: 'https://cus.codesigning.azure.net' }), false)
assert.equal(
azureSigningConfigured({
AZURE_SIGN_ENDPOINT: 'https://cus.codesigning.azure.net',
AZURE_SIGN_ACCOUNT: 'codesign2'
}),
false
)
assert.equal(
azureSigningConfigured({
AZURE_SIGN_ENDPOINT: 'https://cus.codesigning.azure.net',
AZURE_SIGN_ACCOUNT: 'codesign2',
AZURE_SIGN_PROFILE: 'hermesagent'
}),
true
)
})
test('customSign returns true for batch-covered payload files without touching Azure', async () => {
let azureTouched = 0
const result = await customSign(
{ path: 'C:/out/win-unpacked/resources/agent-payload/tools/node.exe' },
{ appInfo: { productFilename: 'Hermes' } },
{ azureSignFile: async () => { azureTouched += 1 } }
)
assert.equal(result, true)
assert.equal(azureTouched, 0)
})
test('customSign skips Store- submission packages (Partner Center signs)', async () => {
const result = await customSign(
{ path: 'C:/out/Store-HermesBundled-0.28.0-win-x64.msix' },
{ appInfo: { productFilename: 'Hermes' } },
{ signMsix: async () => { throw new Error('must not be called') } }
)
assert.equal(result, true)
})
test('customSign delegates the msix package and the product exe to the Azure signer', async () => {
const delegated = []
const deps = {
signMsix: async configuration => delegated.push(['msix', configuration.path]),
azureSignFile: async file => delegated.push(['exe', file])
}
const packager = { appInfo: { productFilename: 'Hermes' } }
await customSign({ path: 'C:/out/HermesBundled-0.28.0-win-x64.msix' }, packager, deps)
await customSign({ path: 'C:/out/win-unpacked/Hermes.exe' }, packager, deps)
// The hook's contract is `true` (handled) even when it delegated —
// electron-builder must not fall back to per-file default signing.
assert.deepEqual(delegated, [
['msix', 'C:/out/HermesBundled-0.28.0-win-x64.msix'],
['exe', 'C:/out/win-unpacked/Hermes.exe']
])
assert.equal(await customSign({ path: 'C:/out/win-unpacked/Hermes.exe' }, packager, deps), true)
})
test('customSign does not mistake a similarly-named payload exe for the product exe', async () => {
const deps = {
azureSignFile: async () => { throw new Error('must not be called') }
}
assert.equal(
await customSign(
{ path: 'C:/out/win-unpacked/resources/Hermes-helper.exe' },
{ appInfo: { productFilename: 'Hermes' } },
deps
),
true
)
})
test('batchSignAppTree is a no-op (skipped=true) without the Azure env, and signs via chunked argv-array invocations when set', () => {
const root = tmpTree()
fs.mkdirSync(path.join(root, 'tools'), { recursive: true })
const exe = path.join(root, 'Hermes.exe')
fs.writeFileSync(exe, 'x')
fs.writeFileSync(path.join(root, 'tools', 'node.exe'), 'x')
fs.writeFileSync(path.join(root, 'tools', 'ffmpeg.dll'), 'x')
// Unsigned lane: loud no-op, nothing invoked.
const skipped = batchSignAppTree(root, exe, { env: {} })
assert.deepEqual(skipped, { signed: 0, chunks: 0, skipped: true })
// Signed lane: product exe excluded, chunked execFileSync with argv arrays.
const invocations = []
const fakeExec = (tool, args) => {
invocations.push({ tool, args })
return Buffer.from('')
}
const result = batchSignAppTree(root, exe, {
env: {
AZURE_SIGN_ENDPOINT: 'https://cus.codesigning.azure.net',
AZURE_SIGN_ACCOUNT: 'codesign2',
AZURE_SIGN_PROFILE: 'hermesagent'
},
exec: fakeExec,
mkdtemp: () => root,
signtool: 'signtool.exe',
dlib: 'azure.codesigning.dlib.dll'
})
assert.deepEqual(result, { signed: 2, chunks: 1, skipped: false })
assert.equal(invocations.length, 1)
assert.equal(invocations[0].tool, 'signtool.exe')
assert.ok(Array.isArray(invocations[0].args), 'argv array, never a shell string')
const args = invocations[0].args
assert.ok(!args.some(arg => typeof arg === 'string' && arg.includes(' ')))
assert.equal(args.includes(exe), false, 'product exe excluded — signed per-file after rcedit')
assert.equal(args.includes(path.join(root, 'tools', 'node.exe')), true)
assert.equal(args.includes(path.join(root, 'tools', 'ffmpeg.dll')), true)
assert.equal(args[args.indexOf('/dlib') + 1], 'azure.codesigning.dlib.dll')
assert.ok(args[args.indexOf('/dmdf') + 1].endsWith('batch-sign.json'))
assert.equal(args[args.indexOf('/fd') + 1], 'SHA256')
assert.equal(args[args.indexOf('/td') + 1], 'SHA256')
})
test('batchSignAppTree chunks large trees into ~100-file signtool invocations', () => {
const root = tmpTree()
fs.mkdirSync(path.join(root, 'tools'), { recursive: true })
for (let i = 0; i < 250; i += 1) {
fs.writeFileSync(path.join(root, 'tools', `bin${i}.exe`), 'x')
}
const invocations = []
const result = batchSignAppTree(root, path.join(root, 'Hermes.exe'), {
env: {
AZURE_SIGN_ENDPOINT: 'https://cus.codesigning.azure.net',
AZURE_SIGN_ACCOUNT: 'codesign2',
AZURE_SIGN_PROFILE: 'hermesagent'
},
exec: (tool, args) => {
invocations.push(args)
return Buffer.from('')
},
mkdtemp: () => root,
signtool: 'signtool.exe',
dlib: 'azure.codesigning.dlib.dll'
})
assert.deepEqual(result, { signed: 250, chunks: 3, skipped: false })
assert.equal(invocations.length, 3)
assert.deepEqual(
invocations.map(batch => batch.filter(arg => arg.endsWith('.exe')).length),
[100, 100, 50]
)
const flat = invocations.flat().filter(arg => arg.endsWith('.exe'))
assert.equal(new Set(flat).size, 250, 'every binary signed exactly once')
})
// ── toolset resolution: signtool + dlib must be same-arch ───────────────────
// The ATS bundle ships the dlib for x64/x86 only and a 32-bit signtool cannot
// load a 64-bit dlib — resolveSigntool/resolveTrustedSigningDlib must pair the
// HOST arch, never "whatever sorted last" (which picked x86 signtool + x64
// dlib and broke signing with "No certificates were found"). Regression for
// the win32-x64 release build failure.
function fakeToolsetCache() {
const cache = path.join(tmpTree(), 'electron-builder', 'Cache')
const kits = path.join(cache, 'win-codesign@1.3.0', 'windows-kits-bundle-10_0_26100_0-abc')
for (const arch of ['arm64', 'x64', 'x86']) {
fs.mkdirSync(path.join(kits, arch), { recursive: true })
fs.writeFileSync(path.join(kits, arch, 'signtool.exe'), 'x')
}
const ats = path.join(cache, 'win-codesign@1.3.0', 'ats-bundle-1_0_95-def')
for (const arch of ['x64', 'x86']) {
fs.mkdirSync(path.join(ats, arch), { recursive: true })
fs.writeFileSync(path.join(ats, arch, 'Azure.CodeSigning.Dlib.dll'), 'x')
}
fs.mkdirSync(path.join(cache, 'win-codesign@1.3.0', 'dotnet-runtime-win-x64-8_0_28-ghi'), { recursive: true })
return { cache, kits, ats }
}
test('signingArch matches the host (x64 unless ia32)', () => {
assert.equal(signingArch(), process.arch === 'ia32' ? 'x86' : 'x64')
})
test('resolveSigntool prefers the host-arch kit over x86', () => {
const { cache } = fakeToolsetCache()
const signtool = resolveSigntool({ ELECTRON_BUILDER_CACHE: cache })
const expectedArch = signingArch()
assert.ok(signtool, 'a signtool must resolve from the fake cache')
assert.ok(
signtool.toLowerCase().includes(`\\${expectedArch}\\signtool.exe`) ||
signtool.toLowerCase().includes(`/${expectedArch}/signtool.exe`),
`expected the ${expectedArch} signtool, got ${signtool}`
)
})
test('resolveTrustedSigningDlib prefers the host-arch dlib over x86', () => {
const { cache } = fakeToolsetCache()
const dlib = resolveTrustedSigningDlib({ ELECTRON_BUILDER_CACHE: cache })
const expectedArch = signingArch()
assert.ok(dlib, 'a dlib must resolve from the fake cache')
assert.ok(
dlib.toLowerCase().includes(`\\${expectedArch}\\azure.codesigning.dlib.dll`) ||
dlib.toLowerCase().includes(`/${expectedArch}/azure.codesigning.dlib.dll`),
`expected the ${expectedArch} dlib, got ${dlib}`
)
})
test('resolveDotnetRuntimeDir finds the bundled .NET runtime dir', () => {
const { cache } = fakeToolsetCache()
const dotnet = resolveDotnetRuntimeDir({ ELECTRON_BUILDER_CACHE: cache })
assert.ok(dotnet, 'the dotnet-runtime bundle must resolve')
assert.ok(/dotnet-runtime-/.test(path.basename(dotnet)), `unexpected dotnet dir ${dotnet}`)
})
test('batchSignBinaries sets DOTNET_ROOT from the bundled runtime for the signtool child', () => {
const { cache } = fakeToolsetCache()
const root = tmpTree()
fs.mkdirSync(path.join(root, 'tools'), { recursive: true })
fs.writeFileSync(path.join(root, 'tools', 'node.exe'), 'x')
const exe = path.join(root, 'Hermes.exe')
const invocations = []
const result = batchSignAppTree(root, exe, {
env: {
ELECTRON_BUILDER_CACHE: cache,
AZURE_SIGN_ENDPOINT: 'https://cus.codesigning.azure.net',
AZURE_SIGN_ACCOUNT: 'codesign2',
AZURE_SIGN_PROFILE: 'hermesagent'
},
exec: (tool, args, opts) => {
invocations.push({ tool, args, opts })
return Buffer.from('')
},
mkdtemp: () => root
})
assert.equal(result.signed, 1)
assert.equal(invocations.length, 1)
const opts = invocations[0].opts
assert.ok(opts && opts.env, 'exec must receive a child env')
assert.equal(opts.env.DOTNET_ROOT, path.join(cache, 'win-codesign@1.3.0', 'dotnet-runtime-win-x64-8_0_28-ghi'))
// The signtool chosen must be the host-arch one, not x86.
assert.match(invocations[0].tool, new RegExp(`[\\\\/]${signingArch()}[\\\\/]signtool\\.exe$`))
// dlib must be same arch.
const dlibArg = invocations[0].args[invocations[0].args.indexOf('/dlib') + 1]
assert.match(dlibArg, new RegExp(`[\\\\/]${signingArch()}[\\\\/]azure\\.codesigning\\.dlib\\.dll$`, 'i'))
})
+100
View File
@@ -6,6 +6,106 @@
* longer bundled; the Electron app fetches it at first launch via
* `install.ps1`'s stage protocol (Windows). See `electron/main.ts`.
*/
import fs from 'node:fs'
import path from 'node:path'
import { createRequire } from 'node:module'
import { CLI_LAUNCHER_SPECS } from '../../../scripts/desktop-cli/cli-entrypoints.mjs'
const require = createRequire(import.meta.url)
const {
light,
displayName,
appNamePascal
} = require('../product-identity.cjs')
export default async function beforeBuild() {
return false
}
function stageMsixAssets() {
const desktop = path.join(import.meta.dirname, '..')
const sourceDir = path.join(desktop, 'assets', 'appx')
const stageDir = path.join(desktop, 'build', 'appx')
const names = [
'Square44x44Logo.png',
'Square150x150Logo.png',
'StoreLogo.png',
'Wide310x150Logo.png'
]
fs.mkdirSync(stageDir, { recursive: true })
for (const name of names) {
const source = path.join(sourceDir, name)
if (!fs.existsSync(source)) {
throw new Error(`missing MSIX asset ${source}`)
}
fs.copyFileSync(source, path.join(stageDir, name))
}
}
function writeMsixExtensions() {
const desktop = path.join(import.meta.dirname, '..')
const output = path.join('build', 'msix-extensions.xml')
const file = path.join(desktop, output)
// One uap5:Extension per payload CLI launcher exe (see
// scripts/desktop-cli/cli-entrypoints.mjs): the minted exes are three
// REAL executables now — the old rust shim's one-exe-argv[0]-dispatch is
// gone — and an AppExecutionAlias's Executable must name the exact exe
// that serves the alias, so each alias gets its own Extension block.
const aliases = light ? '' : appExecutionAliasExtensions()
// The uap3:AppExtension fragment that registers the app as a Windows
// Copilot hardware key provider. The press activates hermes://copilot-key/start.
//
// Content rules (violations are an opaque makeappx 0x80080204):
// * xmlns:uap3 rides on the fragment root — the stock manifest template
// declares no uap3 prefix. A/B-verified fine.
// * children of uap3:Properties are UNPREFIXED (xs:any content, per
// Microsoft's copilot-key-state sample).
const copilot = light
? ''
: `<uap3:Extension
xmlns:uap3="http://schemas.microsoft.com/appx/manifest/uap/windows10/3"
Category="windows.appExtension">
<uap3:AppExtension
Name="com.microsoft.windows.copilotkeyprovider"
Id="${appNamePascal}CopilotKeyProvider"
DisplayName="${displayName}"
Description="Launch ${displayName} with the Copilot key"
PublicFolder="Public">
<uap3:Properties>
<SingleTap>hermes://copilot-key/start?state=Tap</SingleTap>
<PressAndHoldStart>hermes://copilot-key/start?state=Down</PressAndHoldStart>
<PressAndHoldStop>hermes://copilot-key/stop?state=Up</PressAndHoldStop>
</uap3:Properties>
</uap3:AppExtension>
</uap3:Extension>
${aliases}`
fs.mkdirSync(path.dirname(file), { recursive: true })
fs.writeFileSync(file, copilot)
}
/**
* One uap5:Extension block per payload CLI launcher, each naming its own
* Executable (the distlib-minted launcher exes under bin/) and the alias
* that exe serves. Exported pure for tests.
* @param {{ name: string }[]} [launchers] exe stems under bin/
*/
export function appExecutionAliasExtensions(launchers = CLI_LAUNCHER_SPECS.map((s) => s.name)) {
const bs = String.fromCharCode(92)
const executable = (name) => ['app', 'resources', 'agent-payload', 'bin', `${name}.exe`].join(bs)
return launchers
.map(
(name) => `<uap5:Extension
xmlns:uap5="http://schemas.microsoft.com/appx/manifest/uap/windows10/5"
Category="windows.appExecutionAlias"
Executable="${executable(name)}"
EntryPoint="Windows.FullTrustApplication">
<uap5:AppExecutionAlias>
<uap5:ExecutionAlias Alias="${name}.exe" />
</uap5:AppExecutionAlias>
</uap5:Extension>`
)
.join('\n')
}
+115
View File
@@ -0,0 +1,115 @@
import assert from 'node:assert/strict'
import path from 'node:path'
import { fileURLToPath } from 'node:url'
import { test } from 'vitest'
import { CLI_LAUNCHER_SPECS, posixTrampolineScripts, renderWinWrapper } from '../../../scripts/desktop-cli/cli-entrypoints.mjs'
import { appExecutionAliasExtensions } from './before-build.mjs'
const scriptDir = path.dirname(fileURLToPath(import.meta.url))
// The payload layout facts every generator receives (bin-relative, forward
// slashes — the same composition build-bundled-desktop.mjs step 5b feeds in).
const RELS = {
relPython: '../tools/cpython-3.11.16+20260814-win32-x64/bin/python3',
relSite: '../venv/lib/python3.11/site-packages',
relRepo: '../hermes-agent'
}
test('three launcher specs mirror [project.scripts] in pyproject.toml', () => {
assert.deepEqual(CLI_LAUNCHER_SPECS.map((s) => s.name), ['hermes', 'hermes-agent', 'hermes-acp'])
assert.deepEqual(CLI_LAUNCHER_SPECS.map((s) => `${s.module}:${s.func}`), [
'hermes_cli.main:main',
'run_agent:main',
'acp_adapter.entry:main'
])
})
const scripts = posixTrampolineScripts(RELS)
test('one POSIX trampoline per entry, named plainly (no suffix)', () => {
assert.deepEqual(scripts.map((s) => s.name), ['hermes', 'hermes-agent', 'hermes-acp'])
})
test('trampoline is $0-relative: shebang, symlink-chain resolution, own-dir cd', () => {
const text = scripts[0].text
assert.ok(text.startsWith('#!/usr/bin/env bash\n'))
// The whole relocatability contract: nothing in the script may be an
// absolute path or a Windows path — everything resolves off $0.
for (const line of text.split('\n')) {
const code = line.replace(/^#/, '').trim()
assert.ok(!/[A-Za-z]:\\/.test(code), `windows path in generated script: ${line}`)
assert.ok(!/=\s*\/(?!usr\/bin\/env)/.test(code), `absolute path in generated script: ${line}`)
}
assert.match(text, /self="\$0"/)
assert.match(text, /while \[ -L "\$self" \]/)
assert.match(text, /BIN_DIR="\$\(cd -- "\$\(dirname -- "\$self"\)" && pwd\)"/)
assert.match(text, /PYTHON="\$BIN_DIR"\/\.\.\/tools\//)
})
test('trampoline composes the payload import roots (repo first) and replaces inherited PYTHONPATH', () => {
const text = scripts[0].text
assert.match(text, /unset PYTHONPATH PYTHONHOME/)
assert.match(text, /export PYTHONPATH="\$REPO:\$SITE"/)
assert.match(text, /REPO="\$BIN_DIR"\/\.\.\/hermes-agent/)
assert.match(text, /SITE="\$BIN_DIR"\/\.\.\/venv\/lib\/python3\.11\/site-packages/)
})
test('trampoline keeps pycache out of the payload and execs the entry module', () => {
const text = scripts[0].text
assert.match(text, /PYTHONPYCACHEPREFIX="\$HOME\/\.cache\/hermes-pycache"/)
assert.match(text, /if \[ -z "\$\{PYTHONPYCACHEPREFIX:-\}" \]/, 'user-set prefix must win')
assert.match(text, /exec "\$PYTHON" -m hermes_cli\.main "\$@"/)
})
test('trampoline fails loudly (exit 2) when the bundled interpreter is missing', () => {
const text = scripts[0].text
assert.match(text, /bundled interpreter missing at \$PYTHON/)
assert.match(text, /exit 2/)
})
test('each trampoline execs its own entry module', () => {
const modules = scripts.map((s) => /exec "\$PYTHON" -m (\S+) "\$@"/.exec(s.text)?.[1])
assert.deepEqual(modules, ['hermes_cli.main', 'run_agent', 'acp_adapter.entry'])
})
test('win wrapper substitution bakes the entry module and payload layout facts', () => {
const text = renderWinWrapper(CLI_LAUNCHER_SPECS[0], RELS.relRepo, RELS.relSite)
assert.ok(!text.includes('__HERMES_'), 'placeholders must be fully substituted')
assert.match(text, /HERMES_ENTRY_MODULE = "hermes_cli\.main"/)
assert.match(text, /HERMES_ENTRY_FUNC = "main"/)
assert.match(text, /HERMES_REPO_REL = "\.\.\/hermes-agent"/)
assert.match(text, /HERMES_SITE_REL = "\.\.\/venv\/lib\/python3\.11\/site-packages"/)
})
test('win wrapper rejects values that still contain placeholders', () => {
assert.throws(() => renderWinWrapper({ module: '__HERMES_REPO_REL__', func: 'main' }, RELS.relRepo, RELS.relSite))
})
test('MSIX: one uap5 alias Extension per launcher exe, Executable naming that exe', () => {
const xml = appExecutionAliasExtensions()
const blocks = xml.match(/<uap5:Extension\b/g) ?? []
assert.equal(blocks.length, 3)
for (const spec of CLI_LAUNCHER_SPECS) {
const expected = ['app', 'resources', 'agent-payload', 'bin', `${spec.name}.exe`].join('\\')
assert.ok(
xml.includes(`Executable="${expected}"`),
`no alias Extension names ${expected}`
)
// Backslashes: MSIX Executable must be manifest-backslash form.
assert.ok(!xml.includes(`${expected}/`), 'forward slashes are not manifest form')
}
// The old rust shim's single-exe-argv[0]-dispatch shape is gone: each
// block serves exactly ONE alias.
for (const block of (xml.match(/<uap5:Extension[\s\S]*?<\/uap5:Extension>/g) ?? [])) {
assert.equal((block.match(/<uap5:ExecutionAlias /g) ?? []).length, 1)
}
})
test('light variant emits no alias fragments', () => {
// The light gating lives in writeMsixExtensions (light ? '' : ...); the
// pure builder must stay gating-free, so just pin its shape here.
assert.ok(appExecutionAliasExtensions(['hermes']).includes('windows.appExecutionAlias'))
assert.ok(scriptDir.length > 0)
})
+115
View File
@@ -0,0 +1,115 @@
// Generate the AppxManifest.xml that MsixTarget.writeManifest produces on
// the win32 CI lane, using the REAL app-builder-lib helpers and the real
// builder config, so the manifest can be inspected off-Windows — makeappx
// reports every manifest problem as a bare 0x80080204, and this is the
// XML it is rejecting.
//
// Usage (from apps/desktop):
// node scripts/gen-msix-manifest.mjs [bundled|light] [x64|arm64]
//
// The output is the substituted manifest on stdout. Imports reach into
// app-builder-lib's dist because the exports map hides the internals; the
// paths are the same ones MsixTarget itself uses, so a bump that moves
// them fails here loudly.
import { readFile, readdir } from "node:fs/promises"
import { createRequire } from "node:module"
import path from "node:path"
import { fileURLToPath } from "node:url"
import {
buildCapabilitiesXml,
buildExtensionsXml,
defaultTileTag,
lockScreenTag,
resolvePackageApplicationId,
resolvePackageIdentityName,
resourceLanguageTag,
splashScreenTag,
substituteManifestMacros,
} from "../../../node_modules/app-builder-lib/dist/targets/win/winAppUtil.js"
import { appIdentity } from "../../../scripts/msix-shared.mjs"
const desktop = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..")
const repoRoot = path.resolve(desktop, "..", "..")
const variant = process.argv[2] || "bundled"
const arch = process.argv[3] || "x64"
if (!["bundled", "light", "store"].includes(variant)) {
console.error(`variant must be 'bundled', 'light', or 'store', got '${variant}'`)
process.exit(1)
}
process.env.HERMES_DESKTOP_VARIANT = variant
const require = createRequire(import.meta.url)
const config = require(path.join(desktop, "electron-builder.config.cjs"))
const pkg = require(path.join(desktop, "package.json"))
const options = config.msix
// The payload CLI launcher (hermes.exe, the minted distlib launcher) is the
// MSIX entry point; the alias and AppExtension fragments in the config
// reference the same path. The launcher is self-relative and sets the
// payload's own PYTHONPATH.
const executable = `app\\resources\\agent-payload\\bin\\hermes.exe`
const displayName = options.displayName || config.productName
// appInfo.name honours extraMetadata.name the way packager merging does.
const appInfoName = config.extraMetadata?.name || pkg.name
// Same asset discovery as computeUserAssets: requiring the config staged
// assets/appx/ into build/appx/ (the buildResources "appx" dir MsixTarget
// reads), so listing it here sees exactly what the win32 lane packs.
const userAssets = (await readdir(path.join(desktop, "build", "appx")))
.filter((it) => !it.startsWith(".") && !it.endsWith(".db") && it.includes("."))
const extensions = await buildExtensionsXml({
protocols: config.protocols,
fileAssociations: [],
addAutoLaunchExtension: options.addAutoLaunchExtension,
customExtensionsPath: options.customExtensionsPath,
appDir: desktop,
executable,
displayName,
dependencyNames: {},
})
// Same precedence as MsixTarget.writeManifest: the config's custom
// template when set (resolved relative to the app dir, as getResource
// does), else the stock template.
const template = await readFile(
options.customManifestPath
? path.resolve(desktop, options.customManifestPath)
: path.join(repoRoot, "node_modules", "app-builder-lib", "templates", "msix", "appxmanifest.xml"),
"utf8"
)
const manifest = substituteManifestMacros(template, (m) => {
switch (m) {
case "publisher": return options.publisher
case "publisherDisplayName": return options.publisherDisplayName
// Same 4-part derivation the real build uses (msix-shared::appIdentity) —
// a nightly shows its minutes-since-stable component here too, so this
// inspection tool never disagrees with what electron-builder shipped.
case "version": return appIdentity(desktop, process.env.HERMES_PAYLOAD_TAG).version
case "applicationId": return resolvePackageApplicationId(options.applicationId, options.identityName, appInfoName, "MSIX")
case "identityName": return resolvePackageIdentityName(options.identityName, appInfoName, "MSIX")
case "executable": return executable
case "displayName": return displayName
case "description": return pkg.description || config.productName
case "backgroundColor": return options.backgroundColor || "#464646"
case "logo": return "assets\\StoreLogo.png"
case "square150x150Logo": return "assets\\Square150x150Logo.png"
case "square44x44Logo": return "assets\\Square44x44Logo.png"
case "lockScreen": return lockScreenTag(userAssets)
case "defaultTile": return defaultTileTag(userAssets, options.showNameOnTiles || false)
case "splashScreen": return splashScreenTag(userAssets)
case "arch": return arch
case "resourceLanguages": return resourceLanguageTag(options.languages)
case "capabilities": return `<Capabilities>\n${buildCapabilitiesXml(options.capabilities)}\n</Capabilities>`
case "extensions": return extensions
case "minVersion": return options.minVersion || "10.0.17763.0"
case "maxVersionTested": return options.maxVersionTested || options.minVersion || "10.0.17763.0"
case "packageIntegrity": return ""
default: throw new Error(`Macro ${m} is not defined`)
}
})
console.log(manifest)
@@ -0,0 +1,116 @@
// MSIX BUILD_NUMBER round-trip (plan item 3.2): scripts/msix-shared.mjs's
// appIdentity() derivation must equal the Version= the built manifest ships.
//
// The "built manifest" here is the REAL one the win32 lane packs: the repo's
// custom template (assets/msix-manifest.xml) run through app-builder-lib's
// substituteManifestMacros — the same helper MsixTarget.writeManifest uses —
// with the version macro fed by appIdentity(), exactly as
// scripts/gen-msix-manifest.mjs (the offline inspection twin of the build)
// does. The test then reads the Version= back out of the XML and compares.
//
// The git-backed stable lookup is deterministic here because
// node:child_process.execFileSync is mocked; the math it feeds is the
// contract App Installer and makeappx compare.
import assert from 'node:assert/strict'
import fs from 'node:fs'
import os from 'node:os'
import path from 'node:path'
import { fileURLToPath } from 'node:url'
import { createRequire } from 'node:module'
import { beforeEach, test, vi } from 'vitest'
vi.mock('node:child_process', () => ({
execFileSync: vi.fn(),
}))
const { execFileSync } = await import('node:child_process')
const msix = await import('../../../scripts/msix-shared.mjs')
const require = createRequire(import.meta.url)
// The real helpers + template the build itself uses.
const { substituteManifestMacros } = require('../../../node_modules/app-builder-lib/dist/targets/win/winAppUtil.js')
const scriptsDir = path.dirname(fileURLToPath(import.meta.url))
const desktopDir = path.resolve(scriptsDir, '..')
const template = fs.readFileSync(path.join(desktopDir, 'assets', 'msix-manifest.xml'), 'utf8')
// A fake app dir with just enough for appIdentity (same shape as
// msix-shared.test.mjs's fixture).
function makeFakeDesktop(version) {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'msix-roundtrip-'))
fs.writeFileSync(
path.join(dir, 'product-identity.cjs'),
"module.exports = { store: false, light: false, displayName: 'Hermes', appId: 'com.nousresearch.hermes-bundled', channel: 'latest', appNamePascal: 'HermesBundled', msixAppIdWithOrg: 'NousResearch.HermesBundled' }\n"
)
fs.writeFileSync(path.join(dir, 'package.json'), JSON.stringify({ name: 'hermes-desktop', version }))
return dir
}
function gitMock(tags, stableEpoch) {
execFileSync.mockImplementation((cmd, args) => {
if (args[0] === 'tag') return `${tags.join('\n')}\n`
if (args[0] === 'log') return `${stableEpoch}\n`
throw new Error(`unexpected git call ${cmd} ${args.join(' ')}`)
})
}
// Substitute the template the way the build does: the version macro comes
// from appIdentity(); every other macro gets a deterministic placeholder —
// they are identity/packaging strings, not the contract under test here.
function buildManifest(appDir, tag) {
const { version } = msix.appIdentity(appDir, tag)
return { version, xml: substituteManifestMacros(template, (m) => (m === 'version' ? version : `test-${m}`)) }
}
function identityVersion(xml) {
const identity = /<Identity\b[^>]*>/.exec(xml)
assert.ok(identity, 'substituted manifest has no Identity element')
const version = /Version="([^"]*)"/.exec(identity[0])
assert.ok(version, 'Identity element carries no Version attribute')
return version[1]
}
beforeEach(() => {
execFileSync.mockReset()
})
test('stable tag: appIdentity derivation round-trips into the manifest Version', () => {
const app = makeFakeDesktop('0.27.1')
const { version, xml } = buildManifest(app, 'v0.27.1')
assert.equal(version, '0.27.1.0')
assert.equal(identityVersion(xml), version)
})
test('nightly tag: minutes-since-stable build number round-trips into the manifest Version', () => {
const app = makeFakeDesktop('0.27.1')
// Stable v0.27.1 committed 2026-08-01T00:00:00Z; nightly cut 2026-08-29T01:02:03Z.
const stableEpoch = Math.floor(Date.UTC(2026, 7, 1) / 1000)
gitMock(['v0.27.1', 'v0.27.2-nightly.20260829010203'], stableEpoch)
const { version, xml } = buildManifest(app, 'v0.27.2-nightly.20260829010203')
const expectedMinutes = Math.floor((Date.UTC(2026, 7, 29, 1, 2, 3) - Date.UTC(2026, 7, 1)) / 60000)
assert.equal(version, `0.27.2.${expectedMinutes}`)
assert.equal(identityVersion(xml), version)
})
test('manifest Version components are 16-bit (makeappx rejects anything larger)', () => {
const app = makeFakeDesktop('0.27.1')
const stableEpoch = Math.floor(Date.UTC(2026, 7, 1) / 1000)
gitMock(['v0.27.1', 'v0.27.2-nightly.20260829010203'], stableEpoch)
const { xml } = buildManifest(app, 'v0.27.2-nightly.20260829010203')
for (const part of identityVersion(xml).split('.')) {
const n = Number(part)
assert.ok(Number.isInteger(n) && n >= 0 && n <= 65535, `component ${part} outside 16 bits`)
}
})
test('a later nightly stamps a strictly larger BUILD_NUMBER than an earlier one', () => {
// The build number exists so Windows can order nightlies on the same
// base version; monotonicity across the stamp is the actual contract.
const app = makeFakeDesktop('0.27.1')
const stableEpoch = Math.floor(Date.UTC(2026, 7, 1) / 1000)
const early = 'v0.27.2-nightly.20260815080000'
const late = 'v0.27.2-nightly.20260829010203'
gitMock([early, late, 'v0.27.1'], stableEpoch)
const earlyBuild = Number(buildManifest(app, early).version.split('.')[3])
const lateBuild = Number(buildManifest(app, late).version.split('.')[3])
assert.ok(lateBuild > earlyBuild, `${lateBuild} must exceed ${earlyBuild}`)
})
+151
View File
@@ -0,0 +1,151 @@
// electron-builder custom win.sign hook: Azure Trusted Signing for the MSIX
// package only. Windows install validation checks the package signature
// (AppxSignature.p7x over AppxBlockMap.xml); inner files are covered by the
// block-map hashes, NOT per-file Authenticode, so Hermes.exe and every
// payload binary stay unsigned and this hook signs exactly one artifact per
// build.
//
// Wired from electron-builder.config.cjs as
// win.sign = { type: 'signtool', sign: './scripts/sign-msix.mjs', ... }
// The Azure endpoint/account/profile do NOT ride through the config: this
// module reads the AZURE_SIGN_* environment variables directly
// (azureConfigFromEnv) and hands them to app-builder-lib's own
// WindowsSignAzureManager, so the actual signing path is byte-for-byte the
// one a plain { type: 'azure' } config would run.
//
// The Store-submission variant (artifactName "Store-" prefixed) is the one
// exception: its manifest Publisher is the Partner Center publisher ID
// (CN=EE6D86E4-...), which no signable cert subject can match — CA/B CSBR
// requires the legal entity's validated name and Artifact Signing cannot
// customize CN — so SignerSign rejects the package with ERROR_BAD_FORMAT
// (0x8007000B). Partner Center re-signs the package with the Microsoft
// Store certificate on ingestion, so the Store- msix ships unsigned.
import fs from 'node:fs'
import { createRequire } from 'node:module'
import path from 'node:path'
import { pathToFileURL } from 'node:url'
const require = createRequire(import.meta.url)
// Artifacts that carry the only signature Windows validates for a packaged
// app. A .msixbundle envelope is included for the single-invocation bundle
// case (the envelope signature covers the inner .msix packages).
const SIGNED_ARTIFACT_EXTENSIONS = ['.msix', '.msixbundle']
// Store-submission artifacts are not ATS-signable (publisher mismatch, see
// the header) and don't need a signature (Partner Center signs on
// ingestion). Anything with this basename prefix is left unsigned.
const STORE_ARTIFACT_PREFIX = 'Store-'
/**
* Whether `file` is a signing target. Everything except the MSIX package is
* covered by the package's block-map hashes and must stay untouched —
* signing it after packaging would break the hash. The Store-submission
* variant is excluded too: it cannot carry an ATS signature and does not
* need one.
*/
export function shouldSignFile(file) {
if (path.basename(file).startsWith(STORE_ARTIFACT_PREFIX)) return false
const lower = file.toLowerCase()
return SIGNED_ARTIFACT_EXTENSIONS.some(ext => lower.endsWith(ext))
}
/**
* @param {NodeJS.ProcessEnv} [env]
* @returns {{ type: 'azure' } & Record<string, string | undefined>} the
* win.sign azure configuration, composed from the environment.
*/
export function azureConfigFromEnv(env = process.env) {
return {
type: 'azure',
endpoint: env.AZURE_SIGN_ENDPOINT,
codeSigningAccountName: env.AZURE_SIGN_ACCOUNT,
certificateProfileName: env.AZURE_SIGN_PROFILE,
publisherName: env.AZURE_SIGN_PUBLISHER
}
}
// app-builder-lib's exports map exposes only "." and "./internal";
// import('app-builder-lib/dist/...') throws ERR_PACKAGE_PATH_NOT_EXPORTED.
// Resolve the class the way run-electron-builder.mjs finds the CLI: resolve
// the entry module, walk up to the package root, then direct-file import
// (file URLs bypass the exports map). The constructibility of this class is
// pinned by the tripwire test in sign-msix.test.mjs, so a builder bump
// that moves it fails js-tests instead of a release build.
async function loadAzureManagerClass() {
const entry = require.resolve('app-builder-lib')
let root = path.dirname(entry)
while (!fs.existsSync(path.join(root, 'package.json'))) {
const parent = path.dirname(root)
if (parent === root) throw new Error('app-builder-lib package root not found')
root = parent
}
const mod = await import(
pathToFileURL(path.join(root, 'dist', 'codeSign', 'win', 'windowsSignAzureManager.js')).href
)
return mod.WindowsSignAzureManager
}
// One manager per process: electron-builder calls the hook once per file,
// and the manager memoizes toolset downloads / dlib metadata behind it.
let managerPromise = null
function azureManager(packager) {
if (managerPromise == null) {
managerPromise = (async () => {
const WindowsSignAzureManager = await loadAzureManagerClass()
// The manager's constructor re-derives the signing config from
// packager.platformOptions.sign and throws unless type === 'azure' —
// but our config's win.sign is the { type: 'signtool' } hook wiring.
// Shim the packager with the azure config composed from the
// environment; everything else (config.toolsets, buildResourcesDir,
// getTempFile) delegates to the real packager via the prototype.
const shim = Object.create(packager)
Object.defineProperty(shim, 'platformOptions', {
value: { ...packager.platformOptions, sign: azureConfigFromEnv() }
})
const manager = new WindowsSignAzureManager(shim)
// No-op on the modern signtool /dlib path (winCodeSign >= 1.3.0);
// installs the legacy PowerShell module otherwise.
await manager.initialize()
return manager
})()
}
return managerPromise
}
/**
* Sign one file on app-builder-lib's own WindowsSignAzureManager — the exact
* path a plain { type: 'azure' } config runs. Used by the sign-msix hook for
* the package itself and by batch-sign-binaries.mjs's customSign hook for the
* product exe (which must be signed after rcedit, not in the afterPack batch).
*
* @param {string} file
* @param {any} packager WinPackager
*/
export async function azureSignFile(file, packager) {
const mgr = await azureManager(packager)
// signFileWithDlib reads only options.path (plus the manager's own
// signing config), so platformOptions is sufficient here.
await mgr.signFile({ path: file, options: packager.platformOptions })
}
/**
* The electron-builder custom sign hook.
*
* @param {{ path: string }} configuration CustomWindowsSignTaskConfiguration
* @param {any} packager WinPackager
*/
export default async function sign(configuration, packager) {
if (path.basename(configuration.path).startsWith(STORE_ARTIFACT_PREFIX)) {
console.log(`[sign-msix] skip Store- submission package (Partner Center signs on ingestion): ${configuration.path}`)
return
}
if (!shouldSignFile(configuration.path)) {
console.log(`[sign-msix] skip non-MSIX (covered by package block map): ${configuration.path}`)
return
}
await azureSignFile(configuration.path, packager)
}
+47
View File
@@ -0,0 +1,47 @@
/**
* Stage the pm payload into build/agent-payload for a BUNDLED desktop
* build. Runs `hermes pm bundle` with the repo's own venv interpreter —
* the payload carries the repo snapshot (committed state), the tool
* store + facts, and a relocatable venv on the pinned interpreter.
*
* Plain `npm run dist` (no payload) still works: before-build.mjs writes
* an external:true stub and the app resolves a runtime at first launch.
*/
import { execFileSync } from 'node:child_process'
import fs from 'node:fs'
import path from 'node:path'
const desktopRoot = path.join(import.meta.dirname, '..')
const repoRoot = path.join(desktopRoot, '..', '..')
const payloadDir = path.join(desktopRoot, 'build', 'agent-payload')
function venvPython() {
for (const venv of ['.venv', 'venv']) {
for (const rel of [
['Scripts', 'python.exe'],
['bin', 'python']
]) {
const candidate = path.join(repoRoot, venv, ...rel)
if (fs.existsSync(candidate)) {
return candidate
}
}
}
return null
}
const python = venvPython()
if (!python) {
console.error('stage-payload: no repo venv found — run `uv sync` in the repo root first')
process.exit(1)
}
console.log(`stage-payload: ${python} -m pm.cli bundle --out ${payloadDir}`)
execFileSync(python, ['-m', 'pm.cli', 'bundle', '--out', payloadDir], {
cwd: repoRoot,
stdio: 'inherit',
env: { ...process.env, PYTHONUTF8: '1' }
})
@@ -179,7 +179,11 @@ export function BootFailureOverlay() {
await desktop?.oauthLogoutConnectionConfig?.(remoteReauth.url)
let result: { connected?: boolean } | undefined
// `error` matters here: the oauth arm (oauthLoginConnectionConfig) and
// the cloud cascade can both fail with a reason, and the incomplete
// sign-in notice below surfaces it. (DesktopCloudAgentSignInResult has
// no error field; oauthLoginConnectionConfig does.)
let result: { connected?: boolean; error?: string } | undefined
if (connectionConfig?.mode === 'cloud' && desktop?.cloud) {
const status = await desktop.cloud.status()
-15
View File
@@ -1,15 +0,0 @@
# Termux / Android dependency constraints for Hermes Agent.
#
# Usage:
# python -m pip install -e '.[termux]' -c constraints-termux.txt
#
# These pins keep the tested Android install path stable when upstream packages
# move faster than Termux-compatible wheels / sdists.
ipython<10
jedi>=0.18.1,<0.20
parso>=0.8.4,<0.9
stack-data>=0.6,<0.7
pexpect>4.3,<5
matplotlib-inline>=0.1.7,<0.2
asttokens>=2.1,<3
+88 -41
View File
@@ -4037,20 +4037,61 @@ def _get_session_db_timeout() -> float:
return 10.0
def _read_windows_pyvenv_cfg(venv_dir: Path) -> dict[str, str]:
cfg_path = venv_dir / "pyvenv.cfg"
try:
lines = cfg_path.read_text(encoding="utf-8-sig").splitlines()
except OSError:
return {}
def _pm_runtime_venv_dir() -> Optional[Path]:
"""The venv pm provisioned for this install, resolved from pm's own
records (facts.json + store layout) — never from interpreter state.
parsed: dict[str, str] = {}
for raw in lines:
if "=" not in raw:
continue
key, value = raw.split("=", 1)
parsed[key.strip().lower()] = value.strip()
return parsed
Under no-boot-through-venv the scheduler process is the store python
(``sys.prefix == sys.base_prefix``) and ``VIRTUAL_ENV`` is unset in
bundled installs, so prefix/env probing has nothing to offer. pm is
the authority: a bundled install keeps its relocatable venv beside the
manifest (a sibling of the store), a dev install syncs the project
venv (``venv``/``.venv`` — the same layout
``pm.packages.Venv.venv_dir`` materializes). The venv fact must exist:
pm only vouches for what it provisioned.
"""
try:
from pm import paths
from pm.lock import Facts
except Exception:
return None
try:
if not Facts(paths.facts_path()).get("venv"):
return None
except Exception:
return None
store = paths.store_root()
bundled = store.parent / "venv"
if (store.parent / "manifest.json").is_file():
return bundled if bundled.is_dir() else None
try:
from hermes_constants import project_venv_dir
except ImportError:
return None
return project_venv_dir(paths.repo_root())
def _pm_store_python_exe() -> Optional[Path]:
"""The base interpreter pm staged for this install (the ``python``
fact's store entry), or None. This is the store python — never a venv
launcher — so it is the right interpreter to run cron scripts with the
venv overlaid via environment (see
``_windows_cron_python_invocation``)."""
try:
from pm import paths
from pm.lock import Facts
except Exception:
return None
try:
fact = Facts(paths.facts_path()).get("python")
if not fact or "entry" not in fact:
return None
exe = paths.store_root() / fact["entry"] / (
"python.exe" if sys.platform == "win32" else "bin/python3"
)
return exe if exe.is_file() else None
except Exception:
return None
def _windows_cron_python_invocation(python_exe: str) -> tuple[str, dict[str, str]]:
@@ -4059,15 +4100,16 @@ def _windows_cron_python_invocation(python_exe: str) -> tuple[str, dict[str, str
Cron scripts capture stdout/stderr, so using ``pythonw.exe`` directly can
lose script output. uv-created venv ``python.exe`` launchers are also a
problem: even with CREATE_NO_WINDOW, the launcher can re-exec the base
console interpreter and flash a visible window. For uv venvs, bypass the
launcher and run the base ``python.exe`` directly with the venv paths
overlaid in the environment.
console interpreter and flash a visible window. When pm has provisioned
a runtime venv (facts/store resolution — never pyvenv.cfg, which is
dead config under no-boot-through-venv), bypass the launcher and run
the pm store python directly with the venv paths overlaid in the
environment.
"""
if sys.platform != "win32":
return python_exe, {}
interpreter = Path(python_exe)
venv_dir = interpreter.parent.parent
env_overlay: dict[str, str] = {}
if interpreter.name.lower() == "pythonw.exe":
@@ -4075,22 +4117,20 @@ def _windows_cron_python_invocation(python_exe: str) -> tuple[str, dict[str, str
if sibling.exists():
interpreter = sibling
cfg = _read_windows_pyvenv_cfg(venv_dir)
home = cfg.get("home", "")
site_packages = venv_dir / "Lib" / "site-packages"
if "uv" in cfg and home:
base_python = Path(home) / "python.exe"
if base_python.exists() and site_packages.exists():
interpreter = base_python
env_overlay["VIRTUAL_ENV"] = str(venv_dir)
pythonpath_entries = [
str(Path(__file__).resolve().parents[1]),
str(site_packages),
]
existing_pythonpath = os.environ.get("PYTHONPATH", "")
if existing_pythonpath:
pythonpath_entries.append(existing_pythonpath)
env_overlay["PYTHONPATH"] = os.pathsep.join(pythonpath_entries)
venv_dir = _pm_runtime_venv_dir()
base_python = _pm_store_python_exe()
site_packages = venv_dir / "Lib" / "site-packages" if venv_dir else None
if base_python is not None and site_packages is not None and site_packages.exists():
interpreter = base_python
env_overlay["VIRTUAL_ENV"] = str(venv_dir)
pythonpath_entries = [
str(Path(__file__).resolve().parents[1]),
str(site_packages),
]
existing_pythonpath = os.environ.get("PYTHONPATH", "")
if existing_pythonpath:
pythonpath_entries.append(existing_pythonpath)
env_overlay["PYTHONPATH"] = os.pathsep.join(pythonpath_entries)
return str(interpreter), env_overlay
@@ -4248,8 +4288,10 @@ def _windows_cron_bootstrap_argv(
the pre-existing PYTHONPATH behaviour is strictly better than failing
to run at all.
"""
site_packages = Path(env_overlay.get("VIRTUAL_ENV", "")) / "Lib" / "site-packages"
if not site_packages.is_dir():
venv_value = env_overlay.get("VIRTUAL_ENV", "")
venv_dir = Path(venv_value) if venv_value else _pm_runtime_venv_dir()
site_packages = venv_dir / "Lib" / "site-packages" if venv_dir else None
if site_packages is None or not site_packages.is_dir():
# Silent here would make the "editable installs invisible" failure
# undiagnosable; the pre-existing PYTHONPATH-only behaviour applies.
logger.warning(
@@ -4364,14 +4406,19 @@ def _run_job_script(
# choice explicit here keeps the allowed surface small and auditable.
suffix = path.suffix.lower()
if suffix in {".sh", ".bash"}:
# Resolve bash dynamically so Windows (Git Bash) and Linux/macOS
# all work. On native Windows without Git for Windows installed
# shutil.which returns None — fall back to a clear error rather
# Resolve bash dynamically so Windows (Git for Windows) and
# Linux/macOS all work — through pm.shell, the one bash authority
# (it prefers conventional installs over MSIX-package aliases,
# which only spawn inside their package context). On native
# Windows without any bash — fall back to a clear error rather
# than a FileNotFoundError with a confusing "[WinError 2]"
# traceback.
_bash = shutil.which("bash") or (
"/bin/bash" if os.path.isfile("/bin/bash") else None
)
try:
from pm.shell import bash as _pm_bash
_bash = _pm_bash()
except Exception:
_bash = None
if _bash is None:
return False, (
f"Cannot run .sh/.bash script {path.name!r}: bash not found on PATH. "
+22 -42
View File
@@ -247,7 +247,7 @@ if [ "$needs_chown" = true ]; then
# Hermes-owned subdirs: recursive chown is safe here because these are
# created and managed exclusively by hermes (see the s6-setuidgid mkdir
# -p block below for the canonical list).
for sub in cron sessions logs hooks memories skills skins plans workspace home profiles pairing platforms/pairing lazy-packages; do
for sub in cron sessions logs hooks memories skills skins plans workspace home profiles pairing platforms/pairing; do
if [ -e "$HERMES_HOME/$sub" ] && tree_has_non_hermes_owner "$HERMES_HOME/$sub"; then
chown_hermes_tree "$HERMES_HOME/$sub"
fi
@@ -262,16 +262,8 @@ fi
# non-writable prevents an agent session from self-modifying the installed
# source, venv, TUI bundle, or node_modules and bricking the gateway.
#
# Lazy-installable optional backends (Firecrawl, Exa, Feishu, etc.) cannot
# install into the sealed venv, so they are redirected to the writable
# $HERMES_HOME/lazy-packages dir on the data volume (Dockerfile sets
# HERMES_LAZY_INSTALL_TARGET). That dir is appended to the END of sys.path,
# so a package installed there can only ADD modules — it can never shadow or
# break a core module, which is what keeps the sealed-venv guarantee intact
# even though installs are re-enabled. The dir is seeded + chowned to hermes
# in the mkdir/chown blocks above so first-use installs succeed as the
# unprivileged runtime user, and it persists across container recreates /
# image updates (an ABI stamp wipes it if a rebuild bumps the interpreter).
# Lazy installs are fully disabled at runtime (HERMES_DISABLE_LAZY_INSTALLS=1,
# see the Dockerfile), so no writable lazy-install target is provisioned here.
# Always reset ownership of $HERMES_HOME/profiles to hermes on every
# boot. Profile dirs and files can land owned by root when commands
@@ -392,8 +384,7 @@ as_hermes mkdir -p \
"$HERMES_HOME/workspace" \
"$HERMES_HOME/home" \
"$HERMES_HOME/pairing" \
"$HERMES_HOME/platforms/pairing" \
"$HERMES_HOME/lazy-packages"
"$HERMES_HOME/platforms/pairing"
# --- Install-method stamp ---
# The 'docker' stamp is baked into the immutable install tree at
@@ -646,42 +637,31 @@ if [ -d "$INSTALL_DIR/skills" ]; then
|| echo "[stage2] Warning: skills_sync.py failed; continuing"
fi
# --- Discover agent-browser's Chromium binary ---
# The image's Dockerfile runs `npx playwright install chromium`, which
# populates ``$PLAYWRIGHT_BROWSERS_PATH`` (=/opt/hermes/.playwright) with
# a ``chromium_headless_shell-<build>/chrome-headless-shell-linux64/``
# directory. agent-browser (the runtime CLI Hermes spawns for the
# browser tool) doesn't recognise this layout in its own cache scan and
# fails with "Auto-launch failed: Chrome not found" — even though the
# binary is right there (#15697).
# --- Point agent-browser at the pinned Chromium binary ---
# The image's Dockerfile pm-provisions the pinned Chromium pair into
# $HERMES_RUNTIME_DIR (/opt/hermes/tools) at BUILD time and bakes the
# resolved browser binary path into /etc/hermes/agent-browser-executable-path
# (the layout differs per arch — chrome-linux64/chrome on amd64,
# chromium-linux-arm64/chromium on arm64 — so it is resolved at build time,
# not hard-coded). agent-browser (the runtime CLI Hermes spawns for the
# browser tool) doesn't recognise Playwright's directory layout in its own
# cache scan and fails with "Auto-launch failed: Chrome not found" — even
# though the binary is right there (#15697).
#
# Fix: locate the binary at boot and export ``AGENT_BROWSER_EXECUTABLE_PATH``
# Fix: read the baked path and export ``AGENT_BROWSER_EXECUTABLE_PATH``
# via /run/s6/container_environment so the `with-contenv` shebang on
# main-wrapper.sh propagates it into the supervised ``hermes`` process
# and thence to agent-browser subprocesses.
#
# - Skipped when the user has already set ``AGENT_BROWSER_EXECUTABLE_PATH``
# (lets users override with a system Chrome install).
# - Filename-matched (not path-matched): the chromium dir contains many
# shared libraries (libGLESv2.so, libEGL.so, ...) which inherit the
# executable bit from Playwright's tarball but are NOT browser binaries.
# We only accept files whose basename is chrome / chromium /
# chrome-headless-shell / headless_shell / chromium-browser. Compare
# PR #18635's earlier ``find | grep -Ei 'chrome|chromium'`` which would
# match the path ``.../chrome-headless-shell-linux64/libGLESv2.so`` and
# pick a .so.
# - Quietly skipped when $PLAYWRIGHT_BROWSERS_PATH doesn't exist (e.g.
# custom builds that strip Playwright).
# - Quietly skipped when the baked path file is absent (e.g. custom builds
# that strip the pm tool store).
if [ -z "${AGENT_BROWSER_EXECUTABLE_PATH:-}" ] && \
[ -n "${PLAYWRIGHT_BROWSERS_PATH:-}" ] && \
[ -d "$PLAYWRIGHT_BROWSERS_PATH" ]; then
browser_bin=$(find "$PLAYWRIGHT_BROWSERS_PATH" -type f -executable \
\( -name 'chrome' -o -name 'chromium' \
-o -name 'chrome-headless-shell' -o -name 'headless_shell' \
-o -name 'chromium-browser' \) \
2>/dev/null | head -n 1)
if [ -n "$browser_bin" ]; then
echo "[stage2] Found agent-browser Chromium binary: $browser_bin"
[ -f /etc/hermes/agent-browser-executable-path ]; then
browser_bin="$(cat /etc/hermes/agent-browser-executable-path)"
if [ -n "$browser_bin" ] && [ -x "$browser_bin" ]; then
echo "[stage2] Using pinned agent-browser Chromium binary: $browser_bin"
# Write to s6's container_environment so with-contenv picks it
# up for all supervised services (main-hermes, dashboard, etc.).
# Idempotent: each boot overwrites with the current path.
@@ -690,7 +670,7 @@ if [ -z "${AGENT_BROWSER_EXECUTABLE_PATH:-}" ] && \
mkdir -p /run/s6/container_environment
printf '%s' "$browser_bin" > /run/s6/container_environment/AGENT_BROWSER_EXECUTABLE_PATH
else
echo "[stage2] Warning: no Chromium binary under $PLAYWRIGHT_BROWSERS_PATH; browser tool may fail"
echo "[stage2] Warning: baked Chromium binary is missing (${browser_bin:-<empty>}); browser tool may fail"
fi
fi
+1 -1
View File
@@ -87,7 +87,7 @@ class PlatformEntry:
validate_config: Optional[Callable[[Any], bool]] = None
# ACTIVE dependency installer: make the platform's dependencies available,
# installing them (pip / lazy_deps) if needed. Returns True once deps are
# installing them (pm.sync_venv) if needed. Returns True once deps are
# importable, False if they could not be installed. Called by
# ``create_adapter()`` when ``check_fn`` returns False — i.e. exactly at
# the moment the gateway is about to bring the platform up and the user
+1 -1
View File
@@ -455,7 +455,7 @@ class WebSocketRelayTransport:
if not WEBSOCKETS_AVAILABLE:
raise RuntimeError(
"WebSocketRelayTransport requires the 'websockets' package "
"(install the messaging extra)."
"(a hermes-agent core dependency)."
)
self._url = _ws_dial_url(url)
self._platform = platform
+36 -1
View File
@@ -32272,7 +32272,42 @@ def main():
with open(args.config, encoding="utf-8") as f:
data = yaml.safe_load(f) or {}
config = GatewayConfig.from_dict(data)
# Post-update boot bootstrap: when this install's code changed since the
# last boot (app-updater swap, docker/nix image change, git pull without
# `hermes update`), run the idempotent user-state steps (config
# migration, skills sync, state.db guard) before platforms connect.
# Two file reads when nothing changed; never raises.
try:
from pathlib import Path as _Path
from hermes_cli.boot_bootstrap import maybe_run_boot_bootstrap
maybe_run_boot_bootstrap(_Path(__file__).resolve().parents[1])
except Exception as exc:
# maybe_run_boot_bootstrap itself never raises and logs internally;
# this guard covers the import/lookup path. Don't hide it silently.
logger.warning("boot bootstrap setup failed (continuing boot): %s", exc)
# pm startup: same contract as the CLI dispatch path (hermes_cli/main.py)
# — the gateway daemon never passes through CLI dispatch, so without this
# it would boot without the store's tools (git/bash/ffmpeg/...) on PATH.
# O(1) stamp checks, no network, no installs; warns, never blocks.
try:
import pm
pm.adopt()
problems = pm.check()
if problems:
logger.warning(
"install out of sync (%s) — run `hermes pm install`",
"; ".join(problems),
)
else:
pm.activate()
except Exception:
logger.debug("pm startup check failed", exc_info=True)
# start_gateway() performs the full graceful teardown (adapters
# disconnected, sessions saved + flushed, SQLite closed, cron/MCP stopped,
# PID file + runtime lock released) before it returns OR raises SystemExit
-30
View File
@@ -201,39 +201,9 @@ def harden_import_path(src_root: str | None = None) -> None:
sys.path.insert(0, root)
def activate_durable_lazy_target() -> None:
"""Put the durable lazy-install dir on ``sys.path`` if one is configured.
On immutable Docker images the agent venv is sealed and lazy installs
are redirected to a writable dir on the data volume
(``HERMES_LAZY_INSTALL_TARGET``, e.g. ``/opt/data/lazy-packages``).
Packages installed there on a previous run must be importable on this
run, so we activate the dir here — at the very first import, before any
backend module imports its SDK.
The activation appends to the END of ``sys.path`` so the core venv
always wins name collisions (see ``tools.lazy_deps`` for the full
security rationale). Never raises; a missing/empty target is a no-op.
"""
if not os.environ.get("HERMES_LAZY_INSTALL_TARGET", "").strip():
return
try:
from tools import lazy_deps
lazy_deps.activate_durable_lazy_target()
except Exception:
# Bootstrap must never crash an entry point. If activation fails the
# backend simply reports itself unavailable, exactly as before.
pass
# Apply on import — entry points just need ``import hermes_bootstrap``
# (or ``from hermes_bootstrap import apply_windows_utf8_bootstrap``) at
# the very top of their module, before importing anything else. The
# import side effect does the right thing.
apply_windows_utf8_bootstrap()
suppress_platform_ver_console()
# Activate the durable lazy-install target (immutable Docker images) so
# packages installed into the data volume on a previous run are importable
# this run, before any backend module imports its SDK. No-op when unset.
activate_durable_lazy_target()
+66 -102
View File
@@ -11,9 +11,9 @@ Both callers need to run the same core ``.[all]`` reinstall:
This module is deliberately **stdlib-only** so importing it can never fail in
the corrupted-venv state it exists to repair. ``hermes_cli.main`` imports
``managed_uv``, ``hermes_constants``, and friends only in its late path; the
early path must not. Where the late path uses ``managed_uv.ensure_uv`` to
bootstrap uv if missing, the early path uses the stdlib
``pm``, ``hermes_constants``, and friends only in its late path; the
early path must not. Where the late path uses ``pm.uv()`` to
realize uv if missing, the early path uses the stdlib
:func:`hermes_cli._early_recovery._find_uv_binary` lookup and falls back to
plain pip when uv is absent — a degraded but working installer (the late
recovery will bootstrap uv on the next launch if it ever matters).
@@ -24,7 +24,6 @@ from __future__ import annotations
import contextlib
import json
import os
import shutil
import subprocess
import sys
import time
@@ -39,19 +38,6 @@ def _is_windows() -> bool:
return sys.platform == "win32"
def _is_termux_env(env: dict | None = None) -> bool:
"""Stdlib Termux probe (hermes_cli.main's version lives behind imports)."""
env = env if env is not None else os.environ
try:
if env.get("TERMUX_VERSION"):
return True
prefix = env.get("PREFIX", "")
return "com.termux" in prefix
except Exception:
return False
@contextlib.contextmanager
def _stdout_to_stderr():
"""Route fd 1 (and sys.stdout) to stderr for the duration of an install.
@@ -85,20 +71,14 @@ def _stdout_to_stderr():
def _resolve_install_target(root: Path) -> tuple[list[str], dict | None]:
"""(install_cmd_prefix, env) for the project venv — stdlib uv lookup.
Mirrors ``main.py::_default_venv_install_target`` but without
``managed_uv``. ``VIRTUAL_ENV`` steers ``uv pip`` at the project venv even
Mirrors ``main.py::_default_venv_install_target`` but without ``pm``. ``VIRTUAL_ENV`` steers ``uv pip`` at the project venv even
when invoked from the base interpreter (the early-recovery case).
Termux strips leaked interpreter-path env vars so uv resolves the venv
correctly.
"""
uv_bin = _er._find_uv_binary()
if uv_bin:
from hermes_constants import project_venv_dir
env = {**os.environ, "VIRTUAL_ENV": str(project_venv_dir(root) or root / "venv")}
if _is_termux_env(env):
env.pop("PYTHONPATH", None)
env.pop("PYTHONHOME", None)
return [uv_bin, "pip"], env
return [sys.executable, "-m", "pip"], None
@@ -120,36 +100,11 @@ def _venv_scripts_dir(root: Path) -> Path | None:
#: Launcher command names install.ps1's Set-PathVariable exposes from the
#: managed binary dir (the default Hermes root's ``bin``, next to uv.exe)
#: on the user PATH. Keep in lockstep with the launcher list in
#: scripts/install.ps1.
#: on the user PATH. Keep in lockstep with WINDOWS_BIN_LAUNCHERS in
#: hermes_cli/_launchers.py and scripts/install.ps1.
_WINDOWS_BIN_LAUNCHERS = ("hermes", "hermes-acp")
def _venv_is_relocatable(venv_dir: Path) -> bool:
"""True when the venv's pyvenv.cfg declares ``relocatable = true``.
uv writes the flag; ``hermes_cli.managed_uv`` builds its replacement
venvs with ``--relocatable`` (they are constructed aside and swapped
into place). A relocatable venv's console-script trampolines embed a
RELATIVE interpreter reference, so a COPY of one placed outside
``venv\\Scripts`` fails at run time with ``uv trampoline failed to
canonicalize script path``. Non-relocatable venvs (fresh installs)
embed the absolute interpreter path and their trampolines survive
copying. This flag decides which launcher form a PATH dir gets.
"""
try:
cfg = (Path(venv_dir) / "pyvenv.cfg").read_text(
encoding="utf-8", errors="replace"
)
except OSError:
return False
for line in cfg.splitlines():
key, _, value = line.partition("=")
if key.strip().lower() == "relocatable" and value.strip().lower() == "true":
return True
return False
def _normalize_windows_path(value) -> str:
"""Windows path equality key: backslashes, no trailing separator, lowered.
@@ -185,11 +140,15 @@ def ensure_windows_bin_launchers(
windows: bool | None = None,
user_path_entries: list[str] | None = None,
) -> list[str]:
"""Re-stage the Windows ``hermes`` launchers when they vanish.
"""Re-stage the Windows ``hermes`` launchers when they vanish or when
they still boot through the venv.
On Windows, ``hermes`` resolves through launchers derived from the venv
console scripts — never ``venv\\Scripts`` itself on PATH, which would
shadow the user's ``python`` (#83797). The canonical launcher home is
On Windows, ``hermes`` resolves through staged launchers — never
``venv\\Scripts`` itself on PATH, which would shadow the user's
``python`` (#83797) — and under pm the launchers boot the pm STORE
python with ``PYTHONPATH=<repo>;<venv>/site-packages``, never the venv
interpreter (no-boot-through-venv; ``pyvenv.cfg`` is inert dead
config). The canonical launcher home is
the managed binary dir — the default Hermes root's ``bin``
(``%LOCALAPPDATA%\\hermes\\bin``, next to the managed uv) — which lives
OUTSIDE the git checkout so no git operation can ever touch it. It is
@@ -205,15 +164,10 @@ def ensure_windows_bin_launchers(
terminal. That legacy location is re-staged too, during the transition,
for installs whose user PATH still resolves through it.
The launcher FORM depends on the venv (see :func:`_venv_is_relocatable`):
a normal venv's exe trampoline embeds an absolute interpreter path and
survives copying, so it is copied as ``<name>.exe``; a relocatable
venv's trampoline resolves relative to its own location and a copy
dies with ``uv trampoline failed to canonicalize script path``, so a
``<name>.cmd`` delegator invoking the in-venv exe by absolute path is
written instead. A name counts as present when EITHER form exists —
exe copies staged before a venv rebuild keep working (they embed the
swapped-in-place venv's absolute path) and are left alone.
A name counts as present when an exe exists that does NOT boot the
venv interpreter — legacy copied-venv trampolines (detected by their
embedded interpreter path) and placeholder .cmd delegators are replaced
with a store-python launcher as soon as one can be minted.
Two targets, two gates, both failing toward inaction:
@@ -251,6 +205,36 @@ def ensure_windows_bin_launchers(
def _launcher_present(target: Path, name: str) -> bool:
return (target / f"{name}.exe").exists() or (target / f"{name}.cmd").exists()
# Launchers boot the pm STORE python with PYTHONPATH=repo;site-packages
# — never the venv interpreter (no boot through the venv; pyvenv.cfg is
# inert dead config). mint_launcher prefers a distlib exe trampoline
# bound to the store python; a runtime-resolving .cmd is written when
# the store has not materialized a python yet, and the repair upgrades
# it (and any legacy copied-venv trampoline) to the exe once the store
# python exists. See hermes_cli/_launchers.py.
from hermes_cli._launchers import (
exe_is_venv_bound,
mint_launcher,
resolve_store_python,
stage_launcher,
venv_site_packages,
)
from hermes_constants import project_venv_dir
venv_dir = project_venv_dir(root)
site_packages = venv_site_packages(venv_dir) if venv_dir else None
store_python = resolve_store_python(root)
def _needs_attention(target: Path, name: str) -> bool:
"""Missing, a placeholder .cmd, or a launcher that still boots the
venv interpreter — anything the store-python launcher should replace."""
exe = target / f"{name}.exe"
if not exe.exists():
return True
return exe_is_venv_bound(exe, venv_dir)
targets: list[Path] = []
# Canonical target — gate on the managed-clone shape. This runs at
@@ -258,7 +242,9 @@ def ensure_windows_bin_launchers(
# override), so the healthy path must stay at a couple of stat calls.
if _normalize_windows_path(root.parent) == _normalize_windows_path(home):
canonical = home / "bin"
if any(not _launcher_present(canonical, name) for name in _WINDOWS_BIN_LAUNCHERS):
if any(
_needs_attention(canonical, name) for name in _WINDOWS_BIN_LAUNCHERS
):
targets.append(canonical)
# Legacy transition target — the pre-migration in-checkout dir. Only
@@ -268,7 +254,7 @@ def ensure_windows_bin_launchers(
# network shares. An entry stored some other way (8.3 short path,
# subst drive) misses the re-stage, which fails safe: no-op.
legacy = root / "bin"
if any(not _launcher_present(legacy, name) for name in _WINDOWS_BIN_LAUNCHERS):
if any(_needs_attention(legacy, name) for name in _WINDOWS_BIN_LAUNCHERS):
if user_path_entries is None:
user_path_entries = _windows_user_path_entries()
configured = {_normalize_windows_path(entry) for entry in user_path_entries}
@@ -278,44 +264,23 @@ def ensure_windows_bin_launchers(
if not targets:
return []
from hermes_constants import project_venv_dir, venv_bin_dir
venv_dir = project_venv_dir(root)
if venv_dir is None:
return []
scripts_dir = venv_bin_dir(venv_dir, windows=windows)
sources = [
(name, scripts_dir / f"{name}.exe")
for name in _WINDOWS_BIN_LAUNCHERS
if (scripts_dir / f"{name}.exe").is_file()
]
if not sources:
return []
relocatable = _venv_is_relocatable(venv_dir)
restored: list[str] = []
for target in targets:
try:
target.mkdir(parents=True, exist_ok=True)
except OSError:
continue
for name, source in sources:
if _launcher_present(target, name):
for name in _WINDOWS_BIN_LAUNCHERS:
if not _needs_attention(target, name):
# Already a store-python launcher (or a form this heal does
# not understand but that does not boot the venv): leave it.
continue
final = target / (f"{name}.cmd" if relocatable else f"{name}.exe")
staging = target / f"{final.name}.heal.{os.getpid()}"
try:
if relocatable:
staging.write_text(
"@echo off\r\n" f'"{source}" %*\r\n', encoding="ascii"
)
else:
shutil.copy2(source, staging)
os.replace(staging, final)
if store_python is not None:
final = mint_launcher(name, root, target, store_python, site_packages)
else:
final = stage_launcher(name, root, target)
if final is not None:
restored.append(str(final))
except OSError:
with contextlib.suppress(OSError):
staging.unlink()
if restored:
# Guarded like everything else in this never-raises helper: a
# closed/broken stderr must not turn a successful heal into a crash.
@@ -573,7 +538,7 @@ def _run_install_cmd(cmd: list[str], *, env: dict | None, root: Path) -> None:
def _load_installable_optional_extras(root: Path, group: str) -> list[str]:
"""Optional extras referenced by a dependency group (all / termux-all)."""
"""Optional extras referenced by a dependency group (all)."""
try:
import tomllib
@@ -604,17 +569,16 @@ def run_core_install(root: Path) -> None:
pip module at all)
- prefer ``uv pip`` with VIRTUAL_ENV pointed at the project venv; fall back
to ``python -m pip`` when no uv binary is available
- target ``.[all]`` (or ``.[termux-all]`` on Termux) with the per-extra
fallback ladder when the combined extras resolve fails
- target ``.[all]`` with the per-extra fallback ladder when the combined
extras resolve fails
- quarantine live ``hermes*.exe`` shims on Windows so they can be replaced
- route ALL install output to stderr (acp/JSON-RPC safety)
- Termux strips leaked PYTHONPATH/PYTHONHOME from the uv env
Raises ``subprocess.CalledProcessError`` when even the base install fails;
callers own marker lifecycle (clear on success, keep on failure).
"""
prefix, env = _resolve_install_target(root)
group = "termux-all" if _is_termux_env(env) else "all"
group = "all"
with _stdout_to_stderr():
try:
@@ -676,7 +640,7 @@ def bump_marker_attempts(marker_path: Path) -> int:
"""
attempts = 0
try:
raw = marker_path.read_text(encoding="utf-8", errors="replace").strip()
raw = marker_path.read_text(encoding="utf-8-sig", errors="replace").strip()
if raw:
try:
attempts = int(json.loads(raw).get("attempts", 0))
+325
View File
@@ -0,0 +1,325 @@
"""Windows launcher staging: `hermes` / `hermes-acp` launchers that boot the
pm STORE python with ``PYTHONPATH=<repo>;<venv>/site-packages`` — never
``venv\\Scripts\\python.exe`` (pm work item 3, "no boot through the venv";
``pyvenv.cfg`` is inert dead config).
Two consumers:
- ``scripts/install.ps1`` Stage-Path (bootstrap) calls
:func:`ensure_install_launchers` through the venv python — install-time
use of the venv interpreter is fine; it is the materializer, not a boot
path. On a fresh install the pm store interpreter does not exist yet, so
a runtime-resolving ``.cmd`` delegator is staged and
``hermes_cli._install_repair.ensure_windows_bin_launchers`` upgrades it
to an exe once ``hermes pm install`` lands the store python.
- ``hermes_cli/_install_repair.py`` calls the same machinery per-name when
a launcher is missing or still boots through the venv.
Store-root resolution mirrors ``pm/paths.py`` (HERMES_RUNTIME_DIR →
install-stamp.json ``runtimeDir`` → the pm default root); it is re-implemented
here because this module must stay importable from a bare interpreter
(stdlib + distlib only) during repair/bootstrap where ``pm`` imports are not
guaranteed. If pm's layout changes, keep the two in lockstep.
The exe form uses distlib's ScriptMaker with a customized script template
that inserts the repo root and the venv's site-packages into ``sys.path``
before importing the entry point — distlib is taken from pip's vendored
copy when the standalone package is absent (uv-synced venvs carry pip but
rarely standalone distlib). When distlib is unavailable, a ``.cmd``
delegator carrying the same PYTHONPATH composition is written instead.
"""
from __future__ import annotations
import json
import os
from pathlib import Path
#: Launcher command names — keep in lockstep with scripts/install.ps1
#: Stage-Path and hermes_cli/_install_repair.py.
WINDOWS_BIN_LAUNCHERS = ("hermes", "hermes-acp")
#: command name -> (entry module, callable) — mirrors pyproject.toml
#: [project.scripts].
ENTRY_POINTS = {
"hermes": ("hermes_cli.main", "main"),
"hermes-acp": ("acp_adapter.entry", "main"),
}
def _is_windows() -> bool:
return os.name == "nt"
def project_venv_dir(repo_root: Path) -> Path | None:
"""The uv-sync target venv of the install (``venv`` preferred, ``.venv``
for dev checkouts)."""
for name in ("venv", ".venv"):
candidate = Path(repo_root) / name
if candidate.is_dir():
return candidate
return None
def store_root(repo_root: Path) -> Path:
"""The pm byte store, resolved exactly like pm/paths.store_root()."""
env = os.environ.get("HERMES_RUNTIME_DIR")
if env:
return Path(env)
stamp = Path(repo_root) / "install-stamp.json"
if stamp.is_file():
try:
runtime_dir = json.loads(stamp.read_text(encoding="utf-8-sig")).get(
"runtimeDir"
)
except (OSError, ValueError):
runtime_dir = None
if runtime_dir:
return Path(runtime_dir)
from hermes_constants import get_default_hermes_root
return Path(get_default_hermes_root()) / "tools"
def resolve_store_python(repo_root: Path) -> Path | None:
"""The interpreter the store installed from the ``python`` package
(facts.json entry first, newest ``python-*`` entry as fallback), or
None when `hermes pm install` has not materialized one yet."""
runtime = store_root(repo_root)
rel = "python.exe" if _is_windows() else "bin/python3"
facts = runtime / "facts.json"
if facts.is_file():
try:
packages = json.loads(facts.read_text(encoding="utf-8-sig")).get(
"packages", {}
)
entry = (packages.get("python") or {}).get("entry")
except (OSError, ValueError):
entry = None
if entry:
candidate = runtime / entry / rel
if candidate.is_file():
return candidate
for entry_dir in sorted(runtime.glob("python-*"), key=lambda p: p.name):
candidate = entry_dir / rel
if candidate.is_file():
return candidate
return None
def venv_site_packages(venv_dir: Path) -> Path | None:
"""The venv directory uv sync fills (Windows ``Lib\\site-packages``;
POSIX ``lib/python3.X/site-packages``)."""
venv_dir = Path(venv_dir)
if _is_windows():
candidate = venv_dir / "Lib" / "site-packages"
return candidate if candidate.is_dir() else None
for candidate in sorted(venv_dir.glob("lib/python3.*/site-packages")):
if candidate.is_dir():
return candidate
return None
def _load_script_maker():
"""distlib's ScriptMaker — standalone first, then pip's vendored copy."""
try:
from distlib.scripts import ScriptMaker
return ScriptMaker
except ImportError:
pass
try:
from pip._vendor.distlib.scripts import ScriptMaker
return ScriptMaker
except ImportError:
return None
def exe_is_venv_bound(exe: Path, venv_dir: Path | None) -> bool:
"""True when an existing launcher exe embeds the venv interpreter —
i.e. it is a copied venv console-script trampoline from the pre-pm
installer, which boots through ``venv\\Scripts\\python.exe`` and must be
replaced. distlib launchers append the interpreter shebang after the zip
payload; both encodings are scanned to be safe."""
if venv_dir is None:
return False
needles = set()
for interpreter in (
venv_dir / "Scripts" / "hermes.exe",
venv_dir / "Scripts" / "hermes-acp.exe",
venv_dir / "Scripts" / "python.exe",
venv_dir / "bin" / "python3",
venv_dir / "bin" / "hermes",
venv_dir / "bin" / "hermes-acp",
):
for enc in ("utf-8", "utf-16-le"):
try:
needles.add(str(interpreter).encode(enc))
except Exception:
continue
try:
data = Path(exe).read_bytes()
except OSError:
return False
return any(needle in data for needle in needles)
def _write_atomic(target: Path, write) -> Path | None:
"""Stage under a pid-suffixed name then os.replace, so a concurrent
process start never sees a torn launcher."""
staging = target.with_name(f"{target.name}.stage.{os.getpid()}")
try:
write(staging)
os.replace(staging, target)
return target
except OSError:
try:
staging.unlink()
except OSError:
pass
return None
def mint_launcher(
name: str,
repo_root: Path,
out_dir: Path,
python_exe: Path,
site_packages: Path | None,
) -> Path | None:
"""Write one launcher for *name* into out_dir. Returns the written path
or None. Prefers a distlib exe trampoline whose embedded script inserts
the repo root and site-packages before importing the entry point; falls
back to a .cmd delegator (same interpreter, same PYTHONPATH) when
distlib is unavailable."""
module, func = ENTRY_POINTS[name]
out_dir = Path(out_dir)
script_maker_cls = _load_script_maker()
if script_maker_cls is not None:
# The template is %-formatted by distlib with (module, import_name,
# func); the install-time paths are baked in as literals. The repo
# root goes FIRST — its packages win over site-packages (same
# ordering as the desktop shim's PYTHONPATH composition).
esc = lambda p: str(p).replace("%", "%%") # noqa: E731
site_line = (
f"sys.path.append({esc(site_packages)!r})\n" if site_packages else ""
)
class _PathedScriptMaker(script_maker_cls): # type: ignore[misc,valid-type]
script_template = (
"# -*- coding: utf-8 -*-\n"
"import re\n"
"import sys\n"
f"sys.path.insert(0, {esc(repo_root)!r})\n"
f"{site_line}"
"if __name__ == '__main__':\n"
" from %(module)s import %(import_name)s\n"
" sys.argv[0] = re.sub(r'(-script\\.pyw|\\.exe)?$', '', sys.argv[0])\n"
" sys.exit(%(func)s())\n"
)
maker = _PathedScriptMaker(None, str(out_dir), add_launchers=True)
maker.executable = str(python_exe)
maker.variants = {""}
maker.clobber = True
try:
written = maker.make(f"{name} = {module}:{func}")
except Exception:
written = []
for path in written:
if Path(path).suffix.lower() == ".exe":
return Path(path)
# distlib ran but produced no exe (unexpected) — fall through to cmd.
site = f";{site_packages}" if site_packages else ""
code = f"import sys; from {module} import {func}; sys.exit({func}())"
body = (
"@echo off\r\n"
"chcp 65001 >nul\r\n"
f'set "PYTHONPATH={repo_root}{site}"\r\n'
'set "PYTHONHOME="\r\n'
f'"{python_exe}" -c "{code}" %*\r\n'
)
return _write_atomic(out_dir / f"{name}.cmd", lambda p: p.write_text(body, encoding="utf-8"))
def stage_launcher(name: str, repo_root: Path, out_dir: Path) -> Path | None:
"""Stage/refresh ONE launcher for the install rooted at repo_root,
writing what the CURRENT state supports:
- store interpreter present → exe (or .cmd if distlib is missing) bound
to it, with the repo-first PYTHONPATH baked in;
- store interpreter absent → a runtime-resolving .cmd that finds the
store python at boot and fails with a clear message until
`hermes pm install` materializes it.
Never raises; returns the written path or None."""
repo_root = Path(repo_root)
venv_dir = project_venv_dir(repo_root)
site_packages = venv_site_packages(venv_dir) if venv_dir else None
store_python = resolve_store_python(repo_root)
if store_python is not None:
path = mint_launcher(name, repo_root, out_dir, store_python, site_packages)
if path is not None:
return path
return _write_runtime_cmd(name, repo_root, site_packages, out_dir)
def ensure_install_launchers(repo_root: Path, out_dir: Path) -> list[str]:
"""Stage/refresh every launcher in WINDOWS_BIN_LAUNCHERS — see
:func:`stage_launcher` for the per-name contract."""
repo_root = Path(repo_root)
written: list[str] = []
for name in WINDOWS_BIN_LAUNCHERS:
path = stage_launcher(name, repo_root, Path(out_dir))
if path is not None:
written.append(str(path))
return written
def _write_runtime_cmd(
name: str,
repo_root: Path,
site_packages: Path | None,
out_dir: Path,
) -> Path | None:
"""The boot-time-resolving .cmd fallback (fresh install, no store
interpreter yet): glob ``<runtime>\\python-*`` for the store python at
boot, compose PYTHONPATH, and fail with a clear message when the store
is empty. Never references the venv interpreter. The ``endlocal & set``
idiom hoists the boot-resolved interpreter and PYTHONPATH out of the
setlocal scope (percent expansion happens while setlocal is still
active, before endlocal executes)."""
module, func = ENTRY_POINTS[name]
site = ""
if site_packages is not None:
site = (
";%HERMES_REPO%\\"
+ str(site_packages.relative_to(repo_root)).replace("/", "\\")
)
body = (
"@echo off\r\n"
"chcp 65001 >nul\r\n"
"setlocal\r\n"
f'set "HERMES_REPO={repo_root}"\r\n'
'set "PM_RT=%HERMES_RUNTIME_DIR%"\r\n'
'if not defined PM_RT set "PM_RT=%LOCALAPPDATA%\\hermes\\tools"\r\n'
'set "PM_PY="\r\n'
'for /d %%D in ("%PM_RT%\\python-*") do if exist "%%D\\python.exe" set "PM_PY=%%D\\python.exe"\r\n'
'if not defined PM_PY (\r\n'
" echo hermes: no pm store interpreter under %PM_RT% - run \"hermes pm install\" first 1>&2\r\n"
" exit /b 1\r\n"
")\r\n"
f'endlocal & set "PYTHONPATH=%HERMES_REPO%{site}" & set "PM_PY=%PM_PY%"\r\n'
'set "PYTHONHOME="\r\n'
f'set "PM_ENTRY=import sys; from {module} import {func}; sys.exit({func}())"\r\n'
'"%PM_PY%" -c "%PM_ENTRY%" %*\r\n'
)
return _write_atomic(
Path(out_dir) / f"{name}.cmd",
lambda p: p.write_text(body, encoding="utf-8"),
)
+24
View File
@@ -38,6 +38,7 @@ __all__ = [
"IS_WINDOWS",
"resolve_node_command",
"split_command_line",
"restore_ambient_pythonpath",
"suppress_platform_ver_console",
"windows_detach_flags",
"windows_detach_flags_without_breakaway",
@@ -93,6 +94,29 @@ def split_command_line(line: str) -> list[str]:
# -----------------------------------------------------------------------------
def restore_ambient_pythonpath(env: Mapping[str, str]) -> dict:
"""Re-add the ambient ``PYTHONPATH`` to a child environment that a
``build_subprocess_env``-style factory already built.
No-boot-through-venv: the boot interpreter is the pm STORE python, whose
imports arrive via ``PYTHONPATH=<repo>;<venv>/site-packages`` (it has no
editable install). The subprocess-env factories strip Hermes-owned
PYTHONPATH entries so agent-run children on DIFFERENT interpreter
versions never load the backend's C extensions — but a child that
re-execs THIS interpreter (``sys.executable -m hermes_cli.main``) runs
on the same version and needs those entries back. Prepending keeps the
launcher's repo-first ordering intact.
"""
merged = dict(env)
ambient = os.environ.get("PYTHONPATH")
if ambient:
existing = merged.get("PYTHONPATH", "")
merged["PYTHONPATH"] = (
ambient + os.pathsep + existing if existing else ambient
)
return merged
def resolve_node_command(name: str, argv: Sequence[str]) -> list[str]:
"""Resolve a Node-ecosystem command name to an absolute-path argv.
+2 -2
View File
@@ -1731,7 +1731,7 @@ def list_quick_snapshots(
manifest_path = d / "manifest.json"
if manifest_path.exists():
try:
with open(manifest_path, encoding="utf-8") as f:
with open(manifest_path, encoding="utf-8-sig") as f:
results.append(json.load(f))
except (json.JSONDecodeError, OSError):
results.append({"id": d.name, "file_count": 0, "total_size": 0})
@@ -1775,7 +1775,7 @@ def restore_quick_snapshot(
if not manifest_path.exists():
return False
with open(manifest_path, encoding="utf-8") as f:
with open(manifest_path, encoding="utf-8-sig") as f:
meta = json.load(f)
restored = 0
+1 -1
View File
@@ -3253,7 +3253,7 @@ class CLICommandsMixin:
# Fall back to a bare invocation (editor value may not be a
# simple argv-splittable string on some platforms).
subprocess.call(f"{editor} {shlex.quote(path)}", shell=True)
with open(path, "r", encoding="utf-8") as fh:
with open(path, "r", encoding="utf-8-sig") as fh:
raw = fh.read()
finally:
try:
+34 -161
View File
@@ -1,185 +1,58 @@
"""Lazy dependency bootstrapper for non-Python runtime deps.
"""Lazy bootstrapper for non-Python runtime deps, routed through pm.
Detection and prompting live here in Python — not in install.sh — because:
1. shutil.which() works on every platform; install.sh needs bash.
2. Detection is instant; spawning bash for a "is node installed?" check is waste.
3. Python controls the UX (rich prompts, non-interactive fallback, TTY detection).
The old shape spawned install.sh/install.ps1 with an --ensure flag; the
rewritten bootstraps no longer install tools ("heavy deps are pm's job"),
so this module is now a thin adapter: each dep maps to the pm packages
that provide it, pm's lazy-install policy decides whether installing is
allowed right now, and pm's InstallError carries the remedy when not.
install.sh is still the *installation* backend because it has 1900 lines of
battle-tested OS detection and package-manager logic (apt/brew/pacman/dnf/
zypper/Termux/…). Reimplementing that in Python would be huge duplication.
Deps that degrade gracefully (ripgrep → grep fallback, ffmpeg → skip conversion)
don't need ensure_dependency wired in — only hard-fail sites do (TUI needs node,
browser tool needs agent-browser).
Deps that degrade gracefully (ffmpeg → skip conversion) are not wired
here — only hard-fail sites call ensure_dependency (TUI needs node,
browser tools need the browser stack).
"""
from __future__ import annotations
import platform
import shutil
import subprocess
import sys
from pathlib import Path
from hermes_constants import agent_browser_runnable, find_node_executable
from tools.environments.local import hermes_subprocess_env
from hermes_constants import find_node_executable
_IS_WINDOWS = platform.system() == "Windows"
_DEP_CHECKS = {
# find_node_executable() rather than a bare which(): $HERMES_HOME/node is
# not on PATH, so which() would report Node missing on an install that has
# a managed one and trigger a redundant re-install.
"node": lambda: find_node_executable("node") is not None,
"browser": lambda: (
agent_browser_runnable(shutil.which("agent-browser"))
or _has_system_browser()
or _has_hermes_agent_browser()
or _has_npx_agent_browser()
),
"ripgrep": lambda: shutil.which("rg") is not None,
"ffmpeg": lambda: shutil.which("ffmpeg") is not None,
}
_DEP_DESCRIPTIONS = {
"node": "Node.js (required for browser tools and TUI)",
"browser": "Browser engine (Chromium, for web browsing tools)",
"ripgrep": "ripgrep (fast file search)",
"ffmpeg": "ffmpeg (TTS voice messages)",
# dep name -> (availability check, pm packages that provide it)
_DEPS = {
"node": (lambda: find_node_executable("node") is not None, ("node",)),
"browser": (lambda: _browser_available(), ("agent-browser", "chromium")),
"ripgrep": (lambda: shutil.which("rg") is not None, ("ripgrep",)),
}
def _has_system_browser() -> bool:
if _IS_WINDOWS:
names = ("chrome", "msedge", "chromium")
else:
names = ("google-chrome", "google-chrome-stable", "chromium", "chromium-browser", "chrome")
for name in names:
if shutil.which(name):
return True
return False
def _browser_available() -> bool:
from hermes_constants import agent_browser_runnable
def _has_npx_agent_browser() -> bool:
"""agent-browser resolves lazily via npx on the default install (#43564),
invisible to the PATH/managed-dir probes above. Mirror
tools.browser_tool.check_browser_requirements's Termux carve-out so this
check can't diverge from what browser tools actually find."""
if agent_browser_runnable(shutil.which("agent-browser")):
return True
try:
from tools.browser_tool import (
_find_agent_browser,
_is_npx_agent_browser_sentinel,
_requires_real_termux_browser_install,
)
browser_cmd = _find_agent_browser(validate=False)
import pm
return pm.is_installed("agent-browser")
except Exception:
return False
if not _is_npx_agent_browser_sentinel(browser_cmd):
return False
return not _requires_real_termux_browser_install(browser_cmd)
def _has_hermes_agent_browser() -> bool:
from hermes_constants import get_hermes_home
home = get_hermes_home()
if _IS_WINDOWS:
# npm -g --prefix puts .cmd shims directly in the prefix dir on Windows
return (home / "node" / "agent-browser.cmd").is_file()
# install.sh installs globally into $HERMES_HOME/node/bin/ via npm -g --prefix
# Also check legacy node_modules/.bin/ path for git-clone installs.
return (
(home / "node" / "bin" / "agent-browser").is_file()
or (home / "node_modules" / ".bin" / "agent-browser").is_file()
)
def _find_install_script(
package_dir: Path | None = None,
repo_root: Path | None = None,
) -> tuple[Path | None, str | None]:
"""Locate the install script — bundled in wheel or in git checkout.
On Windows, prefers install.ps1; on POSIX, prefers install.sh.
Returns a (path, shell) tuple, or (None, None) if neither is found.
"""
if package_dir is None:
package_dir = Path(__file__).parent
if repo_root is None:
repo_root = package_dir.parent
if _IS_WINDOWS:
preferred = ("install.ps1", "powershell")
fallback = ("install.sh", "bash")
else:
preferred = ("install.sh", "bash")
fallback = ("install.ps1", "powershell")
for script_name, shell in (preferred, fallback):
bundled = package_dir / "scripts" / script_name
if bundled.is_file():
return bundled, shell
repo = repo_root / "scripts" / script_name
if repo.is_file():
return repo, shell
return None, None
def ensure_dependency(
dep: str,
interactive: bool = True,
) -> bool:
def ensure_dependency(dep: str, interactive: bool = True) -> bool:
"""Ensure a non-Python dependency is available. Returns True if available."""
check = _DEP_CHECKS.get(dep)
if check is None:
# Unknown dep — don't silently forward to install script.
entry = _DEPS.get(dep)
if entry is None:
return False
check, packages = entry
if check():
return True
script, shell = _find_install_script()
if script is None:
try:
import pm
for name in packages:
pm.ensure(name)
except Exception as exc:
if interactive:
desc = _DEP_DESCRIPTIONS.get(dep, dep)
print(f" {desc} is not installed and no install script was found.")
print(f" Install {dep} manually and try again.")
print(f" {exc}")
return False
if interactive and sys.stdin.isatty():
desc = _DEP_DESCRIPTIONS.get(dep, dep)
try:
reply = input(f"{desc} is not installed. Install now? [Y/n] ").strip().lower()
except (EOFError, KeyboardInterrupt):
return False
if reply not in ("", "y", "yes"):
return False
if shell == "powershell":
from hermes_constants import get_hermes_home
ps_bin = shutil.which("powershell") or shutil.which("pwsh")
if not ps_bin:
if interactive:
print(" PowerShell not found. Install PowerShell or run install.ps1 manually.")
return False
cmd = [
ps_bin,
"-ExecutionPolicy", "Bypass",
"-File", str(script),
"-Ensure", dep,
"-HermesHome", str(get_hermes_home()),
]
else:
cmd = ["bash", str(script), "--ensure", dep]
run_env = hermes_subprocess_env(inherit_credentials=False)
run_env["IS_INTERACTIVE"] = "false"
result = subprocess.run(
cmd,
env=run_env,
)
if result.returncode != 0:
return False
if check:
return check()
return True
return check()
+41 -9
View File
@@ -706,7 +706,7 @@ def _read_pyproject_version() -> str | None:
"""
pyproject = PROJECT_ROOT / "pyproject.toml"
try:
text = pyproject.read_text(encoding="utf-8")
text = pyproject.read_text(encoding="utf-8-sig")
except OSError:
return None
in_project = False
@@ -1232,6 +1232,34 @@ def check_macos_full_disk_access() -> None:
)
def _pm_venv_active() -> bool:
"""Whether pm provisioned a runtime venv for this install.
Resolved from pm's own records (facts.json + store layout), never from
interpreter state: under no-boot-through-venv ``sys.prefix`` always
equals ``sys.base_prefix`` and ``VIRTUAL_ENV`` is unset in bundled
installs. A bundled install keeps its relocatable venv beside the
manifest; a dev install syncs the project venv (``venv``/``.venv``).
Falls back to the legacy ``sys.prefix`` probe when pm records nothing
here (pre-pm checkouts) or pm itself cannot be read.
"""
try:
from pm import paths
from pm.lock import Facts
if Facts(paths.facts_path()).get("venv"):
store = paths.store_root()
bundled = store.parent / "venv"
if (store.parent / "manifest.json").is_file():
return bundled.is_dir()
from hermes_constants import project_venv_dir
return project_venv_dir(paths.repo_root()) is not None
except Exception:
pass
return sys.prefix != sys.base_prefix
def run_doctor(args):
"""Run diagnostic checks."""
should_fix = getattr(args, 'fix', False)
@@ -1395,8 +1423,12 @@ def run_doctor(args):
_report_database_journal_modes()
except Exception as e:
check_warn(f"SQLite version probe failed: {e}")
# Check if in virtual environment
in_venv = sys.prefix != sys.base_prefix
# Check whether pm provisioned a runtime venv for this install.
# Under no-boot-through-venv the gateway runs the store python
# (sys.prefix == sys.base_prefix always, VIRTUAL_ENV unset in bundled
# installs), so prefix sniffing cannot distinguish; pm's facts are the
# authority. Falls back to the legacy probe when pm has no venv here.
in_venv = _pm_venv_active()
if in_venv:
check_ok("Virtual environment active")
else:
@@ -1464,7 +1496,7 @@ def run_doctor(args):
# latin-1 for Windows Notepad/cp1252 files that are not valid UTF-8 —
# matches hermes_cli.env_loader._load_dotenv_with_fallback.
try:
content = env_path.read_text(encoding="utf-8")
content = env_path.read_text(encoding="utf-8-sig")
except UnicodeDecodeError:
content = env_path.read_text(encoding="latin-1")
if _has_provider_env_config(content):
@@ -1995,7 +2027,7 @@ def run_doctor(args):
# Check for SOUL.md persona file
soul_path = hermes_home / "SOUL.md"
if soul_path.exists():
content = soul_path.read_text(encoding="utf-8").strip()
content = soul_path.read_text(encoding="utf-8-sig").strip()
# Check if it's just the template comments (no real content)
lines = [l for l in content.splitlines() if l.strip() and not l.strip().startswith(("<!--", "-->", "#"))]
if lines:
@@ -2022,12 +2054,12 @@ def run_doctor(args):
memory_file = memories_dir / "MEMORY.md"
user_file = memories_dir / "USER.md"
if memory_file.exists():
size = len(memory_file.read_text(encoding="utf-8").strip())
size = len(memory_file.read_text(encoding="utf-8-sig").strip())
check_ok(f"MEMORY.md exists ({size} chars)")
else:
check_info("MEMORY.md not created yet (will be created when the agent first writes a memory)")
if user_file.exists():
size = len(user_file.read_text(encoding="utf-8").strip())
size = len(user_file.read_text(encoding="utf-8-sig").strip())
check_ok(f"USER.md exists ({size} chars)")
else:
check_info("USER.md not created yet (will be created when the agent first writes a memory)")
@@ -3178,7 +3210,7 @@ def run_doctor(args):
if lock_file.exists():
try:
import json
lock_data = json.loads(lock_file.read_text(encoding="utf-8"))
lock_data = json.loads(lock_file.read_text(encoding="utf-8-sig"))
count = len(lock_data.get("installed", {}))
check_ok(f"Lock file OK ({count} hub-installed skill(s))")
except Exception:
@@ -3344,7 +3376,7 @@ def run_doctor(args):
if not wrapper.is_file():
continue
try:
content = wrapper.read_text(encoding="utf-8")
content = wrapper.read_text(encoding="utf-8-sig")
if "hermes -p" in content:
_m = _re.search(r"hermes -p (\S+)", content)
if _m and not profile_exists(_m.group(1)):
+56 -6
View File
@@ -3525,15 +3525,57 @@ def launchd_gateway_labels_for_install() -> list[str]:
return root_label + sorted(profile_labels)
def _pm_runtime_venv_dir() -> Path | None:
"""The venv pm provisioned for this install, resolved from pm's own
records (facts.json + store layout) — never from interpreter state.
Under no-boot-through-venv the gateway runs the store python with the
venv's site-packages on PYTHONPATH, so ``sys.prefix`` always equals
``sys.base_prefix`` and ``VIRTUAL_ENV`` is unset in bundled installs;
prefix/env probing silently degrades. pm is the authority instead: a
bundled install keeps its relocatable venv beside the manifest (a
sibling of the store), a dev install syncs the project venv
(``venv``/``.venv``, per ``hermes_constants.project_venv_dir`` — the
same layout ``pm.packages.Venv.venv_dir`` materializes). The venv fact
must exist: pm only vouches for what it provisioned.
"""
try:
from pm import paths
from pm.lock import Facts
except Exception:
return None
try:
if not Facts(paths.facts_path()).get("venv"):
return None
except Exception:
return None
store = paths.store_root()
bundled = store.parent / "venv"
if (store.parent / "manifest.json").is_file():
return bundled if bundled.is_dir() else None
try:
from hermes_constants import project_venv_dir
except ImportError:
return None
return project_venv_dir(paths.repo_root())
def _detect_venv_dir() -> Path | None:
"""Detect the active virtualenv directory.
Checks ``sys.prefix`` first (works regardless of the directory name),
then ``VIRTUAL_ENV`` env var (covers uv-managed environments where
sys.prefix == sys.base_prefix), then falls back to probing common
directory names under PROJECT_ROOT.
Resolves the pm-provisioned runtime venv first (facts + store layout —
the authority under no-boot-through-venv, where ``sys.prefix`` no
longer distinguishes and bundled installs carry no ``VIRTUAL_ENV``).
Legacy probes follow for pre-pm environments: ``sys.prefix`` (works
regardless of the directory name), then the ``VIRTUAL_ENV`` env var
(covers uv-managed environments where sys.prefix ==
sys.base_prefix), then common directory names under PROJECT_ROOT.
Returns ``None`` when no virtualenv can be found.
"""
pm_venv = _pm_runtime_venv_dir()
if pm_venv is not None and pm_venv.is_dir():
return pm_venv
# If we're running inside a virtualenv, sys.prefix points to it.
if sys.prefix != sys.base_prefix:
venv = Path(sys.prefix)
@@ -3706,11 +3748,19 @@ def _build_service_path_dirs(project_root: Path | None = None) -> list[str]:
candidates = []
# The interpreter's own bin dir. Under no-boot-through-venv the
# gateway runs the store python (sys.prefix == sys.base_prefix), so
# the venv bin dir comes from pm's facts/store resolution, not from
# prefix sniffing.
venv_bin = project_root / "venv" / "bin"
if _is_dir(venv_bin):
candidates.append(str(venv_bin))
elif sys.prefix != sys.base_prefix:
candidates.append(str(Path(sys.prefix) / "bin"))
else:
pm_venv = _pm_runtime_venv_dir()
if pm_venv is not None:
pm_venv_bin = pm_venv / "bin"
if _is_dir(pm_venv_bin):
candidates.append(str(pm_venv_bin))
node_bin = project_root / "node_modules" / ".bin"
if _is_dir(node_bin):
+18 -1
View File
@@ -57,6 +57,19 @@ def icon_path(project_root: Path) -> Path:
return project_root / "apps" / "desktop" / "assets" / "icon.png"
def _repo_pythonpath_entry() -> str | None:
"""The repo root as a PYTHONPATH entry — under no-boot-through-venv the
boot interpreter is the pm STORE python whose imports arrive via
PYTHONPATH (no editable install), and a .desktop launch by the DE
inherits NO environment, so the Exec line must carry it explicitly."""
try:
import hermes_cli
return str(Path(hermes_cli.__file__).resolve().parent.parent)
except Exception:
return None
def resolve_exec_command() -> str:
"""Build the absolute ``Exec=`` command line for ``hermes desktop``.
@@ -66,6 +79,8 @@ def resolve_exec_command() -> str:
"""
from hermes_cli.relaunch import resolve_hermes_bin
repo = _repo_pythonpath_entry()
prefix = f"env PYTHONPATH={_quote_exec_arg(repo)} " if repo else ""
bin_path = resolve_hermes_bin()
if bin_path:
resolved = Path(bin_path).resolve()
@@ -79,11 +94,13 @@ def resolve_exec_command() -> str:
# sys.executable is the interpreter actually running Hermes (the
# venv one), so prefix it explicitly.
argv = [str(Path(sys.executable).resolve()), str(resolved), "desktop"]
return prefix + " ".join(_quote_exec_arg(a) for a in argv)
else:
argv = [str(resolved), "desktop"]
return " ".join(_quote_exec_arg(a) for a in argv)
else:
argv = [str(Path(sys.executable).resolve()), "-m", "hermes_cli.main", "desktop"]
return " ".join(_quote_exec_arg(a) for a in argv)
return prefix + " ".join(_quote_exec_arg(a) for a in argv)
def _needs_interpreter(bin_path: Path) -> bool:
+3 -3
View File
@@ -147,7 +147,7 @@ def _interpreter_source(venv_dir: Path) -> str | None:
return None
home = ""
try:
for line in cfg.read_text(encoding="utf-8").splitlines():
for line in cfg.read_text(encoding="utf-8-sig").splitlines():
if line.lower().startswith("home"):
_, _, home = line.partition("=")
home = home.strip()
@@ -390,7 +390,7 @@ def ensure_tcc_anchor(project_root: Path | None = None) -> Path | None:
if not venv_py.is_symlink():
marker = _anchor_marker(venv_py.parent)
try:
if marker.is_file() and marker.read_text(encoding="utf-8").strip() == (
if marker.is_file() and marker.read_text(encoding="utf-8-sig").strip() == (
_marker_value(source_file)
):
_provision_libpython(venv_dir, source_file, refresh=False)
@@ -436,7 +436,7 @@ def tcc_anchor_state(project_root: Path | None = None) -> tuple[str, str]:
source_file = _interpreter_file(source)
expected = _marker_value(source_file) if source_file is not None else source
try:
if marker.is_file() and marker.read_text(encoding="utf-8").strip() == expected:
if marker.is_file() and marker.read_text(encoding="utf-8-sig").strip() == expected:
return "active", str(venv_py)
except OSError:
pass
+264 -753
View File
File diff suppressed because it is too large. Load diff
+37 -203
View File
@@ -1,4 +1,4 @@
"""Recover from npm ``EBADENGINE`` failures by upgrading a managed npm.
"""Recover from npm ``EBADENGINE`` failures with the pm-pinned npm.
The repo's ``.npmrc`` sets ``engine-strict=true`` and the root ``package.json``
pins an ``engines.npm`` range, so an npm outside that range aborts every
@@ -13,39 +13,25 @@ an ``npm --version`` probe before work that usually succeeds), we react to it:
npm states the required range in the error, so the recovery reads the
constraint straight out of the output it just produced.
Scope of the repair is deliberately narrow. Hermes only upgrades an npm that
lives inside its **own** managed Node tree (``$HERMES_HOME/node``), installing
in place with ``--prefix`` so ``bin/npm`` keeps resolving to the upgraded
``lib/node_modules/npm``. A system / nvm / brew / Nix npm belongs to the user
and their other projects; Hermes never modifies those. When the failing npm is
one of those foreign installs, Hermes instead provisions its own managed Node
tree (the same tree a fresh install creates), upgrades *that* npm into range,
and hands the caller the managed npm to retry with — leaving the user's
toolchain untouched.
Scope of the repair is deliberately narrow. A system / nvm / brew / Nix npm
belongs to the user and their other projects; Hermes never modifies those.
When the failing npm is a foreign install, Hermes ensures its own pm-pinned
node/npm packages are installed and hands the caller pm's npm to retry with —
leaving the user's toolchain untouched. When the failing npm already *is*
pm's npm, the lockfile pin itself is out of range and no runtime action can
fix that; the caller gets the manual guidance instead.
"""
from __future__ import annotations
import json
import os
import re
import subprocess
import sys
import tempfile
from pathlib import Path
from hermes_constants import (
bootstrap_hermes_managed_node,
get_hermes_home,
managed_node_tree_in_use,
with_hermes_node_path,
)
__all__ = [
"is_ebadengine",
"required_npm_range",
"managed_npm_prefix",
"upgrade_managed_npm",
"maybe_repair_npm_engine",
]
@@ -54,10 +40,6 @@ __all__ = [
_REQUIRED_RE = re.compile(r"Required:\s*(\{.*?\})")
_ACTUAL_RE = re.compile(r"Actual:\s*(\{.*?\})")
# Wall-clock cap for the self-upgrade. The measured in-place upgrade of a
# managed tree takes ~1s; this only has to cover a slow registry.
_UPGRADE_TIMEOUT = 300
def is_ebadengine(output: str) -> bool:
"""Return True when *output* is an npm engine-compatibility failure."""
@@ -123,7 +105,7 @@ def _repo_npm_range() -> str | None:
"""Return ``engines.npm`` from the checkout's root ``package.json``."""
package_json = Path(__file__).resolve().parent.parent / "package.json"
try:
data = json.loads(package_json.read_text(encoding="utf-8"))
data = json.loads(package_json.read_text(encoding="utf-8-sig"))
except (OSError, ValueError):
return None
engines = data.get("engines")
@@ -133,130 +115,26 @@ def _repo_npm_range() -> str | None:
return str(value).strip() if value else None
def managed_npm_prefix(npm: str | os.PathLike[str] | None) -> Path | None:
"""Return the Hermes-managed Node root *npm* lives in, else ``None``.
Symlinks are resolved first: an install links ``~/.local/bin/npm`` at
``$HERMES_HOME/node/bin/npm``, which itself links into
``lib/node_modules/npm/bin/npm-cli.js``. Every one of those spellings is
the managed npm and must be recognised as such, or the repair silently
declines to fix the very install it owns.
"""
if not npm:
return None
prefix = get_hermes_home() / "node"
try:
resolved = Path(npm).resolve()
prefix_resolved = prefix.resolve()
except OSError:
return None
if resolved == prefix_resolved or prefix_resolved in resolved.parents:
return prefix
return None
def _upgrade_env() -> dict[str, str]:
env = with_hermes_node_path()
# The checkout's .npmrc sets `min-release-age`, which would gate the npm
# release we are trying to install. The upgrade runs from a temp cwd so
# that file is out of scope; this neutralises a user-level ~/.npmrc too.
env["npm_config_min_release_age"] = "0"
# `unicode-animations`-style postinstall animations no-op under CI=1.
env["CI"] = "1"
return env
def upgrade_managed_npm(
npm: str,
npm_range: str,
*,
prefix: Path,
quiet: bool = False,
) -> bool:
"""Upgrade the managed npm at *npm* in place to satisfy *npm_range*.
``--prefix`` targets the managed tree explicitly: a managed install writes
``prefix=~/.local`` into ``$HERMES_HOME/node/etc/npmrc`` so that global
installs land on PATH, and without the override the "upgrade" would install
a second npm somewhere else while the managed one stayed stale.
"""
def _pm_npm(*, quiet: bool = False) -> str | None:
"""Install the pm-pinned node/npm packages and return pm's npm path."""
if not quiet:
print(
f"→ Upgrading Hermes-managed npm to satisfy {npm_range}…",
"→ Provisioning the Hermes-pinned Node.js runtime "
"(the resolved npm belongs to your system and is left alone)…",
flush=True,
)
# The managed npm lives inside the very tree the desktop app's Node
# processes execute from; an in-place upgrade while it is in use fails
# with PermissionError: [WinError 5] on npm.cmd (#80926). Defer instead
# of forcing the write — the upgrade re-triggers on the next resolution
# (e.g. the next update once the app is closed).
if managed_node_tree_in_use():
if not quiet:
print(
" ⚠ deferred: the Hermes-managed Node.js tree is in use by a "
"running app; the npm upgrade will apply on a later update "
"once the app is closed.",
file=sys.stderr,
)
return False
try:
# A temp cwd keeps the checkout's .npmrc (engine-strict, min-release-age)
# from applying to the upgrade itself.
with tempfile.TemporaryDirectory(prefix="hermes-npm-upgrade-") as tmp:
result = subprocess.run(
[
npm,
"install",
"--global",
"--prefix",
str(prefix),
f"npm@{npm_range}",
"--no-fund",
"--no-audit",
"--progress=false",
],
cwd=tmp,
env=_upgrade_env(),
capture_output=True,
text=True,
encoding="utf-8",
errors="replace",
timeout=_UPGRADE_TIMEOUT,
check=False,
)
except (OSError, subprocess.SubprocessError):
if not quiet:
print(" ✗ npm upgrade could not be started", file=sys.stderr)
return False
import pm
if result.returncode != 0:
if not quiet:
detail = (result.stderr or result.stdout or "").strip().splitlines()
print(" ✗ npm upgrade failed", file=sys.stderr)
for line in detail[-10:]:
print(f" {line}", file=sys.stderr)
return False
pm.ensure("npm")
from hermes_constants import _pm_node_executable
if not quiet:
print(f" ✓ npm upgraded to {_probe_version(npm) or npm_range}", flush=True)
return True
def _probe_version(npm: str) -> str | None:
try:
result = subprocess.run(
[npm, "--version"],
capture_output=True,
text=True,
encoding="utf-8",
errors="replace",
timeout=30,
env=with_hermes_node_path(),
check=False,
)
except (OSError, subprocess.SubprocessError):
return None
return (result.stdout or "").strip() or None
managed = _pm_node_executable("npm")
except Exception:
managed = None
if not managed and not quiet:
print(" ✗ Managed Node.js provisioning failed", file=sys.stderr)
return managed
def _print_manual_fix(npm: str, npm_range: str, actual: str | None) -> None:
@@ -271,41 +149,6 @@ def _print_manual_fix(npm: str, npm_range: str, actual: str | None) -> None:
)
def _provision_managed_npm(npm_range: str | None, *, quiet: bool = False) -> str | None:
"""Provision a Hermes-managed Node tree and return a satisfying npm.
Installs the managed tree under ``$HERMES_HOME/node`` (reusing a healthy
one when present), then upgrades its bundled npm to *npm_range* — a fresh
Node LTS bundles an npm that may itself be outside the repo's range, so
without the upgrade the caller's single retry would fail the same way.
Falls back to the checkout's own ``engines.npm`` when npm did not state a
range (a Node-only mismatch), so the managed npm ends up in range either
way. Returns the managed npm path, or ``None`` when provisioning failed.
"""
if not quiet:
print(
"→ Provisioning a Hermes-managed Node.js runtime "
"(the resolved npm belongs to your system and is left alone)…",
flush=True,
)
managed_npm = bootstrap_hermes_managed_node()
if not managed_npm:
if not quiet:
print(" ✗ Managed Node.js provisioning failed", file=sys.stderr)
return None
prefix = managed_npm_prefix(managed_npm)
if prefix is None: # pragma: no cover - bootstrap returned a foreign path
return None
target_range = npm_range or _repo_npm_range()
if target_range and not upgrade_managed_npm(
managed_npm, target_range, prefix=prefix, quiet=quiet
):
return None
return managed_npm
def maybe_repair_npm_engine(
npm: str | None,
output: str,
@@ -315,13 +158,13 @@ def maybe_repair_npm_engine(
"""Repair an ``EBADENGINE`` failure, never touching a foreign toolchain.
*output* is the combined stdout/stderr of the npm command that just failed.
Returns the npm executable the caller should retry its command with —
the same *npm* after an in-place upgrade of a Hermes-managed install, or
a freshly provisioned managed npm when the failing npm belongs to the
user (system / nvm / brew / Nix installs are never modified). Returns
``None`` when no repair happened — not an engine failure, a Node mismatch
a managed npm upgrade cannot fix, or a failed upgrade/bootstrap — leaving
the original failure to stand.
Returns the npm executable the caller should retry its command with — the
pm-pinned npm, freshly ensured, when the failing npm was a foreign install
(system / nvm / brew / Nix installs are never modified). Returns ``None``
when no repair happened — not an engine failure, the failing npm already
was pm's own (the lockfile pin is out of range; a runtime install cannot
fix that), or the pm install failed — leaving the original failure to
stand.
The returned value is truthy exactly when the caller should retry once,
so ``if maybe_repair_npm_engine(...)`` call sites keep working; they just
@@ -330,25 +173,16 @@ def maybe_repair_npm_engine(
if not npm or not is_ebadengine(output):
return None
npm_range = required_npm_range(output)
prefix = managed_npm_prefix(npm)
if prefix is not None:
# Hermes owns this npm — upgrade it in place. Only an npm-range
# failure is fixable this way; a Node mismatch needs a Node upgrade.
if not npm_range:
return None
if upgrade_managed_npm(npm, npm_range, prefix=prefix, quiet=quiet):
return npm
return None
# Foreign npm (system / nvm / brew / Nix): provision our own runtime
# instead. This also covers Node-version mismatches — the managed tree
# ships a Node the repo supports.
managed = _provision_managed_npm(npm_range, quiet=quiet)
managed = _pm_npm(quiet=quiet)
if managed:
return managed
try:
already_managed = Path(managed).resolve() == Path(npm).resolve()
except OSError:
already_managed = False
if not already_managed:
return managed
npm_range = required_npm_range(output)
if not quiet and npm_range:
_print_manual_fix(npm, npm_range, actual_npm_version(output))
return None
+5 -5
View File
@@ -353,7 +353,7 @@ def _read_manifest(plugin_dir: Path) -> dict:
try:
import yaml
with open(manifest_file, encoding="utf-8") as f:
with open(manifest_file, encoding="utf-8-sig") as f:
return yaml.safe_load(f) or {}
except Exception as e:
logger.warning("Failed to read plugin.yaml in %s: %s", plugin_dir, e)
@@ -510,7 +510,7 @@ def _display_after_install(plugin_dir: Path, identifier: str) -> None:
after_install = plugin_dir / "after-install.md"
if after_install.exists():
content = after_install.read_text(encoding="utf-8")
content = after_install.read_text(encoding="utf-8-sig")
md = Markdown(content)
console.print()
console.print(Panel(md, border_style="green", expand=False))
@@ -570,7 +570,7 @@ def _read_install_metadata() -> dict[str, dict[str, object]]:
if not path.exists():
return {}
try:
value = json.loads(path.read_text(encoding="utf-8"))
value = json.loads(path.read_text(encoding="utf-8-sig"))
except (OSError, json.JSONDecodeError) as exc:
raise PluginOperationError(f"Could not read plugin install metadata: {exc}") from exc
if not isinstance(value, dict):
@@ -1779,7 +1779,7 @@ def _read_manifest_info(d: Path, prefix: str):
description = ""
if yaml:
try:
with open(manifest_file, encoding="utf-8") as f:
with open(manifest_file, encoding="utf-8-sig") as f:
manifest = yaml.safe_load(f) or {}
name = manifest.get("name", d.name)
version = manifest.get("version", "")
@@ -1823,7 +1823,7 @@ def _bundled_default_on(dir_path) -> bool:
try:
import yaml
with open(manifest_file, encoding="utf-8") as f:
with open(manifest_file, encoding="utf-8-sig") as f:
manifest = yaml.safe_load(f) or {}
kind = str(manifest.get("kind", "standalone")).strip().lower()
return kind in _BUNDLED_DEFAULT_ON_KINDS
+11 -19
View File
@@ -95,26 +95,18 @@ DEFAULT_DIST_OWNED: Tuple[str, ...] = (
)
# Paths that are NEVER part of a distribution. These are user-owned and are
# protected on update. Must stay consistent with
# ``profiles.py::_DEFAULT_EXPORT_EXCLUDE_ROOT`` plus the ``local/``
# convention for user customizations.
USER_OWNED_EXCLUDE: frozenset = frozenset({
# Credentials & runtime secrets
"auth.json", ".env",
# Databases & runtime state
"state.db", "state.db-shm", "state.db-wal",
"hermes_state.db", "response_store.db",
"response_store.db-shm", "response_store.db-wal",
"gateway.pid", "gateway_state.json", "processes.json",
"auth.lock", "active_profile", ".update_check",
"errors.log", ".hermes_history",
# User data
"memories", "sessions", "logs", "plans", "workspace", "home",
"image_cache", "audio_cache", "document_cache",
"browser_screenshots", "checkpoints", "sandboxes",
# protected on update. This is NOT a third copy of the exclude list: it is
# the export-side authority (``profiles.py::DEFAULT_EXPORT_EXCLUDE_ROOT``)
# plus the distribution-specific additions — user data dirs an export archive
# may carry (memories, sessions, ...) but a distribution payload must never
# shadow, and the ``local/`` convention for user customizations.
from hermes_cli.profiles import DEFAULT_EXPORT_EXCLUDE_ROOT
USER_OWNED_EXCLUDE: frozenset = DEFAULT_EXPORT_EXCLUDE_ROOT | frozenset({
# User data a distribution payload must never shadow (an export archive
# intentionally carries these; an install/update must not overwrite)
"memories", "sessions", "plans", "workspace", "home",
"backups", "cache",
# Infrastructure
"hermes-agent", ".worktrees", "profiles", "bin", "node_modules",
# User customization namespace
"local",
})
+6 -1
View File
@@ -209,7 +209,12 @@ def _clone_all_copytree_ignore(source_dir: Path):
# The default profile contains infrastructure (repo checkout, worktrees, DBs,
# caches, binaries) that named profiles don't have. We exclude those so the
# export is a portable, reasonable-size archive of actual profile data.
_DEFAULT_EXPORT_EXCLUDE_ROOT = frozenset({
# The ONE exclude-list authority for "infrastructure / runtime state that is
# not profile data". The export path (``_default_export_ignore``) consumes it
# directly; the distribution path (``profile_distribution.USER_OWNED_EXCLUDE``)
# layers its distribution-specific additions on top of it. Keep new root
# artifacts here — not in a downstream copy.
_DEFAULT_EXPORT_EXCLUDE_ROOT = DEFAULT_EXPORT_EXCLUDE_ROOT = frozenset({
# Infrastructure
"hermes-agent", # repo checkout (multi-GB)
".worktrees", # git worktrees
+6 -5
View File
@@ -922,7 +922,7 @@ def _read_nearest_vercel_project(start: Path | None = None) -> dict[str, str]:
if not project_file.exists():
continue
try:
data = json.loads(project_file.read_text(encoding="utf-8"))
data = json.loads(project_file.read_text(encoding="utf-8-sig"))
except (OSError, json.JSONDecodeError):
return {}
if not isinstance(data, dict):
@@ -1661,19 +1661,20 @@ def setup_terminal_backend(config: dict):
print_info("Installing vercel SDK...")
import subprocess
# Managed uv first: $HERMES_HOME/bin is never on PATH, so a bare
# which() misses the uv Hermes installed. Bootstrapping one is
# Managed uv first: the store is never on PATH, so a bare
# which() misses the uv Hermes installed. Realizing one is
# welcome here — this is the interactive setup wizard, already
# mid-install, and the alternative tier is a pip that a `uv venv`
# venv may not even have.
from hermes_cli.managed_uv import ensure_uv
import pm
uv_bin = ensure_uv()
uv_bin, uv_env = pm.uv()
if uv_bin:
result = subprocess.run(
[uv_bin, "pip", "install", "--python", sys.executable, "vercel"],
capture_output=True,
text=True,
env=uv_env,
)
else:
result = subprocess.run(
+5 -6
View File
@@ -875,17 +875,16 @@ def _pip_install(
(or the last failure for the caller to inspect).
"""
venv_root = Path(sys.executable).parent.parent
uv_env = {**os.environ, "VIRTUAL_ENV": str(venv_root)}
# Managed uv first: $HERMES_HOME/bin is never on PATH, so a bare which()
# Managed uv first: the store is never on PATH, so a bare which()
# misses the uv Hermes installed and prefers a system one when both exist.
# ensure_uv() rather than a pure lookup because this runs during setup,
# pm realizes uv rather than a pure lookup because this runs during setup,
# where installing uv is in scope — and tier 2 is a pip that the Windows
# installer's `uv venv` does not seed, so failing to find uv here is the
# difference between a working post-setup hook and "No module named pip".
from hermes_cli.managed_uv import ensure_uv
import pm
uv_bin = ensure_uv()
uv_bin, uv_env = pm.uv(venv=venv_root)
if uv_bin:
try:
result = subprocess.run(
@@ -1434,7 +1433,7 @@ def _clear_stale_cua_install_lock() -> None:
holder_pid = None
info = lock_dir / "info"
try:
for line in info.read_text(encoding="utf-8", errors="replace").splitlines():
for line in info.read_text(encoding="utf-8-sig", errors="replace").splitlines():
if line.startswith("pid="):
holder_pid = int(line.split("=", 1)[1].strip())
break
+272
View File
@@ -0,0 +1,272 @@
"""Truthful derived build-version metadata for user-facing Hermes displays.
``__version__`` remains the package/API version. This module adds a display
suffix only when it can prove the number of commits since that release.
Resolution order:
1. Install stamp (``install-stamp.json``) — written at build time by
``scripts/write_install_stamp.py`` for every packager (Docker, Nix, and
the desktop app). The stamp is authoritative
for packaged builds.
2. Live git — for source/dev installs with a ``.git`` directory.
3. Unknown — no stamp and no git. The provenance is unknown.
"""
from __future__ import annotations
import json
import os
import subprocess
from dataclasses import dataclass
from pathlib import Path
from typing import Literal, cast
from hermes_cli import __release_date__, __version__
@dataclass(frozen=True)
class VersionInfo:
base_version: str
derived_version: str
distance: int | None
commit: str | None
branch: str | None
source: Literal["build", "ci", "docker", "fallback", "git", "local", "nix", "unknown"]
dirty: bool = False
commit_date: int | None = None
distribution: Literal["docker", "nix", "desktop-app"] | None = None
def _derived_version(base_version: str, distance: int | None, dirty: bool = False) -> str:
if distance and distance > 0:
return f"{base_version}+{distance}"
if dirty and distance is None:
return f"{base_version}+?"
return base_version
def _run_git(repo_dir: Path, *args: str) -> str | None:
try:
result = subprocess.run(
["git", *args], capture_output=True, text=True, timeout=3, cwd=str(repo_dir)
)
except (OSError, subprocess.SubprocessError):
return None
value = (result.stdout or "").strip()
return value if result.returncode == 0 and value else None
def _resolve_repo_dir() -> Path | None:
"""Use the executing checkout before a profile's optional clone."""
repo_dir = Path(__file__).parent.parent.resolve()
if (repo_dir / ".git").exists():
return repo_dir
try:
from hermes_constants import get_hermes_home
candidate = get_hermes_home() / "hermes-agent"
if (candidate / ".git").exists():
return candidate
except Exception:
pass
return None
def _parse_nonnegative(value: str | None) -> int | None:
try:
parsed = int(value or "")
except ValueError:
return None
return parsed if parsed >= 0 else None
# --- Install stamp reader ---------------------------------------------------
# The stamp file lives at the install root: beside the code in source
# checkouts and Docker (which writes it to the project root), and in the
# artifact's resources dir for sealed installs — whose processes carry
# HERMES_INSTALL_ROOT (the Nix wrapper points it at the store path's
# share/hermes-agent, where the stamp is baked). One resolution path for
# every steward; no stamp-specific env override.
_CODE_ROOT = Path(__file__).parent.parent
def _resolve_stamp_file() -> Path | None:
root = os.environ.get("HERMES_INSTALL_ROOT")
base = Path(root) if root else _CODE_ROOT
p = base / "install-stamp.json"
return p if p.is_file() else None
def _stamp_version_info() -> VersionInfo | None:
"""Read provenance from a build-time install stamp."""
stamp_file = _resolve_stamp_file()
if stamp_file is None:
return None
try:
raw = stamp_file.read_text(encoding="utf-8-sig")
data = json.loads(raw)
except (OSError, json.JSONDecodeError):
return None
if not isinstance(data, dict) or "commit" not in data:
return None
# updateMechanism is required in every stamp. A stamp without it means
# the writing build lane must be fixed, not tolerated.
if data.get("updateMechanism") not in ("self", "electron-updater", "external"):
raise RuntimeError(
f"install-stamp.json at {stamp_file} is missing a valid "
"'updateMechanism' (one of self, electron-updater, external). The "
"build lane that wrote this stamp must pass --update-mechanism to "
"scripts/write_install_stamp.py (or bake the field directly)."
)
# A light artifact ships no Python runtime. Reading a light stamp from
# a Python process means the artifact was mispackaged.
if data.get("payload") == "light":
raise RuntimeError(
f"install-stamp.json at {stamp_file} marks this artifact as 'light' "
"(no agent runtime). No Python process can legitimately run from a "
"light artifact — this build is mispackaged."
)
commit = data.get("commit") or None
if not commit or set(commit) == {"0"}:
# All-zero placeholder = fallback stamp, not real provenance.
return None
base_version = data.get("baseVersion") or __version__
display_version = data.get("displayVersion") or base_version
distance = data.get("distance")
if isinstance(distance, str):
distance = _parse_nonnegative(distance)
# ``source`` describes build provenance, while ``distribution`` identifies
# the package form users installed. Keep both facts intact for support.
stamp_source = str(data.get("source") or "")
source = (
cast(Literal["build", "ci", "docker", "fallback", "git", "local", "nix", "unknown"], stamp_source)
if stamp_source in {"ci", "docker", "fallback", "local", "nix"}
else "build"
)
distribution = data.get("distribution")
if distribution not in {"docker", "nix", "desktop-app"}:
distribution = None
commit_date = data.get("commitDate")
if not isinstance(commit_date, int):
commit_date = None
return VersionInfo(
base_version,
display_version,
distance if isinstance(distance, int) else None,
commit,
data.get("branch") or None,
source,
bool(data.get("dirty")),
commit_date,
distribution,
)
# --- Git provenance (source/dev installs) -----------------------------------
def _git_version_info(repo_dir: Path) -> VersionInfo:
commit = _run_git(repo_dir, "rev-parse", "HEAD")
# A detached HEAD has no branch. Leave the field None: every formatter
# already prints the commit separately and handles a missing branch.
branch = _run_git(repo_dir, "branch", "--show-current")
commit_date_raw = _run_git(repo_dir, "log", "-1", "--format=%ct", "HEAD")
commit_date: int | None = None
if commit_date_raw and commit_date_raw.isdigit():
commit_date = int(commit_date_raw)
try:
# -uno: skip the untracked-file scan. This runs on the startup-banner
# path, and a full working-tree walk costs real time on large or cold
# checkouts. Same semantics as write_install_stamp.py.
dirty_result = subprocess.run(
["git", "status", "--porcelain", "-uno"],
capture_output=True,
text=True,
timeout=3,
cwd=str(repo_dir),
)
dirty = dirty_result.returncode == 0 and bool((dirty_result.stdout or "").strip())
except (OSError, subprocess.SubprocessError):
dirty = False
# New releases are SemVer tags. The release-date fallback lets existing
# CalVer-tagged releases display a correct distance during the transition.
distance = None
for tag in (f"v{__version__}", f"v{__release_date__}"):
raw_distance = _run_git(repo_dir, "rev-list", "--count", f"{tag}..HEAD")
parsed_distance = _parse_nonnegative(raw_distance)
if parsed_distance is not None:
distance = parsed_distance
break
return VersionInfo(
__version__, _derived_version(__version__, distance, dirty), distance, commit, branch, "git", dirty, commit_date
)
# --- Cache + public API -----------------------------------------------------
_cached_version_info: VersionInfo | None = None
def _reset_version_info_cache() -> None:
"""Test-only cache reset."""
global _cached_version_info
_cached_version_info = None
def get_version_info() -> VersionInfo:
"""Return cached provenance from install stamp, git, or unknown."""
global _cached_version_info
if _cached_version_info is not None:
return _cached_version_info
# 1. Install stamp (packaged builds: Docker, Nix)
info = _stamp_version_info()
# 2. Live git (source/dev installs)
if info is None:
repo_dir = _resolve_repo_dir()
if repo_dir is not None:
info = _git_version_info(repo_dir)
# 3. Unknown — no stamp, no git
if info is None:
info = VersionInfo(__version__, __version__, None, None, None, "unknown")
_cached_version_info = info
return info
def get_code_identity(refresh: bool = False) -> dict:
"""Return the running install's code identity as a flat dict.
Shape: ``{"sha": full sha | None, "short_sha": str | None, "version":
base package version | None, "source": str}`` — what the update
receipt, runtime inventory, and gateway status stamping consume.
Backed by :func:`get_version_info` (install stamp first, live git
second, unknown third), so every consumer shares one resolution
policy. Returned dicts are fresh copies, never the shared cache.
``refresh=True`` drops the per-process cache first — the updater uses
it to re-read identity after swapping the checkout out from under the
running process.
"""
global _cached_version_info
if refresh:
_cached_version_info = None
info = get_version_info()
return {
"sha": info.commit,
"short_sha": info.commit[:8] if info.commit else None,
"version": info.base_version,
"source": info.source,
}
+15 -15
View File
@@ -151,7 +151,7 @@ def _process_start_marker(pid: int) -> str:
"""
if sys.platform == "linux":
try:
stat_line = Path(f"/proc/{pid}/stat").read_text(encoding="utf-8")
stat_line = Path(f"/proc/{pid}/stat").read_text(encoding="utf-8-sig")
except FileNotFoundError as exc:
raise ProcessLookupError(pid) from exc
@@ -3543,7 +3543,7 @@ def _read_or_create_install_id() -> Optional[str]:
root = get_default_hermes_root()
path = root / _INSTALL_ID_FILENAME
try:
existing = path.read_text(encoding="utf-8").strip().lower()
existing = path.read_text(encoding="utf-8-sig").strip().lower()
if _INSTALL_ID_RE.match(existing):
return existing
except FileNotFoundError:
@@ -6091,7 +6091,7 @@ def _read_flat_json(provider: ProviderConfigSchema) -> Dict[str, Any]:
if not path.exists():
return {}
try:
data = json.loads(path.read_text(encoding="utf-8"))
data = json.loads(path.read_text(encoding="utf-8-sig"))
except Exception:
_log.warning("Failed to read memory provider config from %s", path, exc_info=True)
return {}
@@ -6143,7 +6143,7 @@ def _honcho_read_sources() -> tuple[Dict[str, Any], str, Dict[str, Any]]:
raw: Dict[str, Any] = {}
if path.exists():
try:
loaded = json.loads(path.read_text(encoding="utf-8"))
loaded = json.loads(path.read_text(encoding="utf-8-sig"))
raw = loaded if isinstance(loaded, dict) else {}
except Exception:
_log.warning("Failed to read Honcho config from %s", path, exc_info=True)
@@ -6253,7 +6253,7 @@ def _write_provider_honcho(provider: ProviderConfigSchema, values: Dict[str, str
cfg: Dict[str, Any] = {}
if path.exists():
try:
loaded = json.loads(path.read_text(encoding="utf-8"))
loaded = json.loads(path.read_text(encoding="utf-8-sig"))
cfg = loaded if isinstance(loaded, dict) else {}
except Exception:
_log.warning("Failed to read Honcho config from %s", path, exc_info=True)
@@ -6786,7 +6786,7 @@ def _read_json_file(path: Path) -> Dict[str, Any]:
if not path.exists():
return {}
try:
data = json.loads(path.read_text(encoding="utf-8"))
data = json.loads(path.read_text(encoding="utf-8-sig"))
except Exception:
_log.debug("Failed to read JSON config from %s", path, exc_info=True)
return {}
@@ -9749,7 +9749,7 @@ def _whatsapp_phone_from_identifier(value: Any) -> str | None:
def _whatsapp_linked_account_from_session(session_path: Path) -> tuple[str | None, str | None, str | None]:
creds_path = session_path / "creds.json"
try:
payload = json.loads(creds_path.read_text(encoding="utf-8"))
payload = json.loads(creds_path.read_text(encoding="utf-8-sig"))
except Exception:
return None, None, None
@@ -13464,7 +13464,7 @@ def _profile_env_value(home: Path, key: str) -> str:
env_path = home / ".env"
if not env_path.is_file():
return ""
for line in env_path.read_text(encoding="utf-8").splitlines():
for line in env_path.read_text(encoding="utf-8-sig").splitlines():
line = line.strip()
if not line or line.startswith("#") or "=" not in line:
continue
@@ -13628,7 +13628,7 @@ def _gateway_intentionally_stopped(profile: Optional[str]) -> bool:
state_file = home / "gateway_state.json"
if not state_file.exists():
return False
data = _json.loads(state_file.read_text(encoding="utf-8"))
data = _json.loads(state_file.read_text(encoding="utf-8-sig"))
if not isinstance(data, dict):
return False
return data.get("desired_state") == "stopped"
@@ -15740,7 +15740,7 @@ async def get_config_raw(profile: Optional[str] = None):
path = get_config_path()
if not path.exists():
return {"yaml": "", "path": str(path)}
return {"yaml": path.read_text(encoding="utf-8"), "path": str(path)}
return {"yaml": path.read_text(encoding="utf-8-sig"), "path": str(path)}
return await asyncio.to_thread(_run)
@@ -16901,7 +16901,7 @@ def _active_session_file_for_channel(app: "FastAPI", channel: str) -> Path:
def _read_active_session_file(path: Path) -> Optional[str]:
try:
data = json.loads(path.read_text(encoding="utf-8"))
data = json.loads(path.read_text(encoding="utf-8-sig"))
except (OSError, json.JSONDecodeError):
return None
@@ -17974,7 +17974,7 @@ def mount_spa(application: FastAPI):
auth scheme for /api/pty and /api/ws (ticket vs token).
"""
try:
html = _index_path.read_text(encoding="utf-8")
html = _index_path.read_text(encoding="utf-8-sig")
except OSError:
# The dist dir existed at mount time but index.html is missing or
# unreadable now (partial build, wiped dist, permissions). Without
@@ -18044,7 +18044,7 @@ def mount_spa(application: FastAPI):
):
return JSONResponse({"error": "not found"}, status_code=404)
prefix = _normalise_prefix(request.headers.get("x-forwarded-prefix"))
css = css_path.read_text(encoding="utf-8")
css = css_path.read_text(encoding="utf-8-sig")
if prefix:
for asset_dir in ("/fonts/", "/fonts-terminal/", "/ds-assets/", "/assets/"):
css = css.replace(f"url({asset_dir}", f"url({prefix}{asset_dir}")
@@ -18357,7 +18357,7 @@ def _discover_user_themes() -> list:
result = []
for f in sorted(themes_dir.glob("*.yaml")):
try:
data = yaml.safe_load(f.read_text(encoding="utf-8"))
data = yaml.safe_load(f.read_text(encoding="utf-8-sig"))
except Exception:
continue
normalised = _normalise_theme_definition(data)
@@ -18569,7 +18569,7 @@ def _discover_dashboard_plugins() -> list:
if not manifest_file.exists():
continue
try:
data = json.loads(manifest_file.read_text(encoding="utf-8"))
data = json.loads(manifest_file.read_text(encoding="utf-8-sig"))
name = data.get("name", child.name)
if name in seen_names:
continue
+1 -1
View File
@@ -32,7 +32,7 @@
let
repoRoot = ./..;
npm12 = callPackage ./npm-12-0-2.nix { };
npm12 = callPackage ./npm-pinned.nix { };
node_gyp_11_4_0 = callPackage ./node-gyp-11-4-0.nix { };
nodejs_26_npm_12 = symlinkJoin {
name = "nodejs-26-npm-12";
-29
View File
@@ -1,29 +0,0 @@
{
stdenv,
makeWrapper,
fetchurl,
nodejs_26,
}:
stdenv.mkDerivation rec {
pname = "npm";
version = "12.0.2";
src = fetchurl {
url = "https://registry.npmjs.org/npm/-/npm-${version}.tgz";
hash = "sha256-XbuGxx0HoZV/LpBzQJLdali9zZ68LY1ByhxuaiHTZOE=";
};
nativeBuildInputs = [ makeWrapper ];
dontBuild = true;
installPhase = ''
mkdir -p $out/lib/npm12
cp -r . $out/lib/npm12/
mkdir -p $out/bin
makeWrapper ${nodejs_26}/bin/node $out/bin/npm \
--add-flags "$out/lib/npm12/bin/npm-cli.js"
makeWrapper ${nodejs_26}/bin/node $out/bin/npx \
--add-flags "$out/lib/npm12/bin/npx-cli.js"
'';
}
+36
View File
@@ -0,0 +1,36 @@
{
stdenv,
makeWrapper,
fetchurl,
nodejs_26,
}:
let
# pm/lock.json is the one pin table. Every artifact carries its resolved
# url + sha256, so nix consumes it as pure data — no version or url
# knowledge lives on the nix side.
lock = builtins.fromJSON (builtins.readFile ../pm/lock.json);
pin = lock.packages.npm;
in
stdenv.mkDerivation {
pname = "npm";
version = pin.version;
src = fetchurl {
url = pin.artifacts.any.url;
sha256 = pin.artifacts.any.sha256;
};
nativeBuildInputs = [ makeWrapper ];
dontBuild = true;
installPhase = ''
mkdir -p $out/lib/npm
cp -r . $out/lib/npm/
mkdir -p $out/bin
makeWrapper ${nodejs_26}/bin/node $out/bin/npm \
--add-flags "$out/lib/npm/bin/npm-cli.js"
makeWrapper ${nodejs_26}/bin/node $out/bin/npx \
--add-flags "$out/lib/npm/bin/npx-cli.js"
'';
}
+69
View File
@@ -0,0 +1,69 @@
{
lib,
stdenv,
fetchurl,
unzip,
}:
# The thin translation layer: pm/lock.json -> one fetched, unpacked
# derivation per package for this nix system. Nothing here knows versions,
# urls, or hashes — the lockfile is the complete machine interface, the
# same one `pm install` and `pm bundle` consume.
#
# Consumers pick what they need: (callPackage ./pm-packages.nix { }).ripgrep
# Packages with no artifact for this system are simply absent.
let
lock = builtins.fromJSON (builtins.readFile ../pm/lock.json);
target =
let
arch = if stdenv.hostPlatform.isAarch64 then "arm64" else "x64";
os =
if stdenv.hostPlatform.isDarwin then "darwin"
else if stdenv.hostPlatform.isLinux then "linux"
else "win32";
in
"${os}-${arch}";
# A target pins one artifact or a list of them (a runtime split across
# archives that must land in one directory). Normalize to a list.
artifactsFor =
pin:
let
found = pin.artifacts.${target} or pin.artifacts.any or null;
in
if found == null then null else if builtins.isList found then found else [ found ];
derive = name: pin: artifacts:
stdenv.mkDerivation {
pname = name;
version = pin.version;
srcs = map (artifact: fetchurl {
url = artifact.url;
sha256 = artifact.sha256;
}) artifacts;
# pm's store publishes the unpacked tree; mirror that shape. Several
# archives unpack over one another into the same root, exactly as
# pm merges them into one store entry.
sourceRoot = ".";
nativeBuildInputs =
lib.optional (lib.any (a: lib.hasSuffix ".zip" a.url) artifacts) unzip;
dontBuild = true;
dontConfigure = true;
installPhase = ''
mkdir -p $out
cp -r . $out/
'';
};
in
lib.filterAttrs (_: v: v != null) (
lib.mapAttrs (
name: pin:
let
artifacts = artifactsFor pin;
in
if artifacts == null then null else derive name pin artifacts
) lock.packages
)
+54 -40
View File
@@ -71,7 +71,7 @@ class _RecallResult:
_DEFAULT_API_URL = "https://api.hindsight.vectorize.io"
_DEFAULT_LOCAL_URL = "http://localhost:8888"
# Keep in sync with tools/lazy_deps.py ("memory.hindsight") and plugin.yaml.
# Keep in sync with the pyproject "hindsight" extra and plugin.yaml.
_MIN_CLIENT_VERSION = "0.6.1"
_DEFAULT_TIMEOUT = 120 # seconds — cloud API can take 30-40s per request
_DEFAULT_IDLE_TIMEOUT = 300 # seconds — Hindsight embedded daemon default
@@ -204,8 +204,8 @@ def _local_runtime_hint(reason: str | None) -> str:
def _ensure_cloud_client_dependency() -> None:
"""Install the Hindsight cloud client lazily before importing it."""
try:
from tools.lazy_deps import ensure as _lazy_ensure
_lazy_ensure("memory.hindsight", prompt=False)
from pm import ensure_import as _lazy_ensure
_lazy_ensure("hindsight")
except ImportError:
pass
except Exception as exc:
@@ -431,7 +431,7 @@ def _load_config() -> dict:
profile_path = get_hermes_home() / "hindsight" / "config.json"
if profile_path.exists():
try:
return json.loads(profile_path.read_text(encoding="utf-8"))
return json.loads(profile_path.read_text(encoding="utf-8-sig"))
except Exception:
pass
@@ -439,7 +439,7 @@ def _load_config() -> dict:
legacy_path = Path.home() / ".hindsight" / "config.json"
if legacy_path.exists():
try:
return json.loads(legacy_path.read_text(encoding="utf-8"))
return json.loads(legacy_path.read_text(encoding="utf-8-sig"))
except Exception:
pass
@@ -937,7 +937,7 @@ class HindsightMemoryProvider(MemoryProvider):
existing = {}
if config_path.exists():
try:
existing = json.loads(config_path.read_text(encoding="utf-8"))
existing = json.loads(config_path.read_text(encoding="utf-8-sig"))
except Exception:
pass
existing.update(values)
@@ -981,15 +981,10 @@ class HindsightMemoryProvider(MemoryProvider):
provider_config["mode"] = mode
env_writes: dict = {}
# Step 2: Install/upgrade deps for selected mode
cloud_dep = f"hindsight-client>={_MIN_CLIENT_VERSION}"
local_dep = "hindsight-all"
if mode == "local_embedded":
deps_to_install = [local_dep]
elif mode == "local_external":
deps_to_install = [cloud_dep]
else:
deps_to_install = [cloud_dep]
# Step 2: Install/upgrade deps for selected mode. local_embedded
# installs hindsight-all itself (protobuf conflict bars it from the
# venv extras; side-venv treatment is planned).
extras_needed = [] if mode == "local_embedded" else ["hindsight"]
llm_provider = ""
if mode == "local_embedded":
@@ -1013,18 +1008,41 @@ class HindsightMemoryProvider(MemoryProvider):
provider_config["llm_provider"] = llm_provider
print("\n Checking dependencies...")
# Environment-aware install: sealed hosted venvs redirect to the durable
# data-volume target instead of writing to /opt/hermes (NS-605).
from tools.lazy_deps import install_specs
if mode == "local_embedded":
# hindsight-all cannot live in the hermes venv (its protobuf
# floor conflicts with mem0/modal; side-venv treatment is
# planned). Keep the direct install for this mode only.
import subprocess as _sp
outcome = install_specs(deps_to_install, timeout=120)
if outcome.ok:
print(" ✓ Dependencies up to date")
elif outcome.blocked:
print(f" ⚠ Cannot install dependencies: {outcome.reason}")
import pm as _pm
uv_bin, uv_env = _pm.uv(venv=Path(sys.prefix))
cmd = (
[uv_bin, "pip", "install"]
if uv_bin
else [sys.executable, "-m", "pip", "install"]
)
proc = _sp.run(
[*cmd, "hindsight-all"],
env=uv_env if uv_bin else None,
capture_output=True,
text=True,
timeout=600,
)
if proc.returncode == 0:
print(" ✓ Dependencies up to date")
else:
print(f" ⚠ Install failed:\n{(proc.stderr or '').strip()[:400]}")
print(" Run manually: uv pip install hindsight-all")
else:
print(f" ⚠ Install failed:\n{(outcome.stderr or '').strip()}")
print(f" Run manually: uv pip install --python {sys.executable} {' '.join(deps_to_install)}")
import pm
try:
pm.sync_venv(extras_needed, explicit=True)
print(" ✓ Dependencies up to date")
except Exception as exc:
print(f" ⚠ Install failed: {exc}")
print(" Run manually: hermes pm install")
# Step 3: Mode-specific config
if mode == "cloud":
@@ -1142,7 +1160,7 @@ class HindsightMemoryProvider(MemoryProvider):
materialized_config = dict(provider_config)
config_path = Path(hermes_home) / "hindsight" / "config.json"
try:
materialized_config = json.loads(config_path.read_text(encoding="utf-8"))
materialized_config = json.loads(config_path.read_text(encoding="utf-8-sig"))
except Exception:
pass
@@ -1241,8 +1259,8 @@ class HindsightMemoryProvider(MemoryProvider):
+ (f": {reason}" if reason else "")
)
try:
from tools.lazy_deps import ensure as _lazy_ensure
_lazy_ensure("memory.hindsight", prompt=False)
from pm import ensure_import as _lazy_ensure
_lazy_ensure("hindsight")
except ImportError:
pass
except Exception as _e:
@@ -1608,18 +1626,14 @@ class HindsightMemoryProvider(MemoryProvider):
if Version(installed) < Version(_MIN_CLIENT_VERSION):
logger.warning("hindsight-client %s is outdated (need >=%s), attempting upgrade...",
installed, _MIN_CLIENT_VERSION)
# Environment-aware install: sealed hosted venvs redirect to the
# durable data-volume target instead of /opt/hermes (NS-605).
from tools.lazy_deps import install_specs
outcome = install_specs([f"hindsight-client>={_MIN_CLIENT_VERSION}"], timeout=120)
if outcome.ok:
logger.info("hindsight-client upgraded to >=%s", _MIN_CLIENT_VERSION)
elif outcome.blocked:
logger.warning("Auto-upgrade unavailable: %s. Run: uv pip install 'hindsight-client>=%s'",
outcome.reason, _MIN_CLIENT_VERSION)
else:
logger.warning("Auto-upgrade failed: %s. Run: uv pip install 'hindsight-client>=%s'",
(outcome.stderr or "").strip() or "install error", _MIN_CLIENT_VERSION)
# uv.lock owns the pin; an upgrade is a resync of the extra.
import pm
try:
pm.sync_venv(["hindsight"])
logger.info("hindsight-client resynced against uv.lock")
except Exception as exc:
logger.warning("Auto-upgrade unavailable: %s. Run: hermes pm install", exc)
except Exception:
pass # packaging not available or other issue — proceed anyway
+1 -2
View File
@@ -1,8 +1,7 @@
name: hindsight
version: 1.0.0
description: "Hindsight — long-term memory with knowledge graph, entity resolution, and multi-strategy retrieval."
pip_dependencies:
- "hindsight-client>=0.6.1"
extra: hindsight
requires_env: []
hooks:
- on_session_end
+9 -14
View File
@@ -264,7 +264,7 @@ def _read_config() -> dict:
path = _config_path()
if path.exists():
try:
return json.loads(path.read_text(encoding="utf-8"))
return json.loads(path.read_text(encoding="utf-8-sig"))
except Exception:
pass
return {}
@@ -516,20 +516,15 @@ def _ensure_sdk_installed() -> bool:
return False
print(" Installing honcho-ai...", flush=True)
# Environment-aware install: sealed hosted venvs redirect to the durable
# data-volume target instead of writing to /opt/hermes (NS-605).
from tools.lazy_deps import install_specs
import pm
result = install_specs(["honcho-ai==2.2.0"])
if result.ok:
try:
pm.sync_venv(["honcho"], explicit=True)
print(" Installed.\n")
return True
elif result.blocked:
print(f" Cannot install: {result.reason}\n")
return False
else:
print(f" Install failed:\n{(result.stderr or '').strip()}")
print(" Run manually: uv pip install 'honcho-ai==2.2.0'\n")
except Exception as exc:
print(f" Install failed: {exc}")
print(" Run manually: hermes pm install\n")
return False
@@ -1587,7 +1582,7 @@ def cmd_identity(args) -> None:
print(f" File not found: {p}\n")
return
content = p.read_text(encoding="utf-8").strip()
content = p.read_text(encoding="utf-8-sig").strip()
if not content:
print(f" File is empty: {p}\n")
return
@@ -1767,7 +1762,7 @@ def cmd_migrate(args) -> None:
session_key = hcfg.resolve_session_name()
mgr.get_or_create(session_key)
for f in agent_files:
content = f.read_text(encoding="utf-8").strip()
content = f.read_text(encoding="utf-8-sig").strip()
if content:
ok = mgr.seed_ai_identity(session_key, content, source=f.name)
status = "seeded" if ok else "failed"
+9 -9
View File
@@ -102,7 +102,7 @@ def resolve_active_host() -> str:
try:
path = resolve_config_path()
if path.exists():
raw = json.loads(path.read_text(encoding="utf-8"))
raw = json.loads(path.read_text(encoding="utf-8-sig"))
default_host = str(raw.get("defaultHost", "")).strip()
if default_host:
return default_host
@@ -552,7 +552,7 @@ class HonchoClientConfig:
return cls.from_env(host=resolved_host)
try:
raw = json.loads(path.read_text(encoding="utf-8"))
raw = json.loads(path.read_text(encoding="utf-8-sig"))
except (json.JSONDecodeError, OSError) as e:
logger.warning("Failed to read %s: %s, falling back to env", path, e)
return cls.from_env(host=resolved_host)
@@ -1004,7 +1004,7 @@ def _credential_fingerprint(config: HonchoClientConfig | None) -> str:
# the supported path for background threads).
path = resolve_config_path()
if path.exists():
raw = json.loads(path.read_text(encoding="utf-8"))
raw = json.loads(path.read_text(encoding="utf-8-sig"))
block = _host_block(raw, resolve_active_host())
oauth_block = block.get("oauth")
if isinstance(oauth_block, dict) and oauth_block.get("refreshToken"):
@@ -1124,7 +1124,7 @@ def _honcho_json_timeout() -> float | None:
timeout = None
if mtime_ns != -1:
raw = json.loads(path.read_text(encoding="utf-8"))
raw = json.loads(path.read_text(encoding="utf-8-sig"))
host_block = _host_block(raw, resolve_active_host())
timeout = _resolve_optional_float(
host_block.get("timeout"),
@@ -1255,14 +1255,14 @@ def get_honcho_client(config: HonchoClientConfig | None = None) -> Honcho:
def _build() -> "Honcho":
# Lazy dependency failures fall through to the canonical import error.
try:
from tools.lazy_deps import FeatureUnavailable, ensure as _lazy_ensure
_lazy_ensure("memory.honcho", prompt=False)
from pm import ensure_import as _lazy_ensure
_lazy_ensure("honcho")
except ImportError:
# lazy_deps module missing — fall through to the raw import below.
# pm module missing — fall through to the raw import below.
pass
except Exception:
# FeatureUnavailable or unexpected error. Don't crash here; let the
# actual import attempt produce the canonical error message.
# Lazy-install failure or unexpected error. Don't crash here; let
# the actual import attempt produce the canonical error message.
pass
try:
+1 -2
View File
@@ -1,7 +1,6 @@
name: honcho
version: 1.0.0
description: "Honcho AI-native memory — cross-session user modeling with dialectic Q&A, semantic search, and persistent conclusions."
pip_dependencies:
- honcho-ai
extra: honcho
hooks:
- on_session_end
+4 -4
View File
@@ -101,7 +101,7 @@ def _load_config() -> dict:
config_path = get_hermes_home() / "mem0.json"
if config_path.exists():
try:
file_cfg = json.loads(config_path.read_text(encoding="utf-8"))
file_cfg = json.loads(config_path.read_text(encoding="utf-8-sig"))
config.update({k: v for k, v in file_cfg.items()
if v is not None and v != ""})
except Exception:
@@ -241,7 +241,7 @@ class Mem0MemoryProvider(MemoryProvider):
existing = {}
if config_path.exists():
try:
existing = json.loads(config_path.read_text(encoding="utf-8"))
existing = json.loads(config_path.read_text(encoding="utf-8-sig"))
except Exception:
pass
existing.update(values)
@@ -271,8 +271,8 @@ class Mem0MemoryProvider(MemoryProvider):
# target. On failure we fall through so the import inside the backend
# produces the canonical error, captured below.
try:
from tools.lazy_deps import ensure as _lazy_ensure
_lazy_ensure("memory.mem0", prompt=False)
from pm import ensure_import as _lazy_ensure
_lazy_ensure("mem0")
except ImportError:
pass
except Exception:
+16 -21
View File
@@ -224,7 +224,7 @@ def _save_mem0_json(hermes_home: str, data: dict) -> None:
existing = {}
if config_path.exists():
try:
existing = json.loads(config_path.read_text(encoding="utf-8"))
existing = json.loads(config_path.read_text(encoding="utf-8-sig"))
except Exception:
pass
existing.update(data)
@@ -248,7 +248,7 @@ def _setup_platform(hermes_home: str, config: dict, flags: dict[str, str]) -> No
config_path = Path(hermes_home) / "mem0.json"
if config_path.exists():
try:
existing_config = json.loads(config_path.read_text(encoding="utf-8"))
existing_config = json.loads(config_path.read_text(encoding="utf-8-sig"))
except Exception:
pass
@@ -369,7 +369,7 @@ def _setup_selfhosted(hermes_home: str, config: dict, flags: dict[str, str]) ->
config_path = Path(hermes_home) / "mem0.json"
if config_path.exists():
try:
existing_config = json.loads(config_path.read_text(encoding="utf-8"))
existing_config = json.loads(config_path.read_text(encoding="utf-8-sig"))
except Exception:
pass
@@ -853,32 +853,27 @@ def _setup_oss_interactive(hermes_home: str, config: dict) -> None:
def _install_provider_deps(llm_id: str, embedder_id: str, vector_id: str) -> None:
"""Install all optional pip deps for selected providers."""
"""Point at the pip deps the selected OSS backends need.
These are third-party backend SDKs (ollama, qdrant-client, ...), not
hermes dependencies — pm does not install arbitrary specs into the
hermes venv. Print the exact command instead."""
deps: set[str] = set()
for registry, pid in [(LLM_PROVIDERS, llm_id), (EMBEDDER_PROVIDERS, embedder_id),
(VECTOR_PROVIDERS, vector_id)]:
dep = registry.get(pid, {}).get("pip_dep")
if dep:
deps.add(dep)
missing = []
for dep in sorted(deps):
try:
print(f" Installing {dep}...")
# Environment-aware install: sealed hosted venvs redirect to the
# durable data-volume target instead of /opt/hermes (NS-605).
from tools.lazy_deps import install_specs
import importlib.util
outcome = install_specs([dep], timeout=60)
if outcome.ok:
print(f" ✓ Installed {dep}")
elif outcome.blocked:
print(f" Warning: cannot install {dep}: {outcome.reason}")
else:
print(f" Warning: Could not install {dep}. Install manually: uv pip install {dep}")
except Exception:
print(f" Warning: Could not install {dep}. Install manually: uv pip install {dep}")
if deps:
import importlib
importlib.invalidate_caches()
if importlib.util.find_spec(dep.replace("-", "_").split("[")[0]) is None:
missing.append(dep)
if missing:
print("\n The selected backends need extra packages:")
print(f" uv pip install {' '.join(missing)}")
print(" Run that inside the hermes venv, then re-run setup.")
def _check_qdrant_path(path: str) -> tuple[bool, str]:
+1 -2
View File
@@ -1,5 +1,4 @@
name: mem0
version: 1.3.0
description: "Mem0 — server-side LLM fact extraction with semantic search, automatic deduplication, and opt-in reranking (platform mode)."
pip_dependencies:
- mem0ai>=2.0.10,<3
extra: mem0
-2
View File
@@ -1,8 +1,6 @@
name: openviking
version: 2.0.0
description: "OpenViking context database — session-managed memory with automatic extraction, tiered retrieval, and filesystem-style knowledge browsing."
pip_dependencies:
- httpx
requires_env: []
hooks:
- on_session_end
-2
View File
@@ -1,7 +1,5 @@
name: retaindb
version: 1.0.0
description: "RetainDB — cloud memory API with hybrid search and 7 memory types."
pip_dependencies:
- requests
requires_env:
- RETAINDB_API_KEY
+4 -4
View File
@@ -115,7 +115,7 @@ def _load_supermemory_config(hermes_home: str) -> dict:
config_path = Path(hermes_home) / "supermemory.json"
if config_path.exists():
try:
raw = json.loads(config_path.read_text(encoding="utf-8"))
raw = json.loads(config_path.read_text(encoding="utf-8-sig"))
if isinstance(raw, dict):
config.update({k: v for k, v in raw.items() if v is not None})
except Exception:
@@ -162,7 +162,7 @@ def _save_supermemory_config(values: dict, hermes_home: str) -> None:
existing = {}
if config_path.exists():
try:
raw = json.loads(config_path.read_text(encoding="utf-8"))
raw = json.loads(config_path.read_text(encoding="utf-8-sig"))
if isinstance(raw, dict):
existing = raw
except Exception:
@@ -286,8 +286,8 @@ class _SupermemoryClient:
# fall through so the raw import below produces the canonical
# ImportError message.
try:
from tools.lazy_deps import ensure as _lazy_ensure
_lazy_ensure("memory.supermemory", prompt=False)
from pm import ensure_import as _lazy_ensure
_lazy_ensure("supermemory")
except ImportError:
pass
except Exception:
+1 -2
View File
@@ -1,5 +1,4 @@
name: supermemory
version: 1.0.1
description: "Supermemory semantic long-term memory with profile recall, semantic search, explicit memory tools, and session ingest."
pip_dependencies:
- supermemory
extra: supermemory
+2 -2
View File
@@ -191,8 +191,8 @@ def ensure_dingtalk_deps() -> bool:
if DINGTALK_STREAM_AVAILABLE and HTTPX_AVAILABLE:
return True
try:
from tools.lazy_deps import ensure as _lazy_ensure
_lazy_ensure("platform.dingtalk", prompt=False)
from pm import ensure_import as _lazy_ensure
_lazy_ensure("dingtalk")
except Exception:
return False
try:
+5 -5
View File
@@ -356,7 +356,7 @@ class _DiscordNonConversationalMessageTracker:
if not path.exists():
return []
try:
data = json.loads(path.read_text(encoding="utf-8"))
data = json.loads(path.read_text(encoding="utf-8-sig"))
if isinstance(data, list):
return [str(message_id) for message_id in data if str(message_id).strip()]
except Exception:
@@ -493,7 +493,7 @@ def discord_deps_present() -> bool:
def check_discord_requirements() -> bool:
"""Check if Discord dependencies are available.
Lazy-installs discord.py via ``tools.lazy_deps.ensure("platform.discord")``
Lazy-installs discord.py via ``pm.ensure_import("discord")``
on first call if not present. After successful install, re-binds module
globals so ``DISCORD_AVAILABLE`` becomes True.
"""
@@ -501,8 +501,8 @@ def check_discord_requirements() -> bool:
if DISCORD_AVAILABLE:
return True
try:
from tools.lazy_deps import ensure as _lazy_ensure
_lazy_ensure("platform.discord", prompt=False)
from pm import ensure_import as _lazy_ensure
_lazy_ensure("discord")
except Exception:
return False
try:
@@ -2228,7 +2228,7 @@ class DiscordAdapter(BasePlatformAdapter):
path = self._command_sync_state_path()
if not path.exists():
return {}
data = json.loads(path.read_text(encoding="utf-8"))
data = json.loads(path.read_text(encoding="utf-8-sig"))
except Exception:
return {}
return data if isinstance(data, dict) else {}
+14 -15
View File
@@ -1000,8 +1000,8 @@ def matrix_deps_present() -> bool:
and runs from ``create_adapter()`` when this returns False (#79812).
"""
try:
from tools.lazy_deps import is_available
return is_available("platform.matrix")
from pm import available as is_available
return is_available("matrix")
except Exception: # pragma: no cover — defensive
return False
@@ -1034,22 +1034,20 @@ def ensure_matrix_deps() -> bool:
"""ACTIVE deps-only installer (registry ``ensure_deps_fn``).
Lazy-installs the full ``platform.matrix`` feature group via
``tools.lazy_deps.ensure_and_bind`` whenever any of the declared
``pm.extras.ensure_and_bind`` whenever any of the declared
packages (mautrix, Markdown, aiosqlite, asyncpg, aiohttp-socks) is
missing — not just mautrix itself. Previously this short-circuited on
``import mautrix``, which left the other four packages uninstalled
forever and broke E2EE connect with ``No module named 'asyncpg'``
(#31116). Rebinds module-level type globals on success.
"""
# Check whether any package in the platform.matrix feature group is
# missing. ``feature_missing`` is cheap (per-spec importlib.metadata
# lookups) and correctly handles ``mautrix[encryption]`` by stripping
# the extras marker before checking the bare package.
# Check every anchor of the matrix extra (mautrix alone is not enough:
# a partial install left asyncpg missing forever, #31116).
try:
from tools.lazy_deps import feature_missing, ensure_and_bind
missing = feature_missing("platform.matrix")
from pm.extras import missing as _extra_missing, ensure_and_bind
missing = _extra_missing("matrix")
except Exception as exc: # pragma: no cover — defensive
logger.debug("Matrix: lazy_deps lookup failed: %s", exc)
logger.debug("Matrix: extras lookup failed: %s", exc)
missing = ()
ensure_and_bind = None # type: ignore[assignment]
@@ -1075,12 +1073,12 @@ def ensure_matrix_deps() -> bool:
if ensure_and_bind is None:
return False
if not ensure_and_bind("platform.matrix", _import, globals(), prompt=False):
if not ensure_and_bind("matrix", _import, globals()):
logger.warning(
"Matrix: required packages not installed (%s). "
"Run: pip install 'mautrix[encryption]' asyncpg aiosqlite "
"Markdown aiohttp-socks",
", ".join(missing) if missing else "platform.matrix",
", ".join(missing) if missing else "matrix",
)
return False
@@ -5323,12 +5321,13 @@ def interactive_setup() -> None:
matrix_pkg = "mautrix[encryption]" if want_e2ee else "mautrix"
try:
from tools.lazy_deps import ensure as _lazy_ensure, feature_missing
_missing_before = feature_missing("platform.matrix")
from pm import ensure_import as _lazy_ensure
from pm.extras import missing as _extra_missing
_missing_before = _extra_missing("matrix")
if _missing_before:
print_info(f"Installing {matrix_pkg} (+ {len(_missing_before)} runtime deps)...")
try:
_lazy_ensure("platform.matrix", prompt=False)
_lazy_ensure("matrix")
print_success(f"{matrix_pkg} installed")
except Exception as exc:
print_warning(
+48 -14
View File
@@ -158,7 +158,7 @@ def _write_runtime_record(port: int, token: str, pid: int) -> None:
def _read_runtime_record() -> Optional[Dict[str, Any]]:
try:
raw = json.loads(_runtime_record_path().read_text(encoding="utf-8"))
raw = json.loads(_runtime_record_path().read_text(encoding="utf-8-sig"))
except (OSError, ValueError):
return None
return raw if isinstance(raw, dict) else None
@@ -418,16 +418,50 @@ def _is_timeout_error(exc: BaseException) -> bool:
return "timeout" in type(exc).__name__.lower()
def _node_command(command: str) -> Optional[str]:
"""Resolve a Node toolchain binary (node/npm), pm store first.
Photon's sidecar is Node/TypeScript, so the gateway runs node/npm on
the host. Resolution goes through ``hermes_constants.find_node_executable``
— the pm store's pinned node/npm wins whenever it is installed, with
PATH (Windows shim ordering) as the fallback — never a bare PATH
probe, which would miss the managed install (or resolve a system copy
Hermes does not own). Returns None when the binary is nowhere.
"""
try:
from hermes_constants import find_node_executable
resolved = find_node_executable(command)
if resolved:
return resolved
except Exception: # pragma: no cover — hermes_constants is always present
pass
# pm.ensure wiring: the store's pinned node/npm is the first choice, but
# when it has never been installed, lazily provision it (respecting the
# lazy-install policy — InstallError is swallowed and PATH is tried) so
# the sidecar works on a fresh install without a manual `hermes pm
# install node`.
try:
import pm
pm.ensure("node")
from hermes_constants import find_node_executable as _fne
resolved = _fne(command)
if resolved:
return resolved
except Exception:
pass
return shutil.which(command)
def check_requirements() -> bool:
"""Return True when both Python deps and the Node sidecar are available."""
if not HTTPX_AVAILABLE:
logger.warning("photon: httpx not installed — pip install httpx")
return False
if not shutil.which(os.getenv("PHOTON_NODE_BIN") or "node"):
logger.warning(
"photon: node binary '%s' not found on PATH",
os.getenv("PHOTON_NODE_BIN") or "node",
)
if not _node_command("node"):
logger.warning("photon: node binary not found on PATH or in the pm store")
return False
if not sidecar_deps_installed():
# spectrum-ts not installed yet, or node_modules/ was partially created
@@ -444,7 +478,7 @@ def check_requirements() -> bool:
# user has no CLI to run `hermes photon setup`, so the connect path
# must self-heal). Otherwise keep returning False so
# `hermes setup` / status surface the missing-deps state.
if bool(shutil.which("npm")) and _dir_writable(_sidecar_dir()):
if bool(_node_command("npm")) and _dir_writable(_sidecar_dir()):
return True
# DEBUG (not WARNING): this is the normal pre-setup state.
# check_fn() is called from multiple hot paths in the core
@@ -453,7 +487,7 @@ def check_requirements() -> bool:
npm_error = ""
try:
if _npm_error_log().exists():
npm_error = _npm_error_log().read_text(encoding="utf-8").strip()[:_NPM_ERROR_LOG_MAX_CHARS]
npm_error = _npm_error_log().read_text(encoding="utf-8-sig").strip()[:_NPM_ERROR_LOG_MAX_CHARS]
except OSError:
pass
if npm_error:
@@ -499,9 +533,9 @@ def _reinstall_sidecar_deps() -> None:
Best-effort — a failure here just leaves the (stale) deps in place and the
normal ``_start_sidecar`` readiness check reports the real error.
"""
npm = shutil.which("npm")
npm = _node_command("npm")
if not npm:
logger.warning("[photon] cannot reinstall stale sidecar deps: npm not on PATH")
logger.warning("[photon] cannot reinstall stale sidecar deps: npm not available (pm store or PATH)")
return
# Windows: suppress the console flash these short-lived npm runs would
# otherwise pop (0 elsewhere). Same helper as the sidecar spawn below.
@@ -753,7 +787,7 @@ class PhotonAdapter(BasePlatformAdapter):
self._autostart_sidecar = str(
os.getenv("PHOTON_SIDECAR_AUTOSTART", "true")
).lower() not in ("0", "false", "no")
self._node_bin = os.getenv("PHOTON_NODE_BIN") or shutil.which("node") or "node"
self._node_bin = _node_command("node") or "node"
# Presence watchdog. spectrum-ts only reconnects when its inbound
# iterator throws or ends; a half-open ("zombie") gRPC socket makes the
@@ -1692,11 +1726,11 @@ class PhotonAdapter(BasePlatformAdapter):
creationflags=windows_hide_flags(),
)
except FileNotFoundError as exc:
# Deterministic: node isn't on PATH / PHOTON_NODE_BIN points at
# nothing. Retrying can never fix a missing binary (OOF-153).
# Deterministic: node isn't resolvable (pm store or PATH).
# Retrying can never fix a missing binary (OOF-153).
raise PhotonSidecarStartupError(
f"node binary not found ({self._node_bin!r}) — install Node.js "
f"or set PHOTON_NODE_BIN: {exc}",
f"18+ or run `hermes pm install`: {exc}",
code="SIDECAR_NODE_MISSING",
retryable=False,
) from exc
+4 -5
View File
@@ -21,7 +21,6 @@ from __future__ import annotations
import argparse
import getpass
import os
import shutil
import subprocess
import sys
from pathlib import Path
@@ -29,7 +28,7 @@ from pathlib import Path
from hermes_cli.colors import Colors, color
from . import auth as photon_auth
from .adapter import _NPM_ERROR_LOG_MAX_CHARS, sidecar_deps_installed
from .adapter import _NPM_ERROR_LOG_MAX_CHARS, _node_command, sidecar_deps_installed
from .sidecar_paths import resolve_sidecar_dir
# Writable sidecar runtime dir (mirrors to HERMES_HOME on immutable
@@ -385,7 +384,7 @@ def _cmd_status(_args: argparse.Namespace) -> int:
# callback is the only sink that sees credential-derived strings, so
# cli.py keeps zero taint flow according to CodeQL.
photon_auth.print_credential_summary(print)
node_bin = os.getenv("PHOTON_NODE_BIN") or shutil.which("node")
node_bin = _node_command("node")
sidecar_installed = sidecar_deps_installed()
print(f" node binary : {node_bin or '✗ missing (install Node 18+)'}")
print(f" sidecar deps : {'✓ installed' if sidecar_installed else '✗ run `hermes photon install-sidecar`'}")
@@ -442,8 +441,8 @@ def _cmd_telemetry(args: argparse.Namespace) -> int:
def _install_sidecar() -> int:
npm = shutil.which("npm") or "npm"
if not shutil.which(npm):
npm = _node_command("npm")
if not npm:
print(
"npm is not on PATH. Install Node.js 18+ (https://nodejs.org/) "
"and re-run.",
-4
View File
@@ -38,10 +38,6 @@ optional_env:
description: "Spawn the Node sidecar on connect (true/false, default true)"
prompt: "Auto-start the sidecar?"
password: false
- name: PHOTON_NODE_BIN
description: "Path to the node binary (default: shutil.which('node'))"
prompt: "Node executable path"
password: false
- name: PHOTON_DASHBOARD_HOST
description: "Photon Dashboard API host (default https://app.photon.codes)"
prompt: "Dashboard host"
+8 -11
View File
@@ -702,7 +702,7 @@ def check_teams_requirements() -> bool:
"""Ensure the Teams SDK is importable, lazy-installing it on first use.
Lazy-installs ``microsoft-teams-apps`` via
``tools.lazy_deps.ensure("platform.teams")`` if not present, then rebinds
``pm.ensure_import("teams")`` if not present, then rebinds
all module-level SDK globals on success. Returns True once the SDK (and
aiohttp) are importable, False if they couldn't be installed/imported.
@@ -763,9 +763,9 @@ def check_teams_requirements() -> bool:
"TEAMS_SDK_AVAILABLE": True,
}
from tools.lazy_deps import ensure_and_bind
from pm.extras import ensure_and_bind
return ensure_and_bind("platform.teams", _import, globals(), prompt=False)
return ensure_and_bind("teams", _import, globals())
class TeamsAdapter(BasePlatformAdapter):
@@ -1500,17 +1500,14 @@ def interactive_setup() -> None:
# ── Plugin entry point ────────────────────────────────────────────────────────
def _install_hint() -> str:
"""Build the Teams install hint from the canonical LAZY_DEPS pins.
"""Build the Teams install hint string.
Derived (not hardcoded) so a pin bump in ``tools/lazy_deps.py`` — aiohttp
is CVE-pinned, so bumps happen — never leaves this string stale.
``feature_install_command(venv_pip=True)`` targets the actual Hermes
venv in every layout and sidesteps Ubuntu 24.04's PEP 668 failure that
a bare ``pip install`` hint invites.
Prefers ``uv sync --frozen --extra teams`` (respects pyproject pinning);
falls back to a plain pip install of the two packages if that is
unavailable. Restarting the gateway also auto-installs via pm.
"""
try:
from tools.lazy_deps import feature_install_command
cmd = feature_install_command("platform.teams", venv_pip=True)
cmd = "uv sync --frozen --extra teams"
except Exception: # pragma: no cover — defensive
cmd = None
if not cmd:
+4 -4
View File
@@ -336,7 +336,7 @@ def check_telegram_requirements() -> bool:
"""Check if Telegram dependencies are available.
If python-telegram-bot is missing, attempts to lazy-install it via
``tools.lazy_deps.ensure("platform.telegram")``. After a successful
``pm.ensure_import("telegram")``. After a successful
install, re-imports the SDK and flips ``TELEGRAM_AVAILABLE`` to True
so the adapter's class-level type aliases get rebound.
"""
@@ -347,8 +347,8 @@ def check_telegram_requirements() -> bool:
if TELEGRAM_AVAILABLE:
return True
try:
from tools.lazy_deps import ensure as _lazy_ensure
_lazy_ensure("platform.telegram", prompt=False)
from pm import ensure_import as _lazy_ensure
_lazy_ensure("telegram")
except Exception:
return False
try:
@@ -3896,7 +3896,7 @@ class TelegramAdapter(BasePlatformAdapter):
return
import yaml as _yaml
with open(config_path, "r", encoding="utf-8") as f:
with open(config_path, "r", encoding="utf-8-sig") as f:
config = _yaml.safe_load(f) or {}
# Navigate to platforms.telegram.extra.dm_topics, creating the path
+2 -2
View File
@@ -97,10 +97,10 @@ def ensure_wecom_callback_requirements() -> bool:
return {"ET": _ET, "DEFUSEDXML_AVAILABLE": True}
try:
from tools.lazy_deps import ensure_and_bind
from pm.extras import ensure_and_bind
except Exception: # pragma: no cover — defensive
return False
if not ensure_and_bind("platform.wecom_callback", _import, globals(), prompt=False):
if not ensure_and_bind("wecom", _import, globals()):
return False
return check_wecom_callback_requirements()
+25 -5
View File
@@ -355,15 +355,34 @@ def _file_content_hash(path: Path) -> str:
return ""
def _pm_ensure_node(command: str) -> str | None:
"""Lazily provision node/npm through the pm store, then re-resolve.
``find_node_executable`` prefers the pm store's pinned Node but never
installs it. Node/npm ship as pm packages, so when the store is empty
this is the pm.ensure wiring the pm-unified-toolchain plan asks for:
install (respecting the lazy-install policy — raises InstallError and
returns None when lazy installs are refused) and re-resolve.
"""
try:
import pm
pm.ensure("node" if command == "npm" else command)
return find_node_executable(command)
except Exception:
return None
def check_whatsapp_requirements() -> bool:
"""
Check if WhatsApp dependencies are available.
WhatsApp requires a Node.js bridge for most implementations.
"""
# Prefer Hermes-managed Node/npm so Windows installs are not broken by a
# bad or elevation-triggering system Node on PATH.
_node = find_node_executable("node")
# bad or elevation-triggering system Node on PATH. When the pm store has
# no Node yet, pm.ensure lazily provisions it (policy permitting).
_node = find_node_executable("node") or _pm_ensure_node("node")
if not _node:
return False
try:
@@ -584,8 +603,9 @@ class WhatsAppAdapter(WhatsAppBehaviorMixin, BasePlatformAdapter):
if not _deps_fresh:
print(f"[{self.name}] Installing WhatsApp bridge dependencies...")
# Resolve npm path so Windows uses npm.cmd from the
# Hermes-managed portable Node before falling back to PATH.
_npm_bin = find_node_executable("npm") or "npm"
# Hermes-managed portable Node before falling back to PATH;
# pm.ensure provisions npm when the store has none.
_npm_bin = find_node_executable("npm") or _pm_ensure_node("npm") or "npm"
try:
# Read timeout from environment variable, default to 300 seconds (5 minutes)
# to accommodate slower systems like Unraid NAS
+1 -1
View File
@@ -551,7 +551,7 @@ def _load_fal_client() -> Any:
"""Lazy-load the ``fal_client`` SDK and cache it on this module.
Delegates the actual import to :func:`tools.fal_common.import_fal_client`
so the ``lazy_deps`` ensure-install handling stays in one place.
so the ``pm.ensure_import`` handling stays in one place.
Thread-safe via double-checked locking: concurrent first calls import
the SDK exactly once instead of each racing thread re-running the import.
+11
View File
@@ -159,6 +159,17 @@ def _run_ddgs_search_bounded(query: str, safe_limit: int) -> list[dict[str, Any]
from tools.environments.local import _sanitize_subprocess_env
env = _sanitize_subprocess_env(dict(os.environ))
# No-boot-through-venv: the worker runs under the STORE python, whose
# third-party imports (ddgs/primp) arrive via the launcher-composed
# PYTHONPATH. The sanitizer strips Hermes-owned entries (cross-version
# protection); this worker is the SAME interpreter, so merge the
# ambient PYTHONPATH back in before prepending the plugins entry.
_ambient_pp = os.environ.get("PYTHONPATH")
if _ambient_pp:
_current = env.get("PYTHONPATH", "")
env["PYTHONPATH"] = (
_ambient_pp + os.pathsep + _current if _current else _ambient_pp
)
if _test_hook:
env["HERMES_DDGS_ALLOW_TEST_HOOKS"] = "1"
+4 -4
View File
@@ -2,7 +2,7 @@
Subclasses :class:`agent.web_search_provider.WebSearchProvider`. Uses the
official Exa SDK (``exa-py``) which is lazy-loaded via
:func:`tools.lazy_deps.ensure` so that cold-start CLI users don't pay the
:func:`pm.ensure_import` so that cold-start CLI users don't pay the
SDK import cost when Exa isn't configured.
Config keys this provider responds to::
@@ -61,12 +61,12 @@ def _get_exa_client() -> Any:
)
try:
from tools.lazy_deps import ensure as _lazy_ensure
from pm import ensure_import as _lazy_ensure
_lazy_ensure("search.exa", prompt=False)
_lazy_ensure("exa")
except ImportError:
pass
except Exception as exc: # noqa: BLE001 — lazy_deps surfaces install hints
except Exception as exc: # noqa: BLE001 — pm surfaces install hints
raise ImportError(str(exc))
from exa_py import Exa # noqa: WPS433 — deliberately lazy
+2 -2
View File
@@ -83,9 +83,9 @@ def _load_firecrawl_cls() -> type:
global _FIRECRAWL_CLS_CACHE
if _FIRECRAWL_CLS_CACHE is None:
try:
from tools.lazy_deps import ensure as _lazy_ensure
from pm import ensure_import as _lazy_ensure
_lazy_ensure("search.firecrawl", prompt=False)
_lazy_ensure("firecrawl")
except ImportError:
pass
except Exception as exc: # noqa: BLE001 — surface install hint
+3 -3
View File
@@ -47,14 +47,14 @@ def _ensure_parallel_sdk_installed() -> None:
"""Trigger lazy install of the parallel SDK if it isn't present.
Mirrors the lazy-deps pattern used by the legacy implementation.
Swallows benign ImportError from the lazy_deps helper itself; if the
Swallows benign ImportError from the pm helper itself; if the
SDK is genuinely missing the subsequent ``from parallel import ...``
raises ImportError that the caller can handle.
"""
try:
from tools.lazy_deps import ensure as _lazy_ensure
from pm import ensure_import as _lazy_ensure
_lazy_ensure("search.parallel", prompt=False)
_lazy_ensure("parallel-web")
except ImportError:
pass
except Exception as exc: # noqa: BLE001 — surface install hint as ImportError
+60
View File
@@ -0,0 +1,60 @@
"""pm: the hermes package system.
Everything hermes depends on — tool binaries, the python venv, node_modules
dirs, plugins — is a package in one dependency tree. Package definitions
(pm/packages.py) say what a package IS. The lockfile (pm/lock.json,
machine-written) says exactly which versions and hashes. The installed-state
file (facts.json, per install) says what is actually on this machine.
ensure(name) makes the installed state match the lockfile and returns a
Runner with the composed environment. env_for(*names) composes already-installed
packages' env without installing anything.
"""
from pm.ensure import (
activate,
adopt,
check,
enabled_extras,
ensure,
env_for,
is_installed,
lazy_installs_allowed,
sync_venv,
uv,
)
from pm.extras import available, ensure_import
from pm.lock import Facts, Lockfile
from pm.package import InstallError, Package, Runner, compose_env
from pm.registry import all_packages, get_package, register, walk
from pm.store import Store, current_target
__all__ = [
"ensure",
"env_for",
"is_installed",
"adopt",
"check",
"activate",
"sync_venv",
"available",
"ensure_import",
"enabled_extras",
"lazy_installs_allowed",
# NOTE: pm.uv is importable but deliberately not in __all__ — it is the
# marked transitional bridge for update/repair and must not spread.
"Facts",
"Lockfile",
"InstallError",
"Package",
"Runner",
"compose_env",
"all_packages",
"get_package",
"register",
"walk",
"Store",
"current_target",
]
import pm.packages # noqa: E402,F401 (registers the built-in definitions)
+558
View File
@@ -0,0 +1,558 @@
"""hermes pm: lock / install / env / doctor / gc / bundle."""
from __future__ import annotations
import argparse
import json
import shutil
import subprocess
import sys
import threading
from pathlib import Path
from typing import Optional
from pm.ensure import _facts, _lockfile, _store, ensure
from pm.ensure import uv as pm_uv
from pm.package import InstallError
from pm.registry import get_package
from pm.store import ALL_TARGETS, current_target, hash_url
def cmd_lock(args) -> int:
"""--bump <name> <version>: resolve every target's archives, hash them,
write. A target with one archive pins the object; several pin a list.
Target-independent urls collapse to one "any" artifact."""
lockfile = _lockfile()
package = get_package(args.name)
artifacts: dict[str, object] = {}
def pin(url: str) -> dict:
print(f" {url}")
digest = package.known_sha256(args.version, url) or hash_url(url)
print(f" sha256 {digest}")
return {"url": url, "sha256": digest}
urls = {
target: package.fetch_urls(args.version, target)
for target in ALL_TARGETS
if package.missing_reason(target) is None
}
distinct = {tuple(u) for u in urls.values()}
if len(distinct) == 1:
print(" any:")
pinned = [pin(url) for url in next(iter(urls.values()))]
artifacts["any"] = pinned[0] if len(pinned) == 1 else pinned
else:
for target, target_urls in urls.items():
print(f" {target}:")
pinned = [pin(url) for url in target_urls]
artifacts[target] = pinned[0] if len(pinned) == 1 else pinned
lockfile.set_pin(args.name, args.version, artifacts)
lockfile.save()
print(f"pinned {args.name} {args.version} ({len(artifacts)} targets)")
return 0
def _fmt_bytes(n: int) -> str:
return f"{n / (1024 * 1024):.1f} MiB"
def _live_progress(name: str):
"""Per-package progress for ensure(): download as % + MiB, unpack as a
phase line. Throttled to ~4 MiB steps — a slow line proves it's moving
in a piped (CI) log without flooding it (a 1 MiB tick on a 1.5 GiB
model would be ~1,500 lines)."""
last = 0
def report(stage: str, done: int, total: int, label: str) -> None:
nonlocal last
if stage == "unpack":
last = 0
print(f" {name}: unpacking{(' ' + label) if label else ''}", flush=True)
return
if total <= 0:
return
if done >= total or done - last >= 4 * 1024 * 1024:
last = done
print(
f" {name}: {done / total * 100:5.1f}% {_fmt_bytes(done)} / {_fmt_bytes(total)}",
flush=True,
)
return report
def _install_names(names: list[str]) -> int:
failed = 0
for name in names:
try:
ensure(name, explicit=True, progress=_live_progress(name))
print(f"✓ {name}", flush=True)
except InstallError as e:
print(f"✗ {e}", flush=True)
failed += 1
return failed
def _bundle_package_names() -> list[str]:
names = [
n
for n in _lockfile().names()
if not get_package(n).internal or n == "uv"
]
if "python" not in names:
names.append("python")
return names
def _drop_unloadable_runtime_files(store_dir: Path) -> None:
"""Drop the x64 VC runtime that python-build-standalone ships beside ARM64 Python."""
if current_target() != "win32-arm64":
return
facts = _facts()
facts.reload()
python = facts.get("python")
if python and "entry" in python:
(store_dir / python["entry"] / "vcruntime140_1.dll").unlink(missing_ok=True)
def cmd_install(args) -> int:
names = args.names or [
n for n in _lockfile().names() if not get_package(n).optional
]
failed = _install_names(names)
if not args.names:
from pm.ensure import sync_venv
try:
sync_venv(explicit=True)
print("✓ venv")
except InstallError as e:
print(f"✗ {e}")
failed += 1
return 1 if failed else 0
def cmd_env(args) -> int:
from pm.ensure import env_for
names = args.names or _lockfile().names()
print(json.dumps(env_for(*names), indent=2, sort_keys=True))
return 0
def cmd_doctor(args) -> int:
from pm.ensure import _identity
from pm.store import tree_digest
lockfile = _lockfile()
facts = _facts()
store = _store()
target = current_target()
bad = 0
for name in lockfile.names():
package = get_package(name)
reason = package.missing_reason(target)
if reason is not None:
print(f"- {name}: n/a on {target} ({reason})")
continue
fact = facts.get(name)
soft = package.optional or package.internal
identity = _identity(lockfile, name, target)
if (
fact is not None
and identity is not None
and ("target" not in fact or "artifacts" not in fact)
):
# Legacy fact: pre-dates digest-bound identity; installed()
# treats it as not installed and forces one reinstall.
print(f"{'?' if soft else '✗'} {name}: legacy fact: no recorded identity, run `hermes pm install`")
bad += 0 if soft else 1
continue
if not facts.installed(name, lockfile.version(name), store.root, identity):
state = "not installed" if fact is None else "outdated"
print(f"{'?' if soft else '✗'} {name}: {state}")
bad += 0 if soft else 1
continue
entry = store.entry(fact["entry"])
reason = package.verify(entry, target)
if reason:
print(f"✗ {name}: installed but failed verification: {reason}")
bad += 1
continue
recorded = fact.get("digest")
if recorded is not None and tree_digest(entry) != recorded:
# Doctor is the expensive-path tool: re-hash the realized
# bytes. Boot checks stay O(1) json compares.
print(f"✗ {name}: realized bytes do not match recorded digest")
bad += 1
continue
print(f"✓ {name} {fact['version']}")
return 1 if bad else 0
def _venv_site_packages(venv_dir: Path, win: bool) -> Optional[Path]:
"""The site-packages dir uv sync fills inside the venv (the venv stays
a dependency target; it is never where `python` resolves)."""
if win:
candidate = venv_dir / "Lib" / "site-packages"
return candidate if candidate.is_dir() else None
for candidate in sorted(venv_dir.glob("lib/python3.*/site-packages")):
if candidate.is_dir():
return candidate
return None
def _develop_env(names: list[str]) -> Optional[dict]:
"""The `pm develop` subshell environment. `python` resolves to the pm
STORE python (its bin dir first on PATH); imports come from
PYTHONPATH=<repo>;<venv>/site-packages (repo first — the venv stays a
dependency target only). VIRTUAL_ENV and the venv bin dir are
deliberately absent: nothing in the devshell boots through the venv
(pm work item 3; pyvenv.cfg is inert dead config). Returns None when
the store interpreter has not been materialized (`hermes pm install`)."""
import os
from pm import paths
from pm.ensure import env_for
facts = _facts()
python_fact = facts.get("python")
target = current_target()
if not python_fact or "entry" not in python_fact:
return None
python_bin = get_package("python").binary(
_store().entry(python_fact["entry"]), target
)
if python_bin is None or not python_bin.is_file():
return None
env = env_for(*names, base_env=dict(os.environ))
repo = paths.repo_root()
venv_dir = repo / (".venv" if (repo / ".venv").is_dir() else "venv")
win = target.startswith("win32")
venv_bin = venv_dir / ("Scripts" if win else "bin")
env.pop("VIRTUAL_ENV", None)
env.pop("PYTHONHOME", None)
# The venv bin dir must never be where `python` resolves.
path_entries = [
p for p in env.get("PATH", "").split(os.pathsep) if p and Path(p) != venv_bin
]
env["PATH"] = os.pathsep.join([str(python_bin.parent), *path_entries])
site = _venv_site_packages(venv_dir, win)
ours = str(repo) + (os.pathsep + str(site) if site else "")
existing = env.get("PYTHONPATH", "")
env["PYTHONPATH"] = ours + (os.pathsep + existing if existing else "")
return env
def cmd_develop(args) -> int:
"""Install everything, sync the venv, then activate: spawn a subshell
with every tool's env composed in and the pm STORE python resolving
`python` (PYTHONPATH=repo;venv-site-packages for imports — never the
venv interpreter). The devshell equivalent of nix develop. --print
emits eval-able exports for the current shell."""
import os
import subprocess
from pm.ensure import sync_venv
install_names = [
n for n in _lockfile().names()
if not get_package(n).optional
and get_package(n).missing_reason(current_target()) is None
]
# The devshell boots the STORE python — make sure it is materialized.
if (
"python" not in install_names
and get_package("python").missing_reason(current_target()) is None
):
install_names.append("python")
failed = _install_names(install_names)
try:
sync_venv(explicit=True)
print("✓ venv")
except InstallError as e:
print(f"✗ {e}")
failed += 1
if failed:
return 1
env = _develop_env(_lockfile().names())
if env is None:
print("✗ develop: no pm store interpreter — run 'hermes pm install' first")
return 1
from pm import paths
win = current_target().startswith("win32")
if args.print_env:
changed = {k: v for k, v in env.items() if os.environ.get(k) != v}
for key, value in sorted(changed.items()):
if win and os.environ.get("SHELL") is None:
print(f'$env:{key} = "{value}"')
else:
escaped = value.replace("'", "'\\''")
print(f"export {key}='{escaped}'")
return 0
shell = os.environ.get("SHELL") or os.environ.get("COMSPEC") or (
"cmd.exe" if win else "/bin/sh"
)
print(f"pm develop: entering {shell} (exit to leave)")
return subprocess.call([shell], env=env, cwd=paths.repo_root())
def cmd_gc(args) -> int:
from pm.downloader import gc_protected_names
facts = _facts()
store = _store()
if not store.root.is_dir():
return 0
removed = 0
with store.install_lock():
facts.reload()
keep = facts.entries_in_use()
# Partials an in-flight (or recently interrupted) download still
# owns must survive the sweep.
protected_partials = gc_protected_names(store.root / "partials")
for item in sorted(store.root.iterdir()):
if not item.is_dir() or item.name.startswith("."):
continue
if item.name == "partials" and protected_partials:
for child in sorted(item.iterdir()):
if child.name in protected_partials:
continue
print(f"removing {item.name}/{child.name}")
if child.is_dir():
shutil.rmtree(child, ignore_errors=True)
else:
try:
child.unlink()
except OSError:
pass
continue
if item.name in keep:
continue
print(f"removing {item.name}")
shutil.rmtree(item, ignore_errors=True)
removed += 1
print(f"gc: removed {removed}, kept {len(keep)}")
return 0
def _run_live(cmd: list[str], *, cwd, env, timeout: int = 3600) -> tuple[int, str]:
"""Run cmd with its output streamed through our stdout — a long uv
venv build must prove liveness in a piped (CI) log, not vanish until
exit — while still capturing the tail for the failure message. A
reader thread drains output so proc.wait(timeout) keeps the wall-clock
kill the old subprocess.run(timeout=) had. Returns (returncode, last
~2k chars of combined output)."""
proc = subprocess.Popen(
cmd, cwd=cwd, env=env, stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, text=True, bufsize=1, errors="replace",
)
tail = ""
lock = threading.Lock()
def drain() -> None:
nonlocal tail
for line in proc.stdout:
print(line, end="", flush=True)
with lock:
tail = (tail + line)[-2000:]
thread = threading.Thread(target=drain, daemon=True)
thread.start()
try:
code = proc.wait(timeout=timeout)
except subprocess.TimeoutExpired:
proc.kill()
proc.wait()
raise RuntimeError(f"{cmd[0]} timed out after {timeout}s")
thread.join()
with lock:
return code, tail
def cmd_bundle(args) -> int:
"""Stage a complete payload for THIS machine's target into --out:
repo snapshot + store + facts (via the normal install path, redirected)
+ a relocatable venv built on the staged interpreter and synced from
uv.lock. Built natively per (os, arch); there is no cross-target
staging."""
import os
from pm import paths
out = Path(args.out).resolve()
store_dir = out / "tools"
store_dir.mkdir(parents=True, exist_ok=True)
# A manifest from a previous run would make this payload look sealed
# and refuse its own staging; it is rewritten at the end.
(out / "manifest.json").unlink(missing_ok=True)
repo_dir = out / "hermes-agent"
ref = args.ref or "HEAD"
if repo_dir.exists():
shutil.rmtree(repo_dir)
repo_dir.mkdir(parents=True)
print(f"staging repo snapshot ({ref})…", flush=True)
archive = subprocess.run(
["git", "archive", "--format=tar", ref],
cwd=paths.repo_root(), capture_output=True, timeout=600,
)
if archive.returncode != 0:
print(f"✗ repo: git archive {ref} failed: {archive.stderr.decode()[-500:]}")
return 1
import io
import tarfile
with tarfile.open(fileobj=io.BytesIO(archive.stdout)) as tar:
tar.extractall(repo_dir, filter="data")
print(f"✓ repo ({ref})")
os.environ["HERMES_RUNTIME_DIR"] = str(store_dir)
paths._stamp.cache_clear()
failed = 0
names = _bundle_package_names()
failed += _install_names(
[n for n in names if get_package(n).missing_reason(current_target()) is None]
)
_drop_unloadable_runtime_files(store_dir)
uv_bin, env = pm_uv()
if uv_bin is None:
print("✗ venv: uv did not stage")
return 1
python_fact = _facts().get("python")
if python_fact is None:
print("✗ venv: no staged interpreter to build on")
return 1
python_bin = get_package("python").binary(
_store().entry(python_fact["entry"]), current_target()
)
# Build + sync INSIDE the staged repo: the editable project install
# must point at the payload's own tree, not this checkout.
venv_dir = out / "venv"
if venv_dir.exists():
shutil.rmtree(venv_dir)
env["VIRTUAL_ENV"] = str(venv_dir)
env.pop("UV_NO_CONFIG", None)
if current_target().startswith("darwin"):
# python-build-standalone bakes phantom toolchain paths (its build
# dir's llvm-ar) into sysconfig; sdist builds then fail with
# "No such file or directory: .../tools/llvm/bin/llvm-ar". Point
# sdist builds at the machine's real toolchain.
env.setdefault("AR", "/usr/bin/ar")
env.setdefault("CC", "clang")
for cmd in (
[uv_bin, "venv", "--relocatable", "--python", str(python_bin), str(venv_dir)],
[uv_bin, "sync", "--frozen", "--all-extras", "--active"],
):
print(f" venv: $ {' '.join(cmd)}", flush=True)
code, tail = _run_live(cmd, cwd=repo_dir, env=env)
if code != 0:
print(f"✗ venv: {' '.join(cmd[1:3])} failed:\n{tail}")
return 1
print("✓ venv (relocatable, all extras, on the staged interpreter)")
bad = _arch_guard(store_dir)
for line in bad:
print(f"✗ arch: {line}")
failed += 1
manifest = {
"schema": 1,
"target": current_target(),
"ref": ref,
"repo": "hermes-agent",
"venv": "venv",
"store": "tools",
}
(out / "manifest.json").write_text(
json.dumps(manifest, indent=2, sort_keys=True) + "\n", encoding="utf-8"
)
print(f"✓ manifest ({out / 'manifest.json'})")
return 1 if failed else 0
def _arch_guard(store_dir: Path) -> list[str]:
"""Every staged binary must be built for this machine's target — a
payload staged with a mismatched interpreter or PATH tool ships an
artifact that cannot run. Reads facts, probes each entry binary."""
from pm.lock import Facts
from pm.package import machine_matches_binary
facts = Facts(store_dir / "facts.json")
problems = []
target = current_target()
for name in _lockfile().names():
package = get_package(name)
fact = facts.get(name)
if fact is None or "entry" not in fact:
continue
binary = package.binary(store_dir / fact["entry"], target)
if binary is None or not binary.is_file():
continue
verdict = machine_matches_binary(binary, target)
# A package that declares this target as emulated (x64 binary run
# under Windows ARM64 built-in emulation) is fine with the x64 PE.
if verdict is False and target not in package.emulated_arch_targets:
problems.append(f"{name}: {binary.name} is not a {target} binary")
return problems
def main(argv=None) -> int:
# Windows consoles default to cp1252; pm prints ✓/✗. Never let the
# status glyphs crash the command reporting them. line_buffering:
# pm output must stream live in a piped (CI) log, not sit in a block
# buffer and flush only at exit.
for stream in (sys.stdout, sys.stderr):
try:
stream.reconfigure(errors="replace", line_buffering=True)
except (AttributeError, OSError):
pass
parser = argparse.ArgumentParser(prog="hermes pm")
sub = parser.add_subparsers(dest="cmd", required=True)
p = sub.add_parser("lock", help="write versions+hashes into pm/lock.json")
p.add_argument("--bump", dest="name", required=True)
p.add_argument("version")
p.set_defaults(func=cmd_lock)
p = sub.add_parser("install", help="install packages (default: all required)")
p.add_argument("names", nargs="*")
p.set_defaults(func=cmd_install)
p = sub.add_parser("env", help="print composed env of installed packages")
p.add_argument("names", nargs="*")
p.set_defaults(func=cmd_env)
p = sub.add_parser("doctor", help="check installed state against the lockfile")
p.set_defaults(func=cmd_doctor)
p = sub.add_parser("develop", help="install + sync, then activate a devshell with the composed env")
p.add_argument("--print", dest="print_env", action="store_true",
help="print eval-able exports instead of spawning a shell")
p.set_defaults(func=cmd_develop)
p = sub.add_parser("gc", help="remove store entries nothing references")
p.set_defaults(func=cmd_gc)
p = sub.add_parser("bundle", help="stage a payload (repo+store+facts+relocatable venv) into --out")
p.add_argument("--out", required=True)
p.add_argument("--ref", help="git ref for the repo snapshot (default HEAD)")
p.set_defaults(func=cmd_bundle)
args = parser.parse_args(argv)
return args.func(args)
if __name__ == "__main__":
sys.exit(main())
+427
View File
@@ -0,0 +1,427 @@
"""Resumable, hash-verified, multi-connection downloads.
Every large fetch in Hermes goes through this one downloader: pm
packages (pinned sha256 from lock.json) and local models (deliberately
unverified — catalog sizes may lag an upstream re-upload, so sha256 is
optional per source).
Partial state is the downloader's own. Files land in a managed
partials area keyed by sha256(url), and a finished file is MOVED into
its real destination, so a caller only ever sees a complete file or
nothing. An interrupted download leaves its partial + a .ranges sidecar
(the durable byte-range bitmap) behind, and the next run re-fetches
exactly the ranges the bitmap says are missing.
The progress callback reports the whole job AND the per-dest bitmap:
``progress(overall_done, overall_total, ranges)`` where ``ranges`` maps
``dest.name`` to half-open [start, end) runs. ``done_bytes`` is the sum;
``ranges`` is the shape — one datum, two resolutions.
"""
from __future__ import annotations
import hashlib
import json
import shutil
import threading
import urllib.error
import urllib.request
from dataclasses import dataclass
from pathlib import Path
from typing import Callable, Optional, Sequence
_UA = {"User-Agent": "hermes-pm"}
_LOOPBACK = ("http://127.0.0.1:", "http://localhost:", "http://[::1]:")
_CHUNK = 1 << 20 # read/write block, also the minimum range size
class _HttpsRedirectHandler(urllib.request.HTTPRedirectHandler):
"""The https-only gate must hold across redirects, not just the first
hop — an https URL could otherwise bounce to http mid-download and
carry the payload in the clear."""
def redirect_request(self, req, fp, code, msg, headers, newurl):
if not (newurl.startswith("https://") or newurl.startswith(_LOOPBACK)):
raise DownloadError(f"refusing redirect to non-https url: {newurl}")
return super().redirect_request(req, fp, code, msg, headers, newurl)
_OPENER = urllib.request.build_opener(_HttpsRedirectHandler())
class DownloadError(RuntimeError):
"""Base class for downloader failures."""
class HashError(DownloadError):
"""The downloaded bytes did not match the pinned sha256."""
class DownloadPaused(DownloadError):
"""pause() was called mid-download; partials were left intact."""
@dataclass(frozen=True)
class Source:
url: str
dest: Path
sha256: str = "" # "" = no integrity check (model catalog policy)
_Ranges = list[tuple[int, int]]
ProgressFn = Callable[[int, int, dict[str, _Ranges]], None]
def _coalesce(ranges: _Ranges) -> _Ranges:
"""Merge half-open [start, end) ranges into sorted, disjoint runs."""
runs = sorted((a, b) for a, b in ranges if b > a)
if not runs:
return []
out: _Ranges = []
a, b = runs[0]
for x, y in runs[1:]:
if x <= b:
b = max(b, y)
else:
out.append((a, b))
a, b = x, y
out.append((a, b))
return out
def _missing(total: int, covered: _Ranges) -> _Ranges:
"""The gaps in [0, total) not covered by the coalesced bitmap."""
missing: _Ranges = []
cursor = 0
for a, b in _coalesce(covered):
if a > cursor:
missing.append((cursor, a))
cursor = max(cursor, b)
if cursor < total:
missing.append((cursor, total))
return missing
def _sha256_file(path: Path) -> str:
digest = hashlib.sha256()
with open(path, "rb") as f:
for block in iter(lambda: f.read(_CHUNK), b""):
digest.update(block)
return digest.hexdigest()
def _existing_dest_ok(source: "Source") -> bool:
"""A dest already on disk counts as done only when it is a genuine
complete download. No pinned hash (model-catalog policy): a present
file is accepted — size is the tripwire, same as a fresh fetch. With a
pinned sha256 the existing file must match, or it is re-fetched."""
if not source.dest.exists():
return False
if not source.sha256:
return True
try:
return _sha256_file(source.dest) == source.sha256
except OSError:
return False
def _default_partials() -> Path:
from pm import paths
return paths.store_root() / "partials"
# `pm gc` must not delete partials an in-flight download is still writing.
# A sidecar's mtime is the heartbeat: it is (re)written whenever a download
# pauses or errors, and the .part grows on every chunk. Anything touched
# within the grace window is live; everything older is fair game.
GC_GRACE_SECONDS = 6 * 60 * 60
def gc_protected_names(
partials_dir: Path, *, grace_seconds: float = GC_GRACE_SECONDS
) -> set[str]:
"""Entry names under ``partials_dir`` an in-flight (or recently
interrupted) download still owns — ``<key>.part`` / ``<key>.ranges``
pairs whose newest mtime is within ``grace_seconds`` of now.
gc sweeps call this BEFORE rmtree-ing store entries: partials live in
the store root, and the old name-prefix skip (``startswith(".")``)
does not cover the "partials" directory. A caller deletes only the
names absent from the returned set.
"""
import time
partials_dir = Path(partials_dir)
now = time.time()
protected: set[str] = set()
if not partials_dir.is_dir():
return protected
for entry in partials_dir.iterdir():
try:
# Strict < : grace_seconds=0 protects nothing (the test
# contract for "sweep everything"). mtime can tick a hair
# NEWER than `now` on some filesystems, which <= would
# wrongly treat as inside a zero window.
if now - entry.stat().st_mtime < grace_seconds:
protected.add(entry.name)
except OSError:
protected.add(entry.name) # unreadable → assume live
return protected
class Download:
"""One resumable download job: a plan of sources, run in parallel.
Per source: probe Range support, preallocate a flat .part in the
managed partials area, split into at most ``connections`` byte
ranges, and stream each with a ``Range:`` header from a worker
thread. A lock-protected bitmap records which ranges are actually
durable; on resume only the missing ranges are re-fetched.
A finished source is MOVED into ``Source.dest``. ``pause()`` stops
between chunks and ``run()`` raises :class:`DownloadPaused`,
leaving every partial + sidecar intact for a later resume.
"""
CONNECTIONS = 8
def __init__(
self,
sources: Sequence[Source],
*,
resume: bool = True,
connections: int = CONNECTIONS,
partials_dir: Optional[Path] = None,
):
self.sources = [Source(s.url, Path(s.dest), s.sha256) for s in sources]
self.resume = resume
self.connections = max(1, int(connections))
self.partials_dir = Path(partials_dir) if partials_dir else _default_partials()
self._paused = threading.Event()
def pause(self) -> None:
"""Request a stop between chunks; run() raises DownloadPaused."""
self._paused.set()
def run(self, progress: Optional[ProgressFn] = None) -> list[Path]:
"""Fetch every source; return the moved destination paths."""
self._paused.clear()
self.partials_dir.mkdir(parents=True, exist_ok=True)
for source in self.sources:
if not (source.url.startswith("https://")
or source.url.startswith(_LOOPBACK)):
raise ValueError(f"refusing non-https url: {source.url}")
# Probe every source up front so overall_total is the whole job.
probed = [] # (source, total, range_supported)
for source in self.sources:
if _existing_dest_ok(source):
probed.append((source, source.dest.stat().st_size, True))
continue
# A pre-existing dest that fails its pinned hash is stale — clear
# it so the fetch below replaces it rather than trusting it.
if source.dest.exists():
source.dest.unlink(missing_ok=True)
total, supported = self._probe(source.url)
probed.append((source, total, supported))
overall_total = sum(t for _, t, _ in probed)
moved: list[Path] = []
done_base = 0
completed: dict = {}
for source, total, supported in probed:
if _existing_dest_ok(source):
size = source.dest.stat().st_size
completed[source.dest.name] = [(0, size)]
done_base += size
moved.append(source.dest)
continue
def tick(written: _Ranges) -> None:
if progress is not None:
ranges = dict(completed)
ranges[source.dest.name] = written
progress(done_base + sum(b - a for a, b in written),
overall_total, ranges)
if total and supported:
self._fetch_ranged(source, total, tick)
else:
self._fetch_single(source, total, tick)
done_base += total
completed[source.dest.name] = [(0, total)]
moved.append(source.dest)
return moved
# ── internals ─────────────────────────────────────────────
@staticmethod
def _probe(url: str) -> tuple[int, bool]:
"""(total bytes, range_supported). A server that ignores Range
reports its Content-Length instead; 0 means unknown (the
single-stream fallback judges completeness by the body)."""
req = urllib.request.Request(url, headers={**_UA, "Range": "bytes=0-0"})
try:
with _OPENER.open(req, timeout=60) as r:
if r.status == 206:
content_range = r.headers.get("Content-Range", "")
if "/" in content_range:
return int(content_range.rsplit("/", 1)[1]), True
return 0, False
length = int(r.headers.get("Content-Length") or 0)
return length, False
except urllib.error.HTTPError as exc:
if exc.code in (401, 403):
raise DownloadError(
"the host refused the download (gated or moved). "
"This is a catalog problem, not yours — please report it."
) from exc
raise
except Exception: # noqa: BLE001 - unknown length, fall back
return 0, False
def _key(self, url: str) -> str:
return hashlib.sha256(url.encode("utf-8")).hexdigest()
def _load_sidecar(self, side: Path) -> _Ranges:
if not self.resume or not side.is_file():
return []
try:
return [tuple(r) for r in json.loads(side.read_text(encoding="utf-8"))]
except (OSError, ValueError):
return []
@staticmethod
def _write_sidecar(side: Path, covered: _Ranges) -> None:
side.parent.mkdir(parents=True, exist_ok=True)
side.write_text(json.dumps(covered), encoding="utf-8")
def _fetch_ranged(self, source: Source, total: int, tick) -> None:
key = self._key(source.url)
part = self.partials_dir / f"{key}.part"
side = self.partials_dir / f"{key}.ranges"
covered = self._load_sidecar(side)
if any(b > total for _, b in covered):
covered = [] # server now serves a smaller file: stale partial
# Create the file WITHOUT truncating an existing partial (resume),
# then size it to total (extends with zeros / shrinks stale tails).
open(part, "ab").close()
with open(part, "r+b") as f:
f.truncate(total)
if covered:
ranges = _missing(total, covered)
else:
n = max(1, min(self.connections, (total + _CHUNK - 1) // _CHUNK))
ranges = [(i * total // n, (i + 1) * total // n) for i in range(n)]
# NOT coalesced: the fresh split is the parallel fan-out, and
# adjacent chunks must stay separate workers. (_missing already
# returns disjoint gaps, so resume needs no merge either.)
ranges = [r for r in ranges if r[1] > r[0]]
lock = threading.Lock()
errors: list[Exception] = []
stop = threading.Event()
written = [sum(b - a for a, b in covered)]
def worker(start: int, end: int) -> None:
try:
req = urllib.request.Request(
source.url,
headers={**_UA, "Range": f"bytes={start}-{end - 1}"})
with _OPENER.open(req, timeout=120) as r, \
open(part, "r+b") as f:
f.seek(start)
pos = start
while pos < end:
if self._paused.is_set() or stop.is_set():
return
chunk = r.read(min(_CHUNK, end - pos))
if not chunk:
break
f.write(chunk)
pos += len(chunk)
with lock:
covered[:] = _coalesce(covered + [(start, pos)])
written[0] += len(chunk)
tick(list(covered))
except Exception as exc: # noqa: BLE001
with lock:
errors.append(exc)
stop.set()
threads = [
threading.Thread(target=worker, args=b, daemon=True,
name=f"dl-{key[:8]}-{i}")
for i, b in enumerate(ranges)
]
for t in threads:
t.start()
for t in threads:
t.join()
if self._paused.is_set():
self._write_sidecar(side, covered)
raise DownloadPaused(source.url)
if errors:
self._write_sidecar(side, covered)
raise errors[0]
if written[0] != total:
self._write_sidecar(side, covered)
raise DownloadError(
f"download incomplete ({written[0]} of {total} bytes)")
self._finalize(source, part, side)
def _fetch_single(self, source: Source, total: int, tick) -> None:
"""No-Range fallback: one stream. A server without Range support
cannot resume, so each run restarts the file."""
key = self._key(source.url)
part = self.partials_dir / f"{key}.part"
side = self.partials_dir / f"{key}.ranges"
covered: _Ranges = []
req = urllib.request.Request(source.url, headers=_UA)
try:
with _OPENER.open(req, timeout=120) as r, open(part, "wb") as f:
declared = int(r.headers.get("Content-Length") or 0)
pos = 0
while True:
if self._paused.is_set():
self._write_sidecar(side, covered)
raise DownloadPaused(source.url)
chunk = r.read(_CHUNK)
if not chunk:
break
f.write(chunk)
pos += len(chunk)
covered = [(0, pos)]
tick(list(covered))
if self._paused.is_set():
self._write_sidecar(side, covered)
raise DownloadPaused(source.url)
if declared and pos != declared:
raise DownloadError(
f"download ended at {pos:,} bytes but the server "
f"said {declared:,} — connection dropped?")
if total and pos != total:
raise DownloadError(
f"download incomplete ({pos} of {total} bytes)")
except DownloadError:
self._write_sidecar(side, covered)
raise
except Exception:
self._write_sidecar(side, covered)
raise
self._finalize(source, part, side)
def _finalize(self, source: Source, part: Path, side: Path) -> None:
if source.sha256:
actual = _sha256_file(part)
if actual != source.sha256:
part.unlink(missing_ok=True)
side.unlink(missing_ok=True)
raise HashError(
f"sha256 mismatch for {source.url}: pinned "
f"{source.sha256}, got {actual}")
source.dest.parent.mkdir(parents=True, exist_ok=True)
shutil.move(str(part), str(source.dest))
side.unlink(missing_ok=True)
+538
View File
@@ -0,0 +1,538 @@
"""ensure(): make the installed state match the lockfile for a package,
and hand back its composed environment."""
from __future__ import annotations
import logging
import shutil
from typing import Optional
from pm import paths
from pm.lock import Facts, Lockfile
from pm.package import InstallError, Package, Runner, StatePackage, compose_env
from pm.registry import get_package, walk
from pm.store import Store, current_target, merge_tree, tree_digest
LOG = logging.getLogger(__name__)
# ``progress(stage, done, total, label)`` — stage is "download" | "unpack",
# label is the archive counter ("1/2") when a package has several. Slow
# lines sit in one stage for minutes, so the byte counters are what prove
# liveness to a UI.
def _artifact_progress(progress, index: int, count: int):
if progress is None:
return None
label = f"{index + 1}/{count}" if count > 1 else ""
return lambda done, total: progress("download", done, total, label)
def _lockfile() -> Lockfile:
return Lockfile(paths.lockfile_path())
def _facts() -> Facts:
return Facts(paths.facts_path())
def _store() -> Store:
return Store(paths.store_root())
def _identity(lockfile: Lockfile, name: str, target: str):
"""The identity the lock currently pins for `name` on `target`:
(target, tuple(artifact sha256s)) — or None when the lock pins no
artifacts (nothing digest-bound to compare)."""
artifacts = lockfile.artifacts(name, target)
if not artifacts:
return None
return (target, tuple(a["sha256"] for a in artifacts))
def lazy_installs_allowed() -> bool:
"""Policy: may pm install things on demand right now?
HERMES_DISABLE_LAZY_INSTALLS is an internal bridge var set by the
official Docker image and the hermetic test harness. The user-facing
setting is security.allow_lazy_installs in config.yaml; a config
system that fails to load counts as ALLOWED only when hermes_cli is
genuinely absent (bootstrap) — config errors fail closed.
"""
import os
if os.environ.get("HERMES_DISABLE_LAZY_INSTALLS", "").strip().lower() in (
"1",
"true",
"yes",
):
return False
try:
from hermes_cli.config import get_config_value
except ImportError:
return True
try:
return bool(get_config_value("security.allow_lazy_installs", True))
except Exception:
return False
def enabled_extras() -> list[str]:
"""The venv extras recorded in the installed state."""
return list((_facts().get("venv") or {}).get("extras", []))
def is_installed(name: str) -> bool:
lockfile = _lockfile()
facts = _facts()
return facts.installed(
name,
lockfile.version(name),
_store().root,
_identity(lockfile, name, current_target()),
)
def sealed() -> bool:
"""A bundled payload is read-only: its store sits beside the bundle
manifest. Asking a sealed install for MORE than it shipped is a
Sealed = bundled LAYOUT only; adoption verification is adopt()'s job."""
return (paths.store_root().parent / "manifest.json").is_file()
def _refuse_lazy(name: str, what: str) -> InstallError:
if sealed():
return InstallError(
name,
f"this install is sealed and does not ship: {what}",
"the bundle bakes its entire tree at build time; rebuild the bundle",
)
return InstallError(
name,
f"not installed and lazy installs are disabled: {what}",
"enable security.allow_lazy_installs or run `hermes pm install`",
)
def _remove_entry(store: Store, entry_name: str) -> None:
"""Remove a published store entry before re-realizing it. NOT
fire-and-forget: a silent failure here would leave the stale entry in
place and publish() would then keep it (a concurrent-winner guard),
re-recording facts over bytes the new lock never produced. Defender /
indexer handles on Windows make removal transiently fail, so retry —
then fail loudly."""
import time
entry = store.entry(entry_name)
for attempt in range(5):
try:
shutil.rmtree(entry)
return
except FileNotFoundError:
return
except OSError as e:
if attempt == 4:
raise
time.sleep(0.2 * (attempt + 1))
def _install(
package: Package,
lockfile: Lockfile,
facts: Facts,
store: Store,
target: str,
progress=None,
) -> None:
version = lockfile.version(package.name)
if version is None:
raise InstallError(
package.name, "not in the lockfile", "add it with `hermes pm lock --bump`"
)
reason = package.missing_reason(target)
if reason is not None:
raise InstallError(package.name, f"unavailable on {target}: {reason}", "none")
artifacts = lockfile.artifacts(package.name, target)
entry_name = package.store_entry(version, target)
with store.install_lock():
facts.reload()
if facts.installed(
package.name, version, store.root, _identity(lockfile, package.name, target)
):
return
entry = store.entry(entry_name)
_cond = store.published(entry_name) and package.verify(entry, target) == ""
if _cond:
_remove_entry(store, entry_name)
if not store.published(entry_name):
if not artifacts:
raise InstallError(
package.name,
f"no artifact for {target} in the lockfile",
"run `hermes pm lock --bump` for this package",
)
with store.scratch() as scratch:
staged = scratch / "tree"
try:
for index, artifact in enumerate(artifacts):
label = f"{index + 1}/{len(artifacts)}" if len(artifacts) > 1 else ""
archive = store.fetch(
artifact["url"], artifact["sha256"], scratch,
progress=_artifact_progress(
progress, index, len(artifacts)),
)
if progress is not None:
progress("unpack", 0, 0, label)
if index == 0:
package.unpack(archive, staged, target)
continue
# unpack() empties its destination by contract, so
# a second archive must be unpacked apart and moved
# in — extracting over `staged` would delete the
# first archive's files.
extra = scratch / f"extra-{index}"
package.unpack(archive, extra, target)
merge_tree(extra, staged)
package.stage(store, staged, version, target)
store.publish(staged, entry_name)
except InstallError:
raise
except Exception as e:
raise InstallError(package.name, f"install failed: {e}") from e
reason = package.verify(entry, target)
if reason:
raise InstallError(package.name, f"published entry failed verification: {reason}")
previous = facts.get(package.name)
if previous and "entry" in previous:
# Work item 6: replacing an ESTABLISHED fact is a repair —
# log it, no transaction system, no receipt file.
old_artifact = (previous.get("artifacts") or ["?"])[0]
old = f"{previous.get('version', '?')}/{str(old_artifact)[:12]}"
new = (
f"{version}/{artifacts[0]['sha256'][:12]}"
if artifacts
else version
)
LOG.info("repair: %s re-realized %s -> %s", package.name, old, new)
env = package.env(entry, target)
facts.record(
package.name,
version,
entry_name,
env,
store.root,
target=target,
artifacts=[a["sha256"] for a in artifacts],
digest=tree_digest(entry),
)
if previous and previous.get("version") != version:
package.migrate(previous["version"], version)
def ensure(
name: str,
*,
base_env: Optional[dict] = None,
explicit: bool = False,
progress=None,
) -> Runner:
"""``explicit`` marks a deliberate install command (`hermes pm
install`, `hermes pm bundle`) — those ARE the remedy the lazy-install
policy names, so the policy does not apply to them.
``progress(stage, done, total, label)`` reports the slow parts of an
install to a UI; see _artifact_progress.
"""
if isinstance(get_package(name), StatePackage):
sync_venv(explicit=explicit)
return Runner(name, compose_env([], base=base_env))
lockfile = _lockfile()
facts = _facts()
store = _store()
target = current_target()
chain = walk([name])
missing = [
p
for p in chain
if not facts.installed(
p.name, lockfile.version(p.name), store.root, _identity(lockfile, p.name, target)
)
]
if missing and (sealed() or (not explicit and not lazy_installs_allowed())):
raise _refuse_lazy(name, ", ".join(p.name for p in missing))
for package in missing:
_install(package, lockfile, facts, store, target, progress=progress)
if missing:
facts.reload()
diffs = [facts.env_for(p.name, store.root) for p in chain]
return Runner(name, compose_env(diffs, base=base_env))
def env_for(*names: str, base_env: Optional[dict] = None) -> dict[str, str]:
"""Composed env of already-installed packages only. Never installs,
never raises on missing packages — they contribute nothing."""
facts = _facts()
store = _store()
lockfile = _lockfile()
target = current_target()
diffs: list[dict] = []
for name in names:
try:
chain = walk([name])
except KeyError:
continue
for package in chain:
if facts.installed(
package.name,
lockfile.version(package.name),
store.root,
_identity(lockfile, package.name, target),
):
diffs.append(facts.env_for(package.name, store.root))
return compose_env(diffs, base=base_env)
def sync_venv(extras: Optional[list[str]] = None, *, explicit: bool = False) -> None:
"""Make the venv match uv.lock + the enabled extras. Extras union into
the installed state (one ledger); no-op when the stamp already matches.
``explicit`` marks a deliberate install command (`hermes pm install`,
`hermes update`) — those are the remedy the lazy-install policy points
at, so the policy does not apply to them."""
package = get_package("venv")
facts = _facts()
fact = facts.get("venv") or {}
enabled = sorted(set(fact.get("extras", [])) | set(extras or []))
stamp = package.expected_stamp(enabled)
if fact.get("stamp") == stamp:
return
if sealed() or (not explicit and not lazy_installs_allowed()):
raise _refuse_lazy("venv", str(extras) if extras else "venv out of sync")
with _store().install_lock():
facts.reload()
fact = facts.get("venv") or {}
enabled = sorted(set(fact.get("extras", [])) | set(extras or []))
stamp = package.expected_stamp(enabled)
if fact.get("stamp") == stamp:
return
package.apply(enabled)
facts.record_state("venv", stamp, enabled)
def adopt() -> bool:
"""First boot of a bundled install: verify the shipped payload, then
make it THIS machine's installed state. The payload's own shipped
``pm/lock.json`` (inside the repo snapshot) is the offline authority:
for every package it pins, the shipped fact must record the shipped
identity, package.verify() must pass, and the recorded realized digest
must match a fresh tree_digest() over the actual bytes. Any failure:
log the offending package, do NOT write `.adopted`, return False —
adopt() refuses to vouch for bytes it could not prove.
Idempotent and cheap-ish: returns False when there is nothing to
adopt (no shipped facts, or already adopted)."""
store = _store()
facts = _facts()
if not paths.facts_path().is_file():
return False
marker = store.root.parent / ".adopted"
if marker.is_file():
return False
shipped_lock = paths.repo_root() / "pm" / "lock.json"
if shipped_lock.is_file():
lockfile = Lockfile(shipped_lock)
target = current_target()
for name in lockfile.names():
try:
package = get_package(name)
except KeyError:
continue
if isinstance(package, StatePackage):
continue
fact = facts.get(name)
if not facts.installed(
name, lockfile.version(name), store.root, _identity(lockfile, name, target)
):
LOG.warning(
"pm adopt: refusing %s: fact missing, legacy (no recorded "
"identity), or does not match the shipped lock",
name,
)
return False
entry = store.entry(fact["entry"])
reason = package.verify(entry, target)
if reason:
LOG.warning(
"pm adopt: refusing %s: staged entry failed verification: %s",
name, reason,
)
return False
if fact.get("digest") != tree_digest(entry):
LOG.warning(
"pm adopt: refusing %s: realized bytes do not match the "
"recorded digest",
name,
)
return False
try:
marker.write_text("", encoding="utf-8")
except OSError:
pass
return True
def check() -> list[str]:
"""The startup check: cheap stamp comparisons of the installed state
against the lockfile. Returns problems; empty means healthy. Never
installs, never touches the network. An install pm has never touched
(no installed-state file) reports nothing — pm only vouches for what
it installed. Lockfile packages this build doesn't know (version skew
during a partial update) are skipped, not fatal."""
if not paths.facts_path().is_file():
return []
problems: list[str] = []
lockfile = _lockfile()
facts = _facts()
store = _store()
target = current_target()
for name in lockfile.names():
try:
package = get_package(name)
except KeyError:
continue
if package.optional or package.internal:
continue
if package.missing_reason(target) is not None:
continue
if not facts.installed(
name, lockfile.version(name), store.root, _identity(lockfile, name, target)
):
problems.append(f"{name}: not installed or outdated")
try:
venv = get_package("venv")
except KeyError:
venv = None
fact = facts.get("venv")
if venv is not None and fact is not None:
expected = venv.expected_stamp(fact.get("extras", []))
if fact.get("stamp") != expected:
problems.append("venv: out of sync with uv.lock")
return problems
def _store_path_dirs() -> list[str]:
"""Composed PATH dirs of all installed (non-internal, on_path) store
packages, deps-first, deduped. Includes optional packages that are
*installed* (facts say so) — an installed git/gh must be on PATH even
though it's not in the root closure. Never installs."""
import os
if not paths.facts_path().is_file():
return []
facts = _facts()
store = _store()
lockfile = _lockfile()
target = current_target()
dirs: list[str] = []
for name in lockfile.names():
try:
package = get_package(name)
except KeyError:
continue
if package.internal:
continue
if not getattr(package, "on_path", True):
continue
if package.missing_reason(target) is not None:
continue
if not facts.installed(
name, lockfile.version(name), store.root, _identity(lockfile, name, target)
):
continue
env = facts.env_for(name, store.root)
path_dirs = env.get("PATH") or []
if isinstance(path_dirs, str):
path_dirs = [path_dirs]
for directory in path_dirs:
if directory and directory not in dirs:
dirs.append(str(directory))
return dirs
def activate() -> None:
"""Make the installed store usable: prepend its tool dirs to
os.environ['PATH'] so reactive `shutil.which('git'|'bash'|'ffmpeg'|...)`
resolves the bundled binaries. The gate is `check()` — if the store is
broken, refuse to inject (fail fast rather than serving a partial PATH).
This is the ONE sanctioned global PATH write: PATH is the discovery
contract every `which` reads, not a tool-specific env leak. Store-first
unconditionally — pinned bundled versions win on dev machines too.
"""
import os
if check():
return # broken store → do not provision; callers surface `hermes pm install`
dirs = _store_path_dirs()
if not dirs:
return
existing = os.environ.get("PATH", "")
prefix = os.pathsep.join(dirs)
existing_lower = {p.lower() for p in existing.split(os.pathsep) if p}
missing = [d for d in dirs if d.lower() not in existing_lower]
if missing:
os.environ["PATH"] = os.pathsep.join([*missing, existing]) if existing else os.pathsep.join(missing)
def uv(*, venv=None, realize: bool = True):
"""TRANSITIONAL: (uv path, sanitized env) for call sites that still
drive uv themselves. Two classes remain: update/repair sites (die with
the update collapse, plan step 4) and side-venv installs — browser-use
tool venvs (tools_config, browser_use_cli) and hindsight's
local_embedded daemon — which survive until pm grows the side-venv
package kind (plan step 5's remaining half). Must not spread."""
from pm.packages import uv_env
env = uv_env()
if venv is not None:
env["VIRTUAL_ENV"] = str(venv)
env.pop("UV_NO_CONFIG", None)
lockfile = _lockfile()
facts = _facts()
store = _store()
package = get_package("uv")
if not facts.installed(
"uv", lockfile.version("uv"), store.root, _identity(lockfile, "uv", current_target())
):
if not realize or not lazy_installs_allowed():
return None, env
try:
_install(package, lockfile, facts, store, current_target())
facts.reload()
except Exception:
import logging
logging.getLogger(__name__).debug("pm.uv: install failed", exc_info=True)
return None, env
fact = facts.get("uv")
if fact is None:
return None, env
binary = package.binary(store.entry(fact["entry"]), current_target())
if binary is None or not binary.is_file():
return None, env
return str(binary), env
Loaded 100 of 272 files, more files were not shown because too many files have changed in this diff. Show more