Commit Graph

195 Commits

Author SHA1 Message Date
42871ee64f fix(reinstall): stop reinstall from wiping template link on delete attempt, support ISO-based QEMU reinstall
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 1m44s
Deploy to Production / Trigger production update (push) Failing after 10s
SonarCloud / SonarCloud Analysis (push) Failing after 3m0s
VM delete pre-save was calling save_vm_instance() without template_id/
template_name, which unconditionally overwrites those fields on the
existing cached row - so any non-force delete attempt (even one that
later failed) permanently detached the VM from its source template and
broke reinstall. Now the previous values are carried through.

Also extend reinstall eligibility and the reinstall worker to cover
QEMU VMs created via "Create VM from ISO" (template_name holds the ISO
volid, same storage:content/file pattern already handled for LXC CT
templates), recreating the VM from the same ISO on reinstall.
2026-08-25 09:17:35 +05:00
markmorado
33e5620311 fix(proxmox): re-export cleanup_expired_connections from the package
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 1m44s
Deploy to Production / Trigger production update (push) Failing after 9s
SonarCloud / SonarCloud Analysis (push) Failing after 2m48s
The function lives in client.py but was missing from the package
__init__, unlike its neighbours clear_all_cache and clear_server_cache.
The monitoring worker imports it from app.proxmox inside a try/except
ImportError, so nothing crashed — the hourly job was silently skipped
with "Proxmox cache cleanup not available" and the connection cache was
never pruned, keeping entries older than an hour alive until restart.
2026-08-24 03:30:39 +05:00
abc819294c fix(images): unblock guest-exec in RHEL-family cloud images
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 1m45s
Deploy to Production / Trigger production update (push) Failing after 12s
SonarCloud / SonarCloud Analysis (push) Failing after 3m3s
The RHEL-family qemu-guest-agent package starts the agent with an allow-list
that omits guest-exec, guest-exec-status and guest-file-* (FILTER_RPC_ARGS in
/etc/sysconfig/qemu-ga; older packages spelled the same restriction as
BLOCK_RPCS or BLACKLIST_RPC). The agent answered "The command guest-exec has
been disabled for this instance", so running scripts in a VM, assigning extra
IP addresses and growing the filesystem after a disk resize did not work on
Rocky Linux, AlmaLinux, CentOS Stream and Fedora guests.

- patch-cloud-images.sh: clear all three variants while building the mirror
  (UNBLOCK_RPC=0 opts out) and add an ONLY filter to rebuild only part of the
  catalog. Three `[ ... ] && ...` statements that aborted the script under
  set -e are now plain ifs -- the one before mirrors.json generation killed
  every run that passed BASE_URL
- builtin.py: point the five RHEL-family amd64 entries at the rebuilt mirror
  release 2026-08-22 with their new checksums. Ubuntu and Debian images are
  unaffected and stay in the previous releases, so older panel versions keep
  working
- client.py: annotate the Proxmox error with what to clear inside the guest,
  the raw message names neither the file nor the variable
- WIKI, CHANGELOG: document the restriction and the unblock command
2026-08-22 14:03:49 +05:00
972d54d1f1 fix(ipam): keep extra addresses across a guest reboot
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 1m44s
Deploy to Production / Trigger production update (push) Failing after 11s
SonarCloud / SonarCloud Analysis (push) Failing after 3m0s
Testing on a live AlmaLinux container showed the alias disappearing after
a restart even though it was reported as applied: Proxmox regenerates the
guest network configuration on every start (pve-container rewrites
/etc/network/interfaces, ifcfg or the NM keyfile from netN), wiping any
address added to the native stack.

    before reboot: address1=10.10.10.103/24 address2=10.10.10.106/24
    after reboot:  addresses=10.10.10.103/24     (file recreated by PVE)

Install a oneshot systemd unit that re-adds the recorded addresses after
network-online.target — /etc/systemd is not touched by Proxmox. The native
branches stay, so persist is now reported as e.g. "nm+systemd"; the unit
and its helper are removed once the last address is released.

Also fixed while testing on the two test instances:
- the alias script is ASCII-only now: it travels through the Proxmox API
  and its marker ends up in guest system configs
- a composite persist marker no longer falls through the parser, which
  used to report a successful apply as runtime_only
- allocate_ip marks the first address of a guest as primary on its own,
  so allocations coming from the creation wizard are flagged correctly
2026-08-22 01:36:10 +05:00
6ac505e481 fix(proxmox): send guest-agent scripts as bytes
Proxmox base64-encodes the input-data of an agent exec in Perl, which dies
with "Wide character in subroutine entry" as soon as the payload contains
non-ASCII — a Cyrillic comment in a script is enough, so running any such
script on a VM failed with a 500.

Hand the same UTF-8 bytes over as latin-1: the JSON carries plain bytes,
Perl is happy and the guest still receives valid UTF-8.
2026-08-22 01:35:56 +05:00
6796903712 feat(ipam): several IP addresses per instance
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 1m46s
Deploy to Production / Trigger production update (push) Failing after 10s
SonarCloud / SonarCloud Analysis (push) Failing after 3m7s
A guest was implicitly limited to a single allocation: lookups were
"(server, vmid) -> one row", release_ip_by_vmid freed one address, and
the link tool skipped guests that already had one. An instance can now
hold a primary address plus any number of aliases, managed from the
Network tab of the instance and visible on the IPAM pages.

Schema and service:
- ipam_allocations gains is_primary, assignment_kind, target_interface
  and apply_* columns; migration 43 marks existing rows as primary
- find_allocations_by_resource/set_primary; release_ip_by_vmid now frees
  every address of a deleted guest instead of just the first one
- next-available search may scan the whole subnet when a network has no
  auto-assign pool, skipping node and cached guest addresses so it never
  hands out an address that is already live

Applying to the guest (services/guest_ip_service.py):
- reuses script_engine, so LXC goes through pct exec and VMs through the
  guest agent; the shell script synchronises the whole alias list of an
  interface, which makes add, remove and retry the same code path
- persists via ifupdown, netplan, NetworkManager or sysconfig and never
  restarts guest networking; an unknown stack degrades to runtime_only
  and an unreachable guest keeps the allocation as pending

API and UI:
- /ipam/api/guests/{server_id}/{vmid}/addresses: list, add, apply,
  primary, delete; allocations list accepts guest filters
- instance list exposes ips[] alongside the primary ip
- addresses card on the Network tab, "+N" badge in the IP column,
  primary badge in the allocations table

Fixes found along the way:
- link-allocations registers every address of a guest, carries the MAC
  over, refuses to take an address away from another live guest and no
  longer matches allocations by resource name (which leaked an address
  between same-named guests on different servers)
- IPAM history no longer fails to serialise datetime values, which used
  to abort re-linking with a 500
2026-08-21 23:48:20 +05:00
c9d8e8207f fix(proxmox): read LXC addresses from netN and live interfaces
get_container_interfaces() looked for ipconfigN keys, which exist only on
QEMU cloud-init guests. LXC keeps the address inside the netN string, so
the method never returned anything and containers showed no IP at all.

Parse netN with the existing net_parser, fall back to
/nodes/{node}/lxc/{vmid}/interfaces for DHCP guests (skipped when the
config already carries a static address or the container is stopped), and
validate addresses so loopback, link-local and the literal "dhcp" are
dropped. IPv6 from ip6/inet6 is reported as well.
2026-08-21 23:47:45 +05:00
7d60afb9fa fix(console): stop leaking termproxy sessions and orphaned terminals
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 1m22s
Deploy to Production / Trigger production update (push) Failing after 11s
SonarCloud / SonarCloud Analysis (push) Failing after 2m39s
Opening an LXC console left a failed task on the Proxmox node:

  failed waiting for client: timed out
  TASK ERROR: command '/usr/bin/termproxy 5900 ... vzctlconsole101' failed

connectTerminal() called GET /api/{sid}/container/{vid}/terminal first,
which creates a termproxy session on the node, but only the `node` field
of the response was used - port and ticket were discarded. The ws
endpoint then created a second termproxy session and connected to that
one. Nobody ever attached to the first, so Proxmox waited ~10s for a
client and failed the task.

The console also rendered a black screen in dev: React StrictMode mounts
the effect twice, connect* are async, and the effect cleanup ran before
they had assigned wsRef/termRef, so it had nothing to clean up. Two
xterm instances ended up in the same container - the first one filled
the viewport while the live one was pushed out of the overflow-hidden
area, which is why the badge read "connected" over an empty screen.

- drop the redundant REST call; the ws endpoint owns the termproxy
- remove the now unused GET /api/{server_id}/container/{vmid}/terminal
- add a per-session cancellation guard: a stale connect closes its own
  socket and disposes its own terminal without touching refs that may
  already point at the live session
- route mount and the reconnect button through a single startSession()
  that cancels and tears down the previous session first
- apply the same guard to the node shell page
2026-08-21 21:06:44 +05:00
30e92da8b5 fix(rbac): make network and IPAM access admin-only by default
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 1m26s
Deploy to Production / Trigger production update (push) Failing after 10s
SonarCloud / SonarCloud Analysis (push) Failing after 2m46s
network:view is no longer implied by server:view, and neither network:view
nor ipam:view is granted to the non-admin default roles (operator, user,
demo). The network topology, node/SDN interface configuration and IPAM are
administrative surfaces, not part of the baseline "can see servers"
umbrella, so they should be granted explicitly rather than inherited.
2026-08-21 20:26:38 +05:00
markmorado
31d17fca61 feat: scan NFS server for available exports in storage-add dialog
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 1m20s
Deploy to Production / Trigger production update (push) Failing after 10s
SonarCloud / SonarCloud Analysis (push) Has been cancelled
Add a "scan exports" button next to the server field for NFS storages
that calls Proxmox's /nodes/{node}/scan/nfs and turns the export path
input into a select populated with the real exports found on the
target server, instead of requiring the path to be typed by hand.
2026-08-20 10:23:04 +05:00
markmorado
943207b6b7 fix: support moving LXC container disks to other storage
Disk move was hardcoded to the QEMU move_disk Proxmox API and was
never reachable in the UI for LXC (no Hardware tab for containers,
and the disk list regex only matched VM device names).

- add move_container_disk() using lxc/{vmid}/move_volume
- add POST /api/{server_id}/container/{vmid}/disk/move endpoint
- route useMoveDisk by instance type (vm vs container prefix)
- recognize rootfs/mp\d+ as LXC disk devices
- add a Move disk card on the Settings tab for LXC instances
- filter target storage list by supported content type
  (rootdir for LXC, images for VM) instead of showing all storages
2026-08-20 05:29:28 +05:00
markmorado
da8bbce62a fix: console viewport sizing and add VNC control toolbar
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 1m22s
Deploy to Production / Trigger production update (push) Failing after 10s
SonarCloud / SonarCloud Analysis (push) Failing after 2m30s
Fix nested h-screen flexbox layout bug that let the noVNC canvas overflow
below the viewport until the browser zoom level was changed (missing
min-h-0 on flex children). Also add a fuller console control bar: power
menu (reboot/shutdown/hard reset/hard stop), extra-keys menu (Ctrl+Alt+Del,
Ctrl+Alt+Backspace, Tab, Esc, Ctrl+Alt+F1-F12), scale toggle, clipboard
paste to guest, and console screenshot. Adds a VM hard-reset backend
action (qemu status/reset) alongside the existing start/stop/restart/shutdown.
2026-08-20 02:04:15 +05:00
markmorado
d99e67a60e feat: add network topology page
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 1m22s
Deploy to Production / Trigger production update (push) Failing after 10s
SonarCloud / SonarCloud Analysis (push) Failing after 2m22s
Interactive infrastructure map (panel -> cluster -> node -> guest) rendered
with React Flow, backed by a single aggregated /api/network/topology endpoint
instead of fanning out over servers/nodes/SDN/VMs. Adds a netN NIC parser for
QEMU/LXC guest configs, status/threshold filters, canvas tools, and PNG/SVG
export.
2026-08-20 01:31:37 +05:00
67fe01e9b1 fix: show friendly OS name for LXC deployed from CT template
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 1m26s
Deploy to Production / Trigger production update (push) Failing after 12s
SonarCloud / SonarCloud Analysis (push) Failing after 12m14s
Derive a human-readable OS name (e.g. Debian 13, Ubuntu 24.04) from the
vztmpl volid instead of showing the raw template path like
local:vztmpl/debian-13-standard_13.1-2_amd64.tar.zst in the instances list.
Applied at the display layer, so existing instances are fixed too.
2026-08-19 15:30:34 +05:00
markmorado
73493d7ef2 Add active tasks indicator, propagate LXC create errors, fix Fedora logo
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 1m24s
Deploy to Production / Trigger production update (push) Failing after 10s
- Surface a topbar indicator with a live count of active tasks
- Propagate exceptions from Proxmox LXC container creation instead of
  swallowing them, and validate minimum password length
- Replace the Fedora catalog icon with the official logo artwork
2026-08-18 23:34:32 +05:00
markmorado
9d57e388cd feat: add 5 new cloud images to built-in catalog
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 1m28s
Deploy to Production / Trigger production update (push) Failing after 1m37s
Add AlmaLinux 10, CentOS Stream 9, Debian 13 (Trixie), Fedora 41, and
Ubuntu 26.04 LTS (Resolute) with pre-installed qemu-guest-agent. Images
patched via virt-customize and hosted on GitHub release 2026-08-18.
2026-08-18 23:11:13 +05:00
8cbba05dac fix: validate instance name as DNS label and improve wizard UX
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 1m23s
Deploy to Production / Trigger production update (push) Failing after 10s
Add DNS name validation (RFC 952/1123) for VM, LXC and ISO instance
names on both backend and frontend. Proxmox rejects names with
underscores or special characters but the error was only surfaced as
a raw 400 from the API.

Also require Cloud-Init user and password fields for VM deploys
(highlighted when empty, Next button blocked), and fix numeric input
fields that showed "0" on clear and concatenated values like "04096".
2026-08-17 16:24:55 +05:00
319de5c96e feat: swap SSH keys when admin changes instance owner
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 1m21s
Deploy to Production / Trigger production update (push) Failing after 10s
Remove the previous owner's SSH keys and apply the new owner's keys
(profile + library) when ownership is reassigned. If the new owner has
no keys, the old keys are simply removed.

QEMU VMs: updates cloud-init sshkeys via Proxmox API.
LXC containers: writes /root/.ssh/authorized_keys via exec.
The swap is best-effort — failures are logged but do not block the
ownership change.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-08-14 19:07:59 +05:00
4d00fdf959 catalog: point amd64 images to GitHub mirror with qemu-guest-agent and sha256
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 1m21s
Deploy to Production / Trigger production update (push) Failing after 10s
2026-08-14 16:46:28 +05:00
f29226a735 fix: filter out loop devices and system partitions from VM storage overview
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 1m20s
Deploy to Production / Trigger production update (push) Failing after 10s
Only show user-relevant filesystems by skipping snap/loop mounts
and /boot/* partitions in QEMU guest agent fsinfo response.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-08-14 16:40:01 +05:00
8ff757f8c1 fix: re-download cached import images when checksum is provided
Some checks failed
Deploy to Production / Trigger production update (push) Has been cancelled
Backend Tests / pytest (Python 3.12) (push) Has been cancelled
volume_exists() only checked file presence, not content. A stale image
from a previous download (different URL/version) would be silently reused,
causing templates and VMs to be created from unpatched images even when
the mirror's checksum pointed to a newer patched build.

Add delete_volume() to ProxmoxClient storage mixin, and update both
_do_image_download_sync and _do_image_template_sync to delete the cached
volume and re-download whenever a checksum is provided, letting Proxmox
verify the new download server-side.
2026-08-14 16:19:41 +05:00
8337539e9e i18n(users): translate the terminated-sessions toast
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 2m12s
Deploy to Production / Trigger production update (push) Failing after 11s
Both toasts printed the endpoint's own message field verbatim, so a Russian UI
answered "Terminated 0 sessions". That string is assembled in Python and can
never be translated where it is built, so the endpoints now also return the raw
count and the phrase is composed on the client.

message is kept: it goes to the audit log and may have other consumers, and
adding a field breaks no one.

Russian needs three forms, and zero does not land in the obvious one — 0 is the
"many" category ("сессий", not "сессии") — hence four keys against English's
two.

Fixes both call sites: the per-user button in the table row and Terminate all
sessions on the Sessions tab, which shared the same untranslated string.
2026-08-12 15:00:07 +05:00
bddbf15652 feat(rbac): split setting:view into panel settings and a permission-free profile
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 1m22s
Deploy to Production / Trigger production update (push) Failing after 11s
setting:view used to gate the whole /settings page, which mixed panel-wide
configuration with a user's own account. Revoking it to keep a standard user
out of the panel settings also took away their profile, password, 2FA and SSH
keys, so the permission could not be used for what it names.

The account half now lives at /profile: its endpoints are self-scoped and only
require a session, so the page carries no permission at all and is reached from
the user menu (and Ctrl+K) rather than a sidebar group. /settings keeps the
panel-wide tabs behind setting:view, with saving gated on setting:update,
security on setting:manage and the SMTP/Telegram channels on is_admin, matching
what each endpoint checks.

The shipped "user" role no longer gets setting:view. Migration 42 drops it from
roles holding it without setting:update/manage, since those grants only existed
to reach the profile page; it runs once (marker in panel_settings) so a later
deliberate grant is not reverted on the next start.
2026-08-11 20:07:34 +05:00
c4fda1b0bb feat(migrations): backfill the owner of instances created before the fix
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 1m20s
Deploy to Production / Trigger production update (push) Failing after 10s
Instances deployed while save_vm_instance() dropped the owner are still sitting
in the database with owner_id = NULL, so their users cannot see them. Recover
the creator from the two places it was recorded:

  1. deploy_tasks — user_id is a real foreign key stored when the deploy was
     queued. server_id is absent for LXC deploys, so a NULL there is treated as
     matching any server instead of discarding the row.
  2. audit_logs — vm_create / ct_create / lxc_create entries, which only carry a
     username and are matched by name.

Ownership decides who may see, stop and delete an instance, so a wrong guess is
worse than no owner at all: a row is updated only when every applicable piece of
evidence names the same existing user. VMIDs get reused, so conflicting evidence
is expected and those instances deliberately stay ownerless for an operator to
assign with assign_instances_owner.py. Existing owners, soft-deleted rows,
failed create events and logs from deleted users are all left alone, which also
makes the migration idempotent.

Note this recovers who created an instance, not who it was meant for: a VM an
admin created on behalf of a user is attributed to the admin, because the
intended owner was never persisted before the fix.

The attribution rule is a pure function (pick_backfill_owner) so it can be
tested directly. The migration's own statements are portable, so the tests run
it against SQLite with the two information_schema helpers stubbed out.
2026-08-11 17:28:59 +05:00
1152410860 fix(instances): assign the owner on create and inject the owner's SSH keys
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 2m30s
Deploy to Production / Trigger production update (push) Failing after 10s
Newly created VMs/LXCs ended up with owner_id = NULL, and a VM created by an
admin on behalf of a user received the admin's SSH key instead of the user's.

Ownership: save_vm_instance() only set owner_id when inserting a new row. The
row practically always existed by then — sync_vm_cache runs every 10s and
registers VMs it sees on Proxmox with owner_id = NULL while the clone is still
running, whereas the deploy saves the owner minutes later at 95% progress. So
every deploy landed in the update branch and silently dropped the owner. Set
owner_id there too, but only when it is passed explicitly: callers such as
save-before-delete do not know the owner and must not blank it out.

SSH keys: the profile key was taken from whoever pressed "Create" rather than
from the instance owner, and the LXC path injected no profile key at all. Add
build_deploy_ssh_keys(), which merges pasted keys, keys picked from the library
(same authorization rules as before) and the owner's profile key, deduplicated.
Both the template and LXC deploy endpoints now use it, so the key material is
resolved in one place instead of inside the deploy worker.

ISO VMs are left alone on purpose: they are blank VMs without cloud-init, so
there is nowhere to inject keys.

Covered by backend/tests/test_instance_ownership.py, including the sync race.
2026-08-11 17:19:19 +05:00
markmorado
b7c3a6ed35 feat(instances): ISO eject/mount with boot-order switch and hybrid reboot
Some checks failed
Deploy to Production / Trigger production update (push) Has been cancelled
Backend Tests / pytest (Python 3.12) (push) Has been cancelled
- Images: show real ISO files already on the node storage (incl. those
  present before the node joined the panel) via existing node/isos endpoint
- ISO dialog: rename "detach" to "Eject", show current image inside the
  select (drop overflowing side text), add "boot from disk" checkbox
- Backend: on eject, put disk first in boot order; on mount, optional
  "boot from this ISO" puts CD-ROM first
- Reboot to apply the new boot order via hybrid restart: graceful ACPI
  shutdown with forceStop fallback, then start (fixes guest-ping timeout
  on freshly installed OS without QEMU Guest Agent)
- Update iso_next_steps hint to reflect the automated flow
2026-07-24 23:00:32 +05:00
markmorado
9b493da7ad feat(cluster): import existing Proxmox cluster into the panel
Some checks failed
Deploy to Production / Trigger production update (push) Has been cancelled
Backend Tests / pytest (Python 3.12) (push) Has been cancelled
Add detect/import flow for nodes already clustered on the Proxmox side
(outside the panel). Matches nodelist entries from cluster/config/join
against known panel servers by IP/hostname and links their panel
records to the cluster_name group, without running any pvecm operation.
2026-07-24 06:35:34 +05:00
markmorado
99d8ce750d feat(images): support uploading local ISO/vztmpl files to node storage
Some checks failed
Deploy to Production / Trigger production update (push) Has been cancelled
Backend Tests / pytest (Python 3.12) (push) Has been cancelled
Adds a local file upload path alongside the existing URL download flow:
a new /images/upload endpoint streams the file to a temp path and hands
it to a background worker that proxies it into Proxmox storage via the
upload API (requests-toolbelt handles the large multipart stream). Also
adds /images/resolve-filename to recover the real filename (Content-
Disposition) for URL-based downloads with opaque paths.

Raise nginx client_max_body_size and timeouts on both the frontend proxy
and the standalone nginx configs so multi-GB ISOs don't get cut off.
2026-07-24 06:17:46 +05:00
markmorado
8b4c8c3a30 fix(realtime): sync instance/workspace list state without manual refresh
- Invalidate workspace list cache after assigning servers/users so
  server/user counts on workspace cards update immediately.
- Invalidate node/VM caches after workspace server/user assignment so
  the Nodes page reflects visibility changes right away.
- Soft-delete the cached VM/LXC row and broadcast vm_deleted the moment
  a VM/container is destroyed, instead of waiting for the 10s
  background sync_vm_cache tick.
- Invalidate the VM list on the frontend when a deploy task completes,
  since the backend already persists the new instance to cache
  synchronously before marking the task done.
2026-07-24 05:39:23 +05:00
markmorado
e10f9a2a03 refactor(servers): unify connection status badges
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Has been cancelled
Deploy to Production / Trigger production update (push) Has been cancelled
Extract ServerStatusBadge so every server list renders the auth-error
state identically instead of repeating the online/offline ternary, and
expose last_error_kind from the cluster topology and user-assignment
endpoints, which build their payloads by hand and so did not inherit it
from the server schema.
2026-07-24 05:04:45 +05:00
markmorado
ebc692a324 feat(instances): create blank VMs for ISO installs and download ISOs
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Has been cancelled
Deploy to Production / Trigger production update (push) Has been cancelled
Add a third instance kind to the create wizard: a blank KVM VM with an
empty disk, the installation ISO mounted on ide2 and put first in the
boot order, so the guest boots straight into the installer over noVNC.
Picking a Windows guest presets OVMF, a SATA disk and an e1000 NIC,
which the installer sees without virtio drivers loaded.

Also teach the image downloader content=iso, so ISOs can be fetched
onto a node from the panel: by URL from the images page or from a saved
mirror (mirrors now accept kind=iso), and straight from the ISO step of
the wizard.

The bundled catalog stays qcow2-only on purpose — install ISOs have no
stable "latest" URLs, so hardcoded links would rot on every point release.
2026-07-24 05:00:51 +05:00
markmorado
075b8cdaaa feat(servers): distinguish rejected credentials from unreachable nodes
An offline server used to report a single "Failed to connect" state, but
"the node does not answer" and "the node answered 401" are fixed in
completely different ways — the first is power or network, the second is a
stale token, which is what happens when a node joins a cluster and its
user.cfg and priv/token.cfg are replaced by the cluster ones.

- add ProxmoxClient.check_connection() returning (ok, error_kind, error);
  token auth surfaces 401 as ResourceException rather than
  AuthenticationError, since proxmoxer validates the token on first use
- store the reason in proxmox_servers.last_error_kind (migration 40) and
  expose it through the server response
- surface an auth_error state in the UI with a hint explaining the usual
  cause

Storage listings now also report per-node availability: the cluster config
declares a storage for the whole cluster but says nothing about whether it
actually came up on a given node, so a missing volume group or pool stayed
invisible. Availability comes from a single /cluster/resources call.
2026-07-24 04:42:59 +05:00
markmorado
65c61e7886 feat(servers): auto-provision API tokens and add node metric graphs
Servers can now be connected with a login and password: the panel
authenticates once, creates its own API token on the node (privsep=0) and
uses it from then on. The token can be revoked from the Proxmox UI, cannot
be used for SSH or web login, and appears in the PVE audit log under its
own name. The password stays stored (encrypted) because ProxmoxClient needs
it for SSH fallbacks such as pvecm delnode.

- add proxmox/token_provisioning.py: token creation with free-name
  selection and revoke-on-failure so a failed registration does not leave
  an orphaned token whose secret was never persisted
- POST /api/servers accepts auto_create_token
- add POST /api/servers/{id}/provision-token to migrate servers that are
  still on password auth
- store api_token_value as EncryptedString; it was kept in plaintext while
  the password next to it was already encrypted
- drop the unused ProxmoxServerBase schema, a stale copy of the create
  schema whose validator rejected the new auth mode

Node detail pages gain a Graphs tab (CPU, IO delay, memory, load average,
network, swap, root filesystem) built on the existing node rrddata
endpoint. The chart card is extracted from the instance graphs tab so both
views share it, and X axis ticks now switch to dates on multi-day ranges.
2026-07-24 04:03:30 +05:00
markmorado
ff1396142b fix(api): scope server listings to workspace, stop serialising Proxmox calls
/api/cluster/topology and /api/resources/all read every ProxmoxServer row
regardless of the caller access or the X-Active-Workspace header, so the
Cluster page listed all servers and dashboard aggregates counted the whole
panel instead of the active workspace. Extract the access + workspace filter
out of /api/servers into get_visible_server_ids() and apply it in all three.

Live metrics were also gathered server by server with the cache lock held
across the network calls, so a single unreachable cluster stalled the request
(21-31s in the logs) and concurrent requests queued behind it. Fetch in
parallel, skip servers already known offline, drop the redundant
is_connected() probe, and hold the lock only around the cache dict.

/api/resources/all now orders by server id: without it Postgres returned heap
order, which shifts whenever update_status rewrites a row, so the server list
reshuffled between requests.
2026-07-24 03:18:14 +05:00
markmorado
e67f6b7249 fix(update): recover from stalled panel self-update instead of hanging at 80%
Some checks failed
Deploy to Production / Trigger production update (push) Has been cancelled
Backend Tests / pytest (Python 3.12) (push) Has been cancelled
The panel update status lived only in-memory and was cleared solely by the
host watchdog restarting the container. If the rebuild failed mid-way (e.g. the
build OOMs the host when no swap is configured), the container never restarted
and the UI stayed at "restarting - 80%" forever.

- update_service: get_update_status() now self-heals — after UPDATE_TIMEOUT_SECONDS
  (default 900s) of "is_updating" it transitions to a terminal "failed" state with
  a clear error pointing at the host update service / logs.
- use-settings: poll update status while the server reports is_updating (not only
  right after the click in the same tab), so reloading a stuck page keeps refreshing.
- SettingsPage: banner now renders the "failed" state with the error message.
- update_host.sh: build images sequentially (app, then frontend) to lower peak
  memory and avoid the OOM that killed the build mid-run.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 22:57:41 +05:00
markmorado
0b0b11c689 feat(scripts): run script executions in background and poll for completion
Community scripts (build.func) install packages like Java/OpenSearch and
can take minutes, well past the nginx/proxy HTTP timeout. execute_script
now returns immediately with status "running" and finishes the job on a
background thread with its own DB session; the frontend polls
/api/scripts/executions/{id} every 3s while running.

Also set TERM=xterm for script execution over SSH — community scripts
call `clear` and print colored output, which fails with "TERM
environment variable not set" when run without a PTY.
2026-07-23 21:59:18 +05:00
markmorado
761bd7bca8 fix(workspaces): grant server access via workspace membership alone
Non-privileged users previously needed a server to be both directly
assigned AND inside one of their workspaces (intersection) to see or use
it. Adding a user to a workspace should be sufficient on its own to let
them create instances on that workspace's servers, so the two access
paths are now unioned instead of intersected.
2026-07-23 21:59:09 +05:00
markmorado
ef5b40b2f7 fix(nodes): не давать пустым полям учётных данных затирать токен/пароль сервера
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Has been cancelled
Deploy to Production / Trigger production update (push) Has been cancelled
При редактировании ноды (например, при смене имени) форма отправляла
пустые api_token_name/api_token_value, т.к. секреты не возвращаются
с бэкенда. Бэкенд принимал их как явно заданные и затирал реальные
credentials, из-за чего немедленная проверка подключения падала и
сервер помечался offline. Теперь пустые значения при редактировании
трактуются как «не менять» — и на фронтенде, и на бэкенде.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-23 21:57:27 +05:00
markmorado
43336c02c7 feat(auth): имперсонация — вход админа под пользователем и возврат обратно
Some checks failed
Deploy to Production / Trigger production update (push) Has been cancelled
Backend Tests / pytest (Python 3.12) (push) Has been cancelled
Backend:
- POST /api/auth/impersonate/{user_id} — только для админа, выдаёт JWT
  с клеймом impersonator; запрещает вход под собой, отключённым,
  другим админом и вложенную имперсонацию
- POST /api/auth/stop-impersonate — по клейму impersonator завершает
  сессию имперсонации и перевыпускает токен исходного админа
- /api/auth/me отдаёт поле impersonator для баннера в SPA
- аудит-лог: impersonate_start / impersonate_stop

Frontend:
- auth-store: действия impersonate/stopImpersonate со сбросом кэша
  React Query при смене токена
- UsersPage: кнопка "войти под пользователем" (только админу,
  скрыта для себя/админов/неактивных и во время имперсонации)
- AppLayout: баннер имперсонации с кнопкой "вернуться к администратору"
- локали ru/en

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 05:33:04 +05:00
markmorado
42c6acd52e fix(instances): изоляция по владельцу для живых ресурсов Proxmox
Some checks failed
Deploy to Production / Trigger production update (push) Has been cancelled
Backend Tests / pytest (Python 3.12) (push) Has been cancelled
Список инстансов (/api/virtual-machines) уже фильтровался по owner_id, но
эндпоинты живых ресурсов Proxmox — /api/resources/all (дашборд) и
/api/{server_id}/resources (страница ноды/сервера) — отдавали все VM/LXC без
фильтра. Обычный пользователь (proxmox.view) через страницу ноды видел чужие
инстансы.

- _helpers: get_owned_vmids() — VMID, которыми владеет пользователь
- servers: get_server_resources / get_all_resources фильтруют VM/LXC по
  владельцу для не-привилегированных (сверка vmid с кэшем vm_instances);
  админы и роли с vm:manage видят всё
- assign_instances_owner.py: скрипт массового назначения владельца
  существующим инстансам (owner_id синхронизированных из Proxmox = NULL)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 04:56:17 +05:00
markmorado
4621ffe88a fix(appstore): data files, надёжность установки/удаления и сворачиваемые диалоги
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 57s
Deploy to Production / Trigger production update (push) Successful in 8s
Доставка файлов приложения из каталога (data_files) до compose up,
APP_PASSWORD/default_credentials для Umbrel, сворачивание диалогов
долгих операций (MinimizableDialog), фильтрация pull-шума в ошибках.

Исправления багов:
- teardown: отсутствующий в Proxmox CT считается удалённым — orphaned-приложение
  теперь можно удалить из БД
- install: сдвиг VMID мимо занятых записей (nextid не резервирует номер,
  retry мог подхватить чужой контейнер)
- retry: параметры установки сохраняются в installed_apps.install_params
  (cores/memory/disk/storage/bridge/ip_config/port_explicit, миграция JSONB)
  вместо жёстких дефолтов
- reconcile: недоступный сервер больше не помечает все приложения orphaned
- update: реально опубликованный порт фиксируется в БД (url/health)
- .env: значения с '#' кавычатся (dotenv-парсер обрезал их как inline-комментарий)
- порт 80, выбранный явно в мастере, публикуется как есть
- useCatalog: available_only включён в queryKey (коллизия кэша)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 12:52:14 +05:00
markmorado
f22b24d86a fix(appstore): touch file bind-mounts instead of mkdir, breaking .env mounts
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 56s
Deploy to Production / Trigger production update (push) Successful in 7s
docker compose up failed for apps that bind-mount a single file (e.g.
AnythingLLM's .env) because _prepare_bind_mounts blindly mkdir -p'd every
bind-mount source, turning the file's path into a directory. Docker then
refused to mount a file onto that directory ("not a directory: ... mount
a directory onto a file"). File-looking sources (by name) now get touch'd
instead, with self-healing rm -rf for directories left over from prior
failed installs.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-17 05:19:19 +05:00
markmorado
6a79dd2048 fix(appstore): DNS, per-server golden template, bind-mount perms, delete race
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 1m1s
Deploy to Production / Trigger production update (push) Successful in 8s
App Store install/update pipeline reliability fixes:
- explicit nameserver (APPSTORE_NAMESERVER) on App Store LXCs and the golden
  template build CT, instead of inheriting a node resolver that may be
  unreachable from the container network (e.g. Tailscale-only DNS)
- golden template storage/resolution moved from one global setting to
  per-server_id (migration 37: app_operations.server_id) — building it on one
  Proxmox server no longer overwrites the template configured for another
- golden template automated build fixed: vzdump export step no longer fails
  on "unable to parse directory volume name 'vztmpl/probe'"
- bind-mount host directories are pre-created with permissive ownership
  before `docker compose up`, fixing Permission denied for non-root
  containers (e.g. MariaDB) writing to Docker-auto-created root-owned dirs
- delete/teardown now waits for Proxmox to actually confirm the destroy task
  finished (was silently proceeding after a short timeout, letting a new
  install reuse the VMID while the node was still wiping the old disk)
- pct create retries a few times on transient "got lock" config errors
- failed pipeline steps now report the tail of command output (4000 chars)
  instead of the head, so the real error isn't buried under pull/progress noise
- install error box in the UI wraps long text and scrolls instead of
  stretching the dialog

Also carries forward in-progress script catalog / instances / RBAC changes
that were already uncommitted in the working tree before this session.

Bump to v1.9.1, update WIKI.md/docs/CHANGELOG accordingly.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-17 04:49:40 +05:00
markmorado
8ca33681ca feat(scripts): каталог bash-скриптов для нод и гостей (VM/LXC)
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 59s
Deploy to Production / Trigger production update (push) Successful in 8s
Добавлена библиотека скриптов по образцу VMmanager: свои скрипты (CRUD в UI)
и синхронизация из git-репозитория, параметризация через env-переменные
(без конкатенации в shell), выполнение на нодах Proxmox (SSH), в LXC (pct
exec) и в VM (QEMU guest agent, переиспользует существующий execute_script).
Журнал запусков встроен в общий список задач /api/all-tasks.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-11 04:35:43 +05:00
markmorado
db2f408d57 fix(core): пункты 5-10 аудита — UPID-синк, порт, бэкап all, IPAM, креды
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 56s
Deploy to Production / Trigger production update (push) Successful in 8s
5. UPID-синк: транзитная ошибка опроса статуса больше не помечает живую
   задачу failed — get_task_status(raise_on_error=True) + счётчик из 3
   последовательных неудач; в exit_status пишется реальная причина.
6. Дашборд и бэкапы для серверов на порту != 8006: metrics_broadcaster и
   backup_scheduler строят клиент через ProxmoxClient.from_server (учёт
   порта и token/password) вместо ручной сборки по ip_address.
7. Бэкап "все VM": create_backup при vmid in (None, 0, "all") передаёт
   vzdump параметр all=1 — раньше уходил vmid=0 и PVE отвергал задание.
8. IPAM: get_next_available_ip собирает занятые IP глобально (как проверяет
   allocate_ip), а не только по pool_id — авто-выдача больше не предлагает
   занятый адрес и не срывает создание инстанса.
9. IPAM: fallback в release_ip_by_vmid ищет только записи без
   proxmox_server_id — удаление VM не освобождает IP одноимённого vmid
   на другом сервере.
10. _get_proxmox_client: сервер без валидных кредов теперь возвращает
    понятную 503-ошибку вместо молчаливо пустых списков; перечисление
    серверов в метриках защищено per-server try/except.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 01:06:28 +05:00
markmorado
0efa5968ad fix(core): четыре критических бага — маршруты, очередь задач, логин, shutdown
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 56s
Deploy to Production / Trigger production update (push) Successful in 7s
1. vms.py: catch-all маршруты /vm/{vmid}/{action} и /container/{vmid}/{action}
   перенесены в конец файла — раньше они перехватывали POST /unlock,
   /cloud-init и /exec, и эти операции всегда падали с 400 "Invalid action".
2. task_queue_service: зависшие после рестарта running-задачи помечаются
   failed при старте — иначе очередь bulk-операций блокировалась навсегда.
3. api-client.ts: 401 от /api/auth/login больше не вызывает перезагрузку
   страницы — пользователь видит сообщение "неверный логин/пароль".
4. task_queue_service: bulk shutdown теперь вызывает shutdown_vm/
   shutdown_container (graceful) вместо жёсткого stop.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 00:58:40 +05:00
markmorado
e62626c79b fix(nodes): детерминированная сортировка списка серверов
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 1m0s
Deploy to Production / Trigger production update (push) Successful in 7s
Список серверов терял порядок при фоновом рефетче (каждые 30с),
т.к. запрос с WHERE id IN (...) не гарантирует стабильный порядок строк.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-10 00:40:17 +05:00
markmorado
3424823793 feat(cluster): режим eject "только панель" для нод, не входящих в кластер
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 56s
Deploy to Production / Trigger production update (push) Successful in 7s
При провале join нода остаётся стейл-записью с cluster_name в панели, хотя
в Proxmox она standalone. Обычный eject пытается pvecm delnode через SSH и
падает ("SSH connection failed"), т.к. нода не член кластера и/или узел
недоступен по SSH.

- eject-эндпоинт: новый параметр panel_only — отвязывает панельную запись
  (cluster_name=None) без обращения к Proxmox/SSH.
- UI: чекбокс "Только отвязать запись в панели" в диалоге исключения,
  блокирует поле пароля при выборе.
- Локали ru/en.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 03:36:55 +05:00
markmorado
c88cdb4e26 fix(cluster): join по приватному адресу corosync + подтверждение членства
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 58s
Deploy to Production / Trigger production update (push) Successful in 7s
Проблема: присоединяемая нода подключалась к кластерной по её адресу из
панели (часто публичный FQDN), из-за чего join падал с
"Can't connect to <fqdn>:8006 (Connection timed out)" при выделенной
corosync-сети. Плюс панель помечала ноду как присоединённую сразу после
старта задачи, показывая ложное членство даже при провале join.

- join: новый опциональный параметр cluster_host — приватный IP кластерной
  ноды в corosync-сети; используется как hostname для API-подключения и
  для выбора fingerprint из nodelist. Фолбэк — ip_address из панели.
- join: cluster_name пишется в БД только после подтверждения, что нода
  реально появилась среди нод кластерной ноды (опрос до ~40с, укладываемся
  в nginx proxy_read_timeout 60с). Иначе — confirmed=false и предупреждение.
- UI: поле "Адрес кластерной ноды для подключения" в шаге 3 мастера,
  обработка неподтверждённого join (toast.warning, окно не закрывается).
- Локали ru/en.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 03:28:39 +05:00
markmorado
3b10e903a4 fix(cluster): корректное извлечение fingerprint из nodelist при join
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 56s
Deploy to Production / Trigger production update (push) Successful in 8s
GET /cluster/config/join не возвращает fingerprint на верхнем уровне —
он лежит в nodelist[].pve_fp. Добавлен хелпер _extract_fingerprint,
который выбирает pve_fp целевой ноды (по IP), с фолбэком на
preferred_node и первую ноду. Устраняет ошибку 502
"Не удалось получить fingerprint кластера" при присоединении нод.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 03:07:53 +05:00