Commit Graph

29 Commits

Author SHA1 Message Date
markmorado
075b8cdaaa feat(servers): distinguish rejected credentials from unreachable nodes
An offline server used to report a single "Failed to connect" state, but
"the node does not answer" and "the node answered 401" are fixed in
completely different ways — the first is power or network, the second is a
stale token, which is what happens when a node joins a cluster and its
user.cfg and priv/token.cfg are replaced by the cluster ones.

- add ProxmoxClient.check_connection() returning (ok, error_kind, error);
  token auth surfaces 401 as ResourceException rather than
  AuthenticationError, since proxmoxer validates the token on first use
- store the reason in proxmox_servers.last_error_kind (migration 40) and
  expose it through the server response
- surface an auth_error state in the UI with a hint explaining the usual
  cause

Storage listings now also report per-node availability: the cluster config
declares a storage for the whole cluster but says nothing about whether it
actually came up on a given node, so a missing volume group or pool stayed
invisible. Availability comes from a single /cluster/resources call.
2026-07-24 04:42:59 +05:00
markmorado
65c61e7886 feat(servers): auto-provision API tokens and add node metric graphs
Servers can now be connected with a login and password: the panel
authenticates once, creates its own API token on the node (privsep=0) and
uses it from then on. The token can be revoked from the Proxmox UI, cannot
be used for SSH or web login, and appears in the PVE audit log under its
own name. The password stays stored (encrypted) because ProxmoxClient needs
it for SSH fallbacks such as pvecm delnode.

- add proxmox/token_provisioning.py: token creation with free-name
  selection and revoke-on-failure so a failed registration does not leave
  an orphaned token whose secret was never persisted
- POST /api/servers accepts auto_create_token
- add POST /api/servers/{id}/provision-token to migrate servers that are
  still on password auth
- store api_token_value as EncryptedString; it was kept in plaintext while
  the password next to it was already encrypted
- drop the unused ProxmoxServerBase schema, a stale copy of the create
  schema whose validator rejected the new auth mode

Node detail pages gain a Graphs tab (CPU, IO delay, memory, load average,
network, swap, root filesystem) built on the existing node rrddata
endpoint. The chart card is extracted from the instance graphs tab so both
views share it, and X axis ticks now switch to dates on multi-day ranges.
2026-07-24 04:03:30 +05:00
markmorado
4621ffe88a fix(appstore): data files, надёжность установки/удаления и сворачиваемые диалоги
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 57s
Deploy to Production / Trigger production update (push) Successful in 8s
Доставка файлов приложения из каталога (data_files) до compose up,
APP_PASSWORD/default_credentials для Umbrel, сворачивание диалогов
долгих операций (MinimizableDialog), фильтрация pull-шума в ошибках.

Исправления багов:
- teardown: отсутствующий в Proxmox CT считается удалённым — orphaned-приложение
  теперь можно удалить из БД
- install: сдвиг VMID мимо занятых записей (nextid не резервирует номер,
  retry мог подхватить чужой контейнер)
- retry: параметры установки сохраняются в installed_apps.install_params
  (cores/memory/disk/storage/bridge/ip_config/port_explicit, миграция JSONB)
  вместо жёстких дефолтов
- reconcile: недоступный сервер больше не помечает все приложения orphaned
- update: реально опубликованный порт фиксируется в БД (url/health)
- .env: значения с '#' кавычатся (dotenv-парсер обрезал их как inline-комментарий)
- порт 80, выбранный явно в мастере, публикуется как есть
- useCatalog: available_only включён в queryKey (коллизия кэша)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 12:52:14 +05:00
markmorado
6a79dd2048 fix(appstore): DNS, per-server golden template, bind-mount perms, delete race
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 1m1s
Deploy to Production / Trigger production update (push) Successful in 8s
App Store install/update pipeline reliability fixes:
- explicit nameserver (APPSTORE_NAMESERVER) on App Store LXCs and the golden
  template build CT, instead of inheriting a node resolver that may be
  unreachable from the container network (e.g. Tailscale-only DNS)
- golden template storage/resolution moved from one global setting to
  per-server_id (migration 37: app_operations.server_id) — building it on one
  Proxmox server no longer overwrites the template configured for another
- golden template automated build fixed: vzdump export step no longer fails
  on "unable to parse directory volume name 'vztmpl/probe'"
- bind-mount host directories are pre-created with permissive ownership
  before `docker compose up`, fixing Permission denied for non-root
  containers (e.g. MariaDB) writing to Docker-auto-created root-owned dirs
- delete/teardown now waits for Proxmox to actually confirm the destroy task
  finished (was silently proceeding after a short timeout, letting a new
  install reuse the VMID while the node was still wiping the old disk)
- pct create retries a few times on transient "got lock" config errors
- failed pipeline steps now report the tail of command output (4000 chars)
  instead of the head, so the real error isn't buried under pull/progress noise
- install error box in the UI wraps long text and scrolls instead of
  stretching the dialog

Also carries forward in-progress script catalog / instances / RBAC changes
that were already uncommitted in the working tree before this session.

Bump to v1.9.1, update WIKI.md/docs/CHANGELOG accordingly.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-17 04:49:40 +05:00
markmorado
db2f408d57 fix(core): пункты 5-10 аудита — UPID-синк, порт, бэкап all, IPAM, креды
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 56s
Deploy to Production / Trigger production update (push) Successful in 8s
5. UPID-синк: транзитная ошибка опроса статуса больше не помечает живую
   задачу failed — get_task_status(raise_on_error=True) + счётчик из 3
   последовательных неудач; в exit_status пишется реальная причина.
6. Дашборд и бэкапы для серверов на порту != 8006: metrics_broadcaster и
   backup_scheduler строят клиент через ProxmoxClient.from_server (учёт
   порта и token/password) вместо ручной сборки по ip_address.
7. Бэкап "все VM": create_backup при vmid in (None, 0, "all") передаёт
   vzdump параметр all=1 — раньше уходил vmid=0 и PVE отвергал задание.
8. IPAM: get_next_available_ip собирает занятые IP глобально (как проверяет
   allocate_ip), а не только по pool_id — авто-выдача больше не предлагает
   занятый адрес и не срывает создание инстанса.
9. IPAM: fallback в release_ip_by_vmid ищет только записи без
   proxmox_server_id — удаление VM не освобождает IP одноимённого vmid
   на другом сервере.
10. _get_proxmox_client: сервер без валидных кредов теперь возвращает
    понятную 503-ошибку вместо молчаливо пустых списков; перечисление
    серверов в метриках защищено per-server try/except.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 01:06:28 +05:00
markmorado
0d056a89a5 feat(nodes): TLS-сертификаты ноды и массовая миграция инстансов
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 57s
Deploy to Production / Trigger production update (push) Successful in 8s
- Сертификаты: методы клиента + endpoints GET/POST/DELETE
  /nodes/{node}/certificates, хуки и вкладка «Сертификаты» в
  NodeAdminManager (просмотр, загрузка PEM cert+key, удаление кастомного,
  бейдж срока действия)
- Bulk migrate: тип bulk_migrate в очереди задач с WS-прогрессом,
  target_node/online в BulkOperationItem, диалог выбора целевой ноды и
  кнопка в тулбаре инстансов (в пределах одного сервера/кластера)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 02:48:42 +05:00
markmorado
afde2bf569 feat(nodes): управление нодой как в Proxmox — питание, консоль, журнал/задачи/система
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 58s
Deploy to Production / Trigger production update (push) Successful in 7s
Ядро:
- Reboot/Shutdown ноды (POST /nodes/{node}/status) под новым правом node:power
  с подтверждением в UI; меню действий на карточке ноды + пункт «Консоль (Shell)».

Доп. вкладки в «Администрировании ноды»:
- Журнал (syslog ноды), История задач (tasks + лог задачи),
  Система (DNS / /etc/hosts / часовой пояс).

Бэкенд: методы node_power_action/get_node_syslog/get_node_tasks/get_node_task_log/
get_node_task_status/DNS/hosts/time в NodeAdminMixin + эндпоинты в node_admin.py.
RBAC: право node:power (admin + backfill server:manage).

Проверено: py_compile, tsc --noEmit, vite build; live — маршруты в openapi,
403 без токена, node:power засеян роли admin.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 02:19:07 +05:00
markmorado
782524fff6 feat(hardware): проброс физических дисков ноды в VM (raw passthrough)
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 56s
Deploy to Production / Trigger production update (push) Successful in 7s
Добавлена возможность пробросить физический диск ноды в VM по стабильному
пути /dev/disk/by-id/ (scsiN/virtioN/sataN) в карточке «Проброс устройств».

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 23:37:13 +05:00
markmorado
00e36e1cc6 feat: Proxmox management suite — hardware, firewall, pools, disks, storage, access, node admin
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 54s
Deploy to Production / Trigger production update (push) Successful in 7s
- VM/LXC hardware editor with PCI/USB passthrough (BIOS/OVMF + efidisk,
  machine type, VGA, SCSI controller, CPU sockets); passthrough locked while running
- Firewall at datacenter/node/VM levels: rules, security groups, IP sets,
  aliases, options, log — with rule dialog referencing security groups
- Resource pools (CRUD + membership) and per-instance tags editor
- Node disks: physical disk list, SMART viewer, ZFS pool create/destroy, wipe
- Storage: add LVM/LVM-thin/ZFS/RBD/CephFS types + enable/disable toggle
- PVE Access: API tokens (create with one-time secret, delete) and realms (LDAP/AD/OpenID)
- Node admin: systemd services (start/stop/restart), APT updates & repositories
- Shared: SelectTrigger defaults to full width (fixes narrow dropdowns)

Also bundles related in-progress work: auth screen refresh, node shell console
page, server-updates badge, and node:*/access:* RBAC permissions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 05:40:52 +05:00
markmorado
5c8baefb3f feat: add firewall, pools, access, disks, node admin and VM hardware management
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 1m0s
Deploy to Production / Trigger production update (push) Successful in 7s
Backend: new proxmox mixins and API routes for firewall, pools, access
control, node admin and disks; extend storage and VM endpoints.

Frontend: node managers (firewall, pool, access, disk, node admin), VM
firewall and hardware tabs, PCI/USB device dialogs, related hooks and
locale strings.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 00:02:15 +05:00
markmorado
c12fbf13e5 feat(instances): add VM/LXC migration, autostart/boot order, protection
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 52s
Deploy to Production / Trigger production update (push) Successful in 7s
- Migrate VMs/LXCs between cluster nodes with live progress tracking
- Configure autostart, boot order, startup/shutdown delays, and delete protection from the instance settings tab
- Group OS templates by detected OS (Ubuntu, AlmaLinux, etc.) instead of a single "Cloud Images" bucket
- Expand native Proxmox backup job management (schedule, compression, retention, guest selection, notifications)
- Add ru/en translations for the above
2026-07-03 02:40:58 +05:00
91080d8574 feat(api): expose SDN DNS server entries
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 03:05:06 +05:00
161182b392 feat(api): support netN add/remove in VM and LXC config updates
- update_vm_config: whitelist netN/scsiN/virtioN/ideN/sataN/ipconfigN by
  regex instead of hardcoded 0-3; accept validated delete='netN' key;
  raise with real Proxmox error text instead of returning False
- update_container_config: refactor to dict signature with whitelist +
  netN + delete (fixes LXC config PUT endpoint that passed the whole
  dict as hostname)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 02:32:21 +05:00
7d52297f76 fix(deploy): apply VM config reliably after clone
- wait for clone lock release before config.put (cross-node clones kept
  lock past wait_for_task, dropping ALL requested settings silently)
- configure_vm applies params in independent groups (compute/network/
  cloudinit/disk) with per-group errors and lock retry
- set sockets=1 with cores so total vCPU matches request
- keep template net0 model/MAC/options, replace only bridge
- disk resize: pick boot disk via regex over all buses, report failures
- surface config errors as deploy warnings in UI toast instead of
  silent success

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 02:31:21 +05:00
f880c9e177 fix(images): idempotent download (skip if file exists) + real failure reason
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 52s
Deploy to Production / Trigger production update (push) Successful in 8s
Retrying a template build failed at download because the qcow2 from a prior
attempt already existed and download-url refuses to overwrite. Now skip the
download when the volume already exists (volume_exists), and surface the PVE
task-log tail on download failure instead of a generic message.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 00:38:50 +05:00
c7edf72d19 feat(images): Phase 2 — auto-convert qcow2 to VM template
- client.create_vm_with_import: API-only disk import via scsi0 import-from (PVE 8.2+)
- _do_image_template_sync worker: download(import) -> create+import -> cloud-init
  -> convert_to_template -> register OSTemplate (Cloud Images group)
- images download endpoint: to_template branch + cloud-init/disk_storage params
- clear fallback error when storage lacks import support (file kept)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 23:11:20 +05:00
7aed5dd97e feat(images): backend catalog + mirror model + download API/worker
Phase 1 of VM image mirror feature:
- StorageMixin.download_url (PVE download-url/aria2) + get_download_target_storages
- builtin curated cloud-image catalog (Ubuntu/Debian/Rocky/Alma, amd64+arm64)
- ImageMirror model for admin custom mirrors
- /proxmox/api/images: catalog, lxc-templates proxy, storages, download
- _do_image_download_sync worker with log-tailing progress (DeployTask)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 22:51:21 +05:00
6e67ad7e77 fix(disk-resize): keep growpart script ASCII-only
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Successful in 51s
Deploy to Production / Trigger production update (push) Has been cancelled
Proxmox's guest-agent exec (PVE/QemuServer/Agent.pm) dies with
"Wide character in subroutine entry" when input-data contains non-ASCII
bytes. The growpart script had Cyrillic comments, so the whole auto-grow
call returned 500 and the FS was never resized. Replace those comments
with ASCII so the script reaches the guest.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 03:09:43 +05:00
f67ee24155 Merge branch 'feature/metrics-history'
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 50s
Deploy to Production / Trigger production update (push) Successful in 7s
2026-06-22 02:47:45 +05:00
38d485be4b fix(disk-resize): honest task status, auto-grow FS, CRLF-safe guest exec
Backend fixes for VM/LXC disk resize:
- resize_vm_disk/resize_container_disk now honor wait_for_task and raise
  the real Proxmox exitstatus (e.g. "disk 'scsi0' does not exist") instead
  of always returning success.
- add grow_vm_filesystem: after a VM resize, rescan the disk (retry loop,
  needed for SATA/SCSI), growpart the partition and resize the FS
  (ext*/xfs/btrfs) via the guest agent; surface its output to the API.
- execute_script normalizes CRLF/CR to LF so pasted scripts and growpart
  don't break on trailing \r ("partition-number must be a number").

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 02:43:03 +05:00
8f9f78ef91 feat(metrics): add blockstats + netstat client wrappers 2026-06-21 21:35:02 +05:00
7d003f7fa1 fix(snapshots): wait for Proxmox task to report real outcome
create_vm_snapshot/create_container_snapshot returned success as soon as
the POST returned a UPID, so a snapshot that failed asynchronously (e.g.
storage without snapshot support) still reported "created". Now wait on
the task and surface the actual exitstatus.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 23:35:26 +05:00
a507e3c4bc feat(metrics): real-time VM disk fill, IOPS and network I/O
Add live per-disk usage, disk read/write and network in/out rates to the
instance Overview tab, the instances list and the dashboard top-loaded list.

Backend:
- get_vm_fsinfo() pulls per-filesystem usage via QEMU guest agent
- metrics_broadcaster computes I/O rates (bytes/sec) from cumulative counters
- instances-list loop now refreshes each running VM via status/current
  (1s-granular) instead of cluster/resources (10s pvestatd), with bounded
  concurrency, and paces both realtime loops to a 1s cadence

Frontend:
- OverviewTab: per-disk cards, Network I/O and I/O-rate cards via WS
- InstancesPage: Net and Disk I/O columns; Disk% shown for LXC only
- DashboardPage: top-loaded list overlaid with realtime metrics

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 08:26:29 +05:00
d66928ee78 refactor(proxmox): unify ProxmoxClient construction via from_server
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 48s
Deploy to Production / Trigger production update (push) Successful in 8s
Collapse four duplicated host+auth client-building blocks into a single
ProxmoxClient.from_server() classmethod (the canonical source of truth):

- _helpers._get_proxmox_client now delegates to from_server (107 callers
  untouched)
- ipam._build_proxmox_client removed; its one call site uses from_server
- task_queue_service keeps its per-server cache pool, builds via from_server
- monitoring_worker keeps its ValueError-on-missing-auth contract, builds
  via from_server

Behaviour preserved (token-first, then password; non-default port appended).
Verified: containers rebuild healthy, smoke test of both auth paths passes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-19 22:30:57 +05:00
508be2423a refactor(proxmox): extract _wait_exec_status helper from execute_command
All checks were successful
Deploy to Production / Trigger production update (push) Successful in 8s
Move the guest-agent exec-status polling loop into a shared _wait_exec_status
method reused by execute_command and execute_script. No behavior change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 00:14:35 +05:00
681aa2f303 feat(backups): restore target node/storage selection + task progress tracking
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 1m6s
Deploy to Production / Trigger production update (push) Successful in 8s
Add target node and storage overrides to single and bulk restore flows, and
track Proxmox task progress (parse_task_progress in the monitoring worker,
new ProxmoxTask progress column + migration, enriched task API responses).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 21:26:01 +05:00
2e8558fa51 fix(backend): restore operations — add force param, fix QEMU restore API
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 47s
- restore_vm/restore_lxc: add force=True param (needed for in-place overwrite)
- QEMU restore: remove `restore: 1` from params (breaks Proxmox API — restore
  is triggered by archive presence alone, unlike LXC where restore: 1 required).
  This fixes both single and bulk QEMU restores.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-13 16:32:23 +05:00
50f2ee1966 fix(security): authenticate VNC/terminal WebSockets and harden SSH/config
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 46s
Critical security and robustness fixes from a code audit:

- WS terminal/VNC now require a JWT (?token=), validate the session,
  vms.console permission and VM ownership before bridging to Proxmox
  (previously the LXC terminal gave unauthenticated root shell access).
  Frontend ConsolePage passes the token in both WS URLs.
- Fix SSH fallback for `pvecm delnode`: SSHClient now supports password
  auth (it previously crashed with TypeError); ProxmoxClient stores the
  password for SSH fallbacks and the host port suffix is stripped.
- Add production startup warnings for default ADMIN_PASSWORD and wildcard
  CORS_ORIGINS.
- dashboard: use tz-aware utcnow() instead of naive datetime.utcnow().
- crypto: narrow decrypt_value except to InvalidToken and log failures.
- ssh_client: guard the shared connection cache with a lock; make
  unknown-host-key rejection opt-in via SSH_REJECT_UNKNOWN_HOSTS.
- logging: send loguru console output to stderr instead of stdout.
- Sync version fallback to 1.5.1; remove one-off refactor scripts.
- Add CODE_REVIEW.md audit report.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-13 06:31:26 +05:00
bfd92102cf Refactor backend: Split models and proxmox_client into modules, add alembic
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 2m27s
2026-06-13 03:53:27 +05:00