Commit Graph

22 Commits

Author SHA1 Message Date
abc819294c fix(images): unblock guest-exec in RHEL-family cloud images
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 1m45s
Deploy to Production / Trigger production update (push) Failing after 12s
SonarCloud / SonarCloud Analysis (push) Failing after 3m3s
The RHEL-family qemu-guest-agent package starts the agent with an allow-list
that omits guest-exec, guest-exec-status and guest-file-* (FILTER_RPC_ARGS in
/etc/sysconfig/qemu-ga; older packages spelled the same restriction as
BLOCK_RPCS or BLACKLIST_RPC). The agent answered "The command guest-exec has
been disabled for this instance", so running scripts in a VM, assigning extra
IP addresses and growing the filesystem after a disk resize did not work on
Rocky Linux, AlmaLinux, CentOS Stream and Fedora guests.

- patch-cloud-images.sh: clear all three variants while building the mirror
  (UNBLOCK_RPC=0 opts out) and add an ONLY filter to rebuild only part of the
  catalog. Three `[ ... ] && ...` statements that aborted the script under
  set -e are now plain ifs -- the one before mirrors.json generation killed
  every run that passed BASE_URL
- builtin.py: point the five RHEL-family amd64 entries at the rebuilt mirror
  release 2026-08-22 with their new checksums. Ubuntu and Debian images are
  unaffected and stay in the previous releases, so older panel versions keep
  working
- client.py: annotate the Proxmox error with what to clear inside the guest,
  the raw message names neither the file nor the variable
- WIKI, CHANGELOG: document the restriction and the unblock command
2026-08-22 14:03:49 +05:00
972d54d1f1 fix(ipam): keep extra addresses across a guest reboot
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 1m44s
Deploy to Production / Trigger production update (push) Failing after 11s
SonarCloud / SonarCloud Analysis (push) Failing after 3m0s
Testing on a live AlmaLinux container showed the alias disappearing after
a restart even though it was reported as applied: Proxmox regenerates the
guest network configuration on every start (pve-container rewrites
/etc/network/interfaces, ifcfg or the NM keyfile from netN), wiping any
address added to the native stack.

    before reboot: address1=10.10.10.103/24 address2=10.10.10.106/24
    after reboot:  addresses=10.10.10.103/24     (file recreated by PVE)

Install a oneshot systemd unit that re-adds the recorded addresses after
network-online.target — /etc/systemd is not touched by Proxmox. The native
branches stay, so persist is now reported as e.g. "nm+systemd"; the unit
and its helper are removed once the last address is released.

Also fixed while testing on the two test instances:
- the alias script is ASCII-only now: it travels through the Proxmox API
  and its marker ends up in guest system configs
- a composite persist marker no longer falls through the parser, which
  used to report a successful apply as runtime_only
- allocate_ip marks the first address of a guest as primary on its own,
  so allocations coming from the creation wizard are flagged correctly
2026-08-22 01:36:10 +05:00
6ac505e481 fix(proxmox): send guest-agent scripts as bytes
Proxmox base64-encodes the input-data of an agent exec in Perl, which dies
with "Wide character in subroutine entry" as soon as the payload contains
non-ASCII — a Cyrillic comment in a script is enough, so running any such
script on a VM failed with a 500.

Hand the same UTF-8 bytes over as latin-1: the JSON carries plain bytes,
Perl is happy and the guest still receives valid UTF-8.
2026-08-22 01:35:56 +05:00
6796903712 feat(ipam): several IP addresses per instance
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 1m46s
Deploy to Production / Trigger production update (push) Failing after 10s
SonarCloud / SonarCloud Analysis (push) Failing after 3m7s
A guest was implicitly limited to a single allocation: lookups were
"(server, vmid) -> one row", release_ip_by_vmid freed one address, and
the link tool skipped guests that already had one. An instance can now
hold a primary address plus any number of aliases, managed from the
Network tab of the instance and visible on the IPAM pages.

Schema and service:
- ipam_allocations gains is_primary, assignment_kind, target_interface
  and apply_* columns; migration 43 marks existing rows as primary
- find_allocations_by_resource/set_primary; release_ip_by_vmid now frees
  every address of a deleted guest instead of just the first one
- next-available search may scan the whole subnet when a network has no
  auto-assign pool, skipping node and cached guest addresses so it never
  hands out an address that is already live

Applying to the guest (services/guest_ip_service.py):
- reuses script_engine, so LXC goes through pct exec and VMs through the
  guest agent; the shell script synchronises the whole alias list of an
  interface, which makes add, remove and retry the same code path
- persists via ifupdown, netplan, NetworkManager or sysconfig and never
  restarts guest networking; an unknown stack degrades to runtime_only
  and an unreachable guest keeps the allocation as pending

API and UI:
- /ipam/api/guests/{server_id}/{vmid}/addresses: list, add, apply,
  primary, delete; allocations list accepts guest filters
- instance list exposes ips[] alongside the primary ip
- addresses card on the Network tab, "+N" badge in the IP column,
  primary badge in the allocations table

Fixes found along the way:
- link-allocations registers every address of a guest, carries the MAC
  over, refuses to take an address away from another live guest and no
  longer matches allocations by resource name (which leaked an address
  between same-named guests on different servers)
- IPAM history no longer fails to serialise datetime values, which used
  to abort re-linking with a 500
2026-08-21 23:48:20 +05:00
c9d8e8207f fix(proxmox): read LXC addresses from netN and live interfaces
get_container_interfaces() looked for ipconfigN keys, which exist only on
QEMU cloud-init guests. LXC keeps the address inside the netN string, so
the method never returned anything and containers showed no IP at all.

Parse netN with the existing net_parser, fall back to
/nodes/{node}/lxc/{vmid}/interfaces for DHCP guests (skipped when the
config already carries a static address or the container is stopped), and
validate addresses so loopback, link-local and the literal "dhcp" are
dropped. IPv6 from ip6/inet6 is reported as well.
2026-08-21 23:47:45 +05:00
bddbf15652 feat(rbac): split setting:view into panel settings and a permission-free profile
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 1m22s
Deploy to Production / Trigger production update (push) Failing after 11s
setting:view used to gate the whole /settings page, which mixed panel-wide
configuration with a user's own account. Revoking it to keep a standard user
out of the panel settings also took away their profile, password, 2FA and SSH
keys, so the permission could not be used for what it names.

The account half now lives at /profile: its endpoints are self-scoped and only
require a session, so the page carries no permission at all and is reached from
the user menu (and Ctrl+K) rather than a sidebar group. /settings keeps the
panel-wide tabs behind setting:view, with saving gated on setting:update,
security on setting:manage and the SMTP/Telegram channels on is_admin, matching
what each endpoint checks.

The shipped "user" role no longer gets setting:view. Migration 42 drops it from
roles holding it without setting:update/manage, since those grants only existed
to reach the profile page; it runs once (marker in panel_settings) so a later
deliberate grant is not reverted on the next start.
2026-08-11 20:07:34 +05:00
c4fda1b0bb feat(migrations): backfill the owner of instances created before the fix
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 1m20s
Deploy to Production / Trigger production update (push) Failing after 10s
Instances deployed while save_vm_instance() dropped the owner are still sitting
in the database with owner_id = NULL, so their users cannot see them. Recover
the creator from the two places it was recorded:

  1. deploy_tasks — user_id is a real foreign key stored when the deploy was
     queued. server_id is absent for LXC deploys, so a NULL there is treated as
     matching any server instead of discarding the row.
  2. audit_logs — vm_create / ct_create / lxc_create entries, which only carry a
     username and are matched by name.

Ownership decides who may see, stop and delete an instance, so a wrong guess is
worse than no owner at all: a row is updated only when every applicable piece of
evidence names the same existing user. VMIDs get reused, so conflicting evidence
is expected and those instances deliberately stay ownerless for an operator to
assign with assign_instances_owner.py. Existing owners, soft-deleted rows,
failed create events and logs from deleted users are all left alone, which also
makes the migration idempotent.

Note this recovers who created an instance, not who it was meant for: a VM an
admin created on behalf of a user is attributed to the admin, because the
intended owner was never persisted before the fix.

The attribution rule is a pure function (pick_backfill_owner) so it can be
tested directly. The migration's own statements are portable, so the tests run
it against SQLite with the two information_schema helpers stubbed out.
2026-08-11 17:28:59 +05:00
1152410860 fix(instances): assign the owner on create and inject the owner's SSH keys
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 2m30s
Deploy to Production / Trigger production update (push) Failing after 10s
Newly created VMs/LXCs ended up with owner_id = NULL, and a VM created by an
admin on behalf of a user received the admin's SSH key instead of the user's.

Ownership: save_vm_instance() only set owner_id when inserting a new row. The
row practically always existed by then — sync_vm_cache runs every 10s and
registers VMs it sees on Proxmox with owner_id = NULL while the clone is still
running, whereas the deploy saves the owner minutes later at 95% progress. So
every deploy landed in the update branch and silently dropped the owner. Set
owner_id there too, but only when it is passed explicitly: callers such as
save-before-delete do not know the owner and must not blank it out.

SSH keys: the profile key was taken from whoever pressed "Create" rather than
from the instance owner, and the LXC path injected no profile key at all. Add
build_deploy_ssh_keys(), which merges pasted keys, keys picked from the library
(same authorization rules as before) and the owner's profile key, deduplicated.
Both the template and LXC deploy endpoints now use it, so the key material is
resolved in one place instead of inside the deploy worker.

ISO VMs are left alone on purpose: they are blank VMs without cloud-init, so
there is nowhere to inject keys.

Covered by backend/tests/test_instance_ownership.py, including the sync race.
2026-08-11 17:19:19 +05:00
markmorado
761bd7bca8 fix(workspaces): grant server access via workspace membership alone
Non-privileged users previously needed a server to be both directly
assigned AND inside one of their workspaces (intersection) to see or use
it. Adding a user to a workspace should be sufficient on its own to let
them create instances on that workspace's servers, so the two access
paths are now unioned instead of intersected.
2026-07-23 21:59:09 +05:00
074983a1f6 fix(metrics): correct cpu/disk% chart scaling, evict stale collector state, lighten nic query, drop polling & dead RRD code
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 21:35:02 +05:00
bbec17abd1 feat(metrics): add collector + retention loops and wiring 2026-06-21 21:35:02 +05:00
6769e5b218 feat(metrics): add /metrics read endpoints for vm and container 2026-06-21 21:35:02 +05:00
e380ec1ab2 feat(metrics): add query_instance_metrics read aggregation 2026-06-21 21:35:02 +05:00
8f9f78ef91 feat(metrics): add blockstats + netstat client wrappers 2026-06-21 21:35:02 +05:00
3d55b1ab8d feat(metrics): add timeframe/bucket math and series aggregation 2026-06-21 21:35:02 +05:00
3a2c87bd0a feat(metrics): add rate + blockstats/netstat/fsinfo parsing helpers
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-21 21:35:02 +05:00
3b3841fc0e fix(metrics): portable BigInteger PKs + split test asserts 2026-06-21 21:35:02 +05:00
572bb44ffa feat(metrics): add instance_metric and instance_nic_metric models
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 21:35:02 +05:00
58e73d4af4 fix(tests): repair Proxmox client mock after from_server refactor
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 46s
Deploy to Production / Trigger production update (push) Successful in 8s
The mock_proxmox fixture patched ProxmoxClient with return_value= (constructor
style), but since commit d66928e clients are built via the
ProxmoxClient.from_server classmethod. from_server() therefore returned an
unconfigured child mock, so endpoints serialized a MagicMock and responded 500
— failing 5 tests in test_api_vms.py on every CI run (Backend Tests workflow).

Wire .from_server.return_value (and .return_value) on all patched ProxmoxClient
references to the configured instance so every construction path resolves to
the same mock. Full suite now green (60 passed).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 02:08:28 +05:00
d0badd0a2e refactor(rbac): drop runtime legacy-format resolution from engine
Roles are stored in the new resource:action format and any legacy keys are
converted once at startup by the RBAC v2 DB migration (migrate_rbac_v2), so
the permission engine no longer needs to resolve legacy codes on every check.

- get_user_permissions: store role permission codes as-is (no dual add).
- has_permission: match new-format codes only; drop resolve_permission and the
  dotted-code fallbacks.
- Remove now-unused resolve_permission import in middleware.
- LEGACY_PERMISSION_MAP / convert_* are retained as one-shot DB-migration
  utilities (upgrade path), not a runtime bridge.
- Tests: adapt fixtures to new-format role permissions; assert legacy keys are
  no longer honored at request time.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 02:08:19 +05:00
bfd92102cf Refactor backend: Split models and proxmox_client into modules, add alembic
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 2m27s
2026-06-13 03:53:27 +05:00
markmorado
89d70654fc fix(reinstall): use OSTemplate.vmid (not template_vmid) and convert memory bytes->MB
- AttributeError: OSTemplate has no 'template_vmid' field; the actual column is 'vmid'.
- Use OSTemplate.get_vmid_for_node() to support cross-node replicated templates,
  falling back to source_node/vmid otherwise.
- VMInstance.memory is stored in bytes, but Proxmox config expects MB.
2026-04-29 23:48:48 +02:00