Commit Graph

27 Commits

Author SHA1 Message Date
6796903712 feat(ipam): several IP addresses per instance
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 1m46s
Deploy to Production / Trigger production update (push) Failing after 10s
SonarCloud / SonarCloud Analysis (push) Failing after 3m7s
A guest was implicitly limited to a single allocation: lookups were
"(server, vmid) -> one row", release_ip_by_vmid freed one address, and
the link tool skipped guests that already had one. An instance can now
hold a primary address plus any number of aliases, managed from the
Network tab of the instance and visible on the IPAM pages.

Schema and service:
- ipam_allocations gains is_primary, assignment_kind, target_interface
  and apply_* columns; migration 43 marks existing rows as primary
- find_allocations_by_resource/set_primary; release_ip_by_vmid now frees
  every address of a deleted guest instead of just the first one
- next-available search may scan the whole subnet when a network has no
  auto-assign pool, skipping node and cached guest addresses so it never
  hands out an address that is already live

Applying to the guest (services/guest_ip_service.py):
- reuses script_engine, so LXC goes through pct exec and VMs through the
  guest agent; the shell script synchronises the whole alias list of an
  interface, which makes add, remove and retry the same code path
- persists via ifupdown, netplan, NetworkManager or sysconfig and never
  restarts guest networking; an unknown stack degrades to runtime_only
  and an unreachable guest keeps the allocation as pending

API and UI:
- /ipam/api/guests/{server_id}/{vmid}/addresses: list, add, apply,
  primary, delete; allocations list accepts guest filters
- instance list exposes ips[] alongside the primary ip
- addresses card on the Network tab, "+N" badge in the IP column,
  primary badge in the allocations table

Fixes found along the way:
- link-allocations registers every address of a guest, carries the MAC
  over, refuses to take an address away from another live guest and no
  longer matches allocations by resource name (which leaked an address
  between same-named guests on different servers)
- IPAM history no longer fails to serialise datetime values, which used
  to abort re-linking with a 500
2026-08-21 23:48:20 +05:00
bddbf15652 feat(rbac): split setting:view into panel settings and a permission-free profile
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 1m22s
Deploy to Production / Trigger production update (push) Failing after 11s
setting:view used to gate the whole /settings page, which mixed panel-wide
configuration with a user's own account. Revoking it to keep a standard user
out of the panel settings also took away their profile, password, 2FA and SSH
keys, so the permission could not be used for what it names.

The account half now lives at /profile: its endpoints are self-scoped and only
require a session, so the page carries no permission at all and is reached from
the user menu (and Ctrl+K) rather than a sidebar group. /settings keeps the
panel-wide tabs behind setting:view, with saving gated on setting:update,
security on setting:manage and the SMTP/Telegram channels on is_admin, matching
what each endpoint checks.

The shipped "user" role no longer gets setting:view. Migration 42 drops it from
roles holding it without setting:update/manage, since those grants only existed
to reach the profile page; it runs once (marker in panel_settings) so a later
deliberate grant is not reverted on the next start.
2026-08-11 20:07:34 +05:00
c4fda1b0bb feat(migrations): backfill the owner of instances created before the fix
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Failing after 1m20s
Deploy to Production / Trigger production update (push) Failing after 10s
Instances deployed while save_vm_instance() dropped the owner are still sitting
in the database with owner_id = NULL, so their users cannot see them. Recover
the creator from the two places it was recorded:

  1. deploy_tasks — user_id is a real foreign key stored when the deploy was
     queued. server_id is absent for LXC deploys, so a NULL there is treated as
     matching any server instead of discarding the row.
  2. audit_logs — vm_create / ct_create / lxc_create entries, which only carry a
     username and are matched by name.

Ownership decides who may see, stop and delete an instance, so a wrong guess is
worse than no owner at all: a row is updated only when every applicable piece of
evidence names the same existing user. VMIDs get reused, so conflicting evidence
is expected and those instances deliberately stay ownerless for an operator to
assign with assign_instances_owner.py. Existing owners, soft-deleted rows,
failed create events and logs from deleted users are all left alone, which also
makes the migration idempotent.

Note this recovers who created an instance, not who it was meant for: a VM an
admin created on behalf of a user is attributed to the admin, because the
intended owner was never persisted before the fix.

The attribution rule is a pure function (pick_backfill_owner) so it can be
tested directly. The migration's own statements are portable, so the tests run
it against SQLite with the two information_schema helpers stubbed out.
2026-08-11 17:28:59 +05:00
markmorado
075b8cdaaa feat(servers): distinguish rejected credentials from unreachable nodes
An offline server used to report a single "Failed to connect" state, but
"the node does not answer" and "the node answered 401" are fixed in
completely different ways — the first is power or network, the second is a
stale token, which is what happens when a node joins a cluster and its
user.cfg and priv/token.cfg are replaced by the cluster ones.

- add ProxmoxClient.check_connection() returning (ok, error_kind, error);
  token auth surfaces 401 as ResourceException rather than
  AuthenticationError, since proxmoxer validates the token on first use
- store the reason in proxmox_servers.last_error_kind (migration 40) and
  expose it through the server response
- surface an auth_error state in the UI with a hint explaining the usual
  cause

Storage listings now also report per-node availability: the cluster config
declares a storage for the whole cluster but says nothing about whether it
actually came up on a given node, so a missing volume group or pool stayed
invisible. Availability comes from a single /cluster/resources call.
2026-07-24 04:42:59 +05:00
markmorado
4621ffe88a fix(appstore): data files, надёжность установки/удаления и сворачиваемые диалоги
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 57s
Deploy to Production / Trigger production update (push) Successful in 8s
Доставка файлов приложения из каталога (data_files) до compose up,
APP_PASSWORD/default_credentials для Umbrel, сворачивание диалогов
долгих операций (MinimizableDialog), фильтрация pull-шума в ошибках.

Исправления багов:
- teardown: отсутствующий в Proxmox CT считается удалённым — orphaned-приложение
  теперь можно удалить из БД
- install: сдвиг VMID мимо занятых записей (nextid не резервирует номер,
  retry мог подхватить чужой контейнер)
- retry: параметры установки сохраняются в installed_apps.install_params
  (cores/memory/disk/storage/bridge/ip_config/port_explicit, миграция JSONB)
  вместо жёстких дефолтов
- reconcile: недоступный сервер больше не помечает все приложения orphaned
- update: реально опубликованный порт фиксируется в БД (url/health)
- .env: значения с '#' кавычатся (dotenv-парсер обрезал их как inline-комментарий)
- порт 80, выбранный явно в мастере, публикуется как есть
- useCatalog: available_only включён в queryKey (коллизия кэша)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 12:52:14 +05:00
markmorado
6a79dd2048 fix(appstore): DNS, per-server golden template, bind-mount perms, delete race
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 1m1s
Deploy to Production / Trigger production update (push) Successful in 8s
App Store install/update pipeline reliability fixes:
- explicit nameserver (APPSTORE_NAMESERVER) on App Store LXCs and the golden
  template build CT, instead of inheriting a node resolver that may be
  unreachable from the container network (e.g. Tailscale-only DNS)
- golden template storage/resolution moved from one global setting to
  per-server_id (migration 37: app_operations.server_id) — building it on one
  Proxmox server no longer overwrites the template configured for another
- golden template automated build fixed: vzdump export step no longer fails
  on "unable to parse directory volume name 'vztmpl/probe'"
- bind-mount host directories are pre-created with permissive ownership
  before `docker compose up`, fixing Permission denied for non-root
  containers (e.g. MariaDB) writing to Docker-auto-created root-owned dirs
- delete/teardown now waits for Proxmox to actually confirm the destroy task
  finished (was silently proceeding after a short timeout, letting a new
  install reuse the VMID while the node was still wiping the old disk)
- pct create retries a few times on transient "got lock" config errors
- failed pipeline steps now report the tail of command output (4000 chars)
  instead of the head, so the real error isn't buried under pull/progress noise
- install error box in the UI wraps long text and scrolls instead of
  stretching the dialog

Also carries forward in-progress script catalog / instances / RBAC changes
that were already uncommitted in the working tree before this session.

Bump to v1.9.1, update WIKI.md/docs/CHANGELOG accordingly.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-17 04:49:40 +05:00
markmorado
633f1a424c feat(appstore): кастомный порт при установке + мгновенная чистка инстанса
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 57s
Deploy to Production / Trigger production update (push) Successful in 7s
App Store:
- поле кастомного порта в мастере установки (переопределяет порт каталога)
- валидация диапазона порта 1..65535 (Pydantic Field + EngineError)
- APP_PORT в окружении синхронизируется с выбранным портом
- при удалении приложения инстанс сразу помечается deleted в кэше
  vm_instances (soft_delete_vm_instance), не дожидаясь фонового воркера

Также включены правки IPAM (привязка сетей к рабочей области, UI
WorkspaceSelect) и синхронизации каталога App Store.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-09 01:20:28 +05:00
markmorado
0606e8ac77 feat(appstore): модуль «App Store» — каталог self-hosted приложений (1 app = 1 LXC + Docker Compose)
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 1m4s
Deploy to Production / Trigger production update (push) Successful in 12s
Полный цикл M0–M5 по ТЗ:
- Catalog Service: импорт runtipi/runtipi-appstore → catalog_apps, идемпотентный
  синк, кэш логотипов, авто раз в 24ч (APScheduler), флаг update_available.
- App Engine: установка (клон золотого шаблона → pct push compose/.env →
  docker compose up → health-check) с журналом шагов, шифрованием env (Fernet),
  генерацией random-секретов и прогрессом по WebSocket.
- Lifecycle: start/stop/restart/logs/delete + реконсиляция состояния (orphaned).
- Update/Rollback через снапшоты Proxmox (pre-update, хранение последнего 1).
- UI: экраны App Store (сетка/поиск/категории), страница приложения, мастер
  установки (динамическая form_fields + advanced + прогресс + креденшелы),
  My Apps; i18n ru/en; атрибуция Runtipi.
- RBAC app:view/install/manage; только unprivileged LXC; pct exec/push без
  shell-инъекций (base64 + экранирование).
- Модели catalog_apps/installed_apps/app_operations (миграции 31–32).
- Docs: golden-template, appstore-poc/catalog/engine.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 23:05:20 +05:00
markmorado
bec7b20e31 feat(auth): двухфакторная аутентификация (TOTP) для входа в панель
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 58s
Deploy to Production / Trigger production update (push) Successful in 8s
Бэкенд:
- two_factor_service: генерация секрета, otpauth-URI, QR как inline-SVG,
  одноразовые backup-коды (хранятся как SHA-256)
- эндпоинты /api/auth/2fa/{status,setup,enable,disable}; гейт 2FA в login
  (неверный код -> 400, чтобы SPA не сбрасывала форму); two_factor_enabled в /me
- админский сброс: POST /api/users/{id}/disable-2fa
- модель/схемы/init.sql + идемпотентная миграция №30 (two_factor_backup_codes)
- зависимости: pyotp, qrcode

Фронтенд:
- двухшаговый вход с полем кода (TOTP/backup)
- карточка 2FA в профиле: QR, проверка, backup-коды, отключение
- бейдж/сброс 2FA в списке пользователей; локали ru/en

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 21:08:38 +05:00
markmorado
00e36e1cc6 feat: Proxmox management suite — hardware, firewall, pools, disks, storage, access, node admin
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 54s
Deploy to Production / Trigger production update (push) Successful in 7s
- VM/LXC hardware editor with PCI/USB passthrough (BIOS/OVMF + efidisk,
  machine type, VGA, SCSI controller, CPU sockets); passthrough locked while running
- Firewall at datacenter/node/VM levels: rules, security groups, IP sets,
  aliases, options, log — with rule dialog referencing security groups
- Resource pools (CRUD + membership) and per-instance tags editor
- Node disks: physical disk list, SMART viewer, ZFS pool create/destroy, wipe
- Storage: add LVM/LVM-thin/ZFS/RBD/CephFS types + enable/disable toggle
- PVE Access: API tokens (create with one-time secret, delete) and realms (LDAP/AD/OpenID)
- Node admin: systemd services (start/stop/restart), APT updates & repositories
- Shared: SelectTrigger defaults to full width (fixes narrow dropdowns)

Also bundles related in-progress work: auth screen refresh, node shell console
page, server-updates badge, and node:*/access:* RBAC permissions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 05:40:52 +05:00
35dcbfd18f feat(quotas): per-user resource limits for VM/LXC deployment
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 54s
Deploy to Production / Trigger production update (push) Successful in 8s
Add per-user quotas (max instances, vCPU, RAM, disk) enforced at deploy
time and surfaced to users self-service.

Backend:
- UserQuota model (one-to-one, NULL = unlimited) + migration #28
- quota_service: usage from vm_instances cache, check_quota raises 429
- enforce in VM (templates.py) and LXC (lxc.py) deploy paths
- RBAC quota:view / quota:manage, granted to admin
- admin API GET/PUT /admin/api/users/{id}/quota
- self-service GET /settings/api/quota
- tighten VM list isolation via can_view_all_instances

Frontend:
- admin quota dialog in UsersPage with usage bars
- quota card on Settings page (own limits)
- remaining-quota hint in CreateInstanceWizard config step
- useUserQuota/useSetUserQuota, useMyQuota hooks; RU/EN locales

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 18:15:02 +05:00
8b93161434 feat(auth): self-service password reset (forgot/reset endpoints)
Add an email-based "forgot password" flow:
- PasswordResetToken model (stores only a SHA-256 hash, single-use, TTL)
  and migration 27 for the password_reset_tokens table.
- POST /api/auth/forgot-password — generic response (no account
  enumeration), 60s throttle, emails a one-time reset link via the
  existing EmailChannel as a background task.
- GET /api/auth/reset-password/validate — token validity check for the UI.
- POST /api/auth/reset-password — validates the token, enforces the
  password policy, sets the new password, clears lock state, invalidates
  outstanding tokens and terminates all sessions.
- Configurable link lifetime via PASSWORD_RESET_TOKEN_EXPIRE_MINUTES (30m).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 03:24:19 +05:00
9ced08b88c feat(settings): per-user language preference and panel improvements
Store UI language per user (users.language column, migration 26).
Language selector moves from Panel tab to Profile tab — each account
picks its own language, saved to DB and applied via i18n.changeLanguage()
on save so the interface switches immediately.

Panel tab: panel_name read-only (falls back to env PANEL_NAME),
log_retention_days field exposed in UI, language selector removed.
Panel API accepts panel_name in PUT so it can be overridden in DB
without touching the env var.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-20 08:47:18 +05:00
681aa2f303 feat(backups): restore target node/storage selection + task progress tracking
All checks were successful
Backend Tests / pytest (Python 3.12) (push) Successful in 1m6s
Deploy to Production / Trigger production update (push) Successful in 8s
Add target node and storage overrides to single and bulk restore flows, and
track Proxmox task progress (parse_task_progress in the monitoring worker,
new ProxmoxTask progress column + migration, enriched task API responses).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 21:26:01 +05:00
markmorado
323d1e57fa feat: SSH keys management, backups fixes, instances/nodes/templates improvements, dev mode support
Some checks failed
Backend Tests / pytest (Python 3.12) (push) Has been cancelled
2026-05-07 00:30:21 +02:00
markmorado
79d6864d43 feat(vm-ops): make reinstall/clone/change-password async via DeployTask
Backend:
- Add DeployTask.kind column ('deploy'|'reinstall'|'clone'|'change_password')
  + Migration 23: ALTER TABLE deploy_tasks ADD kind, drop NOT NULL on template_id
- Update DeployTask.to_dict() with kind-aware Russian descriptions
- New backend/app/api/proxmox/async_ops.py with three workers:
    _do_reinstall_sync, _do_clone_sync, _do_change_password_sync
  All run on shared _deploy_executor thread pool from templates.py and
  report progress through _update_deploy_task (DB + WebSocket broadcast)
- Rewrite five vms.py endpoints to return 202 + DeployTaskStartResponse:
    POST /vm/{vmid}/clone, /container/{vmid}/clone,
    POST /vm/{vmid}/reinstall,
    POST /vm/{vmid}/change-password, /container/{vmid}/change-password

Frontend:
- DeployTask: add 'kind' and 'server_id' fields
- useCloneInstance/useReinstallInstance/useChangePassword now return
  AsyncTaskResponse {task_id, status, name}; remove invalidateQueries
- InstanceActionDialogs: each onSuccess pushes task to deploy-tasks-store
  with appropriate kind, shows 'launched in background' toast
- InstancesPage:
  * ghostRows only for kind in {deploy, clone}, deduped by (server_id, vmid)
  * reinstall / change_password tasks shown as inline overlay (badge + ring)
    on the existing VM row's status column
2026-04-30 00:08:41 +02:00
markmorado
89d70654fc fix(reinstall): use OSTemplate.vmid (not template_vmid) and convert memory bytes->MB
- AttributeError: OSTemplate has no 'template_vmid' field; the actual column is 'vmid'.
- Use OSTemplate.get_vmid_for_node() to support cross-node replicated templates,
  falling back to source_node/vmid otherwise.
- VMInstance.memory is stored in bytes, but Proxmox config expects MB.
2026-04-29 23:48:48 +02:00
markmorado
57575d58ce feat: v1.1.8 — user-server assignment, VM/LXC ownership, smart login redirect, RBAC migrations
- Migration 20: user_servers association table (User ↔ ProxmoxServer)
- API: GET/PUT /api/users/{id}/servers — assign servers to users with workspace-conflict validation
- API: GET /api/users/{id}/server-assignments — enriched server list (assigned + compatible flags)
- Non-admin server filter: intersection of assigned_servers and workspace servers
- Migration 21: dashboard:view = false for user role (new key format)
- Migration 22: proxmox.cluster.manage → cluster:manage for admin role
- API: GET/PUT /api/{server_id}/vm/{vmid}/owner — VM/LXC owner assignment (admin-only)
- Login: smart redirect based on user permissions after authentication
- Login: returnUrl param support for post-login redirect
- Roles API: user_count field per role in GET /api/roles
- node_uptime added to server detail API response
- Remove redundant PermissionChecker from GET /users HTML route
2026-03-16 08:27:36 +01:00
markmorado
bef0bcc542 feat: add Workspace feature — scoped server groups with user access and sidebar selector 2026-03-15 01:00:24 +01:00
markmorado
b3db346cfb feat: v1.1.0 — backup module, IPAM node binding, HA, task queue, theme CSS
- Backup module: full CRUD for storages, backup files, scheduled jobs (APScheduler)
- Proxmox client: storage/backup/restore/HA methods
- IPAM: proxmox_node field, server+node binding for networks and allocations
- Task queue service: improvements and new task types
- Monitoring worker: extended metrics and alerts
- HA support: add/remove VMs from Proxmox HA groups
- tasks.html: new task tracking page
- proxmox_vms.html: major UI update (bulk ops, HA, snapshots)
- backups.html: full backup management UI (3 tabs)
- base.html: sidebar backup link, websocket manager
- theme.css: full CSS variables theme, 395 lines added
- migrations: migration #15 (BackupJob), #16 (proxmox_node)
- RBAC: backup resource permissions
- websocket_manager.py: new module for WS connections
2026-03-08 18:29:29 +01:00
markmorado
91dbf0b81c feat: IPAM improvements - server/node subnet binding, theme color fixes
- IPAM link-allocations: filter networks by proxmox_server_id and proxmox_node
  to prevent cross-server/cross-node IP assignment
- IPAM link-allocations: add manual server+network selection modal before linking
- IPAM add-network forms: add Proxmox server and node selectors with dynamic
  cluster-info loading (ipam_dashboard.html, ipam_networks.html)
- models.py / schemas.py: add proxmox_node field to IPAMNetwork
- migrations: migration #16 adds proxmox_node column to ipam_networks table
- Theme fixes: replace all hardcoded rgba/color:#fff/color:white with CSS vars
  across base.html, virtual_machines.html, instance_detail.html,
  os_templates.html, proxmox_vms.html, ipam_network_detail.html,
  ipam_allocations.html
- i18n: add proxmox_node, node selector, link-allocations modal keys (ru/en)
- Version bump: 1.0.4 -> 1.0.5
2026-03-08 14:17:54 +01:00
markmorado
a3abf58453 Update repository URL to git.tzim.uz/markmorado/pvemanager 2026-03-08 12:31:42 +01:00
markmorado
85aa59e7b6 Add favicon, backup scheduler, backup API and templates, RBAC and migration updates 2026-03-08 11:40:41 +01:00
Claude
5f688abff3 security: encrypt sensitive DB fields, clean up dynamic import
Encrypt Proxmox server passwords and VM cloud-init passwords in the database:
- Add backend/app/crypto.py: Fernet-based encrypt/decrypt helpers + EncryptedString
  SQLAlchemy TypeDecorator (transparent to all existing code, no API changes)
- Add FERNET_KEY config field (env var); without it values stay plaintext for
  backward compatibility, with a logged warning
- Update models.py: ProxmoxServer.password and VMInstance.cloud_init_password
  now use EncryptedString(255) instead of String(255)
- Add migration 14 in migrations/migrations.py: idempotent re-encryption of
  existing plaintext rows (skipped when FERNET_KEY is not set)

CSRF analysis:
- /api/auth/change-password is protected by JWT Bearer token (Authorization header)
- Browsers cannot auto-include Authorization headers cross-site → CSRF is not
  applicable here; added explanatory comment

Improve migration loading:
- Add migrations/__init__.py to make it a proper Python package
- Replace fragile importlib dynamic loading in main.py with a standard import

https://claude.ai/code/session_019rK2wvaWg5zXFwmAPPEB7R
2026-03-06 16:17:45 +00:00
8dced5479d Add git repository selection for panel updates
- Add repository URL setting to panel_settings table
- Support for multiple git hosting providers (GitHub, Gitea, GitLab, etc.)
- UI selector for predefined repositories (git.tzim.uz and github.com) or custom URL
- Update service now uses repository from database settings
- API endpoints: GET/PUT /api/updates/repository
- Migration for default repository setting
- Translations for repository selection UI
2025-12-16 20:04:57 +05:00
379afd0353 feat: Add SSH public key storage for users
- Add ssh_public_key column to User model for storing SSH keys
- Add database migration for ssh_public_key column
- Add SSH key section in user profile settings with validation
- Auto-inject user's SSH key when deploying VMs from templates
- Auto-inject user's SSH key when creating LXC containers
- Support for both regular and smart LXC creation endpoints
- Add i18n translations for SSH key UI (ru/en)
- Validate SSH key format (ssh-rsa, ssh-ed25519, ecdsa-sha2, ssh-dss)

Users can now save their SSH public key in Settings -> Account,
and it will be automatically added to any VM or LXC container
they create, enabling passwordless SSH access.
2025-12-15 20:22:42 +05:00
f5a3903f5d Add complete project structure and all files 2025-12-15 13:03:29 +05:00