mirror of
https://github.com/Av0idD4rk/firegex.git
synced 2026-10-23 13:50:59 +00:00
Add one-command installer for the MCP fork
Bootstrap the fork through run.py, build Docker from source, and prepare standalone with a checksum-pinned native runtime plus the MCP backend and Python dependencies. Preserve existing installations and data, propagate Docker build failures, and update README/setup guides. Validation: 347 unit tests and a real standalone-rootfs MCP smoke test passed in Docker.
This commit is contained in:
parent
863422f202
commit
f214cecf9f
44
README.md
44
README.md
@ -20,50 +20,62 @@ for token creation and connecting an agent on your laptop to Firegex on the vuln
|
||||
The Russian [agent guide](inst.md) covers setup, every tool, and common workflows.
|
||||
|
||||
### Docker Mode (Recommended)
|
||||
What you need is a linux machine and docker ( + docker-compose )
|
||||
You need Linux, a current Python 3.10+, git, curl, and Docker with Compose.
|
||||
```bash
|
||||
# One-command installer (works for both Docker and standalone modes)
|
||||
sh <(curl -sLf https://pwnzer0tt1.it/firegex.sh)
|
||||
sh <(curl -sLf https://raw.githubusercontent.com/Av0idD4rk/firegex/main/firegex.sh)
|
||||
```
|
||||
With this command you will download firegex.py, and run it, it will require you the password to use for firegex and start it with docker-compose
|
||||
Run this command from Bash or Zsh. It clones this MCP-enabled fork into `./firegex`
|
||||
and starts it through `run.py`. Choose the UI password when prompted. Docker mode
|
||||
builds the image from this checkout, including MCP. Existing directories are never
|
||||
overwritten; set `FIREGEX_INSTALL_DIR` to choose a different location.
|
||||
|
||||
Or, you can start in a similar way firegex, cloning this repository and executing this command
|
||||
For a POSIX shell, download the script first:
|
||||
```sh
|
||||
curl -sLf https://raw.githubusercontent.com/Av0idD4rk/firegex/main/firegex.sh -o firegex-install.sh && sh firegex-install.sh
|
||||
```
|
||||
|
||||
Or clone this repository yourself and run:
|
||||
```bash
|
||||
python3 run.py start --prebuilt
|
||||
python3 run.py start
|
||||
```
|
||||
|
||||
Without the `--prebuilt` flag, it will build the docker image from source, which may take longer.
|
||||
Do not use `--prebuilt` for this fork: the upstream image does not contain MCP.
|
||||
After installation, run management commands from the installed `firegex` directory.
|
||||
|
||||
### Standalone Mode
|
||||
If Docker is not available or you're running it in a rootless environment, Firegex can run in standalone mode. The one-command installer above also works for standalone mode and will automatically detect and use standalone mode when Docker is unavailable or in rootless mode.
|
||||
|
||||
```bash
|
||||
sh <(curl -sLf https://pwnzer0tt1.it/firegex.sh)
|
||||
sh <(curl -sLf https://raw.githubusercontent.com/Av0idD4rk/firegex/main/firegex.sh)
|
||||
|
||||
# Or manually force standalone mode:
|
||||
python3 run.py start --standalone
|
||||
# Or directly using the one-command installer:
|
||||
sh <(curl -sLf https://pwnzer0tt1.it/firegex.sh) --standalone
|
||||
sh <(curl -sLf https://raw.githubusercontent.com/Av0idD4rk/firegex/main/firegex.sh) --standalone
|
||||
|
||||
# Or manually force standalone mode from a checkout:
|
||||
sudo python3 run.py start --standalone
|
||||
|
||||
# Check status
|
||||
python3 run.py status [--standalone]
|
||||
sudo python3 run.py status --standalone
|
||||
|
||||
# Stop standalone mode
|
||||
python3 run.py stop [--standalone]
|
||||
sudo python3 run.py stop --standalone
|
||||
```
|
||||
|
||||
Standalone mode automatically:
|
||||
- Downloads pre-built rootfs from GitHub releases
|
||||
- Downloads the pinned upstream 5.0.1 native runtime and checks its SHA-256
|
||||
- Detects your architecture (amd64/arm64)
|
||||
- Installs this fork's backend and Python dependencies, including MCP, into that runtime
|
||||
- Sets up chroot environment with necessary bind mounts
|
||||
- Runs as a background daemon process
|
||||
- Manages PID files for process control
|
||||
|
||||
If the server is restarted, docker mode will automatically restart the service, while standalone mode will require you to run the start command again manually.
|
||||
|
||||
Cloning the repository run.py will automatically build the docker image of firegex from source, and start it.
|
||||
Image building of firegex will require more time, so it's recommended to use the version just builded and available in the github packages.
|
||||
This is default behaviour if run.py is not in the firegex source root directory.
|
||||
Standalone installation needs root (the installer uses sudo when required) and
|
||||
network access to GitHub and PyPI. Its native engines, frontend and filter library
|
||||
come from the matching upstream release; the MCP backend comes from this checkout.
|
||||
Configuration and MCP tokens stay in the installation's existing data directory.
|
||||
|
||||
By default firegex will start in a multithread configuration using the number of threads available in your system.
|
||||
The default port of firegex is 4444. At the startup you will choose a password, that is essential for your security.
|
||||
|
||||
17
docs/mcp.md
17
docs/mcp.md
@ -20,6 +20,16 @@ Use `admin` instead of `operator` if the agent needs instance resets or UI auth
|
||||
administration. `observer` can only read configuration, statistics and logs, and
|
||||
test regex samples. Python checks execute submitted code and require `operator`.
|
||||
|
||||
For a standalone installation, run the same CLI inside its rootfs from the checkout:
|
||||
|
||||
```sh
|
||||
sudo chroot ./firegexfs /bin/sh -c 'cd /execute && python3 -m mcp_api.tokens create --name laptop --role operator --hours 168'
|
||||
```
|
||||
|
||||
The [one-command installer](../README.md#docker-mode-recommended) downloads this
|
||||
fork. Docker builds it from source; standalone installs its backend and dependencies
|
||||
over the pinned, checksum-verified 5.0.1 native runtime. Both expose the same MCP API.
|
||||
|
||||
An SSH tunnel is a convenient encrypted connection from the laptop:
|
||||
|
||||
```sh
|
||||
@ -187,3 +197,10 @@ Debian Linux kernel under QEMU inside an unprivileged container. It does not cha
|
||||
the host kernel. Emulation is slower. The harness needs x86-64 Docker images.
|
||||
For the full suite under emulation, allow a longer test budget, for example
|
||||
`python tests/docker_mcp.py --vm --full --timeout 10800`.
|
||||
|
||||
Installer regressions are included in the unit suite. The opt-in
|
||||
`FIREGEX_TEST_INSTALLER=1 ./run_tests.sh standalone/test_installer_rootfs.py`
|
||||
additionally downloads/prepares a real standalone runtime and checks its MCP API.
|
||||
Run it as root in a disposable Linux namespace with `NET_ADMIN` and `SYS_ADMIN`;
|
||||
`FIREGEX_STANDALONE_ARCHIVE` can point to a cached rootfs archive, which is still
|
||||
verified against the pinned checksum.
|
||||
|
||||
46
firegex.sh
Executable file
46
firegex.sh
Executable file
@ -0,0 +1,46 @@
|
||||
#!/bin/sh
|
||||
# Bootstrap this fork; run.py remains the deployment entry point.
|
||||
set -eu
|
||||
|
||||
fail() { printf '%s\n' "Firegex installer: $*" >&2; exit 1; }
|
||||
|
||||
[ "$(uname -s)" = Linux ] || fail 'Linux is required.'
|
||||
command -v python3 >/dev/null 2>&1 || fail 'Install Python 3.10 or newer first.'
|
||||
command -v git >/dev/null 2>&1 || fail 'Install git first.'
|
||||
python3 -c 'import sys, tarfile; sys.exit(not (sys.version_info >= (3, 10) and hasattr(tarfile, "tar_filter")))' \
|
||||
|| fail 'Update Python: version 3.10+ with tarfile extraction filters is required.'
|
||||
|
||||
standalone=0
|
||||
for argument in "$@"; do
|
||||
case "$argument" in
|
||||
--standalone) standalone=1 ;;
|
||||
--prebuilt) fail 'Upstream prebuilt images do not include MCP; omit --prebuilt.' ;;
|
||||
esac
|
||||
done
|
||||
|
||||
# Keep rootless Docker on the standalone path even after privilege elevation.
|
||||
docker_info=$(docker info --format '{{json .SecurityOptions}}' 2>/dev/null) || docker_info=
|
||||
case "$docker_info" in
|
||||
*rootless*) standalone=1; set -- "$@" --standalone ;;
|
||||
esac
|
||||
|
||||
use_sudo=0
|
||||
if [ "$(id -u)" != 0 ] && { [ "$standalone" = 1 ] || [ -z "$docker_info" ]; }; then
|
||||
command -v sudo >/dev/null 2>&1 || fail 'Root privileges are required; run as root or install sudo.'
|
||||
use_sudo=1
|
||||
fi
|
||||
|
||||
install_dir=${FIREGEX_INSTALL_DIR:-"$PWD/firegex"}
|
||||
case "$install_dir" in /*) ;; *) install_dir=$PWD/$install_dir ;; esac
|
||||
if [ -e "$install_dir" ] || [ -L "$install_dir" ]; then
|
||||
fail "$install_dir already exists. Use its run.py, or select a new FIREGEX_INSTALL_DIR."
|
||||
fi
|
||||
|
||||
printf '%s\n' "Downloading Av0idD4rk/firegex into $install_dir"
|
||||
git clone --depth 1 --branch main https://github.com/Av0idD4rk/firegex.git "$install_dir"
|
||||
printf '%s\n' "Manage this installation with: $install_dir/run.py"
|
||||
cd "$install_dir"
|
||||
if [ "$use_sudo" = 1 ]; then
|
||||
exec sudo python3 ./run.py start "$@"
|
||||
fi
|
||||
exec python3 ./run.py start "$@"
|
||||
18
inst.md
18
inst.md
@ -9,7 +9,17 @@ Firegex работает на vulnbox, агент и MCP-клиент — на
|
||||
|
||||
### На vulnbox
|
||||
|
||||
Собери именно этот форк: upstream-образ Firegex не содержит расширение MCP.
|
||||
Установи именно этот форк: upstream-образ Firegex не содержит расширение MCP.
|
||||
Установка одной командой из Bash/Zsh (исходники появятся в `./firegex`):
|
||||
|
||||
```bash
|
||||
sh <(curl -sLf https://raw.githubusercontent.com/Av0idD4rk/firegex/main/firegex.sh)
|
||||
```
|
||||
|
||||
Для принудительного standalone добавь `--standalone`. Docker-режим собирает образ
|
||||
из исходников; standalone устанавливает backend этого форка и MCP-зависимости
|
||||
поверх проверенного runtime 5.0.1. Уже существующий каталог установщик не заменяет.
|
||||
Альтернатива — ручная установка:
|
||||
|
||||
```sh
|
||||
git clone git@github.com:Av0idD4rk/firegex.git
|
||||
@ -33,6 +43,12 @@ docker exec --user nobody -w /execute <container> python3 -m mcp_api.tokens list
|
||||
docker exec --user nobody -w /execute <container> python3 -m mcp_api.tokens revoke <id>
|
||||
```
|
||||
|
||||
В standalone вместо `docker exec` запускай CLI внутри rootfs из каталога установки:
|
||||
|
||||
```sh
|
||||
sudo chroot ./firegexfs /bin/sh -c 'cd /execute && python3 -m mcp_api.tokens create --name laptop-agent --role operator --hours 168'
|
||||
```
|
||||
|
||||
| Роль | Возможности |
|
||||
|---|---|
|
||||
| `observer` | Читать конфигурацию, статистику, журналы и историю; тестировать regex |
|
||||
|
||||
168
run.py
168
run.py
@ -13,6 +13,15 @@ import hashlib
|
||||
import secrets
|
||||
import shlex
|
||||
|
||||
# Native engines, frontend and filter library are unchanged from this release.
|
||||
# Standalone overlays this fork's backend and installs its Python dependencies.
|
||||
# Update the base (and checksums) when changing those compiled components.
|
||||
STANDALONE_BASE_RELEASE = "5.0.1"
|
||||
STANDALONE_BASE_SHA256 = {
|
||||
"amd64": "4cfeeedb230f66bba527403c0d743a4e37b4a1bc62ff971d7744def05b4c25cd",
|
||||
"arm64": "4104054abb7d6189aec8ea30c1514c54dff6c628c76088046087805329217de9",
|
||||
}
|
||||
|
||||
pref = "\033["
|
||||
reset = f"{pref}0m"
|
||||
class g:
|
||||
@ -874,69 +883,118 @@ def download_file(url, filename):
|
||||
return False
|
||||
|
||||
def setup_standalone_rootfs():
|
||||
"""Set up the standalone rootfs"""
|
||||
"""Download a complete runtime before making it the installed rootfs."""
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
puts("Setting up standalone mode...", color=colors.green)
|
||||
|
||||
# Remove and recreate rootfs directory
|
||||
if os.path.exists(g.rootfs_path):
|
||||
rootfs = Path(g.rootfs_path).absolute()
|
||||
if rootfs.exists():
|
||||
if not (rootfs / "execute/app.py").is_file():
|
||||
puts("Existing rootfs is incomplete; select a clean installation directory.", color=colors.red)
|
||||
return False
|
||||
puts("Rootfs already exists, skipping download...", color=colors.yellow)
|
||||
# Clean up any existing mounts
|
||||
cleanup_standalone_mounts()
|
||||
return True
|
||||
|
||||
puts("Creating rootfs directory...", color=colors.green)
|
||||
try:
|
||||
os.makedirs(g.rootfs_path, exist_ok=True)
|
||||
except Exception as e:
|
||||
puts(f"Failed to create rootfs directory: {e}", color=colors.red)
|
||||
return False
|
||||
|
||||
# Get latest release tag
|
||||
release_tag = get_latest_release_tag()
|
||||
|
||||
source_backend = Path(__file__).resolve().parent / "backend/mcp_api"
|
||||
release_tag = STANDALONE_BASE_RELEASE if source_backend.is_dir() else get_latest_release_tag()
|
||||
if not release_tag:
|
||||
puts("Failed to get latest release tag", color=colors.red)
|
||||
return False
|
||||
|
||||
# Get current architecture
|
||||
arch = get_architecture()
|
||||
if not arch:
|
||||
return False
|
||||
|
||||
# Download rootfs from GitHub releases
|
||||
puts(f"Downloading rootfs for {arch} architecture from GitHub releases...", color=colors.green)
|
||||
|
||||
# Construct download URL
|
||||
rootfs_filename = f"firegex-rootfs-{arch}.tar.gz"
|
||||
download_url = f"https://github.com/Pwnzer0tt1/firegex/releases/download/{release_tag}/{rootfs_filename}"
|
||||
tar_path = os.path.join(g.rootfs_path, rootfs_filename)
|
||||
|
||||
# Download the rootfs archive
|
||||
if not download_file(download_url, tar_path):
|
||||
return False
|
||||
|
||||
|
||||
try:
|
||||
# Extract tar.gz file
|
||||
puts("Extracting rootfs...", color=colors.green)
|
||||
with tarfile.open(tar_path, 'r:gz') as tar:
|
||||
# Extract all files with tar filter (allows safe symbolic links)
|
||||
tar.extractall(path=g.rootfs_path, filter='tar')
|
||||
|
||||
# Remove tar.gz file
|
||||
os.remove(tar_path)
|
||||
|
||||
# Create necessary directories
|
||||
os.makedirs(os.path.join(g.rootfs_path, "dev"), exist_ok=True)
|
||||
os.makedirs(os.path.join(g.rootfs_path, "proc"), exist_ok=True)
|
||||
os.makedirs(os.path.join(g.rootfs_path, "sys_host"), exist_ok=True)
|
||||
|
||||
with tempfile.TemporaryDirectory(prefix=".firegex-download-", dir=rootfs.parent) as tmp:
|
||||
archive = Path(tmp) / rootfs_filename
|
||||
if not download_file(download_url, str(archive)):
|
||||
return False
|
||||
if source_backend.is_dir():
|
||||
checksum = hashlib.sha256()
|
||||
with archive.open("rb") as stream:
|
||||
for chunk in iter(lambda: stream.read(1024 * 1024), b""):
|
||||
checksum.update(chunk)
|
||||
if checksum.hexdigest() != STANDALONE_BASE_SHA256[arch]:
|
||||
raise ValueError("Standalone runtime checksum mismatch")
|
||||
staged = Path(tmp) / "rootfs"
|
||||
staged.mkdir()
|
||||
puts("Extracting rootfs...", color=colors.green)
|
||||
with tarfile.open(archive, 'r:gz') as tar:
|
||||
tar.extractall(path=staged, filter='tar')
|
||||
if not (staged / "execute/app.py").is_file():
|
||||
raise ValueError("Standalone runtime does not contain execute/app.py")
|
||||
for directory in ("dev", "proc", "sys_host"):
|
||||
(staged / directory).mkdir(exist_ok=True)
|
||||
(staged / ".firegex-native-base").write_text(release_tag)
|
||||
staged.rename(rootfs)
|
||||
puts("Rootfs setup completed", color=colors.green)
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
puts(f"Failed to extract rootfs: {e}", color=colors.red)
|
||||
# Clean up partial extraction
|
||||
if os.path.exists(tar_path):
|
||||
os.remove(tar_path)
|
||||
puts(f"Failed to prepare rootfs: {e}", color=colors.red)
|
||||
return False
|
||||
|
||||
|
||||
def prepare_standalone_backend():
|
||||
"""Install the fork's backend into the matching native runtime, preserving DBs."""
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
|
||||
backend = Path(__file__).resolve().parent / "backend"
|
||||
if not (backend / "mcp_api").is_dir():
|
||||
return True
|
||||
rootfs = Path(g.rootfs_path).resolve()
|
||||
execute = rootfs / "execute"
|
||||
try:
|
||||
base_marker = rootfs / ".firegex-native-base"
|
||||
if base_marker.exists():
|
||||
version = base_marker.read_text().strip()
|
||||
else:
|
||||
result = subprocess.run(["chroot", str(rootfs), "/bin/python3", "-c",
|
||||
"import sys; sys.path.insert(0,'/execute'); import utils; print(utils.API_VERSION)"],
|
||||
check=True, capture_output=True, text=True)
|
||||
version = result.stdout.strip()
|
||||
if version != STANDALONE_BASE_RELEASE:
|
||||
raise ValueError(f"This fork requires standalone runtime {STANDALONE_BASE_RELEASE}, found {version}")
|
||||
base_marker.write_text(version)
|
||||
|
||||
# Only application files; never copy local DBs, filter uploads, caches or
|
||||
# compiled engines over the runtime's data and native executables.
|
||||
for folder, dirs, files in os.walk(backend):
|
||||
dirs[:] = [d for d in dirs if d not in {"db", "__pycache__", "binsrc", "proxysrc", ".git", ".venv"}]
|
||||
for name in files:
|
||||
if not (name.endswith((".py", ".sh")) or name == "pyproject.toml"):
|
||||
continue
|
||||
source = Path(folder) / name
|
||||
target = execute / source.relative_to(backend)
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copy2(source, target)
|
||||
|
||||
dependencies = hashlib.sha256((backend / "pyproject.toml").read_bytes()).hexdigest()
|
||||
marker = execute / ".firegex-backend-deps"
|
||||
if not marker.exists() or marker.read_text() != dependencies:
|
||||
# Docker's exported resolv.conf can be empty. Do not follow an absolute
|
||||
# symlink from the chroot while installing its resolver configuration.
|
||||
resolver = rootfs / "etc/resolv.conf"
|
||||
if resolver.is_symlink():
|
||||
resolver.unlink()
|
||||
shutil.copyfile("/etc/resolv.conf", resolver)
|
||||
puts("Installing this fork's standalone Python dependencies...", color=colors.green)
|
||||
install = (
|
||||
"import subprocess,sys,tomllib; "
|
||||
"deps=tomllib.load(open('/execute/pyproject.toml','rb'))['project']['dependencies']; "
|
||||
"subprocess.run([sys.executable,'-m','pip','install','--no-cache-dir',"
|
||||
"'--disable-pip-version-check',*deps],check=True)"
|
||||
)
|
||||
subprocess.run(["chroot", str(rootfs), "/bin/python3", "-c", install], check=True)
|
||||
subprocess.run(["chroot", str(rootfs), "/bin/python3", "-c",
|
||||
"from mcp.server import MCPServer"], check=True)
|
||||
marker.write_text(dependencies)
|
||||
return True
|
||||
except Exception as error:
|
||||
puts(f"Failed to prepare standalone backend: {error}", color=colors.red)
|
||||
return False
|
||||
|
||||
def setup_standalone_mounts():
|
||||
@ -1001,6 +1059,10 @@ def run_standalone():
|
||||
puts(f"Process PID: {pid}", color=colors.cyan)
|
||||
return
|
||||
|
||||
if not prepare_standalone_backend():
|
||||
cleanup_standalone_mounts()
|
||||
raise SystemExit(1)
|
||||
|
||||
warn_if_auth_disabled()
|
||||
|
||||
# Set up environment variables
|
||||
@ -1419,13 +1481,13 @@ def main():
|
||||
sudo = "sudo " if g.docker_sudo else ""
|
||||
if not cmd_check(f"{sudo}docker --version"):
|
||||
puts("Docker not found! please install docker and docker compose!", color=colors.red)
|
||||
exit()
|
||||
exit(1)
|
||||
elif not cmd_check(f"{sudo}docker-compose --version") and not cmd_check(f"{sudo}docker compose --version"):
|
||||
puts("Docker compose not found! please install docker compose!", color=colors.red)
|
||||
exit()
|
||||
exit(1)
|
||||
if not cmd_check(f"{sudo}docker ps"):
|
||||
puts("Cannot use docker, the user hasn't the permission or docker isn't running", color=colors.red)
|
||||
exit()
|
||||
exit(1)
|
||||
|
||||
if not is_linux():
|
||||
sep()
|
||||
@ -1453,7 +1515,9 @@ def main():
|
||||
puts(f"Downloading docker image from github packages '{sudo}docker pull ghcr.io/pwnzer0tt1/firegex'", color=colors.green)
|
||||
cmd_check(f"{sudo}docker pull ghcr.io/pwnzer0tt1/firegex:{args.version}", print_output=True)
|
||||
puts("Running 'docker compose up -d --build'\n", color=colors.green)
|
||||
composecmd("up -d --build", g.composefile)
|
||||
result = composecmd("up -d --build", g.composefile)
|
||||
if result is None or result.returncode:
|
||||
raise SystemExit(result.returncode if result is not None else 1)
|
||||
case "compose":
|
||||
write_compose()
|
||||
compose_cmd = " ".join(args.compose_args)
|
||||
|
||||
@ -11,5 +11,6 @@ RUN python3 -m venv /test/tests/.venv && \
|
||||
/test/tests/.venv/bin/pip install -e /test/fgex-lib
|
||||
COPY tests /test/tests
|
||||
COPY backend /test/backend
|
||||
COPY run.py firegex.sh /test/
|
||||
ENV FIREGEX_MCP_TEST_DB=/execute/db/mcp/control.sqlite3
|
||||
ENV FIREGEX_TEST_DEPS_READY=1
|
||||
|
||||
91
tests/standalone/test_installer_rootfs.py
Normal file
91
tests/standalone/test_installer_rootfs.py
Normal file
@ -0,0 +1,91 @@
|
||||
"""Opt-in packaging smoke: a real upstream rootfs must start this fork's MCP.
|
||||
|
||||
Run in a disposable Linux namespace with NET_ADMIN and SYS_ADMIN:
|
||||
FIREGEX_TEST_INSTALLER=1 ./run_tests.sh standalone/test_installer_rootfs.py
|
||||
FIREGEX_STANDALONE_ARCHIVE may name a cached archive; its checksum is still checked.
|
||||
"""
|
||||
import asyncio
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import signal
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
|
||||
import pytest
|
||||
import requests
|
||||
from mcp import Client
|
||||
from mcp.client.streamable_http import streamable_http_client, create_mcp_http_client
|
||||
from helpers.net import free_port
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
|
||||
|
||||
@pytest.mark.root
|
||||
def test_standalone_runtime_exposes_fork_mcp(tmp_path, monkeypatch):
|
||||
if os.getenv("FIREGEX_TEST_INSTALLER") != "1":
|
||||
pytest.skip("Opt in with FIREGEX_TEST_INSTALLER=1 in a disposable namespace")
|
||||
monkeypatch.chdir(tmp_path)
|
||||
monkeypatch.setattr(sys, "argv", [str(ROOT / "run.py"), "status"])
|
||||
spec = importlib.util.spec_from_file_location("standalone_installer", ROOT / "run.py")
|
||||
manager = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(manager)
|
||||
os.chdir(tmp_path)
|
||||
manager.g.rootfs_path = str(tmp_path / "rootfs")
|
||||
manager.g.pid_file = str(tmp_path / "backend.pid")
|
||||
manager.g.configfile = str(tmp_path / "config.json")
|
||||
manager.g.standalone_mode = True
|
||||
port = free_port()
|
||||
manager.args = manager.gen_args(["start", "--standalone", "--host", "127.0.0.1",
|
||||
"--port", str(port), "--threads", "2", "--psw-on-web"])
|
||||
cached = os.getenv("FIREGEX_STANDALONE_ARCHIVE")
|
||||
if cached:
|
||||
monkeypatch.setattr(manager, "download_file", lambda url, path: bool(shutil.copyfile(cached, path)))
|
||||
assert manager.setup_standalone_rootfs()
|
||||
rootfs = Path(manager.g.rootfs_path)
|
||||
pid = None
|
||||
try:
|
||||
assert manager.setup_standalone_mounts()
|
||||
manager.run_standalone()
|
||||
pid = manager.read_pid_file()
|
||||
address = f"http://127.0.0.1:{port}"
|
||||
for _ in range(60):
|
||||
try:
|
||||
if requests.get(address + "/api/status", timeout=1).status_code == 200:
|
||||
break
|
||||
except requests.RequestException:
|
||||
pass
|
||||
time.sleep(0.5)
|
||||
else:
|
||||
pytest.fail(Path("/tmp/app.log").read_text()[-8000:])
|
||||
token = json.loads(subprocess.check_output([
|
||||
"chroot", str(rootfs), "/bin/sh", "-c",
|
||||
"cd /execute && python3 -m mcp_api.tokens create --name installer-test --role operator --hours 1",
|
||||
], text=True))
|
||||
assert requests.post(address + "/mcp/", json={}, timeout=5).status_code == 401
|
||||
|
||||
async def scenario():
|
||||
async with create_mcp_http_client(headers={"Authorization": "Bearer " + token["token"]}) as http:
|
||||
async with Client(streamable_http_client(address + "/mcp/", http_client=http)) as client:
|
||||
catalog = await client.list_tools()
|
||||
assert len(catalog.tools) == 28
|
||||
overview = await client.call_tool("fgx_overview", {})
|
||||
assert overview.structured_content["ok"], overview
|
||||
checked = await client.call_tool("fgx_python_check", {"code": "pass"})
|
||||
assert checked.structured_content["data"]["ok"], checked
|
||||
asyncio.run(scenario())
|
||||
assert (rootfs / "execute/.firegex-backend-deps").is_file()
|
||||
assert (rootfs / "execute/db/mcp/control.sqlite3").is_file()
|
||||
finally:
|
||||
pid = pid or manager.read_pid_file()
|
||||
if pid:
|
||||
try:
|
||||
os.killpg(pid, signal.SIGTERM)
|
||||
os.waitpid(pid, 0)
|
||||
except ProcessLookupError:
|
||||
pass
|
||||
manager.cleanup_standalone_mounts()
|
||||
manager.remove_pid_file()
|
||||
187
tests/unit/test_installer.py
Normal file
187
tests/unit/test_installer.py
Normal file
@ -0,0 +1,187 @@
|
||||
"""Installation must select the fork, preserve data and propagate failures."""
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
|
||||
|
||||
def executable(path, source):
|
||||
path.write_text(f"#!{sys.executable}\n" + source)
|
||||
path.chmod(0o755)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def installer_env(tmp_path):
|
||||
binaries = tmp_path / "bin"
|
||||
binaries.mkdir()
|
||||
executable(binaries / "git", """
|
||||
import os,sys
|
||||
from pathlib import Path
|
||||
assert 'https://github.com/Av0idD4rk/firegex.git' in sys.argv
|
||||
if os.environ.get('CLONE_FAIL'): sys.exit(17)
|
||||
target=Path(sys.argv[-1]); target.mkdir()
|
||||
(target/'run.py').write_text('import os,sys,json; from pathlib import Path; '
|
||||
'Path(os.environ["RESULT"]).write_text(json.dumps(sys.argv[1:])); '
|
||||
'sys.exit(int(os.environ.get("RUN_EXIT", "0")))')
|
||||
""")
|
||||
executable(binaries / "docker", """
|
||||
import os,sys
|
||||
info=os.environ.get('DOCKER_INFO','')
|
||||
print(info)
|
||||
sys.exit(0 if info else 1)
|
||||
""")
|
||||
executable(binaries / "id", "import os; print(os.environ['TEST_UID'])")
|
||||
executable(binaries / "sudo", """
|
||||
import os,sys
|
||||
from pathlib import Path
|
||||
Path(os.environ['SUDO_USED']).touch()
|
||||
os.execvp(sys.argv[1],sys.argv[1:])
|
||||
""")
|
||||
env = {**os.environ, "PATH": str(binaries) + os.pathsep + os.environ["PATH"],
|
||||
"FIREGEX_INSTALL_DIR": str(tmp_path / "installation with spaces"),
|
||||
"RESULT": str(tmp_path / "result.json"), "SUDO_USED": str(tmp_path / "sudo"),
|
||||
"TEST_UID": "1000", "DOCKER_INFO": '["name=seccomp"]'}
|
||||
return env
|
||||
|
||||
|
||||
@pytest.mark.parametrize("info,uid,arguments,sudo,forced", [
|
||||
('["name=seccomp"]', "1000", [], False, False),
|
||||
('["name=seccomp"]', "1000", ["--standalone"], True, True),
|
||||
("", "1000", [], True, False),
|
||||
('["name=rootless"]', "1000", [], True, True),
|
||||
("", "0", [], False, False),
|
||||
])
|
||||
def test_bootstrap_preserves_arguments_and_selects_privileges(installer_env, info, uid, arguments, sudo, forced):
|
||||
env = {**installer_env, "DOCKER_INFO": info, "TEST_UID": uid}
|
||||
literal = "EXAMPLE=spaces; $(not-a-command)"
|
||||
result = subprocess.run(["sh", str(ROOT / "firegex.sh"), *arguments, "--env", literal],
|
||||
env=env, capture_output=True, text=True)
|
||||
assert result.returncode == 0, result.stderr
|
||||
received = json.loads(Path(env["RESULT"]).read_text())
|
||||
assert received[0] == "start" and literal in received
|
||||
assert ("--standalone" in received) == forced
|
||||
assert Path(env["SUDO_USED"]).exists() == sudo
|
||||
|
||||
|
||||
def test_existing_installation_is_untouched(installer_env):
|
||||
target = Path(installer_env["FIREGEX_INSTALL_DIR"])
|
||||
target.mkdir()
|
||||
(target / "important.db").write_bytes(b"keep me")
|
||||
result = subprocess.run(["sh", str(ROOT / "firegex.sh")], env=installer_env, capture_output=True)
|
||||
assert result.returncode != 0 and b"already exists" in result.stderr
|
||||
assert (target / "important.db").read_bytes() == b"keep me"
|
||||
assert not Path(installer_env["RESULT"]).exists()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("extra,arguments,code", [
|
||||
({"CLONE_FAIL": "1"}, [], 17),
|
||||
({"RUN_EXIT": "42"}, [], 42),
|
||||
({}, ["--prebuilt"], 1),
|
||||
])
|
||||
def test_installer_failure_is_reported(installer_env, extra, arguments, code):
|
||||
result = subprocess.run(["sh", str(ROOT / "firegex.sh"), *arguments],
|
||||
env={**installer_env, **extra}, capture_output=True)
|
||||
assert result.returncode == code
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def manager(tmp_path, monkeypatch):
|
||||
monkeypatch.chdir(tmp_path)
|
||||
monkeypatch.setattr(sys, "argv", [str(ROOT / "run.py"), "status"])
|
||||
spec = importlib.util.spec_from_file_location("installer_manager", ROOT / "run.py")
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
os.chdir(tmp_path)
|
||||
source = tmp_path / "source"
|
||||
(source / "backend/mcp_api").mkdir(parents=True)
|
||||
(source / "backend/mcp_api/server.py").write_text("# new MCP backend\n")
|
||||
(source / "backend/pyproject.toml").write_text('[project]\ndependencies=["mcp==2.3.0"]\n')
|
||||
module.__file__ = str(source / "run.py")
|
||||
module.g.rootfs_path = str(tmp_path / "rootfs")
|
||||
monkeypatch.setattr(module, "cleanup_standalone_mounts", lambda: None)
|
||||
return module
|
||||
|
||||
|
||||
def make_runtime(manager):
|
||||
rootfs = Path(manager.g.rootfs_path)
|
||||
for folder in ("execute/db/mcp", "execute/modules", "etc"):
|
||||
(rootfs / folder).mkdir(parents=True, exist_ok=True)
|
||||
(rootfs / "execute/app.py").write_text("# existing backend\n")
|
||||
(rootfs / ".firegex-native-base").write_text(manager.STANDALONE_BASE_RELEASE)
|
||||
return rootfs
|
||||
|
||||
|
||||
def test_backend_install_preserves_databases_and_native_engine(manager, monkeypatch):
|
||||
rootfs = make_runtime(manager)
|
||||
db = rootfs / "execute/db/mcp/control.sqlite3"
|
||||
engine = rootfs / "execute/modules/fgex-proxy"
|
||||
db.write_bytes(b"credentials and changes")
|
||||
engine.write_bytes(b"native engine")
|
||||
source = Path(manager.__file__).parent / "backend"
|
||||
(source / "db").mkdir()
|
||||
(source / "db/unrelated.py").write_text("must not be copied")
|
||||
calls = []
|
||||
monkeypatch.setattr(manager.subprocess, "run", lambda *a, **kw: calls.append(a) or SimpleNamespace(returncode=0))
|
||||
assert manager.prepare_standalone_backend()
|
||||
assert (rootfs / "execute/mcp_api/server.py").read_text() == "# new MCP backend\n"
|
||||
assert db.read_bytes() == b"credentials and changes" and engine.read_bytes() == b"native engine"
|
||||
assert not (rootfs / "execute/db/unrelated.py").exists()
|
||||
assert len(calls) == 2
|
||||
assert manager.prepare_standalone_backend()
|
||||
assert len(calls) == 2 # unchanged requirements do not need a network install
|
||||
|
||||
|
||||
def test_incompatible_runtime_is_refused_before_copy(manager):
|
||||
rootfs = make_runtime(manager)
|
||||
(rootfs / ".firegex-native-base").write_text("4.0.0")
|
||||
assert not manager.prepare_standalone_backend()
|
||||
assert not (rootfs / "execute/mcp_api").exists()
|
||||
|
||||
|
||||
def test_failed_dependencies_are_not_marked_installed(manager, monkeypatch):
|
||||
rootfs = make_runtime(manager)
|
||||
def fail(*a, **kw):
|
||||
raise subprocess.CalledProcessError(1, "pip")
|
||||
monkeypatch.setattr(manager.subprocess, "run", fail)
|
||||
assert not manager.prepare_standalone_backend()
|
||||
assert not (rootfs / "execute/.firegex-backend-deps").exists()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("download_ok", [False, True])
|
||||
def test_failed_download_or_checksum_leaves_no_rootfs(manager, monkeypatch, download_ok):
|
||||
monkeypatch.setattr(manager, "get_architecture", lambda: "amd64")
|
||||
def download(url, filename):
|
||||
assert "/5.0.1/" in url
|
||||
Path(filename).write_bytes(b"truncated or corrupt archive")
|
||||
return download_ok
|
||||
monkeypatch.setattr(manager, "download_file", download)
|
||||
assert not manager.setup_standalone_rootfs()
|
||||
assert not Path(manager.g.rootfs_path).exists()
|
||||
assert not list(Path(manager.g.rootfs_path).parent.glob(".firegex-download-*"))
|
||||
|
||||
|
||||
def test_partial_old_runtime_is_not_treated_as_installed(manager):
|
||||
Path(manager.g.rootfs_path).mkdir()
|
||||
assert not manager.setup_standalone_rootfs()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("result,code", [(SimpleNamespace(returncode=17), 17), (None, 1)])
|
||||
def test_docker_build_failure_reaches_the_installer(manager, monkeypatch, result, code):
|
||||
manager.args = manager.gen_args(["start", "--psw-on-web"])
|
||||
manager.g.build = True
|
||||
for name, value in {"should_use_standalone": False, "is_linux": True,
|
||||
"nfqueue_exists": True, "check_already_running": False}.items():
|
||||
monkeypatch.setattr(manager, name, lambda value=value: value)
|
||||
monkeypatch.setattr(manager, "cmd_check", lambda *a, **kw: True)
|
||||
monkeypatch.setattr(manager, "write_compose", lambda *a, **kw: None)
|
||||
monkeypatch.setattr(manager, "composecmd", lambda *a, **kw: result)
|
||||
with pytest.raises(SystemExit) as error:
|
||||
manager.main()
|
||||
assert error.value.code == code
|
||||
Loading…
Reference in New Issue
Block a user