mirror of
https://github.com/KuekHaoYang/KVideo.git
synced 2026-08-15 00:33:44 +08:00
Merge pull request #232 from KuekHaoYang/agent/strict-verification-suite
feat: add strict verification suite; release 4.9.20
This commit is contained in:
Kuek Hao Yang
@@ -1,5 +1,13 @@
|
||||
# Changelog
|
||||
|
||||
## 4.9.20 - 2026-07-31
|
||||
|
||||
- 新增独立 `verification/` 严格验证套件，可通过 `./verification/run` 一键执行源码行数、AST 复杂度、函数长度、嵌套、依赖环、重复代码、secret、危险构造、ESLint、TypeScript、单元测试、依赖完整性与漏洞审计。
|
||||
- 覆盖 Next.js、Cloudflare Pages 和 Docker 构建/运行，自动发现 API 方法，并验证异常输入、SSE、代理 Range/HLS/CORS/重定向/SSRF、延迟、并发、安全响应头、PWA 与公开发布版本一致性。
|
||||
- 新增四视口页面渲染、Axe 可访问性、递归 UI 状态与全部发现控件交互；每条动作记录完整恢复步骤和匹配依据，并输出截图、像素 diff 与 Playwright trace。
|
||||
- 新增确定性 MP4/HLS、分辨率、掉帧和 200ms 卡顿检测；报告同时提供 HTML、Markdown、JSON、JUnit、NDJSON 与完整原始日志，并自动脱敏。
|
||||
- 验证目录所有自有文件均不超过 150 行。本版本只加入验证基础设施与发布元数据，不修改现有业务代码；套件发现的项目缺陷会以非零退出码和详细证据保留，不伪装为通过。
|
||||
|
||||
## 4.9.19 - 2026-07-31
|
||||
|
||||
- 修复 Docker / Node 自托管下 `/api/user/sync` 与 `/api/user/config` 无法从 `process.env` 读取 Upstash 凭据、持续返回 500 的问题；Redis 客户端改为按请求环境惰性创建，未配置同步时明确返回 503（#226 / PR #227）。
|
||||
|
||||
+12
-1
@@ -4,8 +4,19 @@
|
||||
"name": "KVideo",
|
||||
"branch": "main"
|
||||
},
|
||||
"currentVersion": "4.9.19",
|
||||
"currentVersion": "4.9.20",
|
||||
"releases": [
|
||||
{
|
||||
"version": "4.9.20",
|
||||
"publishedAt": "2026-07-31",
|
||||
"title": "新增严格全链路验证套件",
|
||||
"notes": [
|
||||
"新增独立 verification 目录与一键运行命令，覆盖源码结构、复杂度、依赖、安全、构建、Docker、API、代理、性能、PWA 与发布一致性检查。",
|
||||
"新增四视口页面渲染、Axe 可访问性、递归 UI 状态与控件交互验证，并记录每次动作的恢复步骤、匹配依据、异常、截图和 Playwright trace。",
|
||||
"新增确定性 MP4/HLS 播放、分辨率、掉帧与 200ms 卡顿检测，以及本地和 Cloudflare 的像素差异报告。",
|
||||
"验证结果同时输出 HTML、Markdown、JSON、JUnit、NDJSON 和原始命令日志；验证器自有文件全部限制在 150 行以内，且不会修改业务源码。"
|
||||
]
|
||||
},
|
||||
{
|
||||
"version": "4.9.19",
|
||||
"publishedAt": "2026-07-31",
|
||||
|
||||
Generated
+2
-2
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "kvideo",
|
||||
"version": "4.9.19",
|
||||
"version": "4.9.20",
|
||||
"lockfileVersion": 2,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "kvideo",
|
||||
"version": "4.9.19",
|
||||
"version": "4.9.20",
|
||||
"dependencies": {
|
||||
"@dnd-kit/core": "^6.3.1",
|
||||
"@dnd-kit/sortable": "^10.0.0",
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "kvideo",
|
||||
"version": "4.9.19",
|
||||
"version": "4.9.20",
|
||||
"private": true,
|
||||
"scripts": {
|
||||
"dev": "node scripts/next-with-lan-access.mjs dev",
|
||||
|
||||
@@ -0,0 +1,4 @@
|
||||
artifacts/
|
||||
node_modules/
|
||||
tmp/
|
||||
*.log
|
||||
@@ -0,0 +1,50 @@
|
||||
# KVideo strict verification
|
||||
|
||||
Run the complete read-only validation chain from the repository root:
|
||||
|
||||
```sh
|
||||
./verification/run
|
||||
```
|
||||
|
||||
The runner installs its pinned tools inside this directory, builds and starts
|
||||
KVideo locally, exercises APIs and UI, and writes evidence under
|
||||
`verification/artifacts/<run-id>/`. It does not edit application source.
|
||||
|
||||
Primary outputs:
|
||||
|
||||
- `report.html`: browsable report with every finding and evidence link.
|
||||
- `summary.md`: compact human-readable result and coverage gaps.
|
||||
- `summary.json`: machine-readable run metadata and totals.
|
||||
- `findings.json`: every pass, failure, warning, skip, and explanation.
|
||||
- `junit.xml`: CI-compatible result file.
|
||||
- `events.ndjson`: chronological structured event log.
|
||||
- `run.log`: chronological plain-text log.
|
||||
- `raw/`: complete command output and remote response evidence.
|
||||
- `screenshots/`: viewport, action, and visual-difference images.
|
||||
- `traces/`: browser traces for failed flows.
|
||||
|
||||
Useful options:
|
||||
|
||||
```sh
|
||||
./verification/run --quick
|
||||
./verification/run --offline
|
||||
./verification/run --reference-url https://kvideo.pages.dev
|
||||
./verification/run --keep-server
|
||||
./verification/run --max-actions 10000 --max-action-depth 10
|
||||
```
|
||||
|
||||
Full mode explores up to 5,000 unique control-state operations per route and
|
||||
eight same-route state transitions. Repeated controls with identical structure
|
||||
and state are tested once; changed checked, expanded, pressed, value, and
|
||||
disabled states are separate operations. Reaching either limit is reported as
|
||||
a coverage failure, never as a pass.
|
||||
|
||||
Default mode is deliberately strict. Existing source files over 150 lines,
|
||||
lint/type/build failures, uncaught browser errors, severe accessibility
|
||||
violations, API contract failures, deployment drift, and threshold breaches
|
||||
produce a non-zero exit code. Generated reports and third-party files are not
|
||||
source code and are excluded from the 150-line source policy.
|
||||
|
||||
The suite cannot prove the absence of every defect. It reports exactly what it
|
||||
enumerated, what it executed, what it skipped, and why. A green result means all
|
||||
declared checks passed, not that arbitrary undiscovered states are impossible.
|
||||
Generated
+1
File diff suppressed because one or more lines are too long
@@ -0,0 +1,18 @@
|
||||
{
|
||||
"name": "kvideo-strict-verification",
|
||||
"version": "1.0.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"verify": "node src/main.mjs"
|
||||
},
|
||||
"dependencies": {
|
||||
"@typescript-eslint/typescript-estree": "8.65.0",
|
||||
"axe-core": "4.12.1",
|
||||
"jscpd": "5.0.14",
|
||||
"pixelmatch": "7.2.0",
|
||||
"playwright": "1.62.1",
|
||||
"pngjs": "7.0.0",
|
||||
"wrangler": "4.118.0"
|
||||
}
|
||||
}
|
||||
Executable
+19
@@ -0,0 +1,19 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
|
||||
verify_dir=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
|
||||
repo_dir=$(CDPATH= cd -- "$verify_dir/.." && pwd)
|
||||
|
||||
if ! command -v node >/dev/null 2>&1; then
|
||||
printf '%s\n' 'ERROR: Node.js is required.' >&2
|
||||
exit 127
|
||||
fi
|
||||
|
||||
if ! command -v npm >/dev/null 2>&1; then
|
||||
printf '%s\n' 'ERROR: npm is required.' >&2
|
||||
exit 127
|
||||
fi
|
||||
|
||||
export PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1
|
||||
npm ci --prefix "$verify_dir" --no-audit --no-fund --ignore-scripts
|
||||
exec node "$verify_dir/src/main.mjs" --root "$repo_dir" "$@"
|
||||
@@ -0,0 +1,53 @@
|
||||
function finite(value, fallback) {
|
||||
if (value === '' || value === 'any') return fallback;
|
||||
const parsed = Number(value);
|
||||
return Number.isFinite(parsed) ? parsed : fallback;
|
||||
}
|
||||
|
||||
export function numericCandidate({ min, max, value, step }) {
|
||||
const lower = finite(min, 0);
|
||||
const upper = Math.max(lower, finite(max, lower + 100));
|
||||
const increment = Math.max(Number.EPSILON, finite(step, 1));
|
||||
const midpoint = lower + ((upper - lower) / 2);
|
||||
const snapped = Math.min(upper, lower + (Math.round((midpoint - lower) / increment) * increment));
|
||||
const current = finite(value, lower);
|
||||
const candidate = Math.abs(snapped - current) > Number.EPSILON ? snapped : lower;
|
||||
return String(Number(candidate.toFixed(10)));
|
||||
}
|
||||
|
||||
async function numericValue(locator) {
|
||||
const bounds = await locator.evaluate((element) => ({
|
||||
min: element.min, max: element.max, value: element.value, step: element.step,
|
||||
}));
|
||||
return numericCandidate(bounds);
|
||||
}
|
||||
|
||||
export async function fillInput(locator, type) {
|
||||
if (['number', 'range'].includes(type)) return locator.fill(await numericValue(locator));
|
||||
if (type === 'url') return locator.fill('https://example.com/verification');
|
||||
if (type === 'email') return locator.fill('[email protected]');
|
||||
if (type === 'tel') return locator.fill('0123456789');
|
||||
return locator.fill('验证');
|
||||
}
|
||||
|
||||
export async function toggleInput(locator, type) {
|
||||
const label = locator.locator('xpath=ancestor::label[1]');
|
||||
if (await label.count()) {
|
||||
await label.click({ timeout: 5000 });
|
||||
return 'clickLabel';
|
||||
}
|
||||
const checked = type === 'radio' ? true : !(await locator.isChecked());
|
||||
await locator.setChecked(checked, { force: true });
|
||||
return 'setChecked';
|
||||
}
|
||||
|
||||
export async function prepareActionState(page, action) {
|
||||
const label = action.aria.trim().toLowerCase();
|
||||
const mode = /^(播放|play)$/.test(label) ? 'paused' : /^(暂停|pause)$/.test(label) ? 'playing' : null;
|
||||
if (!mode) return;
|
||||
await page.evaluate(async (expected) => {
|
||||
const videos = [...document.querySelectorAll('video')];
|
||||
if (expected === 'paused') videos.forEach((video) => video.pause());
|
||||
else await Promise.all(videos.map((video) => video.play().catch(() => {})));
|
||||
}, mode);
|
||||
}
|
||||
@@ -0,0 +1,105 @@
|
||||
import crypto from 'node:crypto';
|
||||
import { fillInput, prepareActionState, toggleInput } from './action-state.mjs';
|
||||
|
||||
export async function scanActions(page) {
|
||||
return page.evaluate(() => {
|
||||
const selector = 'button,a[href],input,select,textarea,[role="button"],[data-focusable]';
|
||||
document.querySelectorAll('[data-kv-verify]').forEach((element) => element.removeAttribute('data-kv-verify'));
|
||||
const visible = (element) => {
|
||||
const style = getComputedStyle(element);
|
||||
const box = element.getBoundingClientRect();
|
||||
if (element.closest('[aria-hidden="true"],[hidden],[inert]')) return false;
|
||||
if (style.visibility === 'hidden' || style.display === 'none' || style.pointerEvents === 'none' || Number(style.opacity) === 0) return false;
|
||||
if (box.width <= 0 || box.height <= 0) return false;
|
||||
const fixed = style.position === 'fixed' || getComputedStyle(element.parentElement || element).position === 'fixed';
|
||||
if (fixed && (box.bottom <= 0 || box.top >= innerHeight || box.right <= 0 || box.left >= innerWidth)) return false;
|
||||
const intersects = box.bottom > 0 && box.top < innerHeight && box.right > 0 && box.left < innerWidth;
|
||||
if (!intersects) return true;
|
||||
const x = Math.max(0, Math.min(innerWidth - 1, box.left + box.width / 2));
|
||||
const y = Math.max(0, Math.min(innerHeight - 1, box.top + box.height / 2));
|
||||
const top = document.elementFromPoint(x, y);
|
||||
return !top || top === element || element.contains(top);
|
||||
};
|
||||
const identity = (element) => {
|
||||
const explicit = ['id', 'name', 'data-testid', 'aria-controls'].map((name) => element.getAttribute(name)).find(Boolean);
|
||||
if (explicit) return `${element.tagName.toLowerCase()}#${explicit}`;
|
||||
const parts = [];
|
||||
for (let current = element; current?.parentElement && current !== document.body && parts.length < 6; current = current.parentElement) {
|
||||
const siblings = [...current.parentElement.children].filter((item) => item.tagName === current.tagName);
|
||||
parts.unshift(`${current.tagName.toLowerCase()}:${siblings.indexOf(current) + 1}`);
|
||||
}
|
||||
return parts.join('>');
|
||||
};
|
||||
const counts = new Map();
|
||||
return [...document.querySelectorAll(selector)].filter(visible).map((element, id) => {
|
||||
const text = (element.innerText || element.value || '').trim().replace(/\s+/g, ' ').slice(0, 120);
|
||||
const disabled = element.matches(':disabled') || Boolean(element.closest('[aria-disabled="true"]'));
|
||||
const state = [element.value || '', element.checked ?? '', element.getAttribute('aria-expanded') || '',
|
||||
element.getAttribute('aria-pressed') || '', element.getAttribute('data-state') || '', disabled].join(':');
|
||||
const path = identity(element);
|
||||
const base = [path, element.getAttribute('role') || '', element.getAttribute('aria-label') || '', text, element.getAttribute('href') || '', element.getAttribute('type') || '', state].join('|');
|
||||
const occurrence = counts.get(base) || 0;
|
||||
counts.set(base, occurrence + 1);
|
||||
element.setAttribute('data-kv-verify', String(id));
|
||||
return {
|
||||
id, key: `${base}|${occurrence}`, path, tag: element.tagName.toLowerCase(), text,
|
||||
aria: element.getAttribute('aria-label') || '', href: element.getAttribute('href') || '',
|
||||
type: element.getAttribute('type') || '', state, disabled,
|
||||
};
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
export function stateHash(url, actions) {
|
||||
return crypto.createHash('sha256').update(`${url}\n${actions.map((item) => item.key).join('\n')}`).digest('hex').slice(0, 20);
|
||||
}
|
||||
|
||||
async function findCurrent(page, action) {
|
||||
for (let attempt = 0; attempt < 6; attempt += 1) {
|
||||
const actions = await scanActions(page);
|
||||
const exact = actions.find((item) => item.key === action.key);
|
||||
if (exact) return { current: exact, matchedBy: 'key' };
|
||||
if (['input', 'textarea', 'select'].includes(action.tag)) {
|
||||
const fields = ['path', 'tag', 'aria', 'href', 'type', 'disabled'];
|
||||
const structural = actions.filter((item) => fields.every((field) => item[field] === action[field]));
|
||||
if (structural.length === 1) return { current: structural[0], matchedBy: 'structure' };
|
||||
}
|
||||
const semantic = actions.filter((item) => ['tag', 'aria', 'text', 'href', 'type', 'state'].every((field) => item[field] === action[field]));
|
||||
if (semantic.length === 1) return { current: semantic[0], matchedBy: 'semantic' };
|
||||
await prepareActionState(page, action);
|
||||
const viewport = page.viewportSize();
|
||||
if (viewport) {
|
||||
await page.mouse.move(1, 1);
|
||||
await page.mouse.move(Math.floor(viewport.width / 2), Math.floor(viewport.height / 2));
|
||||
}
|
||||
await page.waitForTimeout(200);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
export async function performAction(page, action, fixtureFile) {
|
||||
const match = await findCurrent(page, action);
|
||||
if (!match) return { ok: false, reason: 'action missing during replay' };
|
||||
const { current, matchedBy } = match;
|
||||
if (current.disabled) return { ok: true, skipped: true, matchedBy, reason: 'control is disabled in this state' };
|
||||
const locator = page.locator(`[data-kv-verify="${current.id}"]`);
|
||||
if (current.tag === 'input' && current.type === 'file') {
|
||||
await locator.setInputFiles(fixtureFile); return { ok: true, operation: 'setInputFiles', matchedBy };
|
||||
}
|
||||
if (current.tag === 'input' && ['checkbox', 'radio'].includes(current.type)) {
|
||||
return { ok: true, operation: await toggleInput(locator, current.type), matchedBy };
|
||||
}
|
||||
if (current.tag === 'input' && ['button', 'submit'].includes(current.type)) {
|
||||
await locator.click({ timeout: 5000 }); return { ok: true, operation: 'click', matchedBy };
|
||||
}
|
||||
if (current.tag === 'input' || current.tag === 'textarea') {
|
||||
await fillInput(locator, current.type); return { ok: true, operation: 'fill', matchedBy };
|
||||
}
|
||||
if (current.tag === 'select') {
|
||||
const values = await locator.locator('option').evaluateAll((options) => options.map((option) => option.value));
|
||||
if (!values.length) return { ok: true, skipped: true, matchedBy, reason: 'select has no options' };
|
||||
await locator.selectOption(values[Math.min(1, values.length - 1)]); return { ok: true, operation: 'select', matchedBy };
|
||||
}
|
||||
await locator.click({ timeout: 5000 });
|
||||
return { ok: true, operation: 'click', matchedBy };
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
|
||||
let axeSource;
|
||||
|
||||
function source(ctx) {
|
||||
if (!axeSource) axeSource = fs.readFileSync(path.join(ctx.config.verifyDir, 'node_modules', 'axe-core', 'axe.min.js'), 'utf8');
|
||||
return axeSource;
|
||||
}
|
||||
|
||||
export async function scanAxe(page, ctx) {
|
||||
await page.addScriptTag({ content: source(ctx) });
|
||||
return page.evaluate(async () => {
|
||||
const result = await window.axe.run(document, {
|
||||
resultTypes: ['violations', 'incomplete'],
|
||||
rules: { 'color-contrast': { enabled: true } },
|
||||
});
|
||||
const compact = (item) => ({
|
||||
id: item.id, impact: item.impact, description: item.description, help: item.help,
|
||||
helpUrl: item.helpUrl, nodes: item.nodes.map((node) => ({ target: node.target, failureSummary: node.failureSummary, html: node.html.slice(0, 500) })),
|
||||
});
|
||||
return { violations: result.violations.map(compact), incomplete: result.incomplete.map(compact) };
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
export function browserInit() {
|
||||
return ({ sourceConfig }) => {
|
||||
localStorage.clear();
|
||||
sessionStorage.clear();
|
||||
try { Object.defineProperty(document, 'startViewTransition', { value: undefined, configurable: true }); } catch {}
|
||||
const settings = {
|
||||
sources: [sourceConfig], premiumSources: [sourceConfig], subscriptions: [], sortBy: 'default', searchHistory: true,
|
||||
watchHistory: true, autoNextEpisode: true, autoSkipIntro: false, skipIntroSeconds: 0, autoSkipOutro: false,
|
||||
skipOutroSeconds: 0, seekStepSeconds: 10, showModeIndicator: true, adFilter: false, adFilterMode: 'off',
|
||||
adKeywords: [], realtimeLatency: true, searchDisplayMode: 'normal', episodeReverseOrder: false,
|
||||
fullscreenType: 'auto', proxyMode: 'none', rememberScrollPosition: false, personalizedRecommendations: false,
|
||||
videoTogetherEnabled: false, danmakuEnabled: false, danmakuApiUrl: '', danmakuOpacity: .7,
|
||||
danmakuFontSize: 20, danmakuDisplayArea: .5, locale: 'zh-CN', blockedCategories: [],
|
||||
};
|
||||
localStorage.setItem('kvideo-settings', JSON.stringify(settings));
|
||||
localStorage.setItem('theme', 'dark');
|
||||
const serviceWorker = { register: async () => ({ update: async () => {} }) };
|
||||
try { Object.defineProperty(navigator, 'serviceWorker', { value: serviceWorker, configurable: true }); } catch { /* browser restriction */ }
|
||||
window.__kvMetrics = { errors: [], rejections: [], longTasks: [], lcp: 0, cls: 0 };
|
||||
addEventListener('error', (event) => window.__kvMetrics.errors.push(String(event.error?.stack || event.message)));
|
||||
addEventListener('unhandledrejection', (event) => window.__kvMetrics.rejections.push(String(event.reason?.stack || event.reason)));
|
||||
try { new PerformanceObserver((list) => list.getEntries().forEach((entry) => window.__kvMetrics.longTasks.push(entry.duration))).observe({ type: 'longtask', buffered: true }); } catch {}
|
||||
try { new PerformanceObserver((list) => list.getEntries().forEach((entry) => { window.__kvMetrics.lcp = entry.startTime; })).observe({ type: 'largest-contentful-paint', buffered: true }); } catch {}
|
||||
try { new PerformanceObserver((list) => list.getEntries().forEach((entry) => { if (!entry.hadRecentInput) window.__kvMetrics.cls += entry.value; })).observe({ type: 'layout-shift', buffered: true }); } catch {}
|
||||
};
|
||||
}
|
||||
|
||||
export const sourceArgument = (fixtureUrl) => ({
|
||||
sourceConfig: { id: 'fixture', name: 'Fixture', baseUrl: fixtureUrl, searchPath: '/source', detailPath: '/source', enabled: true },
|
||||
});
|
||||
@@ -0,0 +1,28 @@
|
||||
export async function pageMetrics(page) {
|
||||
return page.evaluate(() => {
|
||||
const navigation = performance.getEntriesByType('navigation')[0];
|
||||
const resources = performance.getEntriesByType('resource');
|
||||
const metrics = window.__kvMetrics || {};
|
||||
return {
|
||||
url: location.href,
|
||||
title: document.title,
|
||||
scrollWidth: document.documentElement.scrollWidth,
|
||||
clientWidth: document.documentElement.clientWidth,
|
||||
scrollHeight: document.documentElement.scrollHeight,
|
||||
interactive: document.querySelectorAll('button,a[href],input,select,textarea,[role="button"],[data-focusable]').length,
|
||||
navigation: navigation ? {
|
||||
domContentLoaded: navigation.domContentLoadedEventEnd,
|
||||
load: navigation.loadEventEnd,
|
||||
response: navigation.responseEnd,
|
||||
transferSize: navigation.transferSize,
|
||||
} : null,
|
||||
lcp: metrics.lcp || 0,
|
||||
cls: metrics.cls || 0,
|
||||
longTasks: metrics.longTasks || [],
|
||||
errors: metrics.errors || [],
|
||||
rejections: metrics.rejections || [],
|
||||
resourceCount: resources.length,
|
||||
resourceBytes: resources.reduce((sum, item) => sum + (item.transferSize || 0), 0),
|
||||
};
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,63 @@
|
||||
function json(route, body, status = 200) {
|
||||
return route.fulfill({ status, contentType: 'application/json', body: JSON.stringify(body) });
|
||||
}
|
||||
|
||||
function searchStream(fixtureUrl) {
|
||||
const video = {
|
||||
vod_id: 'fixture-video-1', vod_name: '验证视频 1', vod_pic: `${fixtureUrl}/poster.svg?item=1`,
|
||||
vod_remarks: '全2集', vod_year: '2026', type_name: '测试', source: 'fixture', sourceDisplayName: 'Fixture', latency: 20,
|
||||
};
|
||||
return [
|
||||
{ type: 'start', totalSources: 1 },
|
||||
{ type: 'videos', videos: [video], source: 'fixture', completedSources: 1, totalSources: 1, latency: 20 },
|
||||
{ type: 'progress', completedSources: 1, totalSources: 1, totalVideosFound: 1 },
|
||||
{ type: 'complete', totalVideosFound: 1, totalSources: 1, maxPageCount: 1 },
|
||||
].map((item) => `data: ${JSON.stringify(item)}\n\n`).join('');
|
||||
}
|
||||
|
||||
export async function installMocks(page, ctx) {
|
||||
await page.route('**/*', async (route) => {
|
||||
const request = route.request();
|
||||
const url = new URL(request.url());
|
||||
const pathname = url.pathname;
|
||||
if (url.origin === ctx.config.fixtureUrl) {
|
||||
const response = await route.fetch();
|
||||
return route.fulfill({ response });
|
||||
}
|
||||
if (pathname === '/api/auth/session') return json(route, { authenticated: false, session: null });
|
||||
if (pathname === '/api/auth' && request.method() === 'GET') return json(route, {
|
||||
hasAuth: false, persistSession: true, loginMode: 'none', subscriptionSources: '', iptvSources: '', mergeSources: '',
|
||||
});
|
||||
if (pathname === '/api/config') return json(route, { subscriptionSources: '' });
|
||||
if (pathname === '/api/app-update') {
|
||||
const release = { version: ctx.state.version, publishedAt: '2026-07-31', title: 'Verification fixture', notes: ['Deterministic browser response'] };
|
||||
return json(route, {
|
||||
currentVersion: ctx.state.version, currentRelease: release, latestVersion: ctx.state.version, latestRelease: release,
|
||||
status: 'up-to-date', updateAvailable: false, checkedAt: new Date().toISOString(), checkedRemotely: true,
|
||||
usedRemoteManifest: true, source: { repository: 'KuekHaoYang/KVideo', branch: 'main',
|
||||
manifestUrl: 'https://raw.githubusercontent.com/KuekHaoYang/KVideo/main/app-release.json',
|
||||
changelogUrl: 'https://github.com/KuekHaoYang/KVideo/blob/main/CHANGELOG.md', repositoryUrl: 'https://github.com/KuekHaoYang/KVideo' },
|
||||
});
|
||||
}
|
||||
if (pathname === '/api/search-parallel') return route.fulfill({ status: 200, contentType: 'text/event-stream', body: searchStream(ctx.config.fixtureUrl) });
|
||||
if (pathname === '/api/detail') return json(route, { success: true, data: {
|
||||
vod_id: 'fixture-video-1', vod_name: '验证视频 1', vod_pic: `${ctx.config.fixtureUrl}/poster.svg?item=1`,
|
||||
vod_content: 'Deterministic browser fixture', vod_year: '2026', type_name: '测试',
|
||||
episodes: [{ name: '第1集', url: `${ctx.config.fixtureUrl}/test.mp4` }, { name: '第2集', url: `${ctx.config.fixtureUrl}/hls/master.m3u8` }],
|
||||
} });
|
||||
if (pathname === '/api/ping') return json(route, { latency: 20, success: true, timeout: false, method: 'HEAD' });
|
||||
if (pathname === '/api/probe-resolution') return json(route, { width: 640, height: 360, label: '360p' });
|
||||
if (pathname.startsWith('/api/user/')) return json(route, { history: [], favorites: [], config: null, success: true });
|
||||
if (pathname === '/api/premium/category') return json(route, { videos: [] });
|
||||
if (pathname === '/api/premium/types') return json(route, { tags: [{ id: 'recommend', label: '今日推荐', value: '' }] });
|
||||
if (pathname === '/api/danmaku') return json(route, []);
|
||||
if (pathname.startsWith('/api/douban/')) return json(route, { tags: [], subjects: [] });
|
||||
if (pathname.startsWith('/api/') && ['POST', 'PUT', 'PATCH', 'DELETE'].includes(request.method())) {
|
||||
return json(route, { error: 'verification mutation blocked' }, 403);
|
||||
}
|
||||
if (['www.gstatic.com', 'fastly.jsdelivr.net'].includes(url.hostname)) {
|
||||
return route.fulfill({ status: 200, contentType: 'application/javascript', body: '' });
|
||||
}
|
||||
return route.continue();
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
import path from 'node:path';
|
||||
import { relative, walk } from '../core/files.mjs';
|
||||
|
||||
function routeFromFile(root, file) {
|
||||
const part = relative(path.join(root, 'app'), file).replace(/(^|\/)page\.tsx$/, '');
|
||||
const route = `/${part}`.replace(/\/\([^/]+\)/g, '').replace(/\/+/g, '/');
|
||||
return route === '/.' ? '/' : route;
|
||||
}
|
||||
|
||||
export function discoverPages(ctx) {
|
||||
const files = walk(path.join(ctx.config.root, 'app'), (file) => file.endsWith('/page.tsx'));
|
||||
const routes = files.map((file) => routeFromFile(ctx.config.root, file)).filter((route) => !route.includes('['));
|
||||
return [...new Set(routes)].map((route) => {
|
||||
if (route === '/player') return '/player?id=fixture-video-1&source=fixture&title=%E9%AA%8C%E8%AF%81%E8%A7%86%E9%A2%91&episode=0';
|
||||
return route;
|
||||
}).sort();
|
||||
}
|
||||
@@ -0,0 +1,54 @@
|
||||
import { chromium } from 'playwright';
|
||||
import { browserInit, sourceArgument } from './init.mjs';
|
||||
import { installMocks } from './mocks.mjs';
|
||||
|
||||
export async function launchBrowser(ctx) {
|
||||
const browser = await chromium.launch({
|
||||
executablePath: ctx.state.chromePath,
|
||||
headless: true,
|
||||
args: ['--autoplay-policy=no-user-gesture-required', '--disable-background-timer-throttling'],
|
||||
});
|
||||
ctx.services.push({ name: 'playwright-browser', close: () => browser.close() });
|
||||
return browser;
|
||||
}
|
||||
|
||||
export async function newPage(browser, ctx, viewport) {
|
||||
const context = await browser.newContext({ viewport, colorScheme: 'dark', locale: 'zh-CN', reducedMotion: 'reduce' });
|
||||
await context.addInitScript(browserInit(), sourceArgument(ctx.config.fixtureUrl));
|
||||
const page = await context.newPage();
|
||||
const observed = { consoleErrors: [], consoleWarnings: [], pageErrors: [], failedRequests: [], httpErrors: [], dialogs: [], downloads: [], popups: [] };
|
||||
page.on('console', (message) => {
|
||||
if (message.type() === 'error') observed.consoleErrors.push(message.text());
|
||||
if (message.type() === 'warning') observed.consoleWarnings.push(message.text());
|
||||
});
|
||||
page.on('pageerror', (error) => observed.pageErrors.push(error.stack || error.message));
|
||||
page.on('requestfailed', (request) => observed.failedRequests.push({ url: request.url(), error: request.failure()?.errorText }));
|
||||
page.on('response', (response) => {
|
||||
if (response.status() >= 400) observed.httpErrors.push({ method: response.request().method(), status: response.status(), url: response.url() });
|
||||
});
|
||||
page.on('dialog', async (dialog) => {
|
||||
observed.dialogs.push({ type: dialog.type(), message: dialog.message(), defaultValue: dialog.defaultValue() });
|
||||
await dialog.dismiss().catch(() => {});
|
||||
});
|
||||
page.on('download', (download) => observed.downloads.push({ filename: download.suggestedFilename(), url: download.url() }));
|
||||
page.on('popup', (popup) => {
|
||||
observed.popups.push({ url: popup.url() });
|
||||
popup.close().catch(() => {});
|
||||
});
|
||||
await installMocks(page, ctx);
|
||||
return { context, page, observed };
|
||||
}
|
||||
|
||||
export async function stabilize(page) {
|
||||
const css = '*,*::before,*::after{animation-duration:0s!important;transition-duration:0s!important;caret-color:transparent!important}::view-transition-old(root),::view-transition-new(root){animation:none!important}';
|
||||
await page.addStyleTag({ content: css });
|
||||
await page.evaluate(async () => {
|
||||
await document.fonts?.ready;
|
||||
const images = Promise.all([...document.images].map((image) => image.complete ? null : new Promise((resolve) => {
|
||||
image.addEventListener('load', resolve, { once: true });
|
||||
image.addEventListener('error', resolve, { once: true });
|
||||
})));
|
||||
await Promise.race([images, new Promise((resolve) => setTimeout(resolve, 1000))]);
|
||||
});
|
||||
await page.waitForTimeout(250);
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
import path from 'node:path';
|
||||
import { finding } from '../core/finding.mjs';
|
||||
import { jsonBody, request } from '../core/http.mjs';
|
||||
import { writeJson } from '../core/files.mjs';
|
||||
|
||||
export async function checkApiContracts(ctx) {
|
||||
if (!ctx.state.appReady) return;
|
||||
const source = { id: 'fixture', name: 'Fixture', baseUrl: ctx.config.fixtureUrl, searchPath: '/source', detailPath: '/source', enabled: true };
|
||||
const cases = [
|
||||
['config', '/api/config', { method: 'GET' }, [200]],
|
||||
['app-update', '/api/app-update', { method: 'GET' }, [200]],
|
||||
['detail-missing', '/api/detail', { method: 'GET' }, [400]],
|
||||
['detail-fixture', '/api/detail', { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ id: 'fixture-video-1', source }) }, [200]],
|
||||
['search-invalid', '/api/search-parallel', { method: 'POST', headers: { 'content-type': 'application/json' }, body: '{}' }, [200]],
|
||||
['search-fixture', '/api/search-parallel', { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ query: '验证视频', sources: [source] }) }, [200]],
|
||||
['ping-invalid', '/api/ping', { method: 'POST', headers: { 'content-type': 'application/json' }, body: '{}' }, [400]],
|
||||
];
|
||||
const results = [];
|
||||
for (const [name, route, options, expected] of cases) {
|
||||
const response = await request(`${ctx.config.localUrl}${route}`, options);
|
||||
results.push({ name, route, expected, response, parsed: jsonBody(response) });
|
||||
}
|
||||
const target = path.join(ctx.dirs.raw, 'api-contracts.json');
|
||||
writeJson(target, results);
|
||||
const failed = results.filter((item) => !item.expected.includes(item.response.status));
|
||||
const search = results.find((item) => item.name === 'search-fixture');
|
||||
const streamOk = search?.response.body.includes('"type":"videos"') && search.response.body.includes('"type":"complete"');
|
||||
const detail = results.find((item) => item.name === 'detail-fixture');
|
||||
const detailOk = detail?.parsed?.success && detail.parsed?.data?.episodes?.length === 2;
|
||||
finding(ctx, {
|
||||
id: 'api.contract-status', category: 'api', title: 'Core API status contracts match expectations',
|
||||
status: failed.length ? 'FAIL' : 'PASS', severity: 'critical', expected: 'Every core contract returns its declared status',
|
||||
actual: failed.length ? JSON.stringify(failed.map((item) => ({ name: item.name, status: item.response.status, expected: item.expected }))) : `${results.length} cases matched`,
|
||||
reason: failed.length ? 'A core endpoint changed or failed its response contract.' : 'Core status-code contracts are stable.', evidence: [target],
|
||||
remediation: 'Repair the endpoint or intentionally update the declared contract and consumers.',
|
||||
});
|
||||
finding(ctx, {
|
||||
id: 'api.search-stream', category: 'api', title: 'Streaming search emits video and completion events',
|
||||
status: streamOk ? 'PASS' : 'FAIL', severity: 'critical', expected: 'SSE videos event followed by complete', actual: search?.response.body || 'missing',
|
||||
reason: streamOk ? 'The deterministic source traversed the full search stream.' : 'The stream omitted results or completion.', evidence: [target],
|
||||
remediation: 'Inspect streaming serialization, source parsing, and completion handling.',
|
||||
});
|
||||
finding(ctx, {
|
||||
id: 'api.detail-fixture', category: 'api', title: 'Detail parsing returns both MP4 and HLS episodes',
|
||||
status: detailOk ? 'PASS' : 'FAIL', severity: 'critical', expected: 'success=true with 2 episodes', actual: JSON.stringify(detail?.parsed),
|
||||
reason: detailOk ? 'The real detail route parsed deterministic upstream data.' : 'The detail route or episode parser lost fixture data.', evidence: [target],
|
||||
remediation: 'Fix source lookup, upstream parsing, or episode normalization.',
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,55 @@
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { finding } from '../core/finding.mjs';
|
||||
import { relative, walk, writeJson } from '../core/files.mjs';
|
||||
import { request } from '../core/http.mjs';
|
||||
|
||||
const methodPattern = /export\s+(?:async\s+)?function\s+(GET|POST|PUT|PATCH|DELETE|OPTIONS|HEAD)\b/g;
|
||||
|
||||
function routePath(root, file) {
|
||||
return `/${relative(root, file).replace(/^app\//, '').replace(/\/route\.ts$/, '').replace(/\[([^\]]+)\]/g, 'verification-$1')}`;
|
||||
}
|
||||
|
||||
export async function checkApiDiscovery(ctx) {
|
||||
if (!ctx.state.appReady) return finding(ctx, {
|
||||
id: 'api.route-coverage', category: 'api', title: 'Every API method is exercised', status: 'SKIP', severity: 'critical',
|
||||
expected: 'Local server ready', actual: 'server unavailable', reason: 'API calls cannot execute.', remediation: 'Fix local startup.',
|
||||
});
|
||||
const files = walk(path.join(ctx.config.root, 'app', 'api'), (file) => file.endsWith('/route.ts'));
|
||||
const inventory = files.map((file) => ({
|
||||
file: relative(ctx.config.root, file),
|
||||
path: routePath(ctx.config.root, file),
|
||||
methods: [...fs.readFileSync(file, 'utf8').matchAll(methodPattern)].map((match) => match[1]),
|
||||
}));
|
||||
const results = [];
|
||||
for (const route of inventory) {
|
||||
for (const method of route.methods) {
|
||||
const options = { method, timeoutMs: 12_000, headers: {} };
|
||||
if (['POST', 'PUT', 'PATCH'].includes(method)) {
|
||||
options.headers['content-type'] = 'application/json';
|
||||
options.body = '{}';
|
||||
}
|
||||
const response = await request(`${ctx.config.localUrl}${route.path}`, options);
|
||||
results.push({ ...route, methods: undefined, method, response });
|
||||
}
|
||||
}
|
||||
const target = path.join(ctx.dirs.raw, 'api-route-matrix.json');
|
||||
writeJson(target, { inventory, results });
|
||||
const unexercised = inventory.flatMap((route) => route.methods.map((method) => `${method} ${route.path}`))
|
||||
.filter((key) => !results.some((item) => `${item.method} ${item.path}` === key));
|
||||
const crashes = results.filter((item) => !item.response.ok || item.response.status >= 500);
|
||||
finding(ctx, {
|
||||
id: 'api.route-coverage', category: 'api', title: 'Every statically exported API method receives a smoke request',
|
||||
status: unexercised.length ? 'FAIL' : 'PASS', severity: 'critical', expected: '100% exported method invocation',
|
||||
actual: `${results.length} methods invoked; ${unexercised.length} missing`, reason: unexercised.length ? 'Some exported API methods were not reached.' : 'Every discovered method was invoked with a safe anonymous payload.',
|
||||
evidence: [target], remediation: 'Add a safe contract case for every missing method.',
|
||||
});
|
||||
finding(ctx, {
|
||||
id: 'api.anonymous-crashes', category: 'api', title: 'Anonymous malformed requests do not crash API routes',
|
||||
status: crashes.length ? 'FAIL' : 'PASS', severity: 'high', expected: 'No network error or HTTP 5xx for empty safe probes',
|
||||
actual: crashes.length ? JSON.stringify(crashes.map((item) => ({ method: item.method, path: item.path, status: item.response.status, error: item.response.error }))) : 'No crashes',
|
||||
reason: crashes.length ? 'Malformed or anonymous input reaches an internal failure instead of a controlled 4xx response.' : 'All routes rejected or handled generic probes without server failure.',
|
||||
evidence: [target], remediation: 'Validate request inputs and convert expected missing configuration/auth states to explicit 4xx responses.',
|
||||
});
|
||||
ctx.state.apiInventory = inventory;
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { parse } from '@typescript-eslint/typescript-estree';
|
||||
import { finding } from '../core/finding.mjs';
|
||||
import { relative, walk, writeJson } from '../core/files.mjs';
|
||||
import { collectFunctions } from './ast-walk.mjs';
|
||||
|
||||
const extensions = new Set(['.ts', '.tsx']);
|
||||
|
||||
export async function checkAstMetrics(ctx) {
|
||||
const files = walk(ctx.config.root, (file) => extensions.has(path.extname(file)) && !file.includes('/verification/'));
|
||||
const metrics = [];
|
||||
const parseErrors = [];
|
||||
for (const file of files) {
|
||||
try {
|
||||
const ast = parse(fs.readFileSync(file, 'utf8'), { loc: true, jsx: file.endsWith('.tsx'), errorOnUnknownASTType: false });
|
||||
const functions = collectFunctions(ast);
|
||||
metrics.push({ file: relative(ctx.config.root, file), functions });
|
||||
} catch (error) {
|
||||
parseErrors.push({ file: relative(ctx.config.root, file), error: error instanceof Error ? error.message : String(error) });
|
||||
}
|
||||
}
|
||||
const offenders = metrics.flatMap((item) => item.functions.map((fn) => ({ file: item.file, ...fn })))
|
||||
.filter((fn) => fn.lines > 80 || fn.complexity > 15 || fn.maxNesting > 4 || fn.params > 5)
|
||||
.sort((a, b) => (b.complexity + b.lines / 10) - (a.complexity + a.lines / 10));
|
||||
const target = path.join(ctx.dirs.metrics, 'ast-metrics.json');
|
||||
writeJson(target, { parseErrors, offenders, files: metrics });
|
||||
finding(ctx, {
|
||||
id: 'quality.ast-parse', category: 'quality', title: 'TypeScript source is structurally analyzable',
|
||||
status: parseErrors.length ? 'FAIL' : 'PASS', severity: 'high', expected: 'All TS/TSX files parse',
|
||||
actual: parseErrors.length ? JSON.stringify(parseErrors.slice(0, 20)) : `${files.length} files parsed`,
|
||||
reason: parseErrors.length ? 'Unparseable files invalidate structural quality metrics.' : 'AST metrics cover all TS/TSX files.',
|
||||
evidence: [target], remediation: 'Fix syntax/parser incompatibilities before relying on complexity results.',
|
||||
});
|
||||
finding(ctx, {
|
||||
id: 'quality.spaghetti-risk', category: 'quality', title: 'Functions stay within complexity and cohesion limits',
|
||||
status: offenders.length ? 'FAIL' : 'PASS', severity: 'high', expected: 'lines <=80, complexity <=15, nesting <=4, params <=5',
|
||||
actual: offenders.length ? `${offenders.length} risky functions; worst: ${JSON.stringify(offenders.slice(0, 12))}` : 'No threshold breaches',
|
||||
reason: offenders.length ? 'Long, branch-heavy, deeply nested functions are concrete spaghetti-code indicators.' : 'No configured structural risk threshold was exceeded.',
|
||||
evidence: [target], remediation: 'Extract cohesive functions and replace nested conditionals with explicit domain operations.',
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,59 @@
|
||||
const functionTypes = new Set([
|
||||
'FunctionDeclaration', 'FunctionExpression', 'ArrowFunctionExpression',
|
||||
'TSDeclareFunction', 'TSFunctionType', 'ObjectMethod', 'ClassMethod',
|
||||
]);
|
||||
const branchTypes = new Set([
|
||||
'IfStatement', 'ForStatement', 'ForInStatement', 'ForOfStatement',
|
||||
'WhileStatement', 'DoWhileStatement', 'CatchClause', 'ConditionalExpression',
|
||||
]);
|
||||
const nestTypes = new Set([...branchTypes, 'SwitchStatement', 'TryStatement']);
|
||||
|
||||
export function children(node) {
|
||||
const output = [];
|
||||
for (const [key, value] of Object.entries(node || {})) {
|
||||
if (key === 'parent' || key === 'tokens' || key === 'comments') continue;
|
||||
if (Array.isArray(value)) output.push(...value.filter((item) => item?.type));
|
||||
else if (value?.type) output.push(value);
|
||||
}
|
||||
return output;
|
||||
}
|
||||
|
||||
function displayName(node, parent) {
|
||||
if (node.id?.name) return node.id.name;
|
||||
if (parent?.id?.name) return parent.id.name;
|
||||
if (parent?.key?.name) return parent.key.name;
|
||||
return '<anonymous>';
|
||||
}
|
||||
|
||||
function functionMetric(node, parent) {
|
||||
let complexity = 1;
|
||||
let maxNesting = 0;
|
||||
const visit = (current, depth) => {
|
||||
if (current !== node && functionTypes.has(current.type)) return;
|
||||
if (branchTypes.has(current.type)) complexity += 1;
|
||||
if (current.type === 'LogicalExpression' && ['&&', '||', '??'].includes(current.operator)) complexity += 1;
|
||||
if (current.type === 'SwitchCase' && current.test) complexity += 1;
|
||||
const nextDepth = nestTypes.has(current.type) ? depth + 1 : depth;
|
||||
maxNesting = Math.max(maxNesting, nextDepth);
|
||||
for (const child of children(current)) visit(child, nextDepth);
|
||||
};
|
||||
visit(node, 0);
|
||||
return {
|
||||
name: displayName(node, parent),
|
||||
line: node.loc?.start.line || 0,
|
||||
lines: (node.loc?.end.line || 0) - (node.loc?.start.line || 0) + 1,
|
||||
params: node.params?.length || 0,
|
||||
complexity,
|
||||
maxNesting,
|
||||
};
|
||||
}
|
||||
|
||||
export function collectFunctions(ast) {
|
||||
const output = [];
|
||||
const visit = (node, parent) => {
|
||||
if (functionTypes.has(node.type)) output.push(functionMetric(node, parent));
|
||||
for (const child of children(node)) visit(child, node);
|
||||
};
|
||||
visit(ast, null);
|
||||
return output;
|
||||
}
|
||||
@@ -0,0 +1,39 @@
|
||||
import path from 'node:path';
|
||||
import { runCommand } from '../core/command.mjs';
|
||||
import { finding } from '../core/finding.mjs';
|
||||
import { jsonBody, request } from '../core/http.mjs';
|
||||
import { writeJson } from '../core/files.mjs';
|
||||
|
||||
function digest(output) {
|
||||
return output?.match(/^Digest:\s+(sha256:[a-f0-9]+)/m)?.[1] || null;
|
||||
}
|
||||
|
||||
export async function checkDeployment(ctx) {
|
||||
if (ctx.config.offline) return finding(ctx, {
|
||||
id: 'deploy.consistency', category: 'deployment', title: 'Local, GitHub, Cloudflare, and Docker release consistency', status: 'SKIP', severity: 'critical',
|
||||
expected: 'Online verification', actual: '--offline', reason: 'Remote state cannot be verified offline.', remediation: 'Rerun online before release.',
|
||||
});
|
||||
const localSha = (await runCommand(ctx, 'git-head', 'git', ['rev-parse', 'HEAD'], { timeoutMs: 30_000 })).tail.trim();
|
||||
const remoteShaResult = await runCommand(ctx, 'git-remote-main', 'git', ['ls-remote', 'origin', 'refs/heads/main'], { timeoutMs: 30_000 });
|
||||
const remoteSha = remoteShaResult.tail.trim().split(/\s+/)[0] || null;
|
||||
const githubPackage = await request('https://raw.githubusercontent.com/KuekHaoYang/KVideo/main/package.json');
|
||||
const cloudflare = await request(`${ctx.config.referenceUrl}/api/app-update`);
|
||||
const latest = await runCommand(ctx, 'dockerhub-latest', 'docker', ['buildx', 'imagetools', 'inspect', 'kuekhaoyang/kvideo:latest'], { timeoutMs: 120_000 });
|
||||
const versioned = await runCommand(ctx, 'dockerhub-version', 'docker', ['buildx', 'imagetools', 'inspect', `kuekhaoyang/kvideo:${ctx.state.version}`], { timeoutMs: 120_000 });
|
||||
const wrangler = path.join(ctx.config.verifyDir, 'node_modules', '.bin', 'wrangler');
|
||||
const pages = await runCommand(ctx, 'cloudflare-deployments', wrangler, ['pages', 'deployment', 'list', '--project-name', 'kvideo'], { timeoutMs: 120_000 });
|
||||
const githubVersion = jsonBody(githubPackage)?.version;
|
||||
const cloudVersion = jsonBody(cloudflare)?.currentVersion;
|
||||
const latestDigest = digest(latest.tail);
|
||||
const versionDigest = digest(versioned.tail);
|
||||
const facts = { localSha, remoteSha, localVersion: ctx.state.version, githubVersion, cloudVersion, latestDigest, versionDigest, pagesMentionsSha: pages.tail.includes(localSha.slice(0, 7)) };
|
||||
const target = path.join(ctx.dirs.raw, 'deployment-consistency.json');
|
||||
writeJson(target, { facts, githubPackage, cloudflare, evidence: { latest: latest.outputPath, versioned: versioned.outputPath, pages: pages.outputPath } });
|
||||
const ok = localSha === remoteSha && githubVersion === ctx.state.version && cloudVersion === ctx.state.version && latestDigest && latestDigest === versionDigest;
|
||||
finding(ctx, {
|
||||
id: 'deploy.consistency', category: 'deployment', title: 'Local, GitHub main, Cloudflare, and both Docker tags agree',
|
||||
status: ok ? 'PASS' : 'FAIL', severity: 'critical', expected: 'Same Git commit/version; Docker latest and version tags share one digest', actual: JSON.stringify(facts),
|
||||
reason: ok ? 'Every public release surface resolves to the declared release.' : 'At least one release surface is stale, missing, or points at a different artifact.',
|
||||
evidence: [target, latest.outputPath, versioned.outputPath, pages.outputPath], remediation: 'Merge/push main, wait for Pages and Docker workflows, then verify version and digest convergence.',
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,43 @@
|
||||
import path from 'node:path';
|
||||
import { runCommand } from '../core/command.mjs';
|
||||
import { finding } from '../core/finding.mjs';
|
||||
import { jsonBody, request } from '../core/http.mjs';
|
||||
import { waitForUrl } from '../core/service.mjs';
|
||||
import { writeJson } from '../core/files.mjs';
|
||||
|
||||
export async function checkDockerLocal(ctx) {
|
||||
if (ctx.config.quick) return finding(ctx, {
|
||||
id: 'docker.local-image', category: 'docker', title: 'Local Docker image build and runtime smoke', status: 'SKIP', severity: 'critical',
|
||||
expected: 'Full run', actual: '--quick', reason: 'Quick mode omits the expensive image build.', remediation: 'Run ./verification/run without --quick before release.',
|
||||
});
|
||||
const image = `kvideo-verification:${ctx.state.version}`;
|
||||
const name = `kvideo-verification-${Date.now()}`;
|
||||
const build = await runCommand(ctx, 'docker-build', 'docker', ['build', '--pull', '--tag', image, '.'], { timeoutMs: 1_800_000 });
|
||||
if (build.code !== 0) return finding(ctx, {
|
||||
id: 'docker.local-image', category: 'docker', title: 'Local Docker image builds and runs', status: 'FAIL', severity: 'critical',
|
||||
expected: 'docker build exit 0', actual: `exit ${build.code}`, reason: 'The release container cannot be produced from this checkout.',
|
||||
evidence: [build.outputPath], remediation: 'Fix Dockerfile, dependency installation, or standalone build output.', durationMs: build.durationMs,
|
||||
});
|
||||
const run = await runCommand(ctx, 'docker-run', 'docker', ['run', '--detach', '--rm', '--name', name, '--publish', `127.0.0.1:${ctx.config.containerPort}:3000`, image], { timeoutMs: 60_000 });
|
||||
let ready = { ok: false, error: 'container did not start' };
|
||||
let response = null;
|
||||
let inspect = null;
|
||||
if (run.code === 0) {
|
||||
ready = await waitForUrl(ctx.config.containerUrl, 120_000);
|
||||
response = ready.ok ? await request(`${ctx.config.containerUrl}/api/app-update`) : null;
|
||||
inspect = await runCommand(ctx, 'docker-inspect-local', 'docker', ['image', 'inspect', image], { timeoutMs: 30_000 });
|
||||
}
|
||||
const logs = await runCommand(ctx, 'docker-container-logs', 'docker', ['logs', name], { timeoutMs: 30_000 });
|
||||
await runCommand(ctx, 'docker-stop', 'docker', ['stop', name], { timeoutMs: 60_000 });
|
||||
const parsed = response ? jsonBody(response) : null;
|
||||
const ok = run.code === 0 && ready.ok && response?.status === 200 && parsed?.currentVersion === ctx.state.version;
|
||||
const target = path.join(ctx.dirs.raw, 'docker-local.json');
|
||||
writeJson(target, { image, name, build, run, ready, response, parsed, inspect: inspect?.outputPath, logs: logs.outputPath });
|
||||
finding(ctx, {
|
||||
id: 'docker.local-image', category: 'docker', title: 'Local Docker image builds, starts, and reports the expected version',
|
||||
status: ok ? 'PASS' : 'FAIL', severity: 'critical', expected: `HTTP 200 and version ${ctx.state.version}`, actual: JSON.stringify({ run: run.code, ready, status: response?.status, version: parsed?.currentVersion }),
|
||||
reason: ok ? 'The real standalone container passed a runtime smoke test.' : 'The container failed to start, respond, or expose the expected release version.',
|
||||
evidence: [build.outputPath, run.outputPath, logs.outputPath, target], remediation: 'Inspect build and container logs, then correct the Docker release path.',
|
||||
durationMs: build.durationMs + run.durationMs,
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { runCommand } from '../core/command.mjs';
|
||||
import { finding } from '../core/finding.mjs';
|
||||
|
||||
function findReport(dir) {
|
||||
if (!fs.existsSync(dir)) return null;
|
||||
return fs.readdirSync(dir).map((name) => path.join(dir, name)).find((file) => file.endsWith('.json')) || null;
|
||||
}
|
||||
|
||||
export async function checkDuplicates(ctx) {
|
||||
const output = path.join(ctx.dirs.metrics, 'jscpd');
|
||||
const bin = path.join(ctx.config.verifyDir, 'node_modules', '.bin', 'jscpd');
|
||||
const args = ['--min-lines', '8', '--min-tokens', '60', '--reporters', 'json', '--output', output, 'app', 'components', 'lib'];
|
||||
const result = await runCommand(ctx, 'jscpd', bin, args, { cwd: ctx.config.root });
|
||||
const report = findReport(output);
|
||||
let data = null;
|
||||
try { data = report ? JSON.parse(fs.readFileSync(report, 'utf8')) : null; } catch { /* report parse failure */ }
|
||||
const percentage = data?.statistics?.total?.percentage ?? data?.statistics?.total?.percentageTokens ?? null;
|
||||
const clones = data?.duplicates?.length ?? null;
|
||||
const ok = result.code === 0 && data && Number(percentage || 0) <= 5;
|
||||
finding(ctx, {
|
||||
id: 'quality.duplication', category: 'quality', title: 'Copy-paste duplication stays below threshold',
|
||||
status: ok ? 'PASS' : 'FAIL', severity: 'medium', expected: 'jscpd completes and duplicated lines <= 5%',
|
||||
actual: data ? `${percentage}% duplication; ${clones ?? 'unknown'} clone groups` : `jscpd exit ${result.code}; no report`,
|
||||
reason: ok ? 'Token-based clone analysis is below the configured limit.' : 'High duplication or a failed clone scan hides inconsistent parallel implementations.',
|
||||
evidence: [result.outputPath, ...(report ? [report] : [])], remediation: 'Extract shared domain logic and remove copied branches with divergent behavior.',
|
||||
durationMs: result.durationMs,
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { runCommand } from '../core/command.mjs';
|
||||
import { finding } from '../core/finding.mjs';
|
||||
|
||||
export async function checkHarnessSelf(ctx) {
|
||||
const testDir = path.join(ctx.config.verifyDir, 'tests');
|
||||
const tests = fs.readdirSync(testDir).filter((name) => name.endsWith('.test.mjs')).map((name) => path.join(testDir, name));
|
||||
const result = await runCommand(ctx, 'verification-self-tests', 'node', ['--test', ...tests], { cwd: ctx.config.root, timeoutMs: 60_000 });
|
||||
finding(ctx, {
|
||||
id: 'harness.self-tests', category: 'harness', title: 'Verification framework self-tests pass',
|
||||
status: result.code === 0 ? 'PASS' : 'FAIL', severity: 'critical', expected: 'node --test exit 0', actual: `exit ${result.code}`,
|
||||
reason: result.code === 0 ? 'Core redaction, HTTP parsing, graph helpers, and report escaping are verified.' : 'The validation framework failed its own tests.',
|
||||
evidence: [result.outputPath], remediation: 'Fix the harness before treating any project result as authoritative.', durationMs: result.durationMs,
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,78 @@
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { finding } from '../core/finding.mjs';
|
||||
import { lineCount, relative, walk, writeJson } from '../core/files.mjs';
|
||||
|
||||
const ext = ['.ts', '.tsx', '.js', '.jsx', '.mjs'];
|
||||
const importPattern = /(?:import|export)\s+(?:[^'";]+?\s+from\s+)?['"]([^'"]+)['"]|import\(\s*['"]([^'"]+)['"]\s*\)/g;
|
||||
|
||||
function resolveImport(root, from, specifier) {
|
||||
if (!specifier.startsWith('.') && !specifier.startsWith('@/')) return null;
|
||||
const base = specifier.startsWith('@/') ? path.join(root, specifier.slice(2)) : path.resolve(path.dirname(from), specifier);
|
||||
const candidates = [base, ...ext.map((item) => base + item), ...ext.map((item) => path.join(base, `index${item}`))];
|
||||
return candidates.find((item) => fs.existsSync(item) && fs.statSync(item).isFile()) || null;
|
||||
}
|
||||
|
||||
function findCycles(graph) {
|
||||
const cycles = new Set();
|
||||
const active = [];
|
||||
const done = new Set();
|
||||
const visit = (node) => {
|
||||
const index = active.indexOf(node);
|
||||
if (index >= 0) { cycles.add(active.slice(index).concat(node).join(' -> ')); return; }
|
||||
if (done.has(node)) return;
|
||||
active.push(node);
|
||||
for (const child of graph.get(node) || []) visit(child);
|
||||
active.pop();
|
||||
done.add(node);
|
||||
};
|
||||
for (const node of graph.keys()) visit(node);
|
||||
return [...cycles];
|
||||
}
|
||||
|
||||
function closure(graph, start) {
|
||||
const seen = new Set();
|
||||
const visit = (node) => {
|
||||
if (seen.has(node)) return;
|
||||
seen.add(node);
|
||||
for (const child of graph.get(node) || []) visit(child);
|
||||
};
|
||||
visit(start);
|
||||
return seen;
|
||||
}
|
||||
|
||||
export async function checkImportGraph(ctx) {
|
||||
const root = ctx.config.root;
|
||||
const files = walk(root, (file) => ext.includes(path.extname(file)) && !file.includes('/verification/'));
|
||||
const graph = new Map(files.map((file) => [relative(root, file), []]));
|
||||
const unresolved = [];
|
||||
for (const file of files) {
|
||||
const source = fs.readFileSync(file, 'utf8');
|
||||
for (const match of source.matchAll(importPattern)) {
|
||||
const specifier = match[1] || match[2];
|
||||
const resolved = resolveImport(root, file, specifier);
|
||||
if (resolved) graph.get(relative(root, file)).push(relative(root, resolved));
|
||||
else if (specifier.startsWith('.') || specifier.startsWith('@/')) unresolved.push({ file: relative(root, file), specifier });
|
||||
}
|
||||
}
|
||||
const cycles = findCycles(graph);
|
||||
const pages = [...graph.keys()].filter((file) => /(^|\/)app\/.*page\.tsx$/.test(file));
|
||||
const features = pages.map((entry) => {
|
||||
const reached = closure(graph, entry);
|
||||
return { entry, files: reached.size, lines: [...reached].reduce((sum, file) => sum + lineCount(path.join(root, file)), 0) };
|
||||
}).sort((a, b) => b.lines - a.lines);
|
||||
const target = path.join(ctx.dirs.metrics, 'import-graph.json');
|
||||
writeJson(target, { cycles, unresolved, features, graph: Object.fromEntries(graph) });
|
||||
finding(ctx, {
|
||||
id: 'quality.import-cycles', category: 'quality', title: 'Internal dependency graph has no cycles',
|
||||
status: cycles.length ? 'FAIL' : 'PASS', severity: 'high', expected: '0 dependency cycles', actual: cycles.length ? cycles.slice(0, 20).join('\n') : '0',
|
||||
reason: cycles.length ? 'Cycles create order-dependent initialization and make feature boundaries unreliable.' : 'No static import cycles were found.',
|
||||
evidence: [target], remediation: 'Extract shared contracts or invert dependencies to break each cycle.',
|
||||
});
|
||||
finding(ctx, {
|
||||
id: 'quality.feature-footprint', category: 'quality', title: 'Feature transitive code footprint is quantified',
|
||||
status: features.some((item) => item.lines > 10_000) ? 'WARN' : 'PASS', severity: 'medium', expected: 'No page transitively owns more than 10,000 lines',
|
||||
actual: JSON.stringify(features), reason: 'Transitive page footprints expose features that accumulate excessive code through dependencies.',
|
||||
evidence: [target], remediation: 'Split oversized feature graphs into explicit bounded modules and lazy boundaries.',
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
import path from 'node:path';
|
||||
import { finding } from '../core/finding.mjs';
|
||||
import { jsonBody, request } from '../core/http.mjs';
|
||||
import { writeJson } from '../core/files.mjs';
|
||||
|
||||
async function ping(ctx, url) {
|
||||
const response = await request(`${ctx.config.localUrl}/api/ping`, {
|
||||
method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ url }), timeoutMs: 10_000,
|
||||
});
|
||||
return { response, parsed: jsonBody(response) };
|
||||
}
|
||||
|
||||
export async function checkLatency(ctx) {
|
||||
if (!ctx.state.appReady) return;
|
||||
const fast = await ping(ctx, `${ctx.config.fixtureUrl}/fast`);
|
||||
const slow = await ping(ctx, `${ctx.config.fixtureUrl}/slow?ms=300`);
|
||||
const burst = await Promise.all(Array.from({ length: 12 }, () => ping(ctx, `${ctx.config.fixtureUrl}/slow?ms=80`)));
|
||||
const values = burst.map((item) => item.response.durationMs).sort((a, b) => a - b);
|
||||
const p95 = values[Math.ceil(values.length * 0.95) - 1];
|
||||
const target = path.join(ctx.dirs.raw, 'latency-contracts.json');
|
||||
writeJson(target, { fast, slow, burst: burst.map((item) => item.response), p95 });
|
||||
const accurate = fast.response.status === 200 && slow.response.status === 200 && fast.parsed?.success && slow.parsed?.success &&
|
||||
slow.parsed.latency >= 250 && slow.parsed.latency <= 1500 && slow.parsed.latency > fast.parsed.latency;
|
||||
finding(ctx, {
|
||||
id: 'latency.accuracy', category: 'performance', title: 'Latency probe distinguishes fast and delayed sources',
|
||||
status: accurate ? 'PASS' : 'FAIL', severity: 'high', expected: '300ms fixture reports 250-1500ms and exceeds fast fixture',
|
||||
actual: JSON.stringify({ fast: fast.parsed, slow: slow.parsed }), reason: accurate ? 'Measured latency tracks controlled upstream delay.' : 'Latency values are missing, inverted, or outside tolerance.',
|
||||
evidence: [target], remediation: 'Inspect HEAD/GET fallback timing and timeout accounting.',
|
||||
});
|
||||
finding(ctx, {
|
||||
id: 'latency.concurrent-p95', category: 'performance', title: 'Concurrent latency requests stay responsive',
|
||||
status: p95 <= 2000 ? 'PASS' : 'FAIL', severity: 'medium', expected: '12-request p95 <= 2000ms', actual: `${p95}ms`,
|
||||
reason: p95 <= 2000 ? 'The local latency endpoint handled the burst within threshold.' : 'Concurrent probes produced excessive queueing or stalls.',
|
||||
evidence: [target], remediation: 'Bound outbound concurrency and remove serial bottlenecks.',
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,56 @@
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { finding } from '../core/finding.mjs';
|
||||
import { walk, writeJson } from '../core/files.mjs';
|
||||
import { newPage, stabilize } from '../browser/session.mjs';
|
||||
import { pageMetrics } from '../browser/metrics.mjs';
|
||||
|
||||
async function scrollFrames(page) {
|
||||
return page.evaluate(async () => {
|
||||
const frames = [];
|
||||
const started = performance.now();
|
||||
let previous = started;
|
||||
await new Promise((resolve) => {
|
||||
const step = (now) => {
|
||||
frames.push(now - previous); previous = now;
|
||||
const progress = Math.min(1, (now - started) / 2500);
|
||||
scrollTo(0, (document.documentElement.scrollHeight - innerHeight) * progress);
|
||||
if (progress < 1) requestAnimationFrame(step); else resolve();
|
||||
};
|
||||
requestAnimationFrame(step);
|
||||
});
|
||||
frames.sort((a, b) => a - b);
|
||||
return { count: frames.length, p95: frames[Math.ceil(frames.length * .95) - 1] || 0, over34: frames.filter((item) => item > 34).length };
|
||||
});
|
||||
}
|
||||
|
||||
export async function checkPerformance(ctx) {
|
||||
if (!ctx.state.browser || !ctx.state.appReady) return;
|
||||
const session = await newPage(ctx.state.browser, ctx, { width: 1440, height: 1000 });
|
||||
const cdp = await session.context.newCDPSession(session.page);
|
||||
await cdp.send('Emulation.setCPUThrottlingRate', { rate: 4 });
|
||||
await session.page.goto(`${ctx.config.localUrl}/settings`, { waitUntil: 'domcontentloaded', timeout: ctx.config.navigationTimeoutMs });
|
||||
await stabilize(session.page);
|
||||
const frames = await scrollFrames(session.page);
|
||||
const metrics = await pageMetrics(session.page);
|
||||
await session.context.close();
|
||||
const bundles = walk(path.join(ctx.config.root, '.next', 'static'), (file) => file.endsWith('.js')).map((file) => ({ file, bytes: fs.statSync(file).size }));
|
||||
const bundle = { count: bundles.length, totalBytes: bundles.reduce((sum, item) => sum + item.bytes, 0), largest: bundles.sort((a, b) => b.bytes - a.bytes).slice(0, 20) };
|
||||
const target = path.join(ctx.dirs.metrics, 'performance.json');
|
||||
writeJson(target, { frames, metrics, bundle });
|
||||
const longTaskTotal = metrics.longTasks.reduce((sum, item) => sum + item, 0);
|
||||
const good = frames.p95 <= 34 && frames.over34 / Math.max(frames.count, 1) <= .05 && longTaskTotal <= ctx.config.maxLongTaskMs;
|
||||
finding(ctx, {
|
||||
id: 'performance.scroll-jank', category: 'performance', title: 'Settings scrolling remains smooth under 4× CPU throttling',
|
||||
status: good ? 'PASS' : 'FAIL', severity: 'high', expected: 'frame p95 <=34ms, >34ms frames <=5%, long tasks <=500ms total',
|
||||
actual: JSON.stringify({ frames, longTaskTotal }), reason: good ? 'Throttled scroll animation stayed within the frame budget.' : 'Measured frame gaps or main-thread long tasks exceed the budget.',
|
||||
evidence: [target], remediation: 'Profile long tasks, reduce render scope, virtualize long lists, and remove layout thrashing.',
|
||||
});
|
||||
const largest = bundle.largest[0]?.bytes || 0;
|
||||
finding(ctx, {
|
||||
id: 'performance.bundle-size', category: 'performance', title: 'Client JavaScript bundle size is bounded',
|
||||
status: largest <= 1_000_000 ? 'PASS' : 'WARN', severity: 'medium', expected: 'Largest emitted JS asset <= 1,000,000 raw bytes', actual: JSON.stringify(bundle),
|
||||
reason: largest <= 1_000_000 ? 'No single emitted asset exceeds the guardrail.' : 'A very large asset increases parse, compile, and low-end device latency.',
|
||||
evidence: [target], remediation: 'Split heavy dependencies and defer feature code outside initial routes.',
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,46 @@
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { runCommand } from '../core/command.mjs';
|
||||
import { finding } from '../core/finding.mjs';
|
||||
import { readJson } from '../core/files.mjs';
|
||||
|
||||
function major(version) {
|
||||
return Number(String(version).replace(/^v/, '').split('.')[0]);
|
||||
}
|
||||
|
||||
export async function checkPreflight(ctx) {
|
||||
const pkg = readJson(path.join(ctx.config.root, 'package.json'));
|
||||
const release = readJson(path.join(ctx.config.root, 'app-release.json'));
|
||||
const versionsMatch = pkg.version === release.currentVersion && pkg.version === release.releases?.[0]?.version;
|
||||
finding(ctx, {
|
||||
id: 'preflight.version-consistency', category: 'preflight', title: 'Local version metadata agrees',
|
||||
status: versionsMatch ? 'PASS' : 'FAIL', severity: 'high', expected: 'package.json, currentVersion, and first release match',
|
||||
actual: JSON.stringify({ package: pkg.version, current: release.currentVersion, release: release.releases?.[0]?.version }),
|
||||
reason: versionsMatch ? 'All local release sources agree.' : 'Release sources disagree and can publish ambiguous artifacts.',
|
||||
remediation: 'Update all version sources atomically before release.',
|
||||
});
|
||||
const nodeMajor = major(process.version);
|
||||
finding(ctx, {
|
||||
id: 'preflight.node', category: 'preflight', title: 'Node.js runtime is supported',
|
||||
status: nodeMajor >= 20 && nodeMajor <= 26 ? 'PASS' : 'WARN', severity: 'medium', expected: 'Node.js 20 through 26',
|
||||
actual: process.version, reason: nodeMajor >= 20 && nodeMajor <= 26 ? 'Runtime is within the tested range.' : 'Runtime is outside the declared range.',
|
||||
remediation: 'Use an LTS version supported by Next.js and the repository.',
|
||||
});
|
||||
const chrome = '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome';
|
||||
finding(ctx, {
|
||||
id: 'preflight.chrome', category: 'preflight', title: 'Chrome executable is available',
|
||||
status: fs.existsSync(chrome) ? 'PASS' : 'FAIL', severity: 'high', expected: chrome, actual: fs.existsSync(chrome),
|
||||
reason: fs.existsSync(chrome) ? 'UI checks can launch an isolated browser.' : 'UI checks cannot launch the required browser.',
|
||||
remediation: 'Install Google Chrome or configure a supported executable.',
|
||||
});
|
||||
ctx.state.chromePath = chrome;
|
||||
const git = await runCommand(ctx, 'git-status', 'git', ['status', '--porcelain=v1'], { timeoutMs: 30_000 });
|
||||
const businessChanges = git.tail.split('\n').filter(Boolean).filter((line) => !line.slice(3).startsWith('verification/'));
|
||||
finding(ctx, {
|
||||
id: 'preflight.business-tree', category: 'preflight', title: 'Business source tree has no unrelated edits',
|
||||
status: businessChanges.length ? 'FAIL' : 'PASS', severity: 'high', expected: 'No changes outside verification/',
|
||||
actual: businessChanges.length ? businessChanges.join('\n') : 'clean',
|
||||
reason: businessChanges.length ? 'Unrelated edits make attribution and safe publishing ambiguous.' : 'Only the validation scope is changed.',
|
||||
evidence: [git.outputPath], remediation: 'Separate or explicitly approve unrelated changes before publishing.',
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,45 @@
|
||||
import path from 'node:path';
|
||||
import { finding } from '../core/finding.mjs';
|
||||
import { request } from '../core/http.mjs';
|
||||
import { writeJson } from '../core/files.mjs';
|
||||
|
||||
export async function checkProxy(ctx) {
|
||||
if (!ctx.state.appReady) return;
|
||||
const proxy = (url) => `${ctx.config.localUrl}/api/proxy?url=${encodeURIComponent(url)}`;
|
||||
const cases = {
|
||||
missing: await request(`${ctx.config.localUrl}/api/proxy`),
|
||||
mp4: await request(proxy(`${ctx.config.fixtureUrl}/test.mp4`)),
|
||||
range: await request(proxy(`${ctx.config.fixtureUrl}/test.mp4`), { headers: { range: 'bytes=0-99' } }),
|
||||
hls: await request(proxy(`${ctx.config.fixtureUrl}/hls/master.m3u8`)),
|
||||
notFound: await request(proxy(`${ctx.config.fixtureUrl}/status/404`)),
|
||||
redirect: await request(proxy(`${ctx.config.fixtureUrl}/redirect`)),
|
||||
fileProtocol: await request(proxy('file:///etc/hosts')),
|
||||
};
|
||||
const target = path.join(ctx.dirs.raw, 'proxy-contracts.json');
|
||||
writeJson(target, cases);
|
||||
const functional = cases.missing.status === 400 && cases.mp4.status === 200 && cases.range.status === 206 &&
|
||||
cases.range.bytes === 100 && cases.hls.status === 200 && cases.hls.body.includes('/api/proxy?url=') &&
|
||||
cases.notFound.status === 404 && cases.redirect.status === 200;
|
||||
finding(ctx, {
|
||||
id: 'proxy.functional', category: 'proxy', title: 'Media proxy preserves errors, ranges, redirects, CORS, and rewrites HLS',
|
||||
status: functional ? 'PASS' : 'FAIL', severity: 'critical', expected: 'All seven proxy contracts pass',
|
||||
actual: JSON.stringify(Object.fromEntries(Object.entries(cases).map(([key, value]) => [key, { status: value.status, bytes: value.bytes, headers: value.headers }]))),
|
||||
reason: functional ? 'Deterministic upstream behavior survived the proxy contract.' : 'One or more core proxy behaviors are broken.', evidence: [target],
|
||||
remediation: 'Fix forwarding, range/header preservation, redirect handling, or playlist rewriting.',
|
||||
});
|
||||
const blocksUnsupported = cases.fileProtocol.status >= 400 && cases.fileProtocol.status < 500;
|
||||
finding(ctx, {
|
||||
id: 'proxy.protocol-validation', category: 'security', title: 'Proxy rejects unsupported protocols before fetching',
|
||||
status: blocksUnsupported ? 'PASS' : 'FAIL', severity: 'high', expected: 'Controlled HTTP 4xx for file://', actual: cases.fileProtocol.status,
|
||||
reason: blocksUnsupported ? 'Unsupported protocols are rejected as client input.' : 'Unsupported protocols fall into a server error instead of explicit validation.',
|
||||
evidence: [target], remediation: 'Allow only http: and https: before invoking fetch.',
|
||||
});
|
||||
const loopbackFetched = cases.mp4.status === 200;
|
||||
finding(ctx, {
|
||||
id: 'proxy.private-network-ssrf', category: 'security', title: 'Proxy blocks loopback and private-network targets',
|
||||
status: loopbackFetched ? 'FAIL' : 'PASS', severity: 'critical', expected: 'Loopback target rejected', actual: `loopback HTTP ${cases.mp4.status}`,
|
||||
reason: loopbackFetched ? 'The public proxy route can reach 127.0.0.1, demonstrating an SSRF primitive on self-hosted deployments.' : 'Private address access was blocked.',
|
||||
impact: 'An exposed self-hosted instance may reach internal services available to the application host.', evidence: [target],
|
||||
remediation: 'Resolve DNS safely and reject loopback, link-local, private, multicast, and metadata-service address ranges across redirects.',
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
import { finding } from '../core/finding.mjs';
|
||||
import { startProcess, waitForUrl } from '../core/service.mjs';
|
||||
import { createMedia } from '../fixture/media.mjs';
|
||||
import { startFixtureServer } from '../fixture/server.mjs';
|
||||
|
||||
export async function startRuntime(ctx) {
|
||||
await createMedia(ctx);
|
||||
await startFixtureServer(ctx);
|
||||
const fixture = await waitForUrl(`${ctx.config.fixtureUrl}/health`, 5000);
|
||||
finding(ctx, {
|
||||
id: 'runtime.fixture-server', category: 'harness', title: 'Local deterministic fixture server is reachable',
|
||||
status: fixture.ok ? 'PASS' : 'FAIL', severity: 'critical', expected: 'HTTP 200', actual: fixture.ok ? fixture.status : fixture.error,
|
||||
reason: fixture.ok ? 'API, proxy, latency, and media tests can use controlled upstream behavior.' : 'Controlled integration tests cannot run.',
|
||||
remediation: 'Free port 34174 and rerun.',
|
||||
});
|
||||
const production = ctx.state.buildOk;
|
||||
const args = production ? ['start'] : ['run', 'dev'];
|
||||
const service = await startProcess(ctx, production ? 'next-start' : 'next-dev', 'npm', args, {
|
||||
env: { PORT: String(ctx.config.localPort), HOSTNAME: '127.0.0.1', NEXT_TELEMETRY_DISABLED: '1' },
|
||||
url: ctx.config.localUrl, timeoutMs: 120_000,
|
||||
});
|
||||
const ready = service.ready?.ok;
|
||||
ctx.state.appReady = ready;
|
||||
finding(ctx, {
|
||||
id: 'runtime.local-server', category: 'runtime', title: 'KVideo local server starts and responds',
|
||||
status: ready ? 'PASS' : 'FAIL', severity: 'critical', expected: `Reachable ${production ? 'production' : 'development fallback'} server`,
|
||||
actual: ready ? `HTTP ${service.ready.status}` : service.ready?.error || 'not ready',
|
||||
reason: ready ? 'Browser and API integration checks can execute.' : 'No local application endpoint became ready.',
|
||||
impact: production ? '' : 'Production build failed, so UI evidence uses a development fallback.',
|
||||
evidence: [service.outputPath], remediation: 'Fix startup/build errors or release port 34173.',
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,43 @@
|
||||
import path from 'node:path';
|
||||
import { finding } from '../core/finding.mjs';
|
||||
import { request } from '../core/http.mjs';
|
||||
import { writeJson } from '../core/files.mjs';
|
||||
|
||||
function evaluate(headers, remote) {
|
||||
const checks = {
|
||||
contentTypeOptions: headers['x-content-type-options'] === 'nosniff',
|
||||
referrerPolicy: Boolean(headers['referrer-policy']),
|
||||
framing: Boolean(headers['x-frame-options'] || headers['content-security-policy']?.includes('frame-ancestors')),
|
||||
permissionsPolicy: Boolean(headers['permissions-policy']),
|
||||
contentSecurityPolicy: Boolean(headers['content-security-policy']),
|
||||
hsts: !remote || Boolean(headers['strict-transport-security']),
|
||||
noPoweredBy: !headers['x-powered-by'],
|
||||
};
|
||||
return { checks, missing: Object.entries(checks).filter(([, ok]) => !ok).map(([name]) => name) };
|
||||
}
|
||||
|
||||
export async function checkSecurityHeaders(ctx) {
|
||||
if (!ctx.state.appReady) return;
|
||||
const local = await request(ctx.config.localUrl);
|
||||
const remote = ctx.config.offline ? null : await request(ctx.config.referenceUrl);
|
||||
const localEval = evaluate(local.headers, false);
|
||||
const remoteEval = remote ? evaluate(remote.headers, true) : null;
|
||||
const assets = await Promise.all(['/manifest.json', '/sw.js', '/icon.png'].map(async (route) => ({ route, response: await request(`${ctx.config.localUrl}${route}`) })));
|
||||
const target = path.join(ctx.dirs.raw, 'security-headers.json');
|
||||
writeJson(target, { local, remote, localEval, remoteEval, assets });
|
||||
finding(ctx, {
|
||||
id: 'security.response-headers', category: 'security', title: 'Application responses set a complete browser security-header baseline',
|
||||
status: localEval.missing.length || remoteEval?.missing.length ? 'FAIL' : 'PASS', severity: 'high', expected: 'nosniff, referrer, framing, permissions, CSP, HSTS remotely, no powered-by',
|
||||
actual: JSON.stringify({ localMissing: localEval.missing, remoteMissing: remoteEval?.missing || [] }),
|
||||
reason: localEval.missing.length || remoteEval?.missing.length ? 'One or more standard browser defenses are absent.' : 'Both surfaces provide the configured defense-in-depth headers.',
|
||||
evidence: [target], remediation: 'Define headers in Next.js/self-hosted responses and Cloudflare configuration; use a restrictive CSP tested against required scripts.',
|
||||
});
|
||||
const assetFailures = assets.filter((item) => item.response.status !== 200);
|
||||
finding(ctx, {
|
||||
id: 'runtime.pwa-assets', category: 'runtime', title: 'PWA manifest, service worker, and icon are reachable',
|
||||
status: assetFailures.length ? 'FAIL' : 'PASS', severity: 'medium', expected: 'HTTP 200 for all required PWA assets',
|
||||
actual: JSON.stringify(assets.map((item) => ({ route: item.route, status: item.response.status, bytes: item.response.bytes }))),
|
||||
reason: assetFailures.length ? 'At least one install/offline asset is missing.' : 'All declared PWA assets are served.', evidence: [target],
|
||||
remediation: 'Restore the missing public asset and verify its content type and cache behavior.',
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,45 @@
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { finding } from '../core/finding.mjs';
|
||||
import { relative, walk, writeJson } from '../core/files.mjs';
|
||||
|
||||
const textExt = new Set(['.ts', '.tsx', '.js', '.mjs', '.json', '.yml', '.yaml', '.toml', '.md']);
|
||||
const patterns = [
|
||||
['private-key', /-----BEGIN (?:RSA |EC |OPENSSH )?PRIVATE KEY-----/],
|
||||
['github-token', /\bgh[opusr]_[A-Za-z0-9_]{20,}\b/],
|
||||
['aws-access-key', /\bAKIA[0-9A-Z]{16}\b/],
|
||||
['generic-secret-assignment', /(?:secret|token|password|api[_-]?key)\s*[:=]\s*['"][^'"\n]{12,}['"]/i],
|
||||
];
|
||||
|
||||
export async function checkSecurityScan(ctx) {
|
||||
const files = walk(ctx.config.root, (file) => textExt.has(path.extname(file)) && !file.includes('/verification/artifacts/'));
|
||||
const hits = [];
|
||||
for (const file of files) {
|
||||
const lines = fs.readFileSync(file, 'utf8').split(/\r?\n/);
|
||||
lines.forEach((line, index) => {
|
||||
for (const [name, pattern] of patterns) if (pattern.test(line)) hits.push({ file: relative(ctx.config.root, file), line: index + 1, pattern: name });
|
||||
});
|
||||
}
|
||||
const target = path.join(ctx.dirs.metrics, 'secret-scan.json');
|
||||
writeJson(target, hits);
|
||||
finding(ctx, {
|
||||
id: 'security.static-secrets', category: 'security', title: 'Repository contains no obvious committed secrets',
|
||||
status: hits.length ? 'FAIL' : 'PASS', severity: 'critical', expected: '0 credential-pattern matches',
|
||||
actual: hits.length ? JSON.stringify(hits) : '0',
|
||||
reason: hits.length ? 'Credential-like material appears in repository text. Values are intentionally omitted from logs.' : 'No configured credential signature was found.',
|
||||
evidence: [target], remediation: 'Remove and rotate confirmed credentials; replace false positives with safe fixtures.',
|
||||
});
|
||||
const dangerous = files.filter((file) => ['.ts', '.tsx', '.js', '.mjs'].includes(path.extname(file))).flatMap((file) => {
|
||||
const text = fs.readFileSync(file, 'utf8');
|
||||
return [
|
||||
...(text.includes('dangerouslySetInnerHTML') ? [{ file: relative(ctx.config.root, file), construct: 'dangerouslySetInnerHTML' }] : []),
|
||||
...(text.match(/\beval\s*\(/) ? [{ file: relative(ctx.config.root, file), construct: 'eval' }] : []),
|
||||
];
|
||||
});
|
||||
finding(ctx, {
|
||||
id: 'security.dangerous-constructs', category: 'security', title: 'Dangerous runtime constructs are inventoried',
|
||||
status: dangerous.length ? 'WARN' : 'PASS', severity: 'medium', expected: 'No eval or unreviewed raw HTML injection', actual: JSON.stringify(dangerous),
|
||||
reason: dangerous.length ? 'These constructs expand injection risk and require contextual review.' : 'No configured dangerous construct was found.',
|
||||
remediation: 'Verify sanitization and replace raw execution or HTML injection where possible.',
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
import path from 'node:path';
|
||||
import { finding } from '../core/finding.mjs';
|
||||
import { lineCount, relative, walk, writeJson } from '../core/files.mjs';
|
||||
|
||||
const codeExt = new Set(['.ts', '.tsx', '.js', '.jsx', '.mjs', '.cjs', '.css', '.swift', '.kt', '.kts']);
|
||||
|
||||
function inventory(root, files) {
|
||||
return files.map((file) => ({ file: relative(root, file), lines: lineCount(file) })).sort((a, b) => b.lines - a.lines);
|
||||
}
|
||||
|
||||
export async function checkSourcePolicy(ctx) {
|
||||
const root = ctx.config.root;
|
||||
const projectFiles = walk(root, (file) => codeExt.has(path.extname(file)) && !file.includes('/verification/'));
|
||||
const project = inventory(root, projectFiles);
|
||||
const oversized = project.filter((item) => item.lines > ctx.config.maxSourceLines);
|
||||
const totalLines = project.reduce((sum, item) => sum + item.lines, 0);
|
||||
writeJson(path.join(ctx.dirs.metrics, 'source-inventory.json'), { totalLines, files: project, oversized });
|
||||
finding(ctx, {
|
||||
id: 'source.project-line-policy', category: 'source', title: 'Project source files respect the 150-line policy',
|
||||
status: oversized.length ? 'FAIL' : 'PASS', severity: 'high', expected: `Every source file <= ${ctx.config.maxSourceLines} lines`,
|
||||
actual: `${oversized.length}/${project.length} files exceed the limit; ${totalLines} total lines`,
|
||||
reason: oversized.length ? 'Large files concentrate responsibilities and materially increase review and regression risk.' : 'All source files meet the limit.',
|
||||
impact: oversized.slice(0, 20).map((item) => `${item.file}:${item.lines}`).join(', '),
|
||||
evidence: [path.join(ctx.dirs.metrics, 'source-inventory.json')], remediation: 'Split oversized business files by cohesive responsibility in a separate change.',
|
||||
});
|
||||
const authored = walk(ctx.config.verifyDir, (file) => !file.includes('/node_modules/') && !file.includes('/artifacts/') && path.basename(file) !== 'package-lock.json');
|
||||
const validator = inventory(ctx.config.verifyDir, authored);
|
||||
const validatorOversized = validator.filter((item) => item.lines > ctx.config.maxSourceLines);
|
||||
writeJson(path.join(ctx.dirs.metrics, 'verification-inventory.json'), validator);
|
||||
finding(ctx, {
|
||||
id: 'source.verification-line-policy', category: 'harness', title: 'Verification files respect the 150-line policy',
|
||||
status: validatorOversized.length ? 'FAIL' : 'PASS', severity: 'critical', expected: `Every authored verification file <= ${ctx.config.maxSourceLines} lines`,
|
||||
actual: validatorOversized.length ? JSON.stringify(validatorOversized) : `${validator.length} files comply`,
|
||||
reason: validatorOversized.length ? 'The delivered validation code violates the explicit file-size constraint.' : 'The validation implementation is partitioned within the limit.',
|
||||
evidence: [path.join(ctx.dirs.metrics, 'verification-inventory.json')], remediation: 'Split the listed verification files before trusting or publishing the suite.',
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
import fs from 'node:fs';
|
||||
import { runCommand, runNpm } from '../core/command.mjs';
|
||||
import { finding } from '../core/finding.mjs';
|
||||
|
||||
function commandFinding(ctx, id, title, result, severity, expected = 'exit code 0') {
|
||||
const ok = result.code === 0 && !result.timedOut;
|
||||
finding(ctx, {
|
||||
id, category: 'static', title, status: ok ? 'PASS' : 'FAIL', severity, expected,
|
||||
actual: result.timedOut ? `timed out after ${result.durationMs}ms` : `exit code ${result.code}`,
|
||||
reason: ok ? 'The command completed successfully.' : 'The authoritative project command failed; the raw output contains exact diagnostics.',
|
||||
impact: ok ? '' : result.tail.slice(-2000), evidence: [result.outputPath],
|
||||
remediation: 'Resolve every diagnostic in the raw output, then rerun the complete chain.', durationMs: result.durationMs,
|
||||
});
|
||||
return ok;
|
||||
}
|
||||
|
||||
export async function checkStaticTools(ctx) {
|
||||
const test = await runNpm(ctx, 'npm-test', ['test']);
|
||||
ctx.state.testsOk = commandFinding(ctx, 'static.unit-tests', 'Repository unit tests pass', test, 'critical');
|
||||
const lint = await runNpm(ctx, 'npm-lint', ['run', 'lint']);
|
||||
ctx.state.lintOk = commandFinding(ctx, 'static.eslint', 'ESLint reports no errors or warnings', lint, 'high');
|
||||
const types = await runCommand(ctx, 'typescript', 'npx', ['--no-install', 'tsc', '--noEmit', '--incremental', 'false']);
|
||||
ctx.state.typesOk = commandFinding(ctx, 'static.typescript', 'Full TypeScript check passes', types, 'high');
|
||||
const integrity = await runNpm(ctx, 'npm-ls', ['ls', '--all', '--json']);
|
||||
commandFinding(ctx, 'static.dependency-integrity', 'Installed dependency graph is valid', integrity, 'high');
|
||||
if (!ctx.config.offline) await checkAudit(ctx);
|
||||
else finding(ctx, {
|
||||
id: 'static.npm-audit', category: 'static', title: 'Dependency vulnerability audit', status: 'SKIP', severity: 'high',
|
||||
expected: 'Online npm audit', actual: '--offline', reason: 'The run explicitly disabled network checks.', remediation: 'Rerun without --offline.',
|
||||
});
|
||||
const build = await runNpm(ctx, 'next-build', ['run', 'build'], { timeoutMs: ctx.config.commandTimeoutMs });
|
||||
ctx.state.buildOk = commandFinding(ctx, 'static.production-build', 'Production Next.js build succeeds', build, 'critical');
|
||||
if (!ctx.config.quick) {
|
||||
const pages = await runNpm(ctx, 'cloudflare-pages-build', ['run', 'pages:build'], { timeoutMs: ctx.config.commandTimeoutMs });
|
||||
ctx.state.pagesBuildOk = commandFinding(ctx, 'static.cloudflare-build', 'Cloudflare Pages build succeeds', pages, 'critical');
|
||||
}
|
||||
}
|
||||
|
||||
async function checkAudit(ctx) {
|
||||
const result = await runNpm(ctx, 'npm-audit', ['audit', '--omit=dev', '--json']);
|
||||
let audit = null;
|
||||
try { audit = JSON.parse(fs.readFileSync(result.outputPath, 'utf8')); } catch { /* malformed audit output */ }
|
||||
const vulnerabilities = audit?.metadata?.vulnerabilities || {};
|
||||
const severe = (vulnerabilities.critical || 0) + (vulnerabilities.high || 0);
|
||||
const ok = result.code === 0 && severe === 0;
|
||||
finding(ctx, {
|
||||
id: 'static.npm-audit', category: 'security', title: 'Production dependencies have no known high/critical vulnerabilities',
|
||||
status: ok ? 'PASS' : 'FAIL', severity: 'critical', expected: '0 high and 0 critical vulnerabilities',
|
||||
actual: audit ? JSON.stringify(vulnerabilities) : `unparseable output; exit ${result.code}`,
|
||||
reason: ok ? 'npm advisory data reports no severe production vulnerability.' : 'The dependency audit failed or reports severe vulnerabilities.',
|
||||
evidence: [result.outputPath], remediation: 'Upgrade, replace, or explicitly mitigate every severe advisory.', durationMs: result.durationMs,
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,130 @@
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { finding } from '../core/finding.mjs';
|
||||
import { walk, relative, writeJson } from '../core/files.mjs';
|
||||
import { log } from '../core/log.mjs';
|
||||
import { newPage, stabilize } from '../browser/session.mjs';
|
||||
import { performAction, scanActions, stateHash } from '../browser/actions.mjs';
|
||||
|
||||
async function navigate(page, ctx, route, pathSteps, fixtureFile) {
|
||||
await page.goto(`${ctx.config.localUrl}${route}`, { waitUntil: 'domcontentloaded', timeout: ctx.config.navigationTimeoutMs });
|
||||
await stabilize(page);
|
||||
for (const action of pathSteps) {
|
||||
const result = await performAction(page, action, fixtureFile);
|
||||
if (!result.ok) return result;
|
||||
await page.waitForTimeout(250);
|
||||
}
|
||||
return { ok: true };
|
||||
}
|
||||
|
||||
function staysWithinRoute(ctx, route, action, urlAfter) {
|
||||
const start = new URL(route, ctx.config.localUrl);
|
||||
const target = action.href ? new URL(action.href, start) : new URL(urlAfter);
|
||||
return target.origin === start.origin && target.pathname === start.pathname;
|
||||
}
|
||||
|
||||
export async function checkUiActions(ctx) {
|
||||
if (!ctx.state.browser || !ctx.state.pageRoutes) return;
|
||||
const session = await newPage(ctx.state.browser, ctx, { width: 1440, height: 1000 });
|
||||
await session.context.tracing.start({ screenshots: true, snapshots: true, sources: true });
|
||||
const fixtureFile = path.join(ctx.dirs.raw, 'import-fixture.json');
|
||||
fs.writeFileSync(fixtureFile, JSON.stringify({ settings: { sources: [] } }));
|
||||
const results = [];
|
||||
const cappedRoutes = [];
|
||||
for (const route of ctx.state.pageRoutes) {
|
||||
const routeStart = results.length;
|
||||
const queue = [[]];
|
||||
const seenStates = new Set();
|
||||
const testedActions = new Set();
|
||||
let routeActions = 0;
|
||||
let hitCap = false;
|
||||
log(ctx, 'info', 'ui.action-route.start', 'Starting recursive runtime control exploration', { route });
|
||||
while (queue.length && (ctx.config.quick ? results.length : routeActions) < ctx.config.maxActionStates) {
|
||||
const steps = queue.shift();
|
||||
const stepKeys = steps.map((item) => item.key);
|
||||
const replay = await navigate(session.page, ctx, route, steps, fixtureFile);
|
||||
if (!replay.ok) {
|
||||
results.push({ route, depth: steps.length, steps: stepKeys, phase: 'state-replay', result: replay });
|
||||
log(ctx, 'error', 'ui.action-failure', 'State path replay failed', { route, steps: stepKeys, result: replay });
|
||||
await session.page.screenshot({ path: path.join(ctx.dirs.screenshots, `action-failure-${results.length}.png`), fullPage: true }).catch(() => {});
|
||||
continue;
|
||||
}
|
||||
const actions = await scanActions(session.page);
|
||||
const hash = stateHash(session.page.url(), actions);
|
||||
if (seenStates.has(hash)) continue;
|
||||
seenStates.add(hash);
|
||||
for (const action of actions) {
|
||||
if (testedActions.has(action.key)) continue;
|
||||
if ((ctx.config.quick ? results.length : routeActions) >= ctx.config.maxActionStates) {
|
||||
hitCap = true;
|
||||
break;
|
||||
}
|
||||
testedActions.add(action.key);
|
||||
routeActions += 1;
|
||||
const reset = await navigate(session.page, ctx, route, steps, fixtureFile);
|
||||
if (!reset.ok) {
|
||||
results.push({ route, state: hash, depth: steps.length, steps: stepKeys, phase: 'action-reset', action, result: reset });
|
||||
log(ctx, 'error', 'ui.action-failure', 'Action state reset failed', { route, state: hash, steps: stepKeys, action, result: reset });
|
||||
continue;
|
||||
}
|
||||
let result;
|
||||
try { result = await performAction(session.page, action, fixtureFile); await session.page.waitForTimeout(300); }
|
||||
catch (error) { result = { ok: false, reason: error instanceof Error ? error.message : String(error) }; }
|
||||
const after = result.ok ? await scanActions(session.page) : [];
|
||||
const changed = result.ok && stateHash(session.page.url(), after) !== hash;
|
||||
results.push({ route, state: hash, depth: steps.length, steps: stepKeys, action, result, changed, urlAfter: session.page.url() });
|
||||
if (!result.ok) {
|
||||
log(ctx, 'error', 'ui.action-failure', 'Runtime control interaction failed', { route, state: hash, depth: steps.length, steps: stepKeys, action, result, urlAfter: session.page.url() });
|
||||
await session.page.screenshot({ path: path.join(ctx.dirs.screenshots, `action-failure-${results.length}.png`), fullPage: true }).catch(() => {});
|
||||
}
|
||||
if (changed && steps.length < ctx.config.maxActionDepth && staysWithinRoute(ctx, route, action, session.page.url())) {
|
||||
queue.push([...steps, action]);
|
||||
}
|
||||
}
|
||||
}
|
||||
const capped = hitCap || queue.length > 0;
|
||||
if (capped) cappedRoutes.push(route);
|
||||
log(ctx, 'info', 'ui.action-route.end', 'Finished recursive runtime control exploration', {
|
||||
route, actions: results.length - routeStart, uniqueActions: testedActions.size,
|
||||
uniqueStates: seenStates.size, pendingStates: queue.length, capped,
|
||||
});
|
||||
if (ctx.config.quick && results.length >= ctx.config.maxActionStates) break;
|
||||
}
|
||||
const trace = path.join(ctx.dirs.traces, 'ui-actions.zip');
|
||||
await session.context.tracing.stop({ path: trace });
|
||||
await session.context.close();
|
||||
const declared = sourceActionInventory(ctx);
|
||||
const target = path.join(ctx.dirs.raw, 'ui-actions.json');
|
||||
writeJson(target, { results, declared, cappedRoutes, observed: session.observed });
|
||||
const failures = results.filter((item) => item.result && !item.result.ok);
|
||||
const skipped = results.filter((item) => item.result?.skipped);
|
||||
const uniqueRuntime = new Set(results.filter((item) => item.action).map((item) => `${item.route}|${item.action.key}`)).size;
|
||||
finding(ctx, {
|
||||
id: 'ui.action-execution', category: 'ui', title: 'Every discovered runtime control accepts its intended interaction',
|
||||
status: failures.length ? 'FAIL' : 'PASS', severity: 'critical', expected: '0 click/fill/select/upload failures',
|
||||
actual: failures.length ? JSON.stringify(failures.slice(0, 30)) : `${results.length - skipped.length} operated; ${skipped.length} disabled/skipped`,
|
||||
reason: failures.length ? 'A visible runtime control could not be replayed or operated.' : 'All discovered controls were exercised without automation failure.',
|
||||
evidence: [target, trace], remediation: 'Repair unstable selectors, disabled-state logic, click handlers, or the underlying UI exception.',
|
||||
});
|
||||
finding(ctx, {
|
||||
id: 'ui.action-state-coverage', category: 'ui', title: 'Recursive UI state exploration exhausts its queue',
|
||||
status: cappedRoutes.length ? (ctx.config.quick ? 'SKIP' : 'FAIL') : 'PASS', severity: 'high', expected: `Queue exhausted below ${ctx.config.maxActionStates} actions per route and depth ${ctx.config.maxActionDepth}`,
|
||||
actual: cappedRoutes.length ? `Coverage cap reached on ${cappedRoutes.join(', ')} after ${results.length} actions` : `${results.length} actions; queue exhausted`,
|
||||
reason: cappedRoutes.length ? (ctx.config.quick ? 'Quick mode intentionally limits exploration.' : 'Unexplored reachable states remain, so “all buttons” cannot be claimed.') : 'No additional state-changing action remained within the discovered graph.',
|
||||
evidence: [target], remediation: 'Raise limits or split workflows until every reachable state is exhausted.',
|
||||
});
|
||||
finding(ctx, {
|
||||
id: 'ui.action-inventory', category: 'ui', title: 'Static and runtime interaction inventory is recorded',
|
||||
status: 'INFO', severity: 'info', expected: 'Every source declaration and discovered runtime control remains auditable',
|
||||
actual: JSON.stringify({ staticDeclarationSites: declared.length, runtimeActionInstances: results.length, uniqueRuntimeControls: uniqueRuntime, disabledOrSkipped: skipped.length, failures: failures.length, cappedRoutes }),
|
||||
reason: 'Static declaration sites and runtime controls are different populations; the evidence preserves both without claiming a false one-to-one mapping.',
|
||||
evidence: [target], remediation: 'Use file/line declarations and trace-backed runtime states to investigate controls absent from reachable test states.',
|
||||
});
|
||||
}
|
||||
|
||||
function sourceActionInventory(ctx) {
|
||||
const files = walk(ctx.config.root, (file) => /\.(tsx|jsx)$/.test(file) && !file.includes('/verification/'));
|
||||
return files.flatMap((file) => fs.readFileSync(file, 'utf8').split(/\r?\n/).flatMap((line, index) =>
|
||||
/<(button|input|select|textarea)|onClick=|role=["']button/.test(line) ? [{ file: relative(ctx.config.root, file), line: index + 1 }] : [],
|
||||
));
|
||||
}
|
||||
@@ -0,0 +1,59 @@
|
||||
import path from 'node:path';
|
||||
import { finding } from '../core/finding.mjs';
|
||||
import { writeJson } from '../core/files.mjs';
|
||||
import { launchBrowser, newPage, stabilize } from '../browser/session.mjs';
|
||||
import { discoverPages } from '../browser/routes.mjs';
|
||||
import { pageMetrics } from '../browser/metrics.mjs';
|
||||
import { scanAxe } from '../browser/axe.mjs';
|
||||
|
||||
function safeName(value) {
|
||||
return value.replace(/^\//, '').replace(/[^a-zA-Z0-9]+/g, '-') || 'home';
|
||||
}
|
||||
|
||||
export async function checkUiPages(ctx) {
|
||||
if (!ctx.state.appReady) return;
|
||||
const browser = ctx.state.browser || await launchBrowser(ctx);
|
||||
ctx.state.browser = browser;
|
||||
const routes = discoverPages(ctx);
|
||||
ctx.state.pageRoutes = routes;
|
||||
const results = [];
|
||||
for (const viewport of ctx.config.viewports) {
|
||||
for (const route of routes) {
|
||||
const session = await newPage(browser, ctx, viewport);
|
||||
let response = null;
|
||||
let axe = { violations: [], incomplete: [] };
|
||||
try {
|
||||
response = await session.page.goto(`${ctx.config.localUrl}${route}`, { waitUntil: 'domcontentloaded', timeout: ctx.config.navigationTimeoutMs });
|
||||
await stabilize(session.page);
|
||||
axe = await scanAxe(session.page, ctx);
|
||||
const screenshot = path.join(ctx.dirs.screenshots, `${viewport.name}-${safeName(route)}.png`);
|
||||
await session.page.screenshot({ path: screenshot, fullPage: true });
|
||||
results.push({ viewport, route, status: response?.status() || 0, metrics: await pageMetrics(session.page), axe, observed: session.observed, screenshot });
|
||||
} catch (error) {
|
||||
results.push({ viewport, route, status: response?.status() || 0, error: error instanceof Error ? error.stack || error.message : String(error), axe, observed: session.observed });
|
||||
} finally {
|
||||
await session.context.close();
|
||||
}
|
||||
}
|
||||
}
|
||||
const target = path.join(ctx.dirs.raw, 'ui-pages.json');
|
||||
writeJson(target, results);
|
||||
const renderFailures = results.filter((item) => item.error || item.status >= 400 || item.status === 0);
|
||||
const runtimeErrors = results.filter((item) => item.metrics?.errors?.length || item.metrics?.rejections?.length || item.observed.pageErrors.length || item.observed.consoleErrors.length);
|
||||
const overflow = results.filter((item) => item.metrics && item.metrics.scrollWidth > item.metrics.clientWidth + 1);
|
||||
const severeA11y = results.flatMap((item) => item.axe.violations.filter((violation) => ['critical', 'serious'].includes(violation.impact)).map((violation) => ({ route: item.route, viewport: item.viewport.name, ...violation })));
|
||||
aggregate(ctx, 'ui.route-render', 'Every page renders in every target viewport', renderFailures, results.length, 'critical', target);
|
||||
aggregate(ctx, 'ui.runtime-errors', 'Pages emit no uncaught or console errors', runtimeErrors, results.length, 'critical', target);
|
||||
aggregate(ctx, 'ui.horizontal-overflow', 'Pages do not overflow target viewports horizontally', overflow, results.length, 'high', target);
|
||||
aggregate(ctx, 'ui.accessibility', 'Pages have no serious or critical automated accessibility violations', severeA11y, results.length, 'high', target);
|
||||
ctx.state.uiPages = results;
|
||||
}
|
||||
|
||||
function aggregate(ctx, id, title, failures, total, severity, evidence) {
|
||||
finding(ctx, {
|
||||
id, category: 'ui', title, status: failures.length ? 'FAIL' : 'PASS', severity, expected: `0 failures across ${total} page/viewport cases`,
|
||||
actual: failures.length ? JSON.stringify(failures.slice(0, 30)) : `${total} cases passed`,
|
||||
reason: failures.length ? 'At least one enumerated page state violated the declared UI contract.' : 'Every enumerated page state met the contract.',
|
||||
evidence: [evidence], remediation: 'Open the named screenshot and evidence record, reproduce the exact route/viewport, and repair the underlying component.',
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,79 @@
|
||||
import path from 'node:path';
|
||||
import { finding } from '../core/finding.mjs';
|
||||
import { writeJson } from '../core/files.mjs';
|
||||
import { newPage, stabilize } from '../browser/session.mjs';
|
||||
|
||||
async function playCase(ctx, episode) {
|
||||
const session = await newPage(ctx.state.browser, ctx, { width: 1440, height: 1000 });
|
||||
const route = `/player?id=fixture-video-1&source=fixture&title=%E9%AA%8C%E8%AF%81%E8%A7%86%E9%A2%91&episode=${episode}`;
|
||||
try {
|
||||
await session.page.goto(`${ctx.config.localUrl}${route}`, { waitUntil: 'domcontentloaded', timeout: ctx.config.navigationTimeoutMs });
|
||||
await stabilize(session.page);
|
||||
await session.page.locator('video').waitFor({ state: 'attached', timeout: 20_000 });
|
||||
const before = await session.page.locator('video').evaluate(async (video) => {
|
||||
video.muted = true; await video.play();
|
||||
return { currentTime: video.currentTime, readyState: video.readyState, networkState: video.networkState };
|
||||
});
|
||||
await session.page.waitForTimeout(2600);
|
||||
const after = await session.page.locator('video').evaluate((video) => {
|
||||
const quality = video.getVideoPlaybackQuality?.();
|
||||
return { currentTime: video.currentTime, duration: video.duration, paused: video.paused, readyState: video.readyState,
|
||||
width: video.videoWidth, height: video.videoHeight, totalFrames: quality?.totalVideoFrames, droppedFrames: quality?.droppedVideoFrames, error: video.error?.message || null };
|
||||
});
|
||||
await session.page.screenshot({ path: path.join(ctx.dirs.screenshots, `video-episode-${episode}.png`), fullPage: true });
|
||||
return { before, after, observed: session.observed };
|
||||
} catch (error) { return { error: error instanceof Error ? error.stack || error.message : String(error), observed: session.observed }; }
|
||||
finally { await session.context.close(); }
|
||||
}
|
||||
|
||||
async function stallCase(ctx) {
|
||||
const session = await newPage(ctx.state.browser, ctx, { width: 1440, height: 1000 });
|
||||
try {
|
||||
await session.page.goto(`${ctx.config.localUrl}/player?id=fixture-video-1&source=fixture&title=stall&episode=0`, { waitUntil: 'domcontentloaded' });
|
||||
await stabilize(session.page);
|
||||
const video = session.page.locator('video');
|
||||
await video.waitFor({ state: 'attached', timeout: 20_000 });
|
||||
await video.evaluate(async (element) => { element.muted = true; await element.play(); });
|
||||
await session.page.waitForTimeout(600);
|
||||
await video.evaluate((element) => {
|
||||
const descriptor = Object.getOwnPropertyDescriptor(HTMLMediaElement.prototype, 'currentTime');
|
||||
const frozen = descriptor.get.call(element);
|
||||
window.__kvFrozen = true;
|
||||
Object.defineProperty(element, 'currentTime', { configurable: true, get() { return window.__kvFrozen ? frozen : descriptor.get.call(this); }, set(value) { descriptor.set.call(this, value); } });
|
||||
});
|
||||
await session.page.waitForTimeout(600);
|
||||
const detected = await session.page.locator('.loading-overlay-glass').isVisible().catch(() => false);
|
||||
await video.evaluate((element) => { window.__kvFrozen = false; delete element.currentTime; });
|
||||
await session.page.waitForTimeout(500);
|
||||
const recovered = !(await session.page.locator('.loading-overlay-glass').isVisible().catch(() => false));
|
||||
return { detected, recovered };
|
||||
} catch (error) { return { error: error instanceof Error ? error.message : String(error) }; }
|
||||
finally { await session.context.close(); }
|
||||
}
|
||||
|
||||
export async function checkVideo(ctx) {
|
||||
if (!ctx.state.browser || !ctx.state.mediaOk) return;
|
||||
const mp4 = await playCase(ctx, 0);
|
||||
const hls = await playCase(ctx, 1);
|
||||
const stall = await stallCase(ctx);
|
||||
const target = path.join(ctx.dirs.raw, 'video-playback.json');
|
||||
writeJson(target, { mp4, hls, stall });
|
||||
for (const [name, result] of Object.entries({ mp4, hls })) {
|
||||
const advance = result.after ? result.after.currentTime - result.before.currentTime : 0;
|
||||
const dropped = result.after?.droppedFrames || 0;
|
||||
const total = result.after?.totalFrames || 0;
|
||||
const ok = !result.error && advance >= ctx.config.minVideoAdvanceSeconds && result.after?.width === 640 && result.after?.height === 360 && (!total || dropped / total <= .05);
|
||||
finding(ctx, {
|
||||
id: `video.${name}`, category: 'video', title: `${name.toUpperCase()} fixture plays with correct resolution and low frame loss`,
|
||||
status: ok ? 'PASS' : 'FAIL', severity: 'critical', expected: `advance >=${ctx.config.minVideoAdvanceSeconds}s, 640x360, dropped <=5%`,
|
||||
actual: JSON.stringify({ advance, result }), reason: ok ? 'Real browser media playback met timing and quality thresholds.' : 'Playback stalled, decoded incorrectly, errored, or dropped excessive frames.',
|
||||
evidence: [target], remediation: 'Inspect player events, HLS configuration, proxy mode, codec support, and browser console evidence.',
|
||||
});
|
||||
}
|
||||
finding(ctx, {
|
||||
id: 'video.stall-detector', category: 'video', title: '200ms stall detector shows and clears the loading overlay',
|
||||
status: stall.detected && stall.recovered ? 'PASS' : 'FAIL', severity: 'high', expected: 'Overlay appears while currentTime is frozen and disappears after recovery',
|
||||
actual: JSON.stringify(stall), reason: stall.detected && stall.recovered ? 'The live player responded to a controlled playback freeze.' : 'Stall detection or recovery UI did not transition correctly.',
|
||||
evidence: [target], remediation: 'Repair currentTime polling, loading state ownership, or recovery clearing logic.',
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
import crypto from 'node:crypto';
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import pixelmatch from 'pixelmatch';
|
||||
import { PNG } from 'pngjs';
|
||||
import { finding } from '../core/finding.mjs';
|
||||
import { writeJson } from '../core/files.mjs';
|
||||
import { newPage, stabilize } from '../browser/session.mjs';
|
||||
|
||||
function safe(value) {
|
||||
return value.replace(/^\//, '').replace(/[^a-zA-Z0-9]+/g, '-') || 'home';
|
||||
}
|
||||
|
||||
function providerSpecific(route) {
|
||||
return route === '/iptv' || route.endsWith('/settings');
|
||||
}
|
||||
|
||||
function padded(image, width, height) {
|
||||
const output = new PNG({ width, height, fill: true });
|
||||
PNG.bitblt(image, output, 0, 0, image.width, image.height, 0, 0);
|
||||
return output;
|
||||
}
|
||||
|
||||
function compare(leftFile, rightFile, diffFile) {
|
||||
const leftRaw = PNG.sync.read(fs.readFileSync(leftFile));
|
||||
const rightRaw = PNG.sync.read(fs.readFileSync(rightFile));
|
||||
const width = Math.max(leftRaw.width, rightRaw.width);
|
||||
const height = Math.max(leftRaw.height, rightRaw.height);
|
||||
const left = padded(leftRaw, width, height);
|
||||
const right = padded(rightRaw, width, height);
|
||||
const diff = new PNG({ width, height });
|
||||
const pixels = pixelmatch(left.data, right.data, diff.data, width, height, { threshold: 0.12, includeAA: false });
|
||||
fs.writeFileSync(diffFile, PNG.sync.write(diff));
|
||||
return { pixels, total: width * height, ratio: pixels / (width * height), dimensions: { left: [leftRaw.width, leftRaw.height], right: [rightRaw.width, rightRaw.height] } };
|
||||
}
|
||||
|
||||
async function signature(page) {
|
||||
const value = await page.evaluate(() => [...document.body.querySelectorAll('*')].map((element) => {
|
||||
const text = element.children.length ? '' : (element.textContent || '').trim().replace(/\s+/g, ' ').slice(0, 80);
|
||||
return `${element.tagName}:${element.getAttribute('role') || ''}:${element.getAttribute('aria-label') || ''}:${text}`;
|
||||
}).join('\n'));
|
||||
return crypto.createHash('sha256').update(value).digest('hex');
|
||||
}
|
||||
|
||||
async function capture(browser, ctx, base, route, viewport, file) {
|
||||
const session = await newPage(browser, ctx, viewport);
|
||||
try {
|
||||
const response = await session.page.goto(`${base}${route}`, { waitUntil: 'domcontentloaded', timeout: ctx.config.navigationTimeoutMs });
|
||||
await stabilize(session.page);
|
||||
await session.page.screenshot({ path: file, fullPage: true });
|
||||
return { status: response?.status() || 0, signature: await signature(session.page), observed: session.observed };
|
||||
} finally { await session.context.close(); }
|
||||
}
|
||||
|
||||
export async function checkVisual(ctx) {
|
||||
if (ctx.config.offline || !ctx.state.browser || !ctx.state.pageRoutes) return finding(ctx, {
|
||||
id: 'visual.deployment-diff', category: 'visual', title: 'Local and Cloudflare UI visual comparison', status: 'SKIP', severity: 'high',
|
||||
expected: 'Online reference and browser available', actual: ctx.config.offline ? '--offline' : 'browser unavailable', reason: 'Pixel comparison requires both surfaces.', remediation: 'Rerun online after local startup.',
|
||||
});
|
||||
const routes = ctx.config.quick ? ['/'] : ctx.state.pageRoutes;
|
||||
const viewports = ctx.config.quick ? [ctx.config.viewports[0]] : ctx.config.viewports.filter((item) => ['mobile', 'desktop'].includes(item.name));
|
||||
const results = [];
|
||||
for (const viewport of viewports) for (const route of routes) {
|
||||
const stem = `${viewport.name}-${safe(route)}`;
|
||||
const local = path.join(ctx.dirs.screenshots, `visual-local-${stem}.png`);
|
||||
const remote = path.join(ctx.dirs.screenshots, `visual-remote-${stem}.png`);
|
||||
const diff = path.join(ctx.dirs.diffs, `visual-diff-${stem}.png`);
|
||||
try {
|
||||
const localMeta = await capture(ctx.state.browser, ctx, ctx.config.localUrl, route, viewport, local);
|
||||
const remoteMeta = await capture(ctx.state.browser, ctx, ctx.config.referenceUrl, route, viewport, remote);
|
||||
results.push({ route, viewport: viewport.name, localMeta, remoteMeta, comparison: compare(local, remote, diff), local, remote, diff });
|
||||
} catch (error) { results.push({ route, viewport: viewport.name, error: error instanceof Error ? error.message : String(error) }); }
|
||||
}
|
||||
const target = path.join(ctx.dirs.raw, 'visual-comparison.json');
|
||||
writeJson(target, results);
|
||||
const unexpected = results.filter((item) => item.error || (!providerSpecific(item.route) && item.comparison?.ratio > ctx.config.visualDiffRatio));
|
||||
finding(ctx, {
|
||||
id: 'visual.deployment-diff', category: 'visual', title: 'Local and Cloudflare UI stay within visual deviation threshold',
|
||||
status: unexpected.length ? 'FAIL' : 'PASS', severity: 'high', expected: `Pixel difference <= ${ctx.config.visualDiffRatio * 100}% outside provider-specific routes`,
|
||||
actual: JSON.stringify(results.map((item) => ({ route: item.route, viewport: item.viewport, ratio: item.comparison?.ratio, signaturesMatch: item.localMeta?.signature === item.remoteMeta?.signature, error: item.error }))),
|
||||
reason: unexpected.length ? 'Unexpected deployment-specific visual drift exceeds the configured threshold.' : 'Common UI surfaces match; provider-specific differences remain recorded.',
|
||||
evidence: [target, ctx.dirs.diffs], remediation: 'Inspect diff images, stabilize dynamic content, and reconcile deployment build/runtime differences.',
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
import path from 'node:path';
|
||||
|
||||
const viewports = [
|
||||
{ name: 'mobile', width: 390, height: 844 },
|
||||
{ name: 'tablet', width: 820, height: 1180 },
|
||||
{ name: 'desktop', width: 1440, height: 1000 },
|
||||
{ name: 'tv', width: 1920, height: 1080 },
|
||||
];
|
||||
|
||||
function valueAfter(args, flag) {
|
||||
const index = args.indexOf(flag);
|
||||
return index >= 0 ? args[index + 1] : undefined;
|
||||
}
|
||||
|
||||
function positive(value, fallback) {
|
||||
const parsed = Number(value);
|
||||
return Number.isInteger(parsed) && parsed > 0 ? parsed : fallback;
|
||||
}
|
||||
|
||||
export function getConfig(argv) {
|
||||
const args = argv.slice(2);
|
||||
const root = path.resolve(valueAfter(args, '--root') || process.cwd());
|
||||
const verifyDir = path.join(root, 'verification');
|
||||
const quick = args.includes('--quick');
|
||||
const offline = args.includes('--offline');
|
||||
return {
|
||||
args,
|
||||
root,
|
||||
verifyDir,
|
||||
quick,
|
||||
offline,
|
||||
keepServer: args.includes('--keep-server'),
|
||||
strict: !args.includes('--non-strict'),
|
||||
referenceUrl: valueAfter(args, '--reference-url') || 'https://kvideo.pages.dev',
|
||||
localUrl: valueAfter(args, '--base-url') || 'http://127.0.0.1:34173',
|
||||
fixtureUrl: 'http://127.0.0.1:34174',
|
||||
containerUrl: 'http://127.0.0.1:34175',
|
||||
localPort: 34173,
|
||||
fixturePort: 34174,
|
||||
containerPort: 34175,
|
||||
maxSourceLines: 150,
|
||||
maxActionStates: quick ? 30 : positive(valueAfter(args, '--max-actions'), 5000),
|
||||
maxActionDepth: quick ? 1 : positive(valueAfter(args, '--max-action-depth'), 8),
|
||||
commandTimeoutMs: quick ? 180_000 : 900_000,
|
||||
navigationTimeoutMs: 30_000,
|
||||
visualDiffRatio: 0.02,
|
||||
maxLcpMs: 2500,
|
||||
maxCls: 0.1,
|
||||
maxLongTaskMs: 500,
|
||||
minVideoAdvanceSeconds: 1.2,
|
||||
viewports: quick ? [viewports[2]] : viewports,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,56 @@
|
||||
import fs from 'node:fs';
|
||||
import { spawn } from 'node:child_process';
|
||||
import { rawPath } from './log.mjs';
|
||||
|
||||
function terminate(child, signal) {
|
||||
if (!child.pid || child.killed) return;
|
||||
try {
|
||||
process.kill(-child.pid, signal);
|
||||
} catch {
|
||||
try { child.kill(signal); } catch { /* process already ended */ }
|
||||
}
|
||||
}
|
||||
|
||||
export function runCommand(ctx, name, command, args = [], options = {}) {
|
||||
const started = Date.now();
|
||||
const outputPath = rawPath(ctx, `${name}.log`);
|
||||
const stream = fs.createWriteStream(outputPath, { flags: 'w' });
|
||||
const env = { ...process.env, ...(options.env || {}) };
|
||||
return new Promise((resolve) => {
|
||||
const child = spawn(command, args, {
|
||||
cwd: options.cwd || ctx.config.root,
|
||||
env,
|
||||
detached: process.platform !== 'win32',
|
||||
stdio: ['ignore', 'pipe', 'pipe'],
|
||||
});
|
||||
let tail = '';
|
||||
let timedOut = false;
|
||||
const capture = (chunk) => {
|
||||
const text = chunk.toString();
|
||||
stream.write(text);
|
||||
tail = `${tail}${text}`.slice(-12_000);
|
||||
if (options.live) process.stdout.write(text);
|
||||
};
|
||||
child.stdout.on('data', capture);
|
||||
child.stderr.on('data', capture);
|
||||
const timeout = setTimeout(() => {
|
||||
timedOut = true;
|
||||
terminate(child, 'SIGTERM');
|
||||
setTimeout(() => terminate(child, 'SIGKILL'), 3000).unref();
|
||||
}, options.timeoutMs || ctx.config.commandTimeoutMs);
|
||||
child.on('error', (error) => {
|
||||
clearTimeout(timeout);
|
||||
stream.end();
|
||||
resolve({ code: 127, error: error.message, tail, outputPath, timedOut, durationMs: Date.now() - started });
|
||||
});
|
||||
child.on('exit', (code, signal) => {
|
||||
clearTimeout(timeout);
|
||||
stream.end();
|
||||
resolve({ code: code ?? 1, signal, tail, outputPath, timedOut, durationMs: Date.now() - started });
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
export async function runNpm(ctx, name, args, options = {}) {
|
||||
return runCommand(ctx, name, 'npm', args, options);
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { redact } from './redact.mjs';
|
||||
|
||||
function safeRunId() {
|
||||
return new Date().toISOString().replace(/[:.]/g, '-');
|
||||
}
|
||||
|
||||
export function createContext(config) {
|
||||
const runId = safeRunId();
|
||||
const artifacts = path.join(config.verifyDir, 'artifacts', runId);
|
||||
const dirs = Object.fromEntries(
|
||||
['raw', 'screenshots', 'diffs', 'traces', 'metrics', 'media'].map((name) => {
|
||||
const target = path.join(artifacts, name);
|
||||
fs.mkdirSync(target, { recursive: true });
|
||||
return [name, target];
|
||||
}),
|
||||
);
|
||||
const ctx = {
|
||||
config,
|
||||
runId,
|
||||
artifacts,
|
||||
dirs,
|
||||
startedAt: new Date().toISOString(),
|
||||
findings: [],
|
||||
events: [],
|
||||
services: [],
|
||||
state: {},
|
||||
};
|
||||
fs.writeFileSync(path.join(artifacts, 'config.json'), JSON.stringify(redact(config), null, 2));
|
||||
return ctx;
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
|
||||
const ignored = new Set(['.git', '.next', '.vercel', '.wrangler', 'node_modules', 'artifacts']);
|
||||
|
||||
export function walk(root, predicate = () => true) {
|
||||
const output = [];
|
||||
const visit = (dir) => {
|
||||
for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
|
||||
if (ignored.has(entry.name)) continue;
|
||||
const full = path.join(dir, entry.name);
|
||||
if (entry.isDirectory()) visit(full);
|
||||
else if (predicate(full)) output.push(full);
|
||||
}
|
||||
};
|
||||
visit(root);
|
||||
return output;
|
||||
}
|
||||
|
||||
export function relative(root, file) {
|
||||
return path.relative(root, file).split(path.sep).join('/');
|
||||
}
|
||||
|
||||
export function lineCount(file) {
|
||||
const text = fs.readFileSync(file, 'utf8');
|
||||
return text.length === 0 ? 0 : text.split(/\r?\n/).length;
|
||||
}
|
||||
|
||||
export function writeJson(file, value) {
|
||||
fs.mkdirSync(path.dirname(file), { recursive: true });
|
||||
fs.writeFileSync(file, JSON.stringify(value, null, 2));
|
||||
}
|
||||
|
||||
export function readJson(file) {
|
||||
return JSON.parse(fs.readFileSync(file, 'utf8'));
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
import { log } from './log.mjs';
|
||||
|
||||
const validStatus = new Set(['PASS', 'FAIL', 'WARN', 'SKIP', 'INFO']);
|
||||
const validSeverity = new Set(['critical', 'high', 'medium', 'low', 'info']);
|
||||
|
||||
export function finding(ctx, input) {
|
||||
const item = {
|
||||
id: input.id,
|
||||
category: input.category || 'general',
|
||||
title: input.title,
|
||||
status: validStatus.has(input.status) ? input.status : 'INFO',
|
||||
severity: validSeverity.has(input.severity) ? input.severity : 'info',
|
||||
expected: input.expected ?? null,
|
||||
actual: input.actual ?? null,
|
||||
reason: input.reason || '',
|
||||
impact: input.impact || '',
|
||||
remediation: input.remediation || '',
|
||||
evidence: input.evidence || [],
|
||||
durationMs: Math.round(input.durationMs || 0),
|
||||
at: new Date().toISOString(),
|
||||
};
|
||||
ctx.findings.push(item);
|
||||
log(ctx, item.status === 'FAIL' ? 'error' : 'info', item.id, item.title, {
|
||||
status: item.status,
|
||||
severity: item.severity,
|
||||
actual: item.actual,
|
||||
reason: item.reason,
|
||||
});
|
||||
return item;
|
||||
}
|
||||
|
||||
export function hasFailures(ctx) {
|
||||
return ctx.findings.some((item) => item.status === 'FAIL');
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
export async function request(url, options = {}) {
|
||||
const started = performance.now();
|
||||
const { timeoutMs = 20_000, ...fetchOptions } = options;
|
||||
try {
|
||||
const response = await fetch(url, {
|
||||
redirect: fetchOptions.redirect || 'follow',
|
||||
...fetchOptions,
|
||||
signal: AbortSignal.timeout(timeoutMs),
|
||||
});
|
||||
const buffer = Buffer.from(await response.arrayBuffer());
|
||||
return {
|
||||
ok: true,
|
||||
status: response.status,
|
||||
durationMs: Math.round(performance.now() - started),
|
||||
headers: Object.fromEntries(response.headers),
|
||||
bytes: buffer.length,
|
||||
body: buffer.toString('utf8', 0, Math.min(buffer.length, 100_000)),
|
||||
};
|
||||
} catch (error) {
|
||||
return {
|
||||
ok: false,
|
||||
status: 0,
|
||||
durationMs: Math.round(performance.now() - started),
|
||||
error: error instanceof Error ? error.message : String(error),
|
||||
headers: {}, bytes: 0, body: '',
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
export function jsonBody(result) {
|
||||
try { return JSON.parse(result.body); } catch { return null; }
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { redact, redactText } from './redact.mjs';
|
||||
|
||||
export function log(ctx, level, event, message, data = {}) {
|
||||
const entry = {
|
||||
at: new Date().toISOString(),
|
||||
level,
|
||||
event,
|
||||
message: redactText(message),
|
||||
data: redact(data),
|
||||
};
|
||||
ctx.events.push(entry);
|
||||
fs.appendFileSync(path.join(ctx.artifacts, 'events.ndjson'), `${JSON.stringify(entry)}\n`);
|
||||
const suffix = Object.keys(entry.data).length ? ` ${JSON.stringify(entry.data)}` : '';
|
||||
fs.appendFileSync(
|
||||
path.join(ctx.artifacts, 'run.log'),
|
||||
`${entry.at} ${level.toUpperCase()} ${event} ${entry.message}${suffix}\n`,
|
||||
);
|
||||
process.stdout.write(`[${level.toUpperCase()}] ${message}\n`);
|
||||
}
|
||||
|
||||
export function rawPath(ctx, name) {
|
||||
return path.join(ctx.dirs.raw, name.replace(/[^a-zA-Z0-9_.-]+/g, '-'));
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
const secretKey = /(password|passwd|token|secret|authorization|cookie|api[-_]?key)/i;
|
||||
const bearer = /\b(Bearer\s+)[A-Za-z0-9._~+/=-]+/gi;
|
||||
const githubToken = /\b(gh[opusr]_[A-Za-z0-9_]{20,})\b/g;
|
||||
const jwt = /\beyJ[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\b/g;
|
||||
|
||||
export function redactText(value) {
|
||||
return String(value)
|
||||
.replace(bearer, '$1[REDACTED]')
|
||||
.replace(githubToken, '[REDACTED_GITHUB_TOKEN]')
|
||||
.replace(jwt, '[REDACTED_JWT]')
|
||||
.replace(/([?&](?:token|key|secret|password)=)[^&#\s]+/gi, '$1[REDACTED]');
|
||||
}
|
||||
|
||||
export function redact(value, key = '') {
|
||||
if (secretKey.test(key)) return '[REDACTED]';
|
||||
if (typeof value === 'string') return redactText(value);
|
||||
if (Array.isArray(value)) return value.map((item) => redact(item));
|
||||
if (!value || typeof value !== 'object') return value;
|
||||
return Object.fromEntries(
|
||||
Object.entries(value).map(([childKey, child]) => [childKey, redact(child, childKey)]),
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
import fs from 'node:fs';
|
||||
import { spawn } from 'node:child_process';
|
||||
import { rawPath } from './log.mjs';
|
||||
|
||||
export async function waitForUrl(url, timeoutMs = 60_000) {
|
||||
const deadline = Date.now() + timeoutMs;
|
||||
let lastError = '';
|
||||
while (Date.now() < deadline) {
|
||||
try {
|
||||
const response = await fetch(url, { redirect: 'manual', signal: AbortSignal.timeout(3000) });
|
||||
if (response.status < 500) return { ok: true, status: response.status };
|
||||
lastError = `HTTP ${response.status}`;
|
||||
} catch (error) {
|
||||
lastError = error instanceof Error ? error.message : String(error);
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 500));
|
||||
}
|
||||
return { ok: false, error: lastError };
|
||||
}
|
||||
|
||||
export async function startProcess(ctx, name, command, args, options = {}) {
|
||||
const outputPath = rawPath(ctx, `${name}.log`);
|
||||
const stream = fs.createWriteStream(outputPath, { flags: 'w' });
|
||||
const child = spawn(command, args, {
|
||||
cwd: options.cwd || ctx.config.root,
|
||||
env: { ...process.env, ...(options.env || {}) },
|
||||
detached: process.platform !== 'win32',
|
||||
stdio: ['ignore', 'pipe', 'pipe'],
|
||||
});
|
||||
child.stdout.pipe(stream);
|
||||
child.stderr.pipe(stream);
|
||||
const service = { name, child, outputPath, stopped: false };
|
||||
ctx.services.push(service);
|
||||
if (options.url) service.ready = await waitForUrl(options.url, options.timeoutMs);
|
||||
return service;
|
||||
}
|
||||
|
||||
export function stopProcess(service) {
|
||||
if (!service?.child?.pid || service.stopped) return;
|
||||
service.stopped = true;
|
||||
try {
|
||||
process.kill(-service.child.pid, 'SIGTERM');
|
||||
} catch {
|
||||
try { service.child.kill('SIGTERM'); } catch { /* already stopped */ }
|
||||
}
|
||||
}
|
||||
|
||||
export function stopAll(ctx) {
|
||||
for (const service of [...ctx.services].reverse()) {
|
||||
if (typeof service.close === 'function') service.close();
|
||||
else stopProcess(service);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
import { finding } from './finding.mjs';
|
||||
import { log } from './log.mjs';
|
||||
|
||||
export async function runStage(ctx, id, title, task) {
|
||||
const started = Date.now();
|
||||
log(ctx, 'info', `stage.${id}.start`, `Starting ${title}`);
|
||||
try {
|
||||
await task(ctx);
|
||||
log(ctx, 'info', `stage.${id}.end`, `Finished ${title}`, { durationMs: Date.now() - started });
|
||||
} catch (error) {
|
||||
finding(ctx, {
|
||||
id: `stage.${id}.crash`,
|
||||
category: 'harness',
|
||||
title: `${title} crashed`,
|
||||
status: 'FAIL',
|
||||
severity: 'critical',
|
||||
expected: 'Stage completes and records granular findings',
|
||||
actual: error instanceof Error ? error.stack || error.message : String(error),
|
||||
reason: 'The verification harness encountered an unhandled exception.',
|
||||
impact: 'Checks in this stage may be incomplete.',
|
||||
remediation: 'Inspect the stack trace and repair the harness before trusting this run.',
|
||||
durationMs: Date.now() - started,
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
export function escapeXml(value) {
|
||||
return String(value ?? '')
|
||||
.replaceAll('&', '&')
|
||||
.replaceAll('<', '<')
|
||||
.replaceAll('>', '>')
|
||||
.replaceAll('"', '"')
|
||||
.replaceAll("'", ''');
|
||||
}
|
||||
@@ -0,0 +1,33 @@
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { runCommand } from '../core/command.mjs';
|
||||
import { finding } from '../core/finding.mjs';
|
||||
|
||||
export async function createMedia(ctx) {
|
||||
const mp4 = path.join(ctx.dirs.media, 'test.mp4');
|
||||
const hls = path.join(ctx.dirs.media, 'hls');
|
||||
fs.mkdirSync(hls, { recursive: true });
|
||||
const generate = await runCommand(ctx, 'fixture-media', 'ffmpeg', [
|
||||
'-hide_banner', '-loglevel', 'error', '-y',
|
||||
'-f', 'lavfi', '-i', 'testsrc=size=640x360:rate=30',
|
||||
'-f', 'lavfi', '-i', 'sine=frequency=440:sample_rate=44100',
|
||||
'-t', '8', '-c:v', 'libx264', '-preset', 'veryfast', '-pix_fmt', 'yuv420p',
|
||||
'-c:a', 'aac', '-b:a', '96k', '-movflags', '+faststart', mp4,
|
||||
], { timeoutMs: 120_000 });
|
||||
const segment = generate.code === 0 ? await runCommand(ctx, 'fixture-hls', 'ffmpeg', [
|
||||
'-hide_banner', '-loglevel', 'error', '-y', '-i', mp4,
|
||||
'-c', 'copy', '-hls_time', '2', '-hls_list_size', '0',
|
||||
'-hls_segment_filename', path.join(hls, 'segment-%03d.seg'), path.join(hls, 'master.m3u8'),
|
||||
], { timeoutMs: 120_000 }) : { code: 1, outputPath: generate.outputPath, durationMs: 0 };
|
||||
const ok = generate.code === 0 && segment.code === 0 && fs.existsSync(mp4);
|
||||
finding(ctx, {
|
||||
id: 'fixture.media', category: 'harness', title: 'Deterministic MP4 and HLS fixtures were generated',
|
||||
status: ok ? 'PASS' : 'FAIL', severity: 'critical', expected: 'Playable 8-second MP4 and HLS assets',
|
||||
actual: ok ? `${fs.statSync(mp4).size} bytes` : `ffmpeg exits ${generate.code}/${segment.code}`,
|
||||
reason: ok ? 'Video checks use locally generated media and do not depend on third-party streams.' : 'Video evidence cannot be produced without deterministic fixtures.',
|
||||
evidence: [generate.outputPath, segment.outputPath], remediation: 'Install a working ffmpeg with H.264/AAC support.',
|
||||
durationMs: generate.durationMs + segment.durationMs,
|
||||
});
|
||||
ctx.state.mediaOk = ok;
|
||||
return { mp4, hls };
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
import http from 'node:http';
|
||||
import path from 'node:path';
|
||||
import { posterSvg, sourceResponse } from './source.mjs';
|
||||
import { sendFile } from './static.mjs';
|
||||
|
||||
function json(response, status, value, headers = {}) {
|
||||
response.writeHead(status, { 'Content-Type': 'application/json', 'Access-Control-Allow-Origin': '*', ...headers });
|
||||
response.end(JSON.stringify(value));
|
||||
}
|
||||
|
||||
export async function startFixtureServer(ctx) {
|
||||
const baseUrl = ctx.config.fixtureUrl;
|
||||
const server = http.createServer(async (request, response) => {
|
||||
const url = new URL(request.url || '/', baseUrl);
|
||||
if (request.method === 'OPTIONS') { response.writeHead(204, { 'Access-Control-Allow-Origin': '*', 'Access-Control-Allow-Headers': '*' }); response.end(); return; }
|
||||
if (url.pathname === '/health' || url.pathname === '/fast') { json(response, 200, { ok: true, at: Date.now() }); return; }
|
||||
if (url.pathname === '/slow') {
|
||||
const delay = Math.min(Number(url.searchParams.get('ms')) || 250, 5000);
|
||||
await new Promise((resolve) => setTimeout(resolve, delay)); json(response, 200, { ok: true, delay }); return;
|
||||
}
|
||||
if (url.pathname.startsWith('/status/')) { const status = Number(url.pathname.split('/').pop()) || 500; json(response, status, { status }); return; }
|
||||
if (url.pathname === '/redirect') { response.writeHead(302, { Location: `${baseUrl}/fast` }); response.end(); return; }
|
||||
if (url.pathname === '/headers') { json(response, 200, { method: request.method, headers: request.headers }); return; }
|
||||
if (url.pathname === '/source') { json(response, 200, sourceResponse(baseUrl, url.searchParams)); return; }
|
||||
if (url.pathname === '/poster.svg') { response.writeHead(200, { 'Content-Type': 'image/svg+xml', 'Access-Control-Allow-Origin': '*' }); response.end(posterSvg(url.searchParams.get('item'))); return; }
|
||||
if (url.pathname === '/test.mp4') { sendFile(request, response, path.join(ctx.dirs.media, 'test.mp4')); return; }
|
||||
if (url.pathname.startsWith('/hls/')) { sendFile(request, response, path.join(ctx.dirs.media, url.pathname.slice(1))); return; }
|
||||
json(response, 404, { error: 'fixture route not found', path: url.pathname });
|
||||
});
|
||||
await new Promise((resolve, reject) => server.once('error', reject).listen(ctx.config.fixturePort, '127.0.0.1', resolve));
|
||||
const service = { name: 'fixture-server', close: () => server.close() };
|
||||
ctx.services.push(service);
|
||||
return service;
|
||||
}
|
||||
@@ -0,0 +1,31 @@
|
||||
export function sourceResponse(baseUrl, searchParams) {
|
||||
const episodes = `第1集$${baseUrl}/test.mp4#第2集$${baseUrl}/hls/master.m3u8`;
|
||||
const id = searchParams.get('ids') || 'fixture-video-1';
|
||||
const query = searchParams.get('wd') || '验证视频';
|
||||
return {
|
||||
code: 1,
|
||||
msg: 'ok',
|
||||
page: 1,
|
||||
pagecount: 1,
|
||||
total: 3,
|
||||
list: [1, 2, 3].map((number) => ({
|
||||
vod_id: number === 1 ? id : `fixture-video-${number}`,
|
||||
vod_name: `${query} ${number}`,
|
||||
vod_pic: `${baseUrl}/poster.svg?item=${number}`,
|
||||
vod_remarks: number === 1 ? '全2集' : '测试内容',
|
||||
vod_year: '2026',
|
||||
vod_area: 'Fixture',
|
||||
vod_actor: 'Automated Validator',
|
||||
vod_director: 'KVideo Verification',
|
||||
vod_content: 'Deterministic content used only by the local verification suite.',
|
||||
type_name: '测试',
|
||||
vod_play_from: 'm3u8',
|
||||
vod_play_url: episodes,
|
||||
})),
|
||||
};
|
||||
}
|
||||
|
||||
export function posterSvg(item = '1') {
|
||||
const safe = String(item).replace(/[^0-9A-Za-z_-]/g, '');
|
||||
return `<svg xmlns="http://www.w3.org/2000/svg" width="400" height="600"><rect width="100%" height="100%" fill="#101827"/><circle cx="200" cy="220" r="90" fill="#4f8cff"/><text x="200" y="390" text-anchor="middle" fill="white" font-family="sans-serif" font-size="36">KVideo ${safe}</text></svg>`;
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
|
||||
const types = {
|
||||
'.mp4': 'video/mp4', '.m3u8': 'application/vnd.apple.mpegurl', '.seg': 'video/mp2t',
|
||||
};
|
||||
|
||||
export function sendFile(request, response, file) {
|
||||
if (!fs.existsSync(file)) { response.writeHead(404); response.end('not found'); return; }
|
||||
const size = fs.statSync(file).size;
|
||||
const range = request.headers.range;
|
||||
const headers = { 'Content-Type': types[path.extname(file)] || 'application/octet-stream', 'Accept-Ranges': 'bytes', 'Access-Control-Allow-Origin': '*' };
|
||||
if (!range) {
|
||||
response.writeHead(200, { ...headers, 'Content-Length': size });
|
||||
fs.createReadStream(file).pipe(response);
|
||||
return;
|
||||
}
|
||||
const match = /^bytes=(\d*)-(\d*)$/.exec(range);
|
||||
if (!match) { response.writeHead(416, { 'Content-Range': `bytes */${size}` }); response.end(); return; }
|
||||
const start = match[1] ? Number(match[1]) : 0;
|
||||
const end = match[2] ? Math.min(Number(match[2]), size - 1) : size - 1;
|
||||
if (start > end || start >= size) { response.writeHead(416, { 'Content-Range': `bytes */${size}` }); response.end(); return; }
|
||||
response.writeHead(206, { ...headers, 'Content-Length': end - start + 1, 'Content-Range': `bytes ${start}-${end}/${size}` });
|
||||
fs.createReadStream(file, { start, end }).pipe(response);
|
||||
}
|
||||
@@ -0,0 +1,68 @@
|
||||
#!/usr/bin/env node
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { getConfig } from './config.mjs';
|
||||
import { createContext } from './core/context.mjs';
|
||||
import { hasFailures } from './core/finding.mjs';
|
||||
import { log } from './core/log.mjs';
|
||||
import { runStage } from './core/stage.mjs';
|
||||
import { stopAll } from './core/service.mjs';
|
||||
import { writeReports } from './report/write.mjs';
|
||||
import { checkPreflight } from './checks/preflight.mjs';
|
||||
import { checkHarnessSelf } from './checks/harness-self.mjs';
|
||||
import { checkSourcePolicy } from './checks/source-policy.mjs';
|
||||
import { checkAstMetrics } from './checks/ast-metrics.mjs';
|
||||
import { checkImportGraph } from './checks/import-graph.mjs';
|
||||
import { checkSecurityScan } from './checks/security-scan.mjs';
|
||||
import { checkDuplicates } from './checks/duplicates.mjs';
|
||||
import { checkStaticTools } from './checks/static-tools.mjs';
|
||||
import { checkDockerLocal } from './checks/docker-local.mjs';
|
||||
import { startRuntime } from './checks/runtime-start.mjs';
|
||||
import { checkApiDiscovery } from './checks/api-discovery.mjs';
|
||||
import { checkApiContracts } from './checks/api-contracts.mjs';
|
||||
import { checkProxy } from './checks/proxy.mjs';
|
||||
import { checkLatency } from './checks/latency.mjs';
|
||||
import { checkSecurityHeaders } from './checks/security-headers.mjs';
|
||||
import { checkUiPages } from './checks/ui-pages.mjs';
|
||||
import { checkUiActions } from './checks/ui-actions.mjs';
|
||||
import { checkPerformance } from './checks/performance.mjs';
|
||||
import { checkVideo } from './checks/video.mjs';
|
||||
import { checkVisual } from './checks/visual.mjs';
|
||||
import { checkDeployment } from './checks/deployment.mjs';
|
||||
|
||||
const config = getConfig(process.argv);
|
||||
const ctx = createContext(config);
|
||||
ctx.state.version = JSON.parse(fs.readFileSync(path.join(config.root, 'package.json'), 'utf8')).version;
|
||||
process.on('SIGINT', () => { stopAll(ctx); process.exit(130); });
|
||||
process.on('SIGTERM', () => { stopAll(ctx); process.exit(143); });
|
||||
|
||||
async function main() {
|
||||
log(ctx, 'info', 'run.start', 'KVideo strict verification started', { runId: ctx.runId, root: config.root });
|
||||
await runStage(ctx, 'preflight', 'preflight and harness integrity', async () => {
|
||||
await checkPreflight(ctx); await checkHarnessSelf(ctx); await checkSourcePolicy(ctx);
|
||||
});
|
||||
await runStage(ctx, 'quality', 'structural quality and security analysis', async () => {
|
||||
await checkAstMetrics(ctx); await checkImportGraph(ctx); await checkSecurityScan(ctx); await checkDuplicates(ctx);
|
||||
});
|
||||
await runStage(ctx, 'static', 'unit, lint, type, dependency, and production builds', checkStaticTools);
|
||||
await runStage(ctx, 'docker', 'local release container', checkDockerLocal);
|
||||
await runStage(ctx, 'runtime', 'deterministic fixtures and local application', startRuntime);
|
||||
await runStage(ctx, 'api', 'API, proxy, latency, headers, and PWA contracts', async () => {
|
||||
await checkApiDiscovery(ctx); await checkApiContracts(ctx); await checkProxy(ctx); await checkLatency(ctx); await checkSecurityHeaders(ctx);
|
||||
});
|
||||
await runStage(ctx, 'browser', 'UI, accessibility, actions, performance, video, and visual parity', async () => {
|
||||
await checkUiPages(ctx); await checkUiActions(ctx); await checkPerformance(ctx); await checkVideo(ctx); await checkVisual(ctx);
|
||||
});
|
||||
await runStage(ctx, 'deployment', 'public release consistency', checkDeployment);
|
||||
}
|
||||
|
||||
try { await main(); }
|
||||
finally {
|
||||
if (!config.keepServer) stopAll(ctx);
|
||||
const counts = writeReports(ctx);
|
||||
const latest = path.join(config.verifyDir, 'artifacts', 'latest');
|
||||
try { if (fs.lstatSync(latest).isSymbolicLink()) fs.unlinkSync(latest); } catch {}
|
||||
try { fs.symlinkSync(ctx.runId, latest, 'dir'); } catch {}
|
||||
log(ctx, 'info', 'run.end', 'KVideo strict verification finished', { counts, report: path.join(ctx.artifacts, 'report.html') });
|
||||
process.exitCode = hasFailures(ctx) ? 1 : 0;
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
import { escapeXml } from '../core/xml.mjs';
|
||||
|
||||
function findingCard(item) {
|
||||
const evidence = item.evidence.map((entry) => `<li>${escapeXml(entry)}</li>`).join('');
|
||||
return `<article class="finding ${item.status.toLowerCase()}">
|
||||
<header><code>${escapeXml(item.id)}</code><b>${item.status}</b><span>${item.severity}</span></header>
|
||||
<h3>${escapeXml(item.title)}</h3>
|
||||
<dl><dt>Expected</dt><dd>${escapeXml(item.expected)}</dd><dt>Actual</dt><dd>${escapeXml(item.actual)}</dd>
|
||||
<dt>Reason</dt><dd>${escapeXml(item.reason)}</dd><dt>Impact</dt><dd>${escapeXml(item.impact)}</dd>
|
||||
<dt>Remediation</dt><dd>${escapeXml(item.remediation)}</dd></dl><ul>${evidence}</ul></article>`;
|
||||
}
|
||||
|
||||
export function renderHtml(ctx, totals) {
|
||||
const cards = ctx.findings.map(findingCard).join('\n');
|
||||
return `<!doctype html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width">
|
||||
<title>KVideo verification ${escapeXml(ctx.runId)}</title><style>
|
||||
:root{font:14px/1.5 system-ui;color:#18202a;background:#f2f5f8}body{margin:0}main{max-width:1200px;margin:auto;padding:24px}
|
||||
h1{margin:.2em 0}.summary{display:grid;grid-template-columns:repeat(auto-fit,minmax(130px,1fr));gap:12px;margin:20px 0}
|
||||
.metric,.finding{background:white;border:1px solid #dce3ea;border-radius:10px;padding:14px;box-shadow:0 2px 7px #0001}.metric b{font-size:24px;display:block}
|
||||
.finding{margin:12px 0;border-left:7px solid #6b7280}.finding.pass{border-left-color:#0a8f50}.finding.fail{border-left-color:#c62828}.finding.warn{border-left-color:#d97706}
|
||||
.finding header{display:flex;gap:12px;align-items:center}.finding header b{margin-left:auto}dl{display:grid;grid-template-columns:110px 1fr;gap:5px 12px}dt{font-weight:700}dd{margin:0;white-space:pre-wrap;overflow-wrap:anywhere}
|
||||
code{overflow-wrap:anywhere}nav{position:sticky;top:0;background:#18202a;color:white;padding:10px 24px}nav a{color:white;margin-right:15px}</style></head>
|
||||
<body><nav><a href="summary.md">Summary</a><a href="findings.json">JSON</a><a href="events.ndjson">Events</a><a href="junit.xml">JUnit</a></nav>
|
||||
<main><h1>KVideo strict verification</h1><p>Run ${escapeXml(ctx.runId)} · started ${escapeXml(ctx.startedAt)}</p>
|
||||
<section class="summary">${Object.entries(totals).map(([key,value]) => `<div class="metric"><b>${value}</b>${key}</div>`).join('')}</section>
|
||||
<p>A pass means the declared check passed. It is not a proof that undiscovered states cannot fail.</p>${cards}</main></body></html>`;
|
||||
}
|
||||
@@ -0,0 +1,45 @@
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import { writeJson } from '../core/files.mjs';
|
||||
import { escapeXml } from '../core/xml.mjs';
|
||||
import { renderHtml } from './html.mjs';
|
||||
|
||||
function totals(findings) {
|
||||
return findings.reduce((sum, item) => {
|
||||
sum[item.status] = (sum[item.status] || 0) + 1;
|
||||
return sum;
|
||||
}, { PASS: 0, FAIL: 0, WARN: 0, SKIP: 0, INFO: 0 });
|
||||
}
|
||||
|
||||
function renderJunit(ctx, counts) {
|
||||
const cases = ctx.findings.map((item) => {
|
||||
const body = item.status === 'FAIL'
|
||||
? `<failure message="${escapeXml(item.reason)}">${escapeXml(JSON.stringify(item, null, 2))}</failure>`
|
||||
: item.status === 'SKIP' ? '<skipped/>' : '';
|
||||
return `<testcase classname="${escapeXml(item.category)}" name="${escapeXml(item.id)}" time="${item.durationMs / 1000}">${body}</testcase>`;
|
||||
}).join('\n');
|
||||
return `<?xml version="1.0"?><testsuite name="kvideo-verification" tests="${ctx.findings.length}" failures="${counts.FAIL}" skipped="${counts.SKIP}">${cases}</testsuite>`;
|
||||
}
|
||||
|
||||
function renderMarkdown(ctx, counts) {
|
||||
const failures = ctx.findings.filter((item) => item.status === 'FAIL');
|
||||
const warnings = ctx.findings.filter((item) => item.status === 'WARN');
|
||||
const rows = [...failures, ...warnings].map((item) =>
|
||||
`| ${item.status} | ${item.severity} | \`${item.id}\` | ${item.title.replaceAll('|', '\\|')} |`,
|
||||
);
|
||||
return `# KVideo verification ${ctx.runId}\n\n` +
|
||||
`PASS ${counts.PASS} · FAIL ${counts.FAIL} · WARN ${counts.WARN} · SKIP ${counts.SKIP} · INFO ${counts.INFO}\n\n` +
|
||||
`A green run proves only the declared checks. Coverage gaps are explicit findings.\n\n` +
|
||||
`| Status | Severity | Check | Result |\n|---|---|---|---|\n${rows.join('\n') || '| PASS | info | — | No failures or warnings |'}\n`;
|
||||
}
|
||||
|
||||
export function writeReports(ctx) {
|
||||
const counts = totals(ctx.findings);
|
||||
const finishedAt = new Date().toISOString();
|
||||
writeJson(path.join(ctx.artifacts, 'findings.json'), ctx.findings);
|
||||
writeJson(path.join(ctx.artifacts, 'summary.json'), { runId: ctx.runId, startedAt: ctx.startedAt, finishedAt, counts });
|
||||
fs.writeFileSync(path.join(ctx.artifacts, 'junit.xml'), renderJunit(ctx, counts));
|
||||
fs.writeFileSync(path.join(ctx.artifacts, 'summary.md'), renderMarkdown(ctx, counts));
|
||||
fs.writeFileSync(path.join(ctx.artifacts, 'report.html'), renderHtml(ctx, counts));
|
||||
return counts;
|
||||
}
|
||||
@@ -0,0 +1,43 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import test from 'node:test';
|
||||
import { numericCandidate, prepareActionState } from '../src/browser/action-state.mjs';
|
||||
import { getConfig } from '../src/config.mjs';
|
||||
import { redact, redactText } from '../src/core/redact.mjs';
|
||||
import { escapeXml } from '../src/core/xml.mjs';
|
||||
|
||||
test('redacts keyed secrets recursively', () => {
|
||||
assert.deepEqual(redact({ nested: { password: 'value', safe: 'visible' } }), {
|
||||
nested: { password: '[REDACTED]', safe: 'visible' },
|
||||
});
|
||||
});
|
||||
|
||||
test('redacts bearer, GitHub, JWT, and query credentials', () => {
|
||||
const token = ['gho', '_abcdefghijklmnopqrstuvwxyz123456'].join('');
|
||||
const raw = `Bearer abc.def ${token} ?token=secret-value`;
|
||||
const result = redactText(raw);
|
||||
assert.doesNotMatch(result, /abcdefghijklmnopqrstuvwxyz|secret-value/);
|
||||
});
|
||||
|
||||
test('escapes XML metacharacters', () => {
|
||||
assert.equal(escapeXml(`<a x="1">Tom & 'Ada'</a>`), '<a x="1">Tom & 'Ada'</a>');
|
||||
});
|
||||
|
||||
test('chooses valid alternative values for numeric and range inputs', () => {
|
||||
assert.equal(numericCandidate({ min: '0', max: '1', value: '0.5', step: '0.01' }), '0');
|
||||
assert.equal(numericCandidate({ min: '10', max: '100', value: '70', step: '1' }), '55');
|
||||
});
|
||||
|
||||
test('maps play and pause controls to deterministic media preconditions', async () => {
|
||||
const modes = [];
|
||||
const page = { evaluate: async (_callback, mode) => modes.push(mode) };
|
||||
await prepareActionState(page, { aria: '播放' });
|
||||
await prepareActionState(page, { aria: 'Pause' });
|
||||
await prepareActionState(page, { aria: '搜索' });
|
||||
assert.deepEqual(modes, ['paused', 'playing']);
|
||||
});
|
||||
|
||||
test('accepts explicit full-run action budgets', () => {
|
||||
const config = getConfig(['node', 'verify', '--root', process.cwd(), '--max-actions', '1234', '--max-action-depth', '7']);
|
||||
assert.equal(config.maxActionStates, 1234);
|
||||
assert.equal(config.maxActionDepth, 7);
|
||||
});
|
||||
@@ -0,0 +1,19 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import fs from 'node:fs';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import test from 'node:test';
|
||||
import { lineCount, readJson, relative, walk, writeJson } from '../src/core/files.mjs';
|
||||
|
||||
test('file helpers inventory and serialize deterministic fixtures', () => {
|
||||
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'kvideo-verify-'));
|
||||
fs.mkdirSync(path.join(root, 'nested'));
|
||||
fs.writeFileSync(path.join(root, 'nested', 'file.txt'), 'one\ntwo\n');
|
||||
const json = path.join(root, 'result.json');
|
||||
writeJson(json, { ok: true });
|
||||
assert.equal(lineCount(path.join(root, 'nested', 'file.txt')), 3);
|
||||
assert.equal(relative(root, path.join(root, 'nested', 'file.txt')), 'nested/file.txt');
|
||||
assert.deepEqual(readJson(json), { ok: true });
|
||||
assert.equal(walk(root, (file) => file.endsWith('.txt')).length, 1);
|
||||
fs.rmSync(root, { recursive: true, force: true });
|
||||
});
|
||||
@@ -0,0 +1,18 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import http from 'node:http';
|
||||
import test from 'node:test';
|
||||
import { jsonBody, request } from '../src/core/http.mjs';
|
||||
|
||||
test('HTTP evidence captures status, headers, body, and timing', async () => {
|
||||
const server = http.createServer((_request, response) => {
|
||||
response.writeHead(201, { 'content-type': 'application/json', 'x-test': 'yes' });
|
||||
response.end('{"ok":true}');
|
||||
});
|
||||
await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve));
|
||||
const address = server.address();
|
||||
const result = await request(`http://127.0.0.1:${address.port}`);
|
||||
assert.equal(result.status, 201);
|
||||
assert.equal(result.headers['x-test'], 'yes');
|
||||
assert.deepEqual(jsonBody(result), { ok: true });
|
||||
await new Promise((resolve) => server.close(resolve));
|
||||
});
|
||||
Reference in New Issue
Block a user