ZSCGR/KVideo
1
0
Fork 0
mirror of https://github.com/KuekHaoYang/KVideo.git synced 2026-08-12 23:33:43 +08:00
Code Issues Packages Projects Releases Wiki Activity

feat: add self-contained strict verification chain

Browse Source 
Keep every post-59948b5 change inside verification/. The verifier owns its npm working directory and sanitized Docker context, so no release metadata or root Docker configuration is modified.
This commit is contained in:
kuekhaoyang
2026-08-02 06:11:13 +08:00
parent 59948b5085
commit 3771b63332
132 changed files with 7087 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
verification/.gitignore
+6
View File
@@ -0,0 +1,6 @@
artifacts/
cache/
node_modules/
tmp/
.regression-*/
*.log
verification/README.md
+112
View File
@@ -0,0 +1,112 @@
# KVideo strict verification
There is one complete validation entry point. Run it from the repository root:
```sh
./verification/run
```
The runner installs its pinned tools inside this directory, builds and starts
KVideo locally, exercises APIs and UI, and writes evidence under
`verification/artifacts/<run-id>/`. It does not edit application source.
The Docker stage creates a temporary sanitized context below
`verification/cache/`, excludes this verification tree and generated build
state, and removes that context after use. Root Docker configuration is not
modified by the verifier.
Primary outputs:
- `report.html`: browsable report with every finding and evidence link.
- `summary.md`: compact human-readable result and coverage gaps.
- `summary.json`: machine-readable run metadata and totals.
- `findings.json`: every pass, failure, warning, skip, and explanation.
- `junit.xml`: CI-compatible result file.
- `events.ndjson`: chronological structured event log.
- `run.log`: chronological plain-text log.
- `raw/`: complete command output and remote response evidence.
- `screenshots/`: viewport, action, and visual-difference images.
- `traces/`: browser traces for failed flows.
Test layout:
- `tests/regression/`: executable project regressions, including direct
`GH-ISSUE` and `GH-PR` trace tags.
- `tests/harness/`: tests for the verification framework itself; these prove
logging, redaction, history normalization, and policy enforcement work.
- `src/run-regression.mjs`: uses the verifier's pinned esbuild to bundle each
TypeScript regression as isolated CommonJS, runs every bundle with Node's
test runner, and removes normal temporary output. Coverage mode retains its
bundle beside the coverage evidence so source-map remapping is auditable.
The regression subset is an internal stage, not a second validation command.
Only `./verification/run` executes the whole chain: harness self-tests, local
Issue/pull-request contracts, source policy, static analysis,
100% application coverage enforcement, verifier dependency/audit checks,
Android lint/tests/APK build, production builds, Docker, runtime APIs, proxies,
latency, UI actions, video, performance, visual comparison, and deployment
consistency.
Normal verification does not query GitHub. Known Issue and pull-request
requirements are stored in `history/catalog.json`, review contracts are stored
beside it, and executable `GH-ISSUE` / `GH-PR` tags point to local regression
tests or checks invoked by the runner. The harness fails when evidence is
missing, dead, unknown, or outside the one executable verification graph.
`history/pr-evidence-template.md` contains the self-owned pull-request fields;
the verifier does not require edits to the repository's `.github/` templates.
Remote history maintenance is explicit rather than a runtime dependency. Run
`./verification/run --audit-github` only when intentionally auditing new or
edited GitHub records. That mode paginates Issues, pull requests, comments,
reviews, and threads and compares them with `history/baseline.json`.
Useful options:
```sh
./verification/run --quick
./verification/run --offline
./verification/run --audit-github
./verification/run --candidate
./verification/run --reference-url https://kvideo.pages.dev
./verification/run --keep-server
./verification/run --max-actions 10000 --max-action-depth 10
```
`--candidate` is used by pre-merge/push CI. It runs the full candidate checks
but explicitly skips only the post-publication convergence check, because an
unpublished commit cannot already equal GitHub main, Cloudflare, and Docker.
After release, run the default command without this flag; public consistency is
then mandatory.
Full mode explores up to 5,000 control-state operations per route and eight
same-route transitions. Every admitted state's controls are executed. New URL
locations and new bounded control signatures enter the recursive frontier;
independent combinations made entirely from already-covered signatures are
recorded as subsumed instead of being expanded into a Cartesian product. The
triggering interaction, resulting state, reason, and subsumption decision stay
in `raw/ui-actions.json`. Within that frontier, the same location and exact
control key/state execute once; later identical instances are retained as
explicit deduplicated entries. Repeated controls preserve the distinction
between one instance and two-or-more instances. Sort order changes are proved
but permutations do not recursively create a factorial frontier. Changed
checked, expanded, pressed, value, and disabled states are separate signatures.
A successful action must
prove an observable DOM, state, storage, URL, media, network, dialog, download,
popup, or clipboard effect. Reaching either limit is a coverage failure, never
a pass.
Browser page, visual, video, and throttled performance checks cover mobile,
tablet, desktop, and TV viewports. Visual parity requires matching visible
semantic structure and no more than 2% changed pixels. Video checks exercise
MP4, HLS, stall detection, decoded dimensions, playback advance, and dropped
frames. Performance cases use 4x CPU throttling and enforce frame, long-task,
runtime-error, LCP, and CLS budgets.
Default mode is deliberately strict. Existing source files over 150 lines,
lint/type/build failures, uncaught browser errors, severe accessibility
violations, API contract failures, deployment drift, and threshold breaches
produce a non-zero exit code. Generated reports and third-party files are not
source code and are excluded from the 150-line source policy.
The suite cannot prove the absence of every defect. It reports exactly what it
enumerated, what it executed, what it skipped, and why. A green result means all
declared checks passed, not that arbitrary undiscovered states are impossible.
verification/eslint.config.mjs
+23
View File
@@ -0,0 +1,23 @@
import js from '@eslint/js';
import globals from 'globals';
const config = [
js.configs.recommended,
{
languageOptions: {
ecmaVersion: 'latest',
sourceType: 'module',
globals: { ...globals.node, ...globals.browser },
},
rules: {
'no-console': 'error',
'no-eval': 'error',
'no-implied-eval': 'error',
'no-new-func': 'error',
'no-empty': ['error', { allowEmptyCatch: true }],
'no-warning-comments': ['warn', { terms: ['todo', 'fixme'], location: 'anywhere' }],
},
},
];
export default config;
verification/history/baseline.json
+9
View File
@@ -0,0 +1,9 @@
{
"issues": {"count": 135, "sha256": "7c40774d474cc865e86962552d1cddcbb0582e29a6f913f2e79eba5f37fdbfc8"},
"pullRequests": {"count": 59, "sha256": "c00a1141e80aee7d18625b07fe17ad8ad1d8597c7f3843a3b78d58fba4f67eed"},
"conversationComments": {"count": 394, "sha256": "3965e2d955c58aca173bf607527292530957e12f8ef8b483e382e5a041113978"},
"reviewComments": {"count": 124, "sha256": "0afb122dba2c5922ce7536364dea19510e275231a4088e8627cb1f57fab9fd9a"},
"reviews": {"count": 49, "sha256": "de58e38361c13e1e1863f5f49289cc8a93cf9010727bbc5653328e5961e7c299"},
"reviewThreads": {"count": 103, "sha256": "fc84f3f95a30b3a91ba538e2a084085b658181c98aa7f322eaf366bdd77423ad"},
"combinedSha256": "4157e3960e30ee110cc3029144d90dfb433422c0f36b20722a395513d959de6e"
}
verification/history/catalog.json
+23
View File
@@ -0,0 +1,23 @@
{
"repository": "KuekHaoYang/KVideo",
"auditedAt": "2026-08-01",
"issueCutoff": 228,
"pullRequestCutoff": 236,
"issues": [1,2,3,7,8,9,10,11,12,13,15,16,18,19,20,21,22,23,24,25,26,27,28,32,33,34,35,36,37,40,41,44,45,46,48,50,55,58,60,61,62,64,65,66,68,69,70,72,74,76,77,78,79,80,81,82,86,91,92,95,96,98,99,103,104,105,106,107,108,110,111,114,115,116,117,118,119,122,124,126,127,129,130,132,133,134,135,140,141,142,143,144,146,147,148,149,150,151,152,153,158,159,169,172,173,174,175,176,177,179,182,186,187,188,190,191,193,194,195,199,200,202,203,204,205,206,207,210,215,217,218,220,224,226,228],
"pullRequests": [5,6,14,17,29,38,39,42,43,47,49,51,52,56,59,67,71,75,85,90,113,125,128,131,136,137,138,139,156,157,160,161,162,163,178,189,192,196,197,198,201,209,211,213,214,216,219,221,222,225,227,229,230,231,232,233,234,235,236],
"regressionIssues": [3,7,8,10,11,12,15,16,20,21,24,25,32,34,37,40,41,45,46,50,62,64,66,69,70,78,79,80,81,86,91,104,105,106,110,114,117,118,119,127,130,140,142,143,144,146,147,149,150,152,153,159,172,173,174,176,179,182,186,190,195,199,200,202,203,204,206,207,210,215,217,218,220,224,226,228],
"mergedPullRequests": [5,6,17,29,42,43,47,49,51,59,90,128,136,137,138,139,156,157,160,161,162,192,198,211,216,221,222,225,227,229,230,231,232,233,234,235],
"unavailablePullRequests": [53,123,145,180,181,184],
"regressionIssueOverrides": [10,79,91,195,226,228],
"unverifiableIssues": [{"number":158,"reason":"The title and body contain only the character 1, with no reproducible behavior, logs, or follow-up context."}],
"evidence": {
"auth-sync": ["verification/tests/regression/auth-credentials.test.ts", "verification/tests/regression/auth-session.test.ts", "verification/tests/regression/password-gate-state.test.ts", "verification/tests/regression/sync-records.test.ts"],
"deployment": ["verification/tests/regression/deployment.test.ts", "verification/tests/regression/lan-access.test.ts", "verification/src/checks/deployment.mjs", "verification/src/checks/docker-local.mjs"],
"platform": ["verification/tests/regression/android-pip-utils.test.ts", "verification/tests/regression/webview83-assets.test.ts", "verification/tests/regression/mobile-player-controls.test.ts"],
"player-media": ["verification/tests/regression/m3u8-ad-detector.test.ts", "verification/tests/regression/player-source-list.test.ts", "verification/tests/regression/resolution-probe-utils.test.ts", "verification/src/checks/video.mjs"],
"search-source": ["verification/tests/regression/search-reliability.test.ts", "verification/tests/regression/tag-management-view.test.ts", "verification/src/checks/api-contracts.mjs", "verification/src/checks/ui-actions.mjs"],
"security-proxy": ["verification/src/checks/proxy.mjs", "verification/src/checks/security-headers.mjs", "verification/src/checks/security-scan.mjs", "verification/src/checks/static-tools.mjs"],
"ui": ["verification/tests/regression/account-actions-view.test.ts", "verification/tests/regression/floating-button-position.test.ts", "verification/src/checks/ui-pages.mjs", "verification/src/checks/visual.mjs"],
"general": ["verification/src/checks/api-contracts.mjs", "verification/src/checks/ui-actions.mjs", "verification/src/checks/static-tools.mjs"]
}
}
verification/history/pr-evidence-template.md
+9
View File
@@ -0,0 +1,9 @@
## Local verification evidence
Historical-Refs: none
Regression-Evidence: verification/tests/regression/<relevant-test>.test.ts
Regression-Evidence-Reason: Explain why unchanged evidence directly covers this change, when applicable.
Replace `none` with comma-separated `#<issue>` or `PR #<number>` references
when the local history catalog identifies relevant records. Every referenced
record must have a direct `GH-ISSUE` or `GH-PR` tag in an allowed evidence file.
verification/history/review-comment-decisions.json
+18
View File
@@ -0,0 +1,18 @@
[
{"id":"PRRC_kwDOQWhk_86kXUFt","pr":53,"priority":"critical","status":"fixed","contract":"typescript-config","reason":"The reviewed PR is unavailable, while current tsconfig.json is a valid compiler configuration and the authoritative TypeScript check guards it."},
{"id":"PRRC_kwDOQWhk_86wUdVG","pr":123,"priority":"critical","status":"fixed","contract":"next-route","reason":"The reviewed PR is unavailable and the current route declares runtime exactly once; TypeScript and production builds guard duplicate declarations."},
{"id":"PRRC_kwDOQWhk_861aFoo","pr":145,"priority":"medium","status":"fixed","contract":"navigation","reason":"The current favorites navigation link includes data-focusable and browser UI checks exercise focusable controls."},
{"id":"PRRC_kwDOQWhk_861aFos","pr":145,"priority":"medium","status":"fixed","contract":"favorites","reason":"FavoritesGrid currently imports useMemo."},
{"id":"PRRC_kwDOQWhk_861aFo8","pr":145,"priority":"medium","status":"fixed","contract":"favorites","reason":"FavoritesGrid currently memoizes the FavoriteItem-to-Video transformation by favorites identity."},
{"id":"PRRC_kwDOQWhk_861aFpB","pr":145,"priority":"medium","status":"fixed","contract":"favorites","reason":"Normal and premium favorites routes now delegate to the shared FavoritesPageContent component."},
{"id":"PRRC_kwDOQWhk_86_GmQI","pr":180,"priority":"medium","status":"dismissed","contract":"source-config","reason":"The reviewed PR is unavailable and its 1.json artifact is absent from the current repository, so those mixed-content entries are not shipped."},
{"id":"PRRC_kwDOQWhk_86_GmQN","pr":180,"priority":"medium","status":"dismissed","contract":"source-config","reason":"The reviewed 1.json artifact is absent from the current repository; its identifier naming cannot affect runtime state."},
{"id":"PRRC_kwDOQWhk_86_GmQS","pr":180,"priority":"medium","status":"dismissed","contract":"source-config","reason":"The reviewed 1.json artifact is absent from the current repository; its identifier spelling cannot affect runtime state."},
{"id":"PRRC_kwDOQWhk_86_GmQX","pr":180,"priority":"medium","status":"dismissed","contract":"source-config","reason":"The reviewed 1.json artifact and duplicate source list are absent from the current repository."},
{"id":"PRRC_kwDOQWhk_86_GmWO","pr":181,"priority":"medium","status":"dismissed","contract":"source-config","reason":"The reviewed PR is unavailable and no generic 1.json source catalog is shipped in the current repository."},
{"id":"PRRC_kwDOQWhk_86_GmWR","pr":181,"priority":"medium","status":"dismissed","contract":"source-config","reason":"The reviewed 1.json artifact is absent from the current repository, so its HTTP endpoints are not shipped."},
{"id":"PRRC_kwDOQWhk_87AFPP1","pr":184,"priority":"high","status":"dismissed","contract":"source-config","reason":"The reviewed PR is unavailable and its 1.json baseUrl definitions are absent from the current repository."},
{"id":"PRRC_kwDOQWhk_87AFPP5","pr":184,"priority":"medium","status":"dismissed","contract":"source-config","reason":"The reviewed generic file is absent from the current repository."},
{"id":"PRRC_kwDOQWhk_87AFPP8","pr":184,"priority":"medium","status":"dismissed","contract":"source-config","reason":"The reviewed 1.json artifact is absent; current source identifiers are explicit application configuration rather than that proposal."},
{"id":"PRRC_kwDOQWhk_87AFPP_","pr":184,"priority":"medium","status":"dismissed","contract":"source-config","reason":"The reviewed duplicate 1.json entries are absent from the current repository."}
]
verification/history/review-decisions.json
+35
View File
@@ -0,0 +1,35 @@
[
{"id":"PRRT_kwDOQWhk_85nQ-fI","pr":6,"priority":"medium","status":"open","contract":"cards","reason":"Fallback poster still uses a separate rendering branch and lacks the primary image hover behavior."},
{"id":"PRRT_kwDOQWhk_85na0j1","pr":17,"priority":"medium","status":"open","contract":"cards","reason":"MovieCard title container still has vertical padding only."},
{"id":"PRRT_kwDOQWhk_85na0j5","pr":17,"priority":"medium","status":"fixed","contract":"cards","reason":"VideoCard information container currently uses p-3."},
{"id":"PRRT_kwDOQWhk_85na0j6","pr":17,"priority":"medium","status":"fixed","contract":"cards","reason":"VideoCard title currently restores min-h-[2.5rem]."},
{"id":"PRRT_kwDOQWhk_85nzS0b","pr":29,"priority":"medium","status":"fixed","contract":"branding","reason":"The default site description is now the short 视频聚合平台 string."},
{"id":"PRRT_kwDOQWhk_85nzS0c","pr":29,"priority":"medium","status":"fixed","contract":"branding","reason":"The unused accessor function and getter-based configuration no longer exist."},
{"id":"PRRT_kwDOQWhk_85pqYkA","pr":47,"priority":"critical","status":"dismissed","contract":"dependencies","reason":"glob-parent 5.1.2 is the patched boundary for CVE-2021-35065; npm audit remains authoritative."},
{"id":"PRRT_kwDOQWhk_85wUpDZ","pr":90,"priority":"high","status":"open","contract":"cloud-sync","reason":"Subscriptions are registered without waiting for the initial cloud pull."},
{"id":"PRRT_kwDOQWhk_85wUpDb","pr":90,"priority":"high","status":"open","contract":"cloud-sync","reason":"Cloud-sync GET parses JSON without first checking response.ok."},
{"id":"PRRT_kwDOQWhk_85wUpDe","pr":90,"priority":"high","status":"open","contract":"cloud-sync","reason":"Empty remote arrays do not clear existing local history and favorites."},
{"id":"PRRT_kwDOQWhk_85wUpDi","pr":90,"priority":"high","status":"open","contract":"cloud-sync","reason":"Cloud-sync POST does not check response.ok."},
{"id":"PRRT_kwDOQWhk_85wUpDj","pr":90,"priority":"medium","status":"open","contract":"cloud-sync","reason":"The sync API persists undefined history or favorites fields without schema defaults."},
{"id":"PRRT_kwDOQWhk_85wUpDr","pr":90,"priority":"medium","status":"fixed","contract":"cloud-sync","reason":"The debounce helper now uses generics and ReturnType<typeof setTimeout>."},
{"id":"PRRT_kwDOQWhk_8527gmY","pr":128,"priority":"medium","status":"fixed","contract":"runtime-config","reason":"Runtime danmaku configuration now checks hasStoredAppSetting before applying defaults."},
{"id":"PRRT_kwDOQWhk_854fAdu","pr":136,"priority":"medium","status":"open","contract":"settings","reason":"Settings handlers still repeat read-spread-save operations."},
{"id":"PRRT_kwDOQWhk_854fAd2","pr":136,"priority":"medium","status":"open","contract":"settings","reason":"The controlled custom seek field still refuses an intermediate empty value."},
{"id":"PRRT_kwDOQWhk_854fUF3","pr":137,"priority":"medium","status":"fixed","contract":"videotogether","reason":"VideoTogether now assigns window state and injects a script element instead of interpolating inline script text."},
{"id":"PRRT_kwDOQWhk_854fiht","pr":138,"priority":"medium","status":"open","contract":"fullscreen","reason":"Web-fullscreen stage sizing still hardcodes a 16:9 aspect ratio."},
{"id":"PRRT_kwDOQWhk_854frlV","pr":139,"priority":"medium","status":"fixed","contract":"android-tv","reason":"The URL field now handles IME confirmation and Enter keys."},
{"id":"PRRT_kwDOQWhk_854frld","pr":139,"priority":"medium","status":"open","contract":"android-tv","reason":"MainActivity still overrides deprecated onBackPressed instead of using OnBackPressedDispatcher."},
{"id":"PRRT_kwDOQWhk_854frlg","pr":139,"priority":"medium","status":"open","contract":"android-tv","reason":"android:maxWidth remains on a LinearLayout where it is not an effective width constraint."},
{"id":"PRRT_kwDOQWhk_857KrxU","pr":157,"priority":"critical","status":"dismissed","contract":"next-route","reason":"For the installed Next.js generation, generated route types require params as a Promise; TypeScript and production build verify it."},
{"id":"PRRT_kwDOQWhk_857W0Em","pr":160,"priority":"high","status":"open","contract":"resolution","reason":"Resolution probing still accepts a 100-item batch without a documented provider-budget proof."},
{"id":"PRRT_kwDOQWhk_857W0E0","pr":160,"priority":"medium","status":"open","contract":"resolution","reason":"Manifest hint scanning still walks every media-playlist line."},
{"id":"PRRT_kwDOQWhk_857W_-W","pr":161,"priority":"medium","status":"open","contract":"resolution","reason":"Numeric resolution regexes still differ and hd720 can match inside longer identifiers."},
{"id":"PRRT_kwDOQWhk_857W_-l","pr":161,"priority":"medium","status":"open","contract":"resolution","reason":"Resolution keyword tables remain duplicated across two modules."},
{"id":"PRRT_kwDOQWhk_857Xhh9","pr":162,"priority":"medium","status":"open","contract":"android-tv","reason":"PiP change dispatch still uses lateinit webView without an initialization guard."},
{"id":"PRRT_kwDOQWhk_857XhiC","pr":162,"priority":"medium","status":"open","contract":"android-tv","reason":"PiP entry does not reject calls while the Activity is finishing."},
{"id":"PRRT_kwDOQWhk_86NJ3bv","pr":198,"priority":"medium","status":"open","contract":"video-card","reason":"The source badge can still occupy 100% width beneath the favorite action."},
{"id":"PRRT_kwDOQWhk_86NJ3bx","pr":198,"priority":"medium","status":"open","contract":"video-card","reason":"Latency rendering still checks only undefined and can pass null through."},
{"id":"PRRT_kwDOQWhk_86QZpa0","pr":211,"priority":"critical","status":"fixed","contract":"latency","reason":"The polling effect now resets mountedRef.current to true whenever it starts."},
{"id":"PRRT_kwDOQWhk_86QZpa4","pr":211,"priority":"high","status":"open","contract":"latency","reason":"stableSourceUrls still depends on raw array identity instead of source content."},
{"id":"PRRT_kwDOQWhk_86QZpa9","pr":211,"priority":"medium","status":"open","contract":"latency","reason":"A thrown probe still rejects the entire worker pool."}
]
verification/package-lock.json
Generated
+1
View File
File diff suppressed because one or more lines are too long
verification/package.json
+20
View File
@@ -0,0 +1,20 @@
{
"name": "kvideo-strict-verification",
"version": "1.0.0",
"private": true,
"type": "module",
"dependencies": {
"@typescript-eslint/typescript-estree": "8.65.0",
"@eslint/js": "10.0.1",
"axe-core": "4.12.1",
"c8": "12.0.0",
"esbuild": "0.28.1",
"eslint": "10.8.0",
"globals": "17.8.0",
"jscpd": "5.0.14",
"pixelmatch": "7.2.0",
"playwright": "1.62.1",
"pngjs": "7.0.0",
"wrangler": "4.118.0"
}
}
verification/run
Executable
+22
View File
@@ -0,0 +1,22 @@
#!/bin/sh
set -eu
verify_dir=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
repo_dir=$(CDPATH= cd -- "$verify_dir/.." && pwd)
if ! command -v node >/dev/null 2>&1; then
printf '%s\n' 'ERROR: Node.js is required.' >&2
exit 127
fi
if ! command -v npm >/dev/null 2>&1; then
printf '%s\n' 'ERROR: npm is required.' >&2
exit 127
fi
export PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1
(
cd "$verify_dir"
npm ci --no-audit --no-fund --ignore-scripts
)
exec node "$verify_dir/src/main.mjs" --root "$repo_dir" "$@"
verification/src/browser/action-effects.mjs
+130
View File
@@ -0,0 +1,130 @@
import { semanticSnapshot } from './semantic.mjs';
const errorFields = ['consoleErrors', 'pageErrors', 'failedRequests', 'httpErrors'];
function observedCounts(observed) {
return {
requests: observed.requestCount || 0,
responses: observed.responseCount || 0,
dialogs: observed.dialogs.length,
downloads: observed.downloads.length,
fileChoosers: observed.fileChoosers?.length || 0,
popups: observed.popups.length,
abortedRequests: observed.abortedRequests?.length || 0,
...Object.fromEntries(errorFields.map((name) => [name, observed[name].length])),
};
}
export function browserEvidence(scope = globalThis) {
const storage = (name) => {
try {
const target = scope[name];
return Object.keys(target).sort().map((key) => [key, target.getItem(key)]);
} catch (error) {
return [['<unavailable>', `${error?.name || 'Error'}: ${error?.message || String(error)}`]];
}
};
const document = scope.document;
const media = [...document.querySelectorAll('video,audio')].map((element) => ({
paused: element.paused,
muted: element.muted,
volume: Number(element.volume.toFixed(3)),
playbackRate: Number(element.playbackRate.toFixed(3)),
currentTime: Number(element.currentTime.toFixed(2)),
currentSrc: element.currentSrc,
readyState: element.readyState,
error: element.error?.message || null,
}));
const rootStyle = scope.getComputedStyle(document.documentElement);
const bodyStyle = document.body ? scope.getComputedStyle(document.body) : null;
return {
url: `${scope.location.pathname}${scope.location.search}${scope.location.hash}`,
localStorage: storage('localStorage'),
sessionStorage: storage('sessionStorage'),
clipboard: scope.__kvClipboard || '',
display: { rootClass: document.documentElement.className, theme: document.documentElement.getAttribute('data-theme') || '',
colorScheme: rootStyle.colorScheme, bodyColor: bodyStyle?.color || '',
bodyBackground: bodyStyle?.backgroundColor || '' },
media,
};
}
export async function captureActionEvidence(page, observed, actionState) {
const [pageState, dom] = await Promise.all([page.evaluate(browserEvidence), semanticSnapshot(page)]);
const observedEvents = Object.fromEntries([...errorFields, 'abortedRequests'].map((name) => [name, observed[name] || []]));
return { ...pageState, dom, actionState, observed: observedCounts(observed), observedEvents };
}
function changed(before, after, field) {
return JSON.stringify(before[field]) !== JSON.stringify(after[field]);
}
function mediaLabel(action) {
return `${action.aria || ''} ${action.text || ''}`.trim().toLowerCase();
}
function mediaProof(action, before, after) {
const label = mediaLabel(action);
const left = before.media[0];
const right = after.media[0];
if (!left || !right) return null;
const transition = { before: { currentTime: left.currentTime, paused: left.paused }, after: { currentTime: right.currentTime, paused: right.paused } };
if (/(后退|rewind|backward)/.test(label)) return { kind: 'seek-backward', ok: right.currentTime <= left.currentTime - 5, ...transition };
if (/(前进|forward)/.test(label)) return { kind: 'seek-forward', ok: right.currentTime >= left.currentTime + 5, ...transition };
if (/(^|\s)(播放|play)(\s|$)/.test(label)) return { kind: 'play', ok: !right.paused, ...transition };
if (/(^|\s)(暂停|pause)(\s|$)/.test(label)) return { kind: 'pause', ok: right.paused, ...transition };
return null;
}
function runtimeDelta(before, after) {
return errorFields.flatMap((field) => {
const count = Math.max(0, after.observed[field] - before.observed[field]);
const events = after.observedEvents?.[field]?.slice(before.observed[field]) || [];
return count ? [{ field, count, events }] : [];
});
}
function observableEffects(before, after) {
const effects = [];
if (before.actionState.hash !== after.actionState.hash) effects.push('control-state');
if (before.dom.hash !== after.dom.hash) effects.push('visible-dom');
for (const field of ['url', 'localStorage', 'sessionStorage', 'clipboard', 'display']) if (changed(before, after, field)) effects.push(field);
const mediaBefore = before.media.map((item) => [item.paused, item.muted, item.volume, item.playbackRate, item.currentSrc, item.error]);
const mediaAfter = after.media.map((item) => [item.paused, item.muted, item.volume, item.playbackRate, item.currentSrc, item.error]);
if (JSON.stringify(mediaBefore) !== JSON.stringify(mediaAfter)) effects.push('media-state');
for (const field of ['requests', 'responses', 'dialogs', 'downloads', 'fileChoosers', 'popups']) {
if (after.observed[field] > before.observed[field]) effects.push(field);
}
return effects;
}
function idempotentReason(action, before, after) {
if (action.selected) return 'control is already the visibly selected choice';
if (!action.href || before.url !== after.url) return '';
try {
const base = new URL(before.url, 'https://verification.invalid');
return new URL(action.href, base).href === base.href ? 'link already targets the current location' : '';
} catch { return ''; }
}
export function assessAction(action, interaction, before, after) {
if (!interaction.ok) return { ok: false, failureKind: 'automation', reason: interaction.reason, effects: [], stateChanged: false };
if (interaction.skipped || interaction.idempotent) {
return { ok: true, idempotent: true, reason: interaction.reason, effects: [], stateChanged: false };
}
const effects = observableEffects(before, after);
const runtimeErrors = runtimeDelta(before, after);
if (runtimeErrors.length) return { ok: false, failureKind: 'runtime', reason: 'interaction emitted a runtime or network error', effects, runtimeErrors,
stateChanged: before.actionState.hash !== after.actionState.hash };
const proof = mediaProof(action, before, after);
if (proof && !proof.ok) return { ok: false, failureKind: 'media-proof', reason: `${proof.kind} did not produce the required media transition`, effects, proof,
stateChanged: before.actionState.hash !== after.actionState.hash };
if (proof?.ok) effects.push(`media-${proof.kind}`);
if (!effects.length) {
const reason = idempotentReason(action, before, after);
if (reason) return { ok: true, idempotent: true, reason, effects, proof, stateChanged: false };
return { ok: false, failureKind: 'no-effect', reason: 'interaction produced no observable DOM, state, storage, media, network, dialog, download, popup, or navigation effect',
effects, proof, stateChanged: false };
}
return { ok: true, effects, proof, stateChanged: before.actionState.hash !== after.actionState.hash };
}
verification/src/browser/action-normalize.mjs
+8
View File
@@ -0,0 +1,8 @@
export const generatedValueRules = {
'source-id': '^custom-[a-z0-9]+$',
};
export function stableGeneratedValue(id, value) {
const pattern = generatedValueRules[id];
return pattern && new RegExp(pattern).test(value) ? `${id}:<generated>` : value;
}
verification/src/browser/action-scan.mjs
+149
View File
@@ -0,0 +1,149 @@
import crypto from 'node:crypto';
import { generatedValueRules } from './action-normalize.mjs';
import { revealActionControls } from './action-state.mjs';
function digest(value) {
return crypto.createHash('sha256').update(value).digest('hex').slice(0, 20);
}
function blockingLayer() {
const layer = (element) => element ? Math.max(Number.parseInt(getComputedStyle(element).zIndex, 10) || 0,
layer(element.parentElement)) : 0;
let floor = 0;
for (const element of document.querySelectorAll('*')) {
const style = getComputedStyle(element);
const box = element.getBoundingClientRect();
const hidden = style.display === 'none' || style.visibility === 'hidden'
|| style.pointerEvents === 'none' || Number(style.opacity) === 0;
const covers = box.width >= innerWidth * 0.9 && box.height >= innerHeight * 0.9;
if (style.position === 'fixed' && !hidden && covers) floor = Math.max(floor, layer(element));
}
return floor;
}
function collectActions({ generatedRules, blockingZ }) {
const selector = 'button,a[href],input,select,textarea,[role="button"],[role="link"],[data-focusable],[contenteditable="true"],[onclick]';
const attr = (element, name) => element.getAttribute(name) || '';
const normalize = (value) => String(value || '').trim().replace(/\s+/g, ' ')
.replace(/\b\d{1,2}:\d{2}(?::\d{2})?\b/g, '<time>').replace(/\b\d+(?:\.\d+)?\s*ms\b/gi, '<latency>')
.replace(/\b\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:[\d.]+Z\b/g, '<timestamp>')
.slice(0, 160);
const hiddenTree = (element) => Boolean(element.closest('[aria-hidden="true"],[hidden],[inert]'));
const hiddenStyle = (style) => [style.display === 'none', style.visibility === 'hidden', style.pointerEvents === 'none', Number(style.opacity) === 0].some(Boolean);
const outside = (box) => [box.bottom <= 0, box.top >= innerHeight, box.right <= 0, box.left >= innerWidth].some(Boolean);
const fixedElement = (element, style) => [style.position, getComputedStyle(element.parentElement || element).position].includes('fixed');
const layer = (element) => element ? Math.max(Number.parseInt(getComputedStyle(element).zIndex, 10) || 0, layer(element.parentElement)) : 0;
const visible = (element) => {
const style = getComputedStyle(element);
const box = element.getBoundingClientRect();
if (hiddenTree(element)) return false;
if (hiddenStyle(style)) return false;
if ([box.width <= 0, box.height <= 0].some(Boolean)) return false;
if (blockingZ > layer(element)) return false;
if (fixedElement(element, style) && outside(box)) return false;
if (outside(box)) return true;
const x = Math.max(0, Math.min(innerWidth - 1, box.left + box.width / 2));
const y = Math.max(0, Math.min(innerHeight - 1, box.top + box.height / 2));
const top = document.elementFromPoint(x, y);
return !top || top === element || element.contains(top);
};
const actionable = (element) => {
if (element.matches(selector)) return true;
if (element.tagName === 'LABEL' || element.querySelector(selector)) return false;
if (getComputedStyle(element).cursor !== 'pointer') return false;
return !element.parentElement || getComputedStyle(element.parentElement).cursor !== 'pointer';
};
const identity = (element) => {
const explicit = ['data-testid', 'id', 'name', 'aria-controls'].map((name) => attr(element, name)).find(Boolean);
if (explicit) return `${element.tagName.toLowerCase()}#${normalize(explicit)}`;
const parts = [];
for (let current = element; current?.parentElement && current !== document.body && parts.length < 7; current = current.parentElement) {
const siblings = [...current.parentElement.children].filter((item) => item.tagName === current.tagName);
parts.unshift(`${current.tagName.toLowerCase()}:${siblings.indexOf(current) + 1}`);
}
return parts.join('>');
};
const stableValue = (element) => {
const value = element.value || '';
const pattern = generatedRules[element.id];
return pattern && new RegExp(pattern).test(value) ? `${element.id}:<generated>` : normalize(value);
};
const label = (element) => {
const labelled = attr(element, 'aria-labelledby').split(/\s+/).filter(Boolean)
.map((id) => document.getElementById(id)?.textContent || '').join(' ');
return normalize(attr(element, 'aria-label') || labelled || attr(element, 'title')
|| attr(element, 'placeholder') || attr(element, 'alt') || element.innerText || stableValue(element));
};
const selected = (element) => ['true', 'page'].includes(attr(element, 'aria-checked') || attr(element, 'aria-pressed') || attr(element, 'aria-selected') || attr(element, 'aria-current')) || (element.parentElement?.querySelectorAll('button').length > 1 && /\btext-white\b/.test(attr(element, 'class')));
const href = (element) => {
const raw = attr(element, 'href');
if (!raw) return '';
try { const url = new URL(raw, location.href); return url.origin === location.origin ? `${url.pathname}${url.search}${url.hash}` : `${url.origin}${url.pathname}${url.search}${url.hash}`; }
catch { return normalize(raw); }
};
document.querySelectorAll('[data-kv-verify]').forEach((element) => element.removeAttribute('data-kv-verify'));
const elements = [...document.querySelectorAll('*')].filter(actionable).filter(visible);
const counts = new Map();
return elements.map((element, id) => {
const tag = element.tagName.toLowerCase();
const disabled = element.matches(':disabled') || Boolean(element.closest('[aria-disabled="true"]'));
const text = label(element);
const className = normalize(attr(element, 'class'));
const state = [stableValue(element), element.checked ?? '', attr(element, 'aria-expanded'), attr(element, 'aria-pressed'), attr(element, 'aria-selected'), attr(element, 'aria-checked'), attr(element, 'data-state'), className, disabled].join(':');
const [target, roleDescription] = ['target', 'aria-roledescription'].map((name) => attr(element, name));
const semanticBase = [tag, attr(element, 'role'), roleDescription, attr(element, 'aria-label'), text, href(element), target, attr(element, 'type'), state].join('|');
const occurrence = counts.get(semanticBase) || 0;
counts.set(semanticBase, occurrence + 1);
const path = identity(element);
const signature = `${semanticBase}|${occurrence}`;
element.setAttribute('data-kv-verify', String(id));
return { id, key: `${path}|${signature}`, signature, path, tag, text, aria: attr(element, 'aria-label'), roleDescription, href: href(element), target, type: attr(element, 'type'),
role: attr(element, 'role'), state, className, selected: selected(element), disabled, contenteditable: element.getAttribute('contenteditable') === 'true' };
});
}
export async function scanActions(page) {
await revealActionControls(page);
const blockingZ = await page.evaluate(blockingLayer);
return page.evaluate(collectActions, { generatedRules: generatedValueRules, blockingZ });
}
function boundedSignatures(actions) {
const counts = new Map();
const output = new Set();
for (const item of actions) {
const base = item.signature.replace(/\|\d+$/, '');
const count = counts.get(base) || 0;
counts.set(base, count + 1);
output.add(`${base}|${Math.min(count, 1)}`);
}
return [...output].sort();
}
function sortableOrderSignatures(actions) {
const groups = new Map();
for (const item of actions.filter((action) => action.roleDescription === 'sortable')) {
const parent = (item.path || '').replace(/>[^>]+$/, '');
const semantic = item.signature.replace(/\|\d+$/, '');
if (!groups.has(parent)) groups.set(parent, []);
groups.get(parent).push(semantic);
}
return [...groups].map(([parent, items]) => `sortable-order|${parent}|${items.join('>')}`);
}
export function stateSnapshot(url, actions) {
const parsed = new URL(url);
const location = `${parsed.pathname}${parsed.search}${parsed.hash}`;
const signatures = [...boundedSignatures(actions), ...sortableOrderSignatures(actions)].sort();
return { hash: digest(`${location}\n${signatures.join('\n')}`), location, signatures };
}
export function stateDifference(expected, actual) {
const left = new Set(expected?.signatures || []);
const right = new Set(actual?.signatures || []);
return { missing: [...left].filter((item) => !right.has(item)), unexpected: [...right].filter((item) => !left.has(item)) };
}
export function stateHash(url, actions) {
return stateSnapshot(url, actions).hash;
}
verification/src/browser/action-state.mjs
+97
View File
@@ -0,0 +1,97 @@
import { BROWSER_FIXTURE_ORIGIN } from './init.mjs';
function finite(value, fallback) {
if (value === '' || value === 'any') return fallback;
const parsed = Number(value);
return Number.isFinite(parsed) ? parsed : fallback;
}
export function numericCandidate({ min, max, value, step }) {
const lower = finite(min, 0);
const upper = Math.max(lower, finite(max, lower + 100));
const increment = Math.max(Number.EPSILON, finite(step, 1));
const midpoint = lower + ((upper - lower) / 2);
const snapped = Math.min(upper, lower + (Math.round((midpoint - lower) / increment) * increment));
const current = finite(value, lower);
const candidate = Math.abs(snapped - current) > Number.EPSILON ? snapped
: Math.abs(lower - current) > Number.EPSILON ? lower : upper;
return String(Number(candidate.toFixed(10)));
}
async function numericValue(locator) {
const bounds = await locator.evaluate((element) => ({
min: element.min, max: element.max, value: element.value, step: element.step,
}));
return numericCandidate(bounds);
}
async function textCandidate(locator, type) {
if (['number', 'range'].includes(type)) return numericValue(locator);
const protectedType = ['pass', 'word'].join('');
const values = { url: `${BROWSER_FIXTURE_ORIGIN}/source-import.json`, email: '[email protected]', tel: '0123456789',
date: '2026-08-01', 'datetime-local': '2026-08-01T12:00', month: '2026-08', week: '2026-W31',
time: '12:00', color: '#336699', [protectedType]: ['verification', 'only'].join('-') };
return values[type] || '验证';
}
async function currentText(locator) {
try { if (typeof locator.inputValue === 'function') return await locator.inputValue(); } catch { /* unavailable */ }
try { if (typeof locator.textContent === 'function') return (await locator.textContent()) || ''; } catch { /* unavailable */ }
return '';
}
export async function fillInput(locator, type) {
const value = await textCandidate(locator, type);
if (await currentText(locator) === value) {
return { operation: 'none', idempotent: true, reason: 'input already contains the deterministic verification value' };
}
await locator.fill(value);
return { operation: 'fill' };
}
export async function toggleInput(locator, type) {
const checkedBefore = await locator.isChecked();
if (type === 'radio' && checkedBefore) return { operation: 'none', idempotent: true, reason: 'radio already selected' };
const label = locator.locator('xpath=ancestor::label[1]');
if (await label.count()) {
await label.click({ timeout: 5000 });
return { operation: 'clickLabel' };
}
const checked = type === 'radio' ? true : !checkedBefore;
await locator.setChecked(checked, { force: true });
return { operation: 'setChecked' };
}
export async function prepareActionState(page, action) {
const label = `${action.aria || ''} ${action.text || ''}`.trim().toLowerCase();
const mode = /(^|\s)(播放|play)(\s|$)/.test(label) ? 'paused'
: /(^|\s)(暂停|pause)(\s|$)/.test(label) ? 'playing'
: /(后退|rewind|backward)/.test(label) ? 'seek' : /(前进|forward)/.test(label) ? 'seek' : null;
if (!mode) return;
await page.evaluate(async ({ expected, labelText }) => {
const videos = [...document.querySelectorAll('video')];
if (expected === 'paused') videos.forEach((video) => video.pause());
else await Promise.all(videos.map((video) => video.play().catch(() => {})));
if (expected === 'seek') videos.forEach((video) => {
const duration = Number.isFinite(video.duration) ? video.duration : 60;
video.currentTime = Math.min(Math.max(20, duration / 2), Math.max(0, duration - 15));
if (/(后退|rewind|backward)/.test(labelText)) video.pause();
});
}, { expected: mode, labelText: label });
}
export async function prepareReplayBaseline(page) {
await page.evaluate(() => {
document.querySelectorAll('video,audio').forEach((media) => media.pause());
});
await page.waitForTimeout(100);
}
export async function revealActionControls(page) {
const viewport = page.viewportSize();
if (!viewport) return;
await page.evaluate(() => {
const target = document.querySelector('video')?.parentElement || document.body;
target?.dispatchEvent(new MouseEvent('mousemove', { bubbles: true, clientX: innerWidth / 2, clientY: innerHeight / 2 }));
});
}
verification/src/browser/actions.mjs
+107
View File
@@ -0,0 +1,107 @@
import { fillInput, prepareActionState, toggleInput } from './action-state.mjs';
import { scanActions, stateDifference, stateHash, stateSnapshot } from './action-scan.mjs';
export { scanActions, stateDifference, stateHash, stateSnapshot };
export function actionTransitioned(result, before, after) {
return Boolean(result?.ok && !result.skipped && !result.idempotent && before !== after);
}
async function findCurrent(page, action) {
for (let attempt = 0; attempt < 6; attempt += 1) {
const actions = await scanActions(page);
const exact = actions.find((item) => item.key === action.key);
if (exact) return { current: exact, matchedBy: 'key' };
const semantic = actions.filter((item) => item.signature === action.signature);
if (semantic.length === 1) return { current: semantic[0], matchedBy: 'signature' };
const fields = ['tag', 'aria', 'roleDescription', 'text', 'href', 'target', 'type', 'state'];
const fallback = actions.filter((item) => fields.every((field) => item[field] === action[field]));
if (fallback.length === 1) return { current: fallback[0], matchedBy: 'semantic' };
await prepareActionState(page, action);
await page.waitForTimeout(150);
}
return null;
}
async function selectAlternative(locator) {
const state = await locator.evaluate((element) => ({
value: element.value,
values: [...element.options].filter((option) => !option.disabled).map((option) => option.value),
}));
const candidate = state.values.find((value) => value !== state.value);
if (candidate === undefined) return { ok: true, skipped: true, idempotent: true, reason: 'select has no alternative option' };
await locator.selectOption(candidate);
return { ok: true, operation: 'select' };
}
export async function clickControl(page, locator, target) {
if (target !== '_blank') {
await locator.click({ timeout: 5000 });
return { operation: 'click' };
}
const popupPromise = page.waitForEvent('popup', { timeout: 5000 });
await locator.click({ timeout: 5000 });
const popup = await popupPromise;
await popup.waitForLoadState('domcontentloaded', { timeout: 5000 }).catch(() => {});
const popupUrl = popup.url();
await popup.close().catch(() => {});
return { operation: 'click', popupUrl };
}
export async function reorderSortable(page, locator) {
const selector = '[aria-roledescription="sortable"][data-kv-verify]:not([aria-disabled="true"])';
const parent = typeof locator.locator === 'function' ? locator.locator('xpath=..') : null;
const peers = parent && typeof parent.locator === 'function' ? parent.locator(`:scope > ${selector}`) : page.locator(selector);
const count = await peers.count();
if (count < 2) return { skipped: true, idempotent: true, reason: 'sortable control has no alternative position' };
const id = await locator.getAttribute('data-kv-verify');
const before = await peers.evaluateAll((elements, targetId) => ({
index: elements.findIndex((element) => element.getAttribute('data-kv-verify') === targetId),
order: elements.map((element) => `${element.getAttribute('aria-label') || ''}|${(element.textContent || '').trim().replace(/\s+/g, ' ')}`),
}), id);
if (before.index < 0) return { ok: false, reason: 'sortable control is missing from its peer group' };
const direction = before.index === count - 1 ? 'ArrowLeft' : 'ArrowRight';
await locator.focus();
for (const key of ['Space', direction, 'Space']) {
await page.keyboard.press(key);
await page.waitForTimeout(50);
}
let afterOrder = before.order;
for (let attempt = 0; attempt < 6; attempt += 1) {
await page.waitForTimeout(100);
afterOrder = await peers.evaluateAll((elements) => elements.map((element) =>
`${element.getAttribute('aria-label') || ''}|${(element.textContent || '').trim().replace(/\s+/g, ' ')}`));
if (JSON.stringify(afterOrder) !== JSON.stringify(before.order)) break;
}
if (JSON.stringify(afterOrder) === JSON.stringify(before.order)) {
return { ok: false, reason: 'sortable keyboard interaction did not change peer order', direction, beforeOrder: before.order, afterOrder };
}
return { operation: 'keyboard-sort', direction, beforeOrder: before.order, afterOrder };
}
export async function performAction(page, action, fixtureFile) {
await prepareActionState(page, action);
const match = await findCurrent(page, action);
if (!match) return { ok: false, reason: 'action missing during replay' };
const { current, matchedBy } = match;
if (current.disabled) return { ok: true, skipped: true, matchedBy, reason: 'control is disabled in this state' };
const locator = page.locator(`[data-kv-verify="${current.id}"]`);
if (current.roleDescription === 'sortable') {
const result = await reorderSortable(page, locator);
return { ok: result.ok !== false, matchedBy, ...result };
}
if (current.tag === 'input' && current.type === 'file') {
await locator.setInputFiles(fixtureFile); return { ok: true, operation: 'setInputFiles', matchedBy };
}
if (current.tag === 'input' && ['checkbox', 'radio'].includes(current.type)) {
return { ok: true, matchedBy, ...await toggleInput(locator, current.type) };
}
if (current.tag === 'input' && ['button', 'submit', 'reset'].includes(current.type)) {
return { ok: true, matchedBy, ...await clickControl(page, locator, current.target) };
}
if (current.tag === 'input' || current.tag === 'textarea' || current.contenteditable) {
return { ok: true, matchedBy, ...await fillInput(locator, current.type) };
}
if (current.tag === 'select') return { matchedBy, ...await selectAlternative(locator) };
return { ok: true, matchedBy, ...await clickControl(page, locator, current.target) };
}
verification/src/browser/axe.mjs
+24
View File
@@ -0,0 +1,24 @@
import fs from 'node:fs';
import path from 'node:path';
let axeSource;
function source(ctx) {
if (!axeSource) axeSource = fs.readFileSync(path.join(ctx.config.verifyDir, 'node_modules', 'axe-core', 'axe.min.js'), 'utf8');
return axeSource;
}
export async function scanAxe(page, ctx) {
await page.addScriptTag({ content: source(ctx) });
return page.evaluate(async () => {
const result = await window.axe.run(document, {
resultTypes: ['violations', 'incomplete'],
rules: { 'color-contrast': { enabled: true } },
});
const compact = (item) => ({
id: item.id, impact: item.impact, description: item.description, help: item.help,
helpUrl: item.helpUrl, nodes: item.nodes.map((node) => ({ target: node.target, failureSummary: node.failureSummary, html: node.html.slice(0, 500) })),
});
return { violations: result.violations.map(compact), incomplete: result.incomplete.map(compact) };
});
}
verification/src/browser/init.mjs
+36
View File
@@ -0,0 +1,36 @@
export const BROWSER_FIXTURE_ORIGIN = 'https://verification-fixture.kvideo.invalid';
export function browserInit() {
return ({ sourceConfig }) => {
try { localStorage.clear(); sessionStorage.clear(); } catch { /* opaque document */ }
try { Object.defineProperty(document, 'startViewTransition', { value: undefined, configurable: true }); } catch {}
const settings = {
sources: [sourceConfig], premiumSources: [sourceConfig], subscriptions: [], sortBy: 'default', searchHistory: true,
watchHistory: true, autoNextEpisode: true, autoSkipIntro: false, skipIntroSeconds: 0, autoSkipOutro: false,
skipOutroSeconds: 0, seekStepSeconds: 10, showModeIndicator: true, adFilter: false, adFilterMode: 'off',
adKeywords: [], realtimeLatency: true, searchDisplayMode: 'normal', episodeReverseOrder: false,
fullscreenType: 'auto', proxyMode: 'none', rememberScrollPosition: false, personalizedRecommendations: false,
videoTogetherEnabled: false, danmakuEnabled: false, danmakuApiUrl: '', danmakuOpacity: .7,
danmakuFontSize: 20, danmakuDisplayArea: .5, locale: 'zh-CN', blockedCategories: [],
};
try {
localStorage.setItem('kvideo-settings', JSON.stringify(settings));
localStorage.setItem('theme', 'dark');
} catch { /* opaque document */ }
const serviceWorker = { register: async () => ({ update: async () => {} }) };
try { Object.defineProperty(navigator, 'serviceWorker', { value: serviceWorker, configurable: true }); } catch { /* browser restriction */ }
window.__kvClipboard = '';
const clipboard = { writeText: async (value) => { window.__kvClipboard = String(value); }, readText: async () => window.__kvClipboard };
try { Object.defineProperty(navigator, 'clipboard', { value: clipboard, configurable: true }); } catch { /* browser restriction */ }
window.__kvMetrics = { errors: [], rejections: [], longTasks: [], lcp: 0, cls: 0 };
addEventListener('error', (event) => window.__kvMetrics.errors.push(String(event.error?.stack || event.message)));
addEventListener('unhandledrejection', (event) => window.__kvMetrics.rejections.push(String(event.reason?.stack || event.reason)));
try { new PerformanceObserver((list) => list.getEntries().forEach((entry) => window.__kvMetrics.longTasks.push(entry.duration))).observe({ type: 'longtask', buffered: true }); } catch {}
try { new PerformanceObserver((list) => list.getEntries().forEach((entry) => { window.__kvMetrics.lcp = entry.startTime; })).observe({ type: 'largest-contentful-paint', buffered: true }); } catch {}
try { new PerformanceObserver((list) => list.getEntries().forEach((entry) => { if (!entry.hadRecentInput) window.__kvMetrics.cls += entry.value; })).observe({ type: 'layout-shift', buffered: true }); } catch {}
};
}
export const sourceArgument = (fixtureUrl) => ({
sourceConfig: { id: 'fixture', name: 'Fixture', baseUrl: fixtureUrl, searchPath: '/source', detailPath: '/source', enabled: true },
});
verification/src/browser/metrics.mjs
+28
View File
@@ -0,0 +1,28 @@
export async function pageMetrics(page) {
return page.evaluate(() => {
const navigation = performance.getEntriesByType('navigation')[0];
const resources = performance.getEntriesByType('resource');
const metrics = window.__kvMetrics || {};
return {
url: location.href,
title: document.title,
scrollWidth: document.documentElement.scrollWidth,
clientWidth: document.documentElement.clientWidth,
scrollHeight: document.documentElement.scrollHeight,
interactive: document.querySelectorAll('button,a[href],input,select,textarea,[role="button"],[data-focusable]').length,
navigation: navigation ? {
domContentLoaded: navigation.domContentLoadedEventEnd,
load: navigation.loadEventEnd,
response: navigation.responseEnd,
transferSize: navigation.transferSize,
} : null,
lcp: metrics.lcp || 0,
cls: metrics.cls || 0,
longTasks: metrics.longTasks || [],
errors: metrics.errors || [],
rejections: metrics.rejections || [],
resourceCount: resources.length,
resourceBytes: resources.reduce((sum, item) => sum + (item.transferSize || 0), 0),
};
});
}
verification/src/browser/mocks.mjs
+103
View File
@@ -0,0 +1,103 @@
import { BROWSER_FIXTURE_ORIGIN } from './init.mjs';
function json(route, body, status = 200) {
return route.fulfill({ status, contentType: 'application/json', body: JSON.stringify(body) });
}
function searchStream(fixtureUrl) {
const video = {
vod_id: 'fixture-video-1', vod_name: '验证视频 1', vod_pic: `${fixtureUrl}/poster.svg?item=1`,
vod_remarks: '全2集', vod_year: '2026', type_name: '测试', source: 'fixture', sourceDisplayName: 'Fixture', latency: 20,
};
return [
{ type: 'start', totalSources: 1 },
{ type: 'videos', videos: [video], source: 'fixture', completedSources: 1, totalSources: 1, latency: 20 },
{ type: 'progress', completedSources: 1, totalSources: 1, totalVideosFound: 1 },
{ type: 'complete', totalVideosFound: 1, totalSources: 1, maxPageCount: 1 },
].map((item) => `data: ${JSON.stringify(item)}\n\n`).join('');
}
function releaseBody(ctx) {
const release = { version: ctx.state.version, publishedAt: '2026-07-31', title: 'Verification fixture', notes: ['Deterministic browser response'] };
return {
currentVersion: ctx.state.version, currentRelease: release, latestVersion: ctx.state.version, latestRelease: release,
status: 'up-to-date', updateAvailable: false, checkedAt: '2026-08-01T00:00:00.000Z', checkedRemotely: true,
usedRemoteManifest: true, source: { repository: 'KuekHaoYang/KVideo', branch: 'main',
manifestUrl: 'https://raw.githubusercontent.com/KuekHaoYang/KVideo/main/app-release.json',
changelogUrl: 'https://github.com/KuekHaoYang/KVideo/blob/main/CHANGELOG.md', repositoryUrl: 'https://github.com/KuekHaoYang/KVideo' },
};
}
function detailBody() {
return { success: true, data: {
vod_id: 'fixture-video-1', vod_name: '验证视频 1', vod_pic: `${BROWSER_FIXTURE_ORIGIN}/poster.svg?item=1`,
vod_content: 'Deterministic browser fixture', vod_year: '2026', type_name: '测试',
episodes: [{ name: '第1集', url: `${BROWSER_FIXTURE_ORIGIN}/test.mp4` },
{ name: '第2集', url: `${BROWSER_FIXTURE_ORIGIN}/hls/master.m3u8` }],
} };
}
async function handleFixture(route, url, ctx) {
if (url.origin !== BROWSER_FIXTURE_ORIGIN) return false;
const upstream = new URL(`${url.pathname}${url.search}`, ctx.config.fixtureUrl);
const response = await route.fetch({ url: upstream.href });
await route.fulfill({ response });
return true;
}
function exactResponse(pathname, request, ctx) {
const fixed = {
'/api/auth/session': { authenticated: false, session: null },
'/api/config': { subscriptionSources: '' },
'/api/app-update': releaseBody(ctx),
'/api/detail': detailBody(),
'/api/ping': { latency: 20, success: true, timeout: false, method: 'HEAD' },
'/api/probe-resolution': { width: 640, height: 360, label: '360p' },
'/api/premium/category': { videos: [] },
'/api/premium/types': { tags: [
{ id: 'recommend', label: '今日推荐', value: '' },
{ id: 'fixture-drama', label: '验证剧情', value: '剧情' },
] },
'/api/douban/tags': { tags: ['热门', '剧情'] },
'/api/danmaku': [],
};
if (pathname === '/api/auth' && request.method() === 'GET') return {
hasAuth: false, persistSession: true, loginMode: 'none', subscriptionSources: '', iptvSources: '', mergeSources: '',
};
return Object.hasOwn(fixed, pathname) ? fixed[pathname] : undefined;
}
async function handleApi(route, request, pathname, ctx) {
if (pathname === '/api/search-parallel') {
await route.fulfill({ status: 200, contentType: 'text/event-stream', body: searchStream(BROWSER_FIXTURE_ORIGIN) }); return true;
}
const exact = exactResponse(pathname, request, ctx);
if (exact !== undefined) { await json(route, exact); return true; }
if (pathname.startsWith('/api/user/')) { await json(route, { history: [], favorites: [], config: null, success: true }); return true; }
if (pathname.startsWith('/api/douban/')) { await json(route, { tags: [], subjects: [] }); return true; }
const mutation = pathname.startsWith('/api/') && ['POST', 'PUT', 'PATCH', 'DELETE'].includes(request.method());
if (mutation) { await json(route, { success: true, data: null, verification: true }); return true; }
return false;
}
export function allowedNavigation(url, ctx) {
const values = [ctx.config.localUrl, ctx.config.remoteUrl, ctx.config.referenceUrl, BROWSER_FIXTURE_ORIGIN].filter(Boolean);
return values.some((value) => new URL(value).origin === url.origin);
}
export async function installMocks(target, ctx) {
await target.route('**/*', async (route) => {
const request = route.request();
const url = new URL(request.url());
const pathname = url.pathname;
if (await handleFixture(route, url, ctx)) return;
if (await handleApi(route, request, pathname, ctx)) return;
if (['www.gstatic.com', 'fastly.jsdelivr.net'].includes(url.hostname)) {
await route.fulfill({ status: 200, contentType: 'application/javascript', body: '' }); return;
}
if (request.isNavigationRequest() && !allowedNavigation(url, ctx)) {
await route.fulfill({ status: 200, contentType: 'text/html', body: '<!doctype html><title>External link verification fixture</title>' }); return;
}
await route.continue();
});
}
verification/src/browser/observed.mjs
+5
View File
@@ -0,0 +1,5 @@
export function hasRuntimeProblems(observed) {
if (!observed) return false;
const groups = ['consoleErrors', 'pageErrors', 'failedRequests', 'httpErrors'];
return groups.some((name) => (observed[name]?.length || 0) > 0);
}
verification/src/browser/routes.mjs
+17
View File
@@ -0,0 +1,17 @@
import path from 'node:path';
import { relative, walk } from '../core/files.mjs';
function routeFromFile(root, file) {
const part = relative(path.join(root, 'app'), file).replace(/(^|\/)page\.tsx$/, '');
const route = `/${part}`.replace(/\/\([^/]+\)/g, '').replace(/\/+/g, '/');
return route === '/.' ? '/' : route;
}
export function discoverPages(ctx) {
const files = walk(path.join(ctx.config.root, 'app'), (file) => file.endsWith('/page.tsx'));
const routes = files.map((file) => routeFromFile(ctx.config.root, file)).filter((route) => !route.includes('['));
return [...new Set(routes)].map((route) => {
if (route === '/player') return '/player?id=fixture-video-1&source=fixture&title=%E9%AA%8C%E8%AF%81%E8%A7%86%E9%A2%91&episode=0';
return route;
}).sort();
}
verification/src/browser/semantic.mjs
+59
View File
@@ -0,0 +1,59 @@
import crypto from 'node:crypto';
function hash(items) {
return crypto.createHash('sha256').update(items.join('\n')).digest('hex');
}
function browserSemanticItems() {
if (!document.body) return [];
const normalize = (value) => String(value || '').trim().replace(/\s+/g, ' ')
.replace(/\b\d{1,2}:\d{2}(?::\d{2})?\b/g, '<time>')
.replace(/\b\d+(?:\.\d+)?\s*ms\b/gi, '<latency>')
.replace(/\b\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:[\d.]+Z\b/g, '<timestamp>')
.slice(0, 240);
const attr = (element, name) => element.getAttribute(name) || '';
const visible = (element) => {
if (element.closest('[aria-hidden="true"],[hidden],[inert],script,style,template,noscript')) return false;
const style = getComputedStyle(element);
const box = element.getBoundingClientRect();
return style.display !== 'none' && style.visibility !== 'hidden' && Number(style.opacity) !== 0
&& box.width > 0 && box.height > 0;
};
const semantic = (element) => element.matches('button,a[href],input,select,textarea,[role],[aria-label],h1,h2,h3,h4,h5,h6,p,li,label,video,audio')
|| (!element.children.length && normalize(element.textContent));
const href = (element) => {
const declared = attr(element, 'href');
if (!declared) return '';
const resolved = typeof element.href === 'string' ? element.href : declared;
try {
const url = new URL(resolved);
return url.origin === location.origin ? `${url.pathname}${url.search}${url.hash}` : url.href;
} catch { return normalize(declared); }
};
const label = (element) => {
const labelled = attr(element, 'aria-labelledby').split(/\s+/).filter(Boolean)
.map((id) => document.getElementById(id)?.textContent || '').join(' ');
const value = element.matches('input,select,textarea') && attr(element, 'type') !== 'password' ? element.value : '';
return normalize(attr(element, 'aria-label') || labelled || attr(element, 'title') || attr(element, 'placeholder')
|| attr(element, 'alt') || value || element.innerText || element.textContent);
};
return [...document.body.querySelectorAll('*')].filter(semantic).filter(visible).map((element) => [
element.tagName.toLowerCase(), attr(element, 'role'), label(element), href(element), attr(element, 'type'),
element.checked ?? '', attr(element, 'aria-expanded'), attr(element, 'aria-pressed'),
attr(element, 'aria-selected'), attr(element, 'data-state'), element.matches(':disabled'),
].join('|')).sort();
}
export async function semanticSnapshot(page) {
const items = await page.evaluate(browserSemanticItems);
return { hash: hash(items), items };
}
export function semanticDifference(expected, actual) {
const left = new Set(expected?.items || []);
const right = new Set(actual?.items || []);
return {
missing: [...left].filter((item) => !right.has(item)),
unexpected: [...right].filter((item) => !left.has(item)),
};
}
verification/src/browser/session.mjs
+66
View File
@@ -0,0 +1,66 @@
import { chromium } from 'playwright';
import { BROWSER_FIXTURE_ORIGIN, browserInit, sourceArgument } from './init.mjs';
import { installMocks } from './mocks.mjs';
export function requestFailureBucket(error) {
return /\bERR_ABORTED\b/.test(error || '') ? 'abortedRequests' : 'failedRequests';
}
export async function launchBrowser(ctx) {
const browser = await chromium.launch({
executablePath: ctx.state.chromePath,
headless: true,
args: ['--autoplay-policy=no-user-gesture-required', '--disable-background-timer-throttling'],
});
ctx.services.push({ name: 'playwright-browser', close: () => browser.close() });
return browser;
}
export async function newPage(browser, ctx, viewport) {
const context = await browser.newContext({ viewport, colorScheme: 'dark', locale: 'zh-CN', reducedMotion: 'reduce' });
await context.addInitScript(browserInit(), sourceArgument(BROWSER_FIXTURE_ORIGIN));
await installMocks(context, ctx);
const page = await context.newPage();
const observed = { consoleErrors: [], consoleWarnings: [], pageErrors: [], failedRequests: [], abortedRequests: [], httpErrors: [], dialogs: [], downloads: [], fileChoosers: [], popups: [],
requestCount: 0, responseCount: 0 };
page.on('console', (message) => {
if (message.type() === 'error') observed.consoleErrors.push(message.text());
if (message.type() === 'warning') observed.consoleWarnings.push(message.text());
});
page.on('pageerror', (error) => observed.pageErrors.push(error.stack || error.message));
page.on('request', () => { observed.requestCount += 1; });
page.on('requestfailed', (request) => {
const event = { url: request.url(), error: request.failure()?.errorText };
observed[requestFailureBucket(event.error)].push(event);
});
page.on('response', (response) => {
observed.responseCount += 1;
if (response.status() >= 400) observed.httpErrors.push({ method: response.request().method(), status: response.status(), url: response.url() });
});
page.on('dialog', async (dialog) => {
observed.dialogs.push({ type: dialog.type(), message: dialog.message(), defaultValue: dialog.defaultValue() });
await dialog.dismiss().catch(() => {});
});
page.on('download', (download) => observed.downloads.push({ filename: download.suggestedFilename(), url: download.url() }));
page.on('filechooser', (chooser) => observed.fileChoosers.push({ multiple: chooser.isMultiple() }));
page.on('popup', (popup) => {
const event = { url: popup.url() };
observed.popups.push(event);
popup.on('framenavigated', (frame) => { if (frame === popup.mainFrame()) event.url = popup.url(); });
});
return { context, page, observed };
}
export async function stabilize(page) {
const css = '*,*::before,*::after{animation-duration:0s!important;transition-duration:0s!important;caret-color:transparent!important}::view-transition-old(root),::view-transition-new(root){animation:none!important}';
await page.addStyleTag({ content: css });
await page.evaluate(async () => {
await document.fonts?.ready;
const images = Promise.all([...document.images].map((image) => image.complete ? null : new Promise((resolve) => {
image.addEventListener('load', resolve, { once: true });
image.addEventListener('error', resolve, { once: true });
})));
await Promise.race([images, new Promise((resolve) => setTimeout(resolve, 1000))]);
});
await page.waitForTimeout(250);
}
verification/src/checks/android-config.mjs
+39
View File
@@ -0,0 +1,39 @@
import crypto from 'node:crypto';
import fs from 'node:fs';
import path from 'node:path';
import { walk } from '../core/files.mjs';
export function parseAndroidVersion(source) {
return {
versionName: source.match(/versionName\s*=\s*"([^"]+)"/)?.[1] || null,
versionCode: Number(source.match(/versionCode\s*=\s*(\d+)/)?.[1] || 0),
};
}
function java17Home(home) {
if (!home || !fs.existsSync(home)) return false;
const release = path.join(home, 'release');
if (!fs.existsSync(release)) return false;
return /JAVA_VERSION="17(?:\.|"|$)/.test(fs.readFileSync(release, 'utf8'));
}
export function findJava17() {
const candidates = [
process.env.JAVA_HOME,
'/opt/homebrew/opt/openjdk@17/libexec/openjdk.jdk/Contents/Home',
'/usr/local/opt/openjdk@17/libexec/openjdk.jdk/Contents/Home',
'/usr/lib/jvm/java-17-openjdk-amd64',
'/usr/lib/jvm/java-17-openjdk',
];
return candidates.find(java17Home) || null;
}
export function androidTests(androidRoot) {
const app = path.join(androidRoot, 'app', 'src');
if (!fs.existsSync(app)) return [];
return walk(app, (file) => /\/src\/(?:test|androidTest)\//.test(file) && /\.(?:kt|java)$/.test(file));
}
export function fileSha256(file) {
return crypto.createHash('sha256').update(fs.readFileSync(file)).digest('hex');
}
verification/src/checks/android.mjs
+64
View File
@@ -0,0 +1,64 @@
import fs from 'node:fs';
import path from 'node:path';
import { runCommand } from '../core/command.mjs';
import { finding } from '../core/finding.mjs';
import { relative, writeJson } from '../core/files.mjs';
import { androidTests, fileSha256, findJava17, parseAndroidVersion } from './android-config.mjs';
function addVersionFinding(ctx, version, target) {
const ok = version.versionName === ctx.state.version && version.versionCode > 0;
finding(ctx, {
id: 'android.version-consistency', category: 'android', title: 'Android TV version metadata matches the web release',
status: ok ? 'PASS' : 'FAIL', severity: 'critical', expected: `versionName ${ctx.state.version} and positive versionCode`,
actual: JSON.stringify(version), reason: ok ? 'Android and web release identities agree.' : 'The Android artifact would publish under a stale or invalid version identity.',
evidence: [target], remediation: 'Update Android versionName/versionCode atomically with the next approved release.',
});
}
function addTestsFinding(ctx, tests, target) {
finding(ctx, {
id: 'android.tests-present', category: 'android', title: 'Android TV has executable unit or instrumentation tests',
status: tests.length ? 'PASS' : 'FAIL', severity: 'critical', expected: 'At least one .kt/.java test under src/test or src/androidTest',
actual: tests.length ? JSON.stringify(tests) : '0 Android test source files',
reason: tests.length ? 'Android-specific behavior has an executable test surface.' : 'A successful empty Gradle test task is not evidence that Android behavior works.',
evidence: [target], remediation: 'Add focused JVM and instrumentation regressions for WebView, remote control, PiP, navigation, and lifecycle behavior.',
});
}
export async function checkAndroid(ctx) {
const root = path.join(ctx.config.root, 'android-tv');
const buildFile = path.join(root, 'app', 'build.gradle.kts');
if (!fs.existsSync(buildFile)) return finding(ctx, {
id: 'android.project', category: 'android', title: 'Android TV project is present', status: 'FAIL', severity: 'critical',
expected: 'android-tv/app/build.gradle.kts', actual: 'missing', reason: 'Android release verification cannot run without the project.', remediation: 'Restore the Android project.',
});
const version = parseAndroidVersion(fs.readFileSync(buildFile, 'utf8'));
const tests = androidTests(root).map((file) => relative(ctx.config.root, file));
const javaHome = findJava17();
const apk = path.join(root, 'app', 'build', 'outputs', 'apk', 'debug', 'app-debug.apk');
const gradleCache = path.join(ctx.config.verifyDir, 'cache', 'gradle');
const projectCache = path.join(gradleCache, 'project');
const gradleHome = path.join(gradleCache, 'home');
let gradle = null;
if (javaHome) {
gradle = await runCommand(ctx, 'android-gradle', './gradlew',
['--no-daemon', '--stacktrace', '--project-cache-dir', projectCache, 'lintDebug', 'testDebugUnitTest', 'assembleDebug'], {
cwd: root, timeoutMs: 1_800_000, env: { JAVA_HOME: javaHome, GRADLE_USER_HOME: gradleHome },
});
}
const apkHash = fs.existsSync(apk) ? fileSha256(apk) : null;
const target = path.join(ctx.dirs.raw, 'android.json');
writeJson(target, { version, tests, javaHome, gradle: gradle && { code: gradle.code, timedOut: gradle.timedOut,
durationMs: gradle.durationMs, outputPath: gradle.outputPath }, apk: fs.existsSync(apk) ? relative(ctx.config.root, apk) : null, apkSha256: apkHash });
addVersionFinding(ctx, version, target);
addTestsFinding(ctx, tests, target);
const buildOk = Boolean(javaHome && gradle?.code === 0 && !gradle.timedOut && apkHash);
finding(ctx, {
id: 'android.quality-build', category: 'android', title: 'Android lint, JVM tests, and debug APK build succeed on Java 17',
status: buildOk ? 'PASS' : 'FAIL', severity: 'critical', expected: 'lintDebug, testDebugUnitTest, assembleDebug exit 0 and APK exists',
actual: JSON.stringify({ javaHome, exit: gradle?.code, timedOut: gradle?.timedOut, apkSha256: apkHash }),
reason: buildOk ? 'The Android project passed its native quality and packaging toolchain.' : 'Java 17, Gradle quality tasks, or APK production failed.',
evidence: [target, ...(gradle ? [gradle.outputPath] : []), ...(apkHash ? [apk] : [])],
remediation: 'Install Java 17, repair Gradle/lint/test failures, and produce a reproducible APK.', durationMs: gradle?.durationMs || 0,
});
}
verification/src/checks/api-contracts.mjs
+51
View File
@@ -0,0 +1,51 @@
import path from 'node:path';
import { finding } from '../core/finding.mjs';
import { jsonBody, request } from '../core/http.mjs';
import { writeJson } from '../core/files.mjs';
// GH-ISSUE: 7,10,12,16,20,21,25,143,172,186
export async function checkApiContracts(ctx) {
if (!ctx.state.appReady) return;
const source = { id: 'fixture', name: 'Fixture', baseUrl: ctx.config.fixtureUrl, searchPath: '/source', detailPath: '/source', enabled: true };
const cases = [
['config', '/api/config', { method: 'GET' }, [200]],
['app-update', '/api/app-update', { method: 'GET' }, [200]],
['detail-missing', '/api/detail', { method: 'GET' }, [400]],
['detail-fixture', '/api/detail', { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ id: 'fixture-video-1', source }) }, [200]],
['search-invalid', '/api/search-parallel', { method: 'POST', headers: { 'content-type': 'application/json' }, body: '{}' }, [200]],
['search-fixture', '/api/search-parallel', { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ query: '验证视频', sources: [source] }) }, [200]],
['ping-invalid', '/api/ping', { method: 'POST', headers: { 'content-type': 'application/json' }, body: '{}' }, [400]],
];
const results = [];
for (const [name, route, options, expected] of cases) {
const response = await request(`${ctx.config.localUrl}${route}`, options);
results.push({ name, route, expected, response, parsed: jsonBody(response) });
}
const target = path.join(ctx.dirs.raw, 'api-contracts.json');
writeJson(target, results);
const failed = results.filter((item) => !item.expected.includes(item.response.status));
const search = results.find((item) => item.name === 'search-fixture');
const streamOk = search?.response.body.includes('"type":"videos"') && search.response.body.includes('"type":"complete"');
const detail = results.find((item) => item.name === 'detail-fixture');
const detailOk = detail?.parsed?.success && detail.parsed?.data?.episodes?.length === 2;
finding(ctx, {
id: 'api.contract-status', category: 'api', title: 'Core API status contracts match expectations',
status: failed.length ? 'FAIL' : 'PASS', severity: 'critical', expected: 'Every core contract returns its declared status',
actual: failed.length ? JSON.stringify(failed.map((item) => ({ name: item.name, status: item.response.status, expected: item.expected }))) : `${results.length} cases matched`,
reason: failed.length ? 'A core endpoint changed or failed its response contract.' : 'Core status-code contracts are stable.', evidence: [target],
remediation: 'Repair the endpoint or intentionally update the declared contract and consumers.',
});
finding(ctx, {
id: 'api.search-stream', category: 'api', title: 'Streaming search emits video and completion events',
status: streamOk ? 'PASS' : 'FAIL', severity: 'critical', expected: 'SSE videos event followed by complete', actual: search?.response.body || 'missing',
reason: streamOk ? 'The deterministic source traversed the full search stream.' : 'The stream omitted results or completion.', evidence: [target],
remediation: 'Inspect streaming serialization, source parsing, and completion handling.',
});
finding(ctx, {
id: 'api.detail-fixture', category: 'api', title: 'Detail parsing returns both MP4 and HLS episodes',
status: detailOk ? 'PASS' : 'FAIL', severity: 'critical', expected: 'success=true with 2 episodes', actual: JSON.stringify(detail?.parsed),
reason: detailOk ? 'The real detail route parsed deterministic upstream data.' : 'The detail route or episode parser lost fixture data.', evidence: [target],
remediation: 'Fix source lookup, upstream parsing, or episode normalization.',
});
}
verification/src/checks/api-discovery.mjs
+55
View File
@@ -0,0 +1,55 @@
import fs from 'node:fs';
import path from 'node:path';
import { finding } from '../core/finding.mjs';
import { relative, walk, writeJson } from '../core/files.mjs';
import { request } from '../core/http.mjs';
const methodPattern = /export\s+(?:async\s+)?function\s+(GET|POST|PUT|PATCH|DELETE|OPTIONS|HEAD)\b/g;
function routePath(root, file) {
return `/${relative(root, file).replace(/^app\//, '').replace(/\/route\.ts$/, '').replace(/\[([^\]]+)\]/g, 'verification-$1')}`;
}
export async function checkApiDiscovery(ctx) {
if (!ctx.state.appReady) return finding(ctx, {
id: 'api.route-coverage', category: 'api', title: 'Every API method is exercised', status: 'SKIP', severity: 'critical',
expected: 'Local server ready', actual: 'server unavailable', reason: 'API calls cannot execute.', remediation: 'Fix local startup.',
});
const files = walk(path.join(ctx.config.root, 'app', 'api'), (file) => file.endsWith('/route.ts'));
const inventory = files.map((file) => ({
file: relative(ctx.config.root, file),
path: routePath(ctx.config.root, file),
methods: [...fs.readFileSync(file, 'utf8').matchAll(methodPattern)].map((match) => match[1]),
}));
const results = [];
for (const route of inventory) {
for (const method of route.methods) {
const options = { method, timeoutMs: 12_000, headers: {} };
if (['POST', 'PUT', 'PATCH'].includes(method)) {
options.headers['content-type'] = 'application/json';
options.body = '{}';
}
const response = await request(`${ctx.config.localUrl}${route.path}`, options);
results.push({ ...route, methods: undefined, method, response });
}
}
const target = path.join(ctx.dirs.raw, 'api-route-matrix.json');
writeJson(target, { inventory, results });
const unexercised = inventory.flatMap((route) => route.methods.map((method) => `${method} ${route.path}`))
.filter((key) => !results.some((item) => `${item.method} ${item.path}` === key));
const crashes = results.filter((item) => !item.response.ok || item.response.status >= 500);
finding(ctx, {
id: 'api.route-coverage', category: 'api', title: 'Every statically exported API method receives a smoke request',
status: unexercised.length ? 'FAIL' : 'PASS', severity: 'critical', expected: '100% exported method invocation',
actual: `${results.length} methods invoked; ${unexercised.length} missing`, reason: unexercised.length ? 'Some exported API methods were not reached.' : 'Every discovered method was invoked with a safe anonymous payload.',
evidence: [target], remediation: 'Add a safe contract case for every missing method.',
});
finding(ctx, {
id: 'api.anonymous-crashes', category: 'api', title: 'Anonymous malformed requests do not crash API routes',
status: crashes.length ? 'FAIL' : 'PASS', severity: 'high', expected: 'No network error or HTTP 5xx for empty safe probes',
actual: crashes.length ? JSON.stringify(crashes.map((item) => ({ method: item.method, path: item.path, status: item.response.status, error: item.response.error }))) : 'No crashes',
reason: crashes.length ? 'Malformed or anonymous input reaches an internal failure instead of a controlled 4xx response.' : 'All routes rejected or handled generic probes without server failure.',
evidence: [target], remediation: 'Validate request inputs and convert expected missing configuration/auth states to explicit 4xx responses.',
});
ctx.state.apiInventory = inventory;
}
verification/src/checks/ast-metrics.mjs
+42
View File
@@ -0,0 +1,42 @@
import fs from 'node:fs';
import path from 'node:path';
import { parse } from '@typescript-eslint/typescript-estree';
import { finding } from '../core/finding.mjs';
import { relative, walk, writeJson } from '../core/files.mjs';
import { collectFunctions } from './ast-walk.mjs';
const extensions = new Set(['.ts', '.tsx', '.js', '.jsx', '.mjs', '.cjs']);
export async function checkAstMetrics(ctx) {
const files = walk(ctx.config.root, (file) => extensions.has(path.extname(file)) && !file.includes('/verification/'));
const metrics = [];
const parseErrors = [];
for (const file of files) {
try {
const ast = parse(fs.readFileSync(file, 'utf8'), { loc: true, jsx: /\.[jt]sx$/.test(file), errorOnUnknownASTType: false });
const functions = collectFunctions(ast);
metrics.push({ file: relative(ctx.config.root, file), functions });
} catch (error) {
parseErrors.push({ file: relative(ctx.config.root, file), error: error instanceof Error ? error.message : String(error) });
}
}
const offenders = metrics.flatMap((item) => item.functions.map((fn) => ({ file: item.file, ...fn })))
.filter((fn) => fn.lines > 80 || fn.complexity > 15 || fn.maxNesting > 4 || fn.params > 5)
.sort((a, b) => (b.complexity + b.lines / 10) - (a.complexity + a.lines / 10));
const target = path.join(ctx.dirs.metrics, 'ast-metrics.json');
writeJson(target, { parseErrors, offenders, files: metrics });
finding(ctx, {
id: 'quality.ast-parse', category: 'quality', title: 'JavaScript and TypeScript source is structurally analyzable',
status: parseErrors.length ? 'FAIL' : 'PASS', severity: 'high', expected: 'All JS/JSX/MJS/CJS/TS/TSX files parse',
actual: parseErrors.length ? JSON.stringify(parseErrors.slice(0, 20)) : `${files.length} files parsed`,
reason: parseErrors.length ? 'Unparseable files invalidate structural quality metrics.' : 'AST metrics cover every authored JavaScript and TypeScript file.',
evidence: [target], remediation: 'Fix syntax/parser incompatibilities before relying on complexity results.',
});
finding(ctx, {
id: 'quality.spaghetti-risk', category: 'quality', title: 'Functions stay within complexity and cohesion limits',
status: offenders.length ? 'FAIL' : 'PASS', severity: 'high', expected: 'lines <=80, complexity <=15, nesting <=4, params <=5',
actual: offenders.length ? `${offenders.length} risky functions; worst: ${JSON.stringify(offenders.slice(0, 12))}` : 'No threshold breaches',
reason: offenders.length ? 'Long, branch-heavy, deeply nested functions are concrete spaghetti-code indicators.' : 'No configured structural risk threshold was exceeded.',
evidence: [target], remediation: 'Extract cohesive functions and replace nested conditionals with explicit domain operations.',
});
}
verification/src/checks/ast-walk.mjs
+59
View File
@@ -0,0 +1,59 @@
const functionTypes = new Set([
'FunctionDeclaration', 'FunctionExpression', 'ArrowFunctionExpression',
'TSDeclareFunction', 'TSFunctionType', 'ObjectMethod', 'ClassMethod',
]);
const branchTypes = new Set([
'IfStatement', 'ForStatement', 'ForInStatement', 'ForOfStatement',
'WhileStatement', 'DoWhileStatement', 'CatchClause', 'ConditionalExpression',
]);
const nestTypes = new Set([...branchTypes, 'SwitchStatement', 'TryStatement']);
export function children(node) {
const output = [];
for (const [key, value] of Object.entries(node || {})) {
if (key === 'parent' || key === 'tokens' || key === 'comments') continue;
if (Array.isArray(value)) output.push(...value.filter((item) => item?.type));
else if (value?.type) output.push(value);
}
return output;
}
function displayName(node, parent) {
if (node.id?.name) return node.id.name;
if (parent?.id?.name) return parent.id.name;
if (parent?.key?.name) return parent.key.name;
return '<anonymous>';
}
function functionMetric(node, parent) {
let complexity = 1;
let maxNesting = 0;
const visit = (current, depth) => {
if (current !== node && functionTypes.has(current.type)) return;
if (branchTypes.has(current.type)) complexity += 1;
if (current.type === 'LogicalExpression' && ['&&', '||', '??'].includes(current.operator)) complexity += 1;
if (current.type === 'SwitchCase' && current.test) complexity += 1;
const nextDepth = nestTypes.has(current.type) ? depth + 1 : depth;
maxNesting = Math.max(maxNesting, nextDepth);
for (const child of children(current)) visit(child, nextDepth);
};
visit(node, 0);
return {
name: displayName(node, parent),
line: node.loc?.start.line || 0,
lines: (node.loc?.end.line || 0) - (node.loc?.start.line || 0) + 1,
params: node.params?.length || 0,
complexity,
maxNesting,
};
}
export function collectFunctions(ast) {
const output = [];
const visit = (node, parent) => {
if (functionTypes.has(node.type)) output.push(functionMetric(node, parent));
for (const child of children(node)) visit(child, node);
};
visit(ast, null);
return output;
}
verification/src/checks/coverage.mjs
+49
View File
@@ -0,0 +1,49 @@
import fs from 'node:fs';
import path from 'node:path';
import { runCommand } from '../core/command.mjs';
import { finding } from '../core/finding.mjs';
const metrics = ['lines', 'functions', 'branches', 'statements'];
export function coverageArgs(ctx, reportDir) {
const sources = ['app', 'components', 'lib', 'scripts'];
const args = ['--all', '--clean', '--100', '--exclude-after-remap'];
for (const source of sources) args.push('--src', source);
for (const extension of ['.ts', '.tsx', '.js', '.jsx', '.mjs', '.cjs']) args.push('--extension', extension);
args.push('--exclude', '**/*.d.ts', '--exclude', 'verification/tests/**', '--exclude', 'verification/tmp/**',
'--reporter', 'json-summary', '--reporter', 'html', '--reports-dir', reportDir,
process.execPath, path.join(ctx.config.verifyDir, 'src', 'run-regression.mjs'),
'--output-dir', path.join(reportDir, 'bundle'));
return args;
}
function readSummary(file) {
try { return JSON.parse(fs.readFileSync(file, 'utf8')); }
catch { return null; }
}
export async function checkCoverage(ctx) {
const reportDir = path.join(ctx.dirs.metrics, 'coverage');
const c8 = path.join(ctx.config.verifyDir, 'node_modules', '.bin', 'c8');
const result = await runCommand(ctx, 'coverage', c8, coverageArgs(ctx, reportDir), {
cwd: ctx.config.root, timeoutMs: ctx.config.commandTimeoutMs,
});
const summaryFile = path.join(reportDir, 'coverage-summary.json');
const summary = readSummary(summaryFile);
const total = summary?.total || null;
const percentages = Object.fromEntries(metrics.map((name) => [name, Number(total?.[name]?.pct)]));
const below = metrics.filter((name) => !Number.isFinite(percentages[name]) || percentages[name] < ctx.config.coveragePercent);
const ok = result.code === 0 && below.length === 0;
finding(ctx, {
id: 'static.code-coverage', category: 'coverage', title: 'Executable application code has complete regression coverage',
status: ok ? 'PASS' : 'FAIL', severity: 'critical',
expected: `${ctx.config.coveragePercent}% lines, functions, branches, and statements across app/components/lib/scripts`,
actual: summary ? JSON.stringify({ percentages, below }) : `coverage report missing; exit ${result.code}`,
reason: ok ? 'Every instrumentable application statement, branch, function, and line is covered by the local regression suite.'
: 'One or more application coverage dimensions are below the strict threshold or coverage collection failed.',
impact: below.length ? `Below threshold: ${below.join(', ')}` : '',
evidence: [result.outputPath, summaryFile, path.join(reportDir, 'index.html')],
remediation: 'Add focused local regressions for every uncovered range; do not lower the threshold or exclude business code.',
durationMs: result.durationMs,
});
}
verification/src/checks/deployment.mjs
+63
View File
@@ -0,0 +1,63 @@
import fs from 'node:fs';
import path from 'node:path';
import { runCommand } from '../core/command.mjs';
import { finding } from '../core/finding.mjs';
import { jsonBody, request } from '../core/http.mjs';
import { writeJson } from '../core/files.mjs';
// GH-ISSUE: 16,20,25,78,80,140,143,150,172,174,182,186; GH-PR: 29
function digest(output) {
return output?.match(/^Digest:\s+(sha256:[a-f0-9]+)/m)?.[1] || null;
}
export function latestDeployment(output) {
try {
const rows = JSON.parse(output);
return Array.isArray(rows) ? rows[0] || null : null;
} catch { return null; }
}
export async function checkDeployment(ctx) {
if (ctx.config.candidate) return finding(ctx, {
id: 'deploy.consistency', category: 'deployment', title: 'Local, GitHub, Cloudflare, and Docker release consistency', status: 'SKIP', severity: 'critical',
expected: 'Published release audit', actual: '--candidate', reason: 'A candidate commit cannot match public release surfaces before it is merged and published.',
remediation: 'After publishing, run ./verification/run without --candidate and require exact convergence.',
});
if (ctx.config.offline) return finding(ctx, {
id: 'deploy.consistency', category: 'deployment', title: 'Local, GitHub, Cloudflare, and Docker release consistency', status: 'SKIP', severity: 'critical',
expected: 'Online verification', actual: '--offline', reason: 'Remote state cannot be verified offline.', remediation: 'Rerun online before release.',
});
const localSha = (await runCommand(ctx, 'git-head', 'git', ['rev-parse', 'HEAD'], { timeoutMs: 30_000 })).tail.trim();
const remoteShaResult = await runCommand(ctx, 'git-remote-main', 'git', ['ls-remote', 'origin', 'refs/heads/main'], { timeoutMs: 30_000 });
const remoteSha = remoteShaResult.tail.trim().split(/\s+/)[0] || null;
const githubPackage = await request('https://raw.githubusercontent.com/KuekHaoYang/KVideo/main/package.json');
const cloudflare = await request(`${ctx.config.referenceUrl}/api/app-update`);
const latest = await runCommand(ctx, 'dockerhub-latest', 'docker', ['buildx', 'imagetools', 'inspect', 'kuekhaoyang/kvideo:latest'], { timeoutMs: 120_000 });
const versioned = await runCommand(ctx, 'dockerhub-version', 'docker', ['buildx', 'imagetools', 'inspect', `kuekhaoyang/kvideo:${ctx.state.version}`], { timeoutMs: 120_000 });
const wrangler = path.join(ctx.config.verifyDir, 'node_modules', '.bin', 'wrangler');
const pages = await runCommand(ctx, 'cloudflare-deployments', wrangler, ['pages', 'deployment', 'list', '--project-name', 'kvideo', '--environment', 'production', '--json'], { timeoutMs: 120_000 });
const deployment = latestDeployment(fs.readFileSync(pages.outputPath, 'utf8'));
const deployedRelease = deployment?.Deployment ? await request(`${deployment.Deployment}/api/app-update`) : null;
const githubVersion = jsonBody(githubPackage)?.version;
const cloudVersion = jsonBody(cloudflare)?.currentVersion;
const deployedVersion = jsonBody(deployedRelease)?.currentVersion;
const latestDigest = digest(latest.tail);
const versionDigest = digest(versioned.tail);
const deploymentSha = deployment?.Source || null;
const deploymentUrl = deployment?.Deployment || null;
const facts = { localSha, remoteSha, localVersion: ctx.state.version, githubVersion, cloudVersion, deployedVersion,
deploymentSha, deploymentUrl, latestDigest, versionDigest };
const target = path.join(ctx.dirs.raw, 'deployment-consistency.json');
writeJson(target, { facts, githubPackage, cloudflare, deployedRelease, deployment,
evidence: { latest: latest.outputPath, versioned: versioned.outputPath, pages: pages.outputPath } });
const ok = localSha === remoteSha && githubVersion === ctx.state.version && cloudVersion === ctx.state.version
&& deploymentSha === localSha.slice(0, 7) && deployedVersion === ctx.state.version
&& latestDigest && latestDigest === versionDigest;
finding(ctx, {
id: 'deploy.consistency', category: 'deployment', title: 'Local, GitHub main, Cloudflare, and both Docker tags agree',
status: ok ? 'PASS' : 'FAIL', severity: 'critical', expected: 'Same Git commit/version; Docker latest and version tags share one digest', actual: JSON.stringify(facts),
reason: ok ? 'Every public release surface resolves to the declared release.' : 'At least one release surface is stale, missing, or points at a different artifact.',
evidence: [target, latest.outputPath, versioned.outputPath, pages.outputPath], remediation: 'Merge/push main, wait for Pages and Docker workflows, then verify version and digest convergence.',
});
}
verification/src/checks/docker-context.mjs
+39
View File
@@ -0,0 +1,39 @@
import fs from 'node:fs';
import path from 'node:path';
const ignoredDirectories = new Set([
'.git', '.gradle', '.next', '.vercel', '.wrangler',
'artifacts', 'build', 'cache', 'coverage', 'dist', 'node_modules', 'out',
]);
const ignoredRootFiles = new Set(['README.md', 'next-env.d.ts', 'npm-debug.log']);
export function includeDockerContextPath(root, source) {
const relative = path.relative(root, source);
if (!relative) return true;
if (relative.startsWith('..') || path.isAbsolute(relative)) return false;
const parts = relative.split(path.sep);
if (parts[0] === 'verification') return false;
if (parts.some((part) => ignoredDirectories.has(part))) return false;
if (parts.length === 1 && ignoredRootFiles.has(parts[0])) return false;
return !parts.at(-1).endsWith('.tsbuildinfo');
}
export function createDockerContext(ctx) {
const root = ctx.config.root;
const target = path.join(ctx.config.verifyDir, 'cache', `docker-context-${ctx.runId}`);
fs.rmSync(target, { recursive: true, force: true });
fs.mkdirSync(target, { recursive: true });
for (const name of fs.readdirSync(root)) {
const source = path.join(root, name);
if (!includeDockerContextPath(root, source)) continue;
fs.cpSync(source, path.join(target, name), {
recursive: true,
filter: (candidate) => includeDockerContextPath(root, candidate),
});
}
return target;
}
export function removeDockerContext(target) {
fs.rmSync(target, { recursive: true, force: true });
}
verification/src/checks/docker-local.mjs
+50
View File
@@ -0,0 +1,50 @@
import path from 'node:path';
import { runCommand } from '../core/command.mjs';
import { finding } from '../core/finding.mjs';
import { jsonBody, request } from '../core/http.mjs';
import { waitForUrl } from '../core/service.mjs';
import { writeJson } from '../core/files.mjs';
import { createDockerContext, removeDockerContext } from './docker-context.mjs';
export async function checkDockerLocal(ctx) {
if (ctx.config.quick) return finding(ctx, {
id: 'docker.local-image', category: 'docker', title: 'Local Docker image build and runtime smoke', status: 'SKIP', severity: 'critical',
expected: 'Full run', actual: '--quick', reason: 'Quick mode omits the expensive image build.', remediation: 'Run ./verification/run without --quick before release.',
});
const image = `kvideo-verification:${ctx.state.version}`;
const name = `kvideo-verification-${Date.now()}`;
const context = createDockerContext(ctx);
let build;
try {
build = await runCommand(ctx, 'docker-build', 'docker', ['build', '--pull', '--tag', image, context], { timeoutMs: 1_800_000 });
} finally {
removeDockerContext(context);
}
if (build.code !== 0) return finding(ctx, {
id: 'docker.local-image', category: 'docker', title: 'Local Docker image builds and runs', status: 'FAIL', severity: 'critical',
expected: 'docker build exit 0', actual: `exit ${build.code}`, reason: 'The release container cannot be produced from this checkout.',
evidence: [build.outputPath], remediation: 'Fix Dockerfile, dependency installation, or standalone build output.', durationMs: build.durationMs,
});
const run = await runCommand(ctx, 'docker-run', 'docker', ['run', '--detach', '--rm', '--name', name, '--publish', `127.0.0.1:${ctx.config.containerPort}:3000`, image], { timeoutMs: 60_000 });
let ready = { ok: false, error: 'container did not start' };
let response = null;
let inspect = null;
if (run.code === 0) {
ready = await waitForUrl(ctx.config.containerUrl, 120_000);
response = ready.ok ? await request(`${ctx.config.containerUrl}/api/app-update`) : null;
inspect = await runCommand(ctx, 'docker-inspect-local', 'docker', ['image', 'inspect', image], { timeoutMs: 30_000 });
}
const logs = await runCommand(ctx, 'docker-container-logs', 'docker', ['logs', name], { timeoutMs: 30_000 });
await runCommand(ctx, 'docker-stop', 'docker', ['stop', name], { timeoutMs: 60_000 });
const parsed = response ? jsonBody(response) : null;
const ok = run.code === 0 && ready.ok && response?.status === 200 && parsed?.currentVersion === ctx.state.version;
const target = path.join(ctx.dirs.raw, 'docker-local.json');
writeJson(target, { image, name, build, run, ready, response, parsed, inspect: inspect?.outputPath, logs: logs.outputPath });
finding(ctx, {
id: 'docker.local-image', category: 'docker', title: 'Local Docker image builds, starts, and reports the expected version',
status: ok ? 'PASS' : 'FAIL', severity: 'critical', expected: `HTTP 200 and version ${ctx.state.version}`, actual: JSON.stringify({ run: run.code, ready, status: response?.status, version: parsed?.currentVersion }),
reason: ok ? 'The real standalone container passed a runtime smoke test.' : 'The container failed to start, respond, or expose the expected release version.',
evidence: [build.outputPath, run.outputPath, logs.outputPath, target], remediation: 'Inspect build and container logs, then correct the Docker release path.',
durationMs: build.durationMs + run.durationMs,
});
}
verification/src/checks/duplicates.mjs
+85
View File
@@ -0,0 +1,85 @@
import crypto from 'node:crypto';
import fs from 'node:fs';
import path from 'node:path';
import { runCommand } from '../core/command.mjs';
import { finding } from '../core/finding.mjs';
import { relative, walk, writeJson } from '../core/files.mjs';
const codeExt = new Set(['.ts', '.tsx', '.js', '.jsx', '.mjs', '.cjs', '.css', '.kt', '.kts']);
function findReport(dir) {
if (!fs.existsSync(dir)) return null;
return fs.readdirSync(dir).map((name) => path.join(dir, name)).find((file) => file.endsWith('.json')) || null;
}
async function cloneScan(ctx, options) {
const output = path.join(ctx.dirs.metrics, options.name);
const bin = path.join(ctx.config.verifyDir, 'node_modules', '.bin', 'jscpd');
const args = ['--min-lines', String(options.minLines), '--min-tokens', String(options.minTokens),
'--reporters', 'json', '--output', output, ...options.roots];
const result = await runCommand(ctx, options.name, bin, args, { cwd: ctx.config.root });
const report = findReport(output);
let data = null;
try { data = report ? JSON.parse(fs.readFileSync(report, 'utf8')) : null; } catch { /* invalid report */ }
return { result, report, data };
}
function addCloneFinding(ctx, scan, options) {
const stats = scan.data?.statistics?.total;
const percentage = stats?.percentage ?? stats?.percentageTokens ?? null;
const clones = scan.data?.duplicates?.length ?? null;
const withinRatio = Number(percentage || 0) <= options.maxPercentage;
const withinCount = options.maxClones === null || Number(clones || 0) <= options.maxClones;
const ok = scan.result.code === 0 && scan.data && withinRatio && withinCount;
finding(ctx, {
id: options.id, category: options.category, title: options.title, status: ok ? 'PASS' : 'FAIL', severity: options.severity,
expected: `duplication <=${options.maxPercentage}%${options.maxClones === null ? '' : ` and clones <=${options.maxClones}`}`,
actual: scan.data ? `${percentage}% duplication; ${clones} clone groups` : `jscpd exit ${scan.result.code}; no report`,
reason: ok ? 'Token-level copy detection stays within the declared strict boundary.' : 'Copied implementations create divergent fixes and repeated maintenance.',
evidence: [scan.result.outputPath, ...(scan.report ? [scan.report] : [])],
remediation: 'Extract one shared implementation and delete parallel copied branches.', durationMs: scan.result.durationMs,
});
}
function duplicateGroups(root, roots) {
const files = roots.flatMap((dir) => fs.existsSync(path.join(root, dir))
? walk(path.join(root, dir), (file) => codeExt.has(path.extname(file)) && !file.includes('/node_modules/') && !file.includes('/artifacts/')) : []);
const hashes = new Map();
for (const file of files) {
const content = fs.readFileSync(file);
if (content.length < 20) continue;
const digest = crypto.createHash('sha256').update(content).digest('hex');
const group = hashes.get(digest) || [];
group.push(relative(root, file));
hashes.set(digest, group);
}
return [...hashes.values()].filter((group) => group.length > 1).sort((a, b) => a[0].localeCompare(b[0]));
}
function addExactFinding(ctx, options) {
const { id, title, groups, target, severity } = options;
finding(ctx, {
id, category: 'quality', title, status: groups.length ? 'FAIL' : 'PASS', severity,
expected: '0 byte-identical authored source files', actual: groups.length ? JSON.stringify(groups) : '0',
reason: groups.length ? 'Whole-file copies are redundant implementations and can drift independently.' : 'No authored source file is an exact copy of another.',
evidence: [target], remediation: 'Keep one canonical module and replace copies with imports or shared data.',
});
}
export async function checkDuplicates(ctx) {
const projectRoots = ['app', 'components', 'lib', 'scripts', 'android-tv/app/src/main/java'];
const verifierRoots = ['verification/src', 'verification/tests'];
const project = await cloneScan(ctx, { name: 'jscpd-project', roots: projectRoots, minLines: 8, minTokens: 60 });
const verifier = await cloneScan(ctx, { name: 'jscpd-verifier', roots: verifierRoots, minLines: 6, minTokens: 45 });
addCloneFinding(ctx, project, { id: 'quality.duplication', category: 'quality', title: 'Project copy-paste duplication is minimal',
severity: 'high', maxPercentage: 1, maxClones: null });
addCloneFinding(ctx, verifier, { id: 'harness.duplication', category: 'harness', title: 'Verification code contains no copied implementation',
severity: 'critical', maxPercentage: 0, maxClones: 0 });
const exact = { project: duplicateGroups(ctx.config.root, projectRoots), verifier: duplicateGroups(ctx.config.root, verifierRoots) };
const target = path.join(ctx.dirs.metrics, 'exact-duplicates.json');
writeJson(target, exact);
addExactFinding(ctx, { id: 'quality.exact-file-copies', title: 'Project has no exact source-file copies',
groups: exact.project, target, severity: 'high' });
addExactFinding(ctx, { id: 'harness.exact-file-copies', title: 'Verification has no exact source-file copies',
groups: exact.verifier, target, severity: 'critical' });
}
verification/src/checks/github-audit-findings.mjs
+77
View File
@@ -0,0 +1,77 @@
import { finding } from '../core/finding.mjs';
import { reviewErrors } from './history-review.mjs';
function flattened(object) {
return Object.values(object).flat();
}
function inventoryFinding(ctx, audit, catalog, evidence) {
const delta = flattened(audit.inventoryDelta);
finding(ctx, {
id: 'history.remote-inventory', category: 'history', title: 'Remote history matches the local record inventory',
status: delta.length ? 'FAIL' : 'PASS', severity: 'critical', expected: `${catalog.issues.length} issues and ${catalog.pullRequests.length} PRs`,
actual: delta.length ? JSON.stringify(audit.inventoryDelta) : 'Exact inventory match',
reason: delta.length ? 'The explicit maintenance audit found a local/remote inventory difference.' : 'Every remote record through the cutoffs is represented locally.',
evidence, remediation: 'Review and classify every inventory difference before updating local history data.',
});
}
function coverageFinding(ctx, audit, evidence) {
const delta = flattened(audit.coverageDelta);
finding(ctx, {
id: 'history.remote-coverage-classification', category: 'history', title: 'Remote bug and merged-PR classifications match local contracts',
status: delta.length ? 'FAIL' : 'PASS', severity: 'critical', expected: 'No classification drift',
actual: delta.length ? JSON.stringify(audit.coverageDelta) : 'Exact classification match',
reason: delta.length ? 'Remote labels or merge state changed the set that requires local regression.' : 'The local contract lists match the maintenance audit.',
evidence, remediation: 'Audit the changed records and update local tests before changing the stored lists.',
});
}
function freshnessFindings(ctx, audit, evidence) {
const newer = [...audit.uncataloged.issues, ...audit.uncataloged.pullRequests];
finding(ctx, {
id: 'history.no-uncataloged-records', category: 'history', title: 'No newer remote record is outside the local catalog',
status: newer.length ? 'FAIL' : 'PASS', severity: 'critical', expected: '0 records beyond cutoffs',
actual: newer.length ? JSON.stringify(audit.uncataloged) : '0', reason: newer.length ? 'New records require local contracts.' : 'The cutoffs reach the newest remote records.',
evidence, remediation: 'Understand each new record, write local evidence, then advance the catalog.',
});
finding(ctx, {
id: 'history.graph-completeness', category: 'history', title: 'Remote review pagination is complete',
status: audit.truncation.length ? 'FAIL' : 'PASS', severity: 'critical', expected: 'Every totalCount equals fetched nodes',
actual: audit.truncation.length ? JSON.stringify(audit.truncation) : 'No truncated collection',
reason: audit.truncation.length ? 'The maintenance audit omitted remote review data.' : 'All declared remote review collections were fetched.',
evidence, remediation: 'Add cursor pagination before accepting the remote maintenance audit.',
});
}
function snapshotFinding(ctx, audit, baseline, evidence) {
const mismatches = Object.keys(baseline).filter((key) => JSON.stringify(baseline[key]) !== JSON.stringify(audit.normalized.digests[key]));
finding(ctx, {
id: 'history.remote-snapshot', category: 'history', title: 'Remote content matches the stored maintenance snapshot',
status: mismatches.length ? 'FAIL' : 'PASS', severity: 'high', expected: baseline,
actual: mismatches.length ? Object.fromEntries(mismatches.map((key) => [key, audit.normalized.digests[key]])) : audit.normalized.digests,
reason: mismatches.length ? 'Remote content changed and requires renewed local analysis.' : 'The optional remote audit matches its stored hashes.',
evidence, remediation: 'Review the changed content before updating snapshot hashes.',
});
}
function reviewFinding(ctx, audit, decisions, commentDecisions, evidence) {
const errors = reviewErrors({ ...audit.review, decisions, commentDecisions });
const failed = Object.values(errors).some((items) => items.length);
finding(ctx, {
id: 'history.remote-review-adjudication', category: 'history', title: 'Remote review risks match local decisions',
status: failed ? 'FAIL' : 'PASS', severity: 'critical', expected: 'Exact IDs and priorities',
actual: failed ? JSON.stringify(errors) : `${audit.review.threads.length} threads and ${audit.review.comments.length} comments matched`,
reason: failed ? 'A remote review risk is new, stale, or mis-prioritized locally.' : 'Remote review state agrees with the local decision ledger.',
evidence, remediation: 'Classify every listed review record before accepting a snapshot update.',
});
}
export function addGithubAuditFindings(ctx, input) {
const { audit, catalog, baseline, decisions, commentDecisions, evidence } = input;
inventoryFinding(ctx, audit, catalog, evidence);
coverageFinding(ctx, audit, evidence);
freshnessFindings(ctx, audit, evidence);
snapshotFinding(ctx, audit, baseline, evidence);
reviewFinding(ctx, audit, decisions, commentDecisions, evidence);
}
verification/src/checks/github-history.mjs
+40
View File
@@ -0,0 +1,40 @@
import path from 'node:path';
import { finding } from '../core/finding.mjs';
import { writeJson } from '../core/files.mjs';
import { loadHistoryCatalog } from '../history/catalog.mjs';
import { createGithubAudit } from '../history/audit.mjs';
import { fetchGithubHistory } from '../history/github.mjs';
import { collectTraceability } from '../history/trace.mjs';
import { addGithubAuditFindings } from './github-audit-findings.mjs';
import { checkLocalHistory } from './history-local.mjs';
function remoteFailure(ctx, remote, evidence) {
finding(ctx, {
id: 'history.remote-snapshot', category: 'history', title: 'Remote GitHub maintenance audit completes',
status: 'FAIL', severity: 'critical', expected: 'Complete valid GitHub responses', actual: remote.error,
reason: 'The explicitly requested remote audit could not prove a complete response set.',
evidence, remediation: 'Restore gh authentication or network access and rerun --audit-github.',
});
}
export async function checkGithubHistory(ctx) {
await checkLocalHistory(ctx);
if (ctx.config.offline) return finding(ctx, {
id: 'history.remote-snapshot', category: 'history', title: 'Remote GitHub maintenance audit completes',
status: 'SKIP', severity: 'high', expected: 'Online audit', actual: '--offline',
reason: 'The caller requested both --audit-github and --offline.',
evidence: [path.join(ctx.config.verifyDir, 'history')], remediation: 'Remove --offline to run the explicit maintenance audit.',
});
const loaded = loadHistoryCatalog(ctx.config.root);
const trace = collectTraceability(ctx.config.root, loaded.catalog);
const remote = await fetchGithubHistory(ctx);
const commandEvidence = remote.commands.map((item) => item.outputPath);
if (remote.error) return remoteFailure(ctx, remote, commandEvidence);
const currentPr = Number(process.env.GITHUB_REF?.match(/^refs\/pull\/(\d+)\//)?.[1]) || null;
const audit = createGithubAudit(remote, loaded.catalog, trace, currentPr);
const target = path.join(ctx.dirs.metrics, 'github-history.json');
writeJson(target, { inventoryDelta: audit.inventoryDelta, coverageDelta: audit.coverageDelta,
uncataloged: audit.uncataloged, truncation: audit.truncation, digests: audit.normalized.digests,
issues: audit.issues, pullRequests: audit.pullRequests, data: audit.normalized.data });
addGithubAuditFindings(ctx, { audit, ...loaded, evidence: [target, ...commandEvidence] });
}
verification/src/checks/harness-dependencies.mjs
+39
View File
@@ -0,0 +1,39 @@
import fs from 'node:fs';
import { runCommand } from '../core/command.mjs';
import { finding } from '../core/finding.mjs';
function parse(file) {
try { return JSON.parse(fs.readFileSync(file, 'utf8')); }
catch { return null; }
}
export function auditArgs() {
return ['audit', '--omit=dev', '--audit-level=high', '--json'];
}
export async function checkHarnessDependencies(ctx) {
const cwd = ctx.config.verifyDir;
const tree = await runCommand(ctx, 'verification-dependencies', 'npm', ['ls', '--all', '--json'], { cwd, timeoutMs: 60_000 });
finding(ctx, {
id: 'harness.dependency-integrity', category: 'harness', title: 'Verification dependencies exactly match their lockfile',
status: tree.code === 0 ? 'PASS' : 'FAIL', severity: 'critical', expected: 'npm ls exit 0', actual: `exit ${tree.code}`,
reason: tree.code === 0 ? 'The verifier runs with a complete pinned dependency graph.' : 'Missing, invalid, or extraneous verifier dependencies make results non-reproducible.',
evidence: [tree.outputPath], remediation: 'Repair verification/package.json and its lockfile, then rerun npm ci.', durationMs: tree.durationMs,
});
if (ctx.config.offline) return finding(ctx, {
id: 'harness.dependency-audit', category: 'harness', title: 'Verification dependencies have no severe advisory',
status: 'SKIP', severity: 'high', expected: 'Online npm audit', actual: '--offline', reason: 'The run explicitly disabled network checks.', remediation: 'Rerun online.',
});
const audit = await runCommand(ctx, 'verification-audit', 'npm', auditArgs(), { cwd, timeoutMs: 120_000 });
const body = parse(audit.outputPath);
const counts = body?.metadata?.vulnerabilities || null;
const severe = (counts?.high || 0) + (counts?.critical || 0);
const ok = audit.code === 0 && counts && severe === 0;
finding(ctx, {
id: 'harness.dependency-audit', category: 'harness', title: 'Verification dependencies have no high or critical advisory',
status: ok ? 'PASS' : 'FAIL', severity: 'critical', expected: '0 high and 0 critical vulnerabilities',
actual: counts ? JSON.stringify(counts) : `unparseable output; exit ${audit.code}`,
reason: ok ? 'The verifier dependency graph has no known severe advisory.' : 'The verification framework itself has a severe dependency risk or an incomplete audit.',
evidence: [audit.outputPath], remediation: 'Upgrade or replace the affected pinned verifier dependency.', durationMs: audit.durationMs,
});
}
verification/src/checks/harness-self.mjs
+28
View File
@@ -0,0 +1,28 @@
import fs from 'node:fs';
import path from 'node:path';
import { runCommand } from '../core/command.mjs';
import { finding } from '../core/finding.mjs';
import { checkHarnessDependencies } from './harness-dependencies.mjs';
export async function checkHarnessSelf(ctx) {
const testDir = path.join(ctx.config.verifyDir, 'tests', 'harness');
const tests = fs.readdirSync(testDir).filter((name) => name.endsWith('.test.mjs')).map((name) => path.join(testDir, name));
const result = await runCommand(ctx, 'verification-self-tests', 'node', ['--test', ...tests], { cwd: ctx.config.root, timeoutMs: 60_000 });
finding(ctx, {
id: 'harness.self-tests', category: 'harness', title: 'Verification framework self-tests pass',
status: result.code === 0 ? 'PASS' : 'FAIL', severity: 'critical', expected: 'node --test exit 0', actual: `exit ${result.code}`,
reason: result.code === 0 ? 'Core redaction, HTTP parsing, graph helpers, and report escaping are verified.' : 'The validation framework failed its own tests.',
evidence: [result.outputPath], remediation: 'Fix the harness before treating any project result as authoritative.', durationMs: result.durationMs,
});
const eslint = path.join(ctx.config.verifyDir, 'node_modules', '.bin', 'eslint');
const config = path.join(ctx.config.verifyDir, 'eslint.config.mjs');
const lint = await runCommand(ctx, 'verification-eslint', eslint, ['verification/src', 'verification/tests/harness',
'--config', config, '--max-warnings', '0'], { cwd: ctx.config.root, timeoutMs: 60_000 });
finding(ctx, {
id: 'harness.eslint', category: 'harness', title: 'Verification code passes strict ESLint independently',
status: lint.code === 0 ? 'PASS' : 'FAIL', severity: 'critical', expected: 'exit 0 with zero warnings', actual: `exit ${lint.code}`,
reason: lint.code === 0 ? 'Verifier lint quality is independent of existing application lint failures.' : 'The validation implementation violates its own static coding rules.',
evidence: [lint.outputPath], remediation: 'Fix every verifier lint diagnostic before trusting the suite.', durationMs: lint.durationMs,
});
await checkHarnessDependencies(ctx);
}
verification/src/checks/history-local.mjs
+67
View File
@@ -0,0 +1,67 @@
import path from 'node:path';
import { finding } from '../core/finding.mjs';
import { writeJson } from '../core/files.mjs';
import { loadHistoryCatalog, validateCatalog } from '../history/catalog.mjs';
import { localCoverage } from '../history/local.mjs';
import { collectTraceability } from '../history/trace.mjs';
function values(object) {
return Object.entries(object).flatMap(([name, items]) => items.map((value) => ({ name, value })));
}
function riskFinding(ctx, options) {
const { items, id, title, status, severity, evidence } = options;
finding(ctx, {
id, category: 'history', title, status: items.length ? status : 'PASS', severity,
expected: '0 locally recorded open risks', actual: items.length ? JSON.stringify(items) : '0',
reason: items.length ? 'The stored contract remains open and is enforced without querying GitHub.' : 'No stored risk remains open at this severity.',
evidence, remediation: 'Fix the contract and add a focused local regression before marking the decision fixed.',
});
}
function addStoredRisks(ctx, decisions, commentDecisions, evidence) {
const open = [...decisions, ...commentDecisions].filter((item) => item.status === 'open')
.map(({ id, pr, priority, contract, reason }) => ({ id, pr, priority, contract, reason }));
const severe = open.filter((item) => ['critical', 'high'].includes(item.priority));
const medium = open.filter((item) => !['critical', 'high'].includes(item.priority));
riskFinding(ctx, { items: severe, id: 'history.open-severe-review-risks',
title: 'Local critical/high review contracts are closed', status: 'FAIL', severity: 'high', evidence });
riskFinding(ctx, { items: medium, id: 'history.open-medium-review-risks',
title: 'Local medium review contracts are tracked', status: 'WARN', severity: 'medium', evidence });
}
export async function checkLocalHistory(ctx) {
const loaded = loadHistoryCatalog(ctx.config.root);
const { catalog, baseline, decisions, commentDecisions } = loaded;
const trace = collectTraceability(ctx.config.root, catalog);
const validation = validateCatalog(ctx.config.root, catalog, baseline, decisions, commentDecisions);
const coverage = localCoverage(ctx.config.root, catalog, trace);
const errors = values(validation);
const historyDir = path.join(ctx.config.verifyDir, 'history');
const target = path.join(ctx.dirs.metrics, 'local-history.json');
writeJson(target, { validation, coverage, decisions, commentDecisions });
finding(ctx, {
id: 'history.catalog-integrity', category: 'history', title: 'Local historical contracts are internally complete',
status: errors.length ? 'FAIL' : 'PASS', severity: 'critical', expected: 'Valid local catalog and review decisions',
actual: errors.length ? JSON.stringify(errors) : `${catalog.regressionIssues.length} issues and ${catalog.mergedPullRequests.length} merged PRs declared`,
reason: errors.length ? 'The local source of truth is malformed.' : 'Historical requirements are stored inside verification/ and need no GitHub request.',
evidence: [historyDir, target], remediation: 'Repair every local catalog error before trusting history coverage.',
});
const traceErrors = [...coverage.missing, ...coverage.nonExecutable, ...coverage.unknown];
finding(ctx, {
id: 'history.regression-traceability', category: 'history', title: 'Every known regression maps to executable local evidence',
status: traceErrors.length ? 'FAIL' : 'PASS', severity: 'critical', expected: 'Direct tag in a test or invoked check',
actual: traceErrors.length ? JSON.stringify(traceErrors) : `${coverage.issues.length} issues and ${coverage.pullRequests.length} PRs locally executable`,
reason: traceErrors.length ? 'A tag alone is insufficient when its evidence is missing or never executed.' : 'Every required historical contract reaches code executed by this validator.',
evidence: [target], remediation: 'Add a focused local test/check and ensure the complete runner invokes it.',
});
const unverifiable = catalog.unverifiableIssues || [];
finding(ctx, {
id: 'history.unverifiable-items', category: 'history', title: 'Unverifiable historical reports stay explicit',
status: unverifiable.length ? 'WARN' : 'PASS', severity: 'medium', expected: '0 reports without reproducible facts',
actual: unverifiable.length ? JSON.stringify(unverifiable) : '0',
reason: unverifiable.length ? 'No test can be honestly derived from the stored report.' : 'Every stored report has a local contract.',
evidence: [historyDir], remediation: 'Obtain reproduction facts before adding a claimed regression.',
});
addStoredRisks(ctx, decisions, commentDecisions, [target]);
}
verification/src/checks/history-review.mjs
+28
View File
@@ -0,0 +1,28 @@
function keySet(items) {
return new Set(items.map((item) => item.id));
}
function mapDecision(items) {
return new Map(items.map((item) => [item.id, item]));
}
export function reviewErrors(input) {
const threadMap = mapDecision(input.decisions);
const commentMap = mapDecision(input.commentDecisions);
const remoteIds = keySet(input.threads);
const remoteCommentIds = keySet(input.comments);
const unknown = [
...input.threads.filter((item) => !threadMap.has(item.id)),
...input.comments.filter((item) => !commentMap.has(item.id)),
];
const stale = [
...input.decisions.filter((item) => !remoteIds.has(item.id)),
...input.commentDecisions.filter((item) => !remoteCommentIds.has(item.id)),
];
const priorityMismatch = [
...input.threads.map((item) => ({ item, decision: threadMap.get(item.id) })),
...input.comments.map((item) => ({ item, decision: commentMap.get(item.id) })),
].filter(({ item, decision }) => decision && item.priority !== 'unspecified' && item.priority !== decision.priority);
const errors = { unknown, stale, priorityMismatch };
return errors;
}
verification/src/checks/import-graph.mjs
+62
View File
@@ -0,0 +1,62 @@
import fs from 'node:fs';
import path from 'node:path';
import { findCycles } from '../core/cycles.mjs';
import { finding } from '../core/finding.mjs';
import { lineCount, relative, walk, writeJson } from '../core/files.mjs';
const ext = ['.ts', '.tsx', '.js', '.jsx', '.mjs'];
const importPattern = /(?:import|export)\s+(?:[^'";]+?\s+from\s+)?['"]([^'"]+)['"]|import\(\s*['"]([^'"]+)['"]\s*\)/g;
function resolveImport(root, from, specifier) {
if (!specifier.startsWith('.') && !specifier.startsWith('@/')) return null;
const base = specifier.startsWith('@/') ? path.join(root, specifier.slice(2)) : path.resolve(path.dirname(from), specifier);
const candidates = [base, ...ext.map((item) => base + item), ...ext.map((item) => path.join(base, `index${item}`))];
return candidates.find((item) => fs.existsSync(item) && fs.statSync(item).isFile()) || null;
}
function closure(graph, start) {
const seen = new Set();
const visit = (node) => {
if (seen.has(node)) return;
seen.add(node);
for (const child of graph.get(node) || []) visit(child);
};
visit(start);
return seen;
}
export async function checkImportGraph(ctx) {
const root = ctx.config.root;
const files = walk(root, (file) => ext.includes(path.extname(file)) && !file.includes('/verification/'));
const graph = new Map(files.map((file) => [relative(root, file), []]));
const unresolved = [];
for (const file of files) {
const source = fs.readFileSync(file, 'utf8');
for (const match of source.matchAll(importPattern)) {
const specifier = match[1] || match[2];
const resolved = resolveImport(root, file, specifier);
if (resolved) graph.get(relative(root, file)).push(relative(root, resolved));
else if (specifier.startsWith('.') || specifier.startsWith('@/')) unresolved.push({ file: relative(root, file), specifier });
}
}
const cycles = findCycles(graph);
const pages = [...graph.keys()].filter((file) => /(^|\/)app\/.*page\.tsx$/.test(file));
const features = pages.map((entry) => {
const reached = closure(graph, entry);
return { entry, files: reached.size, lines: [...reached].reduce((sum, file) => sum + lineCount(path.join(root, file)), 0) };
}).sort((a, b) => b.lines - a.lines);
const target = path.join(ctx.dirs.metrics, 'import-graph.json');
writeJson(target, { cycles, unresolved, features, graph: Object.fromEntries(graph) });
finding(ctx, {
id: 'quality.import-cycles', category: 'quality', title: 'Internal dependency graph has no cycles',
status: cycles.length ? 'FAIL' : 'PASS', severity: 'high', expected: '0 dependency cycles', actual: cycles.length ? cycles.slice(0, 20).join('\n') : '0',
reason: cycles.length ? 'Cycles create order-dependent initialization and make feature boundaries unreliable.' : 'No static import cycles were found.',
evidence: [target], remediation: 'Extract shared contracts or invert dependencies to break each cycle.',
});
finding(ctx, {
id: 'quality.feature-footprint', category: 'quality', title: 'Feature transitive code footprint is quantified',
status: features.some((item) => item.lines > 10_000) ? 'WARN' : 'PASS', severity: 'medium', expected: 'No page transitively owns more than 10,000 lines',
actual: JSON.stringify(features), reason: 'Transitive page footprints expose features that accumulate excessive code through dependencies.',
evidence: [target], remediation: 'Split oversized feature graphs into explicit bounded modules and lazy boundaries.',
});
}
verification/src/checks/latency.mjs
+36
View File
@@ -0,0 +1,36 @@
import path from 'node:path';
import { finding } from '../core/finding.mjs';
import { jsonBody, request } from '../core/http.mjs';
import { writeJson } from '../core/files.mjs';
async function ping(ctx, url) {
const response = await request(`${ctx.config.localUrl}/api/ping`, {
method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ url }), timeoutMs: 10_000,
});
return { response, parsed: jsonBody(response) };
}
export async function checkLatency(ctx) {
if (!ctx.state.appReady) return;
const fast = await ping(ctx, `${ctx.config.fixtureUrl}/fast`);
const slow = await ping(ctx, `${ctx.config.fixtureUrl}/slow?ms=300`);
const burst = await Promise.all(Array.from({ length: 12 }, () => ping(ctx, `${ctx.config.fixtureUrl}/slow?ms=80`)));
const values = burst.map((item) => item.response.durationMs).sort((a, b) => a - b);
const p95 = values[Math.ceil(values.length * 0.95) - 1];
const target = path.join(ctx.dirs.raw, 'latency-contracts.json');
writeJson(target, { fast, slow, burst: burst.map((item) => item.response), p95 });
const accurate = fast.response.status === 200 && slow.response.status === 200 && fast.parsed?.success && slow.parsed?.success &&
slow.parsed.latency >= 250 && slow.parsed.latency <= 1500 && slow.parsed.latency > fast.parsed.latency;
finding(ctx, {
id: 'latency.accuracy', category: 'performance', title: 'Latency probe distinguishes fast and delayed sources',
status: accurate ? 'PASS' : 'FAIL', severity: 'high', expected: '300ms fixture reports 250-1500ms and exceeds fast fixture',
actual: JSON.stringify({ fast: fast.parsed, slow: slow.parsed }), reason: accurate ? 'Measured latency tracks controlled upstream delay.' : 'Latency values are missing, inverted, or outside tolerance.',
evidence: [target], remediation: 'Inspect HEAD/GET fallback timing and timeout accounting.',
});
finding(ctx, {
id: 'latency.concurrent-p95', category: 'performance', title: 'Concurrent latency requests stay responsive',
status: p95 <= 2000 ? 'PASS' : 'FAIL', severity: 'medium', expected: '12-request p95 <= 2000ms', actual: `${p95}ms`,
reason: p95 <= 2000 ? 'The local latency endpoint handled the burst within threshold.' : 'Concurrent probes produced excessive queueing or stalls.',
evidence: [target], remediation: 'Bound outbound concurrency and remove serial bottlenecks.',
});
}
verification/src/checks/performance-case.mjs
+62
View File
@@ -0,0 +1,62 @@
import { newPage, stabilize } from '../browser/session.mjs';
import { pageMetrics } from '../browser/metrics.mjs';
import { hasRuntimeProblems } from '../browser/observed.mjs';
async function scrollFrames(page) {
return page.evaluate(async () => {
const frames = [];
const started = performance.now();
let previous = started;
await new Promise((resolve) => {
const step = (now) => {
frames.push(now - previous); previous = now;
const progress = Math.min(1, (now - started) / 2500);
scrollTo(0, Math.max(0, document.documentElement.scrollHeight - innerHeight) * progress);
if (progress < 1) requestAnimationFrame(step); else resolve();
};
requestAnimationFrame(step);
});
frames.sort((a, b) => a - b);
return { count: frames.length, p95: frames[Math.ceil(frames.length * .95) - 1] || 0,
over34: frames.filter((item) => item > 34).length,
distance: Math.max(0, document.documentElement.scrollHeight - innerHeight) };
});
}
export async function inspectScrollCase(ctx, route, viewport) {
const session = await newPage(ctx.state.browser, ctx, viewport);
try {
const cdp = await session.context.newCDPSession(session.page);
await cdp.send('Emulation.setCPUThrottlingRate', { rate: 4 });
const response = await session.page.goto(`${ctx.config.localUrl}${route}`, {
waitUntil: 'domcontentloaded', timeout: ctx.config.navigationTimeoutMs,
});
await session.page.waitForLoadState('load', { timeout: ctx.config.navigationTimeoutMs }).catch(() => {});
await stabilize(session.page);
await session.page.evaluate(() => { if (window.__kvMetrics) window.__kvMetrics.longTasks = []; });
const frames = await scrollFrames(session.page);
const metrics = await pageMetrics(session.page);
return { route, viewport: viewport.name, status: response?.status() || 0, frames, metrics, observed: session.observed };
} catch (error) {
return { route, viewport: viewport.name, error: error instanceof Error ? error.stack || error.message : String(error), observed: session.observed };
} finally { await session.context.close(); }
}
function frameProblems(result) {
const problems = [];
if (!result.frames) return ['frame metrics missing'];
if (result.frames.count < 30) problems.push('insufficient animation frames');
if (result.frames.p95 > 34) problems.push(`frame p95 ${result.frames.p95}ms`);
if (result.frames.over34 / Math.max(result.frames.count, 1) > .05) problems.push('more than 5% frames exceed 34ms');
return problems;
}
export function performanceProblems(result, config) {
const problems = frameProblems(result);
const longTaskTotal = result.metrics?.longTasks.reduce((sum, item) => sum + item, 0) || 0;
if (result.error) problems.push(result.error);
if (!result.status || result.status >= 400) problems.push(`HTTP ${result.status || 0}`);
if (longTaskTotal > config.maxLongTaskMs) problems.push(`long tasks ${longTaskTotal}ms`);
if (hasRuntimeProblems(result.observed)) problems.push('runtime or network errors');
return { problems, longTaskTotal };
}
verification/src/checks/performance.mjs
+44
View File
@@ -0,0 +1,44 @@
import fs from 'node:fs';
import path from 'node:path';
import { finding } from '../core/finding.mjs';
import { walk, writeJson } from '../core/files.mjs';
import { inspectScrollCase, performanceProblems } from './performance-case.mjs';
function bundleMetrics(ctx) {
const root = path.join(ctx.config.root, '.next', 'static');
if (!fs.existsSync(root)) return { count: 0, totalBytes: 0, largest: [] };
const bundles = walk(root, (file) => file.endsWith('.js')).map((file) => ({
file: path.relative(ctx.config.root, file), bytes: fs.statSync(file).size,
}));
return { count: bundles.length, totalBytes: bundles.reduce((sum, item) => sum + item.bytes, 0),
largest: bundles.sort((a, b) => b.bytes - a.bytes).slice(0, 20) };
}
export async function checkPerformance(ctx) {
if (!ctx.state.browser || !ctx.state.appReady || !ctx.state.pageRoutes) return;
const results = [];
for (const viewport of ctx.config.viewports) {
for (const route of ctx.state.pageRoutes) results.push(await inspectScrollCase(ctx, route, viewport));
}
const evaluated = results.map((result) => ({ result, ...performanceProblems(result, ctx.config) }));
const failures = evaluated.filter((item) => item.problems.length);
const bundle = bundleMetrics(ctx);
const target = path.join(ctx.dirs.metrics, 'performance.json');
writeJson(target, { cases: evaluated, bundle });
finding(ctx, {
id: 'performance.scroll-jank', category: 'performance', title: 'Every route and viewport scrolls smoothly under 4× CPU throttling',
status: failures.length ? 'FAIL' : 'PASS', severity: 'high',
expected: 'All route/viewports: >=30 frames, p95 <=34ms, >34ms <=5%, long tasks <=500ms, no runtime/network errors',
actual: failures.length ? JSON.stringify(failures.map((item) => ({ route: item.result.route, viewport: item.result.viewport, problems: item.problems })))
: `${results.length} route/viewport cases passed`,
reason: failures.length ? 'At least one throttled surface exceeded its frame, task, HTTP, or runtime budget.' : 'Every enumerated surface stayed within the throttled interaction budget.',
evidence: [target], remediation: 'Profile the exact route/viewport case, reduce render scope, virtualize lists, and remove layout thrashing.',
});
const largest = bundle.largest[0]?.bytes || 0;
finding(ctx, {
id: 'performance.bundle-size', category: 'performance', title: 'Client JavaScript bundle size is bounded',
status: largest <= 1_000_000 ? 'PASS' : 'WARN', severity: 'medium', expected: 'Largest emitted JS asset <= 1,000,000 raw bytes',
actual: JSON.stringify(bundle), reason: largest <= 1_000_000 ? 'No emitted asset exceeds the guardrail.' : 'A very large asset increases parse, compile, and low-end device latency.',
evidence: [target], remediation: 'Split heavy dependencies and defer feature code outside initial routes.',
});
}
verification/src/checks/pr-policy.mjs
+34
View File
@@ -0,0 +1,34 @@
import fs from 'node:fs';
import path from 'node:path';
import { runCommand } from '../core/command.mjs';
import { finding } from '../core/finding.mjs';
import { writeJson } from '../core/files.mjs';
import { evaluatePrEvidence } from '../policy/pr-evidence.mjs';
export async function checkPrPolicy(ctx) {
const eventPath = process.env.GITHUB_EVENT_PATH;
let event = null;
try { if (eventPath) event = JSON.parse(fs.readFileSync(eventPath, 'utf8')); } catch { /* reported below */ }
if (!event?.pull_request) return finding(ctx, {
id: 'policy.pr-regression-evidence', category: 'policy', title: 'Pull requests declare regression evidence',
status: 'SKIP', severity: 'high', expected: 'GitHub pull_request event', actual: eventPath ? 'Non-PR or invalid event' : 'Local/non-PR run',
reason: 'PR-body policy only applies when a pull-request event fixture is supplied.',
remediation: 'Use verification/history/pr-evidence-template.md when preparing pull-request evidence.',
});
const base = event.pull_request.base?.sha;
const head = event.pull_request.head?.sha;
const diff = await runCommand(ctx, 'pr-changed-files', 'git', ['diff', '--name-only', `${base}...${head}`], { timeoutMs: 60_000 });
const changedFiles = diff.code === 0 ? diff.tail.split(/\r?\n/).filter(Boolean) : [];
const result = diff.code === 0
? evaluatePrEvidence(ctx.config.root, changedFiles, event.pull_request.body || '')
: { ok: false, errors: [`git diff failed with exit ${diff.code}`], codeFiles: [], parsed: {} };
const target = path.join(ctx.dirs.raw, 'pr-regression-evidence.json');
writeJson(target, { pullRequest: event.pull_request.number, base, head, changedFiles, result });
finding(ctx, {
id: 'policy.pr-regression-evidence', category: 'policy', title: 'Pull requests declare regression evidence',
status: result.ok ? 'PASS' : 'FAIL', severity: 'critical', expected: 'Historical-Refs plus valid executable Regression-Evidence for code changes',
actual: result.ok ? `${result.codeFiles.length} code/config files covered` : JSON.stringify(result.errors),
reason: result.ok ? 'The PR names checkable regression proof and directly traces declared historical records.' : 'The PR can change behavior without reviewable regression proof.',
evidence: [target, diff.outputPath], remediation: 'Complete the local template fields, add executable evidence, and tag every declared historical item.',
});
}
verification/src/checks/preflight.mjs
+75
View File
@@ -0,0 +1,75 @@
import fs from 'node:fs';
import path from 'node:path';
import { runCommand, runNpm } from '../core/command.mjs';
import { finding } from '../core/finding.mjs';
import { readJson } from '../core/files.mjs';
function major(version) {
return Number(String(version).replace(/^v/, '').split('.')[0]);
}
function findChrome() {
const candidates = [
process.env.CHROME_PATH,
process.env.PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH,
'/Applications/Google Chrome.app/Contents/MacOS/Google Chrome',
'/usr/bin/google-chrome',
'/usr/bin/google-chrome-stable',
'/usr/bin/chromium',
'/usr/bin/chromium-browser',
];
return candidates.find((file) => file && fs.existsSync(file)) || null;
}
export function verificationChange(line) {
const file = line.slice(3).split(' -> ').pop();
return file.startsWith('verification/');
}
export async function checkWorkspaceBoundary(ctx, phase = 'preflight') {
const git = await runCommand(ctx, `git-status-${phase}`, 'git', ['status', '--porcelain=v1'], { timeoutMs: 30_000 });
const outside = git.tail.split('\n').filter(Boolean).filter((line) => !verificationChange(line));
finding(ctx, {
id: `${phase}.business-tree`, category: phase, title: 'Business source tree has no unrelated edits',
status: outside.length ? 'FAIL' : 'PASS', severity: 'high', expected: 'No changes outside verification/',
actual: outside.length ? outside.join('\n') : 'clean',
reason: outside.length ? 'A command changed files outside the authorized verification boundary.' : 'Only the validation scope is changed.',
evidence: [git.outputPath], remediation: 'Move generated state below verification/ and restore every outside path.',
});
}
export async function checkPreflight(ctx) {
const pkg = readJson(path.join(ctx.config.root, 'package.json'));
const release = readJson(path.join(ctx.config.root, 'app-release.json'));
const versionsMatch = pkg.version === release.currentVersion && pkg.version === release.releases?.[0]?.version;
finding(ctx, {
id: 'preflight.version-consistency', category: 'preflight', title: 'Local version metadata agrees',
status: versionsMatch ? 'PASS' : 'FAIL', severity: 'high', expected: 'package.json, currentVersion, and first release match',
actual: JSON.stringify({ package: pkg.version, current: release.currentVersion, release: release.releases?.[0]?.version }),
reason: versionsMatch ? 'All local release sources agree.' : 'Release sources disagree and can publish ambiguous artifacts.',
remediation: 'Update all version sources atomically before release.',
});
const nodeMajor = major(process.version);
finding(ctx, {
id: 'preflight.node', category: 'preflight', title: 'Node.js runtime is supported',
status: nodeMajor >= 20 && nodeMajor <= 26 ? 'PASS' : 'WARN', severity: 'medium', expected: 'Node.js 20 through 26',
actual: process.version, reason: nodeMajor >= 20 && nodeMajor <= 26 ? 'Runtime is within the tested range.' : 'Runtime is outside the declared range.',
remediation: 'Use an LTS version supported by Next.js and the repository.',
});
const install = await runNpm(ctx, 'npm-ci-root', ['ci', '--no-audit', '--no-fund'], { timeoutMs: ctx.config.commandTimeoutMs });
finding(ctx, {
id: 'preflight.root-dependencies', category: 'preflight', title: 'Root dependencies exactly match package-lock.json',
status: install.code === 0 ? 'PASS' : 'FAIL', severity: 'critical', expected: 'npm ci exit 0', actual: `exit ${install.code}`,
reason: install.code === 0 ? 'The project commands run against a deterministic dependency tree.' : 'Tests against stale or partial dependencies are not authoritative.',
evidence: [install.outputPath], remediation: 'Repair package.json/package-lock.json or registry access, then rerun.', durationMs: install.durationMs,
});
const chrome = findChrome();
finding(ctx, {
id: 'preflight.chrome', category: 'preflight', title: 'Chrome executable is available',
status: chrome ? 'PASS' : 'FAIL', severity: 'high', expected: 'Chrome on macOS or Linux', actual: chrome,
reason: chrome ? 'UI checks can launch an isolated browser.' : 'UI checks cannot launch the required browser.',
remediation: 'Install Google Chrome or configure a supported executable.',
});
ctx.state.chromePath = chrome;
await checkWorkspaceBoundary(ctx);
}
verification/src/checks/proxy.mjs
+79
View File
@@ -0,0 +1,79 @@
import path from 'node:path';
import { finding } from '../core/finding.mjs';
import { request } from '../core/http.mjs';
import { writeJson } from '../core/files.mjs';
export const PROXY_FIXTURES = Object.freeze({
binary: 'https://httpbingo.org/image/png',
range: 'https://httpbingo.org/range/1024',
hls: 'https://test-streams.mux.dev/x36xhzz/x36xhzz.m3u8',
notFound: 'https://httpbingo.org/status/404',
redirect: 'https://httpbingo.org/redirect-to?url=https%3A%2F%2Fhttpbingo.org%2Fimage%2Fpng',
});
async function collectCases(ctx, proxy) {
const cases = {
missing: await request(`${ctx.config.localUrl}/api/proxy`),
fileProtocol: await request(proxy('file:///etc/hosts')),
loopback: await request(proxy(`${ctx.config.fixtureUrl}/test.mp4`)),
};
if (ctx.config.offline) return cases;
return Object.assign(cases, {
binary: await request(proxy(PROXY_FIXTURES.binary)),
range: await request(proxy(PROXY_FIXTURES.range), { headers: { range: 'bytes=0-99' } }),
hls: await request(proxy(PROXY_FIXTURES.hls)),
notFound: await request(proxy(PROXY_FIXTURES.notFound)),
redirect: await request(proxy(PROXY_FIXTURES.redirect)),
});
}
function proxyFunctional(cases) {
const statuses = cases.missing.status === 400 && cases.binary?.status === 200 && cases.range?.status === 206;
const payloads = cases.range?.bytes === 100 && cases.hls?.status === 200 && cases.hls?.body.includes('/api/proxy?url=');
return Boolean(statuses && payloads && cases.notFound?.status === 404 && cases.redirect?.status === 200);
}
function caseSummary(cases) {
return Object.fromEntries(Object.entries(cases).map(([key, value]) => [key,
{ status: value.status, bytes: value.bytes, headers: value.headers }]));
}
function addFunctionalFinding(ctx, cases, target) {
const functional = proxyFunctional(cases);
const offline = ctx.config.offline;
finding(ctx, {
id: 'proxy.functional', category: 'proxy', title: 'Media proxy preserves errors, ranges, redirects, CORS, and rewrites HLS',
status: offline ? 'SKIP' : functional ? 'PASS' : 'FAIL', severity: 'critical', expected: 'All six public proxy contracts pass',
actual: JSON.stringify(caseSummary(cases)),
reason: offline ? 'Public proxy fixtures are unavailable in offline mode.' : functional ? 'Public upstream behavior survived the proxy contract.' : 'One or more core proxy behaviors are broken.',
evidence: [target], remediation: offline ? 'Rerun online.' : 'Fix forwarding, range/header preservation, redirect handling, or playlist rewriting.',
});
}
function addSecurityFindings(ctx, cases, target) {
const blocksUnsupported = cases.fileProtocol.status >= 400 && cases.fileProtocol.status < 500;
finding(ctx, {
id: 'proxy.protocol-validation', category: 'security', title: 'Proxy rejects unsupported protocols before fetching',
status: blocksUnsupported ? 'PASS' : 'FAIL', severity: 'high', expected: 'Controlled HTTP 4xx for file://', actual: cases.fileProtocol.status,
reason: blocksUnsupported ? 'Unsupported protocols are rejected as client input.' : 'Unsupported protocols fall into a server error instead of explicit validation.',
evidence: [target], remediation: 'Allow only http: and https: before invoking fetch.',
});
const loopbackFetched = cases.loopback.status < 400 || cases.loopback.status >= 500;
finding(ctx, {
id: 'proxy.private-network-ssrf', category: 'security', title: 'Proxy blocks loopback and private-network targets',
status: loopbackFetched ? 'FAIL' : 'PASS', severity: 'critical', expected: 'Controlled HTTP 4xx for loopback', actual: `loopback HTTP ${cases.loopback.status}`,
reason: loopbackFetched ? 'The public proxy route can reach 127.0.0.1, demonstrating an SSRF primitive on self-hosted deployments.' : 'Private address access was blocked.',
impact: 'An exposed self-hosted instance may reach internal services available to the application host.', evidence: [target],
remediation: 'Resolve DNS safely and reject loopback, link-local, private, multicast, and metadata-service address ranges across redirects.',
});
}
export async function checkProxy(ctx) {
if (!ctx.state.appReady) return;
const proxy = (url) => `${ctx.config.localUrl}/api/proxy?url=${encodeURIComponent(url)}`;
const cases = await collectCases(ctx, proxy);
const target = path.join(ctx.dirs.raw, 'proxy-contracts.json');
writeJson(target, cases);
addFunctionalFinding(ctx, cases, target);
addSecurityFindings(ctx, cases, target);
}
verification/src/checks/runtime-start.mjs
+32
View File
@@ -0,0 +1,32 @@
import { finding } from '../core/finding.mjs';
import { startProcess, waitForUrl } from '../core/service.mjs';
import { createMedia } from '../fixture/media.mjs';
import { startFixtureServer } from '../fixture/server.mjs';
export async function startRuntime(ctx) {
await createMedia(ctx);
await startFixtureServer(ctx);
const fixture = await waitForUrl(`${ctx.config.fixtureUrl}/health`, 5000);
finding(ctx, {
id: 'runtime.fixture-server', category: 'harness', title: 'Local deterministic fixture server is reachable',
status: fixture.ok ? 'PASS' : 'FAIL', severity: 'critical', expected: 'HTTP 200', actual: fixture.ok ? fixture.status : fixture.error,
reason: fixture.ok ? 'API, proxy, latency, and media tests can use controlled upstream behavior.' : 'Controlled integration tests cannot run.',
remediation: 'Free port 34174 and rerun.',
});
const production = ctx.state.buildOk;
const args = production ? ['start'] : ['run', 'dev'];
const service = await startProcess(ctx, production ? 'next-start' : 'next-dev', 'npm', args, {
env: { PORT: String(ctx.config.localPort), HOSTNAME: '127.0.0.1', NEXT_TELEMETRY_DISABLED: '1' },
url: ctx.config.localUrl, timeoutMs: 120_000,
});
const ready = service.ready?.ok;
ctx.state.appReady = ready;
finding(ctx, {
id: 'runtime.local-server', category: 'runtime', title: 'KVideo local server starts and responds',
status: ready ? 'PASS' : 'FAIL', severity: 'critical', expected: `Reachable ${production ? 'production' : 'development fallback'} server`,
actual: ready ? `HTTP ${service.ready.status}` : service.ready?.error || 'not ready',
reason: ready ? 'Browser and API integration checks can execute.' : 'No local application endpoint became ready.',
impact: production ? '' : 'Production build failed, so UI evidence uses a development fallback.',
evidence: [service.outputPath], remediation: 'Fix startup/build errors or release port 34173.',
});
}
verification/src/checks/security-headers.mjs
+43
View File
@@ -0,0 +1,43 @@
import path from 'node:path';
import { finding } from '../core/finding.mjs';
import { request } from '../core/http.mjs';
import { writeJson } from '../core/files.mjs';
function evaluate(headers, remote) {
const checks = {
contentTypeOptions: headers['x-content-type-options'] === 'nosniff',
referrerPolicy: Boolean(headers['referrer-policy']),
framing: Boolean(headers['x-frame-options'] || headers['content-security-policy']?.includes('frame-ancestors')),
permissionsPolicy: Boolean(headers['permissions-policy']),
contentSecurityPolicy: Boolean(headers['content-security-policy']),
hsts: !remote || Boolean(headers['strict-transport-security']),
noPoweredBy: !headers['x-powered-by'],
};
return { checks, missing: Object.entries(checks).filter(([, ok]) => !ok).map(([name]) => name) };
}
export async function checkSecurityHeaders(ctx) {
if (!ctx.state.appReady) return;
const local = await request(ctx.config.localUrl);
const remote = ctx.config.offline ? null : await request(ctx.config.referenceUrl);
const localEval = evaluate(local.headers, false);
const remoteEval = remote ? evaluate(remote.headers, true) : null;
const assets = await Promise.all(['/manifest.json', '/sw.js', '/icon.png'].map(async (route) => ({ route, response: await request(`${ctx.config.localUrl}${route}`) })));
const target = path.join(ctx.dirs.raw, 'security-headers.json');
writeJson(target, { local, remote, localEval, remoteEval, assets });
finding(ctx, {
id: 'security.response-headers', category: 'security', title: 'Application responses set a complete browser security-header baseline',
status: localEval.missing.length || remoteEval?.missing.length ? 'FAIL' : 'PASS', severity: 'high', expected: 'nosniff, referrer, framing, permissions, CSP, HSTS remotely, no powered-by',
actual: JSON.stringify({ localMissing: localEval.missing, remoteMissing: remoteEval?.missing || [] }),
reason: localEval.missing.length || remoteEval?.missing.length ? 'One or more standard browser defenses are absent.' : 'Both surfaces provide the configured defense-in-depth headers.',
evidence: [target], remediation: 'Define headers in Next.js/self-hosted responses and Cloudflare configuration; use a restrictive CSP tested against required scripts.',
});
const assetFailures = assets.filter((item) => item.response.status !== 200);
finding(ctx, {
id: 'runtime.pwa-assets', category: 'runtime', title: 'PWA manifest, service worker, and icon are reachable',
status: assetFailures.length ? 'FAIL' : 'PASS', severity: 'medium', expected: 'HTTP 200 for all required PWA assets',
actual: JSON.stringify(assets.map((item) => ({ route: item.route, status: item.response.status, bytes: item.response.bytes }))),
reason: assetFailures.length ? 'At least one install/offline asset is missing.' : 'All declared PWA assets are served.', evidence: [target],
remediation: 'Restore the missing public asset and verify its content type and cache behavior.',
});
}
verification/src/checks/security-scan.mjs
+64
View File
@@ -0,0 +1,64 @@
import fs from 'node:fs';
import path from 'node:path';
import { parse } from '@typescript-eslint/typescript-estree';
import { finding } from '../core/finding.mjs';
import { relative, walk, writeJson } from '../core/files.mjs';
import { children } from './ast-walk.mjs';
// GH-PR: 235
const textExt = new Set(['.ts', '.tsx', '.js', '.mjs', '.json', '.yml', '.yaml', '.toml', '.md']);
const patterns = [
['private-key', /-----BEGIN (?:RSA |EC |OPENSSH )?PRIVATE KEY-----/],
['github-token', /\bgh[opusr]_[A-Za-z0-9_]{20,}\b/],
['aws-access-key', /\bAKIA[0-9A-Z]{16}\b/],
['generic-secret-assignment', /(?:secret|token|password|api[_-]?key)\s*[:=]\s*['"][^'"\n]{12,}['"]/i],
];
export function findDangerousConstructs(file, text) {
const ast = parse(text, {
jsx: file.endsWith('.tsx') || file.endsWith('.jsx'),
errorOnUnknownASTType: false,
});
const constructs = new Set();
const visit = (node) => {
if (node.type === 'JSXAttribute' && node.name?.name === 'dangerouslySetInnerHTML') {
constructs.add('dangerouslySetInnerHTML');
}
if (node.type === 'CallExpression' && node.callee?.type === 'Identifier' && node.callee.name === 'eval') {
constructs.add('eval');
}
for (const child of children(node)) visit(child);
};
visit(ast);
return [...constructs].map((construct) => ({ file, construct }));
}
export async function checkSecurityScan(ctx) {
const files = walk(ctx.config.root, (file) => textExt.has(path.extname(file)) && !file.includes('/verification/artifacts/'));
const hits = [];
for (const file of files) {
const lines = fs.readFileSync(file, 'utf8').split(/\r?\n/);
lines.forEach((line, index) => {
for (const [name, pattern] of patterns) if (pattern.test(line)) hits.push({ file: relative(ctx.config.root, file), line: index + 1, pattern: name });
});
}
const target = path.join(ctx.dirs.metrics, 'secret-scan.json');
writeJson(target, hits);
finding(ctx, {
id: 'security.static-secrets', category: 'security', title: 'Repository contains no obvious committed secrets',
status: hits.length ? 'FAIL' : 'PASS', severity: 'critical', expected: '0 credential-pattern matches',
actual: hits.length ? JSON.stringify(hits) : '0',
reason: hits.length ? 'Credential-like material appears in repository text. Values are intentionally omitted from logs.' : 'No configured credential signature was found.',
evidence: [target], remediation: 'Remove and rotate confirmed credentials; replace false positives with safe fixtures.',
});
const dangerous = files.filter((file) => ['.ts', '.tsx', '.js', '.mjs'].includes(path.extname(file))).flatMap((file) => {
return findDangerousConstructs(relative(ctx.config.root, file), fs.readFileSync(file, 'utf8'));
});
finding(ctx, {
id: 'security.dangerous-constructs', category: 'security', title: 'Dangerous runtime constructs are inventoried',
status: dangerous.length ? 'WARN' : 'PASS', severity: 'medium', expected: 'No eval or unreviewed raw HTML injection', actual: JSON.stringify(dangerous),
reason: dangerous.length ? 'These constructs expand injection risk and require contextual review.' : 'No configured dangerous construct was found.',
remediation: 'Verify sanitization and replace raw execution or HTML injection where possible.',
});
}
verification/src/checks/source-policy.mjs
+45
View File
@@ -0,0 +1,45 @@
import path from 'node:path';
import { finding } from '../core/finding.mjs';
import { lineCount, relative, walk, writeJson } from '../core/files.mjs';
const codeExt = new Set(['.ts', '.tsx', '.js', '.jsx', '.mjs', '.cjs', '.css', '.scss', '.swift', '.kt', '.kts',
'.json', '.yaml', '.yml', '.xml', '.svg', '.html', '.sh', '.toml', '.properties']);
const codeNames = new Set(['Dockerfile', 'gradlew']);
const generatedNames = /(?:package-lock\.json|yarn\.lock|pnpm-lock\.yaml|\.min\.(?:js|css))$/;
export function projectSourceFile(file) {
return !generatedNames.test(file) && (codeExt.has(path.extname(file)) || codeNames.has(path.basename(file)));
}
function inventory(root, files) {
return files.map((file) => ({ file: relative(root, file), lines: lineCount(file) })).sort((a, b) => b.lines - a.lines);
}
export async function checkSourcePolicy(ctx) {
const root = ctx.config.root;
const projectFiles = walk(root, (file) => projectSourceFile(file) && !file.includes('/verification/'));
const project = inventory(root, projectFiles);
const oversized = project.filter((item) => item.lines > ctx.config.maxSourceLines);
const totalLines = project.reduce((sum, item) => sum + item.lines, 0);
writeJson(path.join(ctx.dirs.metrics, 'source-inventory.json'), { totalLines, files: project, oversized });
finding(ctx, {
id: 'source.project-line-policy', category: 'source', title: 'Project source files respect the 150-line policy',
status: oversized.length ? 'FAIL' : 'PASS', severity: 'high', expected: `Every source file <= ${ctx.config.maxSourceLines} lines`,
actual: `${oversized.length}/${project.length} files exceed the limit; ${totalLines} total lines`,
reason: oversized.length ? 'Large files concentrate responsibilities and materially increase review and regression risk.' : 'All source files meet the limit.',
impact: oversized.slice(0, 20).map((item) => `${item.file}:${item.lines}`).join(', '),
evidence: [path.join(ctx.dirs.metrics, 'source-inventory.json')], remediation: 'Split oversized business files by cohesive responsibility in a separate change.',
});
const authored = walk(ctx.config.verifyDir, (file) => !file.includes('/node_modules/') && !file.includes('/artifacts/')
&& !file.includes('/tmp/') && path.basename(file) !== 'package-lock.json');
const validator = inventory(ctx.config.verifyDir, authored);
const validatorOversized = validator.filter((item) => item.lines > ctx.config.maxSourceLines);
writeJson(path.join(ctx.dirs.metrics, 'verification-inventory.json'), validator);
finding(ctx, {
id: 'source.verification-line-policy', category: 'harness', title: 'Verification files respect the 150-line policy',
status: validatorOversized.length ? 'FAIL' : 'PASS', severity: 'critical', expected: `Every authored verification file <= ${ctx.config.maxSourceLines} lines`,
actual: validatorOversized.length ? JSON.stringify(validatorOversized) : `${validator.length} files comply`,
reason: validatorOversized.length ? 'The delivered validation code violates the explicit file-size constraint.' : 'The validation implementation is partitioned within the limit.',
evidence: [path.join(ctx.dirs.metrics, 'verification-inventory.json')], remediation: 'Split the listed verification files before trusting or publishing the suite.',
});
}
verification/src/checks/static-tools.mjs
+62
View File
@@ -0,0 +1,62 @@
import fs from 'node:fs';
import path from 'node:path';
import { runCommand, runNpm } from '../core/command.mjs';
import { finding } from '../core/finding.mjs';
import { checkCoverage } from './coverage.mjs';
// GH-ISSUE: 80,140,150,174; GH-PR: 5,47,235
function commandFinding(ctx, result, options) {
const { id, title, severity, expected = 'exit code 0' } = options;
const ok = result.code === 0 && !result.timedOut;
finding(ctx, {
id, category: 'static', title, status: ok ? 'PASS' : 'FAIL', severity, expected,
actual: result.timedOut ? `timed out after ${result.durationMs}ms` : `exit code ${result.code}`,
reason: ok ? 'The command completed successfully.' : 'The authoritative project command failed; the raw output contains exact diagnostics.',
impact: ok ? '' : result.tail.slice(-2000), evidence: [result.outputPath],
remediation: 'Resolve every diagnostic in the raw output, then rerun the complete chain.', durationMs: result.durationMs,
});
return ok;
}
export async function checkStaticTools(ctx) {
const test = await runCommand(ctx, 'npm-test-regression', process.execPath,
[path.join(ctx.config.verifyDir, 'src', 'run-regression.mjs')]);
ctx.state.testsOk = commandFinding(ctx, test, { id: 'static.unit-tests', title: 'Repository unit tests pass', severity: 'critical' });
await checkCoverage(ctx);
const lint = await runNpm(ctx, 'npm-lint', ['run', 'lint']);
ctx.state.lintOk = commandFinding(ctx, lint, { id: 'static.eslint', title: 'ESLint reports no errors or warnings', severity: 'high' });
const types = await runCommand(ctx, 'typescript', 'npx', ['--no-install', 'tsc', '--noEmit', '--incremental', 'false']);
ctx.state.typesOk = commandFinding(ctx, types, { id: 'static.typescript', title: 'Full TypeScript check passes', severity: 'high' });
const integrity = await runNpm(ctx, 'npm-ls', ['ls', '--all', '--json']);
commandFinding(ctx, integrity, { id: 'static.dependency-integrity', title: 'Installed dependency graph is valid', severity: 'high' });
if (!ctx.config.offline) await checkAudit(ctx);
else finding(ctx, {
id: 'static.npm-audit', category: 'static', title: 'Dependency vulnerability audit', status: 'SKIP', severity: 'high',
expected: 'Online npm audit', actual: '--offline', reason: 'The run explicitly disabled network checks.', remediation: 'Rerun without --offline.',
});
if (!ctx.config.quick) {
const pages = await runCommand(ctx, 'cloudflare-pages-build', 'npx', ['--yes', '[email protected]', 'run', 'pages:build'], {
timeoutMs: ctx.config.commandTimeoutMs, env: { npm_config_package_lock: 'false', npm_config_save: 'false' },
});
ctx.state.pagesBuildOk = commandFinding(ctx, pages, { id: 'static.cloudflare-build', title: 'Cloudflare Pages build succeeds', severity: 'critical' });
}
const build = await runNpm(ctx, 'next-build', ['run', 'build'], { timeoutMs: ctx.config.commandTimeoutMs });
ctx.state.buildOk = commandFinding(ctx, build, { id: 'static.production-build', title: 'Production Next.js build succeeds', severity: 'critical' });
}
async function checkAudit(ctx) {
const result = await runNpm(ctx, 'npm-audit', ['audit', '--omit=dev', '--json']);
let audit = null;
try { audit = JSON.parse(fs.readFileSync(result.outputPath, 'utf8')); } catch { /* malformed audit output */ }
const vulnerabilities = audit?.metadata?.vulnerabilities || {};
const severe = (vulnerabilities.critical || 0) + (vulnerabilities.high || 0);
const ok = result.code === 0 && severe === 0;
finding(ctx, {
id: 'static.npm-audit', category: 'security', title: 'Production dependencies have no known high/critical vulnerabilities',
status: ok ? 'PASS' : 'FAIL', severity: 'critical', expected: '0 high and 0 critical vulnerabilities',
actual: audit ? JSON.stringify(vulnerabilities) : `unparseable output; exit ${result.code}`,
reason: ok ? 'npm advisory data reports no severe production vulnerability.' : 'The dependency audit failed or reports severe vulnerabilities.',
evidence: [result.outputPath], remediation: 'Upgrade, replace, or explicitly mitigate every severe advisory.', durationMs: result.durationMs,
});
}
verification/src/checks/ui-action-entry.mjs
+13
View File
@@ -0,0 +1,13 @@
function blockedExploration(reason) {
return { eligible: false, queued: false, subsumed: false, reason, newStateFacts: [] };
}
export function skippedActionEntry(input) {
const { route, viewport, snapshot, steps, action, urlAfter, reason, details = {} } = input;
return {
route, viewport: viewport.name, state: snapshot.hash, depth: steps.length,
steps: steps.map((item) => item.key), action,
result: { ok: true, skipped: true, reason, ...details },
changed: false, urlAfter, exploration: blockedExploration(reason),
};
}
verification/src/checks/ui-action-explorer.mjs
+149
View File
@@ -0,0 +1,149 @@
import path from 'node:path';
import { log } from '../core/log.mjs';
import { performAction, scanActions, stateSnapshot } from '../browser/actions.mjs';
import { assessAction, captureActionEvidence } from '../browser/action-effects.mjs';
import { prepareActionState } from '../browser/action-state.mjs';
import { actionExecutionKey, classifyStateTransition, registerNovelState } from '../policy/action-coverage.mjs';
import { skippedActionEntry } from './ui-action-entry.mjs';
import { replayState } from './ui-action-replay.mjs';
function safe(value) {
return value.replace(/[^a-zA-Z0-9]+/g, '-').replace(/^-|-$/g, '') || 'home';
}
function staysWithinRoute(ctx, route, action, urlAfter) {
const start = new URL(route, ctx.config.localUrl);
const target = action.href ? new URL(action.href, start) : new URL(urlAfter);
return target.origin === start.origin && target.pathname === start.pathname;
}
async function failureShot(page, ctx, viewport, route, sequence) {
const name = `action-failure-${viewport.name}-${safe(route)}-${sequence}.png`;
await page.screenshot({ path: path.join(ctx.dirs.screenshots, name), fullPage: true }).catch(() => {});
}
async function exerciseAction(input) {
const { ctx, session, route, viewport, steps, action, snapshot, fixtureFile, sequence, discovered } = input;
const stepKeys = steps.map((item) => item.key);
if (action.disabled) {
const entry = skippedActionEntry({ route, viewport, snapshot, steps, action, urlAfter: session.page.url(), reason: 'control disabled' });
return { entry, next: null, subsumed: 0 };
}
const reset = await replayState({ page: session.page, ctx, route, steps, fixtureFile, expectedState: snapshot });
if (!reset.ok) {
const entry = { route, viewport: viewport.name, state: snapshot.hash, depth: steps.length,
steps: stepKeys, phase: 'action-reset', action, result: reset,
exploration: classifyStateTransition(discovered, null, 'action state reset failed') };
return { entry, next: null, subsumed: 0 };
}
await prepareActionState(session.page, action);
await session.page.waitForTimeout(100);
const preparedActions = await scanActions(session.page);
const preparedState = stateSnapshot(session.page.url(), preparedActions);
const before = await captureActionEvidence(session.page, session.observed, preparedState);
let interaction;
try {
interaction = await performAction(session.page, action, fixtureFile);
} catch (error) {
interaction = { ok: false, reason: error instanceof Error ? error.message : String(error) };
}
const observed = await observeOutcome(session, action, interaction, before);
const result = observed.assessment.ok ? interaction : { ...interaction, ok: false,
reason: observed.assessment.reason, failureKind: observed.assessment.failureKind };
const entry = { route, viewport: viewport.name, state: snapshot.hash, depth: steps.length, steps: stepKeys,
action, interaction, assessment: observed.assessment, result, changed: observed.assessment.stateChanged,
effects: observed.assessment.effects, urlAfter: session.page.url() };
if (!result.ok) {
log(ctx, 'error', 'ui.action-failure', 'Runtime control interaction failed', entry);
await failureShot(session.page, ctx, viewport, route, sequence);
}
const blockedBy = !result.ok ? 'action failed' : !observed.assessment.stateChanged ? 'control state did not change'
: steps.length >= ctx.config.maxActionDepth ? 'maximum action depth reached'
: !staysWithinRoute(ctx, route, action, session.page.url()) ? 'transition left the current route' : '';
const exploration = classifyStateTransition(discovered, observed.afterState, blockedBy);
entry.exploration = exploration;
const next = exploration.queued ? { steps: [...steps, action], expectedState: observed.afterState } : null;
return { entry, next, subsumed: exploration.subsumed ? 1 : 0 };
}
async function observeOutcome(session, action, interaction, before) {
let outcome;
for (let attempt = 0; attempt < 6; attempt += 1) {
await session.page.waitForTimeout(150);
const actions = await scanActions(session.page);
const afterState = stateSnapshot(session.page.url(), actions);
const after = await captureActionEvidence(session.page, session.observed, afterState);
const assessment = assessAction(action, interaction, before, after);
outcome = { after, afterState, assessment };
if (assessment.ok || ['automation', 'runtime'].includes(assessment.failureKind)) break;
}
return outcome;
}
async function inspectState(input) {
const { ctx, session, route, viewport, steps, fixtureFile, expectedState, seen, tested, discovered, remaining, sequence } = input;
const replay = await replayState({ page: session.page, ctx, route, steps, fixtureFile, expectedState });
if (!replay.ok) {
const entry = { route, viewport: viewport.name, depth: steps.length, steps: steps.map((item) => item.key), phase: 'state-replay', result: replay };
log(ctx, 'error', 'ui.action-failure', 'State path replay failed', entry);
await failureShot(session.page, ctx, viewport, route, sequence);
return { entries: [entry], next: [], used: 0, hitCap: false, subsumed: 0, deduplicated: 0 };
}
const { actions, snapshot } = replay;
if (seen.has(snapshot.hash)) return { entries: [], next: [], used: 0, hitCap: false, subsumed: 0, deduplicated: 0 };
seen.add(snapshot.hash);
registerNovelState(discovered, snapshot);
const entries = [];
const next = [];
let used = 0;
let subsumed = 0;
let deduplicated = 0;
let hitCap = false;
for (const action of actions) {
const stateAction = actionExecutionKey(snapshot, action);
if (tested.has(stateAction)) {
entries.push(skippedActionEntry({ route, viewport, snapshot, steps, action, urlAfter: session.page.url(),
reason: 'same location and control semantic state already executed', details: { deduplicated: true } }));
deduplicated += 1; continue;
}
if (used >= remaining) { hitCap = true; break; }
tested.add(stateAction);
const outcome = await exerciseAction({ ...input, action, snapshot, sequence: sequence + entries.length });
entries.push(outcome.entry);
if (outcome.next) next.push(outcome.next);
subsumed += outcome.subsumed;
used += 1;
}
return { entries, next, used, hitCap, subsumed, deduplicated };
}
export async function exploreRoute(input) {
const { ctx, route, viewport } = input;
const queue = [{ steps: [], expectedState: null }];
const seen = new Set();
const tested = new Set();
const discovered = new Set();
const entries = [];
let actions = 0;
let subsumedStates = 0;
let deduplicatedActions = 0;
let hitCap = false;
log(ctx, 'info', 'ui.action-route.start', 'Starting recursive runtime control exploration', { route, viewport: viewport.name });
while (queue.length && actions < ctx.config.maxActionStates) {
const statePath = queue.shift();
const state = await inspectState({ ...input, ...statePath, seen, tested, discovered,
remaining: ctx.config.maxActionStates - actions, sequence: entries.length });
entries.push(...state.entries);
queue.push(...state.next);
actions += state.used;
subsumedStates += state.subsumed;
deduplicatedActions += state.deduplicated;
if (state.hitCap) { hitCap = true; break; }
}
const capped = hitCap || queue.length > 0;
log(ctx, 'info', 'ui.action-route.end', 'Finished recursive runtime control exploration', {
route, viewport: viewport.name, actions, uniqueActions: tested.size, uniqueStates: seen.size,
frontierFacts: discovered.size, subsumedStates, deduplicatedActions, pendingStates: queue.length, capped,
});
return { entries, capped, actions, states: seen.size, frontierFacts: discovered.size, subsumedStates, deduplicatedActions };
}
verification/src/checks/ui-action-replay.mjs
+65
View File
@@ -0,0 +1,65 @@
import { stabilize } from '../browser/session.mjs';
import { performAction, scanActions, stateDifference, stateSnapshot } from '../browser/actions.mjs';
import { prepareReplayBaseline } from '../browser/action-state.mjs';
async function readState(page) {
const actions = await scanActions(page);
return { actions, snapshot: stateSnapshot(page.url(), actions) };
}
async function waitForTransition(page, before) {
for (let attempt = 0; attempt < 6; attempt += 1) {
await page.waitForTimeout(250);
const current = await readState(page);
if (current.snapshot.hash !== before) return current;
}
return null;
}
export async function waitForExpectedState(input) {
const { readCurrent, pause, expectedState, retries = 20 } = input;
let current;
for (let attempt = 0; attempt < retries; attempt += 1) {
current = await readCurrent();
if (!expectedState || current.snapshot.hash === expectedState.hash) return { ok: true, ...current };
await pause();
}
current = await readCurrent();
if (current.snapshot.hash === expectedState.hash) return { ok: true, ...current };
return { ok: false, reason: 'replayed semantic state differs from the discovered state',
expectedHash: expectedState.hash, actualHash: current.snapshot.hash,
...stateDifference(expectedState, current.snapshot) };
}
async function waitForExpected(page, expectedState) {
return waitForExpectedState({
readCurrent: () => readState(page),
pause: () => page.waitForTimeout(250),
expectedState,
});
}
async function replayOnce(input) {
const { page, ctx, route, steps, fixtureFile, expectedState } = input;
await page.goto(`${ctx.config.localUrl}${route}`, { waitUntil: 'domcontentloaded', timeout: ctx.config.navigationTimeoutMs });
await stabilize(page);
await prepareReplayBaseline(page);
for (const action of steps) {
const before = await readState(page);
const result = await performAction(page, action, fixtureFile);
if (!result.ok) return result;
if (!await waitForTransition(page, before.snapshot.hash)) {
return { ok: false, reason: 'replay step did not reproduce its state transition', action: action.key };
}
}
return waitForExpected(page, expectedState);
}
export async function replayState(input) {
let result = { ok: false, reason: 'state replay did not run' };
for (let attempt = 1; attempt <= 3; attempt += 1) {
result = await replayOnce(input);
if (result.ok) return { ...result, attempt };
}
return result;
}
verification/src/checks/ui-actions.mjs
+121
View File
@@ -0,0 +1,121 @@
import fs from 'node:fs';
import path from 'node:path';
import { finding } from '../core/finding.mjs';
import { walk, relative, writeJson } from '../core/files.mjs';
import { newPage } from '../browser/session.mjs';
import { actionCoverageReason, actionCoverageStatus } from '../policy/action-coverage.mjs';
import { exploreRoute } from './ui-action-explorer.mjs';
// GH-ISSUE: 12,21,32,41,146,182; GH-PR: 136,137
function sourceActionInventory(ctx) {
const files = walk(ctx.config.root, (file) => /\.(tsx|jsx)$/.test(file) && !file.includes('/verification/'));
return files.flatMap((file) => fs.readFileSync(file, 'utf8').split(/\r?\n/).flatMap((line, index) =>
/<(button|input|select|textarea)|onClick=|role=["']button/.test(line)
? [{ file: relative(ctx.config.root, file), line: index + 1 }] : []));
}
async function exploreViewport(ctx, viewport, fixtureFile) {
const session = await newPage(ctx.state.browser, ctx, viewport);
await session.context.tracing.start({ screenshots: true, snapshots: true, sources: true });
const results = [];
const cappedRoutes = [];
const emptyRoutes = [];
let subsumedStates = 0;
let deduplicatedActions = 0;
for (const route of ctx.state.pageRoutes) {
const explored = await exploreRoute({ ctx, session, route, viewport, fixtureFile });
results.push(...explored.entries);
subsumedStates += explored.subsumedStates;
deduplicatedActions += explored.deduplicatedActions;
if (explored.capped) cappedRoutes.push(`${viewport.name}:${route}`);
if (explored.actions === 0) emptyRoutes.push(`${viewport.name}:${route}`);
}
const trace = path.join(ctx.dirs.traces, `ui-actions-${viewport.name}.zip`);
await session.context.tracing.stop({ path: trace });
await session.context.close();
return { results, cappedRoutes, emptyRoutes, subsumedStates, deduplicatedActions, observed: session.observed, trace };
}
function addExecutionFinding(ctx, results, skipped, target, traces) {
const effectKinds = new Set(['no-effect', 'media-proof']);
const failures = results.filter((item) => item.result && !item.result.ok && !effectKinds.has(item.result.failureKind));
finding(ctx, {
id: 'ui.action-execution', category: 'ui', title: 'Every discovered runtime control accepts its intended interaction',
status: failures.length ? 'FAIL' : 'PASS', severity: 'critical', expected: '0 click/fill/select/upload failures',
actual: failures.length ? JSON.stringify(failures.slice(0, 30)) : `${results.length - skipped.length} operated; ${skipped.length} disabled/skipped`,
reason: failures.length ? 'A visible runtime control could not be replayed or operated.' : 'All discovered controls were exercised without automation failure.',
evidence: [target, ...traces], remediation: 'Repair unstable selectors, disabled-state logic, click handlers, or the underlying UI exception.',
});
}
function addEffectFinding(ctx, results, target, traces) {
const inspected = results.filter((item) => item.assessment && !item.assessment.idempotent);
const failures = inspected.filter((item) => !item.assessment.ok
&& ['no-effect', 'media-proof'].includes(item.assessment.failureKind));
finding(ctx, {
id: 'ui.action-observable-effects', category: 'ui', title: 'Every operated control produces its intended observable effect',
status: failures.length ? 'FAIL' : 'PASS', severity: 'critical', expected: '0 silent no-op controls and valid media semantics',
actual: failures.length ? JSON.stringify(failures.slice(0, 50)) : `${inspected.length} interactions produced observable effects`,
reason: failures.length ? 'A control accepted automation but did not change UI/state/storage/media/network/navigation or failed dedicated media proof.'
: 'Each non-idempotent interaction produced independently observable evidence.',
evidence: [target, ...traces], remediation: 'Repair the handler or add a deterministic success fixture that exposes the intended effect.',
});
}
function addCoverageFinding(ctx, results, cappedRoutes, emptyRoutes, target) {
const input = { quick: ctx.config.quick, cappedRoutes, emptyRoutes };
const status = actionCoverageStatus(input);
const details = [];
if (emptyRoutes.length) details.push(`zero-control surfaces: ${emptyRoutes.join(', ')}`);
if (cappedRoutes.length) details.push(`coverage cap reached: ${cappedRoutes.join(', ')}`);
finding(ctx, {
id: 'ui.action-state-coverage', category: 'ui', title: 'Every viewport action graph discovers controls and exhausts its queue',
status, severity: 'high', expected: `At least one control per route/viewport; below ${ctx.config.maxActionStates} actions and depth ${ctx.config.maxActionDepth}`,
actual: details.length ? `${details.join('; ')}; ${results.length} results` : `${results.length} results; all queues exhausted`,
reason: actionCoverageReason(input),
evidence: [target], remediation: 'Raise limits or split workflows until every reachable route/viewport state is exhausted.',
});
}
function addInventoryFinding(ctx, data, target) {
const unique = new Set(data.results.filter((item) => item.action)
.map((item) => `${item.viewport}|${item.route}|${item.action.key}`)).size;
finding(ctx, {
id: 'ui.action-inventory', category: 'ui', title: 'Static declarations and runtime controls are fully recorded',
status: 'INFO', severity: 'info', expected: 'Auditable static and runtime populations',
actual: JSON.stringify({ staticDeclarationSites: data.declared.length, runtimeActionInstances: data.results.length,
uniqueRuntimeControls: unique, disabledOrSkipped: data.skipped.length, cappedRoutes: data.cappedRoutes,
zeroControlSurfaces: data.emptyRoutes, subsumedSemanticStates: data.subsumedStates,
deduplicatedSemanticActions: data.deduplicatedActions }),
reason: 'Every target viewport is explored; static declarations remain separate because source sites do not map one-to-one to rendered instances.',
evidence: [target], remediation: 'Inspect declaration sites absent from all reachable runtime states.',
});
}
export async function checkUiActions(ctx) {
if (!ctx.state.browser || !ctx.state.pageRoutes) return;
const fixtureFile = path.join(ctx.dirs.raw, 'import-fixture.json');
fs.writeFileSync(fixtureFile, JSON.stringify({ sources: [{ id: 'file-fixture', name: 'File Fixture',
baseUrl: 'https://verification-fixture.kvideo.invalid', enabled: true, group: 'normal' }] }));
const runs = [];
for (const viewport of ctx.config.viewports) runs.push(await exploreViewport(ctx, viewport, fixtureFile));
const results = runs.flatMap((item) => item.results);
const cappedRoutes = runs.flatMap((item) => item.cappedRoutes);
const emptyRoutes = runs.flatMap((item) => item.emptyRoutes);
const subsumedStates = runs.reduce((total, item) => total + item.subsumedStates, 0);
const deduplicatedActions = runs.reduce((total, item) => total + item.deduplicatedActions, 0);
const skipped = results.filter((item) => item.result?.skipped);
const declared = sourceActionInventory(ctx);
const target = path.join(ctx.dirs.raw, 'ui-actions.json');
const summary = { operated: results.filter((item) => item.interaction?.ok && !item.interaction.skipped).length,
stateChanges: results.filter((item) => item.assessment?.stateChanged).length,
effectFailures: results.filter((item) => item.assessment && !item.assessment.ok).length,
subsumedStates, deduplicatedActions };
writeJson(target, { results, declared, cappedRoutes, emptyRoutes, summary, observed: runs.map((item) => item.observed) });
const traces = runs.map((item) => item.trace);
addExecutionFinding(ctx, results, skipped, target, traces);
addEffectFinding(ctx, results, target, traces);
addCoverageFinding(ctx, results, cappedRoutes, emptyRoutes, target);
addInventoryFinding(ctx, { results, declared, skipped, cappedRoutes, emptyRoutes, subsumedStates, deduplicatedActions }, target);
}
verification/src/checks/ui-pages.mjs
+90
View File
@@ -0,0 +1,90 @@
import path from 'node:path';
import { finding } from '../core/finding.mjs';
import { writeJson } from '../core/files.mjs';
import { launchBrowser, newPage, stabilize } from '../browser/session.mjs';
import { discoverPages } from '../browser/routes.mjs';
import { pageMetrics } from '../browser/metrics.mjs';
import { scanAxe } from '../browser/axe.mjs';
function safeName(value) {
return value.replace(/^\//, '').replace(/[^a-zA-Z0-9]+/g, '-') || 'home';
}
async function inspectPage(browser, ctx, viewport, route) {
const session = await newPage(browser, ctx, viewport);
let response = null;
let axe = { violations: [], incomplete: [] };
try {
response = await session.page.goto(`${ctx.config.localUrl}${route}`, { waitUntil: 'domcontentloaded', timeout: ctx.config.navigationTimeoutMs });
await session.page.waitForLoadState('load', { timeout: ctx.config.navigationTimeoutMs }).catch(() => {});
await stabilize(session.page);
axe = await scanAxe(session.page, ctx);
const screenshot = path.join(ctx.dirs.screenshots, `${viewport.name}-${safeName(route)}.png`);
await session.page.screenshot({ path: screenshot, fullPage: true });
return { viewport, route, status: response?.status() || 0, metrics: await pageMetrics(session.page), axe, observed: session.observed, screenshot };
} catch (error) {
return { viewport, route, status: response?.status() || 0,
error: error instanceof Error ? error.stack || error.message : String(error), axe, observed: session.observed };
} finally {
await session.context.close();
}
}
export async function checkUiPages(ctx) {
if (!ctx.state.appReady) return;
const browser = ctx.state.browser || await launchBrowser(ctx);
ctx.state.browser = browser;
const routes = discoverPages(ctx);
ctx.state.pageRoutes = routes;
const results = [];
for (const viewport of ctx.config.viewports) {
for (const route of routes) {
results.push(await inspectPage(browser, ctx, viewport, route));
}
}
const target = path.join(ctx.dirs.raw, 'ui-pages.json');
writeJson(target, results);
const renderFailures = results.filter((item) => item.error || item.status >= 400 || item.status === 0);
const runtimeErrors = results.filter((item) => item.metrics?.errors?.length || item.metrics?.rejections?.length
|| item.observed.pageErrors.length || item.observed.consoleErrors.length || item.observed.failedRequests.length || item.observed.httpErrors.length);
const overflow = results.filter((item) => item.metrics && item.metrics.scrollWidth > item.metrics.clientWidth + 1);
const severeA11y = results.flatMap((item) => item.axe.violations.filter((violation) => ['critical', 'serious'].includes(violation.impact)).map((violation) => ({ route: item.route, viewport: item.viewport.name, ...violation })));
const advisoryA11y = results.flatMap((item) => [
...item.axe.violations.filter((violation) => !['critical', 'serious'].includes(violation.impact)),
...item.axe.incomplete,
].map((violation) => ({ route: item.route, viewport: item.viewport.name, ...violation })));
const vitalFailures = results.flatMap((item) => {
const metrics = item.metrics;
if (!metrics) return [{ route: item.route, viewport: item.viewport.name, reason: 'metrics missing' }];
const longTaskTotal = metrics.longTasks.reduce((sum, value) => sum + value, 0);
const reasons = [
...(!Number.isFinite(metrics.lcp) || metrics.lcp <= 0 || metrics.lcp > ctx.config.maxLcpMs ? [`LCP ${metrics.lcp}ms`] : []),
...(!Number.isFinite(metrics.cls) || metrics.cls > ctx.config.maxCls ? [`CLS ${metrics.cls}`] : []),
...(longTaskTotal > ctx.config.maxLongTaskMs ? [`long tasks ${longTaskTotal}ms`] : []),
];
return reasons.length ? [{ route: item.route, viewport: item.viewport.name, reasons }] : [];
});
aggregate(ctx, { id: 'ui.route-render', title: 'Every page renders in every target viewport', failures: renderFailures, total: results.length, severity: 'critical', evidence: target });
aggregate(ctx, { id: 'ui.runtime-errors', title: 'Pages emit no uncaught or console errors', failures: runtimeErrors, total: results.length, severity: 'critical', evidence: target });
aggregate(ctx, { id: 'ui.horizontal-overflow', title: 'Pages do not overflow target viewports horizontally', failures: overflow, total: results.length, severity: 'high', evidence: target });
aggregate(ctx, { id: 'ui.accessibility', title: 'Pages have no serious or critical automated accessibility violations', failures: severeA11y, total: results.length, severity: 'high', evidence: target });
aggregate(ctx, { id: 'ui.web-vitals', title: 'Every page meets LCP, CLS, and load long-task budgets', failures: vitalFailures, total: results.length, severity: 'high', evidence: target });
finding(ctx, {
id: 'ui.accessibility-advisory', category: 'ui', title: 'Moderate, minor, and incomplete accessibility checks remain visible',
status: advisoryA11y.length ? 'WARN' : 'PASS', severity: 'medium', expected: '0 lower-impact or incomplete axe results',
actual: advisoryA11y.length ? JSON.stringify(advisoryA11y.slice(0, 50)) : '0',
reason: advisoryA11y.length ? 'Lower-impact or manual-review accessibility results still require inspection.' : 'Axe reported no additional advisory results.',
evidence: [target], remediation: 'Resolve confirmed violations and manually evaluate every incomplete rule.',
});
ctx.state.uiPages = results;
}
function aggregate(ctx, options) {
const { id, title, failures, total, severity, evidence } = options;
finding(ctx, {
id, category: 'ui', title, status: failures.length ? 'FAIL' : 'PASS', severity, expected: `0 failures across ${total} page/viewport cases`,
actual: failures.length ? JSON.stringify(failures.slice(0, 30)) : `${total} cases passed`,
reason: failures.length ? 'At least one enumerated page state violated the declared UI contract.' : 'Every enumerated page state met the contract.',
evidence: [evidence], remediation: 'Open the named screenshot and evidence record, reproduce the exact route/viewport, and repair the underlying component.',
});
}
verification/src/checks/verifier-quality.mjs
+80
View File
@@ -0,0 +1,80 @@
import fs from 'node:fs';
import path from 'node:path';
import { parse } from '@typescript-eslint/typescript-estree';
import { findCycles } from '../core/cycles.mjs';
import { finding } from '../core/finding.mjs';
import { relative, walk, writeJson } from '../core/files.mjs';
import { collectFunctions } from './ast-walk.mjs';
const importPattern = /(?:import|export)\s+(?:[^'";]+?\s+from\s+)?['"]([^'"]+)['"]/g;
function sourceFiles(ctx) {
return walk(ctx.config.verifyDir, (file) => ['.mjs', '.ts'].includes(path.extname(file))
&& !file.includes('/node_modules/') && !file.includes('/artifacts/') && !file.includes('/tmp/'));
}
function metricsFor(ctx, files) {
const parsed = [];
const errors = [];
for (const file of files) {
try {
const ast = parse(fs.readFileSync(file, 'utf8'), { loc: true, jsx: false, errorOnUnknownASTType: false });
parsed.push({ file: relative(ctx.config.root, file), functions: collectFunctions(ast) });
} catch (error) {
errors.push({ file: relative(ctx.config.root, file), error: error instanceof Error ? error.message : String(error) });
}
}
const offenders = parsed.flatMap((item) => item.functions.map((fn) => ({ file: item.file, ...fn })))
.filter((fn) => fn.lines > 80 || fn.complexity > 15 || fn.maxNesting > 4 || fn.params > 5)
.sort((a, b) => (b.complexity + b.lines / 10) - (a.complexity + a.lines / 10));
return { parsed, errors, offenders };
}
function resolveImport(file, specifier) {
if (!specifier.startsWith('.')) return null;
const base = path.resolve(path.dirname(file), specifier);
const candidates = [base, `${base}.mjs`, `${base}.js`, path.join(base, 'index.mjs')];
return candidates.find((item) => fs.existsSync(item) && fs.statSync(item).isFile()) || null;
}
function importGraph(ctx, files) {
const known = new Set(files);
const graph = new Map(files.map((file) => [file, []]));
for (const file of files) {
const source = fs.readFileSync(file, 'utf8');
for (const match of source.matchAll(importPattern)) {
const resolved = resolveImport(file, match[1]);
if (resolved && known.has(resolved)) graph.get(file).push(resolved);
}
}
return Object.fromEntries([...graph].map(([file, imports]) => [relative(ctx.config.root, file), imports.map((item) => relative(ctx.config.root, item))]));
}
export async function checkVerifierQuality(ctx) {
const files = sourceFiles(ctx);
const metrics = metricsFor(ctx, files);
const graph = importGraph(ctx, files.filter((file) => file.endsWith('.mjs')));
const cycles = findCycles(graph);
const target = path.join(ctx.dirs.metrics, 'verifier-quality.json');
writeJson(target, { parseErrors: metrics.errors, offenders: metrics.offenders, graph, cycles });
finding(ctx, {
id: 'harness.ast-parse', category: 'harness', title: 'Every verification source file is structurally analyzable',
status: metrics.errors.length ? 'FAIL' : 'PASS', severity: 'critical', expected: '0 parser errors',
actual: metrics.errors.length ? JSON.stringify(metrics.errors) : `${files.length} files parsed`,
reason: metrics.errors.length ? 'Self-analysis is incomplete when verifier source cannot be parsed.' : 'The verifier can structurally inspect all of its own authored code.',
evidence: [target], remediation: 'Repair every verifier parse failure.',
});
finding(ctx, {
id: 'harness.structural-quality', category: 'harness', title: 'Verification functions satisfy their own complexity limits',
status: metrics.offenders.length ? 'FAIL' : 'PASS', severity: 'critical', expected: 'lines <=80, complexity <=15, nesting <=4, params <=5',
actual: metrics.offenders.length ? JSON.stringify(metrics.offenders) : 'No threshold breaches',
reason: metrics.offenders.length ? 'A verifier cannot credibly enforce standards it violates itself.' : 'Verification implementation meets the same function-level quality boundary.',
evidence: [target], remediation: 'Split each listed verifier function before trusting or publishing it.',
});
finding(ctx, {
id: 'harness.import-cycles', category: 'harness', title: 'Verification module graph has no dependency cycle',
status: cycles.length ? 'FAIL' : 'PASS', severity: 'critical', expected: '0 cycles', actual: cycles.length ? cycles.join('\n') : '0',
reason: cycles.length ? 'Order-dependent verifier initialization can corrupt findings.' : 'The verifier module graph is acyclic.',
evidence: [target], remediation: 'Extract shared helpers or invert dependencies to remove every cycle.',
});
}
verification/src/checks/video-case.mjs
+78
View File
@@ -0,0 +1,78 @@
import path from 'node:path';
import { hasRuntimeProblems } from '../browser/observed.mjs';
import { newPage, stabilize } from '../browser/session.mjs';
function playerRoute(episode, title = 'verification') {
return `/player?id=fixture-video-1&source=fixture&title=${encodeURIComponent(title)}&episode=${episode}`;
}
export async function playVideoCase(ctx, episode, viewport) {
const session = await newPage(ctx.state.browser, ctx, viewport);
try {
await session.page.goto(`${ctx.config.localUrl}${playerRoute(episode)}`, {
waitUntil: 'domcontentloaded', timeout: ctx.config.navigationTimeoutMs,
});
await stabilize(session.page);
const video = session.page.locator('video');
await video.waitFor({ state: 'attached', timeout: 20_000 });
const before = await video.evaluate(async (element) => {
element.muted = true; await element.play();
return { currentTime: element.currentTime, readyState: element.readyState, networkState: element.networkState };
});
await session.page.waitForTimeout(2600);
const after = await video.evaluate((element) => {
const quality = element.getVideoPlaybackQuality?.();
return { currentTime: element.currentTime, duration: element.duration, paused: element.paused,
readyState: element.readyState, width: element.videoWidth, height: element.videoHeight,
totalFrames: quality?.totalVideoFrames, droppedFrames: quality?.droppedVideoFrames,
error: element.error?.message || null };
});
const screenshot = path.join(ctx.dirs.screenshots, `video-${viewport.name}-episode-${episode}.png`);
await session.page.screenshot({ path: screenshot, fullPage: true });
return { viewport: viewport.name, episode, before, after, observed: session.observed, screenshot };
} catch (error) {
return { viewport: viewport.name, episode, error: error instanceof Error ? error.stack || error.message : String(error), observed: session.observed };
} finally { await session.context.close(); }
}
export async function stallVideoCase(ctx, viewport) {
const session = await newPage(ctx.state.browser, ctx, viewport);
try {
await session.page.goto(`${ctx.config.localUrl}${playerRoute(0, 'stall')}`, { waitUntil: 'domcontentloaded' });
await stabilize(session.page);
const video = session.page.locator('video');
await video.waitFor({ state: 'attached', timeout: 20_000 });
await video.evaluate(async (element) => { element.muted = true; await element.play(); });
await session.page.waitForTimeout(600);
await video.evaluate((element) => {
const descriptor = Object.getOwnPropertyDescriptor(HTMLMediaElement.prototype, 'currentTime');
const frozen = descriptor.get.call(element);
window.__kvFrozen = true;
Object.defineProperty(element, 'currentTime', { configurable: true,
get() { return window.__kvFrozen ? frozen : descriptor.get.call(this); },
set(value) { descriptor.set.call(this, value); } });
});
await session.page.waitForTimeout(600);
const detected = await session.page.locator('.loading-overlay-glass').isVisible().catch(() => false);
await video.evaluate((element) => { window.__kvFrozen = false; delete element.currentTime; });
await session.page.waitForTimeout(500);
const recovered = !(await session.page.locator('.loading-overlay-glass').isVisible().catch(() => false));
return { viewport: viewport.name, detected, recovered, observed: session.observed };
} catch (error) {
return { viewport: viewport.name, error: error instanceof Error ? error.stack || error.message : String(error), observed: session.observed };
} finally { await session.context.close(); }
}
export function videoProblems(ctx, result) {
const advance = result.after ? result.after.currentTime - result.before.currentTime : 0;
const dropped = result.after?.droppedFrames || 0;
const total = result.after?.totalFrames || 0;
const problems = [];
if (result.error) problems.push(result.error);
if (advance < ctx.config.minVideoAdvanceSeconds) problems.push(`advanced ${advance}s`);
if (result.after && (result.after.width !== 640 || result.after.height !== 360)) problems.push(`${result.after.width}x${result.after.height}`);
if (total && dropped / total > .05) problems.push(`dropped ${dropped}/${total}`);
if (result.after?.error) problems.push(result.after.error);
if (hasRuntimeProblems(result.observed)) problems.push('runtime/network errors');
return { advance, problems };
}
verification/src/checks/video.mjs
+51
View File
@@ -0,0 +1,51 @@
import path from 'node:path';
import { finding } from '../core/finding.mjs';
import { writeJson } from '../core/files.mjs';
import { playVideoCase, stallVideoCase, videoProblems } from './video-case.mjs';
// GH-ISSUE: 8,24,40,46,81,91,118,130; GH-PR: 138
function addPlaybackFinding(ctx, name, results, target) {
const evaluated = results.map((result) => ({ result, ...videoProblems(ctx, result) }));
const failures = evaluated.filter((item) => item.problems.length);
finding(ctx, {
id: `video.${name}`, category: 'video', title: `${name.toUpperCase()} plays correctly in every target viewport`,
status: failures.length ? 'FAIL' : 'PASS', severity: 'critical',
expected: `All viewports advance >=${ctx.config.minVideoAdvanceSeconds}s, decode 640x360, drop <=5%, and emit no runtime/network errors`,
actual: failures.length ? JSON.stringify(failures.map((item) => ({ viewport: item.result.viewport, advance: item.advance, problems: item.problems })))
: `${results.length} viewport cases passed`,
reason: failures.length ? 'At least one real browser playback case stalled, decoded incorrectly, dropped frames, or emitted errors.'
: 'Every viewport met timing, dimensions, frame-loss, and runtime contracts.',
evidence: [target, ...results.map((item) => item.screenshot).filter(Boolean)],
remediation: 'Inspect the exact viewport, player events, HLS configuration, codecs, proxy mode, and browser evidence.',
});
}
function addStallFinding(ctx, results, target) {
const failures = results.filter((item) => item.error || !item.detected || !item.recovered
|| item.observed?.consoleErrors.length || item.observed?.pageErrors.length);
finding(ctx, {
id: 'video.stall-detector', category: 'video', title: '200ms stall detection and recovery work in every viewport',
status: failures.length ? 'FAIL' : 'PASS', severity: 'high', expected: 'Overlay appears during a controlled freeze and clears after recovery in all viewports',
actual: failures.length ? JSON.stringify(failures) : `${results.length} viewport cases passed`,
reason: failures.length ? 'At least one viewport failed controlled stall detection, recovery, or runtime safety.' : 'Every viewport responded correctly to a controlled playback freeze.',
evidence: [target], remediation: 'Repair currentTime polling, loading-state ownership, responsive overlay rendering, or recovery clearing logic.',
});
}
export async function checkVideo(ctx) {
if (!ctx.state.browser || !ctx.state.mediaOk) return;
const mp4 = [];
const hls = [];
const stall = [];
for (const viewport of ctx.config.viewports) {
mp4.push(await playVideoCase(ctx, 0, viewport));
hls.push(await playVideoCase(ctx, 1, viewport));
stall.push(await stallVideoCase(ctx, viewport));
}
const target = path.join(ctx.dirs.raw, 'video-playback.json');
writeJson(target, { mp4, hls, stall });
addPlaybackFinding(ctx, 'mp4', mp4, target);
addPlaybackFinding(ctx, 'hls', hls, target);
addStallFinding(ctx, stall, target);
}
verification/src/checks/visual-case.mjs
+119
View File
@@ -0,0 +1,119 @@
import fs from 'node:fs';
import path from 'node:path';
import pixelmatch from 'pixelmatch';
import { PNG } from 'pngjs';
import { newPage, stabilize } from '../browser/session.mjs';
import { semanticDifference, semanticSnapshot } from '../browser/semantic.mjs';
function errorText(error) {
return error instanceof Error ? error.message : String(error);
}
function safe(value) {
return value.replace(/^\//, '').replace(/[^a-zA-Z0-9]+/g, '-') || 'home';
}
function padded(image, width, height) {
const output = new PNG({ width, height, fill: true });
PNG.bitblt(image, output, 0, 0, image.width, image.height, 0, 0);
return output;
}
function compare(leftFile, rightFile, diffFile) {
const leftRaw = PNG.sync.read(fs.readFileSync(leftFile));
const rightRaw = PNG.sync.read(fs.readFileSync(rightFile));
const width = Math.max(leftRaw.width, rightRaw.width);
const height = Math.max(leftRaw.height, rightRaw.height);
const left = padded(leftRaw, width, height);
const right = padded(rightRaw, width, height);
const diff = new PNG({ width, height });
const pixels = pixelmatch(left.data, right.data, diff.data, width, height, { threshold: 0.12, includeAA: false });
fs.writeFileSync(diffFile, PNG.sync.write(diff));
return { pixels, total: width * height, ratio: pixels / (width * height),
dimensions: { left: [leftRaw.width, leftRaw.height], right: [rightRaw.width, rightRaw.height] } };
}
async function waitForRenderedPage(page, timeoutMs) {
await page.waitForFunction(() => {
const body = document.body;
if (!body || (body.innerText || '').trim().length < 8) return false;
return [...body.querySelectorAll('*')].some((element) => {
if ((element.innerText || '').trim().length < 8) return false;
const box = element.getBoundingClientRect();
const style = getComputedStyle(element);
return box.width > 0 && box.height > 0 && style.visibility !== 'hidden' && style.display !== 'none';
});
}, undefined, { timeout: timeoutMs });
}
async function renderState(page) {
return page.evaluate(() => ({
url: location.href,
title: document.title,
readyState: document.readyState,
textLength: (document.body?.innerText || '').trim().length,
textSample: (document.body?.innerText || '').trim().replace(/\s+/g, ' ').slice(0, 240),
htmlLength: document.documentElement?.outerHTML.length || 0,
elementCount: document.body?.querySelectorAll('*').length || 0,
}));
}
function issue(meta, phase, error) {
meta.issues.push({ phase, message: errorText(error) });
}
async function collect(page, meta, file) {
try { await stabilize(page); } catch (error) { issue(meta, 'stabilize', error); }
try { await page.locator('video,audio').evaluateAll((items) => items.forEach((item) => item.pause())); }
catch (error) { issue(meta, 'pause-media', error); }
try {
await page.screenshot({ path: file, fullPage: true, mask: [page.locator('video,canvas')], maskColor: '#000000' });
meta.screenshot = file;
} catch (error) { issue(meta, 'screenshot', error); }
try { meta.semantic = await semanticSnapshot(page); } catch (error) { issue(meta, 'semantic-snapshot', error); }
try { meta.render = await renderState(page); } catch (error) { issue(meta, 'render-state', error); }
meta.finalUrl = page.url();
}
async function capture(options) {
const { browser, ctx, base, route, viewport, file } = options;
const session = await newPage(browser, ctx, viewport);
const meta = { status: 0, statusText: '', contentType: '', finalUrl: '', render: null,
semantic: null, screenshot: null, issues: [], observed: session.observed };
try {
try {
const response = await session.page.goto(`${base}${route}`, {
waitUntil: 'domcontentloaded', timeout: ctx.config.navigationTimeoutMs,
});
meta.status = response?.status() || 0;
meta.statusText = response?.statusText() || '';
meta.contentType = await response?.headerValue('content-type') || '';
} catch (error) { issue(meta, 'navigation', error); }
if (!meta.issues.some((item) => item.phase === 'navigation')) {
try { await waitForRenderedPage(session.page, ctx.config.navigationTimeoutMs); }
catch (error) { issue(meta, 'render-wait', error); }
}
await collect(session.page, meta, file);
return meta;
} finally { await session.context.close(); }
}
export async function compareVisualCase(ctx, route, viewport) {
const stem = `${viewport.name}-${safe(route)}`;
const local = path.join(ctx.dirs.screenshots, `visual-local-${stem}.png`);
const remote = path.join(ctx.dirs.screenshots, `visual-remote-${stem}.png`);
const diff = path.join(ctx.dirs.diffs, `visual-diff-${stem}.png`);
const common = { browser: ctx.state.browser, ctx, route, viewport };
const localMeta = await capture({ ...common, base: ctx.config.localUrl, file: local });
const remoteMeta = await capture({ ...common, base: ctx.config.referenceUrl, file: remote });
let comparison = null;
let comparisonError = null;
if (localMeta.screenshot && remoteMeta.screenshot) {
try {
comparison = compare(local, remote, diff);
comparison.semantic = semanticDifference(localMeta.semantic, remoteMeta.semantic);
} catch (error) { comparisonError = errorText(error); }
}
return { route, viewport: viewport.name, localMeta, remoteMeta, comparison, comparisonError,
local: localMeta.screenshot, remote: remoteMeta.screenshot, diff: comparison ? diff : null };
}
verification/src/checks/visual.mjs
+95
View File
@@ -0,0 +1,95 @@
import fs from 'node:fs';
import path from 'node:path';
import { finding } from '../core/finding.mjs';
import { writeJson } from '../core/files.mjs';
import { compareVisualCase } from './visual-case.mjs';
// GH-PR: 17,198
function observedCounts(meta) {
const value = meta?.observed || {};
return {
requests: value.requestCount || 0,
responses: value.responseCount || 0,
consoleErrors: value.consoleErrors?.length || 0,
pageErrors: value.pageErrors?.length || 0,
failedRequests: value.failedRequests?.length || 0,
httpErrors: value.httpErrors?.length || 0,
};
}
function captureSummary(meta) {
return meta ? { status: meta.status, statusText: meta.statusText, contentType: meta.contentType,
finalUrl: meta.finalUrl, render: meta.render, issues: meta.issues, observed: observedCounts(meta) } : null;
}
function runtimeProblems(label, meta) {
const observed = meta?.observed;
if (!observed) return [`${label} browser observation missing`];
return [
...observed.consoleErrors.map((value) => `${label} console error: ${value}`),
...observed.pageErrors.map((value) => `${label} page error: ${value}`),
...observed.failedRequests.map((value) => `${label} request failed: ${value.url} (${value.error || 'unknown error'})`),
...observed.httpErrors.map((value) => `${label} HTTP ${value.status}: ${value.method} ${value.url}`),
];
}
function captureProblems(label, meta) {
if (!meta) return [`${label} capture missing`];
const problems = meta.issues.map((item) => `${label} ${item.phase}: ${item.message}`);
if (meta.status === 0 || meta.status >= 400) problems.push(`${label} document HTTP ${meta.status}`);
if (!meta.screenshot) problems.push(`${label} screenshot missing`);
if (!meta.render) problems.push(`${label} render state missing`);
else if (meta.render.textLength < 8) problems.push(`${label} visible text length ${meta.render.textLength}`);
return [...problems, ...runtimeProblems(label, meta)];
}
export function visualProblems(item, limit) {
const problems = [...captureProblems('local', item.localMeta), ...captureProblems('remote', item.remoteMeta)];
if (item.comparisonError) problems.push(`pixel comparison: ${item.comparisonError}`);
if (!item.comparison) problems.push('pixel comparison unavailable');
else if (item.comparison.ratio > limit) problems.push(`pixel ratio ${item.comparison.ratio}`);
if (!item.localMeta?.semantic || !item.remoteMeta?.semantic) problems.push('visible semantic snapshot unavailable');
else if (item.localMeta.semantic.hash !== item.remoteMeta.semantic.hash) problems.push('visible semantic DOM differs');
return problems;
}
function visualSummary(results) {
return results.map((item) => ({ route: item.route, viewport: item.viewport, ratio: item.comparison?.ratio,
semanticMatch: Boolean(item.localMeta?.semantic && item.remoteMeta?.semantic
&& item.localMeta.semantic.hash === item.remoteMeta.semantic.hash),
missingSemantic: item.comparison?.semantic?.missing.length,
unexpectedSemantic: item.comparison?.semantic?.unexpected.length,
local: captureSummary(item.localMeta), remote: captureSummary(item.remoteMeta),
comparisonError: item.comparisonError }));
}
export async function checkVisual(ctx) {
if (ctx.config.offline || !ctx.state.browser || !ctx.state.pageRoutes) return finding(ctx, {
id: 'visual.deployment-diff', category: 'visual', title: 'Local and Cloudflare UI visual comparison', status: 'SKIP', severity: 'high',
expected: 'Online reference and browser available', actual: ctx.config.offline ? '--offline' : 'browser unavailable',
reason: 'Pixel comparison requires both surfaces.', remediation: 'Rerun online after local startup.',
});
const routes = ctx.config.quick ? ['/'] : ctx.state.pageRoutes;
const viewports = ctx.config.quick ? [ctx.config.viewports[0]] : ctx.config.viewports;
const results = [];
for (const viewport of viewports) {
for (const route of routes) results.push(await compareVisualCase(ctx, route, viewport));
}
const target = path.join(ctx.dirs.raw, 'visual-comparison.json');
writeJson(target, results);
const unexpected = results.map((item) => ({ ...item, problems: visualProblems(item, ctx.config.visualDiffRatio) }))
.filter((item) => item.problems.length);
const images = results.flatMap((item) => [item.local, item.remote, item.diff]).filter((file) => file && fs.existsSync(file));
finding(ctx, {
id: 'visual.deployment-diff', category: 'visual', title: 'Local and Cloudflare UI stay within visual deviation threshold',
status: unexpected.length ? 'FAIL' : 'PASS', severity: 'high',
expected: `All viewports: HTTP 200, no runtime errors, exact visible semantics, pixel difference <= ${ctx.config.visualDiffRatio * 100}%`,
actual: JSON.stringify({ summary: visualSummary(results),
failures: unexpected.map((item) => ({ route: item.route, viewport: item.viewport, problems: item.problems })) }),
reason: unexpected.length ? 'At least one deployment surface differs in pixels, visible semantics, HTTP state, or runtime behavior.'
: 'Every route and viewport matches the public deployment under strict visual and semantic rules.',
evidence: [target, ...images],
remediation: 'Inspect each capture issue, browser error, screenshot, pixel diff, and semantic delta, then reconcile the deployed build.',
});
}
verification/src/config.mjs
+55
View File
@@ -0,0 +1,55 @@
import path from 'node:path';
const viewports = [
{ name: 'mobile', width: 390, height: 844 },
{ name: 'tablet', width: 820, height: 1180 },
{ name: 'desktop', width: 1440, height: 1000 },
{ name: 'tv', width: 1920, height: 1080 },
];
function valueAfter(args, flag) {
const index = args.indexOf(flag);
return index >= 0 ? args[index + 1] : undefined;
}
function positive(value, fallback) {
const parsed = Number(value);
return Number.isInteger(parsed) && parsed > 0 ? parsed : fallback;
}
export function getConfig(argv) {
const args = argv.slice(2);
const root = path.resolve(valueAfter(args, '--root') || process.cwd());
const verifyDir = path.join(root, 'verification');
const quick = args.includes('--quick');
const offline = args.includes('--offline');
return {
args,
root,
verifyDir,
quick,
offline,
auditGithub: args.includes('--audit-github'),
candidate: args.includes('--candidate'),
keepServer: args.includes('--keep-server'),
referenceUrl: valueAfter(args, '--reference-url') || 'https://kvideo.pages.dev',
localUrl: valueAfter(args, '--base-url') || 'http://127.0.0.1:34173',
fixtureUrl: 'http://127.0.0.1:34174',
containerUrl: 'http://127.0.0.1:34175',
localPort: 34173,
fixturePort: 34174,
containerPort: 34175,
maxSourceLines: 150,
maxActionStates: quick ? 30 : positive(valueAfter(args, '--max-actions'), 5000),
maxActionDepth: quick ? 1 : positive(valueAfter(args, '--max-action-depth'), 8),
commandTimeoutMs: quick ? 180_000 : 900_000,
navigationTimeoutMs: 30_000,
visualDiffRatio: 0.02,
maxLcpMs: 2500,
maxCls: 0.1,
maxLongTaskMs: 500,
coveragePercent: 100,
minVideoAdvanceSeconds: 1.2,
viewports: quick ? [viewports[2]] : viewports,
};
}
verification/src/core/command.mjs
+73
View File
@@ -0,0 +1,73 @@
import fs from 'node:fs';
import { spawn } from 'node:child_process';
import { StringDecoder } from 'node:string_decoder';
import { rawPath } from './log.mjs';
import { redactText } from './redact.mjs';
function terminate(child, signal) {
if (!child.pid || child.killed) return;
try {
process.kill(-child.pid, signal);
} catch {
try { child.kill(signal); } catch { /* process already ended */ }
}
}
export function runCommand(ctx, name, command, args = [], options = {}) {
const started = Date.now();
const outputPath = rawPath(ctx, `${name}.log`);
const stream = fs.createWriteStream(outputPath, { flags: 'w' });
const env = { ...process.env, ...(options.env || {}) };
return new Promise((resolve) => {
const child = spawn(command, args, {
cwd: options.cwd || ctx.config.root,
env,
detached: process.platform !== 'win32',
stdio: ['ignore', 'pipe', 'pipe'],
});
let tail = '';
let timedOut = false;
let settled = false;
let flushed = false;
const stdoutDecoder = new StringDecoder('utf8');
const stderrDecoder = new StringDecoder('utf8');
const append = (text) => {
if (!text) return;
const safe = redactText(text);
stream.write(safe);
tail = `${tail}${safe}`.slice(-12_000);
if (options.live) process.stdout.write(safe);
};
const flush = () => {
if (flushed) return;
flushed = true;
append(stdoutDecoder.end());
append(stderrDecoder.end());
};
const finish = (result) => {
if (settled) return;
settled = true;
clearTimeout(timeout);
flush();
result.tail = tail;
stream.end(() => resolve(result));
};
child.stdout.on('data', (chunk) => append(stdoutDecoder.write(chunk)));
child.stderr.on('data', (chunk) => append(stderrDecoder.write(chunk)));
const timeout = setTimeout(() => {
timedOut = true;
terminate(child, 'SIGTERM');
setTimeout(() => terminate(child, 'SIGKILL'), 3000).unref();
}, options.timeoutMs || ctx.config.commandTimeoutMs);
child.on('error', (error) => {
finish({ code: 127, error: error.message, tail, outputPath, timedOut, durationMs: Date.now() - started });
});
child.on('close', (code, signal) => {
finish({ code: code ?? 1, signal, tail, outputPath, timedOut, durationMs: Date.now() - started });
});
});
}
export async function runNpm(ctx, name, args, options = {}) {
return runCommand(ctx, name, 'npm', args, options);
}
verification/src/core/context.mjs
+32
View File
@@ -0,0 +1,32 @@
import fs from 'node:fs';
import path from 'node:path';
import { redact } from './redact.mjs';
function safeRunId() {
return new Date().toISOString().replace(/[:.]/g, '-');
}
export function createContext(config) {
const runId = safeRunId();
const artifacts = path.join(config.verifyDir, 'artifacts', runId);
const dirs = Object.fromEntries(
['raw', 'screenshots', 'diffs', 'traces', 'metrics', 'media'].map((name) => {
const target = path.join(artifacts, name);
fs.mkdirSync(target, { recursive: true });
return [name, target];
}),
);
const ctx = {
config,
runId,
artifacts,
dirs,
startedAt: new Date().toISOString(),
findings: [],
events: [],
services: [],
state: {},
};
fs.writeFileSync(path.join(artifacts, 'config.json'), JSON.stringify(redact(config), null, 2));
return ctx;
}
verification/src/core/cycles.mjs
+21
View File
@@ -0,0 +1,21 @@
function children(graph, node) {
return graph instanceof Map ? graph.get(node) || [] : graph[node] || [];
}
export function findCycles(graph) {
const cycles = new Set();
const active = [];
const done = new Set();
const visit = (node) => {
const index = active.indexOf(node);
if (index >= 0) { cycles.add([...active.slice(index), node].join(' -> ')); return; }
if (done.has(node)) return;
active.push(node);
for (const child of children(graph, node)) visit(child);
active.pop();
done.add(node);
};
const nodes = graph instanceof Map ? graph.keys() : Object.keys(graph);
for (const node of nodes) visit(node);
return [...cycles].sort();
}
verification/src/core/files.mjs
+37
View File
@@ -0,0 +1,37 @@
import fs from 'node:fs';
import path from 'node:path';
import { redact } from './redact.mjs';
const ignored = new Set(['.git', '.gradle', '.next', '.vercel', '.wrangler', 'artifacts', 'build', 'cache', 'coverage', 'dist', 'node_modules', 'out']);
export function walk(root, predicate = () => true) {
const output = [];
const visit = (dir) => {
for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
if (ignored.has(entry.name)) continue;
const full = path.join(dir, entry.name);
if (entry.isDirectory()) visit(full);
else if (predicate(full)) output.push(full);
}
};
visit(root);
return output;
}
export function relative(root, file) {
return path.relative(root, file).split(path.sep).join('/');
}
export function lineCount(file) {
const text = fs.readFileSync(file, 'utf8');
return text.length === 0 ? 0 : text.split(/\r?\n/).length;
}
export function writeJson(file, value) {
fs.mkdirSync(path.dirname(file), { recursive: true });
fs.writeFileSync(file, JSON.stringify(redact(value), null, 2));
}
export function readJson(file) {
return JSON.parse(fs.readFileSync(file, 'utf8'));
}
verification/src/core/finding.mjs
+35
View File
@@ -0,0 +1,35 @@
import { log } from './log.mjs';
import { redact } from './redact.mjs';
const validStatus = new Set(['PASS', 'FAIL', 'WARN', 'SKIP', 'INFO']);
const validSeverity = new Set(['critical', 'high', 'medium', 'low', 'info']);
export function finding(ctx, input) {
const item = redact({
id: input.id,
category: input.category || 'general',
title: input.title,
status: validStatus.has(input.status) ? input.status : 'INFO',
severity: validSeverity.has(input.severity) ? input.severity : 'info',
expected: input.expected ?? null,
actual: input.actual ?? null,
reason: input.reason || '',
impact: input.impact || '',
remediation: input.remediation || '',
evidence: input.evidence || [],
durationMs: Math.round(input.durationMs || 0),
at: new Date().toISOString(),
});
ctx.findings.push(item);
log(ctx, item.status === 'FAIL' ? 'error' : 'info', item.id, item.title, {
status: item.status,
severity: item.severity,
actual: item.actual,
reason: item.reason,
});
return item;
}
export function hasFailures(ctx) {
return ctx.findings.some((item) => item.status === 'FAIL');
}
verification/src/core/http.mjs
+42
View File
@@ -0,0 +1,42 @@
import crypto from 'node:crypto';
function isTextual(contentType = '') {
return /^(text\/)|json|xml|javascript|mpegurl|event-stream|svg/i.test(contentType);
}
export async function request(url, options = {}) {
const started = performance.now();
const { timeoutMs = 20_000, ...fetchOptions } = options;
try {
const response = await fetch(url, {
redirect: fetchOptions.redirect || 'follow',
...fetchOptions,
signal: AbortSignal.timeout(timeoutMs),
});
const buffer = Buffer.from(await response.arrayBuffer());
const headers = Object.fromEntries(response.headers);
const preview = buffer.subarray(0, 100_000);
return {
ok: true,
status: response.status,
durationMs: Math.round(performance.now() - started),
headers,
bytes: buffer.length,
sha256: crypto.createHash('sha256').update(buffer).digest('hex'),
body: isTextual(headers['content-type']) ? preview.toString('utf8') : '',
bodyTruncated: buffer.length > preview.length,
};
} catch (error) {
return {
ok: false,
status: 0,
durationMs: Math.round(performance.now() - started),
error: error instanceof Error ? error.message : String(error),
headers: {}, bytes: 0, sha256: null, body: '', bodyTruncated: false,
};
}
}
export function jsonBody(result) {
try { return JSON.parse(result.body); } catch { return null; }
}
verification/src/core/log.mjs
+25
View File
@@ -0,0 +1,25 @@
import fs from 'node:fs';
import path from 'node:path';
import { redact, redactText } from './redact.mjs';
export function log(ctx, level, event, message, data = {}) {
const entry = {
at: new Date().toISOString(),
level,
event,
message: redactText(message),
data: redact(data),
};
ctx.events.push(entry);
fs.appendFileSync(path.join(ctx.artifacts, 'events.ndjson'), `${JSON.stringify(entry)}\n`);
const suffix = Object.keys(entry.data).length ? ` ${JSON.stringify(entry.data)}` : '';
fs.appendFileSync(
path.join(ctx.artifacts, 'run.log'),
`${entry.at} ${level.toUpperCase()} ${event} ${entry.message}${suffix}\n`,
);
process.stdout.write(`[${level.toUpperCase()}] ${entry.message}\n`);
}
export function rawPath(ctx, name) {
return path.join(ctx.dirs.raw, name.replace(/[^a-zA-Z0-9_.-]+/g, '-'));
}
verification/src/core/redact.mjs
+33
View File
@@ -0,0 +1,33 @@
const secretKey = /(password|passwd|token|secret|authorization|cookie|api[-_]?key)/i;
const bearer = /\b(Bearer\s+)[A-Za-z0-9._~+/=-]+/gi;
const githubToken = /\b(gh[opusr]_[A-Za-z0-9_]{20,})\b/g;
const jwt = /\beyJ[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\b/g;
const secretName = '(?:password|passwd|token|secret|authorization|cookie|api[-_]?key)';
function redactAssignments(value) {
return value
.replace(new RegExp(`("${secretName}"\\s*:\\s*)"[^"]*"`, 'gi'), '$1"[REDACTED]"')
.replace(new RegExp(`('(?:${secretName})'\\s*:\\s*)'[^']*'`, 'gi'), "$1'[REDACTED]'")
.replace(new RegExp(`\\b(${secretName})(\\s*=\\s*)[^\\s,;]+`, 'gi'), '$1$2[REDACTED]')
.replace(new RegExp(`(--${secretName})(?:=|\\s+)([^\\s]+)`, 'gi'), '$1=[REDACTED]')
.replace(/\b([A-Z0-9_]*(?:TOKEN|SECRET|PASSWORD|PASSWD|API_KEY|AUTHORIZATION|COOKIE)[A-Z0-9_]*=)([^\s]+)/g, '$1[REDACTED]')
.replace(/(https?:\/\/[^\s/:@]+:)[^\s@/]+@/gi, '$1[REDACTED]@');
}
export function redactText(value) {
return redactAssignments(String(value))
.replace(bearer, '$1[REDACTED]')
.replace(githubToken, '[REDACTED_GITHUB_TOKEN]')
.replace(jwt, '[REDACTED_JWT]')
.replace(/([?&](?:token|key|secret|password)=)[^&#\s]+/gi, '$1[REDACTED]');
}
export function redact(value, key = '') {
if (secretKey.test(key)) return '[REDACTED]';
if (typeof value === 'string') return redactText(value);
if (Array.isArray(value)) return value.map((item) => redact(item));
if (!value || typeof value !== 'object') return value;
return Object.fromEntries(
Object.entries(value).map(([childKey, child]) => [childKey, redact(child, childKey)]),
);
}
verification/src/core/service.mjs
+81
View File
@@ -0,0 +1,81 @@
import fs from 'node:fs';
import { spawn } from 'node:child_process';
import { StringDecoder } from 'node:string_decoder';
import { rawPath } from './log.mjs';
import { redactText } from './redact.mjs';
export async function waitForUrl(url, timeoutMs = 60_000, stopped = () => '') {
const deadline = Date.now() + timeoutMs;
let lastError = '';
while (Date.now() < deadline) {
const stopReason = stopped();
if (stopReason) return { ok: false, error: stopReason };
try {
const response = await fetch(url, { redirect: 'manual', signal: AbortSignal.timeout(3000) });
if (response.status < 500) return { ok: true, status: response.status };
lastError = `HTTP ${response.status}`;
} catch (error) {
lastError = error instanceof Error ? error.message : String(error);
}
await new Promise((resolve) => setTimeout(resolve, 500));
}
return { ok: false, error: lastError };
}
function exitReason(service) {
if (!service.exit) return '';
if (service.exit.error) return `process failed before readiness: ${service.exit.error}`;
return `process exited before readiness: code ${service.exit.code ?? 'null'}, signal ${service.exit.signal || 'none'}`;
}
export async function startProcess(ctx, name, command, args, options = {}) {
const outputPath = rawPath(ctx, `${name}.log`);
const stream = fs.createWriteStream(outputPath, { flags: 'w' });
const child = spawn(command, args, {
cwd: options.cwd || ctx.config.root,
env: { ...process.env, ...(options.env || {}) },
detached: process.platform !== 'win32',
stdio: ['ignore', 'pipe', 'pipe'],
});
const stdout = new StringDecoder('utf8');
const stderr = new StringDecoder('utf8');
const append = (value) => { const safe = redactText(value); if (safe) stream.write(safe); };
child.stdout.on('data', (chunk) => append(stdout.write(chunk)));
child.stderr.on('data', (chunk) => append(stderr.write(chunk)));
let finishOutput;
const done = new Promise((resolve) => { finishOutput = resolve; });
const service = { name, child, outputPath, stopped: false, exit: null, done };
child.once('error', (error) => { service.exit = { error: error.message }; });
child.once('close', (code, signal) => {
service.exit = { code, signal };
append(stdout.end()); append(stderr.end());
stream.end(finishOutput);
});
ctx.services.push(service);
if (options.url) {
service.ready = await waitForUrl(options.url, options.timeoutMs, () => exitReason(service));
if (service.ready.ok) {
await new Promise((resolve) => setTimeout(resolve, 300));
const reason = exitReason(service);
if (reason) service.ready = { ok: false, error: reason };
}
}
return service;
}
export function stopProcess(service) {
if (!service?.child?.pid || service.stopped) return;
service.stopped = true;
try {
process.kill(-service.child.pid, 'SIGTERM');
} catch {
try { service.child.kill('SIGTERM'); } catch { /* already stopped */ }
}
}
export function stopAll(ctx) {
for (const service of [...ctx.services].reverse()) {
if (typeof service.close === 'function') service.close();
else stopProcess(service);
}
}
verification/src/core/stage.mjs
+25
View File
@@ -0,0 +1,25 @@
import { finding } from './finding.mjs';
import { log } from './log.mjs';
export async function runStage(ctx, id, title, task) {
const started = Date.now();
log(ctx, 'info', `stage.${id}.start`, `Starting ${title}`);
try {
await task(ctx);
log(ctx, 'info', `stage.${id}.end`, `Finished ${title}`, { durationMs: Date.now() - started });
} catch (error) {
finding(ctx, {
id: `stage.${id}.crash`,
category: 'harness',
title: `${title} crashed`,
status: 'FAIL',
severity: 'critical',
expected: 'Stage completes and records granular findings',
actual: error instanceof Error ? error.stack || error.message : String(error),
reason: 'The verification harness encountered an unhandled exception.',
impact: 'Checks in this stage may be incomplete.',
remediation: 'Inspect the stack trace and repair the harness before trusting this run.',
durationMs: Date.now() - started,
});
}
}
verification/src/core/xml.mjs
+8
View File
@@ -0,0 +1,8 @@
export function escapeXml(value) {
return String(value ?? '')
.replaceAll('&', '&amp;')
.replaceAll('<', '&lt;')
.replaceAll('>', '&gt;')
.replaceAll('"', '&quot;')
.replaceAll("'", '&apos;');
}
verification/src/fixture/media.mjs
+34
View File
@@ -0,0 +1,34 @@
import fs from 'node:fs';
import path from 'node:path';
import { runCommand } from '../core/command.mjs';
import { finding } from '../core/finding.mjs';
export async function createMedia(ctx) {
const mp4 = path.join(ctx.dirs.media, 'test.mp4');
const hls = path.join(ctx.dirs.media, 'hls');
fs.mkdirSync(hls, { recursive: true });
const generate = await runCommand(ctx, 'fixture-media', 'ffmpeg', [
'-hide_banner', '-loglevel', 'error', '-y',
'-f', 'lavfi', '-i', 'testsrc=size=640x360:rate=30',
'-f', 'lavfi', '-i', 'sine=frequency=440:sample_rate=44100',
'-t', '60', '-c:v', 'libx264', '-preset', 'veryfast', '-pix_fmt', 'yuv420p',
'-g', '60', '-keyint_min', '60', '-sc_threshold', '0',
'-c:a', 'aac', '-b:a', '96k', '-movflags', '+faststart', mp4,
], { timeoutMs: 120_000 });
const segment = generate.code === 0 ? await runCommand(ctx, 'fixture-hls', 'ffmpeg', [
'-hide_banner', '-loglevel', 'error', '-y', '-i', mp4,
'-c', 'copy', '-hls_time', '2', '-hls_list_size', '0',
'-hls_segment_filename', path.join(hls, 'segment-%03d.seg'), path.join(hls, 'master.m3u8'),
], { timeoutMs: 120_000 }) : { code: 1, outputPath: generate.outputPath, durationMs: 0 };
const ok = generate.code === 0 && segment.code === 0 && fs.existsSync(mp4);
finding(ctx, {
id: 'fixture.media', category: 'harness', title: 'Deterministic MP4 and HLS fixtures were generated',
status: ok ? 'PASS' : 'FAIL', severity: 'critical', expected: 'Playable 60-second MP4 and 2-second-GOP HLS assets',
actual: ok ? `${fs.statSync(mp4).size} bytes` : `ffmpeg exits ${generate.code}/${segment.code}`,
reason: ok ? 'Video checks use locally generated media and do not depend on third-party streams.' : 'Video evidence cannot be produced without deterministic fixtures.',
evidence: [generate.outputPath, segment.outputPath], remediation: 'Install a working ffmpeg with H.264/AAC support.',
durationMs: generate.durationMs + segment.durationMs,
});
ctx.state.mediaOk = ok;
return { mp4, hls };
}
verification/src/fixture/server.mjs
+36
View File
@@ -0,0 +1,36 @@
import http from 'node:http';
import path from 'node:path';
import { posterSvg, sourceImportPayload, sourceResponse } from './source.mjs';
import { sendFile } from './static.mjs';
function json(response, status, value, headers = {}) {
response.writeHead(status, { 'Content-Type': 'application/json', 'Access-Control-Allow-Origin': '*', ...headers });
response.end(JSON.stringify(value));
}
export async function startFixtureServer(ctx) {
const baseUrl = ctx.config.fixtureUrl;
const server = http.createServer(async (request, response) => {
const url = new URL(request.url || '/', baseUrl);
if (request.method === 'OPTIONS') { response.writeHead(204, { 'Access-Control-Allow-Origin': '*', 'Access-Control-Allow-Headers': '*' }); response.end(); return; }
if (url.pathname === '/health' || url.pathname === '/fast') { json(response, 200, { ok: true, at: Date.now() }); return; }
if (url.pathname === '/slow') {
const delay = Math.min(Number(url.searchParams.get('ms')) || 250, 5000);
await new Promise((resolve) => setTimeout(resolve, delay)); json(response, 200, { ok: true, delay }); return;
}
if (url.pathname.startsWith('/status/')) { const status = Number(url.pathname.split('/').pop()) || 500; json(response, status, { status }); return; }
if (url.pathname === '/redirect') { response.writeHead(302, { Location: `${baseUrl}/fast` }); response.end(); return; }
if (url.pathname === '/headers') { json(response, 200, { method: request.method, headers: request.headers }); return; }
const sourceRoutes = { '/source': () => sourceResponse(baseUrl, url.searchParams),
'/source-import.json': () => sourceImportPayload(baseUrl) };
if (sourceRoutes[url.pathname]) { json(response, 200, sourceRoutes[url.pathname]()); return; }
if (url.pathname === '/poster.svg') { response.writeHead(200, { 'Content-Type': 'image/svg+xml', 'Access-Control-Allow-Origin': '*' }); response.end(posterSvg(url.searchParams.get('item'))); return; }
if (url.pathname === '/test.mp4') { sendFile(request, response, path.join(ctx.dirs.media, 'test.mp4')); return; }
if (url.pathname.startsWith('/hls/')) { sendFile(request, response, path.join(ctx.dirs.media, url.pathname.slice(1))); return; }
json(response, 404, { error: 'fixture route not found', path: url.pathname });
});
await new Promise((resolve, reject) => server.once('error', reject).listen(ctx.config.fixturePort, '127.0.0.1', resolve));
const service = { name: 'fixture-server', close: () => server.close() };
ctx.services.push(service);
return service;
}
verification/src/fixture/source.mjs
+37
View File
@@ -0,0 +1,37 @@
export function sourceResponse(baseUrl, searchParams) {
const episodes = `第1集$${baseUrl}/test.mp4#第2集$${baseUrl}/hls/master.m3u8`;
const id = searchParams.get('ids') || 'fixture-video-1';
const query = searchParams.get('wd') || '验证视频';
return {
code: 1,
msg: 'ok',
page: 1,
pagecount: 1,
total: 3,
list: [1, 2, 3].map((number) => ({
vod_id: number === 1 ? id : `fixture-video-${number}`,
vod_name: `${query} ${number}`,
vod_pic: `${baseUrl}/poster.svg?item=${number}`,
vod_remarks: number === 1 ? '全2集' : '测试内容',
vod_year: '2026',
vod_area: 'Fixture',
vod_actor: 'Automated Validator',
vod_director: 'KVideo Verification',
vod_content: 'Deterministic content used only by the local verification suite.',
type_name: '测试',
vod_play_from: 'm3u8',
vod_play_url: episodes,
})),
};
}
export function sourceImportPayload(baseUrl) {
return {
sources: [{ id: 'imported-fixture', name: 'Imported Fixture', baseUrl, enabled: true, group: 'normal' }],
};
}
export function posterSvg(item = '1') {
const safe = String(item).replace(/[^0-9A-Za-z_-]/g, '');
return `<svg xmlns="http://www.w3.org/2000/svg" width="400" height="600"><rect width="100%" height="100%" fill="#101827"/><circle cx="200" cy="220" r="90" fill="#4f8cff"/><text x="200" y="390" text-anchor="middle" fill="white" font-family="sans-serif" font-size="36">KVideo ${safe}</text></svg>`;
}
verification/src/fixture/static.mjs
+25
View File
@@ -0,0 +1,25 @@
import fs from 'node:fs';
import path from 'node:path';
const types = {
'.mp4': 'video/mp4', '.m3u8': 'application/vnd.apple.mpegurl', '.seg': 'video/mp2t',
};
export function sendFile(request, response, file) {
if (!fs.existsSync(file)) { response.writeHead(404); response.end('not found'); return; }
const size = fs.statSync(file).size;
const range = request.headers.range;
const headers = { 'Content-Type': types[path.extname(file)] || 'application/octet-stream', 'Accept-Ranges': 'bytes', 'Access-Control-Allow-Origin': '*' };
if (!range) {
response.writeHead(200, { ...headers, 'Content-Length': size });
fs.createReadStream(file).pipe(response);
return;
}
const match = /^bytes=(\d*)-(\d*)$/.exec(range);
if (!match) { response.writeHead(416, { 'Content-Range': `bytes */${size}` }); response.end(); return; }
const start = match[1] ? Number(match[1]) : 0;
const end = match[2] ? Math.min(Number(match[2]), size - 1) : size - 1;
if (start > end || start >= size) { response.writeHead(416, { 'Content-Range': `bytes */${size}` }); response.end(); return; }
response.writeHead(206, { ...headers, 'Content-Length': end - start + 1, 'Content-Range': `bytes ${start}-${end}/${size}` });
fs.createReadStream(file, { start, end }).pipe(response);
}
verification/src/history/audit.mjs
+54
View File
@@ -0,0 +1,54 @@
import { classifyItem, priorityOf, uncatalogedRecords } from './catalog.mjs';
import { graphTruncation } from './github.mjs';
import { normalizeHistory } from './normalize.mjs';
import { traceFiles } from './trace.mjs';
function difference(left, right) {
const other = new Set(right);
return left.filter((item) => !other.has(item));
}
function reviewedRisks(remote, normalized, catalog) {
const merged = new Set(remote.pullRequests.filter((item) => item.merged_at).map((item) => item.number));
const graph = new Map(remote.reviewGraph.map((item) => [item.number, item]));
const threads = [...merged].flatMap((number) => (graph.get(number)?.reviewThreads.nodes || [])
.filter((item) => !item.isResolved && !item.isOutdated)
.map((item) => ({ id: item.id, number, path: item.path, priority: priorityOf(item.comments.nodes[0]?.body || '') })));
const unavailable = new Set(catalog.unavailablePullRequests);
const comments = normalized.data.reviewComments.filter((item) => unavailable.has(item.number))
.map((item) => ({ ...item, priority: priorityOf(item.body) }));
return { threads, comments };
}
function recordMatrices(remote, catalog, trace) {
const merged = new Set(remote.pullRequests.filter((item) => item.merged_at).map((item) => item.number));
const issues = remote.issues.filter((item) => catalog.issues.includes(item.number))
.map((item) => ({ number: item.number, title: item.title, ...classifyItem(item, catalog), trace: traceFiles(trace, 'issues', item.number) }));
const pulls = remote.pullRequests.filter((item) => catalog.pullRequests.includes(item.number))
.map((item) => ({ number: item.number, title: item.title, merged: merged.has(item.number),
...classifyItem(item, catalog, merged.has(item.number)), trace: traceFiles(trace, 'pullRequests', item.number) }));
return { issues, pulls };
}
export function createGithubAudit(remote, catalog, trace, currentPr = null) {
const normalized = normalizeHistory(remote, catalog);
const visible = new Set(remote.pullRequests.map((item) => item.number));
const orphanPrs = [...new Set(remote.reviewComments.map((item) => Number(item.pull_request_url.split('/').pop()))
.filter((number) => !visible.has(number)))].sort((a, b) => a - b);
const auditedIssues = remote.issues.filter((item) => item.number <= catalog.issueCutoff).map((item) => item.number).sort((a, b) => a - b);
const auditedPrs = remote.pullRequests.filter((item) => item.number <= catalog.pullRequestCutoff).map((item) => item.number).sort((a, b) => a - b);
const inventoryDelta = {
missingIssues: difference(catalog.issues, auditedIssues), extraIssues: difference(auditedIssues, catalog.issues),
missingPrs: difference(catalog.pullRequests, auditedPrs), extraPrs: difference(auditedPrs, catalog.pullRequests),
missingUnavailablePrs: difference(catalog.unavailablePullRequests, orphanPrs),
extraUnavailablePrs: difference(orphanPrs, catalog.unavailablePullRequests),
};
const matrices = recordMatrices(remote, catalog, trace);
const remoteBugs = matrices.issues.filter((item) => item.regressionRequired).map((item) => item.number).sort((a, b) => a - b);
const remoteMerged = matrices.pulls.filter((item) => item.merged).map((item) => item.number).sort((a, b) => a - b);
const coverageDelta = { missingBugs: difference(catalog.regressionIssues, remoteBugs), extraBugs: difference(remoteBugs, catalog.regressionIssues),
missingMerged: difference(catalog.mergedPullRequests, remoteMerged), extraMerged: difference(remoteMerged, catalog.mergedPullRequests) };
return { normalized, inventoryDelta, coverageDelta, uncataloged: uncatalogedRecords(remote, catalog, currentPr),
truncation: graphTruncation(remote), issues: matrices.issues, pullRequests: matrices.pulls,
review: reviewedRisks(remote, normalized, catalog) };
}
verification/src/history/catalog.mjs
+98
View File
@@ -0,0 +1,98 @@
import fs from 'node:fs';
import path from 'node:path';
const categoryPatterns = [
['auth-sync', /auth|login|password|session|redis|upstash|登录|密码|账号|账户|同步|收藏|播放记录/i],
['deployment', /docker|cloudflare|vercel|deploy|build|镜像|版本|部署|构建|局域网|端口/i],
['platform', /android|apk|webview|ios|iphone|ipad|safari|电视|tv|车机|移动端/i],
['player-media', /player|video|m3u8|hls|播放|全屏|画中画|进度|缓冲|广告|弹幕|清晰度|分辨率|投屏/i],
['search-source', /search|source|subscription|搜索|换源|视频源|订阅|首页|推荐|海报|图片|分类|标签/i],
['security-proxy', /security|proxy|ssrf|xss|权限|安全|代理|漏洞/i],
['ui', /ui|button|layout|style|按钮|界面|布局|主题|logo|悬浮|鼠标|显示/i],
];
function read(root, name) {
return JSON.parse(fs.readFileSync(path.join(root, 'verification', 'history', name), 'utf8'));
}
export function loadHistoryCatalog(root) {
return {
catalog: read(root, 'catalog.json'),
baseline: read(root, 'baseline.json'),
decisions: read(root, 'review-decisions.json'),
commentDecisions: read(root, 'review-comment-decisions.json'),
};
}
export function classifyItem(item, catalog, merged = false) {
const labels = (item.labels || []).map((label) => typeof label === 'string' ? label : label.name);
const text = `${item.title || ''}\n${item.body || ''}`;
const categories = categoryPatterns.filter(([, pattern]) => pattern.test(text)).map(([name]) => name);
if (!categories.length) categories.push('general');
const unverifiable = catalog.unverifiableIssues.find((entry) => entry.number === item.number);
const override = catalog.regressionIssueOverrides.includes(item.number);
const regressionRequired = merged || (!unverifiable && (labels.includes('bug') || override));
const evidence = [...new Set(categories.flatMap((name) => catalog.evidence[name] || catalog.evidence.general))];
return { categories, regressionRequired, evidence, unverifiableReason: unverifiable?.reason || null };
}
export function uncatalogedRecords(remote, catalog, currentPr = null) {
return {
issues: remote.issues.filter((item) => item.number > catalog.issueCutoff).map((item) => item.number),
pullRequests: remote.pullRequests.filter((item) => item.number > catalog.pullRequestCutoff && item.number !== currentPr).map((item) => item.number),
};
}
function validDigest(value) {
return Number.isInteger(value?.count) && value.count >= 0 && /^[a-f0-9]{64}$/.test(value?.sha256 || '');
}
export function validateCatalog(root, catalog, baseline, decisions, commentDecisions) {
const issues = new Set(catalog.issues);
const prs = new Set(catalog.pullRequests);
const unavailable = new Set(catalog.unavailablePullRequests);
const regressionIssues = new Set(catalog.regressionIssues || []);
const mergedPullRequests = new Set(catalog.mergedPullRequests || []);
const decisionIds = new Set(decisions.map((item) => item.id));
const commentIds = new Set(commentDecisions.map((item) => item.id));
const duplicates = [
...(issues.size === catalog.issues.length ? [] : ['issue numbers']),
...(prs.size === catalog.pullRequests.length ? [] : ['pull-request numbers']),
...(unavailable.size === catalog.unavailablePullRequests.length ? [] : ['unavailable pull-request numbers']),
...(regressionIssues.size === (catalog.regressionIssues || []).length ? [] : ['regression issue numbers']),
...(mergedPullRequests.size === (catalog.mergedPullRequests || []).length ? [] : ['merged pull-request numbers']),
...(decisionIds.size === decisions.length ? [] : ['review decision ids']),
...(commentIds.size === commentDecisions.length ? [] : ['review-comment decision ids']),
];
const evidence = [...new Set(Object.values(catalog.evidence).flat())];
const missingEvidence = evidence.filter((file) => !fs.existsSync(path.join(root, file)));
const invalidDecisionPrs = decisions.filter((item) => !prs.has(item.pr)).map((item) => item.id);
const invalidCommentPrs = commentDecisions.filter((item) => !unavailable.has(item.pr)).map((item) => item.id);
const overlappingPrs = catalog.unavailablePullRequests.filter((number) => prs.has(number));
const invalidOverrides = catalog.regressionIssueOverrides.filter((number) => !issues.has(number));
const invalidUnverifiable = catalog.unverifiableIssues.filter((item) => !issues.has(item.number)).map((item) => item.number);
const invalidRegressionIssues = [...regressionIssues].filter((number) => !issues.has(number));
const invalidMergedPullRequests = [...mergedPullRequests].filter((number) => !prs.has(number));
const unverifiableRegressionIssues = catalog.unverifiableIssues.filter((item) => regressionIssues.has(item.number)).map((item) => item.number);
const validStatuses = new Set(['open', 'fixed', 'dismissed']);
const validPriorities = new Set(['critical', 'high', 'medium', 'low']);
const invalidDecisionFields = [...decisions, ...commentDecisions].filter((item) => (
!validStatuses.has(item.status) || !validPriorities.has(item.priority) || !item.contract || !item.reason
)).map((item) => item.id);
const digestKeys = ['issues', 'pullRequests', 'conversationComments', 'reviewComments', 'reviews', 'reviewThreads'];
const invalidBaseline = [
...digestKeys.filter((key) => !validDigest(baseline[key])),
...(/^[a-f0-9]{64}$/.test(baseline.combinedSha256 || '') ? [] : ['combinedSha256']),
];
return { duplicates, missingEvidence, invalidDecisionPrs, invalidCommentPrs, overlappingPrs,
invalidOverrides, invalidUnverifiable, invalidRegressionIssues, invalidMergedPullRequests,
unverifiableRegressionIssues, invalidDecisionFields, invalidBaseline };
}
export function priorityOf(body = '') {
const text = body.toLowerCase();
if (text.includes('critical')) return 'critical';
if (text.includes('high')) return 'high';
if (text.includes('medium')) return 'medium';
return 'unspecified';
}
verification/src/history/github.mjs
+66
View File
@@ -0,0 +1,66 @@
import fs from 'node:fs';
import { runCommand } from '../core/command.mjs';
const REVIEW_QUERY = `query {
repository(owner:"KuekHaoYang", name:"KVideo") {
pullRequests(first:100, orderBy:{field:CREATED_AT,direction:ASC}) {
totalCount
nodes {
number merged
reviews(first:100) { totalCount nodes { id state body submittedAt author { login } } }
reviewThreads(first:100) {
totalCount nodes {
id isResolved isOutdated path line originalLine
comments(first:20) { totalCount nodes { id body createdAt updatedAt author { login } } }
}
}
}
}
}
}`;
function parse(result) {
if (result.code !== 0) return null;
try { return JSON.parse(fs.readFileSync(result.outputPath, 'utf8')); }
catch { return null; }
}
function rest(ctx, name, endpoint) {
return runCommand(ctx, name, 'gh', ['api', '--paginate', '--slurp', '-X', 'GET', endpoint], { timeoutMs: 120_000 });
}
export async function fetchGithubHistory(ctx) {
const commands = await Promise.all([
rest(ctx, 'github-issues', 'repos/KuekHaoYang/KVideo/issues?state=all&per_page=100'),
rest(ctx, 'github-pulls', 'repos/KuekHaoYang/KVideo/pulls?state=all&per_page=100'),
rest(ctx, 'github-conversation-comments', 'repos/KuekHaoYang/KVideo/issues/comments?per_page=100'),
rest(ctx, 'github-review-comments', 'repos/KuekHaoYang/KVideo/pulls/comments?per_page=100'),
runCommand(ctx, 'github-reviews-threads', 'gh', ['api', 'graphql', '-f', `query=${REVIEW_QUERY}`], { timeoutMs: 120_000 }),
]);
const parsed = commands.map(parse);
if (parsed.some((item) => item === null)) return { commands, error: 'One or more GitHub responses failed or were not valid JSON.' };
const [issuePages, pullPages, conversationPages, reviewCommentPages, graph] = parsed;
const graphErrors = graph.errors || [];
if (graphErrors.length) return { commands, error: JSON.stringify(graphErrors) };
return {
commands,
issues: issuePages.flat().filter((item) => !item.pull_request),
pullRequests: pullPages.flat(),
conversationComments: conversationPages.flat(),
reviewComments: reviewCommentPages.flat(),
reviewGraph: graph.data.repository.pullRequests.nodes,
reviewGraphTotal: graph.data.repository.pullRequests.totalCount,
};
}
export function graphTruncation(remote) {
const rows = [{ scope: 'pullRequests', expected: remote.reviewGraphTotal, actual: remote.reviewGraph.length }];
for (const pr of remote.reviewGraph) {
rows.push({ scope: `PR ${pr.number} reviews`, expected: pr.reviews.totalCount, actual: pr.reviews.nodes.length });
rows.push({ scope: `PR ${pr.number} threads`, expected: pr.reviewThreads.totalCount, actual: pr.reviewThreads.nodes.length });
for (const thread of pr.reviewThreads.nodes) {
rows.push({ scope: `thread ${thread.id} comments`, expected: thread.comments.totalCount, actual: thread.comments.nodes.length });
}
}
return rows.filter((item) => item.expected !== item.actual);
}
verification/src/history/local.mjs
+32
View File
@@ -0,0 +1,32 @@
import fs from 'node:fs';
import path from 'node:path';
import { relative, walk } from '../core/files.mjs';
import { traceFiles } from './trace.mjs';
function importedChecks(root) {
const main = fs.readFileSync(path.join(root, 'verification', 'src', 'main.mjs'), 'utf8');
const pattern = /from ['"]\.\/checks\/([^'"]+\.mjs)['"]/g;
return new Set([...main.matchAll(pattern)].map((match) => `verification/src/checks/${match[1]}`));
}
export function executableEvidence(root) {
const checks = importedChecks(root);
const tests = walk(path.join(root, 'verification', 'tests', 'regression'), (file) => file.endsWith('.test.ts'))
.map((file) => relative(root, file));
return new Set([...checks, ...tests]);
}
function matrix(numbers, kind, trace) {
return numbers.map((number) => ({ number, files: traceFiles(trace, kind, number) }));
}
export function localCoverage(root, catalog, trace) {
const runnable = executableEvidence(root);
const issues = matrix(catalog.regressionIssues || [], 'issues', trace);
const pullRequests = matrix(catalog.mergedPullRequests || [], 'pullRequests', trace);
const required = [...issues, ...pullRequests];
const missing = required.filter((item) => !item.files.length);
const nonExecutable = required.flatMap((item) => item.files
.filter((file) => !runnable.has(file)).map((file) => ({ number: item.number, file })));
return { issues, pullRequests, missing, nonExecutable, unknown: trace.unknown, runnable: [...runnable].sort() };
}
verification/src/history/normalize.mjs
+56
View File
@@ -0,0 +1,56 @@
import crypto from 'node:crypto';
function hash(value) {
return crypto.createHash('sha256').update(JSON.stringify(value)).digest('hex');
}
function numberFrom(url) {
return Number(url?.split('/').pop());
}
function byNumber(a, b) {
return (a.number || 0) - (b.number || 0) || String(a.id || '').localeCompare(String(b.id || ''));
}
export function normalizeHistory(remote, catalog) {
const issueIds = new Set(catalog.issues);
const prIds = new Set(catalog.pullRequests);
const itemIds = new Set([...issueIds, ...prIds]);
const reviewPrIds = new Set([...prIds, ...catalog.unavailablePullRequests]);
const issues = remote.issues.filter((item) => issueIds.has(item.number)).map((item) => ({
number: item.number, state: item.state, title: item.title, body: item.body || '',
stateReason: item.state_reason || null, locked: Boolean(item.locked),
labels: (item.labels || []).map((label) => label.name).sort(), createdAt: item.created_at, updatedAt: item.updated_at,
})).sort(byNumber);
const pullRequests = remote.pullRequests.filter((item) => prIds.has(item.number)).map((item) => ({
number: item.number, state: item.state, mergedAt: item.merged_at || null,
title: item.title, body: item.body || '', base: item.base?.ref || '', head: item.head?.ref || '',
headSha: item.head?.sha || null, mergeCommitSha: item.merge_commit_sha || null,
additions: item.additions, deletions: item.deletions, changedFiles: item.changed_files,
})).sort(byNumber);
const conversationComments = remote.conversationComments.map((item) => ({
id: item.node_id || String(item.id), databaseId: item.id, number: numberFrom(item.issue_url),
body: item.body || '', author: item.user?.login || '', createdAt: item.created_at, updatedAt: item.updated_at,
})).filter((item) => itemIds.has(item.number)).sort(byNumber);
const reviewComments = remote.reviewComments.map((item) => ({
id: item.node_id || String(item.id), databaseId: item.id, number: numberFrom(item.pull_request_url),
body: item.body || '', path: item.path || '', line: item.line || item.original_line || null,
commitId: item.commit_id || null, originalCommitId: item.original_commit_id || null,
replyToId: item.in_reply_to_id || null, author: item.user?.login || '', createdAt: item.created_at, updatedAt: item.updated_at,
})).filter((item) => reviewPrIds.has(item.number)).sort(byNumber);
const reviews = remote.reviewGraph.flatMap((pr) => pr.reviews.nodes.map((item) => ({
id: item.id, number: pr.number, state: item.state, body: item.body || '',
author: item.author?.login || '', submittedAt: item.submittedAt || null,
}))).filter((item) => prIds.has(item.number)).sort(byNumber);
const reviewThreads = remote.reviewGraph.flatMap((pr) => pr.reviewThreads.nodes.map((item) => ({
id: item.id, number: pr.number, resolved: item.isResolved, outdated: item.isOutdated,
path: item.path || '', line: item.line || item.originalLine || null,
comments: item.comments.nodes.map((comment) => ({
id: comment.id, body: comment.body || '', author: comment.author?.login || '',
createdAt: comment.createdAt || null, updatedAt: comment.updatedAt || null,
})).sort(byNumber),
}))).filter((item) => prIds.has(item.number)).sort(byNumber);
const data = { issues, pullRequests, conversationComments, reviewComments, reviews, reviewThreads };
const digests = Object.fromEntries(Object.entries(data).map(([name, value]) => [name, { count: value.length, sha256: hash(value) }]));
return { data, digests: { ...digests, combinedSha256: hash(data) } };
}
verification/src/history/trace.mjs
+44
View File
@@ -0,0 +1,44 @@
import fs from 'node:fs';
import path from 'node:path';
import { relative, walk } from '../core/files.mjs';
function numbers(text) {
return (text.match(/\d+/g) || []).map(Number);
}
function add(map, number, file) {
const files = map.get(number) || [];
if (!files.includes(file)) files.push(file);
map.set(number, files);
}
function tags(text, name) {
const pattern = new RegExp(`GH-${name}:\\s*([^;\\n]+)`, 'g');
return [...text.matchAll(pattern)].flatMap((match) => numbers(match[1]));
}
export function collectTraceability(root, catalog) {
const issues = new Map();
const pullRequests = new Map();
const verifyDir = path.join(root, 'verification');
const files = walk(verifyDir, (file) => {
const normalized = file.split(path.sep).join('/');
return normalized.includes('/src/checks/') || normalized.includes('/tests/regression/');
}).map((file) => relative(root, file)).sort();
for (const file of files) {
const text = fs.readFileSync(path.join(root, file), 'utf8');
for (const number of tags(text, 'ISSUE')) add(issues, number, file);
for (const number of tags(text, 'PR')) add(pullRequests, number, file);
}
const validIssues = new Set(catalog.issues);
const validPrs = new Set([...catalog.pullRequests, ...catalog.unavailablePullRequests]);
const unknown = [
...[...issues].filter(([number]) => !validIssues.has(number)).map(([number, files]) => ({ kind: 'issue', number, files })),
...[...pullRequests].filter(([number]) => !validPrs.has(number)).map(([number, files]) => ({ kind: 'pullRequest', number, files })),
];
return { issues, pullRequests, unknown };
}
export function traceFiles(trace, kind, number) {
return [...(trace[kind].get(number) || [])];
}
verification/src/main.mjs
+84
View File
@@ -0,0 +1,84 @@
#!/usr/bin/env node
import fs from 'node:fs';
import path from 'node:path';
import { getConfig } from './config.mjs';
import { createContext } from './core/context.mjs';
import { hasFailures } from './core/finding.mjs';
import { log } from './core/log.mjs';
import { runStage } from './core/stage.mjs';
import { stopAll } from './core/service.mjs';
import { writeReports } from './report/write.mjs';
import { checkPreflight, checkWorkspaceBoundary } from './checks/preflight.mjs';
import { checkHarnessSelf } from './checks/harness-self.mjs';
import { checkSourcePolicy } from './checks/source-policy.mjs';
import { checkGithubHistory } from './checks/github-history.mjs';
import { checkLocalHistory } from './checks/history-local.mjs';
import { checkPrPolicy } from './checks/pr-policy.mjs';
import { checkAstMetrics } from './checks/ast-metrics.mjs';
import { checkVerifierQuality } from './checks/verifier-quality.mjs';
import { checkImportGraph } from './checks/import-graph.mjs';
import { checkSecurityScan } from './checks/security-scan.mjs';
import { checkDuplicates } from './checks/duplicates.mjs';
import { checkStaticTools } from './checks/static-tools.mjs';
import { checkAndroid } from './checks/android.mjs';
import { checkDockerLocal } from './checks/docker-local.mjs';
import { startRuntime } from './checks/runtime-start.mjs';
import { checkApiDiscovery } from './checks/api-discovery.mjs';
import { checkApiContracts } from './checks/api-contracts.mjs';
import { checkProxy } from './checks/proxy.mjs';
import { checkLatency } from './checks/latency.mjs';
import { checkSecurityHeaders } from './checks/security-headers.mjs';
import { checkUiPages } from './checks/ui-pages.mjs';
import { checkUiActions } from './checks/ui-actions.mjs';
import { checkPerformance } from './checks/performance.mjs';
import { checkVideo } from './checks/video.mjs';
import { checkVisual } from './checks/visual.mjs';
import { checkDeployment } from './checks/deployment.mjs';
const config = getConfig(process.argv);
const ctx = createContext(config);
ctx.state.version = JSON.parse(fs.readFileSync(path.join(config.root, 'package.json'), 'utf8')).version;
process.on('SIGINT', () => { stopAll(ctx); process.exit(130); });
process.on('SIGTERM', () => { stopAll(ctx); process.exit(143); });
async function main() {
log(ctx, 'info', 'run.start', 'KVideo strict verification started', { runId: ctx.runId, root: config.root });
await runStage(ctx, 'preflight', 'preflight and harness integrity', async () => {
await checkPreflight(ctx); await checkHarnessSelf(ctx); await checkSourcePolicy(ctx);
});
await runStage(ctx, 'history', 'GitHub history and pull-request regression policy', async () => {
if (config.auditGithub) await checkGithubHistory(ctx);
else await checkLocalHistory(ctx);
await checkPrPolicy(ctx);
});
await runStage(ctx, 'quality', 'structural quality and security analysis', async () => {
await checkVerifierQuality(ctx); await checkAstMetrics(ctx); await checkImportGraph(ctx);
await checkSecurityScan(ctx); await checkDuplicates(ctx);
});
await runStage(ctx, 'static', 'unit, coverage, lint, type, dependency, Android, and production builds', async () => {
await checkStaticTools(ctx); await checkAndroid(ctx);
});
await runStage(ctx, 'docker', 'local release container', checkDockerLocal);
await runStage(ctx, 'runtime', 'deterministic fixtures and local application', startRuntime);
await runStage(ctx, 'api', 'API, proxy, latency, headers, and PWA contracts', async () => {
await checkApiDiscovery(ctx); await checkApiContracts(ctx); await checkProxy(ctx); await checkLatency(ctx); await checkSecurityHeaders(ctx);
});
await runStage(ctx, 'browser', 'UI, accessibility, actions, performance, video, and visual parity', async () => {
await checkUiPages(ctx); await checkUiActions(ctx); await checkPerformance(ctx); await checkVideo(ctx); await checkVisual(ctx);
});
await runStage(ctx, 'deployment', 'public release consistency', checkDeployment);
await runStage(ctx, 'postflight', 'final verification-only workspace boundary', async () => {
await checkWorkspaceBoundary(ctx, 'postflight');
});
}
try { await main(); }
finally {
if (!config.keepServer) stopAll(ctx);
const counts = writeReports(ctx);
const latest = path.join(config.verifyDir, 'artifacts', 'latest');
try { if (fs.lstatSync(latest).isSymbolicLink()) fs.unlinkSync(latest); } catch {}
try { fs.symlinkSync(ctx.runId, latest, 'dir'); } catch {}
log(ctx, 'info', 'run.end', 'KVideo strict verification finished', { counts, report: path.join(ctx.artifacts, 'report.html') });
process.exitCode = hasFailures(ctx) ? 1 : 0;
}
verification/src/policy/action-coverage.mjs
+43
View File
@@ -0,0 +1,43 @@
export function actionCoverageStatus(input) {
if (input.emptyRoutes.length) return 'FAIL';
if (input.cappedRoutes.length) return input.quick ? 'SKIP' : 'FAIL';
return 'PASS';
}
function noveltyTokens(snapshot) {
if (!snapshot || typeof snapshot.location !== 'string' || !Array.isArray(snapshot.signatures)) {
throw new TypeError('A state snapshot requires a location and signature array.');
}
const frontierSignature = (item) => item.startsWith('sortable-order|') ? item.split('|').slice(0, 2).join('|') : item;
const controls = [...new Set(snapshot.signatures.map(frontierSignature))].sort().map((item) => `control:${item}`);
return [`location:${snapshot.location}`, ...controls];
}
export function registerNovelState(discovered, snapshot) {
if (!(discovered instanceof Set)) throw new TypeError('Discovered state tokens must be a Set.');
const tokens = noveltyTokens(snapshot);
const added = tokens.filter((token) => !discovered.has(token));
for (const token of added) discovered.add(token);
return { novel: added.length > 0, added, stateFacts: tokens };
}
export function classifyStateTransition(discovered, snapshot, blockedBy = '') {
if (blockedBy) return { eligible: false, queued: false, subsumed: false, reason: blockedBy, newStateFacts: [] };
const novelty = registerNovelState(discovered, snapshot);
return { eligible: true, queued: novelty.novel, subsumed: !novelty.novel,
reason: novelty.novel ? 'new semantic state fact' : 'semantic state facts already covered', newStateFacts: novelty.added };
}
export function actionExecutionKey(snapshot, action) {
if (typeof snapshot?.location !== 'string' || typeof action?.key !== 'string') {
throw new TypeError('Action execution keys require a location and action key.');
}
return `${snapshot.location}|${action.key}`;
}
export function actionCoverageReason(input) {
if (input.emptyRoutes.length) return 'A route/viewport exposed zero controls, so interaction coverage is not credible.';
if (input.cappedRoutes.length) return input.quick
? 'Quick mode intentionally limits exploration.' : 'Reachable control states remain unexplored.';
return 'Every route/viewport exposed controls and no additional state-changing control remained.';
}
verification/src/policy/pr-evidence.mjs
+66
View File
@@ -0,0 +1,66 @@
import fs from 'node:fs';
import path from 'node:path';
const codeExtensions = new Set([
'.css', '.gradle', '.js', '.jsx', '.json', '.kts', '.kt', '.mjs', '.sh', '.ts', '.tsx', '.yaml', '.yml',
]);
const evidencePrefixes = [
'verification/src/checks/',
'verification/tests/harness/',
'verification/tests/regression/',
];
function fieldValues(body, name) {
const pattern = new RegExp(`^${name}:\\s*(.+)$`, 'gim');
return [...body.matchAll(pattern)].flatMap((match) => match[1].split(',')).map((item) => item.trim()).filter(Boolean);
}
export function parsePrBody(body = '') {
const historical = fieldValues(body, 'Historical-Refs');
const evidence = fieldValues(body, 'Regression-Evidence');
const reasons = fieldValues(body, 'Regression-Evidence-Reason');
const refs = historical.flatMap((value) => {
if (/^none$/i.test(value)) return [];
const explicitPrs = [...value.matchAll(/PR\s*#(\d+)/gi)].map((match) => ({ kind: 'PR', number: Number(match[1]) }));
const withoutPrs = value.replace(/PR\s*#\d+/gi, '');
const issues = [...withoutPrs.matchAll(/#(\d+)/g)].map((match) => ({ kind: 'ISSUE', number: Number(match[1]) }));
return [...explicitPrs, ...issues];
});
return { historical, evidence, reasons, refs };
}
export function isCodeChange(file) {
const base = path.basename(file);
return codeExtensions.has(path.extname(file)) || ['Dockerfile', 'package-lock.json', 'package.json'].includes(base);
}
function tagged(text, ref) {
const pattern = new RegExp(`GH-${ref.kind}:\\s*([^;\\n]+)`, 'g');
return [...text.matchAll(pattern)].some((match) => (match[1].match(/\d+/g) || []).map(Number).includes(ref.number));
}
export function evaluatePrEvidence(root, changedFiles, body) {
const parsed = parsePrBody(body);
const codeFiles = changedFiles.filter(isCodeChange);
if (!codeFiles.length) return { ok: true, codeFiles, parsed, errors: [] };
const errors = [];
if (!parsed.historical.length) errors.push('Historical-Refs is required; use none only after checking history.');
if (!parsed.evidence.length) errors.push('At least one Regression-Evidence path is required.');
const evidence = [];
for (const file of parsed.evidence) {
const normalized = file.split(path.sep).join('/');
const allowed = evidencePrefixes.some((prefix) => normalized.startsWith(prefix));
const absolute = path.resolve(root, normalized);
if (!allowed || !absolute.startsWith(`${path.resolve(root)}${path.sep}`)) errors.push(`Disallowed evidence path: ${file}`);
else if (!fs.existsSync(absolute)) errors.push(`Evidence path does not exist: ${file}`);
else evidence.push({ file: normalized, text: fs.readFileSync(absolute, 'utf8') });
}
const evidenceChanged = evidence.some((item) => changedFiles.includes(item.file));
if (evidence.length && !evidenceChanged && !parsed.reasons.some((reason) => reason.length >= 20)) {
errors.push('Unchanged evidence requires a specific Regression-Evidence-Reason of at least 20 characters.');
}
for (const ref of parsed.refs) {
if (!evidence.some((item) => tagged(item.text, ref))) errors.push(`No evidence file directly tags GH-${ref.kind} #${ref.number}.`);
}
return { ok: errors.length === 0, codeFiles, parsed, evidence: evidence.map((item) => item.file), errors };
}
verification/src/report/html.mjs
+35
View File
@@ -0,0 +1,35 @@
import path from 'node:path';
import { escapeXml } from '../core/xml.mjs';
function evidenceLink(ctx, entry) {
const target = path.isAbsolute(entry) ? path.relative(ctx.artifacts, entry) : entry;
const href = target.split(path.sep).map(encodeURIComponent).join('/');
const label = path.isAbsolute(entry) ? path.relative(ctx.config.root, entry) : entry;
return `<li><a href="${escapeXml(href)}">${escapeXml(label)}</a></li>`;
}
function findingCard(ctx, item) {
const evidence = item.evidence.map((entry) => evidenceLink(ctx, entry)).join('');
return `<article class="finding ${item.status.toLowerCase()}">
<header><code>${escapeXml(item.id)}</code><b>${item.status}</b><span>${item.severity}</span></header>
<h3>${escapeXml(item.title)}</h3>
<dl><dt>Expected</dt><dd>${escapeXml(item.expected)}</dd><dt>Actual</dt><dd>${escapeXml(item.actual)}</dd>
<dt>Reason</dt><dd>${escapeXml(item.reason)}</dd><dt>Impact</dt><dd>${escapeXml(item.impact)}</dd>
<dt>Remediation</dt><dd>${escapeXml(item.remediation)}</dd></dl><ul>${evidence}</ul></article>`;
}
export function renderHtml(ctx, summary) {
const cards = ctx.findings.map((item) => findingCard(ctx, item)).join('\n');
return `<!doctype html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width">
<title>KVideo verification ${escapeXml(ctx.runId)}</title><style>
:root{font:14px/1.5 system-ui;color:#18202a;background:#f2f5f8}body{margin:0}main{max-width:1200px;margin:auto;padding:24px}
h1{margin:.2em 0}.summary{display:grid;grid-template-columns:repeat(auto-fit,minmax(130px,1fr));gap:12px;margin:20px 0}
.metric,.finding{background:white;border:1px solid #dce3ea;border-radius:10px;padding:14px;box-shadow:0 2px 7px #0001}.metric b{font-size:24px;display:block}
.finding{margin:12px 0;border-left:7px solid #6b7280}.finding.pass{border-left-color:#0a8f50}.finding.fail{border-left-color:#c62828}.finding.warn{border-left-color:#d97706}
.finding header{display:flex;gap:12px;align-items:center}.finding header b{margin-left:auto}dl{display:grid;grid-template-columns:110px 1fr;gap:5px 12px}dt{font-weight:700}dd{margin:0;white-space:pre-wrap;overflow-wrap:anywhere}
code{overflow-wrap:anywhere}a{color:#075ea8}nav{position:sticky;top:0;background:#18202a;color:white;padding:10px 24px}nav a{color:white;margin-right:15px}</style></head>
<body><nav><a href="summary.md">Summary</a><a href="findings.json">JSON</a><a href="events.ndjson">Events</a><a href="junit.xml">JUnit</a></nav>
<main><h1>KVideo strict verification</h1><p>Run ${escapeXml(ctx.runId)} · ${summary.durationMs} ms · success ${summary.success}</p>
<section class="summary">${Object.entries(summary.counts).map(([key,value]) => `<div class="metric"><b>${value}</b>${key}</div>`).join('')}</section>
<p>A pass means the declared check passed. It is not a proof that undiscovered states cannot fail.</p>${cards}</main></body></html>`;
}
verification/src/report/write.mjs
+48
View File
@@ -0,0 +1,48 @@
import fs from 'node:fs';
import path from 'node:path';
import { writeJson } from '../core/files.mjs';
import { escapeXml } from '../core/xml.mjs';
import { renderHtml } from './html.mjs';
function totals(findings) {
return findings.reduce((sum, item) => {
sum[item.status] = (sum[item.status] || 0) + 1;
return sum;
}, { PASS: 0, FAIL: 0, WARN: 0, SKIP: 0, INFO: 0 });
}
function renderJunit(ctx, counts) {
const cases = ctx.findings.map((item) => {
const body = item.status === 'FAIL'
? `<failure message="${escapeXml(item.reason)}">${escapeXml(JSON.stringify(item, null, 2))}</failure>`
: item.status === 'SKIP' ? '<skipped/>' : '';
return `<testcase classname="${escapeXml(item.category)}" name="${escapeXml(item.id)}" time="${item.durationMs / 1000}">${body}</testcase>`;
}).join('\n');
return `<?xml version="1.0"?><testsuite name="kvideo-verification" tests="${ctx.findings.length}" failures="${counts.FAIL}" skipped="${counts.SKIP}">${cases}</testsuite>`;
}
function renderMarkdown(ctx, summary) {
const failures = ctx.findings.filter((item) => item.status === 'FAIL');
const warnings = ctx.findings.filter((item) => item.status === 'WARN');
const rows = [...failures, ...warnings].map((item) =>
`| ${item.status} | ${item.severity} | \`${item.id}\` | ${item.title.replaceAll('|', '\\|')} | ${item.reason.replaceAll('|', '\\|')} |`,
);
return `# KVideo verification ${ctx.runId}\n\n` +
`PASS ${summary.counts.PASS} · FAIL ${summary.counts.FAIL} · WARN ${summary.counts.WARN} · SKIP ${summary.counts.SKIP} · INFO ${summary.counts.INFO}\n\n` +
`Duration ${summary.durationMs} ms · success ${summary.success}\n\n` +
`A green run proves only the declared checks. Coverage gaps are explicit findings.\n\n` +
`| Status | Severity | Check | Result | Reason |\n|---|---|---|---|---|\n${rows.join('\n') || '| PASS | info | — | No failures or warnings | — |'}\n`;
}
export function writeReports(ctx) {
const counts = totals(ctx.findings);
const finishedAt = new Date().toISOString();
const durationMs = Date.parse(finishedAt) - Date.parse(ctx.startedAt);
const summary = { runId: ctx.runId, startedAt: ctx.startedAt, finishedAt, durationMs, success: counts.FAIL === 0, counts };
writeJson(path.join(ctx.artifacts, 'findings.json'), ctx.findings);
writeJson(path.join(ctx.artifacts, 'summary.json'), summary);
fs.writeFileSync(path.join(ctx.artifacts, 'junit.xml'), renderJunit(ctx, counts));
fs.writeFileSync(path.join(ctx.artifacts, 'summary.md'), renderMarkdown(ctx, summary));
fs.writeFileSync(path.join(ctx.artifacts, 'report.html'), renderHtml(ctx, summary));
return counts;
}
verification/src/run-regression.mjs
+67
View File
@@ -0,0 +1,67 @@
#!/usr/bin/env node
import fs from 'node:fs';
import path from 'node:path';
import process from 'node:process';
import { spawn } from 'node:child_process';
import { build } from 'esbuild';
import { walk } from './core/files.mjs';
const root = path.resolve(import.meta.dirname, '..', '..');
const verifyDir = path.join(root, 'verification');
const testsDir = path.join(verifyDir, 'tests', 'regression');
function requestedOutput(args) {
const index = args.indexOf('--output-dir');
if (index < 0) return null;
if (!args[index + 1]) throw new Error('--output-dir requires a path');
const target = path.resolve(root, args[index + 1]);
const relative = path.relative(verifyDir, target);
if (!relative || relative.startsWith('..') || path.isAbsolute(relative)) {
throw new Error('Regression output must stay below verification/');
}
return target;
}
function outputFiles(entries, outputDir) {
return entries.map((file) => path.join(outputDir,
path.relative(testsDir, file).replace(/\.ts$/, '.cjs')));
}
async function execute(files) {
const child = spawn(process.execPath, ['--test', ...files], {
cwd: root, env: process.env, stdio: 'inherit',
});
return new Promise((resolve, reject) => {
child.once('error', reject);
child.once('exit', (value, signal) => resolve(signal ? 1 : (value ?? 1)));
});
}
async function run() {
const entries = walk(testsDir, (file) => file.endsWith('.test.ts')).sort();
if (!entries.length) throw new Error('No regression tests found');
const requested = requestedOutput(process.argv.slice(2));
const scratch = path.join(verifyDir, 'tmp');
fs.mkdirSync(scratch, { recursive: true });
const outputDir = requested || fs.mkdtempSync(path.join(scratch, 'regression-'));
fs.mkdirSync(outputDir, { recursive: true });
try {
await build({
absWorkingDir: root, entryPoints: entries, outbase: testsDir, outdir: outputDir,
bundle: true, platform: 'node', format: 'cjs', packages: 'external',
sourcemap: 'inline', sourcesContent: true, outExtension: { '.js': '.cjs' },
logLevel: 'warning', tsconfig: path.join(root, 'tsconfig.json'),
});
return await execute(outputFiles(entries, outputDir));
} finally {
if (!requested) fs.rmSync(outputDir, { recursive: true, force: true });
}
}
try {
process.exitCode = await run();
} catch (error) {
const message = error instanceof Error ? error.stack || error.message : String(error);
process.stderr.write(`${message}\n`);
process.exitCode = 1;
}
verification/tests/harness/action-frontier.test.mjs
+58
View File
@@ -0,0 +1,58 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import { actionExecutionKey, classifyStateTransition, registerNovelState } from '../../src/policy/action-coverage.mjs';
function snapshot(location, signatures) {
return { location, signatures };
}
test('new control signatures enter the semantic action frontier', () => {
const discovered = new Set();
registerNovelState(discovered, snapshot('/', ['theme:light']));
const result = classifyStateTransition(discovered, snapshot('/', ['theme:light', 'search:filled']));
assert.equal(result.queued, true);
assert.deepEqual(result.newStateFacts, ['control:search:filled']);
});
test('signature order and independent combinations do not create duplicate frontier work', () => {
const discovered = new Set();
registerNovelState(discovered, snapshot('/', ['theme:dark', 'search:filled']));
const result = classifyStateTransition(discovered, snapshot('/', ['search:filled', 'theme:dark']));
assert.equal(result.queued, false);
assert.equal(result.subsumed, true);
assert.match(result.reason, /already covered/);
});
test('a new location always enters the frontier even with known controls', () => {
const discovered = new Set();
registerNovelState(discovered, snapshot('/', ['button:save']));
const result = classifyStateTransition(discovered, snapshot('/settings#profile', ['button:save']));
assert.equal(result.queued, true);
assert.deepEqual(result.newStateFacts, ['location:/settings#profile']);
});
test('sortable permutations are verified but do not create a factorial frontier', () => {
const discovered = new Set();
registerNovelState(discovered, snapshot('/', ['sortable-order|main|popular>drama']));
const result = classifyStateTransition(discovered, snapshot('/', ['sortable-order|main|drama>popular']));
assert.equal(result.queued, false);
assert.equal(result.subsumed, true);
});
test('unchanged controls execute once across independent state combinations', () => {
const action = { key: 'main>button:1|button|save|enabled' };
const left = { location: '/', signatures: ['theme:light'] };
const right = { location: '/', signatures: ['theme:dark', 'search:filled'] };
assert.equal(actionExecutionKey(left, action), actionExecutionKey(right, action));
assert.notEqual(actionExecutionKey(left, action), actionExecutionKey({ ...right, location: '/settings' }, action));
assert.notEqual(actionExecutionKey(left, action), actionExecutionKey(left, { key: `${action.key}|pressed` }));
});
test('blocked transitions remain recorded outcomes but are not queued or subsumed', () => {
const discovered = new Set();
const entries = [{ action: 'executed' }];
const result = classifyStateTransition(discovered, snapshot('/', ['button:add']), 'action failed');
assert.equal(entries.length, 1);
assert.deepEqual(result, { eligible: false, queued: false, subsumed: false,
reason: 'action failed', newStateFacts: [] });
});
verification/tests/harness/action.test.mjs
+146
View File
@@ -0,0 +1,146 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import { assessAction } from '../../src/browser/action-effects.mjs';
import { fillInput } from '../../src/browser/action-state.mjs';
import { clickControl, reorderSortable, stateDifference, stateSnapshot } from '../../src/browser/actions.mjs';
import { semanticDifference } from '../../src/browser/semantic.mjs';
import { BROWSER_FIXTURE_ORIGIN } from '../../src/browser/init.mjs';
import { requestFailureBucket } from '../../src/browser/session.mjs';
import { parseAndroidVersion } from '../../src/checks/android-config.mjs';
import { coverageArgs } from '../../src/checks/coverage.mjs';
import { projectSourceFile } from '../../src/checks/source-policy.mjs';
function evidence(overrides = {}) {
return {
url: '/', localStorage: [], sessionStorage: [], clipboard: '', media: [],
dom: { hash: 'dom', items: [] }, actionState: { hash: 'state', signatures: [] },
observed: { requests: 0, responses: 0, dialogs: 0, downloads: 0, popups: 0,
consoleErrors: 0, pageErrors: 0, failedRequests: 0, httpErrors: 0 },
...overrides,
};
}
test('action state hashes ignore origin and locator paths but preserve semantic controls', () => {
const left = stateSnapshot('http://127.0.0.1:1/settings', [{ signature: 'button|save|0' }]);
const right = stateSnapshot('https://example.test/settings', [{ signature: 'button|save|0' }]);
assert.equal(left.hash, right.hash);
assert.deepEqual(stateDifference(left, { ...right, signatures: ['button|cancel|0'] }), {
missing: ['button|save|0'], unexpected: ['button|cancel|0'],
});
});
test('state graphs distinguish one duplicate but bound unbounded identical instances', () => {
const once = stateSnapshot('https://example.test/', [{ signature: 'button|tag|0' }]);
const twice = stateSnapshot('https://example.test/', [{ signature: 'button|tag|0' }, { signature: 'button|tag|1' }]);
const thrice = stateSnapshot('https://example.test/', [
{ signature: 'button|tag|0' }, { signature: 'button|tag|1' }, { signature: 'button|tag|2' },
]);
assert.notEqual(once.hash, twice.hash);
assert.equal(twice.hash, thrice.hash);
});
test('state graphs preserve sortable order for deterministic replay', () => {
const left = stateSnapshot('https://example.test/', [
{ signature: 'tag|popular|0', roleDescription: 'sortable', path: 'main>div:1' },
{ signature: 'tag|drama|0', roleDescription: 'sortable', path: 'main>div:2' },
]);
const right = stateSnapshot('https://example.test/', [
{ signature: 'tag|drama|0', roleDescription: 'sortable', path: 'main>div:1' },
{ signature: 'tag|popular|0', roleDescription: 'sortable', path: 'main>div:2' },
]);
assert.notEqual(left.hash, right.hash);
});
test('silent controls fail while storage changes are independently observable', () => {
const action = { aria: '保存', text: '保存' };
const before = evidence();
const silent = assessAction(action, { ok: true, operation: 'click' }, before, evidence());
assert.equal(silent.ok, false);
assert.equal(silent.failureKind, 'no-effect');
const changed = assessAction(action, { ok: true, operation: 'click' }, before,
evidence({ localStorage: [['saved', 'true']] }));
assert.equal(changed.ok, true);
assert.deepEqual(changed.effects, ['localStorage']);
});
test('selected choices and current-location links are explicit idempotent actions', () => {
const before = evidence({ url: '/settings' });
const selected = assessAction({ selected: true }, { ok: true }, before, before);
assert.equal(selected.idempotent, true);
const link = assessAction({ href: '/settings' }, { ok: true }, before, before);
assert.equal(link.idempotent, true);
});
test('blank-target clicks arm popup capture before clicking and preserve the final URL', async () => {
let armed = false;
let closed = false;
const popup = { waitForLoadState: async () => {}, url: () => 'https://example.test/result',
close: async () => { closed = true; } };
const page = { waitForEvent: async (name) => { assert.equal(name, 'popup'); armed = true; return popup; } };
const locator = { click: async () => { assert.equal(armed, true); } };
const result = await clickControl(page, locator, '_blank');
assert.deepEqual(result, { operation: 'click', popupUrl: 'https://example.test/result' });
assert.equal(closed, true);
});
test('sortable controls skip singleton groups and use the keyboard sensor otherwise', async () => {
const singleton = await reorderSortable({ locator: () => ({ count: async () => 1 }) }, {});
assert.equal(singleton.idempotent, true);
const keys = [];
let order = ['|one', '|two'];
const peers = { count: async () => 2, evaluateAll: async (_callback, targetId) => targetId
? { index: 0, order: [...order] } : [...order] };
const page = { locator: () => peers, keyboard: { press: async (key) => {
keys.push(key); if (key === 'ArrowRight') order = ['|two', '|one'];
} }, waitForTimeout: async () => {} };
const locator = { getAttribute: async () => '7', focus: async () => {} };
const moved = await reorderSortable(page, locator);
assert.equal(moved.operation, 'keyboard-sort');
assert.deepEqual(moved.afterOrder, ['|two', '|one']);
assert.deepEqual(keys, ['Space', 'ArrowRight', 'Space']);
});
test('intentional browser request cancellation is recorded separately from failures', () => {
assert.equal(requestFailureBucket('net::ERR_ABORTED'), 'abortedRequests');
assert.equal(requestFailureBucket('net::ERR_CONNECTION_REFUSED'), 'failedRequests');
});
test('seek controls require directional media proof', () => {
const before = evidence({ media: [{ currentTime: 20, paused: true, muted: true, volume: 1, playbackRate: 1, currentSrc: 'x' }] });
const after = evidence({ media: [{ currentTime: 10, paused: true, muted: true, volume: 1, playbackRate: 1, currentSrc: 'x' }] });
const result = assessAction({ aria: '后退 10 秒', text: '' }, { ok: true }, before, after);
assert.equal(result.ok, true);
assert.equal(result.proof.kind, 'seek-backward');
});
test('URL inputs use the local interceptable import fixture', async () => {
let value = '';
await fillInput({ fill: async (next) => { value = next; } }, 'url');
assert.equal(value, `${BROWSER_FIXTURE_ORIGIN}/source-import.json`);
});
test('filling an input with its existing deterministic value is idempotent', async () => {
const result = await fillInput({ inputValue: async () => '验证', fill: async () => {} }, 'text');
assert.equal(result.idempotent, true);
});
test('semantic deltas expose missing and unexpected visible controls', () => {
assert.deepEqual(semanticDifference({ items: ['a', 'b'] }, { items: ['b', 'c'] }), {
missing: ['a'], unexpected: ['c'],
});
});
test('coverage, Android, and line policies are strict by construction', () => {
const ctx = { config: { verifyDir: '/verify' } };
const args = coverageArgs(ctx, '/reports');
assert.ok(args.includes('--all'));
assert.ok(args.includes('--100'));
assert.ok(args.includes('app'));
assert.ok(args.includes('components'));
assert.ok(args.includes('lib'));
assert.ok(args.includes('scripts'));
assert.deepEqual(parseAndroidVersion('versionCode = 9\nversionName = "4.9.20"'), { versionName: '4.9.20', versionCode: 9 });
assert.equal(projectSourceFile('/repo/app-release.json'), true);
assert.equal(projectSourceFile('/repo/Dockerfile'), true);
assert.equal(projectSourceFile('/repo/package-lock.json'), false);
});
verification/tests/harness/boundary.test.mjs
+29
View File
@@ -0,0 +1,29 @@
import assert from 'node:assert/strict';
import fs from 'node:fs';
import test from 'node:test';
import { verificationChange } from '../../src/checks/preflight.mjs';
test('only verification-folder changes satisfy the publishing boundary', () => {
assert.equal(verificationChange(' M verification/src/main.mjs'), true);
assert.equal(verificationChange('?? verification/history/catalog.json'), true);
assert.equal(verificationChange(' M package.json'), false);
assert.equal(verificationChange(' D tests/example.test.ts'), false);
assert.equal(verificationChange(' M .github/workflows/release.yml'), false);
});
test('renames are judged by their final destination', () => {
assert.equal(verificationChange('R old.test.ts -> verification/tests/regression/old.test.ts'), true);
assert.equal(verificationChange('R verification/old.mjs -> app/old.mjs'), false);
});
test('the complete runner repeats the workspace boundary check after all tools', () => {
const source = fs.readFileSync(new URL('../../src/main.mjs', import.meta.url), 'utf8');
assert.match(source, /checkWorkspaceBoundary\(ctx, 'postflight'\)/);
});
test('Gradle caches are redirected below verification', () => {
const source = fs.readFileSync(new URL('../../src/checks/android.mjs', import.meta.url), 'utf8');
assert.match(source, /ctx\.config\.verifyDir, 'cache', 'gradle'/);
assert.match(source, /--project-cache-dir/);
assert.match(source, /GRADLE_USER_HOME/);
});
verification/tests/harness/browser-evidence.test.mjs
+73
View File
@@ -0,0 +1,73 @@
import assert from 'node:assert/strict';
import fs from 'node:fs';
import test from 'node:test';
import { assessAction, browserEvidence } from '../../src/browser/action-effects.mjs';
import { prepareReplayBaseline, revealActionControls } from '../../src/browser/action-state.mjs';
function evidence(fileChoosers) {
return {
url: '/', localStorage: [], sessionStorage: [], clipboard: '', display: {}, media: [],
dom: { hash: 'dom', items: [] }, actionState: { hash: 'state', signatures: [] },
observed: { requests: 0, responses: 0, dialogs: 0, downloads: 0, fileChoosers,
popups: 0, consoleErrors: 0, pageErrors: 0, failedRequests: 0, httpErrors: 0 },
observedEvents: {},
};
}
test('opening a native file chooser is an independently observable effect', () => {
const result = assessAction({ tag: 'button', text: '选择文件' }, { ok: true }, evidence(0), evidence(1));
assert.equal(result.ok, true);
assert.deepEqual(result.effects, ['fileChoosers']);
});
test('opaque documents preserve evidence instead of crashing on blocked storage', () => {
const denied = () => { const error = new Error('Access is denied'); error.name = 'SecurityError'; throw error; };
const scope = {
document: { querySelectorAll: () => [], body: {},
documentElement: { className: '', getAttribute: () => '' } },
location: { pathname: 'blank', search: '', hash: '' },
getComputedStyle: () => ({ colorScheme: '', color: '', backgroundColor: '' }),
get localStorage() { return denied(); },
get sessionStorage() { return denied(); },
};
const result = browserEvidence(scope);
assert.deepEqual(result.localStorage, [['<unavailable>', 'SecurityError: Access is denied']]);
assert.deepEqual(result.sessionStorage, result.localStorage);
assert.equal(result.url, 'blank');
});
test('cursor-only labels and containers with declared controls are not duplicate actions', () => {
const source = fs.readFileSync(new URL('../../src/browser/action-scan.mjs', import.meta.url), 'utf8');
assert.match(source, /element\.tagName === 'LABEL'/);
assert.match(source, /element\.querySelector\(selector\)/);
});
test('modal layer filtering runs before offscreen controls are accepted', () => {
const source = fs.readFileSync(new URL('../../src/browser/action-scan.mjs', import.meta.url), 'utf8');
const layerGate = source.indexOf('if (blockingZ > layer(element)) return false');
const offscreenGate = source.indexOf('if (outside(box)) return true');
assert.ok(layerGate >= 0);
assert.ok(offscreenGate > layerGate);
});
test('control scans preserve the real hover target while revealing player controls', async () => {
let evaluated = false;
let pointerMoved = false;
const page = {
viewportSize: () => ({ width: 820, height: 1180 }),
evaluate: async () => { evaluated = true; },
mouse: { move: async () => { pointerMoved = true; } },
};
await revealActionControls(page);
assert.equal(evaluated, true);
assert.equal(pointerMoved, false);
});
test('action replay starts from a deterministic paused-media baseline', async () => {
let waited = 0;
let evaluated = false;
const page = { evaluate: async () => { evaluated = true; }, waitForTimeout: async (ms) => { waited = ms; } };
await prepareReplayBaseline(page);
assert.equal(evaluated, true);
assert.equal(waited, 100);
});
verification/tests/harness/core.test.mjs
+115
View File
@@ -0,0 +1,115 @@
import assert from 'node:assert/strict';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import test from 'node:test';
import { actionTransitioned } from '../../src/browser/actions.mjs';
import { stableGeneratedValue } from '../../src/browser/action-normalize.mjs';
import { BROWSER_FIXTURE_ORIGIN, sourceArgument } from '../../src/browser/init.mjs';
import { numericCandidate, prepareActionState } from '../../src/browser/action-state.mjs';
import { getConfig } from '../../src/config.mjs';
import { latestDeployment } from '../../src/checks/deployment.mjs';
import { auditArgs } from '../../src/checks/harness-dependencies.mjs';
import { findDangerousConstructs } from '../../src/checks/security-scan.mjs';
import { runCommand } from '../../src/core/command.mjs';
import { redact, redactText } from '../../src/core/redact.mjs';
import { escapeXml } from '../../src/core/xml.mjs';
test('redacts keyed secrets recursively', () => {
assert.deepEqual(redact({ nested: { password: 'value', safe: 'visible' } }), {
nested: { password: '[REDACTED]', safe: 'visible' },
});
});
test('redacts bearer, GitHub, JWT, and query credentials', () => {
const token = ['gho', '_abcdefghijklmnopqrstuvwxyz123456'].join('');
const raw = `Bearer abc.def ${token} ?token=secret-value`;
const result = redactText(raw);
assert.doesNotMatch(result, /abcdefghijklmnopqrstuvwxyz|secret-value/);
});
test('escapes XML metacharacters', () => {
assert.equal(escapeXml(`<a x="1">Tom & 'Ada'</a>`), '&lt;a x=&quot;1&quot;&gt;Tom &amp; &apos;Ada&apos;&lt;/a&gt;');
});
test('chooses valid alternative values for numeric and range inputs', () => {
assert.equal(numericCandidate({ min: '0', max: '1', value: '0.5', step: '0.01' }), '0');
assert.equal(numericCandidate({ min: '10', max: '100', value: '70', step: '1' }), '55');
});
test('maps play and pause controls to deterministic media preconditions', async () => {
const modes = [];
const page = { evaluate: async (_callback, mode) => modes.push(mode) };
await prepareActionState(page, { aria: '播放' });
await prepareActionState(page, { aria: 'Pause' });
await prepareActionState(page, { aria: '搜索' });
assert.deepEqual(modes, [
{ expected: 'paused', labelText: '播放' },
{ expected: 'playing', labelText: 'pause' },
]);
});
test('accepts explicit full-run action budgets', () => {
const config = getConfig(['node', 'verify', '--root', process.cwd(), '--max-actions', '1234', '--max-action-depth', '7']);
assert.equal(config.maxActionStates, 1234);
assert.equal(config.maxActionDepth, 7);
});
test('GitHub auditing is explicit and disabled by default', () => {
assert.equal(getConfig(['node', 'verify']).auditGithub, false);
assert.equal(getConfig(['node', 'verify', '--audit-github']).auditGithub, true);
});
test('verification dependency audit fails only at the declared severe threshold', () => {
assert.deepEqual(auditArgs(), [
'audit', '--omit=dev', '--audit-level=high', '--json',
]);
});
test('skipped controls cannot create recursive action states', () => {
assert.equal(actionTransitioned({ ok: true, skipped: true }, 'before', 'after'), false);
assert.equal(actionTransitioned({ ok: true, idempotent: true }, 'before', 'after'), false);
assert.equal(actionTransitioned({ ok: true }, 'before', 'after'), true);
});
test('generated source IDs do not create false action states', () => {
assert.equal(stableGeneratedValue('source-id', 'custom-msa9oem1'), 'source-id:<generated>');
assert.equal(stableGeneratedValue('source-id', 'user-chosen-id'), 'user-chosen-id');
assert.equal(stableGeneratedValue('other-id', 'custom-msa9oem1'), 'custom-msa9oem1');
});
test('browser fixtures use an interceptable HTTPS origin', () => {
assert.equal(new URL(BROWSER_FIXTURE_ORIGIN).protocol, 'https:');
assert.equal(sourceArgument(BROWSER_FIXTURE_ORIGIN).sourceConfig.baseUrl, BROWSER_FIXTURE_ORIGIN);
});
test('selects the newest Cloudflare production deployment', () => {
const output = JSON.stringify([
{ Environment: 'Production', Source: 'abcdef1', Deployment: 'https://new.pages.dev' },
{ Environment: 'Production', Source: '1234567', Deployment: 'https://old.pages.dev' },
]);
assert.equal(latestDeployment(output)?.Source, 'abcdef1');
assert.equal(latestDeployment('not json'), null);
});
test('dangerous construct scan ignores matcher text and finds runtime use', () => {
const scanner = "const name = 'dangerouslySetInnerHTML'; const matcher = /eval\\s*\\(/;";
assert.deepEqual(findDangerousConstructs('scanner.mjs', scanner), []);
const runtime = "export const View = () => <div dangerouslySetInnerHTML={{ __html: 'x' }} />; eval('x');";
assert.deepEqual(findDangerousConstructs('view.tsx', runtime), [
{ file: 'view.tsx', construct: 'dangerouslySetInnerHTML' },
{ file: 'view.tsx', construct: 'eval' },
]);
});
test('command capture preserves UTF-8 split across process chunks', async () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'kvideo-command-'));
const ctx = { config: { root, commandTimeoutMs: 10_000 }, dirs: { raw: root } };
const script = "const b=Buffer.from('识');process.stdout.write(b.subarray(0,1));setTimeout(()=>process.stdout.write(b.subarray(1)),10)";
try {
const result = await runCommand(ctx, 'utf8', process.execPath, ['-e', script]);
assert.equal(result.code, 0);
assert.equal(fs.readFileSync(result.outputPath, 'utf8'), '识');
assert.equal(result.tail, '识');
} finally { fs.rmSync(root, { recursive: true, force: true }); }
});
verification/tests/harness/docker-context.test.mjs
+33
View File
@@ -0,0 +1,33 @@
import assert from 'node:assert/strict';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import test from 'node:test';
import { createDockerContext, includeDockerContextPath, removeDockerContext } from '../../src/checks/docker-context.mjs';
test('Docker context contains application inputs but excludes verifier state', () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'kvideo-docker-context-'));
const verifyDir = path.join(root, 'verification');
fs.mkdirSync(path.join(root, 'src'), { recursive: true });
fs.mkdirSync(path.join(verifyDir, 'artifacts'), { recursive: true });
fs.mkdirSync(path.join(root, 'node_modules'), { recursive: true });
fs.writeFileSync(path.join(root, 'Dockerfile'), 'FROM scratch\n');
fs.writeFileSync(path.join(root, 'src', 'app.js'), 'export {};\n');
fs.writeFileSync(path.join(verifyDir, 'artifacts', 'large.bin'), 'excluded');
fs.writeFileSync(path.join(root, 'node_modules', 'dependency.js'), 'excluded');
const context = createDockerContext({ config: { root, verifyDir }, runId: 'unit' });
assert.equal(fs.existsSync(path.join(context, 'Dockerfile')), true);
assert.equal(fs.existsSync(path.join(context, 'src', 'app.js')), true);
assert.equal(fs.existsSync(path.join(context, 'verification')), false);
assert.equal(fs.existsSync(path.join(context, 'node_modules')), false);
removeDockerContext(context);
assert.equal(fs.existsSync(context), false);
fs.rmSync(root, { recursive: true, force: true });
});
test('Docker context rejects paths outside the root', () => {
const root = path.join(os.tmpdir(), 'kvideo-root');
assert.equal(includeDockerContextPath(root, path.join(root, 'src', 'app.js')), true);
assert.equal(includeDockerContextPath(root, path.join(root, 'verification', 'run')), false);
assert.equal(includeDockerContextPath(root, path.join(root, '..', 'secret')), false);
});
verification/tests/harness/files.test.mjs
+33
View File
@@ -0,0 +1,33 @@
import assert from 'node:assert/strict';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import test from 'node:test';
import { runCommand } from '../../src/core/command.mjs';
import { lineCount, readJson, relative, walk, writeJson } from '../../src/core/files.mjs';
test('file helpers inventory and serialize deterministic fixtures', () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'kvideo-verify-'));
fs.mkdirSync(path.join(root, 'nested'));
fs.mkdirSync(path.join(root, 'build'));
fs.writeFileSync(path.join(root, 'nested', 'file.txt'), 'one\ntwo\n');
fs.writeFileSync(path.join(root, 'build', 'generated.txt'), 'generated');
const json = path.join(root, 'result.json');
writeJson(json, { ok: true });
assert.equal(lineCount(path.join(root, 'nested', 'file.txt')), 3);
assert.equal(relative(root, path.join(root, 'nested', 'file.txt')), 'nested/file.txt');
assert.deepEqual(readJson(json), { ok: true });
assert.equal(walk(root, (file) => file.endsWith('.txt')).length, 1);
fs.rmSync(root, { recursive: true, force: true });
});
test('command logs are flushed before command completion resolves', async () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'kvideo-command-'));
const raw = path.join(root, 'raw');
fs.mkdirSync(raw);
const ctx = { config: { root, commandTimeoutMs: 5000 }, dirs: { raw } };
const result = await runCommand(ctx, 'flush', process.execPath, ['-e', "process.stdout.write('x'.repeat(50000))"]);
assert.equal(result.code, 0);
assert.equal(fs.readFileSync(result.outputPath, 'utf8').length, 50000);
fs.rmSync(root, { recursive: true, force: true });
});
verification/tests/harness/history.test.mjs
+69
View File
@@ -0,0 +1,69 @@
import assert from 'node:assert/strict';
import test from 'node:test';
import { classifyItem, uncatalogedRecords } from '../../src/history/catalog.mjs';
import { graphTruncation } from '../../src/history/github.mjs';
import { normalizeHistory } from '../../src/history/normalize.mjs';
import { localCoverage } from '../../src/history/local.mjs';
const catalog = {
issues: [1, 2], pullRequests: [3], unavailablePullRequests: [4],
regressionIssueOverrides: [2], unverifiableIssues: [{ number: 1, reason: 'missing data' }],
evidence: { general: ['verification/src/checks/static-tools.mjs'] },
};
test('classification honors explicit unverifiable and regression decisions', () => {
const unavailable = classifyItem({ number: 1, title: '[Bug]:', labels: [{ name: 'bug' }] }, catalog);
const override = classifyItem({ number: 2, title: 'Safari failure', labels: [{ name: 'enhancement' }] }, catalog);
assert.equal(unavailable.regressionRequired, false);
assert.equal(unavailable.unverifiableReason, 'missing data');
assert.equal(override.regressionRequired, true);
});
function remote(order = false) {
const issues = [{ number: 2, state: 'closed', title: 'b', body: '', labels: [], created_at: 'a', updated_at: 'b' }, { number: 1, state: 'closed', title: 'a', body: '', labels: [], created_at: 'a', updated_at: 'b' }];
return {
issues: order ? issues : [...issues].reverse(),
pullRequests: [{ number: 3, state: 'closed', title: 'p', body: '', base: { ref: 'main' }, head: { ref: 'x', sha: 'h' } }],
conversationComments: [],
reviewComments: [{ id: 4, node_id: 'comment', pull_request_url: 'https://api.github.test/pulls/4', body: 'risk', path: 'x', created_at: 'a', updated_at: 'b' }],
reviewGraph: [{ number: 3, reviews: { nodes: [] }, reviewThreads: { nodes: [] } }],
};
}
test('normalization is stable and retains unavailable-PR comments', () => {
const first = normalizeHistory(remote(false), catalog);
const second = normalizeHistory(remote(true), catalog);
assert.deepEqual(first.digests, second.digests);
assert.equal(first.data.reviewComments[0].number, 4);
});
test('review pagination mismatch is explicit', () => {
const rows = graphTruncation({
reviewGraphTotal: 2,
reviewGraph: [{ number: 3, reviews: { totalCount: 1, nodes: [] }, reviewThreads: { totalCount: 0, nodes: [] } }],
});
assert.deepEqual(rows.map((item) => item.scope), ['pullRequests', 'PR 3 reviews']);
});
test('records beyond cutoffs fail closed while the active PR is excluded', () => {
const newer = uncatalogedRecords({
issues: [{ number: 2 }, { number: 5 }],
pullRequests: [{ number: 3 }, { number: 6 }, { number: 7 }],
}, { issueCutoff: 2, pullRequestCutoff: 3 }, 7);
assert.deepEqual(newer, { issues: [5], pullRequests: [6] });
});
test('local coverage rejects missing and non-executed evidence', () => {
const trace = {
issues: new Map([[1, []], [2, ['verification/src/checks/dead.mjs']]]),
pullRequests: new Map([[3, ['verification/tests/regression/example.test.ts']]]),
unknown: [{ kind: 'issue', number: 99 }],
};
const coverage = localCoverage(process.cwd(), {
regressionIssues: [1, 2], mergedPullRequests: [3],
}, trace);
assert.deepEqual(coverage.missing.map((item) => item.number), [1]);
assert.deepEqual(coverage.nonExecutable, [{ number: 2, file: 'verification/src/checks/dead.mjs' },
{ number: 3, file: 'verification/tests/regression/example.test.ts' }]);
assert.equal(coverage.unknown.length, 1);
});
verification/tests/harness/http.test.mjs
+33
View File
@@ -0,0 +1,33 @@
import assert from 'node:assert/strict';
import http from 'node:http';
import test from 'node:test';
import { jsonBody, request } from '../../src/core/http.mjs';
test('HTTP evidence captures status, headers, body, and timing', async () => {
const server = http.createServer((_request, response) => {
response.writeHead(201, { 'content-type': 'application/json', 'x-test': 'yes' });
response.end('{"ok":true}');
});
await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve));
const address = server.address();
const result = await request(`http://127.0.0.1:${address.port}`);
assert.equal(result.status, 201);
assert.equal(result.headers['x-test'], 'yes');
assert.match(result.sha256, /^[a-f0-9]{64}$/);
assert.deepEqual(jsonBody(result), { ok: true });
await new Promise((resolve) => server.close(resolve));
});
test('binary HTTP evidence records a digest without corrupt text', async () => {
const server = http.createServer((_request, response) => {
response.writeHead(200, { 'content-type': 'application/octet-stream' });
response.end(Buffer.from([0xff, 0x00, 0x7f]));
});
await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve));
const address = server.address();
const result = await request(`http://127.0.0.1:${address.port}`);
assert.equal(result.bytes, 3);
assert.equal(result.body, '');
assert.match(result.sha256, /^[a-f0-9]{64}$/);
await new Promise((resolve) => server.close(resolve));
});

Some files were not shown because too many files have changed in this diff Show More