mirror of
https://github.com/KuekHaoYang/KVideo.git
synced 2026-08-13 07:43:43 +08:00
Merge pull request #49 from Troray/feature/add-ad-filter
新增M3U8启发式广告检测算法并支持动态关键词配置
This commit is contained in:
@@ -0,0 +1,118 @@
|
||||
# 🛡️ 广告过滤功能使用指南
|
||||
|
||||
KVideo 新增了强大的 M3U8 广告过滤系统,支持多种过滤模式,可有效去除视频中插入的切片广告。
|
||||
|
||||
## ✨ 功能特性
|
||||
|
||||
- **多模式选择**:支持关闭、关键词过滤、智能启发式过滤(Beta)和激进模式。
|
||||
- **UI 集成**:在播放器设置菜单中直接切换模式,实时生效。
|
||||
- **自定义关键词**:支持通过环境变量扩展过滤关键词。
|
||||
- **高性能**:基于流式处理,对播放加载速度几乎无影响。
|
||||
|
||||
## 🎮 使用说明
|
||||
|
||||
1. 打开任意视频播放。
|
||||
|
||||
2. 点击播放器左上角的 **(···)** 按钮。
|
||||
|
||||
3. 在 **广告过滤** 下拉菜单中选择模式:
|
||||
- **关闭**:不过滤任何内容(默认)。
|
||||
- **关键词**:仅过滤 URL 中包含特定广告关键词(如 `adjump`, `pre-roll`)的片段。安全稳定,误杀率极低。
|
||||
- **智能(Beta)**:推荐使用。结合关键词、文件名特征、时间戳跳变标记 (DISCONTINUITY) 和 HLS 标签进行综合评分。能识别大多数无明显关键词的广告。
|
||||
- **激进**:降低判定阈值,过滤更多疑似广告,但可能会误伤正片。仅在智能模式无效时尝试。
|
||||
|
||||
> ⚠️重要提示:添加关键词时,请勿使用类似`ad` `ads` `video` `20260118` 这类简短相对通用的词汇,会造成误杀率极高。
|
||||
|
||||
---
|
||||
|
||||
## 🛠️ 部署配置指南
|
||||
|
||||
你可以通过环境变量 `NEXT_PUBLIC_AD_KEYWORDS` 自定义额外的广告关键词。
|
||||
|
||||
### 1. 本地开发 (Local Development)
|
||||
|
||||
在项目根目录的 `.env.local` 文件中添加。支持多行格式(推荐使用引号包裹):
|
||||
|
||||
```env
|
||||
# 单行模式
|
||||
NEXT_PUBLIC_AD_KEYWORDS=spam,promo,intro-ad
|
||||
|
||||
# 多行模式 (更清晰)
|
||||
NEXT_PUBLIC_AD_KEYWORDS="
|
||||
spam
|
||||
promo
|
||||
intro-ad
|
||||
/unwanted-segment/
|
||||
"
|
||||
```
|
||||
|
||||
### 2. Docker 部署 (运行时热加载)
|
||||
|
||||
我们支持三种方式注入关键词,优先级从高到低:
|
||||
|
||||
1. **文件加载 (推荐)**:通过挂载文件并在运行时读取。
|
||||
2. **运行时环境变量**:直接通过 `-e` 传入 `AD_KEYWORDS`。
|
||||
3. **构建时环境变量**:构建镜像时打入的 `NEXT_PUBLIC_AD_KEYWORDS`。
|
||||
|
||||
#### 方案 A:挂载配置文件 (支持运行时修改)
|
||||
|
||||
创建一个关键词文件 `ad_keywords.txt`:
|
||||
```text
|
||||
spam
|
||||
promo
|
||||
/ad-segment/
|
||||
intro
|
||||
```
|
||||
|
||||
运行 Docker 时挂载该文件并设置 `AD_KEYWORDS_FILE` 环境变量:
|
||||
|
||||
```bash
|
||||
docker run -d -p 3000:3000 \
|
||||
-v $(pwd)/ad_keywords.txt:/app/ad_keywords.txt \
|
||||
-e AD_KEYWORDS_FILE=/app/ad_keywords.txt \
|
||||
--name kvideo kuekhaoyang/kvideo:latest
|
||||
```
|
||||
*注:修改宿主机文件后重启容器即可生效,无需重新构建镜像。*
|
||||
|
||||
#### 方案 B:直接注入环境变量
|
||||
|
||||
```bash
|
||||
docker run -d -p 3000:3000 \
|
||||
-e AD_KEYWORDS="spam,promo,intro-ad" \
|
||||
--name kvideo kuekhaoyang/kvideo:latest
|
||||
```
|
||||
|
||||
### 3. Vercel 部署
|
||||
|
||||
1. 进入 Vercel 项目 Dashboard。
|
||||
2. 点击 **Settings** -> **Environment Variables**。
|
||||
3. 添加新变量:
|
||||
- **Key**: `NEXT_PUBLIC_AD_KEYWORDS`
|
||||
- **Value**: `spam,promo,intro-ad`
|
||||
4. 重新部署项目以生效。
|
||||
|
||||
### 4. Cloudflare Pages 部署
|
||||
|
||||
1. 进入 Cloudflare Pages 项目 Dashboard。
|
||||
2. 点击 **Settings** -> **Environment variables**。
|
||||
3. 添加变量:
|
||||
- **Variable name**: `NEXT_PUBLIC_AD_KEYWORDS`
|
||||
- **Value**: `spam,promo,intro-ad`
|
||||
4. 重新部署项目 (Retry deployment) 以生效。
|
||||
|
||||
---
|
||||
|
||||
## 🔍 技术实现细节
|
||||
|
||||
广告检测逻辑位于 `lib/utils/m3u8-ad-detector.ts`,采用分块评分机制:
|
||||
|
||||
1. **Block 解析**:将 M3U8 播放列表按 `#EXT-X-DISCONTINUITY` 分割成多个块。
|
||||
2. **特征学习**:分析最长的块(通常是正片)提取特征,如文件命名模式、路径前缀等。
|
||||
3. **评分系统**:
|
||||
- **HLS 标签**:`#EXT-X-CUE-OUT`/`IN` (+10分,确认为广告)
|
||||
- **路径前缀**:与正片路径不一致 (+5分,高度疑似)
|
||||
- **关键词**:URL 包含已知广告词 (+2.5分)
|
||||
- **文件名模式**:与正片命名规则不符 (+1.5分)
|
||||
4. **决策过滤**:
|
||||
- **智能模式**:移除评分 >= 5.0 的块。
|
||||
- **激进模式**:移除评分 >= 3.0 的块。
|
||||
@@ -83,6 +83,11 @@
|
||||
- **内容隔离**:高级内容与普通内容完全物理隔离,互不干扰
|
||||
- **专属设置**:拥有独立的内容源管理和功能设置
|
||||
|
||||
### 🛡️ 广告过滤
|
||||
- **多模式选择**:支持关闭、关键词过滤、智能启发式过滤(Beta)和激进模式。
|
||||
- **UI 集成**:在播放器设置菜单中直接切换模式,实时生效。
|
||||
- **自定义关键词**:支持通过环境变量扩展过滤关键词。
|
||||
- **高性能**:基于流式处理,对播放加载速度几乎无影响。
|
||||
|
||||
## 🔐 隐私保护
|
||||
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import React from 'react';
|
||||
import type { Metadata } from "next";
|
||||
import { Geist, Geist_Mono } from "next/font/google";
|
||||
import "./globals.css";
|
||||
@@ -6,7 +7,48 @@ import { Analytics } from "@vercel/analytics/react";
|
||||
import { ServiceWorkerRegister } from "@/components/ServiceWorkerRegister";
|
||||
import { PasswordGate } from "@/components/PasswordGate";
|
||||
import { siteConfig } from "@/lib/config/site-config";
|
||||
import { AdKeywordsInjector } from "@/components/AdKeywordsInjector";
|
||||
import fs from 'fs';
|
||||
import path from 'path';
|
||||
|
||||
// Server Component specifically for reading env/file (async for best practices)
|
||||
async function AdKeywordsWrapper() {
|
||||
let keywords: string[] = [];
|
||||
|
||||
try {
|
||||
// 1. Try reading from file (Docker runtime support)
|
||||
const keywordsFile = process.env.AD_KEYWORDS_FILE;
|
||||
if (keywordsFile) {
|
||||
// Resolve absolute path or relative to CWD
|
||||
const filePath = path.isAbsolute(keywordsFile)
|
||||
? keywordsFile
|
||||
: path.join(process.cwd(), keywordsFile);
|
||||
|
||||
try {
|
||||
const content = await fs.promises.readFile(filePath, 'utf-8');
|
||||
keywords = content.split(/[\n,]/).map((k: string) => k.trim()).filter((k: string) => k);
|
||||
console.log(`[AdFilter] Loaded ${keywords.length} keywords from file: ${filePath}`);
|
||||
} catch (fileError: unknown) {
|
||||
// Handle file not found (ENOENT) gracefully
|
||||
if ((fileError as NodeJS.ErrnoException).code !== 'ENOENT') {
|
||||
console.warn('[AdFilter] Error reading keywords file:', fileError);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 2. Fallback to Env var (Runtime or Build time)
|
||||
if (keywords.length === 0) {
|
||||
const envKeywords = process.env.AD_KEYWORDS || process.env.NEXT_PUBLIC_AD_KEYWORDS;
|
||||
if (envKeywords) {
|
||||
keywords = envKeywords.split(/[\n,]/).map((k: string) => k.trim()).filter((k: string) => k);
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
console.warn('[AdFilter] Failed to load keywords:', error);
|
||||
}
|
||||
|
||||
return <AdKeywordsInjector keywords={keywords} />;
|
||||
}
|
||||
|
||||
const geistSans = Geist({
|
||||
variable: "--font-geist-sans",
|
||||
@@ -39,6 +81,7 @@ export default function RootLayout({
|
||||
>
|
||||
<ThemeProvider>
|
||||
<PasswordGate hasEnvPassword={!!process.env.ACCESS_PASSWORD}>
|
||||
<AdKeywordsWrapper />
|
||||
{children}
|
||||
</PasswordGate>
|
||||
<Analytics />
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
'use client';
|
||||
|
||||
import { useEffect, useRef } from 'react';
|
||||
import { usePlayerSettings } from '@/components/player/hooks/usePlayerSettings';
|
||||
|
||||
interface AdKeywordsInjectorProps {
|
||||
keywords: string[];
|
||||
}
|
||||
|
||||
export function AdKeywordsInjector({ keywords }: AdKeywordsInjectorProps) {
|
||||
const { setAdKeywords } = usePlayerSettings();
|
||||
const initialized = useRef(false);
|
||||
|
||||
useEffect(() => {
|
||||
if (!initialized.current && keywords.length > 0) {
|
||||
setAdKeywords(keywords);
|
||||
initialized.current = true;
|
||||
}
|
||||
}, [keywords, setAdKeywords]);
|
||||
|
||||
return null;
|
||||
}
|
||||
@@ -3,6 +3,7 @@
|
||||
import React from 'react';
|
||||
import { Icons } from '@/components/ui/Icon';
|
||||
import { usePlayerSettings } from '../hooks/usePlayerSettings';
|
||||
import { settingsStore, AdFilterMode } from '@/lib/store/settings-store';
|
||||
|
||||
import { createPortal } from 'react-dom';
|
||||
|
||||
@@ -32,16 +33,28 @@ export function DesktopMoreMenu({
|
||||
autoSkipOutro,
|
||||
skipOutroSeconds,
|
||||
showModeIndicator,
|
||||
adFilter,
|
||||
setAutoNextEpisode,
|
||||
setAutoSkipIntro,
|
||||
setSkipIntroSeconds,
|
||||
setAutoSkipOutro,
|
||||
setSkipOutroSeconds,
|
||||
setShowModeIndicator,
|
||||
setAdFilter,
|
||||
adFilterMode,
|
||||
setAdFilterMode,
|
||||
} = usePlayerSettings();
|
||||
|
||||
const buttonRef = React.useRef<HTMLButtonElement>(null);
|
||||
const [menuPosition, setMenuPosition] = React.useState({ top: 0, left: 0 });
|
||||
const [isAdFilterOpen, setAdFilterOpen] = React.useState(false);
|
||||
|
||||
const AD_FILTER_LABELS: Record<string, string> = {
|
||||
off: '关闭',
|
||||
keyword: '关键词',
|
||||
heuristic: '智能(Beta)',
|
||||
aggressive: '激进'
|
||||
};
|
||||
|
||||
React.useEffect(() => {
|
||||
if (showMoreMenu && buttonRef.current && containerRef.current) {
|
||||
@@ -130,6 +143,46 @@ export function DesktopMoreMenu({
|
||||
</button>
|
||||
</div>
|
||||
|
||||
{/* Ad Filter Mode Selector */}
|
||||
<div className="px-4 py-2.5 flex items-center justify-between">
|
||||
<div className="flex items-center gap-3 text-sm text-[var(--text-color)]">
|
||||
<Icons.ShieldAlert size={18} />
|
||||
<span>广告过滤</span>
|
||||
</div>
|
||||
{/* Custom Ad Filter Mode Selector */}
|
||||
<div className="relative">
|
||||
<button
|
||||
onClick={() => setAdFilterOpen(!isAdFilterOpen)}
|
||||
className="flex items-center gap-1.5 bg-[var(--glass-bg)] border border-[var(--glass-border)] text-[var(--text-color)] text-xs rounded-md px-2.5 py-1.5 outline-none hover:border-[var(--accent-color)] hover:bg-[color-mix(in_srgb,var(--accent-color)_5%,transparent)] transition-all cursor-pointer"
|
||||
>
|
||||
<span>{AD_FILTER_LABELS[adFilterMode] || '关闭'}</span>
|
||||
<Icons.ChevronDown size={14} className={`text-[var(--text-color-secondary)] transition-transform duration-200 ${isAdFilterOpen ? 'rotate-180' : ''}`} />
|
||||
</button>
|
||||
|
||||
{isAdFilterOpen && (
|
||||
<>
|
||||
<div className="fixed inset-0 z-10 cursor-default" onClick={() => setAdFilterOpen(false)} />
|
||||
<div className="absolute right-0 top-full mt-1 w-28 bg-[var(--glass-bg)] backdrop-blur-xl border border-[var(--glass-border)] rounded-lg shadow-xl overflow-hidden z-20 flex flex-col animate-in fade-in zoom-in-95 duration-150">
|
||||
{Object.entries(AD_FILTER_LABELS).map(([mode, label]) => (
|
||||
<button
|
||||
key={mode}
|
||||
onClick={() => {
|
||||
setAdFilterMode(mode as AdFilterMode);
|
||||
setAdFilterOpen(false);
|
||||
}}
|
||||
className={`text-left text-xs px-3 py-2.5 hover:bg-[color-mix(in_srgb,var(--accent-color)_15%,transparent)] transition-colors w-full flex items-center justify-between group ${adFilterMode === mode ? 'text-[var(--accent-color)] font-medium bg-[color-mix(in_srgb,var(--accent-color)_5%,transparent)]' : 'text-[var(--text-color)]'
|
||||
}`}
|
||||
>
|
||||
<span>{label}</span>
|
||||
{adFilterMode === mode && <Icons.Check size={12} className="text-[var(--accent-color)]" />}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Auto Next Episode Switch */}
|
||||
<div className="px-4 py-2.5 flex items-center justify-between">
|
||||
<div className="flex items-center gap-3 text-sm text-[var(--text-color)]">
|
||||
@@ -225,7 +278,7 @@ export function DesktopMoreMenu({
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
</div >
|
||||
);
|
||||
|
||||
return (
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
import { useEffect, useRef } from 'react';
|
||||
import Hls from 'hls.js';
|
||||
import { usePlayerSettings } from './usePlayerSettings';
|
||||
import { filterM3u8Ad } from '@/lib/utils/m3u8-utils';
|
||||
|
||||
interface UseHlsPlayerProps {
|
||||
videoRef: React.RefObject<HTMLVideoElement | null>;
|
||||
@@ -17,6 +19,8 @@ export function useHlsPlayer({
|
||||
onError
|
||||
}: UseHlsPlayerProps) {
|
||||
const hlsRef = useRef<Hls | null>(null);
|
||||
const { adFilterMode, adKeywords } = usePlayerSettings();
|
||||
const isAdFilterEnabled = adFilterMode !== 'off';
|
||||
|
||||
useEffect(() => {
|
||||
const video = videoRef.current;
|
||||
@@ -29,45 +33,66 @@ export function useHlsPlayer({
|
||||
}
|
||||
|
||||
let hls: Hls | null = null;
|
||||
let extraBlobs: string[] = [];
|
||||
|
||||
// Check if HLS is supported natively (Safari, Mobile Chrome)
|
||||
// We prefer native playback if available as it's usually more battery efficient
|
||||
const isNativeHlsSupported = video.canPlayType('application/vnd.apple.mpegurl');
|
||||
|
||||
if (Hls.isSupported()) {
|
||||
// Use hls.js for browsers without native support (Desktop Chrome, Firefox, Edge)
|
||||
// OR if we want to force hls.js for better control (optional, but sticking to native first is safer)
|
||||
|
||||
// Note: Some desktop browsers (like Safari) support native HLS.
|
||||
// We usually prefer native, BUT sometimes native implementation is buggy or lacks features.
|
||||
// For now, we follow the standard pattern: Native first, then HLS.js.
|
||||
// EXCEPT for Chrome on Desktop which reports canPlayType as '' (false).
|
||||
// Define custom loader class to intercept manifest loading
|
||||
// We use 'any' cast because default loader type might not be strictly exposed in all typings
|
||||
const DefaultLoader = (Hls as any).DefaultConfig.loader;
|
||||
|
||||
if (!isNativeHlsSupported) {
|
||||
hls = new Hls({
|
||||
class AdFilterLoader extends DefaultLoader {
|
||||
load(context: any, config: any, callbacks: any) {
|
||||
if (isAdFilterEnabled && (context.type === 'manifest' || context.type === 'level')) {
|
||||
const originalOnSuccess = callbacks.onSuccess;
|
||||
callbacks.onSuccess = (response: any, stats: any, context: any, networkDetails: any) => {
|
||||
if (typeof response.data === 'string') {
|
||||
try {
|
||||
// Filter the content
|
||||
response.data = filterM3u8Ad(response.data, context.url, adFilterMode, adKeywords);
|
||||
} catch (e) {
|
||||
console.warn('[HLS] Ad filter error:', e);
|
||||
}
|
||||
}
|
||||
originalOnSuccess(response, stats, context, networkDetails);
|
||||
};
|
||||
}
|
||||
super.load(context, config, callbacks);
|
||||
}
|
||||
}
|
||||
|
||||
if (!isNativeHlsSupported || isAdFilterEnabled) {
|
||||
// If ad filtering is on, we force Hls.js even on native-supported desktop browsers
|
||||
// Exceptions might exist for iOS where MSE is strictly not available, check Hls.isSupported() result carefully.
|
||||
// Hls.isSupported() is false on iOS Safari usually, so this block won't run there.
|
||||
|
||||
const config: any = {
|
||||
// Worker & Performance
|
||||
enableWorker: true,
|
||||
lowLatencyMode: false, // Disable low latency for more stable playback
|
||||
lowLatencyMode: false,
|
||||
|
||||
// Buffer Settings - More aggressive buffering for smoother playback
|
||||
maxBufferLength: 60, // Buffer up to 60 seconds ahead
|
||||
maxMaxBufferLength: 120, // Allow up to 2 minutes of buffer
|
||||
maxBufferSize: 60 * 1000 * 1000, // 60MB buffer size
|
||||
maxBufferHole: 0.5, // Allow small gaps in buffer
|
||||
// Buffer Settings
|
||||
maxBufferLength: 60,
|
||||
maxMaxBufferLength: 120,
|
||||
maxBufferSize: 60 * 1000 * 1000,
|
||||
maxBufferHole: 0.5,
|
||||
|
||||
// Start with more buffer before playing
|
||||
startFragPrefetch: true, // Enable prefetching next fragment
|
||||
// Start with more buffer
|
||||
startFragPrefetch: true,
|
||||
|
||||
// ABR (Adaptive Bitrate) Settings - Be more conservative
|
||||
abrEwmaDefaultEstimate: 500000, // Start with conservative bandwidth estimate (500kbps)
|
||||
abrEwmaFastLive: 3, // Fast adaptation for live
|
||||
abrEwmaSlowLive: 9, // Slow adaptation for live
|
||||
abrEwmaFastVoD: 3, // Fast adaptation for VoD
|
||||
abrEwmaSlowVoD: 9, // Slow adaptation for VoD
|
||||
abrBandWidthFactor: 0.8, // Use 80% of estimated bandwidth (conservative)
|
||||
abrBandWidthUpFactor: 0.7, // Even more conservative when switching up
|
||||
// ABR Settings
|
||||
abrEwmaDefaultEstimate: 500000,
|
||||
abrEwmaFastLive: 3,
|
||||
abrEwmaSlowLive: 9,
|
||||
abrEwmaFastVoD: 3,
|
||||
abrEwmaSlowVoD: 9,
|
||||
abrBandWidthFactor: 0.8,
|
||||
abrBandWidthUpFactor: 0.7,
|
||||
|
||||
// Loading Settings - More retries and longer timeouts
|
||||
// Loading Settings
|
||||
fragLoadingMaxRetry: 6,
|
||||
fragLoadingRetryDelay: 1000,
|
||||
fragLoadingMaxRetryTimeout: 64000,
|
||||
@@ -79,29 +104,35 @@ export function useHlsPlayer({
|
||||
levelLoadingMaxRetryTimeout: 64000,
|
||||
|
||||
// Timeouts
|
||||
fragLoadingTimeOut: 20000, // 20 seconds for fragment loading
|
||||
manifestLoadingTimeOut: 10000, // 10 seconds for manifest
|
||||
levelLoadingTimeOut: 10000, // 10 seconds for level
|
||||
fragLoadingTimeOut: 20000,
|
||||
manifestLoadingTimeOut: 10000,
|
||||
levelLoadingTimeOut: 10000,
|
||||
|
||||
// Backbuffer - Keep some played content for seeking back
|
||||
backBufferLength: 30, // Keep 30 seconds of played content
|
||||
});
|
||||
// Backbuffer
|
||||
backBufferLength: 30,
|
||||
};
|
||||
|
||||
// Use custom loader if ad filtering is enabled
|
||||
if (isAdFilterEnabled) {
|
||||
config.loader = AdFilterLoader;
|
||||
}
|
||||
|
||||
hls = new Hls(config);
|
||||
hlsRef.current = hls;
|
||||
|
||||
hls.loadSource(src);
|
||||
hls.attachMedia(video);
|
||||
|
||||
// Auto Play Handler
|
||||
hls.on(Hls.Events.FRAG_LOADED, (event, data) => {
|
||||
// Force play if we have the first segment and it's not playing yet
|
||||
// detailed: data.frag.sn is the sequence number
|
||||
if (autoPlay && video.paused && data.frag.start === 0) {
|
||||
video.play().catch(console.warn);
|
||||
}
|
||||
});
|
||||
|
||||
// Manifest Parsed Handler
|
||||
hls.on(Hls.Events.MANIFEST_PARSED, () => {
|
||||
|
||||
// Check for HEVC/H.265 codec (limited browser support)
|
||||
// Check for HEVC
|
||||
if (hls) {
|
||||
const levels = hls.levels;
|
||||
if (levels && levels.length > 0) {
|
||||
@@ -110,10 +141,7 @@ export function useHlsPlayer({
|
||||
level.videoCodec?.toLowerCase().includes('h265')
|
||||
);
|
||||
if (hasHEVC) {
|
||||
console.warn('[HLS] ⚠️ HEVC/H.265 codec detected - may not play in all browsers');
|
||||
console.warn('[HLS] Supported: Safari with hardware acceleration, some Edge versions');
|
||||
console.warn('[HLS] Not supported: Most Chrome/Firefox versions');
|
||||
// Notify parent about potential codec issues
|
||||
console.warn('[HLS] ⚠️ HEVC detected');
|
||||
onError?.('检测到 HEVC/H.265 编码,当前浏览器可能不支持');
|
||||
}
|
||||
}
|
||||
@@ -121,12 +149,13 @@ export function useHlsPlayer({
|
||||
|
||||
if (autoPlay) {
|
||||
video.play().catch((err) => {
|
||||
console.warn('[HLS] Autoplay prevented:', err);
|
||||
// console.warn('[HLS] Autoplay prevented:', err);
|
||||
onAutoPlayPrevented?.(err);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// Error Handling
|
||||
let networkErrorRetries = 0;
|
||||
let mediaErrorRetries = 0;
|
||||
const MAX_RETRIES = 3;
|
||||
@@ -136,22 +165,18 @@ export function useHlsPlayer({
|
||||
switch (data.type) {
|
||||
case Hls.ErrorTypes.NETWORK_ERROR:
|
||||
networkErrorRetries++;
|
||||
console.error(`[HLS] Network error (${networkErrorRetries}/${MAX_RETRIES}), trying to recover...`, data);
|
||||
if (networkErrorRetries <= MAX_RETRIES) {
|
||||
hls?.startLoad();
|
||||
} else {
|
||||
console.error('[HLS] Too many network errors, giving up');
|
||||
onError?.('网络错误:无法加载视频流');
|
||||
hls?.destroy();
|
||||
}
|
||||
break;
|
||||
case Hls.ErrorTypes.MEDIA_ERROR:
|
||||
mediaErrorRetries++;
|
||||
console.error(`[HLS] Media error (${mediaErrorRetries}/${MAX_RETRIES}), trying to recover...`, data);
|
||||
if (mediaErrorRetries <= MAX_RETRIES) {
|
||||
hls?.recoverMediaError();
|
||||
} else {
|
||||
console.error('[HLS] Too many media errors, giving up');
|
||||
onError?.('媒体错误:视频格式不支持或已损坏');
|
||||
hls?.destroy();
|
||||
}
|
||||
@@ -162,20 +187,142 @@ export function useHlsPlayer({
|
||||
hls?.destroy();
|
||||
break;
|
||||
}
|
||||
} else {
|
||||
// Non-fatal errors
|
||||
console.warn('[HLS] Non-fatal error:', data.type, data.details);
|
||||
}
|
||||
});
|
||||
} else {
|
||||
// Native HLS support
|
||||
// Native HLS (Desktop Safari, no Filter)
|
||||
video.src = src;
|
||||
}
|
||||
} else if (isNativeHlsSupported) {
|
||||
// Fallback for environments where Hls.js is not supported but native is (e.g. iOS without MSE?)
|
||||
video.src = src;
|
||||
// Native HLS (iOS, Mobile Safari)
|
||||
// Limitations: Native HLS cannot easily intercept sub-playlist requests.
|
||||
// We use fetch+blob for the master playlist as a best 'first-level' filter.
|
||||
// If the ad discontinuity is in the master playlist (rare for ads, common for periods), it works.
|
||||
// If it's in sub-playlists, it might fail unless we parse and blob those too (complex).
|
||||
|
||||
if (isAdFilterEnabled) {
|
||||
const processMasterPlaylist = async (masterSrc: string) => {
|
||||
// Move blob tracking outside try to ensure cleanup on error
|
||||
const createdBlobs: string[] = [];
|
||||
|
||||
// Safely resolve relative URLs to absolute (handles iOS Safari scenarios)
|
||||
let absoluteMasterSrc: string;
|
||||
try {
|
||||
absoluteMasterSrc = new URL(masterSrc, window.location.href).toString();
|
||||
} catch {
|
||||
absoluteMasterSrc = masterSrc; // Fallback if URL parsing fails
|
||||
}
|
||||
|
||||
try {
|
||||
const response = await fetch(absoluteMasterSrc);
|
||||
const masterContent = await response.text();
|
||||
|
||||
// If it's a simple playlist (no variants), just filter and play
|
||||
if (!masterContent.includes('#EXT-X-STREAM-INF')) {
|
||||
const filtered = filterM3u8Ad(masterContent, absoluteMasterSrc, adFilterMode, adKeywords);
|
||||
const blob = new Blob([filtered], { type: 'application/vnd.apple.mpegurl' });
|
||||
const blobUrl = URL.createObjectURL(blob);
|
||||
createdBlobs.push(blobUrl);
|
||||
return { masterBlobUrl: blobUrl, allBlobs: createdBlobs };
|
||||
}
|
||||
|
||||
// It IS a master playlist. Use map + Promise.all for clean concurrent processing.
|
||||
const lines = masterContent.split(/\r?\n/);
|
||||
|
||||
|
||||
|
||||
// Process each line, looking back at previous line to determine context
|
||||
const lineProcessingPromises = lines.map(async (line, index) => {
|
||||
const trimmedLine = line.trim();
|
||||
|
||||
// Handle #EXT-X-MEDIA:URI="..."
|
||||
if (trimmedLine.startsWith('#EXT-X-MEDIA') && trimmedLine.includes('URI="')) {
|
||||
const uriMatch = trimmedLine.match(/URI="([^"]+)"/);
|
||||
const uri = uriMatch?.[1];
|
||||
if (uri) {
|
||||
// Process if relative or absolute URL; fetch will handle CORS
|
||||
const isRelative = !uri.startsWith('http');
|
||||
|
||||
if (isRelative || uri.startsWith('http')) {
|
||||
try {
|
||||
const absoluteUrl = isRelative ? new URL(uri, absoluteMasterSrc).toString() : uri;
|
||||
const subRes = await fetch(absoluteUrl);
|
||||
const subContent = await subRes.text();
|
||||
const filteredSub = filterM3u8Ad(subContent, absoluteUrl, adFilterMode, adKeywords);
|
||||
const subBlob = new Blob([filteredSub], { type: 'application/vnd.apple.mpegurl' });
|
||||
const subBlobUrl = URL.createObjectURL(subBlob);
|
||||
createdBlobs.push(subBlobUrl);
|
||||
return line.replace(`URI="${uri}"`, `URI="${subBlobUrl}"`);
|
||||
} catch (e) {
|
||||
console.warn('[HLS Native] Failed to process EXT-X-MEDIA URI:', e);
|
||||
return line;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Handle playlist URL (line after #EXT-X-STREAM-INF)
|
||||
const prevLine = index > 0 ? lines[index - 1].trim() : '';
|
||||
if (prevLine.startsWith('#EXT-X-STREAM-INF') && trimmedLine && !trimmedLine.startsWith('#')) {
|
||||
// Process if relative or absolute URL; fetch will handle CORS
|
||||
const isRelative = !trimmedLine.startsWith('http');
|
||||
|
||||
if (isRelative || trimmedLine.startsWith('http')) {
|
||||
try {
|
||||
const absoluteUrl = isRelative ? new URL(trimmedLine, absoluteMasterSrc).toString() : trimmedLine;
|
||||
const subRes = await fetch(absoluteUrl);
|
||||
const subContent = await subRes.text();
|
||||
const filteredSub = filterM3u8Ad(subContent, absoluteUrl, adFilterMode, adKeywords);
|
||||
const subBlob = new Blob([filteredSub], { type: 'application/vnd.apple.mpegurl' });
|
||||
const subBlobUrl = URL.createObjectURL(subBlob);
|
||||
createdBlobs.push(subBlobUrl);
|
||||
return subBlobUrl;
|
||||
} catch (e) {
|
||||
console.warn('[HLS Native] Failed to process variant playlist:', e);
|
||||
return line;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// All other lines pass through unchanged
|
||||
return line;
|
||||
});
|
||||
|
||||
const processedLines = await Promise.all(lineProcessingPromises);
|
||||
|
||||
// Join back
|
||||
const finalMasterContent = processedLines.join('\n');
|
||||
const masterBlob = new Blob([finalMasterContent], { type: 'application/vnd.apple.mpegurl' });
|
||||
const masterBlobUrl = URL.createObjectURL(masterBlob);
|
||||
createdBlobs.push(masterBlobUrl);
|
||||
|
||||
return { masterBlobUrl, allBlobs: createdBlobs };
|
||||
} catch (e) {
|
||||
// Critical: Clean up any blobs created before the error
|
||||
for (const blobUrl of createdBlobs) {
|
||||
try {
|
||||
URL.revokeObjectURL(blobUrl);
|
||||
} catch { /* ignore cleanup errors */ }
|
||||
}
|
||||
console.error('[HLS Native] Recursive fetch failed', e);
|
||||
throw e;
|
||||
}
|
||||
};
|
||||
|
||||
processMasterPlaylist(src).then((result) => {
|
||||
video.src = result.masterBlobUrl;
|
||||
extraBlobs = result.allBlobs;
|
||||
}).catch((e) => {
|
||||
console.warn('[HLS Native] Ad filtering failed, falling back to original source.', e);
|
||||
onError?.('广告过滤失败,已回退到原始视频流');
|
||||
video.src = src;
|
||||
});
|
||||
|
||||
} else {
|
||||
video.src = src;
|
||||
}
|
||||
} else {
|
||||
console.error('[HLS] HLS not supported in this browser');
|
||||
console.error('[HLS] HLS not supported');
|
||||
onError?.('当前浏览器不支持 HLS 视频播放');
|
||||
}
|
||||
|
||||
@@ -183,6 +330,7 @@ export function useHlsPlayer({
|
||||
if (hls) {
|
||||
hls.destroy();
|
||||
}
|
||||
extraBlobs.forEach(url => URL.revokeObjectURL(url));
|
||||
};
|
||||
}, [src, videoRef, autoPlay, onAutoPlayPrevented, onError]);
|
||||
}, [src, videoRef, autoPlay, onAutoPlayPrevented, onError, isAdFilterEnabled, adFilterMode, adKeywords]);
|
||||
}
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
'use client';
|
||||
|
||||
import { useState, useEffect, useCallback } from 'react';
|
||||
import { settingsStore } from '@/lib/store/settings-store';
|
||||
import { settingsStore, AdFilterMode } from '@/lib/store/settings-store';
|
||||
|
||||
/**
|
||||
* Hook to access and update player settings from the settings store
|
||||
@@ -17,6 +17,9 @@ export function usePlayerSettings() {
|
||||
autoSkipOutro: stored.autoSkipOutro,
|
||||
skipOutroSeconds: stored.skipOutroSeconds,
|
||||
showModeIndicator: stored.showModeIndicator,
|
||||
adFilter: stored.adFilter,
|
||||
adFilterMode: stored.adFilterMode,
|
||||
adKeywords: stored.adKeywords,
|
||||
};
|
||||
});
|
||||
|
||||
@@ -31,6 +34,9 @@ export function usePlayerSettings() {
|
||||
autoSkipOutro: stored.autoSkipOutro,
|
||||
skipOutroSeconds: stored.skipOutroSeconds,
|
||||
showModeIndicator: stored.showModeIndicator,
|
||||
adFilter: stored.adFilter,
|
||||
adFilterMode: stored.adFilterMode,
|
||||
adKeywords: stored.adKeywords,
|
||||
});
|
||||
});
|
||||
return unsubscribe;
|
||||
@@ -71,6 +77,18 @@ export function usePlayerSettings() {
|
||||
updateSetting('showModeIndicator', value);
|
||||
}, [updateSetting]);
|
||||
|
||||
const setAdFilter = useCallback((value: boolean) => {
|
||||
updateSetting('adFilter', value);
|
||||
}, [updateSetting]);
|
||||
|
||||
const setAdFilterMode = useCallback((value: AdFilterMode) => {
|
||||
updateSetting('adFilterMode', value);
|
||||
}, [updateSetting]);
|
||||
|
||||
const setAdKeywords = useCallback((value: string[]) => {
|
||||
updateSetting('adKeywords', value);
|
||||
}, [updateSetting]);
|
||||
|
||||
return {
|
||||
...settings,
|
||||
setAutoNextEpisode,
|
||||
@@ -79,5 +97,8 @@ export function usePlayerSettings() {
|
||||
setAutoSkipOutro,
|
||||
setSkipOutroSeconds,
|
||||
setShowModeIndicator,
|
||||
setAdFilter,
|
||||
setAdFilterMode,
|
||||
setAdKeywords,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -15,6 +15,14 @@ export const UtilityIcons = {
|
||||
</svg>
|
||||
),
|
||||
|
||||
ShieldAlert: ({ className = "", size = 24 }: IconProps) => (
|
||||
<svg width={size} height={size} viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round" className={className}>
|
||||
<path d="M12 22s8-4 8-10V5l-8-3-8 3v7c0 6 8 10 8 10z" />
|
||||
<line x1="12" y1="8" x2="12" y2="12" />
|
||||
<line x1="12" y1="16" x2="12.01" y2="16" />
|
||||
</svg>
|
||||
),
|
||||
|
||||
Sparkles: ({ className = "", size = 24 }: IconProps) => (
|
||||
<svg width={size} height={size} viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round" className={className}>
|
||||
<path d="M12 3v18M5.2 8.2l13.6 7.6M18.8 8.2L5.2 15.8" />
|
||||
|
||||
+38
-59
@@ -18,6 +18,7 @@ export type SortOption =
|
||||
| 'name-desc';
|
||||
|
||||
export type SearchDisplayMode = 'normal' | 'grouped';
|
||||
export type AdFilterMode = 'off' | 'keyword' | 'heuristic' | 'aggressive';
|
||||
|
||||
export interface AppSettings {
|
||||
sources: VideoSource[];
|
||||
@@ -35,6 +36,9 @@ export interface AppSettings {
|
||||
autoSkipOutro: boolean;
|
||||
skipOutroSeconds: number;
|
||||
showModeIndicator: boolean; // Show '直连模式'/'代理模式' badge on player
|
||||
adFilter: boolean; // Filter ad tags from m3u8 (legacy, kept for compatibility)
|
||||
adFilterMode: AdFilterMode; // 'off' | 'keyword' | 'heuristic' | 'aggressive'
|
||||
adKeywords: string[]; // Dynamically loaded ad keywords
|
||||
// Search & Display settings
|
||||
realtimeLatency: boolean; // Enable real-time latency ping updates
|
||||
searchDisplayMode: SearchDisplayMode; // 'normal' = individual cards, 'grouped' = group same-name videos
|
||||
@@ -87,51 +91,43 @@ function getEnvSubscriptions(customValue?: string): SourceSubscription[] {
|
||||
// Debugging helper
|
||||
// console.log("Environment Subscriptions:", getEnvSubscriptions());
|
||||
|
||||
// Shared default settings factory to avoid code duplication
|
||||
function getDefaultAppSettings(): AppSettings {
|
||||
return {
|
||||
sources: getDefaultSources(),
|
||||
premiumSources: getDefaultPremiumSources(),
|
||||
subscriptions: getEnvSubscriptions(),
|
||||
sortBy: 'default',
|
||||
searchHistory: true,
|
||||
watchHistory: true,
|
||||
passwordAccess: false,
|
||||
accessPasswords: [],
|
||||
autoNextEpisode: true,
|
||||
autoSkipIntro: false,
|
||||
skipIntroSeconds: 0,
|
||||
autoSkipOutro: false,
|
||||
skipOutroSeconds: 0,
|
||||
showModeIndicator: false,
|
||||
adFilter: false,
|
||||
adFilterMode: 'heuristic',
|
||||
adKeywords: [],
|
||||
realtimeLatency: false,
|
||||
searchDisplayMode: 'normal',
|
||||
episodeReverseOrder: false,
|
||||
};
|
||||
}
|
||||
|
||||
export const settingsStore = {
|
||||
getSettings(): AppSettings {
|
||||
// SSR: Return defaults
|
||||
if (typeof window === 'undefined') {
|
||||
return {
|
||||
sources: getDefaultSources(),
|
||||
premiumSources: getDefaultPremiumSources(),
|
||||
subscriptions: getEnvSubscriptions(),
|
||||
sortBy: 'default',
|
||||
searchHistory: true,
|
||||
watchHistory: true,
|
||||
passwordAccess: false,
|
||||
accessPasswords: [],
|
||||
autoNextEpisode: true,
|
||||
autoSkipIntro: false,
|
||||
skipIntroSeconds: 0,
|
||||
autoSkipOutro: false,
|
||||
skipOutroSeconds: 0,
|
||||
showModeIndicator: false,
|
||||
realtimeLatency: false,
|
||||
searchDisplayMode: 'normal',
|
||||
episodeReverseOrder: false,
|
||||
};
|
||||
return getDefaultAppSettings();
|
||||
}
|
||||
|
||||
// Client: No stored settings, return defaults
|
||||
const stored = localStorage.getItem(SETTINGS_KEY);
|
||||
if (!stored) {
|
||||
return {
|
||||
sources: getDefaultSources(),
|
||||
premiumSources: getDefaultPremiumSources(),
|
||||
subscriptions: getEnvSubscriptions(),
|
||||
sortBy: 'default',
|
||||
searchHistory: true,
|
||||
watchHistory: true,
|
||||
passwordAccess: false,
|
||||
accessPasswords: [],
|
||||
autoNextEpisode: true,
|
||||
autoSkipIntro: false,
|
||||
skipIntroSeconds: 0,
|
||||
autoSkipOutro: false,
|
||||
skipOutroSeconds: 0,
|
||||
showModeIndicator: false,
|
||||
realtimeLatency: false,
|
||||
searchDisplayMode: 'normal',
|
||||
episodeReverseOrder: false,
|
||||
};
|
||||
return getDefaultAppSettings();
|
||||
}
|
||||
|
||||
try {
|
||||
@@ -187,33 +183,16 @@ export const settingsStore = {
|
||||
autoSkipOutro: parsed.autoSkipOutro !== undefined ? parsed.autoSkipOutro : false,
|
||||
skipOutroSeconds: typeof parsed.skipOutroSeconds === 'number' ? parsed.skipOutroSeconds : 0,
|
||||
showModeIndicator: parsed.showModeIndicator !== undefined ? parsed.showModeIndicator : false,
|
||||
adFilter: parsed.adFilter !== undefined ? parsed.adFilter : false,
|
||||
adFilterMode: parsed.adFilterMode || 'heuristic',
|
||||
adKeywords: Array.isArray(parsed.adKeywords) ? parsed.adKeywords : [],
|
||||
realtimeLatency: parsed.realtimeLatency !== undefined ? parsed.realtimeLatency : false,
|
||||
searchDisplayMode: parsed.searchDisplayMode === 'grouped' ? 'grouped' : 'normal',
|
||||
episodeReverseOrder: parsed.episodeReverseOrder !== undefined ? parsed.episodeReverseOrder : false,
|
||||
};
|
||||
} catch {
|
||||
// Even if localStorage fails, we should return defaults + ENV subscriptions
|
||||
const envSubscriptions = getEnvSubscriptions();
|
||||
|
||||
return {
|
||||
sources: getDefaultSources(),
|
||||
premiumSources: getDefaultPremiumSources(),
|
||||
subscriptions: envSubscriptions,
|
||||
sortBy: 'default',
|
||||
searchHistory: true,
|
||||
watchHistory: true,
|
||||
passwordAccess: false,
|
||||
accessPasswords: [],
|
||||
autoNextEpisode: true,
|
||||
autoSkipIntro: false,
|
||||
skipIntroSeconds: 0,
|
||||
autoSkipOutro: false,
|
||||
skipOutroSeconds: 0,
|
||||
showModeIndicator: false,
|
||||
realtimeLatency: false,
|
||||
searchDisplayMode: 'normal',
|
||||
episodeReverseOrder: false,
|
||||
};
|
||||
return getDefaultAppSettings();
|
||||
}
|
||||
},
|
||||
|
||||
|
||||
@@ -0,0 +1,265 @@
|
||||
/**
|
||||
* Heuristic Ad Detection Module
|
||||
*
|
||||
* Provides block-based analysis for detecting ads in M3U8 playlists
|
||||
* using filename pattern matching and other heuristics.
|
||||
*/
|
||||
|
||||
// Ad-related path keywords for scoring
|
||||
export const AD_PATH_KEYWORDS = [
|
||||
'advert', 'preroll', 'midroll', 'postroll',
|
||||
'dai', 'vast', 'ima', 'adjump', 'commercial', 'sponsor'
|
||||
];
|
||||
|
||||
/**
|
||||
* Represents a segment in the playlist
|
||||
*/
|
||||
interface Segment {
|
||||
url: string;
|
||||
duration: number;
|
||||
lineIndex: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Represents a block of segments between DISCONTINUITY markers
|
||||
*/
|
||||
interface Block {
|
||||
segments: Segment[];
|
||||
startLineIndex: number;
|
||||
endLineIndex: number;
|
||||
hasCueTag: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Pattern extracted from main content for comparison
|
||||
*/
|
||||
interface MainPattern {
|
||||
filenameRegex: RegExp | null;
|
||||
avgDuration: number;
|
||||
commonPrefix: string;
|
||||
pathPrefix: string; // Directory path prefix (e.g., "/20230907/73PWifvT/1392kb/hls/")
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse M3U8 content into blocks separated by DISCONTINUITY markers
|
||||
*/
|
||||
export function parseBlocks(lines: string[]): Block[] {
|
||||
const blocks: Block[] = [];
|
||||
let currentBlock: Block = {
|
||||
segments: [],
|
||||
startLineIndex: 0,
|
||||
endLineIndex: 0,
|
||||
hasCueTag: false
|
||||
};
|
||||
|
||||
for (let i = 0; i < lines.length; i++) {
|
||||
const line = lines[i].trim();
|
||||
|
||||
// Check for CUE tags
|
||||
if (line.startsWith('#EXT-X-CUE-OUT') || line.startsWith('#EXT-X-CUE-IN')) {
|
||||
currentBlock.hasCueTag = true;
|
||||
}
|
||||
|
||||
// DISCONTINUITY marks block boundary
|
||||
if (line === '#EXT-X-DISCONTINUITY') {
|
||||
if (currentBlock.segments.length > 0) {
|
||||
currentBlock.endLineIndex = i - 1;
|
||||
blocks.push(currentBlock);
|
||||
}
|
||||
currentBlock = {
|
||||
segments: [],
|
||||
startLineIndex: i + 1,
|
||||
endLineIndex: 0,
|
||||
hasCueTag: false
|
||||
};
|
||||
continue;
|
||||
}
|
||||
|
||||
// Parse EXTINF and the following URL
|
||||
if (line.startsWith('#EXTINF:')) {
|
||||
const durationMatch = line.match(/#EXTINF:([\d.]+)/);
|
||||
const duration = durationMatch ? parseFloat(durationMatch[1]) : 0;
|
||||
|
||||
// Next line should be the URL
|
||||
if (i + 1 < lines.length) {
|
||||
const url = lines[i + 1].trim();
|
||||
if (url && !url.startsWith('#')) {
|
||||
currentBlock.segments.push({
|
||||
url,
|
||||
duration,
|
||||
lineIndex: i + 1
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Don't forget the last block
|
||||
if (currentBlock.segments.length > 0) {
|
||||
currentBlock.endLineIndex = lines.length - 1;
|
||||
blocks.push(currentBlock);
|
||||
}
|
||||
|
||||
return blocks;
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract filename from URL (handles both relative and absolute URLs)
|
||||
*/
|
||||
function extractFilename(url: string): string {
|
||||
try {
|
||||
const path = url.includes('://') ? new URL(url).pathname : url;
|
||||
const parts = path.split('/');
|
||||
return parts[parts.length - 1] || '';
|
||||
} catch {
|
||||
return url.split('/').pop() || '';
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Find common prefix among an array of strings
|
||||
*/
|
||||
function findCommonPrefix(strings: string[]): string {
|
||||
if (!strings || strings.length < 2) return '';
|
||||
|
||||
let prefix = '';
|
||||
const first = strings[0];
|
||||
|
||||
for (let i = 0; i < first.length; i++) {
|
||||
const char = first[i];
|
||||
if (strings.every(s => s[i] === char)) {
|
||||
prefix += char;
|
||||
} else {
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
return prefix;
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract path prefix (directory) from URL
|
||||
* e.g., "/20230907/73PWifvT/1392kb/hls/" from "/20230907/73PWifvT/1392kb/hls/gFE6lwIk.ts"
|
||||
*/
|
||||
function extractPathPrefix(url: string): string {
|
||||
try {
|
||||
const path = url.includes('://') ? new URL(url).pathname : url;
|
||||
const lastSlash = path.lastIndexOf('/');
|
||||
return lastSlash >= 0 ? path.substring(0, lastSlash + 1) : '';
|
||||
} catch {
|
||||
const lastSlash = url.lastIndexOf('/');
|
||||
return lastSlash >= 0 ? url.substring(0, lastSlash + 1) : '';
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Learn pattern from the largest block (assumed to be main content)
|
||||
*/
|
||||
export function learnMainPattern(blocks: Block[]): MainPattern {
|
||||
// Find the largest block by segment count (likely main content)
|
||||
const mainBlock = blocks.length > 0 ? blocks.reduce((largest, block) =>
|
||||
block.segments.length > largest.segments.length ? block : largest
|
||||
) : null;
|
||||
|
||||
if (!mainBlock || mainBlock.segments.length === 0) {
|
||||
return { filenameRegex: null, avgDuration: 0, commonPrefix: '', pathPrefix: '' };
|
||||
}
|
||||
|
||||
// Extract filenames
|
||||
const filenames = mainBlock.segments.map(s => extractFilename(s.url));
|
||||
|
||||
// Find common prefix
|
||||
const commonPrefix = findCommonPrefix(filenames);
|
||||
|
||||
// Calculate average duration
|
||||
const totalDuration = mainBlock.segments.reduce((sum, s) => sum + s.duration, 0);
|
||||
const avgDuration = totalDuration / mainBlock.segments.length;
|
||||
|
||||
// Try to build a regex pattern from the filenames
|
||||
// Common patterns: "0000001.ts", "seg-1.ts", "segment_001.ts"
|
||||
let filenameRegex: RegExp | null = null;
|
||||
if (commonPrefix.length >= 2) {
|
||||
// Escape special regex characters in prefix
|
||||
const escapedPrefix = commonPrefix.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
||||
filenameRegex = new RegExp(`^${escapedPrefix}`);
|
||||
}
|
||||
|
||||
// Extract path prefix (directory path without filename)
|
||||
// e.g., "/20230907/73PWifvT/1392kb/hls/" from "/20230907/73PWifvT/1392kb/hls/gFE6lwIk.ts"
|
||||
const firstUrl = mainBlock.segments[0].url;
|
||||
const pathPrefix = extractPathPrefix(firstUrl);
|
||||
|
||||
return { filenameRegex, avgDuration, commonPrefix, pathPrefix };
|
||||
}
|
||||
|
||||
/**
|
||||
* Score a block for ad likelihood based on heuristics
|
||||
* Returns a score where higher = more likely to be an ad
|
||||
*/
|
||||
export function scoreBlock(block: Block, mainPattern: MainPattern, extraKeywords: string[] = []): number {
|
||||
let score = 0;
|
||||
|
||||
// If block has CUE tag, it's definitely an ad
|
||||
if (block.hasCueTag) {
|
||||
return 10; // Max score
|
||||
}
|
||||
|
||||
// Check path keywords (Built-in + Custom)
|
||||
// We filter out very short custom keywords to avoid false positives in scoring
|
||||
const safeExtraKeywords = extraKeywords.filter(k => k.length > 2);
|
||||
const allKeywords = [...AD_PATH_KEYWORDS, ...safeExtraKeywords];
|
||||
|
||||
for (const segment of block.segments) {
|
||||
const urlLower = segment.url.toLowerCase();
|
||||
for (const keyword of allKeywords) {
|
||||
if (urlLower.includes(keyword.toLowerCase())) {
|
||||
score += 2.5;
|
||||
break; // Only count once per segment
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Check filename pattern mismatch
|
||||
if (mainPattern.filenameRegex) {
|
||||
const mismatchCount = block.segments.filter(s => {
|
||||
if (!mainPattern.filenameRegex) return false; // No pattern to compare against
|
||||
const filename = extractFilename(s.url);
|
||||
return !mainPattern.filenameRegex.test(filename);
|
||||
}).length;
|
||||
|
||||
if (mismatchCount === block.segments.length && block.segments.length > 0) {
|
||||
score += 1.5; // All filenames differ from main pattern
|
||||
}
|
||||
}
|
||||
|
||||
// **KEY FEATURE**: Check path prefix mismatch (e.g., different date/folder/bitrate)
|
||||
// This is the most reliable indicator for ads that come from different CDN paths
|
||||
if (mainPattern.pathPrefix && block.segments.length > 0) {
|
||||
const pathMismatchCount = block.segments.filter(s => {
|
||||
const segmentPathPrefix = extractPathPrefix(s.url);
|
||||
return segmentPathPrefix !== mainPattern.pathPrefix;
|
||||
}).length;
|
||||
|
||||
if (pathMismatchCount === block.segments.length) {
|
||||
// ALL segments have different path prefix - strong ad indicator
|
||||
score += 5.0;
|
||||
}
|
||||
}
|
||||
|
||||
return score;
|
||||
}
|
||||
|
||||
/**
|
||||
* Threshold configuration
|
||||
*/
|
||||
export const THRESHOLDS = {
|
||||
HIGH: 5.0, // Definitely an ad
|
||||
LOW: 3.0 // Possibly an ad (for future "fuzzy" mode)
|
||||
};
|
||||
|
||||
/**
|
||||
* Determine if a block should be filtered based on its score
|
||||
*/
|
||||
export function shouldFilterBlock(score: number, threshold: number = THRESHOLDS.HIGH): boolean {
|
||||
return score >= threshold;
|
||||
}
|
||||
@@ -0,0 +1,157 @@
|
||||
|
||||
/**
|
||||
* Utility functions for M3U8 playlist manipulation
|
||||
*/
|
||||
|
||||
import { parseBlocks, learnMainPattern, scoreBlock, shouldFilterBlock } from './m3u8-ad-detector';
|
||||
|
||||
/**
|
||||
* Filters ads from specific M3U8 content using multiple detection strategies:
|
||||
* 1. Keyword matching (configurable via env)
|
||||
* 2. CUE-OUT/CUE-IN standard tags
|
||||
* 3. Heuristic block analysis (filename patterns, ad path keywords)
|
||||
*
|
||||
* Also converts relative URLs to absolute URLs for Blob playback.
|
||||
*
|
||||
* @param content The raw M3U8 content string
|
||||
* @param baseUrl The base URL of the M3U8 file (to resolve relative paths)
|
||||
* @returns The filtered M3U8 content
|
||||
*/
|
||||
export type AdFilterMode = 'off' | 'keyword' | 'heuristic' | 'aggressive';
|
||||
|
||||
export function filterM3u8Ad(content: string, baseUrl: string, mode: AdFilterMode = 'heuristic', customKeywords: string[] = []): string {
|
||||
if (!content) return '';
|
||||
|
||||
// Use keywords passed from AdKeywordsWrapper (already loaded from env/file)
|
||||
const keywords = customKeywords;
|
||||
|
||||
const basePath = baseUrl.substring(0, baseUrl.lastIndexOf('/') + 1);
|
||||
let origin = '';
|
||||
try {
|
||||
origin = new URL(baseUrl).origin;
|
||||
} catch (e) { /* ignore */ }
|
||||
|
||||
// 2. Global Scan: Check if any ad keywords exist in the content
|
||||
const hasKeywordMatch = mode !== 'off' && keywords.some(k => content.includes(k));
|
||||
const hasCueTag = mode !== 'off' && (content.includes('#EXT-X-CUE-OUT') || content.includes('#EXT-X-CUE-IN'));
|
||||
|
||||
// 3. Heuristic Analysis: If no explicit ad signals, use block-based detection
|
||||
const lines = content.split(/\r?\n/);
|
||||
let adLineIndices = new Set<number>();
|
||||
|
||||
if (!hasCueTag && (mode === 'heuristic' || mode === 'aggressive')) {
|
||||
// No obvious ad signals - run heuristic analysis
|
||||
const blocks = parseBlocks(lines);
|
||||
if (blocks.length > 1) {
|
||||
const mainPattern = learnMainPattern(blocks);
|
||||
for (const block of blocks) {
|
||||
// Pass all keywords (including custom ones) to heuristic scorer
|
||||
const score = scoreBlock(block, mainPattern, keywords);
|
||||
const threshold = mode === 'aggressive' ? 3.0 : 5.0;
|
||||
if (shouldFilterBlock(score, threshold)) {
|
||||
// Mark all lines in this block for removal
|
||||
for (const segment of block.segments) {
|
||||
adLineIndices.add(segment.lineIndex);
|
||||
adLineIndices.add(segment.lineIndex - 1); // EXTINF line
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const processedLines: string[] = [];
|
||||
|
||||
// State machine for CUE-OUT/CUE-IN tracking
|
||||
let insideCueAdBlock = false;
|
||||
|
||||
for (let i = 0; i < lines.length; i++) {
|
||||
const line = lines[i];
|
||||
const trimmedLine = line.trim();
|
||||
|
||||
// Skip lines marked by heuristic analysis
|
||||
if (adLineIndices.has(i)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
// 3. CUE Tag Detection (SCTE-35 Standard)
|
||||
// EXT-X-CUE-OUT marks start of ad, EXT-X-CUE-IN marks end
|
||||
if (mode !== 'off' && trimmedLine.startsWith('#EXT-X-CUE-OUT')) {
|
||||
insideCueAdBlock = true;
|
||||
// Remove preceding DISCONTINUITY if present
|
||||
if (processedLines.length > 0 && processedLines[processedLines.length - 1].trim() === '#EXT-X-DISCONTINUITY') {
|
||||
processedLines.pop();
|
||||
}
|
||||
continue; // Skip the CUE-OUT tag itself
|
||||
}
|
||||
|
||||
if (trimmedLine.startsWith('#EXT-X-CUE-IN')) {
|
||||
insideCueAdBlock = false;
|
||||
// Also skip the next line if it's a DISCONTINUITY (ad block ending marker)
|
||||
if (i + 1 < lines.length && lines[i + 1].trim() === '#EXT-X-DISCONTINUITY') {
|
||||
i++; // Skip the following DISCONTINUITY
|
||||
}
|
||||
continue; // Skip the CUE-IN tag itself
|
||||
}
|
||||
|
||||
// Skip all content inside CUE ad block
|
||||
if (insideCueAdBlock) {
|
||||
continue;
|
||||
}
|
||||
|
||||
// 4. Keyword-based Ad Detection & Backtrack (skip if no keywords configured)
|
||||
if (keywords.length > 0 && hasKeywordMatch && keywords.some(keyword => trimmedLine.includes(keyword))) {
|
||||
// Found Ad: Remove it and backtrack to remove associated metadata
|
||||
while (processedLines.length > 0) {
|
||||
const lastIndex = processedLines.length - 1;
|
||||
const lastLine = processedLines[lastIndex].trim();
|
||||
|
||||
if (lastLine.startsWith('#EXTINF:') || lastLine === '#EXT-X-DISCONTINUITY') {
|
||||
processedLines.pop();
|
||||
} else {
|
||||
break;
|
||||
}
|
||||
}
|
||||
continue; // Skip the ad line itself
|
||||
}
|
||||
|
||||
// 5. Discontinuity Handling (Conservative Mode)
|
||||
// Keep all Discontinuity tags by default.
|
||||
// They will ONLY be removed via backtracking when a confirmed ad segment is found.
|
||||
// This prevents false positives on legitimate concatenated streams.
|
||||
if (trimmedLine === '#EXT-X-DISCONTINUITY') {
|
||||
processedLines.push(line);
|
||||
continue;
|
||||
}
|
||||
|
||||
// 6. General Cleanup & URL Normalization
|
||||
if (!trimmedLine || trimmedLine.startsWith('http') || trimmedLine.startsWith('blob:')) {
|
||||
processedLines.push(line);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (trimmedLine.startsWith('#')) {
|
||||
// Handle URI="..." in attributes (e.g. #EXT-X-KEY)
|
||||
if (trimmedLine.includes('URI="')) {
|
||||
processedLines.push(line.replace(/URI="([^"]+)"/g, (match, uri) => {
|
||||
if (uri.startsWith('http')) return match; // Already absolute
|
||||
if (uri.startsWith('/')) {
|
||||
return `URI="${origin}${uri}"`; // Root-relative
|
||||
}
|
||||
return `URI="${basePath}${uri}"`; // Path-relative
|
||||
}));
|
||||
} else {
|
||||
processedLines.push(line);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
// 7. Resolve Relative URLs (for Blob support)
|
||||
if (trimmedLine.startsWith('/')) {
|
||||
processedLines.push(origin ? `${origin}${trimmedLine}` : trimmedLine);
|
||||
} else {
|
||||
processedLines.push(`${basePath}${trimmedLine}`);
|
||||
}
|
||||
}
|
||||
|
||||
return processedLines.join('\n');
|
||||
}
|
||||
Reference in New Issue
Block a user