mirror of
https://github.com/ZSCGR/CloudFlare-ImgBed.git
synced 2026-08-13 04:03:42 +08:00
@@ -0,0 +1,191 @@
|
||||
# CloudFlare ImgBed KV 到 D1 数据库迁移指南
|
||||
|
||||
本指南将帮助您将现有的 KV 存储数据迁移到 D1 数据库。
|
||||
|
||||
## 迁移前准备
|
||||
|
||||
### 1. 配置 D1 数据库
|
||||
|
||||
首先,您需要在 Cloudflare 控制台创建一个 D1 数据库:
|
||||
|
||||
```bash
|
||||
# 创建 D1 数据库
|
||||
wrangler d1 create imgbed-database
|
||||
|
||||
# 执行数据库初始化脚本
|
||||
wrangler d1 execute imgbed-database --file=./database/init.sql
|
||||
```
|
||||
|
||||
### 2. 更新 wrangler.toml
|
||||
|
||||
在您的 `wrangler.toml` 文件中添加 D1 数据库绑定:
|
||||
|
||||
```toml
|
||||
[[d1_databases]]
|
||||
binding = "DB"
|
||||
database_name = "imgbed-database"
|
||||
database_id = "your-database-id"
|
||||
```
|
||||
|
||||
### 3. 备份现有数据
|
||||
|
||||
在迁移前,强烈建议备份您的现有数据:
|
||||
|
||||
1. 访问管理后台的系统设置 → 备份恢复
|
||||
2. 点击"备份数据"下载完整备份文件
|
||||
3. 保存备份文件到安全位置
|
||||
|
||||
## 迁移步骤
|
||||
|
||||
### 1. 检查迁移环境
|
||||
|
||||
访问迁移工具检查环境:
|
||||
```
|
||||
GET /api/manage/migrate?action=check
|
||||
```
|
||||
|
||||
确保返回结果中 `canMigrate` 为 `true`。
|
||||
|
||||
### 2. 查看迁移状态
|
||||
|
||||
查看当前数据统计:
|
||||
```
|
||||
GET /api/manage/migrate?action=status
|
||||
```
|
||||
|
||||
### 3. 执行迁移
|
||||
|
||||
开始数据迁移:
|
||||
```
|
||||
GET /api/manage/migrate?action=migrate
|
||||
```
|
||||
|
||||
迁移过程包括:
|
||||
- 文件元数据迁移
|
||||
- 系统设置迁移
|
||||
- 索引操作迁移
|
||||
|
||||
## 迁移后验证
|
||||
|
||||
### 1. 检查数据完整性
|
||||
|
||||
- 访问管理后台,确认文件列表正常显示
|
||||
- 测试文件上传功能
|
||||
- 检查系统设置是否保持不变
|
||||
|
||||
### 2. 性能测试
|
||||
|
||||
- 测试文件访问速度
|
||||
- 验证管理功能正常工作
|
||||
|
||||
### 3. 功能验证
|
||||
|
||||
- 文件上传/删除
|
||||
- 备份恢复功能
|
||||
- 用户认证
|
||||
- API Token 管理
|
||||
|
||||
## 数据库结构说明
|
||||
|
||||
迁移后的 D1 数据库包含以下表:
|
||||
|
||||
### files 表
|
||||
存储文件元数据,包含以下字段:
|
||||
- `id`: 文件ID(主键)
|
||||
- `value`: 文件值(用于分块文件)
|
||||
- `metadata`: JSON格式的文件元数据
|
||||
- 其他索引字段:`file_name`, `file_type`, `timestamp` 等
|
||||
|
||||
### settings 表
|
||||
存储系统配置:
|
||||
- `key`: 配置键(主键)
|
||||
- `value`: 配置值
|
||||
- `category`: 配置分类
|
||||
|
||||
### index_operations 表
|
||||
存储索引操作记录:
|
||||
- `id`: 操作ID(主键)
|
||||
- `type`: 操作类型
|
||||
- `timestamp`: 时间戳
|
||||
- `data`: 操作数据
|
||||
- `processed`: 是否已处理
|
||||
|
||||
### index_metadata 表
|
||||
存储索引元数据:
|
||||
- `key`: 元数据键
|
||||
- `last_updated`: 最后更新时间
|
||||
- `total_count`: 总文件数
|
||||
- `last_operation_id`: 最后操作ID
|
||||
|
||||
### other_data 表
|
||||
存储其他数据(如黑名单IP等):
|
||||
- `key`: 数据键
|
||||
- `value`: 数据值
|
||||
- `type`: 数据类型
|
||||
|
||||
## 兼容性说明
|
||||
|
||||
### 向后兼容
|
||||
- 系统会自动检测可用的数据库类型(D1 或 KV)
|
||||
- 如果 D1 不可用,会自动回退到 KV 存储
|
||||
- 所有现有的 API 接口保持不变
|
||||
|
||||
### 环境变量
|
||||
- `DB`: D1 数据库绑定(新增)
|
||||
- `img_url`: KV 存储绑定(保留作为备用)
|
||||
|
||||
## 故障排除
|
||||
|
||||
### 常见问题
|
||||
|
||||
1. **迁移失败**
|
||||
- 检查 D1 数据库是否正确配置
|
||||
- 确认数据库初始化脚本已执行
|
||||
- 查看迁移日志中的错误信息
|
||||
|
||||
2. **数据不完整**
|
||||
- 检查迁移结果中的错误列表
|
||||
- 重新运行迁移(支持增量迁移)
|
||||
- 使用备份文件恢复数据
|
||||
|
||||
3. **性能问题**
|
||||
- D1 数据库查询比 KV 稍慢,这是正常的
|
||||
- 确保使用了适当的索引
|
||||
- 考虑优化查询语句
|
||||
|
||||
### 回滚方案
|
||||
|
||||
如果迁移后出现问题,可以:
|
||||
|
||||
1. 临时禁用 D1 绑定,系统会自动回退到 KV
|
||||
2. 使用备份文件恢复到迁移前状态
|
||||
3. 重新配置和测试 D1 数据库
|
||||
|
||||
## 性能优化建议
|
||||
|
||||
### 1. 索引优化
|
||||
D1 数据库已预设了必要的索引,包括:
|
||||
- 文件时间戳索引
|
||||
- 目录索引
|
||||
- 文件类型索引
|
||||
|
||||
### 2. 查询优化
|
||||
- 使用分页查询大量数据
|
||||
- 避免全表扫描
|
||||
- 合理使用 WHERE 条件
|
||||
|
||||
### 3. 监控
|
||||
- 定期检查数据库性能
|
||||
- 监控查询执行时间
|
||||
- 关注错误日志
|
||||
|
||||
## 支持
|
||||
|
||||
如果在迁移过程中遇到问题,请:
|
||||
|
||||
1. 查看浏览器控制台的错误信息
|
||||
2. 检查 Cloudflare Workers 的日志
|
||||
3. 确认所有配置正确
|
||||
4. 参考本指南的故障排除部分
|
||||
|
||||
迁移完成后,您的图床将使用更强大的 D1 数据库,享受更好的查询性能和数据管理能力。
|
||||
@@ -1,8 +1,8 @@
|
||||
<div align="center">
|
||||
<a href="https://github.com/MarSeventh/CloudFlare-ImgBed"><img width="80%" alt="logo" src="static/readme/banner.png"/></a>
|
||||
<p><em>🗂️开源文件托管解决方案,支持 Docker 和无服务器部署,支持 Telegram Bot 、 Cloudflare R2 、S3 等多种存储渠道</em></p>
|
||||
<p><em>🗂️开源文件托管解决方案,支持 Docker 和无服务器部署,支持 Telegram Bot 、 Cloudflare R2 、S3 等多种存储渠道.</em> 魔改原版将KV改为D1存储</p>
|
||||
<p>
|
||||
<a href="https://github.com/MarSeventh/CloudFlare-ImgBed/blob/main/README.md">简体中文</a> | <a href="https://github.com/MarSeventh/CloudFlare-ImgBed/blob/main/README_en.md">English</a> | <a href="https://cfbed.sanyue.de">官方网站</a>
|
||||
<a href="https://github.com/ccxyChuzhong/CloudFlare-ImgBed-D1/blob/main/README.md">简体中文</a> | <a href="https://github.com/ccxyChuzhong/CloudFlare-ImgBed-D1/blob/main/README_en.md">English</a> | <a href="https://github.com/MarSeventh/CloudFlare-ImgBed">KV版本(原版)</a> | <a href="https://github.com/ccxyChuzhong/CloudFlare-ImgBed-D1">D1版本</a> | <a href="https://cfbed.sanyue.de">官方网站</a>
|
||||
</p>
|
||||
<div>
|
||||
<a href="https://github.com/MarSeventh/CloudFlare-ImgBed/blob/main/LICENSE">
|
||||
@@ -78,6 +78,137 @@
|
||||
</details>
|
||||
|
||||
|
||||
# 必看!必看 !必看!
|
||||
如果是使用KV存储想转D1存储。建议重新创建一个图床。使用系统的备份和恢复功能进行数据迁移!!!!
|
||||
|
||||
<details>
|
||||
<summary>KV转D1存储详细如下</summary>
|
||||
|
||||
- 首先确认您的 D1 数据库已经创建:数据库名称必须为: `imgbed-database` 将数据库sql语句一段一段的全部执行
|
||||
```sql
|
||||
-- CloudFlare ImgBed D1 Database Initialization Script
|
||||
-- 这个脚本用于初始化D1数据库
|
||||
|
||||
-- 删除已存在的表(如果需要重新初始化)
|
||||
-- 注意:在生产环境中使用时请谨慎
|
||||
-- DROP TABLE IF EXISTS files;
|
||||
-- DROP TABLE IF EXISTS settings;
|
||||
-- DROP TABLE IF EXISTS index_operations;
|
||||
-- DROP TABLE IF EXISTS index_metadata;
|
||||
-- DROP TABLE IF EXISTS other_data;
|
||||
|
||||
-- 执行主要的数据库架构创建
|
||||
-- 这里会包含 schema.sql 的内容
|
||||
|
||||
-- 1. 文件表 - 存储文件元数据
|
||||
CREATE TABLE IF NOT EXISTS files (
|
||||
id TEXT PRIMARY KEY,
|
||||
value TEXT,
|
||||
metadata TEXT NOT NULL,
|
||||
file_name TEXT,
|
||||
file_type TEXT,
|
||||
file_size TEXT,
|
||||
upload_ip TEXT,
|
||||
upload_address TEXT,
|
||||
list_type TEXT,
|
||||
timestamp INTEGER,
|
||||
label TEXT,
|
||||
directory TEXT,
|
||||
channel TEXT,
|
||||
channel_name TEXT,
|
||||
tg_file_id TEXT,
|
||||
tg_chat_id TEXT,
|
||||
tg_bot_token TEXT,
|
||||
is_chunked BOOLEAN DEFAULT FALSE,
|
||||
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- 2. 系统配置表
|
||||
CREATE TABLE IF NOT EXISTS settings (
|
||||
key TEXT PRIMARY KEY,
|
||||
value TEXT NOT NULL,
|
||||
category TEXT,
|
||||
description TEXT,
|
||||
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- 3. 索引操作表
|
||||
CREATE TABLE IF NOT EXISTS index_operations (
|
||||
id TEXT PRIMARY KEY,
|
||||
type TEXT NOT NULL,
|
||||
timestamp INTEGER NOT NULL,
|
||||
data TEXT NOT NULL,
|
||||
processed BOOLEAN DEFAULT FALSE,
|
||||
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- 4. 索引元数据表
|
||||
CREATE TABLE IF NOT EXISTS index_metadata (
|
||||
key TEXT PRIMARY KEY,
|
||||
last_updated INTEGER,
|
||||
total_count INTEGER DEFAULT 0,
|
||||
last_operation_id TEXT,
|
||||
chunk_count INTEGER DEFAULT 0,
|
||||
chunk_size INTEGER DEFAULT 0,
|
||||
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- 5. 其他数据表
|
||||
CREATE TABLE IF NOT EXISTS other_data (
|
||||
key TEXT PRIMARY KEY,
|
||||
value TEXT NOT NULL,
|
||||
type TEXT,
|
||||
description TEXT,
|
||||
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
-- 初始化完成
|
||||
```
|
||||
|
||||
### 在 Cloudflare Dashboard 配置 Pages 绑定
|
||||
|
||||
#### 步骤 A: 登录 Cloudflare Dashboard
|
||||
1. 访问 https://dash.cloudflare.com
|
||||
2. 登录您的账户
|
||||
|
||||
#### 步骤 B: 进入 Pages 项目
|
||||
1. 在左侧菜单中点击 **"Pages"**
|
||||
2. 找到并点击您的图床项目
|
||||
|
||||
#### 步骤 C: 配置 Functions 绑定
|
||||
1. 在项目页面中,点击 **"Settings"** 标签
|
||||
2. 在左侧菜单中点击 **"Functions"**
|
||||
3. 向下滚动找到 **"D1 database bindings"** 部分
|
||||
|
||||
#### 步骤 D: 添加 D1 绑定
|
||||
1. 点击 **"Add binding"** 按钮
|
||||
2. 填写以下信息:
|
||||
- **Variable name**: `DB` (必须是大写的 DB)
|
||||
- **D1 database**: 从下拉菜单中选择您创建的 `imgbed-database`
|
||||
3. 点击 **"Save"** 按钮
|
||||
|
||||
#### 步骤 E: 重新部署 Pages
|
||||
|
||||
配置绑定后,需要重新部署:
|
||||
|
||||
#### 步骤 F: 验证配置
|
||||
|
||||
部署完成后,访问以下URL验证配置:
|
||||
|
||||
```
|
||||
https://your-domain.com/api/manage/migrate?action=check
|
||||
```
|
||||
|
||||
查看详细的配置状态
|
||||
```
|
||||
https://your-domain.com/api/manage/migrate?action=status
|
||||
```
|
||||
</details>
|
||||
|
||||
|
||||
|
||||
|
||||
# 1. Introduction
|
||||
|
||||
+141
-3
@@ -1,9 +1,8 @@
|
||||
<div align="center">
|
||||
<a href="https://github.com/MarSeventh/CloudFlare-ImgBed"><img width="80%" alt="logo" src="static/readme/banner.png"/></a>
|
||||
<p><em>🗂️Open-source file hosting solution, supporting Docker and serverless deployment, supporting multiple storage channels such as Telegram Bot, Cloudflare R2, S3, etc.</em></p>
|
||||
<p><em>🗂️Open-source file hosting solution, supporting Docker and serverless deployment, supporting multiple storage channels such as Telegram Bot, Cloudflare R2, S3, etc.</em> Modified version that replaces KV with D1 storage</p>
|
||||
<p>
|
||||
<a href="https://github.com/MarSeventh/CloudFlare-ImgBed/blob/main/README.md">简体中文</a> | <a href="https://github.com/MarSeventh/CloudFlare-ImgBed/blob/main/README_en.md">English</a> | <a
|
||||
href="https://cfbed.sanyue.de/en">Official Website</a>
|
||||
<a href="https://github.com/ccxyChuzhong/CloudFlare-ImgBed-D1/blob/main/README.md">简体中文</a> | <a href="https://github.com/ccxyChuzhong/CloudFlare-ImgBed-D1/blob/main/README_en.md">English</a> | <a href="https://github.com/MarSeventh/CloudFlare-ImgBed">KV Version (Original)</a> | <a href="https://github.com/ccxyChuzhong/CloudFlare-ImgBed-D1">D1 Version</a> | <a href="https://cfbed.sanyue.de/en">Official Website</a>
|
||||
</p>
|
||||
<div>
|
||||
<a href="https://github.com/MarSeventh/CloudFlare-ImgBed/blob/main/LICENSE">
|
||||
@@ -68,6 +67,145 @@
|
||||
</details>
|
||||
|
||||
|
||||
# Important! Important! Important!
|
||||
If you are using KV storage and want to migrate to D1 storage, it is recommended to create a new image hosting service. Use the system's backup and restore functions for data migration!!!!
|
||||
|
||||
<details>
|
||||
<summary>Detailed KV to D1 Storage Migration Guide</summary>
|
||||
|
||||
- First, confirm that your D1 database has been created: The database name must be: `imgbed-database`. Execute all SQL statements section by section:
|
||||
```sql
|
||||
-- CloudFlare ImgBed D1 Database Initialization Script
|
||||
-- This script is used to initialize the D1 database
|
||||
|
||||
-- Drop existing tables (if re-initialization is needed)
|
||||
-- Note: Use with caution in production environment
|
||||
-- DROP TABLE IF EXISTS files;
|
||||
-- DROP TABLE IF EXISTS settings;
|
||||
-- DROP TABLE IF EXISTS index_operations;
|
||||
-- DROP TABLE IF EXISTS index_metadata;
|
||||
-- DROP TABLE IF EXISTS other_data;
|
||||
|
||||
-- Execute main database schema creation
|
||||
-- This will include the content of schema.sql
|
||||
|
||||
-- 1. Files table - stores file metadata
|
||||
CREATE TABLE IF NOT EXISTS files (
|
||||
id TEXT PRIMARY KEY,
|
||||
value TEXT,
|
||||
metadata TEXT NOT NULL,
|
||||
file_name TEXT,
|
||||
file_type TEXT,
|
||||
file_size TEXT,
|
||||
upload_ip TEXT,
|
||||
upload_address TEXT,
|
||||
list_type TEXT,
|
||||
timestamp INTEGER,
|
||||
label TEXT,
|
||||
directory TEXT,
|
||||
channel TEXT,
|
||||
channel_name TEXT,
|
||||
tg_file_id TEXT,
|
||||
tg_chat_id TEXT,
|
||||
tg_bot_token TEXT,
|
||||
is_chunked BOOLEAN DEFAULT FALSE,
|
||||
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- 2. System configuration table
|
||||
CREATE TABLE IF NOT EXISTS settings (
|
||||
key TEXT PRIMARY KEY,
|
||||
value TEXT NOT NULL,
|
||||
category TEXT,
|
||||
description TEXT,
|
||||
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- 3. Index operations table
|
||||
CREATE TABLE IF NOT EXISTS index_operations (
|
||||
id TEXT PRIMARY KEY,
|
||||
type TEXT NOT NULL,
|
||||
timestamp INTEGER NOT NULL,
|
||||
data TEXT NOT NULL,
|
||||
processed BOOLEAN DEFAULT FALSE,
|
||||
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- 4. Index metadata table
|
||||
CREATE TABLE IF NOT EXISTS index_metadata (
|
||||
key TEXT PRIMARY KEY,
|
||||
last_updated INTEGER,
|
||||
total_count INTEGER DEFAULT 0,
|
||||
last_operation_id TEXT,
|
||||
chunk_count INTEGER DEFAULT 0,
|
||||
chunk_size INTEGER DEFAULT 0,
|
||||
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- 5. Other data table
|
||||
CREATE TABLE IF NOT EXISTS other_data (
|
||||
key TEXT PRIMARY KEY,
|
||||
value TEXT NOT NULL,
|
||||
type TEXT,
|
||||
description TEXT,
|
||||
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- Insert initial index metadata
|
||||
INSERT OR REPLACE INTO index_metadata (key, last_updated, total_count, last_operation_id)
|
||||
VALUES ('main_index', 0, 0, NULL);
|
||||
|
||||
-- Initialization complete
|
||||
-- Database is ready, data migration can begin
|
||||
|
||||
```
|
||||
|
||||
### Configure Pages Bindings in Cloudflare Dashboard
|
||||
|
||||
#### Step A: Login to Cloudflare Dashboard
|
||||
1. Visit https://dash.cloudflare.com
|
||||
2. Login to your account
|
||||
|
||||
#### Step B: Enter Pages Project
|
||||
1. Click **"Pages"** in the left menu
|
||||
2. Find and click your image hosting project
|
||||
|
||||
#### Step C: Configure Functions Bindings
|
||||
1. Click the **"Settings"** tab on the project page
|
||||
2. Click **"Functions"** in the left menu
|
||||
3. Scroll down to find the **"D1 database bindings"** section
|
||||
|
||||
#### Step D: Add D1 Binding
|
||||
1. Click the **"Add binding"** button
|
||||
2. Fill in the following information:
|
||||
- **Variable name**: `DB` (must be uppercase DB)
|
||||
- **D1 database**: Select your created `imgbed-database` from the dropdown
|
||||
3. Click the **"Save"** button
|
||||
|
||||
#### Step E: Redeploy Pages
|
||||
|
||||
After configuring bindings, you need to redeploy:
|
||||
|
||||
#### Step F: Verify Configuration
|
||||
|
||||
After deployment is complete, visit the following URL to verify configuration:
|
||||
|
||||
```
|
||||
https://your-domain.com/api/manage/migrate?action=check
|
||||
```
|
||||
|
||||
View detailed configuration status:
|
||||
```
|
||||
https://your-domain.com/api/manage/migrate?action=status
|
||||
```
|
||||
</details>
|
||||
|
||||
|
||||
|
||||
|
||||
# 1. Introduction
|
||||
|
||||
|
||||
@@ -0,0 +1,127 @@
|
||||
-- CloudFlare ImgBed D1 Database Initialization Script
|
||||
-- 这个脚本用于初始化D1数据库
|
||||
|
||||
-- 删除已存在的表(如果需要重新初始化)
|
||||
-- 注意:在生产环境中使用时请谨慎
|
||||
-- DROP TABLE IF EXISTS files;
|
||||
-- DROP TABLE IF EXISTS settings;
|
||||
-- DROP TABLE IF EXISTS index_operations;
|
||||
-- DROP TABLE IF EXISTS index_metadata;
|
||||
-- DROP TABLE IF EXISTS other_data;
|
||||
|
||||
-- 执行主要的数据库架构创建
|
||||
-- 这里会包含 schema.sql 的内容
|
||||
|
||||
-- 1. 文件表 - 存储文件元数据
|
||||
CREATE TABLE IF NOT EXISTS files (
|
||||
id TEXT PRIMARY KEY,
|
||||
value TEXT,
|
||||
metadata TEXT NOT NULL,
|
||||
file_name TEXT,
|
||||
file_type TEXT,
|
||||
file_size TEXT,
|
||||
upload_ip TEXT,
|
||||
upload_address TEXT,
|
||||
list_type TEXT,
|
||||
timestamp INTEGER,
|
||||
label TEXT,
|
||||
directory TEXT,
|
||||
channel TEXT,
|
||||
channel_name TEXT,
|
||||
tg_file_id TEXT,
|
||||
tg_chat_id TEXT,
|
||||
tg_bot_token TEXT,
|
||||
is_chunked BOOLEAN DEFAULT FALSE,
|
||||
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- 2. 系统配置表
|
||||
CREATE TABLE IF NOT EXISTS settings (
|
||||
key TEXT PRIMARY KEY,
|
||||
value TEXT NOT NULL,
|
||||
category TEXT,
|
||||
description TEXT,
|
||||
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- 3. 索引操作表
|
||||
CREATE TABLE IF NOT EXISTS index_operations (
|
||||
id TEXT PRIMARY KEY,
|
||||
type TEXT NOT NULL,
|
||||
timestamp INTEGER NOT NULL,
|
||||
data TEXT NOT NULL,
|
||||
processed BOOLEAN DEFAULT FALSE,
|
||||
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- 4. 索引元数据表
|
||||
CREATE TABLE IF NOT EXISTS index_metadata (
|
||||
key TEXT PRIMARY KEY,
|
||||
last_updated INTEGER,
|
||||
total_count INTEGER DEFAULT 0,
|
||||
last_operation_id TEXT,
|
||||
chunk_count INTEGER DEFAULT 0,
|
||||
chunk_size INTEGER DEFAULT 0,
|
||||
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- 5. 其他数据表
|
||||
CREATE TABLE IF NOT EXISTS other_data (
|
||||
key TEXT PRIMARY KEY,
|
||||
value TEXT NOT NULL,
|
||||
type TEXT,
|
||||
description TEXT,
|
||||
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- 创建索引
|
||||
CREATE INDEX IF NOT EXISTS idx_files_timestamp ON files(timestamp DESC);
|
||||
CREATE INDEX IF NOT EXISTS idx_files_directory ON files(directory);
|
||||
CREATE INDEX IF NOT EXISTS idx_files_channel ON files(channel);
|
||||
CREATE INDEX IF NOT EXISTS idx_files_file_type ON files(file_type);
|
||||
CREATE INDEX IF NOT EXISTS idx_files_upload_ip ON files(upload_ip);
|
||||
CREATE INDEX IF NOT EXISTS idx_files_created_at ON files(created_at DESC);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_settings_category ON settings(category);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_index_operations_timestamp ON index_operations(timestamp);
|
||||
CREATE INDEX IF NOT EXISTS idx_index_operations_processed ON index_operations(processed);
|
||||
CREATE INDEX IF NOT EXISTS idx_index_operations_type ON index_operations(type);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_other_data_type ON other_data(type);
|
||||
|
||||
-- 创建触发器
|
||||
CREATE TRIGGER IF NOT EXISTS update_files_updated_at
|
||||
AFTER UPDATE ON files
|
||||
BEGIN
|
||||
UPDATE files SET updated_at = CURRENT_TIMESTAMP WHERE id = NEW.id;
|
||||
END;
|
||||
|
||||
CREATE TRIGGER IF NOT EXISTS update_settings_updated_at
|
||||
AFTER UPDATE ON settings
|
||||
BEGIN
|
||||
UPDATE settings SET updated_at = CURRENT_TIMESTAMP WHERE key = NEW.key;
|
||||
END;
|
||||
|
||||
CREATE TRIGGER IF NOT EXISTS update_index_metadata_updated_at
|
||||
AFTER UPDATE ON index_metadata
|
||||
BEGIN
|
||||
UPDATE index_metadata SET updated_at = CURRENT_TIMESTAMP WHERE key = NEW.key;
|
||||
END;
|
||||
|
||||
CREATE TRIGGER IF NOT EXISTS update_other_data_updated_at
|
||||
AFTER UPDATE ON other_data
|
||||
BEGIN
|
||||
UPDATE other_data SET updated_at = CURRENT_TIMESTAMP WHERE key = NEW.key;
|
||||
END;
|
||||
|
||||
-- 插入初始的索引元数据
|
||||
INSERT OR REPLACE INTO index_metadata (key, last_updated, total_count, last_operation_id)
|
||||
VALUES ('main_index', 0, 0, NULL);
|
||||
|
||||
-- 初始化完成
|
||||
-- 数据库已准备就绪,可以开始迁移数据
|
||||
@@ -0,0 +1,116 @@
|
||||
-- CloudFlare ImgBed D1 Database Schema
|
||||
-- 用于替代原有的KV存储
|
||||
|
||||
-- 1. 文件表 - 存储文件元数据
|
||||
CREATE TABLE IF NOT EXISTS files (
|
||||
id TEXT PRIMARY KEY, -- 文件ID (原KV的key)
|
||||
value TEXT, -- 文件值 (对于分块文件,存储实际内容)
|
||||
metadata TEXT NOT NULL, -- 文件元数据 (JSON格式)
|
||||
file_name TEXT, -- 文件名 (从metadata中提取,便于查询)
|
||||
file_type TEXT, -- 文件类型 (从metadata中提取)
|
||||
file_size TEXT, -- 文件大小 (从metadata中提取)
|
||||
upload_ip TEXT, -- 上传IP (从metadata中提取)
|
||||
upload_address TEXT, -- 上传地址 (从metadata中提取)
|
||||
list_type TEXT, -- 列表类型 (从metadata中提取)
|
||||
timestamp INTEGER, -- 时间戳 (从metadata中提取,便于排序)
|
||||
label TEXT, -- 标签 (从metadata中提取)
|
||||
directory TEXT, -- 目录 (从metadata中提取,便于查询)
|
||||
channel TEXT, -- 渠道 (从metadata中提取)
|
||||
channel_name TEXT, -- 渠道名称 (从metadata中提取)
|
||||
tg_file_id TEXT, -- Telegram文件ID (从metadata中提取)
|
||||
tg_chat_id TEXT, -- Telegram聊天ID (从metadata中提取)
|
||||
tg_bot_token TEXT, -- Telegram Bot Token (从metadata中提取)
|
||||
is_chunked BOOLEAN DEFAULT FALSE, -- 是否为分块文件
|
||||
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- 为文件表创建索引
|
||||
CREATE INDEX IF NOT EXISTS idx_files_timestamp ON files(timestamp DESC);
|
||||
CREATE INDEX IF NOT EXISTS idx_files_directory ON files(directory);
|
||||
CREATE INDEX IF NOT EXISTS idx_files_channel ON files(channel);
|
||||
CREATE INDEX IF NOT EXISTS idx_files_file_type ON files(file_type);
|
||||
CREATE INDEX IF NOT EXISTS idx_files_upload_ip ON files(upload_ip);
|
||||
CREATE INDEX IF NOT EXISTS idx_files_created_at ON files(created_at DESC);
|
||||
|
||||
-- 2. 系统配置表 - 存储各种系统配置
|
||||
CREATE TABLE IF NOT EXISTS settings (
|
||||
key TEXT PRIMARY KEY, -- 配置键 (原KV的key)
|
||||
value TEXT NOT NULL, -- 配置值 (JSON格式)
|
||||
category TEXT, -- 配置分类 (page, security, upload, others等)
|
||||
description TEXT, -- 配置描述
|
||||
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- 为设置表创建索引
|
||||
CREATE INDEX IF NOT EXISTS idx_settings_category ON settings(category);
|
||||
|
||||
-- 3. 索引操作表 - 存储原子操作记录
|
||||
CREATE TABLE IF NOT EXISTS index_operations (
|
||||
id TEXT PRIMARY KEY, -- 操作ID
|
||||
type TEXT NOT NULL, -- 操作类型 (add, remove, move, batch_add等)
|
||||
timestamp INTEGER NOT NULL, -- 时间戳
|
||||
data TEXT NOT NULL, -- 操作数据 (JSON格式)
|
||||
processed BOOLEAN DEFAULT FALSE, -- 是否已处理
|
||||
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- 为索引操作表创建索引
|
||||
CREATE INDEX IF NOT EXISTS idx_index_operations_timestamp ON index_operations(timestamp);
|
||||
CREATE INDEX IF NOT EXISTS idx_index_operations_processed ON index_operations(processed);
|
||||
CREATE INDEX IF NOT EXISTS idx_index_operations_type ON index_operations(type);
|
||||
|
||||
-- 4. 索引元数据表 - 存储索引的元信息
|
||||
CREATE TABLE IF NOT EXISTS index_metadata (
|
||||
key TEXT PRIMARY KEY, -- 元数据键 (如 'main_index')
|
||||
last_updated INTEGER, -- 最后更新时间
|
||||
total_count INTEGER DEFAULT 0, -- 总文件数
|
||||
last_operation_id TEXT, -- 最后处理的操作ID
|
||||
chunk_count INTEGER DEFAULT 0, -- 分块数量 (保留字段,D1中可能不需要)
|
||||
chunk_size INTEGER DEFAULT 0, -- 分块大小 (保留字段)
|
||||
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- 5. 其他数据表 - 存储黑名单IP等其他数据
|
||||
CREATE TABLE IF NOT EXISTS other_data (
|
||||
key TEXT PRIMARY KEY, -- 数据键
|
||||
value TEXT NOT NULL, -- 数据值
|
||||
type TEXT, -- 数据类型 (blacklist_ip, whitelist等)
|
||||
description TEXT, -- 描述
|
||||
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
-- 为其他数据表创建索引
|
||||
CREATE INDEX IF NOT EXISTS idx_other_data_type ON other_data(type);
|
||||
|
||||
-- 6. 创建触发器来自动更新 updated_at 字段
|
||||
-- 文件表触发器
|
||||
CREATE TRIGGER IF NOT EXISTS update_files_updated_at
|
||||
AFTER UPDATE ON files
|
||||
BEGIN
|
||||
UPDATE files SET updated_at = CURRENT_TIMESTAMP WHERE id = NEW.id;
|
||||
END;
|
||||
|
||||
-- 设置表触发器
|
||||
CREATE TRIGGER IF NOT EXISTS update_settings_updated_at
|
||||
AFTER UPDATE ON settings
|
||||
BEGIN
|
||||
UPDATE settings SET updated_at = CURRENT_TIMESTAMP WHERE key = NEW.key;
|
||||
END;
|
||||
|
||||
-- 索引元数据表触发器
|
||||
CREATE TRIGGER IF NOT EXISTS update_index_metadata_updated_at
|
||||
AFTER UPDATE ON index_metadata
|
||||
BEGIN
|
||||
UPDATE index_metadata SET updated_at = CURRENT_TIMESTAMP WHERE key = NEW.key;
|
||||
END;
|
||||
|
||||
-- 其他数据表触发器
|
||||
CREATE TRIGGER IF NOT EXISTS update_other_data_updated_at
|
||||
AFTER UPDATE ON other_data
|
||||
BEGIN
|
||||
UPDATE other_data SET updated_at = CURRENT_TIMESTAMP WHERE key = NEW.key;
|
||||
END;
|
||||
@@ -0,0 +1,37 @@
|
||||
import { errorHandling, telemetryData, checkDatabaseConfig } from './utils/middleware';
|
||||
|
||||
// 安全的中间件链,带错误处理
|
||||
export async function onRequest(context) {
|
||||
try {
|
||||
// 检查数据库配置
|
||||
var dbCheckResult = await checkDatabaseConfig(context);
|
||||
if (dbCheckResult instanceof Response) {
|
||||
return dbCheckResult;
|
||||
}
|
||||
|
||||
// 错误处理中间件
|
||||
var errorResult = await errorHandling(context);
|
||||
if (errorResult instanceof Response) {
|
||||
return errorResult;
|
||||
}
|
||||
|
||||
// 遥测数据中间件
|
||||
var telemetryResult = await telemetryData(context);
|
||||
if (telemetryResult instanceof Response) {
|
||||
return telemetryResult;
|
||||
}
|
||||
|
||||
return await context.next();
|
||||
} catch (error) {
|
||||
console.error('Middleware chain error:', error);
|
||||
return new Response(JSON.stringify({
|
||||
error: 'Middleware error: ' + error.message,
|
||||
stack: error.stack
|
||||
}), {
|
||||
status: 500,
|
||||
headers: {
|
||||
'Content-Type': 'application/json'
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,114 @@
|
||||
/**
|
||||
* 备份功能测试工具
|
||||
*/
|
||||
|
||||
import { getDatabase } from '../../utils/databaseAdapter.js';
|
||||
|
||||
export async function onRequest(context) {
|
||||
var env = context.env;
|
||||
|
||||
try {
|
||||
var db = getDatabase(env);
|
||||
|
||||
var results = {
|
||||
databaseType: db.constructor.name || 'Unknown',
|
||||
settings: {
|
||||
all: [],
|
||||
manage: [],
|
||||
sysConfig: []
|
||||
},
|
||||
files: {
|
||||
count: 0,
|
||||
sample: []
|
||||
}
|
||||
};
|
||||
|
||||
// 测试列出所有设置
|
||||
try {
|
||||
var allSettings = await db.listSettings({});
|
||||
results.settings.all = allSettings.keys.map(function(item) {
|
||||
return {
|
||||
key: item.name,
|
||||
hasValue: !!item.value,
|
||||
valueLength: item.value ? item.value.length : 0
|
||||
};
|
||||
});
|
||||
} catch (error) {
|
||||
results.settings.allError = error.message;
|
||||
}
|
||||
|
||||
// 测试列出manage@开头的设置
|
||||
try {
|
||||
var manageSettings = await db.listSettings({ prefix: 'manage@' });
|
||||
results.settings.manage = manageSettings.keys.map(function(item) {
|
||||
return {
|
||||
key: item.name,
|
||||
hasValue: !!item.value,
|
||||
valueLength: item.value ? item.value.length : 0
|
||||
};
|
||||
});
|
||||
} catch (error) {
|
||||
results.settings.manageError = error.message;
|
||||
}
|
||||
|
||||
// 测试列出sysConfig设置
|
||||
try {
|
||||
var sysConfigSettings = await db.listSettings({ prefix: 'manage@sysConfig@' });
|
||||
results.settings.sysConfig = sysConfigSettings.keys.map(function(item) {
|
||||
return {
|
||||
key: item.name,
|
||||
hasValue: !!item.value,
|
||||
valueLength: item.value ? item.value.length : 0,
|
||||
valuePreview: item.value ? item.value.substring(0, 100) + '...' : null
|
||||
};
|
||||
});
|
||||
} catch (error) {
|
||||
results.settings.sysConfigError = error.message;
|
||||
}
|
||||
|
||||
// 测试列出文件
|
||||
try {
|
||||
var filesList = await db.listFiles({ limit: 5 });
|
||||
results.files.count = filesList.keys.length;
|
||||
results.files.sample = filesList.keys.map(function(item) {
|
||||
return {
|
||||
id: item.name,
|
||||
hasMetadata: !!item.metadata
|
||||
};
|
||||
});
|
||||
} catch (error) {
|
||||
results.files.error = error.message;
|
||||
}
|
||||
|
||||
// 测试特定设置的读取
|
||||
try {
|
||||
var pageConfig = await db.get('manage@sysConfig@page');
|
||||
results.specificTests = {
|
||||
pageConfig: {
|
||||
exists: !!pageConfig,
|
||||
length: pageConfig ? pageConfig.length : 0,
|
||||
preview: pageConfig ? pageConfig.substring(0, 200) + '...' : null
|
||||
}
|
||||
};
|
||||
} catch (error) {
|
||||
results.specificTests = { error: error.message };
|
||||
}
|
||||
|
||||
return new Response(JSON.stringify(results, null, 2), {
|
||||
headers: {
|
||||
'Content-Type': 'application/json'
|
||||
}
|
||||
});
|
||||
|
||||
} catch (error) {
|
||||
return new Response(JSON.stringify({
|
||||
error: error.message,
|
||||
stack: error.stack
|
||||
}), {
|
||||
status: 500,
|
||||
headers: {
|
||||
'Content-Type': 'application/json'
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,112 @@
|
||||
/**
|
||||
* D1数据库查询测试工具
|
||||
*/
|
||||
|
||||
import { getDatabase } from '../../utils/databaseAdapter.js';
|
||||
|
||||
export async function onRequest(context) {
|
||||
var env = context.env;
|
||||
var url = new URL(context.request.url);
|
||||
var query = url.searchParams.get('query') || 'files';
|
||||
|
||||
try {
|
||||
var db = getDatabase(env);
|
||||
var results = {
|
||||
databaseType: db.constructor.name,
|
||||
query: query,
|
||||
results: null,
|
||||
error: null
|
||||
};
|
||||
|
||||
if (query === 'files') {
|
||||
// 直接查询files表
|
||||
try {
|
||||
var stmt = db.db.prepare('SELECT COUNT(*) as count FROM files');
|
||||
var countResult = await stmt.first();
|
||||
results.totalFiles = countResult.count;
|
||||
|
||||
// 查询前5条记录
|
||||
var stmt2 = db.db.prepare('SELECT id, metadata, created_at FROM files ORDER BY created_at DESC LIMIT 5');
|
||||
var fileResults = await stmt2.all();
|
||||
|
||||
// 检查结果格式
|
||||
console.log('fileResults type:', typeof fileResults);
|
||||
console.log('fileResults:', fileResults);
|
||||
|
||||
if (Array.isArray(fileResults)) {
|
||||
results.sampleFiles = fileResults.map(function(row) {
|
||||
return {
|
||||
id: row.id,
|
||||
metadata: JSON.parse(row.metadata || '{}'),
|
||||
created_at: row.created_at
|
||||
};
|
||||
});
|
||||
} else {
|
||||
results.sampleFiles = [];
|
||||
results.fileResultsType = typeof fileResults;
|
||||
results.fileResultsValue = fileResults;
|
||||
}
|
||||
} catch (error) {
|
||||
results.error = 'Direct query failed: ' + error.message;
|
||||
}
|
||||
} else if (query === 'list') {
|
||||
// 测试listFiles方法
|
||||
try {
|
||||
var listResult = await db.listFiles({ limit: 5 });
|
||||
results.listResult = listResult;
|
||||
} catch (error) {
|
||||
results.error = 'listFiles failed: ' + error.message;
|
||||
}
|
||||
} else if (query === 'listall') {
|
||||
// 测试通用list方法
|
||||
try {
|
||||
var listAllResult = await db.list({ limit: 5 });
|
||||
results.listAllResult = listAllResult;
|
||||
} catch (error) {
|
||||
results.error = 'list failed: ' + error.message;
|
||||
}
|
||||
} else if (query === 'prefix') {
|
||||
// 测试带前缀的查询
|
||||
var prefix = url.searchParams.get('prefix') || 'cosplay/';
|
||||
try {
|
||||
var prefixResult = await db.list({ prefix: prefix, limit: 10 });
|
||||
results.prefixResult = prefixResult;
|
||||
results.prefix = prefix;
|
||||
} catch (error) {
|
||||
results.error = 'prefix query failed: ' + error.message;
|
||||
}
|
||||
} else if (query === 'settings') {
|
||||
// 查询设置表
|
||||
try {
|
||||
var stmt = db.db.prepare('SELECT COUNT(*) as count FROM settings');
|
||||
var countResult = await stmt.first();
|
||||
results.totalSettings = countResult.count;
|
||||
|
||||
var stmt2 = db.db.prepare('SELECT key, value FROM settings LIMIT 5');
|
||||
var settingResults = await stmt2.all();
|
||||
results.sampleSettings = settingResults;
|
||||
} catch (error) {
|
||||
results.error = 'Settings query failed: ' + error.message;
|
||||
}
|
||||
} else {
|
||||
results.error = 'Unknown query type. Use: files, list, listall, prefix, settings';
|
||||
}
|
||||
|
||||
return new Response(JSON.stringify(results, null, 2), {
|
||||
headers: {
|
||||
'Content-Type': 'application/json'
|
||||
}
|
||||
});
|
||||
|
||||
} catch (error) {
|
||||
return new Response(JSON.stringify({
|
||||
error: error.message,
|
||||
stack: error.stack
|
||||
}), {
|
||||
status: 500,
|
||||
headers: {
|
||||
'Content-Type': 'application/json'
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,89 @@
|
||||
/**
|
||||
* 简单的D1测试
|
||||
*/
|
||||
|
||||
export async function onRequest(context) {
|
||||
var env = context.env;
|
||||
|
||||
try {
|
||||
var results = {
|
||||
hasDB: !!env.DB,
|
||||
dbType: env.DB ? typeof env.DB : 'undefined'
|
||||
};
|
||||
|
||||
if (!env.DB) {
|
||||
return new Response(JSON.stringify(results), {
|
||||
headers: { 'Content-Type': 'application/json' }
|
||||
});
|
||||
}
|
||||
|
||||
// 测试简单查询
|
||||
try {
|
||||
var stmt = env.DB.prepare('SELECT COUNT(*) as count FROM files');
|
||||
var countResult = await stmt.first();
|
||||
results.countQuery = {
|
||||
success: true,
|
||||
count: countResult.count,
|
||||
resultType: typeof countResult
|
||||
};
|
||||
} catch (error) {
|
||||
results.countQuery = {
|
||||
success: false,
|
||||
error: error.message
|
||||
};
|
||||
}
|
||||
|
||||
// 测试all()查询
|
||||
try {
|
||||
var stmt2 = env.DB.prepare('SELECT id FROM files LIMIT 3');
|
||||
var allResult = await stmt2.all();
|
||||
results.allQuery = {
|
||||
success: true,
|
||||
resultType: typeof allResult,
|
||||
isArray: Array.isArray(allResult),
|
||||
length: allResult ? allResult.length : 'N/A',
|
||||
sample: allResult
|
||||
};
|
||||
} catch (error) {
|
||||
results.allQuery = {
|
||||
success: false,
|
||||
error: error.message
|
||||
};
|
||||
}
|
||||
|
||||
// 测试带参数的查询
|
||||
try {
|
||||
var stmt3 = env.DB.prepare('SELECT id FROM files WHERE id LIKE ? LIMIT 2');
|
||||
var paramResult = await stmt3.bind('cosplay/%').all();
|
||||
results.paramQuery = {
|
||||
success: true,
|
||||
resultType: typeof paramResult,
|
||||
isArray: Array.isArray(paramResult),
|
||||
length: paramResult ? paramResult.length : 'N/A',
|
||||
sample: paramResult
|
||||
};
|
||||
} catch (error) {
|
||||
results.paramQuery = {
|
||||
success: false,
|
||||
error: error.message
|
||||
};
|
||||
}
|
||||
|
||||
return new Response(JSON.stringify(results, null, 2), {
|
||||
headers: {
|
||||
'Content-Type': 'application/json'
|
||||
}
|
||||
});
|
||||
|
||||
} catch (error) {
|
||||
return new Response(JSON.stringify({
|
||||
error: error.message,
|
||||
stack: error.stack
|
||||
}), {
|
||||
status: 500,
|
||||
headers: {
|
||||
'Content-Type': 'application/json'
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,109 @@
|
||||
/**
|
||||
* 数据库功能测试工具
|
||||
*/
|
||||
|
||||
import { getDatabase } from '../../utils/databaseAdapter.js';
|
||||
|
||||
export async function onRequest(context) {
|
||||
var env = context.env;
|
||||
|
||||
try {
|
||||
var results = {
|
||||
databaseType: null,
|
||||
listTest: null,
|
||||
getTest: null,
|
||||
putTest: null,
|
||||
errors: []
|
||||
};
|
||||
|
||||
// 测试数据库连接
|
||||
try {
|
||||
var db = getDatabase(env);
|
||||
results.databaseType = db.constructor.name || 'Unknown';
|
||||
} catch (error) {
|
||||
results.errors.push('Database connection failed: ' + error.message);
|
||||
return new Response(JSON.stringify(results, null, 2), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json' }
|
||||
});
|
||||
}
|
||||
|
||||
// 测试list方法
|
||||
try {
|
||||
var listResult = await db.list({ limit: 5 });
|
||||
results.listTest = {
|
||||
success: true,
|
||||
hasKeys: !!(listResult && listResult.keys),
|
||||
isArray: Array.isArray(listResult.keys),
|
||||
keyCount: listResult.keys ? listResult.keys.length : 0,
|
||||
structure: listResult ? Object.keys(listResult) : []
|
||||
};
|
||||
} catch (error) {
|
||||
results.listTest = {
|
||||
success: false,
|
||||
error: error.message
|
||||
};
|
||||
results.errors.push('List test failed: ' + error.message);
|
||||
}
|
||||
|
||||
// 测试get方法
|
||||
try {
|
||||
var getResult = await db.get('test_key_that_does_not_exist');
|
||||
results.getTest = {
|
||||
success: true,
|
||||
result: getResult,
|
||||
isNull: getResult === null
|
||||
};
|
||||
} catch (error) {
|
||||
results.getTest = {
|
||||
success: false,
|
||||
error: error.message
|
||||
};
|
||||
results.errors.push('Get test failed: ' + error.message);
|
||||
}
|
||||
|
||||
// 测试put方法(使用临时键)
|
||||
try {
|
||||
var testKey = 'test_' + Date.now();
|
||||
var testValue = 'test_value_' + Date.now();
|
||||
await db.put(testKey, testValue);
|
||||
|
||||
// 立即读取验证
|
||||
var retrievedValue = await db.get(testKey);
|
||||
|
||||
// 清理测试数据
|
||||
await db.delete(testKey);
|
||||
|
||||
results.putTest = {
|
||||
success: true,
|
||||
valueMatch: retrievedValue === testValue,
|
||||
testKey: testKey,
|
||||
testValue: testValue,
|
||||
retrievedValue: retrievedValue
|
||||
};
|
||||
} catch (error) {
|
||||
results.putTest = {
|
||||
success: false,
|
||||
error: error.message
|
||||
};
|
||||
results.errors.push('Put test failed: ' + error.message);
|
||||
}
|
||||
|
||||
return new Response(JSON.stringify(results, null, 2), {
|
||||
headers: {
|
||||
'Content-Type': 'application/json'
|
||||
}
|
||||
});
|
||||
|
||||
} catch (error) {
|
||||
return new Response(JSON.stringify({
|
||||
error: error.message,
|
||||
stack: error.stack
|
||||
}), {
|
||||
status: 500,
|
||||
headers: {
|
||||
'Content-Type': 'application/json'
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,121 @@
|
||||
/**
|
||||
* 环境变量调试工具
|
||||
* 用于检查 D1 和 KV 绑定状态
|
||||
*/
|
||||
import { getDatabase } from '../../utils/databaseAdapter.js';
|
||||
|
||||
export async function onRequest(context) {
|
||||
const { env } = context;
|
||||
|
||||
try {
|
||||
// 检查环境变量
|
||||
const envInfo = {
|
||||
hasDB: !!env.DB,
|
||||
hasImgUrl: !!env.img_url,
|
||||
dbType: env.DB ? typeof env.DB : 'undefined',
|
||||
imgUrlType: env.img_url ? typeof env.img_url : 'undefined',
|
||||
dbPrepare: env.DB && typeof env.DB.prepare === 'function',
|
||||
imgUrlGet: env.img_url && typeof env.img_url.get === 'function'
|
||||
};
|
||||
|
||||
// 尝试测试 D1 连接
|
||||
let d1Test = null;
|
||||
if (env.DB) {
|
||||
try {
|
||||
const stmt = env.DB.prepare('SELECT 1 as test');
|
||||
const result = await stmt.first();
|
||||
d1Test = { success: true, result: result };
|
||||
} catch (error) {
|
||||
d1Test = { success: false, error: error.message };
|
||||
}
|
||||
}
|
||||
|
||||
// 尝试测试 KV 连接
|
||||
let kvTest = null;
|
||||
if (env.img_url) {
|
||||
try {
|
||||
const result = await getDatabase(env).list({ limit: 1 });
|
||||
kvTest = { success: true, hasKeys: result.keys.length > 0 };
|
||||
} catch (error) {
|
||||
kvTest = { success: false, error: error.message };
|
||||
}
|
||||
}
|
||||
|
||||
const debugInfo = {
|
||||
timestamp: new Date().toISOString(),
|
||||
environment: envInfo,
|
||||
d1Test: d1Test,
|
||||
kvTest: kvTest,
|
||||
recommendation: getRecommendation(envInfo, d1Test, kvTest)
|
||||
};
|
||||
|
||||
return new Response(JSON.stringify(debugInfo, null, 2), {
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Access-Control-Allow-Origin': '*'
|
||||
}
|
||||
});
|
||||
|
||||
} catch (error) {
|
||||
return new Response(JSON.stringify({
|
||||
error: 'Debug failed',
|
||||
message: error.message,
|
||||
stack: error.stack
|
||||
}, null, 2), {
|
||||
status: 500,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Access-Control-Allow-Origin': '*'
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
function getRecommendation(envInfo, d1Test, kvTest) {
|
||||
const recommendations = [];
|
||||
|
||||
if (!envInfo.hasDB && !envInfo.hasImgUrl) {
|
||||
recommendations.push('❌ 没有配置任何数据库绑定');
|
||||
recommendations.push('🔧 请在 Cloudflare Pages Dashboard 中配置 D1 或 KV 绑定');
|
||||
}
|
||||
|
||||
if (envInfo.hasDB) {
|
||||
if (!envInfo.dbPrepare) {
|
||||
recommendations.push('⚠️ D1 绑定存在但 prepare 方法不可用');
|
||||
recommendations.push('🔧 请检查 D1 数据库是否正确绑定');
|
||||
} else if (d1Test && !d1Test.success) {
|
||||
recommendations.push('❌ D1 数据库连接失败: ' + d1Test.error);
|
||||
recommendations.push('🔧 请检查数据库是否已初始化表结构');
|
||||
recommendations.push('💡 运行: npx wrangler d1 execute imgbed-database --file=./database/init.sql');
|
||||
} else if (d1Test && d1Test.success) {
|
||||
recommendations.push('✅ D1 数据库连接正常');
|
||||
}
|
||||
} else {
|
||||
recommendations.push('ℹ️ 没有检测到 D1 绑定 (env.DB)');
|
||||
recommendations.push('🔧 在 Pages Settings → Functions → D1 database bindings 中添加:');
|
||||
recommendations.push(' Variable name: DB');
|
||||
recommendations.push(' D1 database: imgbed-database');
|
||||
}
|
||||
|
||||
if (envInfo.hasImgUrl) {
|
||||
if (!envInfo.imgUrlGet) {
|
||||
recommendations.push('⚠️ KV 绑定存在但 get 方法不可用');
|
||||
} else if (kvTest && !kvTest.success) {
|
||||
recommendations.push('❌ KV 连接失败: ' + kvTest.error);
|
||||
} else if (kvTest && kvTest.success) {
|
||||
recommendations.push('✅ KV 存储连接正常');
|
||||
}
|
||||
} else {
|
||||
recommendations.push('ℹ️ 没有检测到 KV 绑定 (env.img_url)');
|
||||
}
|
||||
|
||||
if (!envInfo.hasDB && !envInfo.hasImgUrl) {
|
||||
recommendations.push('');
|
||||
recommendations.push('🚀 快速解决方案:');
|
||||
recommendations.push('1. 重新部署项目 (配置可能还没生效)');
|
||||
recommendations.push('2. 等待 2-3 分钟让绑定生效');
|
||||
recommendations.push('3. 检查 Pages 项目的 Functions 设置');
|
||||
}
|
||||
|
||||
return recommendations;
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
/**
|
||||
* 最简单的测试页面
|
||||
*/
|
||||
|
||||
export async function onRequest(context) {
|
||||
try {
|
||||
return new Response(JSON.stringify({
|
||||
success: true,
|
||||
message: "Minimal test works",
|
||||
timestamp: new Date().toISOString()
|
||||
}), {
|
||||
headers: {
|
||||
'Content-Type': 'application/json'
|
||||
}
|
||||
});
|
||||
} catch (error) {
|
||||
return new Response(JSON.stringify({
|
||||
error: error.message,
|
||||
stack: error.stack
|
||||
}), {
|
||||
status: 500,
|
||||
headers: {
|
||||
'Content-Type': 'application/json'
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,163 @@
|
||||
/**
|
||||
* 恢复功能检查工具
|
||||
*/
|
||||
|
||||
import { getDatabase } from '../../utils/databaseAdapter.js';
|
||||
|
||||
export async function onRequest(context) {
|
||||
var env = context.env;
|
||||
var url = new URL(context.request.url);
|
||||
var action = url.searchParams.get('action') || 'status';
|
||||
|
||||
try {
|
||||
var db = getDatabase(env);
|
||||
var results = {
|
||||
action: action,
|
||||
timestamp: new Date().toISOString()
|
||||
};
|
||||
|
||||
if (action === 'status') {
|
||||
// 检查当前数据库状态
|
||||
|
||||
// 统计文件数量
|
||||
var fileCount = 0;
|
||||
var cursor = null;
|
||||
while (true) {
|
||||
var response = await db.listFiles({
|
||||
limit: 1000,
|
||||
cursor: cursor
|
||||
});
|
||||
|
||||
if (!response || !response.keys || !Array.isArray(response.keys)) {
|
||||
break;
|
||||
}
|
||||
|
||||
for (var item of response.keys) {
|
||||
if (!item.name.startsWith('manage@') && !item.name.startsWith('chunk_')) {
|
||||
if (item.metadata && item.metadata.TimeStamp) {
|
||||
fileCount++;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
cursor = response.cursor;
|
||||
if (!cursor) break;
|
||||
}
|
||||
|
||||
// 统计设置数量
|
||||
var settingsResponse = await db.listSettings({});
|
||||
var settingsCount = 0;
|
||||
if (settingsResponse && settingsResponse.keys) {
|
||||
settingsCount = settingsResponse.keys.length;
|
||||
}
|
||||
|
||||
// 检查关键设置
|
||||
var keySettings = {};
|
||||
var settingKeys = ['manage@sysConfig@page', 'manage@sysConfig@security'];
|
||||
for (var key of settingKeys) {
|
||||
try {
|
||||
var value = await db.get(key);
|
||||
keySettings[key] = {
|
||||
exists: !!value,
|
||||
length: value ? value.length : 0
|
||||
};
|
||||
} catch (error) {
|
||||
keySettings[key] = {
|
||||
exists: false,
|
||||
error: error.message
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
results.status = {
|
||||
fileCount: fileCount,
|
||||
settingsCount: settingsCount,
|
||||
keySettings: keySettings
|
||||
};
|
||||
|
||||
} else if (action === 'test') {
|
||||
// 测试恢复一个简单的设置
|
||||
var testKey = 'test_restore_' + Date.now();
|
||||
var testValue = 'test_value_' + Date.now();
|
||||
|
||||
try {
|
||||
// 写入测试数据
|
||||
await db.put(testKey, testValue);
|
||||
|
||||
// 读取验证
|
||||
var retrieved = await db.get(testKey);
|
||||
|
||||
// 清理测试数据
|
||||
await db.delete(testKey);
|
||||
|
||||
results.test = {
|
||||
success: true,
|
||||
valueMatch: retrieved === testValue,
|
||||
testKey: testKey,
|
||||
testValue: testValue,
|
||||
retrievedValue: retrieved
|
||||
};
|
||||
} catch (error) {
|
||||
results.test = {
|
||||
success: false,
|
||||
error: error.message
|
||||
};
|
||||
}
|
||||
|
||||
} else if (action === 'sample') {
|
||||
// 提供样本恢复数据
|
||||
// 创建样本数据
|
||||
var testFileKey = "test_file_" + Date.now();
|
||||
var testSettingKey = "test_setting_" + Date.now();
|
||||
var testSettingValue = "test_value_" + Date.now();
|
||||
|
||||
results.sampleData = {
|
||||
timestamp: Date.now(),
|
||||
version: "2.0.2",
|
||||
data: {
|
||||
fileCount: 1,
|
||||
files: {},
|
||||
settings: {}
|
||||
}
|
||||
};
|
||||
|
||||
// 动态添加文件和设置
|
||||
results.sampleData.data.files[testFileKey] = {
|
||||
metadata: {
|
||||
FileName: "test.jpg",
|
||||
FileType: "image/jpeg",
|
||||
FileSize: "0.1",
|
||||
TimeStamp: Date.now(),
|
||||
Channel: "Test",
|
||||
ListType: "None"
|
||||
},
|
||||
value: null
|
||||
};
|
||||
|
||||
results.sampleData.data.settings[testSettingKey] = testSettingValue;
|
||||
|
||||
results.instructions = {
|
||||
usage: "Use this sample data to test restore functionality",
|
||||
endpoint: "/api/manage/sysConfig/backup",
|
||||
method: "POST with action=restore"
|
||||
};
|
||||
}
|
||||
|
||||
return new Response(JSON.stringify(results, null, 2), {
|
||||
headers: {
|
||||
'Content-Type': 'application/json'
|
||||
}
|
||||
});
|
||||
|
||||
} catch (error) {
|
||||
return new Response(JSON.stringify({
|
||||
error: error.message,
|
||||
stack: error.stack
|
||||
}), {
|
||||
status: 500,
|
||||
headers: {
|
||||
'Content-Type': 'application/json'
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,104 @@
|
||||
/**
|
||||
* 恢复样本数据测试
|
||||
*/
|
||||
|
||||
import { getDatabase } from '../../utils/databaseAdapter.js';
|
||||
|
||||
export async function onRequest(context) {
|
||||
var env = context.env;
|
||||
|
||||
try {
|
||||
var db = getDatabase(env);
|
||||
|
||||
// 使用您提供的样本数据
|
||||
var sampleSettings = {
|
||||
"manage@sysConfig@page": "{\"config\":[{\"id\":\"siteTitle\",\"label\":\"网站标题\",\"placeholder\":\"Sanyue ImgHub\",\"category\":\"全局设置\",\"value\":\"ChuZhong ImgHub\"},{\"id\":\"siteIcon\",\"label\":\"网站图标\",\"category\":\"全局设置\"},{\"id\":\"ownerName\",\"label\":\"图床名称\",\"placeholder\":\"Sanyue ImgHub\",\"category\":\"全局设置\",\"value\":\"ChuZhong ImgHub\"},{\"id\":\"logoUrl\",\"label\":\"图床Logo\",\"category\":\"全局设置\"},{\"id\":\"bkInterval\",\"label\":\"背景切换间隔\",\"placeholder\":\"3000\",\"tooltip\":\"单位:毫秒 ms\",\"category\":\"全局设置\"},{\"id\":\"bkOpacity\",\"label\":\"背景图透明度\",\"placeholder\":\"1\",\"tooltip\":\"0-1 之间的小数\",\"category\":\"全局设置\"},{\"id\":\"urlPrefix\",\"label\":\"默认URL前缀\",\"tooltip\":\"自定义URL前缀,如:https://img.a.com/file/,留空则使用当前域名 <br/> 设置后将应用于客户端和管理端\",\"category\":\"全局设置\"},{\"id\":\"announcement\",\"label\":\"公告\",\"tooltip\":\"支持HTML标签\",\"category\":\"客户端设置\"},{\"id\":\"defaultUploadChannel\",\"label\":\"默认上传渠道\",\"type\":\"select\",\"options\":[{\"label\":\"Telegram\",\"value\":\"telegram\"},{\"label\":\"Cloudflare R2\",\"value\":\"cfr2\"},{\"label\":\"S3\",\"value\":\"s3\"}],\"placeholder\":\"telegram\",\"category\":\"客户端设置\"},{\"id\":\"defaultUploadFolder\",\"label\":\"默认上传目录\",\"placeholder\":\"/ 开头的合法目录,不能包含特殊字符, 默认为根目录\",\"category\":\"客户端设置\"},{\"id\":\"defaultUploadNameType\",\"label\":\"默认命名方式\",\"type\":\"select\",\"options\":[{\"label\":\"默认\",\"value\":\"default\"},{\"label\":\"仅前缀\",\"value\":\"index\"},{\"label\":\"仅原名\",\"value\":\"origin\"},{\"label\":\"短链接\",\"value\":\"short\"}],\"placeholder\":\"default\",\"category\":\"客户端设置\"},{\"id\":\"loginBkImg\",\"label\":\"登录页背景图\",\"tooltip\":\"1.填写 bing 使用必应壁纸轮播 <br/> 2.填写 [\\\"url1\\\",\\\"url2\\\"] 使用多张图片轮播 <br/> 3.填写 [\\\"url\\\"] 使用单张图片\",\"category\":\"客户端设置\"},{\"id\":\"uploadBkImg\",\"label\":\"上传页背景图\",\"tooltip\":\"1.填写 bing 使用必应壁纸轮播 <br/> 2.填写 [\\\"url1\\\",\\\"url2\\\"] 使用多张图片轮播 <br/> 3.填写 [\\\"url\\\"] 使用单张图片\",\"category\":\"客户端设置\"},{\"id\":\"footerLink\",\"label\":\"页脚传送门链接\",\"category\":\"客户端设置\"},{\"id\":\"disableFooter\",\"label\":\"隐藏页脚\",\"type\":\"boolean\",\"default\":false,\"category\":\"客户端设置\",\"value\":false},{\"id\":\"adminLoginBkImg\",\"label\":\"登录页背景图\",\"tooltip\":\"1.填写 bing 使用必应壁纸轮播 <br/> 2.填写 [\\\"url1\\\",\\\"url2\\\"] 使用多张图片轮播 <br/> 3.填写 [\\\"url\\\"] 使用单张图片\",\"category\":\"管理端设置\"}]}",
|
||||
"manage@sysConfig@security": "{\"auth\":{\"user\":{\"authCode\":\"ccxy211008\"},\"admin\":{\"adminUsername\":\"chuzhong\",\"adminPassword\":\"ccxy211008\"}},\"upload\":{\"moderate\":{\"enabled\":false,\"channel\":\"default\",\"moderateContentApiKey\":\"\",\"nsfwApiPath\":\"\"}},\"access\":{\"allowedDomains\":\"\",\"whiteListMode\":false}}"
|
||||
};
|
||||
|
||||
var results = {
|
||||
beforeRestore: {},
|
||||
afterRestore: {},
|
||||
restoreResults: [],
|
||||
errors: []
|
||||
};
|
||||
|
||||
// 检查恢复前的状态
|
||||
for (var key in sampleSettings) {
|
||||
try {
|
||||
var beforeValue = await db.get(key);
|
||||
results.beforeRestore[key] = {
|
||||
exists: !!beforeValue,
|
||||
length: beforeValue ? beforeValue.length : 0
|
||||
};
|
||||
} catch (error) {
|
||||
results.beforeRestore[key] = { error: error.message };
|
||||
}
|
||||
}
|
||||
|
||||
// 执行恢复
|
||||
for (var key in sampleSettings) {
|
||||
try {
|
||||
var value = sampleSettings[key];
|
||||
console.log('恢复设置:', key, '长度:', value.length);
|
||||
|
||||
await db.put(key, value);
|
||||
|
||||
// 立即验证
|
||||
var retrieved = await db.get(key);
|
||||
var success = retrieved === value;
|
||||
|
||||
results.restoreResults.push({
|
||||
key: key,
|
||||
success: success,
|
||||
originalLength: value.length,
|
||||
retrievedLength: retrieved ? retrieved.length : 0,
|
||||
matches: success
|
||||
});
|
||||
|
||||
if (!success) {
|
||||
console.error('恢复验证失败:', key);
|
||||
console.error('原始长度:', value.length);
|
||||
console.error('检索长度:', retrieved ? retrieved.length : 0);
|
||||
}
|
||||
|
||||
} catch (error) {
|
||||
results.errors.push({
|
||||
key: key,
|
||||
error: error.message
|
||||
});
|
||||
console.error('恢复失败:', key, error);
|
||||
}
|
||||
}
|
||||
|
||||
// 检查恢复后的状态
|
||||
for (var key in sampleSettings) {
|
||||
try {
|
||||
var afterValue = await db.get(key);
|
||||
results.afterRestore[key] = {
|
||||
exists: !!afterValue,
|
||||
length: afterValue ? afterValue.length : 0
|
||||
};
|
||||
} catch (error) {
|
||||
results.afterRestore[key] = { error: error.message };
|
||||
}
|
||||
}
|
||||
|
||||
return new Response(JSON.stringify(results, null, 2), {
|
||||
headers: {
|
||||
'Content-Type': 'application/json'
|
||||
}
|
||||
});
|
||||
|
||||
} catch (error) {
|
||||
return new Response(JSON.stringify({
|
||||
error: error.message,
|
||||
stack: error.stack
|
||||
}), {
|
||||
status: 500,
|
||||
headers: {
|
||||
'Content-Type': 'application/json'
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,87 @@
|
||||
/**
|
||||
* 设置检查工具
|
||||
*/
|
||||
|
||||
import { getDatabase } from '../../utils/databaseAdapter.js';
|
||||
|
||||
export async function onRequest(context) {
|
||||
var env = context.env;
|
||||
|
||||
try {
|
||||
var db = getDatabase(env);
|
||||
|
||||
var results = {
|
||||
allSettings: [],
|
||||
expectedSettings: [
|
||||
'manage@sysConfig@page',
|
||||
'manage@sysConfig@security',
|
||||
'manage@sysConfig@upload',
|
||||
'manage@sysConfig@others'
|
||||
],
|
||||
missingSettings: [],
|
||||
existingSettings: {}
|
||||
};
|
||||
|
||||
// 列出所有设置
|
||||
var allSettings = await db.listSettings({});
|
||||
results.allSettings = allSettings.keys.map(function(item) {
|
||||
return {
|
||||
key: item.name,
|
||||
hasValue: !!item.value,
|
||||
valueLength: item.value ? item.value.length : 0
|
||||
};
|
||||
});
|
||||
|
||||
// 检查每个预期的设置
|
||||
for (var i = 0; i < results.expectedSettings.length; i++) {
|
||||
var settingKey = results.expectedSettings[i];
|
||||
try {
|
||||
var value = await db.get(settingKey);
|
||||
if (value) {
|
||||
results.existingSettings[settingKey] = {
|
||||
exists: true,
|
||||
length: value.length,
|
||||
preview: value.substring(0, 200) + (value.length > 200 ? '...' : '')
|
||||
};
|
||||
} else {
|
||||
results.missingSettings.push(settingKey);
|
||||
results.existingSettings[settingKey] = {
|
||||
exists: false
|
||||
};
|
||||
}
|
||||
} catch (error) {
|
||||
results.existingSettings[settingKey] = {
|
||||
exists: false,
|
||||
error: error.message
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
// 检查是否有其他manage@开头的设置
|
||||
var manageSettings = await db.listSettings({ prefix: 'manage@' });
|
||||
results.manageSettings = manageSettings.keys.map(function(item) {
|
||||
return {
|
||||
key: item.name,
|
||||
hasValue: !!item.value,
|
||||
valueLength: item.value ? item.value.length : 0
|
||||
};
|
||||
});
|
||||
|
||||
return new Response(JSON.stringify(results, null, 2), {
|
||||
headers: {
|
||||
'Content-Type': 'application/json'
|
||||
}
|
||||
});
|
||||
|
||||
} catch (error) {
|
||||
return new Response(JSON.stringify({
|
||||
error: error.message,
|
||||
stack: error.stack
|
||||
}), {
|
||||
status: 500,
|
||||
headers: {
|
||||
'Content-Type': 'application/json'
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,80 @@
|
||||
/**
|
||||
* 上传功能测试工具
|
||||
*/
|
||||
|
||||
import { getDatabase } from '../../utils/databaseAdapter.js';
|
||||
import { fetchUploadConfig, fetchSecurityConfig } from '../../utils/sysConfig.js';
|
||||
|
||||
export async function onRequest(context) {
|
||||
var env = context.env;
|
||||
|
||||
try {
|
||||
var results = {
|
||||
databaseCheck: null,
|
||||
configCheck: null,
|
||||
uploadConfigCheck: null,
|
||||
securityConfigCheck: null
|
||||
};
|
||||
|
||||
// 检查数据库
|
||||
try {
|
||||
var db = getDatabase(env);
|
||||
results.databaseCheck = {
|
||||
success: true,
|
||||
type: db.constructor.name || 'Unknown'
|
||||
};
|
||||
} catch (error) {
|
||||
results.databaseCheck = {
|
||||
success: false,
|
||||
error: error.message
|
||||
};
|
||||
}
|
||||
|
||||
// 检查上传配置
|
||||
try {
|
||||
var uploadConfig = await fetchUploadConfig(env);
|
||||
results.uploadConfigCheck = {
|
||||
success: true,
|
||||
hasChannels: !!(uploadConfig.telegram && uploadConfig.telegram.channels),
|
||||
channelCount: uploadConfig.telegram ? uploadConfig.telegram.channels.length : 0
|
||||
};
|
||||
} catch (error) {
|
||||
results.uploadConfigCheck = {
|
||||
success: false,
|
||||
error: error.message
|
||||
};
|
||||
}
|
||||
|
||||
// 检查安全配置
|
||||
try {
|
||||
var securityConfig = await fetchSecurityConfig(env);
|
||||
results.securityConfigCheck = {
|
||||
success: true,
|
||||
hasAuth: !!(securityConfig.auth),
|
||||
hasUpload: !!(securityConfig.upload)
|
||||
};
|
||||
} catch (error) {
|
||||
results.securityConfigCheck = {
|
||||
success: false,
|
||||
error: error.message
|
||||
};
|
||||
}
|
||||
|
||||
return new Response(JSON.stringify(results, null, 2), {
|
||||
headers: {
|
||||
'Content-Type': 'application/json'
|
||||
}
|
||||
});
|
||||
|
||||
} catch (error) {
|
||||
return new Response(JSON.stringify({
|
||||
error: error.message,
|
||||
stack: error.stack
|
||||
}), {
|
||||
status: 500,
|
||||
headers: {
|
||||
'Content-Type': 'application/json'
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,19 +1,12 @@
|
||||
import { fetchSecurityConfig } from "../../utils/sysConfig";
|
||||
import { checkKVConfig } from "../../utils/middleware";
|
||||
import { checkDatabaseConfig, errorHandling } from "../../utils/middleware";
|
||||
import { validateApiToken } from "../../utils/tokenValidator";
|
||||
import { getDatabase } from "../../utils/databaseAdapter.js";
|
||||
|
||||
let securityConfig = {}
|
||||
let basicUser = ""
|
||||
let basicPass = ""
|
||||
|
||||
async function errorHandling(context) {
|
||||
try {
|
||||
return await context.next();
|
||||
} catch (err) {
|
||||
return new Response(`${err.message}\n${err.stack}`, { status: 500 });
|
||||
}
|
||||
}
|
||||
|
||||
function basicAuthentication(request) {
|
||||
const Authorization = request.headers.get('Authorization');
|
||||
|
||||
@@ -116,7 +109,8 @@ async function authentication(context) {
|
||||
const pathname = new URL(context.request.url).pathname;
|
||||
const requiredPermission = extractRequiredPermission(pathname);
|
||||
|
||||
const tokenValidation = await validateApiToken(context.request, context.env.img_url, requiredPermission);
|
||||
const db = getDatabase(context.env);
|
||||
const tokenValidation = await validateApiToken(context.request, db, requiredPermission);
|
||||
if (tokenValidation.valid) {
|
||||
// Token验证通过,继续处理请求
|
||||
return context.next();
|
||||
@@ -146,4 +140,22 @@ async function authentication(context) {
|
||||
|
||||
}
|
||||
|
||||
export const onRequest = [checkKVConfig, errorHandling, authentication];
|
||||
// 暂时禁用中间件来排查问题
|
||||
// export const onRequest = [checkDatabaseConfig, errorHandling, authentication];
|
||||
|
||||
// 临时的简单中间件
|
||||
export async function onRequest(context) {
|
||||
try {
|
||||
return await context.next();
|
||||
} catch (error) {
|
||||
console.error('Manage middleware error:', error);
|
||||
return new Response(JSON.stringify({
|
||||
error: 'Manage middleware error: ' + error.message
|
||||
}), {
|
||||
status: 500,
|
||||
headers: {
|
||||
'Content-Type': 'application/json'
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,3 +1,5 @@
|
||||
import { getDatabase } from '../../utils/databaseAdapter.js';
|
||||
|
||||
export async function onRequest(context) {
|
||||
// API Token管理,支持创建、删除、列出Token
|
||||
const {
|
||||
@@ -9,13 +11,13 @@ export async function onRequest(context) {
|
||||
data,
|
||||
} = context;
|
||||
|
||||
const kv = env.img_url
|
||||
const db = getDatabase(env);
|
||||
const url = new URL(request.url)
|
||||
const method = request.method
|
||||
|
||||
// GET - 获取所有Token列表
|
||||
if (method === 'GET') {
|
||||
const tokens = await getApiTokens(kv)
|
||||
const tokens = await getApiTokens(db)
|
||||
return new Response(JSON.stringify(tokens), {
|
||||
headers: {
|
||||
'content-type': 'application/json',
|
||||
@@ -37,7 +39,7 @@ export async function onRequest(context) {
|
||||
})
|
||||
}
|
||||
|
||||
const token = await createApiToken(kv, name, permissions, owner)
|
||||
const token = await createApiToken(db, name, permissions, owner)
|
||||
return new Response(JSON.stringify(token), {
|
||||
headers: {
|
||||
'content-type': 'application/json',
|
||||
@@ -58,7 +60,7 @@ export async function onRequest(context) {
|
||||
})
|
||||
}
|
||||
|
||||
const result = await deleteApiToken(kv, tokenId)
|
||||
const result = await deleteApiToken(db, tokenId)
|
||||
return new Response(JSON.stringify(result), {
|
||||
headers: {
|
||||
'content-type': 'application/json',
|
||||
@@ -80,7 +82,7 @@ export async function onRequest(context) {
|
||||
})
|
||||
}
|
||||
|
||||
const result = await updateApiToken(kv, tokenId, permissions)
|
||||
const result = await updateApiToken(db, tokenId, permissions)
|
||||
return new Response(JSON.stringify(result), {
|
||||
headers: {
|
||||
'content-type': 'application/json',
|
||||
@@ -92,8 +94,8 @@ export async function onRequest(context) {
|
||||
}
|
||||
|
||||
// 获取所有API Token
|
||||
async function getApiTokens(kv) {
|
||||
const settingsStr = await kv.get('manage@sysConfig@security')
|
||||
async function getApiTokens(db) {
|
||||
const settingsStr = await db.get('manage@sysConfig@security')
|
||||
const settings = settingsStr ? JSON.parse(settingsStr) : {}
|
||||
const tokens = settings.apiTokens?.tokens || {}
|
||||
|
||||
@@ -115,8 +117,8 @@ async function getApiTokens(kv) {
|
||||
}
|
||||
|
||||
// 创建新的API Token
|
||||
async function createApiToken(kv, name, permissions, owner) {
|
||||
const settingsStr = await kv.get('manage@sysConfig@security')
|
||||
async function createApiToken(db, name, permissions, owner) {
|
||||
const settingsStr = await db.get('manage@sysConfig@security')
|
||||
const settings = settingsStr ? JSON.parse(settingsStr) : {}
|
||||
|
||||
if (!settings.apiTokens) {
|
||||
@@ -139,8 +141,8 @@ async function createApiToken(kv, name, permissions, owner) {
|
||||
|
||||
settings.apiTokens.tokens[tokenId] = tokenData
|
||||
|
||||
// 保存到KV
|
||||
await kv.put('manage@sysConfig@security', JSON.stringify(settings))
|
||||
// 保存到数据库
|
||||
await db.put('manage@sysConfig@security', JSON.stringify(settings))
|
||||
|
||||
return {
|
||||
id: tokenId,
|
||||
@@ -154,8 +156,8 @@ async function createApiToken(kv, name, permissions, owner) {
|
||||
}
|
||||
|
||||
// 删除API Token
|
||||
async function deleteApiToken(kv, tokenId) {
|
||||
const settingsStr = await kv.get('manage@sysConfig@security')
|
||||
async function deleteApiToken(db, tokenId) {
|
||||
const settingsStr = await db.get('manage@sysConfig@security')
|
||||
const settings = settingsStr ? JSON.parse(settingsStr) : {}
|
||||
|
||||
if (!settings.apiTokens?.tokens?.[tokenId]) {
|
||||
@@ -164,15 +166,15 @@ async function deleteApiToken(kv, tokenId) {
|
||||
|
||||
delete settings.apiTokens.tokens[tokenId]
|
||||
|
||||
// 保存到KV
|
||||
await kv.put('manage@sysConfig@security', JSON.stringify(settings))
|
||||
// 保存到数据库
|
||||
await db.put('manage@sysConfig@security', JSON.stringify(settings))
|
||||
|
||||
return { success: true, message: 'Token 已删除' }
|
||||
}
|
||||
|
||||
// 更新API Token权限
|
||||
async function updateApiToken(kv, tokenId, permissions) {
|
||||
const settingsStr = await kv.get('manage@sysConfig@security')
|
||||
async function updateApiToken(db, tokenId, permissions) {
|
||||
const settingsStr = await db.get('manage@sysConfig@security')
|
||||
const settings = settingsStr ? JSON.parse(settingsStr) : {}
|
||||
|
||||
if (!settings.apiTokens?.tokens?.[tokenId]) {
|
||||
@@ -182,8 +184,8 @@ async function updateApiToken(kv, tokenId, permissions) {
|
||||
settings.apiTokens.tokens[tokenId].permissions = permissions
|
||||
settings.apiTokens.tokens[tokenId].updatedAt = new Date().toISOString()
|
||||
|
||||
// 保存到KV
|
||||
await kv.put('manage@sysConfig@security', JSON.stringify(settings))
|
||||
// 保存到数据库
|
||||
await db.put('manage@sysConfig@security', JSON.stringify(settings))
|
||||
|
||||
return {
|
||||
success: true,
|
||||
@@ -208,8 +210,8 @@ function generateTokenId() {
|
||||
}
|
||||
|
||||
// 根据Token获取权限(供其他API使用)
|
||||
export async function getTokenPermissions(kv, token) {
|
||||
const settingsStr = await kv.get('manage@sysConfig@security')
|
||||
export async function getTokenPermissions(db, token) {
|
||||
const settingsStr = await db.get('manage@sysConfig@security')
|
||||
const settings = settingsStr ? JSON.parse(settingsStr) : {}
|
||||
const tokens = settings.apiTokens?.tokens || {}
|
||||
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { purgeCFCache } from "../../../utils/purgeCache";
|
||||
import { addFileToIndex } from "../../../utils/indexManager.js";
|
||||
import { getDatabase } from "../../../utils/databaseAdapter.js";
|
||||
|
||||
export async function onRequest(context) {
|
||||
// Contents of context object
|
||||
@@ -24,11 +25,12 @@ export async function onRequest(context) {
|
||||
params.path = decodeURIComponent(params.path);
|
||||
|
||||
//read the metadata
|
||||
const value = await env.img_url.getWithMetadata(params.path);
|
||||
const db = getDatabase(env);
|
||||
const value = await db.getWithMetadata(params.path);
|
||||
|
||||
//change the metadata
|
||||
value.metadata.ListType = "Block"
|
||||
await env.img_url.put(params.path,"",{metadata: value.metadata});
|
||||
await db.put(params.path,"",{metadata: value.metadata});
|
||||
const info = JSON.stringify(value.metadata);
|
||||
|
||||
// 清除CDN缓存
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import { getDatabase } from '../../../utils/databaseAdapter.js';
|
||||
|
||||
export async function onRequest(context) {
|
||||
// Contents of context object
|
||||
const {
|
||||
@@ -11,7 +13,7 @@ export async function onRequest(context) {
|
||||
|
||||
try {
|
||||
|
||||
const kv = env.img_url;
|
||||
const kv = getDatabase(env);
|
||||
let list = await kv.get("manage@blockipList");
|
||||
if (list == null) {
|
||||
list = [];
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import { getDatabase } from '../../../utils/databaseAdapter.js';
|
||||
|
||||
export async function onRequest(context) {
|
||||
// Contents of context object
|
||||
const {
|
||||
@@ -9,8 +11,8 @@ export async function onRequest(context) {
|
||||
data, // arbitrary space for passing data between middlewares
|
||||
} = context;
|
||||
try {
|
||||
const kv = env.img_url;
|
||||
const list = await kv.get("manage@blockipList");
|
||||
const db = getDatabase(env);
|
||||
const list = await db.get("manage@blockipList");
|
||||
if (list == null) {
|
||||
return new Response('', { status: 200 });
|
||||
} else {
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import { getDatabase } from '../../../utils/databaseAdapter.js';
|
||||
|
||||
export async function onRequest(context) {
|
||||
// Contents of context object
|
||||
const {
|
||||
@@ -9,7 +11,7 @@ export async function onRequest(context) {
|
||||
data, // arbitrary space for passing data between middlewares
|
||||
} = context;
|
||||
try {
|
||||
const kv = env.img_url;
|
||||
const kv = getDatabase(env);
|
||||
let list = await kv.get("manage@blockipList");
|
||||
if (list == null) {
|
||||
list = [];
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { S3Client, DeleteObjectCommand } from "@aws-sdk/client-s3";
|
||||
import { purgeCFCache } from "../../../utils/purgeCache";
|
||||
import { removeFileFromIndex, batchRemoveFilesFromIndex } from "../../../utils/indexManager.js";
|
||||
import { getDatabase } from '../../../utils/databaseAdapter.js';
|
||||
|
||||
export async function onRequest(context) {
|
||||
const { request, env, params, waitUntil } = context;
|
||||
@@ -104,7 +105,8 @@ export async function onRequest(context) {
|
||||
async function deleteFile(env, fileId, cdnUrl, url) {
|
||||
try {
|
||||
// 读取图片信息
|
||||
const img = await env.img_url.getWithMetadata(fileId);
|
||||
const db = getDatabase(env);
|
||||
const img = await db.getWithMetadata(fileId);
|
||||
|
||||
// 如果是R2渠道的图片,需要删除R2中对应的图片
|
||||
if (img.metadata?.Channel === 'CloudflareR2') {
|
||||
@@ -117,8 +119,8 @@ async function deleteFile(env, fileId, cdnUrl, url) {
|
||||
await deleteS3File(img);
|
||||
}
|
||||
|
||||
// 删除KV存储中的记录
|
||||
await env.img_url.delete(fileId);
|
||||
// 删除数据库中的记录
|
||||
await db.delete(fileId);
|
||||
|
||||
// 清除CDN缓存
|
||||
await purgeCFCache(env, cdnUrl);
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { readIndex, mergeOperationsToIndex, deleteAllOperations, rebuildIndex,
|
||||
getIndexInfo, getIndexStorageStats } from '../../utils/indexManager.js';
|
||||
import { readIndex, getIndexInfo, rebuildIndex, getIndexStorageStats } from '../../utils/indexManager.js';
|
||||
import { getDatabase } from '../../utils/databaseAdapter.js';
|
||||
|
||||
export async function onRequest(context) {
|
||||
const { request, waitUntil } = context;
|
||||
@@ -41,24 +41,6 @@ export async function onRequest(context) {
|
||||
});
|
||||
}
|
||||
|
||||
// 特殊操作:合并挂起的原子操作到索引
|
||||
if (action === 'merge-operations') {
|
||||
waitUntil(mergeOperationsToIndex(context));
|
||||
|
||||
return new Response('Operations merged into index asynchronously', {
|
||||
headers: { "Content-Type": "text/plain" }
|
||||
});
|
||||
}
|
||||
|
||||
// 特殊操作:清除所有原子操作
|
||||
if (action === 'delete-operations') {
|
||||
waitUntil(deleteAllOperations(context));
|
||||
|
||||
return new Response('All operations deleted asynchronously', {
|
||||
headers: { "Content-Type": "text/plain" }
|
||||
});
|
||||
}
|
||||
|
||||
// 特殊操作:获取索引存储信息
|
||||
if (action === 'index-storage-stats') {
|
||||
const stats = await getIndexStorageStats(context);
|
||||
@@ -153,28 +135,37 @@ async function getAllFileRecords(env, dir) {
|
||||
const allRecords = [];
|
||||
let cursor = null;
|
||||
|
||||
while (true) {
|
||||
const response = await env.img_url.list({
|
||||
prefix: dir,
|
||||
limit: 1000,
|
||||
cursor: cursor
|
||||
});
|
||||
try {
|
||||
const db = getDatabase(env);
|
||||
|
||||
cursor = response.cursor;
|
||||
while (true) {
|
||||
const response = await db.list({
|
||||
prefix: dir,
|
||||
limit: 1000,
|
||||
cursor: cursor
|
||||
});
|
||||
|
||||
for (const item of response.keys) {
|
||||
// 跳过管理相关的键
|
||||
if (item.name.startsWith('manage@') || item.name.startsWith('chunk_')) {
|
||||
continue;
|
||||
// 检查响应格式
|
||||
if (!response || !response.keys || !Array.isArray(response.keys)) {
|
||||
console.error('Invalid response from database list:', response);
|
||||
break;
|
||||
}
|
||||
|
||||
// 跳过没有元数据的文件
|
||||
if (!item.metadata || !item.metadata.TimeStamp) {
|
||||
continue;
|
||||
}
|
||||
cursor = response.cursor;
|
||||
|
||||
allRecords.push(item);
|
||||
}
|
||||
for (const item of response.keys) {
|
||||
// 跳过管理相关的键
|
||||
if (item.name.startsWith('manage@') || item.name.startsWith('chunk_')) {
|
||||
continue;
|
||||
}
|
||||
|
||||
// 跳过没有元数据的文件
|
||||
if (!item.metadata || !item.metadata.TimeStamp) {
|
||||
continue;
|
||||
}
|
||||
|
||||
allRecords.push(item);
|
||||
}
|
||||
|
||||
if (!cursor) break;
|
||||
|
||||
@@ -201,4 +192,15 @@ async function getAllFileRecords(env, dir) {
|
||||
totalCount: allRecords.length,
|
||||
returnedCount: filteredRecords.length
|
||||
};
|
||||
|
||||
} catch (error) {
|
||||
console.error('Error in getAllFileRecords:', error);
|
||||
return {
|
||||
files: [],
|
||||
directories: [],
|
||||
totalCount: 0,
|
||||
returnedCount: 0,
|
||||
error: error.message
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,262 @@
|
||||
/**
|
||||
* 数据迁移工具
|
||||
* 用于将KV数据迁移到D1数据库
|
||||
*/
|
||||
|
||||
import { getDatabase, checkDatabaseConfig } from '../../utils/databaseAdapter.js';
|
||||
|
||||
export async function onRequest(context) {
|
||||
const { request, env } = context;
|
||||
const url = new URL(request.url);
|
||||
const action = url.searchParams.get('action');
|
||||
|
||||
try {
|
||||
switch (action) {
|
||||
case 'check':
|
||||
return await handleCheck(env);
|
||||
case 'migrate':
|
||||
return await handleMigrate(env);
|
||||
case 'status':
|
||||
return await handleStatus(env);
|
||||
default:
|
||||
return new Response(JSON.stringify({ error: '不支持的操作' }), {
|
||||
status: 400,
|
||||
headers: { 'Content-Type': 'application/json' }
|
||||
});
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('迁移操作错误:', error);
|
||||
return new Response(JSON.stringify({ error: '操作失败: ' + error.message }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json' }
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// 检查迁移环境
|
||||
async function handleCheck(env) {
|
||||
const dbConfig = checkDatabaseConfig(env);
|
||||
|
||||
const result = {
|
||||
hasKV: dbConfig.hasKV,
|
||||
hasD1: dbConfig.hasD1,
|
||||
canMigrate: dbConfig.hasKV && dbConfig.hasD1,
|
||||
currentDatabase: dbConfig.usingD1 ? 'D1' : (dbConfig.usingKV ? 'KV' : 'None'),
|
||||
message: ''
|
||||
};
|
||||
|
||||
if (!result.canMigrate) {
|
||||
if (!result.hasKV) {
|
||||
result.message = '未找到KV存储,无法进行迁移';
|
||||
} else if (!result.hasD1) {
|
||||
result.message = '未找到D1数据库,请先配置D1数据库';
|
||||
}
|
||||
} else {
|
||||
result.message = '环境检查通过,可以开始迁移';
|
||||
}
|
||||
|
||||
return new Response(JSON.stringify(result), {
|
||||
headers: { 'Content-Type': 'application/json' }
|
||||
});
|
||||
}
|
||||
|
||||
// 执行迁移
|
||||
async function handleMigrate(env) {
|
||||
const dbConfig = checkDatabaseConfig(env);
|
||||
|
||||
if (!dbConfig.hasKV || !dbConfig.hasD1) {
|
||||
return new Response(JSON.stringify({
|
||||
error: '迁移环境不满足要求',
|
||||
hasKV: dbConfig.hasKV,
|
||||
hasD1: dbConfig.hasD1
|
||||
}), {
|
||||
status: 400,
|
||||
headers: { 'Content-Type': 'application/json' }
|
||||
});
|
||||
}
|
||||
|
||||
const migrationResult = {
|
||||
startTime: new Date().toISOString(),
|
||||
files: { migrated: 0, failed: 0, errors: [] },
|
||||
settings: { migrated: 0, failed: 0, errors: [] },
|
||||
operations: { migrated: 0, failed: 0, errors: [] },
|
||||
status: 'running'
|
||||
};
|
||||
|
||||
try {
|
||||
// 1. 迁移文件数据
|
||||
console.log('开始迁移文件数据...');
|
||||
await migrateFiles(env, migrationResult);
|
||||
|
||||
// 2. 迁移系统设置
|
||||
console.log('开始迁移系统设置...');
|
||||
await migrateSettings(env, migrationResult);
|
||||
|
||||
// 3. 迁移索引操作
|
||||
console.log('开始迁移索引操作...');
|
||||
await migrateIndexOperations(env, migrationResult);
|
||||
|
||||
migrationResult.status = 'completed';
|
||||
migrationResult.endTime = new Date().toISOString();
|
||||
|
||||
} catch (error) {
|
||||
migrationResult.status = 'failed';
|
||||
migrationResult.error = error.message;
|
||||
migrationResult.endTime = new Date().toISOString();
|
||||
}
|
||||
|
||||
return new Response(JSON.stringify(migrationResult), {
|
||||
headers: { 'Content-Type': 'application/json' }
|
||||
});
|
||||
}
|
||||
|
||||
// 迁移文件数据
|
||||
async function migrateFiles(env, result) {
|
||||
const db = getDatabase(env);
|
||||
let cursor = null;
|
||||
const batchSize = 100;
|
||||
|
||||
while (true) {
|
||||
const response = await getDatabase(env).list({
|
||||
limit: batchSize,
|
||||
cursor: cursor
|
||||
});
|
||||
|
||||
for (const item of response.keys) {
|
||||
// 跳过管理相关的键
|
||||
if (item.name.startsWith('manage@') || item.name.startsWith('chunk_')) {
|
||||
continue;
|
||||
}
|
||||
|
||||
try {
|
||||
const fileData = await getDatabase(env).getWithMetadata(item.name);
|
||||
|
||||
if (fileData && fileData.metadata) {
|
||||
await db.putFile(item.name, fileData.value || '', {
|
||||
metadata: fileData.metadata
|
||||
});
|
||||
result.files.migrated++;
|
||||
}
|
||||
} catch (error) {
|
||||
result.files.failed++;
|
||||
result.files.errors.push({
|
||||
file: item.name,
|
||||
error: error.message
|
||||
});
|
||||
console.error(`迁移文件 ${item.name} 失败:`, error);
|
||||
}
|
||||
}
|
||||
|
||||
cursor = response.cursor;
|
||||
if (!cursor) break;
|
||||
|
||||
// 添加延迟避免过载
|
||||
await new Promise(resolve => setTimeout(resolve, 10));
|
||||
}
|
||||
}
|
||||
|
||||
// 迁移系统设置
|
||||
async function migrateSettings(env, result) {
|
||||
const db = getDatabase(env);
|
||||
|
||||
const settingsList = await getDatabase(env).list({ prefix: 'manage@' });
|
||||
|
||||
for (const item of settingsList.keys) {
|
||||
// 跳过索引相关的键
|
||||
if (item.name.startsWith('manage@index')) {
|
||||
continue;
|
||||
}
|
||||
|
||||
try {
|
||||
const value = await getDatabase(env).get(item.name);
|
||||
if (value) {
|
||||
await db.putSetting(item.name, value);
|
||||
result.settings.migrated++;
|
||||
}
|
||||
} catch (error) {
|
||||
result.settings.failed++;
|
||||
result.settings.errors.push({
|
||||
setting: item.name,
|
||||
error: error.message
|
||||
});
|
||||
console.error(`迁移设置 ${item.name} 失败:`, error);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 迁移索引操作
|
||||
async function migrateIndexOperations(env, result) {
|
||||
const db = getDatabase(env);
|
||||
const operationPrefix = 'manage@index@operation_';
|
||||
|
||||
const operationsList = await getDatabase(env).list({ prefix: operationPrefix });
|
||||
|
||||
for (const item of operationsList.keys) {
|
||||
try {
|
||||
const operationData = await getDatabase(env).get(item.name);
|
||||
if (operationData) {
|
||||
const operation = JSON.parse(operationData);
|
||||
const operationId = item.name.replace(operationPrefix, '');
|
||||
|
||||
await db.putIndexOperation(operationId, operation);
|
||||
result.operations.migrated++;
|
||||
}
|
||||
} catch (error) {
|
||||
result.operations.failed++;
|
||||
result.operations.errors.push({
|
||||
operation: item.name,
|
||||
error: error.message
|
||||
});
|
||||
console.error(`迁移操作 ${item.name} 失败:`, error);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 获取迁移状态
|
||||
async function handleStatus(env) {
|
||||
const dbConfig = checkDatabaseConfig(env);
|
||||
|
||||
let fileCount = { kv: 0, d1: 0 };
|
||||
let settingCount = { kv: 0, d1: 0 };
|
||||
|
||||
try {
|
||||
// 统计KV中的数据
|
||||
if (dbConfig.hasKV) {
|
||||
const kvFiles = await getDatabase(env).list({ limit: 1000 });
|
||||
fileCount.kv = kvFiles.keys.filter(k =>
|
||||
!k.name.startsWith('manage@') && !k.name.startsWith('chunk_')
|
||||
).length;
|
||||
|
||||
const kvSettings = await getDatabase(env).list({ prefix: 'manage@', limit: 1000 });
|
||||
settingCount.kv = kvSettings.keys.filter(k =>
|
||||
!k.name.startsWith('manage@index')
|
||||
).length;
|
||||
}
|
||||
|
||||
// 统计D1中的数据
|
||||
if (dbConfig.hasD1) {
|
||||
const db = getDatabase(env);
|
||||
|
||||
const fileCountStmt = db.db.prepare('SELECT COUNT(*) as count FROM files');
|
||||
const fileResult = await fileCountStmt.first();
|
||||
fileCount.d1 = fileResult.count;
|
||||
|
||||
const settingCountStmt = db.db.prepare('SELECT COUNT(*) as count FROM settings');
|
||||
const settingResult = await settingCountStmt.first();
|
||||
settingCount.d1 = settingResult.count;
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('获取状态失败:', error);
|
||||
}
|
||||
|
||||
return new Response(JSON.stringify({
|
||||
database: dbConfig,
|
||||
counts: {
|
||||
files: fileCount,
|
||||
settings: settingCount
|
||||
},
|
||||
migrationNeeded: fileCount.kv > 0 && fileCount.d1 === 0
|
||||
}), {
|
||||
headers: { 'Content-Type': 'application/json' }
|
||||
});
|
||||
}
|
||||
@@ -1,7 +1,7 @@
|
||||
import { S3Client, CopyObjectCommand, DeleteObjectCommand } from "@aws-sdk/client-s3";
|
||||
import { purgeCFCache } from "../../../utils/purgeCache";
|
||||
import { moveFileInIndex, batchMoveFilesInIndex } from "../../../utils/indexManager.js";
|
||||
|
||||
import { getDatabase } from '../../../utils/databaseAdapter.js';
|
||||
export async function onRequest(context) {
|
||||
const { request, env, params, waitUntil } = context;
|
||||
|
||||
@@ -125,7 +125,7 @@ export async function onRequest(context) {
|
||||
async function moveFile(env, fileId, newFileId, cdnUrl, url) {
|
||||
try {
|
||||
// 读取图片信息
|
||||
const img = await env.img_url.getWithMetadata(fileId);
|
||||
const img = await getDatabase(env).getWithMetadata(fileId);
|
||||
|
||||
// 如果是R2渠道的图片,需要移动R2中对应的图片
|
||||
if (img.metadata?.Channel === 'CloudflareR2') {
|
||||
@@ -168,8 +168,8 @@ async function moveFile(env, fileId, newFileId, cdnUrl, url) {
|
||||
img.metadata.Folder = folderPath;
|
||||
|
||||
// 更新KV存储
|
||||
await env.img_url.put(newFileId, img.value, { metadata: img.metadata });
|
||||
await env.img_url.delete(fileId);
|
||||
await getDatabase(env).put(newFileId, img.value, { metadata: img.metadata });
|
||||
await getDatabase(env).delete(fileId);
|
||||
|
||||
// 清除CDN缓存
|
||||
await purgeCFCache(env, cdnUrl);
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { readIndex } from '../../../utils/indexManager.js';
|
||||
import { getDatabase } from '../../../utils/databaseAdapter.js';
|
||||
|
||||
export async function onRequest(context) {
|
||||
const { request, env } = context;
|
||||
@@ -41,23 +42,54 @@ async function handleBackup(context) {
|
||||
}
|
||||
};
|
||||
|
||||
// 首先从索引中读取所有文件信息
|
||||
const indexResult = await readIndex(context, {
|
||||
count: -1, // 获取所有文件
|
||||
start: 0,
|
||||
includeSubdirFiles: true // 包含子目录下的文件
|
||||
});
|
||||
backupData.data.fileCount = indexResult.files.length;
|
||||
// 直接从数据库读取所有文件信息,不依赖索引
|
||||
const db = getDatabase(env);
|
||||
let allFiles = [];
|
||||
let cursor = null;
|
||||
|
||||
// 分批获取所有文件
|
||||
while (true) {
|
||||
const response = await db.listFiles({
|
||||
limit: 1000,
|
||||
cursor: cursor
|
||||
});
|
||||
|
||||
if (!response || !response.keys || !Array.isArray(response.keys)) {
|
||||
break;
|
||||
}
|
||||
|
||||
for (const item of response.keys) {
|
||||
// 跳过管理相关的键和分块数据
|
||||
if (item.name.startsWith('manage@') || item.name.startsWith('chunk_')) {
|
||||
continue;
|
||||
}
|
||||
|
||||
// 跳过没有元数据的文件
|
||||
if (!item.metadata || !item.metadata.TimeStamp) {
|
||||
continue;
|
||||
}
|
||||
|
||||
allFiles.push({
|
||||
id: item.name,
|
||||
metadata: item.metadata
|
||||
});
|
||||
}
|
||||
|
||||
cursor = response.cursor;
|
||||
if (!cursor) break;
|
||||
}
|
||||
|
||||
backupData.data.fileCount = allFiles.length;
|
||||
|
||||
// 备份文件数据
|
||||
for (const file of indexResult.files) {
|
||||
for (const file of allFiles) {
|
||||
const fileId = file.id;
|
||||
const metadata = file.metadata;
|
||||
|
||||
// 对于TelegramNew渠道且IsChunked为true的文件,需要从KV读取其值
|
||||
// 对于TelegramNew渠道且IsChunked为true的文件,需要从数据库读取其值
|
||||
if (metadata.Channel === 'TelegramNew' && metadata.IsChunked === true) {
|
||||
try {
|
||||
const fileData = await env.img_url.getWithMetadata(fileId);
|
||||
const fileData = await db.getWithMetadata(fileId);
|
||||
backupData.data.files[fileId] = {
|
||||
metadata: metadata,
|
||||
value: fileData.value
|
||||
@@ -80,17 +112,32 @@ async function handleBackup(context) {
|
||||
}
|
||||
|
||||
// 备份系统设置
|
||||
const settingsList = await env.img_url.list({ prefix: 'manage@' });
|
||||
for (const key of settingsList.keys) {
|
||||
// db 已经在上面定义了
|
||||
|
||||
// 备份所有设置,不仅仅是manage@开头的
|
||||
const allSettingsList = await db.listSettings({});
|
||||
for (const key of allSettingsList.keys) {
|
||||
// 忽略索引文件
|
||||
if (key.name.startsWith('manage@index')) continue;
|
||||
|
||||
const setting = await env.img_url.get(key.name);
|
||||
const setting = key.value;
|
||||
if (setting) {
|
||||
backupData.data.settings[key.name] = setting;
|
||||
}
|
||||
}
|
||||
|
||||
// 额外确保备份manage@开头的设置
|
||||
const manageSettingsList = await db.listSettings({ prefix: 'manage@' });
|
||||
for (const key of manageSettingsList.keys) {
|
||||
// 忽略索引文件
|
||||
if (key.name.startsWith('manage@index')) continue;
|
||||
|
||||
const setting = key.value;
|
||||
if (setting && !backupData.data.settings[key.name]) {
|
||||
backupData.data.settings[key.name] = setting;
|
||||
}
|
||||
}
|
||||
|
||||
const backupJson = JSON.stringify(backupData, null, 2);
|
||||
|
||||
return new Response(backupJson, {
|
||||
@@ -131,30 +178,56 @@ async function handleRestore(request, env) {
|
||||
let restoredSettings = 0;
|
||||
|
||||
// 恢复文件数据
|
||||
for (const [key, fileData] of Object.entries(backupData.data.files)) {
|
||||
try {
|
||||
if (fileData.value) {
|
||||
// 对于有value的文件(如telegram分块文件),恢复完整数据
|
||||
await env.img_url.put(key, fileData.value, {
|
||||
metadata: fileData.metadata
|
||||
});
|
||||
} else if (fileData.metadata) {
|
||||
// 只恢复元数据
|
||||
await env.img_url.put(key, '', {
|
||||
metadata: fileData.metadata
|
||||
});
|
||||
const db = getDatabase(env);
|
||||
const fileEntries = Object.entries(backupData.data.files);
|
||||
const batchSize = 50; // 批量处理,避免超时
|
||||
|
||||
for (let i = 0; i < fileEntries.length; i += batchSize) {
|
||||
const batch = fileEntries.slice(i, i + batchSize);
|
||||
|
||||
for (const [key, fileData] of batch) {
|
||||
try {
|
||||
if (fileData.value) {
|
||||
// 对于有value的文件(如telegram分块文件),恢复完整数据
|
||||
await db.put(key, fileData.value, {
|
||||
metadata: fileData.metadata
|
||||
});
|
||||
} else if (fileData.metadata) {
|
||||
// 只恢复元数据
|
||||
await db.put(key, '', {
|
||||
metadata: fileData.metadata
|
||||
});
|
||||
}
|
||||
restoredFiles++;
|
||||
} catch (error) {
|
||||
console.error(`恢复文件 ${key} 失败:`, error);
|
||||
}
|
||||
restoredFiles++;
|
||||
} catch (error) {
|
||||
console.error(`恢复文件 ${key} 失败:`, error);
|
||||
}
|
||||
|
||||
// 每批处理后短暂暂停,避免过载
|
||||
if (i + batchSize < fileEntries.length) {
|
||||
await new Promise(resolve => setTimeout(resolve, 10));
|
||||
}
|
||||
}
|
||||
|
||||
// 恢复系统设置
|
||||
for (const [key, value] of Object.entries(backupData.data.settings)) {
|
||||
const settingEntries = Object.entries(backupData.data.settings);
|
||||
console.log(`开始恢复 ${settingEntries.length} 个设置`);
|
||||
|
||||
for (const [key, value] of settingEntries) {
|
||||
try {
|
||||
await env.img_url.put(key, value);
|
||||
restoredSettings++;
|
||||
console.log(`恢复设置: ${key}, 长度: ${value.length}`);
|
||||
await db.put(key, value);
|
||||
|
||||
// 验证是否成功保存
|
||||
const retrieved = await db.get(key);
|
||||
if (retrieved === value) {
|
||||
restoredSettings++;
|
||||
console.log(`设置 ${key} 恢复成功`);
|
||||
} else {
|
||||
console.error(`设置 ${key} 恢复后验证失败`);
|
||||
console.error(`原始长度: ${value.length}, 检索长度: ${retrieved ? retrieved.length : 'null'}`);
|
||||
}
|
||||
} catch (error) {
|
||||
console.error(`恢复设置 ${key} 失败:`, error);
|
||||
}
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import { getDatabase } from '../../../utils/databaseAdapter.js';
|
||||
|
||||
export async function onRequest(context) {
|
||||
// 其他设置相关,GET方法读取设置,POST方法保存设置
|
||||
const {
|
||||
@@ -9,11 +11,11 @@ export async function onRequest(context) {
|
||||
data, // arbitrary space for passing data between middlewares
|
||||
} = context;
|
||||
|
||||
const kv = env.img_url
|
||||
const db = getDatabase(env);
|
||||
|
||||
// GET读取设置
|
||||
if (request.method === 'GET') {
|
||||
const settings = await getOthersConfig(kv, env)
|
||||
const settings = await getOthersConfig(db, env)
|
||||
|
||||
return new Response(JSON.stringify(settings), {
|
||||
headers: {
|
||||
@@ -27,8 +29,8 @@ export async function onRequest(context) {
|
||||
const body = await request.json()
|
||||
const settings = body
|
||||
|
||||
// 写入 KV
|
||||
await kv.put('manage@sysConfig@others', JSON.stringify(settings))
|
||||
// 写入数据库
|
||||
await db.put('manage@sysConfig@others', JSON.stringify(settings))
|
||||
|
||||
return new Response(JSON.stringify(settings), {
|
||||
headers: {
|
||||
@@ -39,10 +41,10 @@ export async function onRequest(context) {
|
||||
|
||||
}
|
||||
|
||||
export async function getOthersConfig(kv, env) {
|
||||
export async function getOthersConfig(db, env) {
|
||||
const settings = {}
|
||||
// 读取KV中的设置
|
||||
const settingsStr = await kv.get('manage@sysConfig@others')
|
||||
// 读取数据库中的设置
|
||||
const settingsStr = await db.get('manage@sysConfig@others')
|
||||
const settingsKV = settingsStr ? JSON.parse(settingsStr) : {}
|
||||
|
||||
// 远端遥测
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import { getDatabase } from '../../../utils/databaseAdapter.js';
|
||||
|
||||
export async function onRequest(context) {
|
||||
// 页面设置相关,GET方法读取设置,POST方法保存设置
|
||||
const {
|
||||
@@ -9,11 +11,11 @@ export async function onRequest(context) {
|
||||
data, // arbitrary space for passing data between middlewares
|
||||
} = context;
|
||||
|
||||
const kv = env.img_url
|
||||
const db = getDatabase(env);
|
||||
|
||||
// GET读取设置
|
||||
if (request.method === 'GET') {
|
||||
const settings = await getPageConfig(kv, env)
|
||||
const settings = await getPageConfig(db, env)
|
||||
|
||||
return new Response(JSON.stringify(settings), {
|
||||
headers: {
|
||||
@@ -26,8 +28,8 @@ export async function onRequest(context) {
|
||||
if (request.method === 'POST') {
|
||||
const body = await request.json()
|
||||
const settings = body
|
||||
// 写入 KV
|
||||
await kv.put('manage@sysConfig@page', JSON.stringify(settings))
|
||||
// 写入数据库
|
||||
await db.put('manage@sysConfig@page', JSON.stringify(settings))
|
||||
|
||||
return new Response(JSON.stringify(settings), {
|
||||
headers: {
|
||||
@@ -38,10 +40,10 @@ export async function onRequest(context) {
|
||||
|
||||
}
|
||||
|
||||
export async function getPageConfig(kv, env) {
|
||||
export async function getPageConfig(db, env) {
|
||||
const settings = {}
|
||||
// 读取KV中的设置
|
||||
const settingsStr = await kv.get('manage@sysConfig@page')
|
||||
// 读取数据库中的设置
|
||||
const settingsStr = await db.get('manage@sysConfig@page')
|
||||
const settingsKV = settingsStr ? JSON.parse(settingsStr) : {}
|
||||
|
||||
const config = []
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import { getDatabase } from '../../../utils/databaseAdapter.js';
|
||||
|
||||
export async function onRequest(context) {
|
||||
// 安全设置相关,GET方法读取设置,POST方法保存设置
|
||||
const {
|
||||
@@ -9,11 +11,11 @@ export async function onRequest(context) {
|
||||
data, // arbitrary space for passing data between middlewares
|
||||
} = context;
|
||||
|
||||
const kv = env.img_url
|
||||
const db = getDatabase(env);
|
||||
|
||||
// GET读取设置
|
||||
if (request.method === 'GET') {
|
||||
const settings = await getSecurityConfig(kv, env)
|
||||
const settings = await getSecurityConfig(db, env)
|
||||
|
||||
return new Response(JSON.stringify(settings), {
|
||||
headers: {
|
||||
@@ -27,8 +29,8 @@ export async function onRequest(context) {
|
||||
const body = await request.json()
|
||||
const settings = body
|
||||
|
||||
// 写入 KV
|
||||
await kv.put('manage@sysConfig@security', JSON.stringify(settings))
|
||||
// 写入数据库
|
||||
await db.put('manage@sysConfig@security', JSON.stringify(settings))
|
||||
|
||||
return new Response(JSON.stringify(settings), {
|
||||
headers: {
|
||||
@@ -39,10 +41,10 @@ export async function onRequest(context) {
|
||||
|
||||
}
|
||||
|
||||
export async function getSecurityConfig(kv, env) {
|
||||
export async function getSecurityConfig(db, env) {
|
||||
const settings = {}
|
||||
// 读取KV中的设置
|
||||
const settingsStr = await kv.get('manage@sysConfig@security')
|
||||
// 读取数据库中的设置
|
||||
const settingsStr = await db.get('manage@sysConfig@security')
|
||||
const settingsKV = settingsStr ? JSON.parse(settingsStr) : {}
|
||||
|
||||
// 认证管理
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import { getDatabase } from '../../../utils/databaseAdapter.js';
|
||||
|
||||
export async function onRequest(context) {
|
||||
// 上传设置相关,GET方法读取设置,POST方法保存设置
|
||||
const {
|
||||
@@ -9,11 +11,11 @@ export async function onRequest(context) {
|
||||
data, // arbitrary space for passing data between middlewares
|
||||
} = context;
|
||||
|
||||
const kv = env.img_url
|
||||
const db = getDatabase(env);
|
||||
|
||||
// GET读取设置
|
||||
if (request.method === 'GET') {
|
||||
const settings = await getUploadConfig(kv, env)
|
||||
const settings = await getUploadConfig(db, env)
|
||||
|
||||
return new Response(JSON.stringify(settings), {
|
||||
headers: {
|
||||
@@ -27,8 +29,8 @@ export async function onRequest(context) {
|
||||
const body = await request.json()
|
||||
const settings = body
|
||||
|
||||
// 写入 KV
|
||||
await kv.put('manage@sysConfig@upload', JSON.stringify(settings))
|
||||
// 写入数据库
|
||||
await db.put('manage@sysConfig@upload', JSON.stringify(settings))
|
||||
|
||||
return new Response(JSON.stringify(settings), {
|
||||
headers: {
|
||||
@@ -39,10 +41,10 @@ export async function onRequest(context) {
|
||||
|
||||
}
|
||||
|
||||
export async function getUploadConfig(kv, env) {
|
||||
export async function getUploadConfig(db, env) {
|
||||
const settings = {}
|
||||
// 读取KV中的设置
|
||||
const settingsStr = await kv.get('manage@sysConfig@upload')
|
||||
// 读取数据库中的设置
|
||||
const settingsStr = await db.get('manage@sysConfig@upload')
|
||||
const settingsKV = settingsStr ? JSON.parse(settingsStr) : {}
|
||||
|
||||
// =====================读取tg渠道配置=====================
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { purgeCFCache } from "../../../utils/purgeCache";
|
||||
import { addFileToIndex } from "../../../utils/indexManager.js";
|
||||
import { getDatabase } from "../../../utils/databaseAdapter.js";
|
||||
|
||||
export async function onRequest(context) {
|
||||
// Contents of context object
|
||||
@@ -24,11 +25,12 @@ export async function onRequest(context) {
|
||||
params.path = decodeURIComponent(params.path);
|
||||
|
||||
//read the metadata
|
||||
const value = await env.img_url.getWithMetadata(params.path);
|
||||
const db = getDatabase(env);
|
||||
const value = await db.getWithMetadata(params.path);
|
||||
|
||||
//change the metadata
|
||||
value.metadata.ListType = "White"
|
||||
await env.img_url.put(params.path,"",{metadata: value.metadata});
|
||||
await db.put(params.path,"",{metadata: value.metadata});
|
||||
const info = JSON.stringify(value.metadata);
|
||||
|
||||
// 清除CDN缓存
|
||||
|
||||
@@ -1,8 +1,9 @@
|
||||
import { S3Client, GetObjectCommand } from "@aws-sdk/client-s3";
|
||||
import { fetchSecurityConfig } from "../utils/sysConfig";
|
||||
import { TelegramAPI } from "../utils/telegramAPI";
|
||||
import { setCommonHeaders, setRangeHeaders, handleHeadRequest, getFileContent, isTgChannel,
|
||||
import { setCommonHeaders, setRangeHeaders, handleHeadRequest, getFileContent, isTgChannel,
|
||||
returnWithCheck, return404, isDomainAllowed } from './fileTools';
|
||||
import { getDatabase } from '../utils/databaseAdapter.js';
|
||||
|
||||
|
||||
export async function onRequest(context) { // Contents of context object
|
||||
@@ -39,8 +40,9 @@ export async function onRequest(context) { // Contents of context object
|
||||
return await returnBlockImg(url);
|
||||
}
|
||||
|
||||
// 从KV中获取图片记录
|
||||
const imgRecord = await env.img_url.getWithMetadata(fileId);
|
||||
// 从数据库中获取图片记录
|
||||
const db = getDatabase(env);
|
||||
const imgRecord = await db.getWithMetadata(fileId);
|
||||
if (!imgRecord) {
|
||||
return new Response('Error: Image Not Found', { status: 404 });
|
||||
}
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
import { createResponse, getUploadIp, getIPAddress, selectConsistentChannel, buildUniqueFileId, endUpload } from './uploadTools';
|
||||
import { retryFailedChunks, cleanupFailedMultipartUploads, checkChunkUploadStatuses, cleanupChunkData, cleanupUploadSession } from './chunkUpload';
|
||||
import { S3Client, CompleteMultipartUploadCommand } from "@aws-sdk/client-s3";
|
||||
import { getDatabase } from '../utils/databaseAdapter.js';
|
||||
|
||||
// 处理分块合并
|
||||
export async function handleChunkMerge(context) {
|
||||
@@ -23,8 +24,9 @@ export async function handleChunkMerge(context) {
|
||||
}
|
||||
|
||||
// 验证上传会话
|
||||
const db = getDatabase(env);
|
||||
const sessionKey = `upload_session_${uploadId}`;
|
||||
const sessionData = await env.img_url.get(sessionKey);
|
||||
const sessionData = await db.get(sessionKey);
|
||||
if (!sessionData) {
|
||||
return createResponse('Error: Invalid or expired upload session', { status: 400 });
|
||||
}
|
||||
@@ -94,7 +96,7 @@ async function startMerge(context, uploadId, totalChunks, originalFileName, orig
|
||||
|
||||
// 存储合并状态
|
||||
const statusKey = `merge_status_${uploadId}`;
|
||||
await env.img_url.put(statusKey, JSON.stringify(mergeStatus), {
|
||||
await db.put(statusKey, JSON.stringify(mergeStatus), {
|
||||
expirationTtl: 3600 // 1小时过期
|
||||
});
|
||||
|
||||
@@ -306,7 +308,8 @@ async function handleChannelBasedMerge(context, uploadId, totalChunks, originalF
|
||||
// 对于仍在上传的分块,标记为超时
|
||||
for (const chunk of uploadingChunks) {
|
||||
try {
|
||||
const chunkRecord = await env.img_url.getWithMetadata(chunk.key);
|
||||
const db = getDatabase(env);
|
||||
const chunkRecord = await db.getWithMetadata(chunk.key);
|
||||
if (chunkRecord && chunkRecord.metadata) {
|
||||
const timeoutMetadata = {
|
||||
...chunkRecord.metadata,
|
||||
@@ -315,8 +318,8 @@ async function handleChannelBasedMerge(context, uploadId, totalChunks, originalF
|
||||
timeoutDuringMerge: true,
|
||||
timeoutTime: Date.now()
|
||||
};
|
||||
|
||||
await env.img_url.put(chunk.key, chunkRecord.value, {
|
||||
|
||||
await db.put(chunk.key, chunkRecord.value, {
|
||||
metadata: timeoutMetadata,
|
||||
expirationTtl: 3600
|
||||
});
|
||||
@@ -379,7 +382,8 @@ async function mergeR2ChunksInfo(context, uploadId, completedChunks, metadata) {
|
||||
const multipartKey = `multipart_${uploadId}`;
|
||||
|
||||
// 获取multipart info
|
||||
const multipartInfoData = await env.img_url.get(multipartKey);
|
||||
const db = getDatabase(env);
|
||||
const multipartInfoData = await db.get(multipartKey);
|
||||
if (!multipartInfoData) {
|
||||
throw new Error('Multipart upload info not found');
|
||||
}
|
||||
@@ -412,10 +416,10 @@ async function mergeR2ChunksInfo(context, uploadId, completedChunks, metadata) {
|
||||
metadata.FileSize = (totalSize / 1024 / 1024).toFixed(2);
|
||||
|
||||
// 清理multipart info
|
||||
await env.img_url.delete(multipartKey);
|
||||
|
||||
// 写入KV数据库
|
||||
await env.img_url.put(finalFileId, "", { metadata });
|
||||
await db.delete(multipartKey);
|
||||
|
||||
// 写入数据库
|
||||
await db.put(finalFileId, "", { metadata });
|
||||
|
||||
// 结束上传
|
||||
waitUntil(endUpload(context, finalFileId, metadata));
|
||||
@@ -462,7 +466,8 @@ async function mergeS3ChunksInfo(context, uploadId, completedChunks, metadata) {
|
||||
const multipartKey = `multipart_${uploadId}`;
|
||||
|
||||
// 获取multipart info
|
||||
const multipartInfoData = await env.img_url.get(multipartKey);
|
||||
const db = getDatabase(env);
|
||||
const multipartInfoData = await db.get(multipartKey);
|
||||
if (!multipartInfoData) {
|
||||
throw new Error('Multipart upload info not found');
|
||||
}
|
||||
@@ -513,10 +518,10 @@ async function mergeS3ChunksInfo(context, uploadId, completedChunks, metadata) {
|
||||
metadata.S3FileKey = finalFileId;
|
||||
|
||||
// 清理multipart info
|
||||
await env.img_url.delete(multipartKey);
|
||||
await db.delete(multipartKey);
|
||||
|
||||
// 写入KV数据库
|
||||
await env.img_url.put(finalFileId, "", { metadata });
|
||||
// 写入数据库
|
||||
await db.put(finalFileId, "", { metadata });
|
||||
|
||||
// 异步结束上传
|
||||
waitUntil(endUpload(context, finalFileId, metadata));
|
||||
@@ -583,8 +588,9 @@ async function mergeTelegramChunksInfo(context, uploadId, completedChunks, metad
|
||||
// 将分片信息存储到value中
|
||||
const chunksData = JSON.stringify(chunks);
|
||||
|
||||
// 写入KV数据库
|
||||
await env.img_url.put(finalFileId, chunksData, { metadata });
|
||||
// 写入数据库
|
||||
const db = getDatabase(env);
|
||||
await db.put(finalFileId, chunksData, { metadata });
|
||||
|
||||
// 异步结束上传
|
||||
waitUntil(endUpload(context, finalFileId, metadata));
|
||||
@@ -612,8 +618,9 @@ async function mergeTelegramChunksInfo(context, uploadId, completedChunks, metad
|
||||
// 检查合并状态
|
||||
export async function checkMergeStatus(env, uploadId) {
|
||||
try {
|
||||
const db = getDatabase(env);
|
||||
const statusKey = `merge_status_${uploadId}`;
|
||||
const statusData = await env.img_url.get(statusKey);
|
||||
const statusData = await db.get(statusKey);
|
||||
|
||||
if (!statusData) {
|
||||
return createResponse(JSON.stringify({
|
||||
@@ -644,7 +651,7 @@ export async function checkMergeStatus(env, uploadId) {
|
||||
};
|
||||
|
||||
// 更新状态
|
||||
await env.img_url.put(statusKey, JSON.stringify(timeoutStatus), {
|
||||
await db.put(statusKey, JSON.stringify(timeoutStatus), {
|
||||
expirationTtl: 3600
|
||||
}).catch(err => console.warn('Failed to update timeout status:', err));
|
||||
|
||||
@@ -698,11 +705,12 @@ export async function checkMergeStatus(env, uploadId) {
|
||||
// 更新合并状态
|
||||
async function updateMergeStatus(env, statusKey, updates) {
|
||||
try {
|
||||
const currentData = await env.img_url.get(statusKey);
|
||||
const db = getDatabase(env);
|
||||
const currentData = await db.get(statusKey);
|
||||
if (currentData) {
|
||||
const status = JSON.parse(currentData);
|
||||
const updatedStatus = { ...status, ...updates, updatedAt: Date.now() };
|
||||
await env.img_url.put(statusKey, JSON.stringify(updatedStatus), {
|
||||
await db.put(statusKey, JSON.stringify(updatedStatus), {
|
||||
expirationTtl: 3600 // 1小时过期
|
||||
});
|
||||
}
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
import { createResponse, selectConsistentChannel, getUploadIp, getIPAddress, buildUniqueFileId, endUpload } from './uploadTools';
|
||||
import { TelegramAPI } from '../utils/telegramAPI';
|
||||
import { S3Client, CreateMultipartUploadCommand, UploadPartCommand, AbortMultipartUploadCommand } from "@aws-sdk/client-s3";
|
||||
import { getDatabase } from '../utils/databaseAdapter.js';
|
||||
|
||||
// 初始化分块上传
|
||||
export async function initializeChunkedUpload(context) {
|
||||
@@ -46,8 +47,9 @@ export async function initializeChunkedUpload(context) {
|
||||
};
|
||||
|
||||
// 保存会话信息
|
||||
const db = getDatabase(env);
|
||||
const sessionKey = `upload_session_${uploadId}`;
|
||||
await env.img_url.put(sessionKey, JSON.stringify(sessionInfo), {
|
||||
await db.put(sessionKey, JSON.stringify(sessionInfo), {
|
||||
expirationTtl: 3600 // 1小时过期
|
||||
});
|
||||
|
||||
@@ -93,7 +95,7 @@ export async function handleChunkUpload(context) {
|
||||
|
||||
// 验证上传会话
|
||||
const sessionKey = `upload_session_${uploadId}`;
|
||||
const sessionData = await env.img_url.get(sessionKey);
|
||||
const sessionData = await getDatabase(env).get(sessionKey);
|
||||
if (!sessionData) {
|
||||
return createResponse('Error: Invalid or expired upload session', { status: 400 });
|
||||
}
|
||||
@@ -133,7 +135,7 @@ export async function handleChunkUpload(context) {
|
||||
};
|
||||
|
||||
// 立即保存分块记录和数据,设置过期时间
|
||||
await env.img_url.put(chunkKey, chunkData, {
|
||||
await getDatabase(env).put(chunkKey, chunkData, {
|
||||
metadata: initialChunkMetadata,
|
||||
expirationTtl: 3600 // 1小时过期
|
||||
});
|
||||
@@ -211,7 +213,7 @@ async function uploadChunkToStorageWithTimeout(context, chunkIndex, totalChunks,
|
||||
|
||||
// 超时或失败时,更新状态为超时/失败
|
||||
try {
|
||||
const chunkRecord = await env.img_url.getWithMetadata(chunkKey, { type: 'arrayBuffer' });
|
||||
const chunkRecord = await getDatabase(env).getWithMetadata(chunkKey, { type: 'arrayBuffer' });
|
||||
if (chunkRecord && chunkRecord.metadata) {
|
||||
const isTimeout = error.message === 'Upload timeout';
|
||||
const errorMetadata = {
|
||||
@@ -223,7 +225,7 @@ async function uploadChunkToStorageWithTimeout(context, chunkIndex, totalChunks,
|
||||
};
|
||||
|
||||
// 保留原始数据以便重试
|
||||
await env.img_url.put(chunkKey, chunkRecord.value, {
|
||||
await getDatabase(env).put(chunkKey, chunkRecord.value, {
|
||||
metadata: errorMetadata,
|
||||
expirationTtl: 3600
|
||||
});
|
||||
@@ -244,7 +246,7 @@ async function uploadChunkToStorage(context, chunkIndex, totalChunks, uploadId,
|
||||
|
||||
try {
|
||||
// 从KV获取分块数据和metadata
|
||||
const chunkRecord = await env.img_url.getWithMetadata(chunkKey, { type: 'arrayBuffer' });
|
||||
const chunkRecord = await getDatabase(env).getWithMetadata(chunkKey, { type: 'arrayBuffer' });
|
||||
if (!chunkRecord || !chunkRecord.value) {
|
||||
console.error(`Chunk ${chunkIndex} data not found in KV`);
|
||||
return;
|
||||
@@ -275,7 +277,7 @@ async function uploadChunkToStorage(context, chunkIndex, totalChunks, uploadId,
|
||||
};
|
||||
|
||||
// 只保存metadata,不保存原始数据,设置过期时间
|
||||
await env.img_url.put(chunkKey, '', {
|
||||
await getDatabase(env).put(chunkKey, '', {
|
||||
metadata: updatedMetadata,
|
||||
expirationTtl: 3600 // 1小时过期
|
||||
});
|
||||
@@ -293,7 +295,7 @@ async function uploadChunkToStorage(context, chunkIndex, totalChunks, uploadId,
|
||||
};
|
||||
|
||||
// 保留原始数据以便重试,设置过期时间
|
||||
await env.img_url.put(chunkKey, chunkData, {
|
||||
await getDatabase(env).put(chunkKey, chunkData, {
|
||||
metadata: failedMetadata,
|
||||
expirationTtl: 3600 // 1小时过期
|
||||
});
|
||||
@@ -307,7 +309,7 @@ async function uploadChunkToStorage(context, chunkIndex, totalChunks, uploadId,
|
||||
|
||||
// 发生异常时,确保保留原始数据并标记为失败
|
||||
try {
|
||||
const chunkRecord = await env.img_url.getWithMetadata(chunkKey, { type: 'arrayBuffer' });
|
||||
const chunkRecord = await getDatabase(env).getWithMetadata(chunkKey, { type: 'arrayBuffer' });
|
||||
if (chunkRecord && chunkRecord.metadata) {
|
||||
const errorMetadata = {
|
||||
...chunkRecord.metadata,
|
||||
@@ -316,7 +318,7 @@ async function uploadChunkToStorage(context, chunkIndex, totalChunks, uploadId,
|
||||
failedTime: Date.now()
|
||||
};
|
||||
|
||||
await env.img_url.put(chunkKey, chunkRecord.value, {
|
||||
await getDatabase(env).put(chunkKey, chunkRecord.value, {
|
||||
metadata: errorMetadata,
|
||||
expirationTtl: 3600 // 1小时过期
|
||||
});
|
||||
@@ -353,7 +355,7 @@ async function uploadSingleChunkToR2Multipart(context, chunkData, chunkIndex, to
|
||||
};
|
||||
|
||||
// 保存multipart info
|
||||
await env.img_url.put(multipartKey, JSON.stringify(multipartInfo), {
|
||||
await getDatabase(env).put(multipartKey, JSON.stringify(multipartInfo), {
|
||||
expirationTtl: 3600 // 1小时过期
|
||||
});
|
||||
} else {
|
||||
@@ -363,7 +365,7 @@ async function uploadSingleChunkToR2Multipart(context, chunkData, chunkIndex, to
|
||||
const maxRetries = 30; // 最多等待60秒
|
||||
|
||||
while (!multipartInfoData && retryCount < maxRetries) {
|
||||
multipartInfoData = await env.img_url.get(multipartKey);
|
||||
multipartInfoData = await getDatabase(env).get(multipartKey);
|
||||
if (!multipartInfoData) {
|
||||
// 等待2秒后重试
|
||||
await new Promise(resolve => setTimeout(resolve, 2000));
|
||||
@@ -381,7 +383,7 @@ async function uploadSingleChunkToR2Multipart(context, chunkData, chunkIndex, to
|
||||
}
|
||||
|
||||
// 获取multipart info
|
||||
const multipartInfoData = await env.img_url.get(multipartKey);
|
||||
const multipartInfoData = await getDatabase(env).get(multipartKey);
|
||||
if (!multipartInfoData) {
|
||||
return { success: false, error: 'Multipart upload not initialized' };
|
||||
}
|
||||
@@ -459,7 +461,7 @@ async function uploadSingleChunkToS3Multipart(context, chunkData, chunkIndex, to
|
||||
};
|
||||
|
||||
// 保存multipart info
|
||||
await env.img_url.put(multipartKey, JSON.stringify(multipartInfo), {
|
||||
await getDatabase(env).put(multipartKey, JSON.stringify(multipartInfo), {
|
||||
expirationTtl: 3600 // 1小时过期
|
||||
});
|
||||
} else {
|
||||
@@ -469,7 +471,7 @@ async function uploadSingleChunkToS3Multipart(context, chunkData, chunkIndex, to
|
||||
const maxRetries = 30; // 最多等待60秒
|
||||
|
||||
while (!multipartInfoData && retryCount < maxRetries) {
|
||||
multipartInfoData = await env.img_url.get(multipartKey);
|
||||
multipartInfoData = await getDatabase(env).get(multipartKey);
|
||||
if (!multipartInfoData) {
|
||||
// 等待2秒后重试
|
||||
await new Promise(resolve => setTimeout(resolve, 2000));
|
||||
@@ -487,7 +489,7 @@ async function uploadSingleChunkToS3Multipart(context, chunkData, chunkIndex, to
|
||||
}
|
||||
|
||||
// 获取multipart info
|
||||
const multipartInfoData = await env.img_url.get(multipartKey);
|
||||
const multipartInfoData = await getDatabase(env).get(multipartKey);
|
||||
if (!multipartInfoData) {
|
||||
return { success: false, error: 'Multipart upload not initialized' };
|
||||
}
|
||||
@@ -686,7 +688,7 @@ async function retrySingleChunk(context, chunk, uploadChannel, maxRetries = 5, r
|
||||
let lastError = null;
|
||||
|
||||
try {
|
||||
const chunkRecord = await env.img_url.getWithMetadata(chunk.key, { type: 'arrayBuffer' });
|
||||
const chunkRecord = await getDatabase(env).getWithMetadata(chunk.key, { type: 'arrayBuffer' });
|
||||
if (!chunkRecord || !chunkRecord.value) {
|
||||
console.error(`Chunk ${chunk.index} data missing for retry`);
|
||||
return { success: false, chunk, reason: 'data_missing', error: 'Chunk data not found' };
|
||||
@@ -704,7 +706,7 @@ async function retrySingleChunk(context, chunk, uploadChannel, maxRetries = 5, r
|
||||
status: 'retrying',
|
||||
};
|
||||
|
||||
await env.img_url.put(chunk.key, chunkData, {
|
||||
await getDatabase(env).put(chunk.key, chunkData, {
|
||||
metadata: retryMetadata,
|
||||
expirationTtl: 3600
|
||||
});
|
||||
@@ -742,7 +744,7 @@ async function retrySingleChunk(context, chunk, uploadChannel, maxRetries = 5, r
|
||||
};
|
||||
|
||||
// 删除原始数据,只保留上传结果,设置过期时间
|
||||
await env.img_url.put(chunk.key, '', {
|
||||
await getDatabase(env).put(chunk.key, '', {
|
||||
metadata: updatedMetadata,
|
||||
expirationTtl: 3600 // 1小时过期
|
||||
});
|
||||
@@ -764,14 +766,14 @@ async function retrySingleChunk(context, chunk, uploadChannel, maxRetries = 5, r
|
||||
|
||||
// 更新重试失败状态
|
||||
try {
|
||||
const chunkRecord = await env.img_url.getWithMetadata(chunk.key, { type: 'arrayBuffer' });
|
||||
const chunkRecord = await getDatabase(env).getWithMetadata(chunk.key, { type: 'arrayBuffer' });
|
||||
if (chunkRecord) {
|
||||
const failedRetryMetadata = {
|
||||
...chunkRecord.metadata,
|
||||
status: isTimeout ? 'retry_timeout' : 'retry_failed'
|
||||
};
|
||||
|
||||
await env.img_url.put(chunk.key, chunkRecord.value, {
|
||||
await getDatabase(env).put(chunk.key, chunkRecord.value, {
|
||||
metadata: failedRetryMetadata,
|
||||
expirationTtl: 3600
|
||||
});
|
||||
@@ -798,7 +800,7 @@ export async function cleanupFailedMultipartUploads(context, uploadId, uploadCha
|
||||
|
||||
try {
|
||||
const multipartKey = `multipart_${uploadId}`;
|
||||
const multipartInfoData = await env.img_url.get(multipartKey);
|
||||
const multipartInfoData = await getDatabase(env).get(multipartKey);
|
||||
|
||||
if (!multipartInfoData) {
|
||||
return; // 没有multipart upload需要清理
|
||||
@@ -837,7 +839,7 @@ export async function cleanupFailedMultipartUploads(context, uploadId, uploadCha
|
||||
}
|
||||
|
||||
// 清理multipart info
|
||||
await env.img_url.delete(multipartKey);
|
||||
await getDatabase(env).delete(multipartKey);
|
||||
console.log(`Cleaned up failed multipart upload for ${uploadId}`);
|
||||
|
||||
} catch (error) {
|
||||
@@ -854,7 +856,7 @@ export async function checkChunkUploadStatuses(env, uploadId, totalChunks) {
|
||||
for (let i = 0; i < totalChunks; i++) {
|
||||
const chunkKey = `chunk_${uploadId}_${i.toString().padStart(3, '0')}`;
|
||||
try {
|
||||
const chunkRecord = await env.img_url.getWithMetadata(chunkKey, { type: 'arrayBuffer' });
|
||||
const chunkRecord = await getDatabase(env).getWithMetadata(chunkKey, { type: 'arrayBuffer' });
|
||||
if (chunkRecord && chunkRecord.metadata) {
|
||||
let status = chunkRecord.metadata.status || 'unknown';
|
||||
|
||||
@@ -870,7 +872,7 @@ export async function checkChunkUploadStatuses(env, uploadId, totalChunks) {
|
||||
timeoutDetectedTime: currentTime
|
||||
};
|
||||
|
||||
await env.img_url.put(chunkKey, chunkRecord.value, {
|
||||
await getDatabase(env).put(chunkKey, chunkRecord.value, {
|
||||
metadata: timeoutMetadata,
|
||||
expirationTtl: 3600
|
||||
}).catch(err => console.warn(`Failed to update timeout status for chunk ${i}:`, err));
|
||||
@@ -932,12 +934,12 @@ export async function cleanupChunkData(env, uploadId, totalChunks) {
|
||||
const chunkKey = `chunk_${uploadId}_${i.toString().padStart(3, '0')}`;
|
||||
|
||||
// 删除KV中的分块记录
|
||||
await env.img_url.delete(chunkKey);
|
||||
await getDatabase(env).delete(chunkKey);
|
||||
}
|
||||
|
||||
// 清理multipart info(如果存在)
|
||||
const multipartKey = `multipart_${uploadId}`;
|
||||
await env.img_url.delete(multipartKey);
|
||||
await getDatabase(env).delete(multipartKey);
|
||||
|
||||
} catch (cleanupError) {
|
||||
console.warn('Failed to cleanup chunk data:', cleanupError);
|
||||
@@ -948,7 +950,7 @@ export async function cleanupChunkData(env, uploadId, totalChunks) {
|
||||
export async function cleanupUploadSession(env, uploadId) {
|
||||
try {
|
||||
const sessionKey = `upload_session_${uploadId}`;
|
||||
await env.img_url.delete(sessionKey);
|
||||
await getDatabase(env).delete(sessionKey);
|
||||
console.log(`Cleaned up upload session for ${uploadId}`);
|
||||
} catch (cleanupError) {
|
||||
console.warn('Failed to cleanup upload session:', cleanupError);
|
||||
@@ -962,7 +964,7 @@ export async function forceCleanupUpload(context, uploadId, totalChunks) {
|
||||
try {
|
||||
// 读取 session 信息
|
||||
const sessionKey = `upload_session_${uploadId}`;
|
||||
const sessionRecord = await env.img_url.get(sessionKey);
|
||||
const sessionRecord = await getDatabase(env).get(sessionKey);
|
||||
const uploadChannel = sessionRecord ? JSON.parse(sessionRecord).uploadChannel : 'cfr2'; // 默认使用 cfr2
|
||||
|
||||
// 清理 multipart upload信息
|
||||
@@ -973,7 +975,7 @@ export async function forceCleanupUpload(context, uploadId, totalChunks) {
|
||||
// 清理所有分块
|
||||
for (let i = 0; i < totalChunks; i++) {
|
||||
const chunkKey = `chunk_${uploadId}_${i.toString().padStart(3, '0')}`;
|
||||
cleanupPromises.push(env.img_url.delete(chunkKey).catch(err =>
|
||||
cleanupPromises.push(getDatabase(env).delete(chunkKey).catch(err =>
|
||||
console.warn(`Failed to delete chunk ${i}:`, err)
|
||||
));
|
||||
}
|
||||
@@ -986,7 +988,7 @@ export async function forceCleanupUpload(context, uploadId, totalChunks) {
|
||||
];
|
||||
|
||||
keysToCleanup.forEach(key => {
|
||||
cleanupPromises.push(env.img_url.delete(key).catch(err =>
|
||||
cleanupPromises.push(getDatabase(env).delete(key).catch(err =>
|
||||
console.warn(`Failed to delete key ${key}:`, err)
|
||||
));
|
||||
});
|
||||
@@ -1078,7 +1080,7 @@ export async function uploadLargeFileToTelegram(context, file, fullId, metadata,
|
||||
}
|
||||
|
||||
// 写入最终的KV记录,分片信息作为value
|
||||
await env.img_url.put(fullId, chunksData, { metadata });
|
||||
await getDatabase(env).put(fullId, chunksData, { metadata });
|
||||
|
||||
// 异步结束上传
|
||||
waitUntil(endUpload(context, fullId, metadata));
|
||||
|
||||
+16
-10
@@ -1,11 +1,12 @@
|
||||
import { userAuthCheck, UnauthorizedResponse } from "../utils/userAuth";
|
||||
import { fetchUploadConfig, fetchSecurityConfig } from "../utils/sysConfig";
|
||||
import { createResponse, getUploadIp, getIPAddress, isExtValid,
|
||||
import { createResponse, getUploadIp, getIPAddress, isExtValid,
|
||||
moderateContent, purgeCDNCache, isBlockedUploadIp, buildUniqueFileId, endUpload } from "./uploadTools";
|
||||
import { initializeChunkedUpload, handleChunkUpload, uploadLargeFileToTelegram, handleCleanupRequest} from "./chunkUpload";
|
||||
import { handleChunkMerge, checkMergeStatus } from "./chunkMerge";
|
||||
import { TelegramAPI } from "../utils/telegramAPI";
|
||||
import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3";
|
||||
import { getDatabase } from '../utils/databaseAdapter.js';
|
||||
|
||||
|
||||
export async function onRequest(context) { // Contents of context object
|
||||
@@ -241,13 +242,14 @@ async function uploadFileToCloudflareR2(context, fullId, metadata, returnLink) {
|
||||
let moderateUrl = `${R2PublicUrl}/${fullId}`;
|
||||
metadata.Label = await moderateContent(env, moderateUrl);
|
||||
|
||||
// 写入KV数据库
|
||||
// 写入数据库
|
||||
try {
|
||||
await env.img_url.put(fullId, "", {
|
||||
const db = getDatabase(env);
|
||||
await db.put(fullId, "", {
|
||||
metadata: metadata,
|
||||
});
|
||||
} catch (error) {
|
||||
return createResponse('Error: Failed to write to KV database', { status: 500 });
|
||||
return createResponse('Error: Failed to write to database', { status: 500 });
|
||||
}
|
||||
|
||||
// 结束上传
|
||||
@@ -337,7 +339,8 @@ async function uploadFileToS3(context, fullId, metadata, returnLink) {
|
||||
// 图像审查
|
||||
if (uploadModerate && uploadModerate.enabled) {
|
||||
try {
|
||||
await env.img_url.put(fullId, "", { metadata });
|
||||
const db = getDatabase(env);
|
||||
await db.put(fullId, "", { metadata });
|
||||
} catch {
|
||||
return createResponse("Error: Failed to write to KV database", { status: 500 });
|
||||
}
|
||||
@@ -347,11 +350,12 @@ async function uploadFileToS3(context, fullId, metadata, returnLink) {
|
||||
metadata.Label = await moderateContent(env, moderateUrl);
|
||||
}
|
||||
|
||||
// 写入 KV 数据库
|
||||
// 写入数据库
|
||||
try {
|
||||
await env.img_url.put(fullId, "", { metadata });
|
||||
const db = getDatabase(env);
|
||||
await db.put(fullId, "", { metadata });
|
||||
} catch {
|
||||
return createResponse("Error: Failed to write to KV database", { status: 500 });
|
||||
return createResponse("Error: Failed to write to database", { status: 500 });
|
||||
}
|
||||
|
||||
// 结束上传
|
||||
@@ -465,7 +469,8 @@ async function uploadFileToTelegram(context, fullId, metadata, fileExt, fileName
|
||||
metadata.TgFileId = id;
|
||||
metadata.TgChatId = tgChatId;
|
||||
metadata.TgBotToken = tgBotToken;
|
||||
await env.img_url.put(fullId, "", {
|
||||
const db = getDatabase(env);
|
||||
await db.put(fullId, "", {
|
||||
metadata: metadata,
|
||||
});
|
||||
} catch (error) {
|
||||
@@ -498,7 +503,8 @@ async function uploadFileToExternal(context, fullId, metadata, returnLink) {
|
||||
metadata.ExternalLink = extUrl;
|
||||
// 写入KV数据库
|
||||
try {
|
||||
await env.img_url.put(fullId, "", {
|
||||
const db = getDatabase(env);
|
||||
await db.put(fullId, "", {
|
||||
metadata: metadata,
|
||||
});
|
||||
} catch (error) {
|
||||
|
||||
@@ -205,21 +205,42 @@ export function getUploadIp(request) {
|
||||
|
||||
// 检查上传IP是否被封禁
|
||||
export async function isBlockedUploadIp(env, uploadIp) {
|
||||
const kv = env.img_url;
|
||||
let list = await kv.get("manage@blockipList");
|
||||
if (list == null) {
|
||||
list = [];
|
||||
} else {
|
||||
list = list.split(",");
|
||||
}
|
||||
try {
|
||||
// 使用数据库适配器而不是直接访问KV
|
||||
const { getDatabase } = await import('../utils/databaseAdapter.js');
|
||||
const db = getDatabase(env);
|
||||
let list = await db.get("manage@blockipList");
|
||||
if (list == null) {
|
||||
list = [];
|
||||
} else {
|
||||
list = list.split(",");
|
||||
}
|
||||
|
||||
return list.includes(uploadIp);
|
||||
return list.includes(uploadIp);
|
||||
} catch (error) {
|
||||
console.error('Failed to check blocked IP:', error);
|
||||
// 如果数据库未配置,默认不阻止任何IP
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
// 构建唯一文件ID
|
||||
export async function buildUniqueFileId(context, fileName, fileType = 'application/octet-stream') {
|
||||
const { env, url } = context;
|
||||
|
||||
// 获取数据库适配器
|
||||
const { getDatabase } = await import('../utils/databaseAdapter.js');
|
||||
let db;
|
||||
try {
|
||||
db = getDatabase(env);
|
||||
} catch (error) {
|
||||
console.error('Database not configured for buildUniqueFileId:', error);
|
||||
// 如果数据库未配置,生成一个简单的唯一ID
|
||||
const timestamp = Date.now();
|
||||
const random = Math.random().toString(36).substring(2, 8);
|
||||
return `${timestamp}_${random}_${fileName}`;
|
||||
}
|
||||
|
||||
let fileExt = fileName.split('.').pop();
|
||||
if (!fileExt || fileExt === fileName) {
|
||||
fileExt = fileType.split('/').pop();
|
||||
@@ -257,7 +278,7 @@ export async function buildUniqueFileId(context, fileName, fileType = 'applicati
|
||||
while (true) {
|
||||
const shortId = generateShortId(8);
|
||||
const testFullId = normalizedFolder ? `${normalizedFolder}/${shortId}.${fileExt}` : `${shortId}.${fileExt}`;
|
||||
if (await env.img_url.get(testFullId) === null) {
|
||||
if (await db.get(testFullId) === null) {
|
||||
return testFullId;
|
||||
}
|
||||
}
|
||||
@@ -266,7 +287,7 @@ export async function buildUniqueFileId(context, fileName, fileType = 'applicati
|
||||
}
|
||||
|
||||
// 检查基础ID是否已存在
|
||||
if (await env.img_url.get(baseId) === null) {
|
||||
if (await db.get(baseId) === null) {
|
||||
return baseId;
|
||||
}
|
||||
|
||||
@@ -296,7 +317,7 @@ export async function buildUniqueFileId(context, fileName, fileType = 'applicati
|
||||
}
|
||||
|
||||
// 检查新ID是否已存在
|
||||
if (await env.img_url.get(duplicateId) === null) {
|
||||
if (await db.get(duplicateId) === null) {
|
||||
return duplicateId;
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,405 @@
|
||||
/**
|
||||
* D1 数据库操作工具类
|
||||
* 用于替代原有的KV存储操作
|
||||
*/
|
||||
|
||||
function D1Database(db) {
|
||||
this.db = db;
|
||||
}
|
||||
|
||||
// ==================== 文件操作 ====================
|
||||
|
||||
/**
|
||||
* 保存文件记录 (替代 KV.put)
|
||||
*/
|
||||
D1Database.prototype.putFile = function(fileId, value, options) {
|
||||
value = value || '';
|
||||
options = options || {};
|
||||
var metadata = options.metadata || {};
|
||||
|
||||
// 从metadata中提取字段用于索引
|
||||
var extractedFields = this.extractMetadataFields(metadata);
|
||||
|
||||
var stmt = this.db.prepare(
|
||||
'INSERT OR REPLACE INTO files (' +
|
||||
'id, value, metadata, file_name, file_type, file_size, ' +
|
||||
'upload_ip, upload_address, list_type, timestamp, ' +
|
||||
'label, directory, channel, channel_name, ' +
|
||||
'tg_file_id, tg_chat_id, tg_bot_token, is_chunked' +
|
||||
') VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)'
|
||||
);
|
||||
|
||||
return stmt.bind(
|
||||
fileId,
|
||||
value,
|
||||
JSON.stringify(metadata),
|
||||
extractedFields.fileName,
|
||||
extractedFields.fileType,
|
||||
extractedFields.fileSize,
|
||||
extractedFields.uploadIP,
|
||||
extractedFields.uploadAddress,
|
||||
extractedFields.listType,
|
||||
extractedFields.timestamp,
|
||||
extractedFields.label,
|
||||
extractedFields.directory,
|
||||
extractedFields.channel,
|
||||
extractedFields.channelName,
|
||||
extractedFields.tgFileId,
|
||||
extractedFields.tgChatId,
|
||||
extractedFields.tgBotToken,
|
||||
extractedFields.isChunked
|
||||
).run();
|
||||
};
|
||||
|
||||
/**
|
||||
* 获取文件记录 (替代 KV.get)
|
||||
*/
|
||||
D1Database.prototype.getFile = function(fileId) {
|
||||
var self = this;
|
||||
var stmt = this.db.prepare('SELECT * FROM files WHERE id = ?');
|
||||
return stmt.bind(fileId).first().then(function(result) {
|
||||
if (!result) return null;
|
||||
|
||||
return {
|
||||
value: result.value,
|
||||
metadata: JSON.parse(result.metadata || '{}')
|
||||
};
|
||||
});
|
||||
};
|
||||
|
||||
/**
|
||||
* 获取文件记录包含元数据 (替代 KV.getWithMetadata)
|
||||
*/
|
||||
D1Database.prototype.getFileWithMetadata = function(fileId) {
|
||||
return this.getFile(fileId);
|
||||
};
|
||||
|
||||
/**
|
||||
* 删除文件记录 (替代 KV.delete)
|
||||
*/
|
||||
D1Database.prototype.deleteFile = function(fileId) {
|
||||
var stmt = this.db.prepare('DELETE FROM files WHERE id = ?');
|
||||
return stmt.bind(fileId).run();
|
||||
};
|
||||
|
||||
/**
|
||||
* 列出文件 (替代 KV.list)
|
||||
*/
|
||||
D1Database.prototype.listFiles = function(options) {
|
||||
options = options || {};
|
||||
var prefix = options.prefix || '';
|
||||
var limit = options.limit || 1000;
|
||||
var cursor = options.cursor || null;
|
||||
|
||||
var query = 'SELECT id, metadata FROM files';
|
||||
var params = [];
|
||||
|
||||
if (prefix) {
|
||||
query += ' WHERE id LIKE ?';
|
||||
params.push(prefix + '%');
|
||||
}
|
||||
|
||||
if (cursor) {
|
||||
query += prefix ? ' AND' : ' WHERE';
|
||||
query += ' id > ?';
|
||||
params.push(cursor);
|
||||
}
|
||||
|
||||
query += ' ORDER BY id LIMIT ?';
|
||||
params.push(limit + 1);
|
||||
|
||||
var stmt = this.db.prepare(query);
|
||||
if (params.length > 0) {
|
||||
stmt = stmt.bind.apply(stmt, params);
|
||||
}
|
||||
return stmt.all().then(function(response) {
|
||||
var results = response.results || [];
|
||||
var hasMore = results.length > limit;
|
||||
if (hasMore) {
|
||||
results.pop();
|
||||
}
|
||||
|
||||
var keys = results.map(function(row) {
|
||||
return {
|
||||
name: row.id,
|
||||
metadata: JSON.parse(row.metadata || '{}')
|
||||
};
|
||||
});
|
||||
|
||||
return {
|
||||
keys: keys,
|
||||
cursor: hasMore && keys.length > 0 ? keys[keys.length - 1].name : null,
|
||||
list_complete: !hasMore
|
||||
};
|
||||
});
|
||||
};
|
||||
|
||||
// ==================== 设置操作 ====================
|
||||
|
||||
/**
|
||||
* 保存设置 (替代 KV.put)
|
||||
*/
|
||||
D1Database.prototype.putSetting = function(key, value, category) {
|
||||
if (!category && key.startsWith('manage@sysConfig@')) {
|
||||
category = key.split('@')[2];
|
||||
}
|
||||
|
||||
var stmt = this.db.prepare(
|
||||
'INSERT OR REPLACE INTO settings (key, value, category) VALUES (?, ?, ?)'
|
||||
);
|
||||
|
||||
return stmt.bind(key, value, category).run();
|
||||
};
|
||||
|
||||
/**
|
||||
* 获取设置 (替代 KV.get)
|
||||
*/
|
||||
D1Database.prototype.getSetting = function(key) {
|
||||
var stmt = this.db.prepare('SELECT value FROM settings WHERE key = ?');
|
||||
return stmt.bind(key).first().then(function(result) {
|
||||
return result ? result.value : null;
|
||||
});
|
||||
};
|
||||
|
||||
/**
|
||||
* 删除设置 (替代 KV.delete)
|
||||
*/
|
||||
D1Database.prototype.deleteSetting = function(key) {
|
||||
var stmt = this.db.prepare('DELETE FROM settings WHERE key = ?');
|
||||
return stmt.bind(key).run();
|
||||
};
|
||||
|
||||
/**
|
||||
* 列出设置 (替代 KV.list)
|
||||
*/
|
||||
D1Database.prototype.listSettings = function(options) {
|
||||
options = options || {};
|
||||
var prefix = options.prefix || '';
|
||||
var limit = options.limit || 1000;
|
||||
|
||||
var query = 'SELECT key, value FROM settings';
|
||||
var params = [];
|
||||
|
||||
if (prefix) {
|
||||
query += ' WHERE key LIKE ?';
|
||||
params.push(prefix + '%');
|
||||
}
|
||||
|
||||
query += ' ORDER BY key LIMIT ?';
|
||||
params.push(limit);
|
||||
|
||||
var stmt = this.db.prepare(query);
|
||||
if (params.length > 0) {
|
||||
stmt = stmt.bind.apply(stmt, params);
|
||||
}
|
||||
return stmt.all().then(function(response) {
|
||||
var results = response.results || [];
|
||||
var keys = results.map(function(row) {
|
||||
return {
|
||||
name: row.key,
|
||||
value: row.value
|
||||
};
|
||||
});
|
||||
|
||||
return { keys: keys };
|
||||
});
|
||||
};
|
||||
|
||||
// ==================== 索引操作 ====================
|
||||
|
||||
/**
|
||||
* 保存索引操作记录
|
||||
*/
|
||||
D1Database.prototype.putIndexOperation = function(operationId, operation) {
|
||||
var stmt = this.db.prepare(
|
||||
'INSERT OR REPLACE INTO index_operations (id, type, timestamp, data) VALUES (?, ?, ?, ?)'
|
||||
);
|
||||
|
||||
return stmt.bind(
|
||||
operationId,
|
||||
operation.type,
|
||||
operation.timestamp,
|
||||
JSON.stringify(operation.data)
|
||||
).run();
|
||||
};
|
||||
|
||||
/**
|
||||
* 获取索引操作记录
|
||||
*/
|
||||
D1Database.prototype.getIndexOperation = function(operationId) {
|
||||
var stmt = this.db.prepare('SELECT * FROM index_operations WHERE id = ?');
|
||||
return stmt.bind(operationId).first().then(function(result) {
|
||||
if (!result) return null;
|
||||
|
||||
return {
|
||||
type: result.type,
|
||||
timestamp: result.timestamp,
|
||||
data: JSON.parse(result.data)
|
||||
};
|
||||
});
|
||||
};
|
||||
|
||||
/**
|
||||
* 删除索引操作记录
|
||||
*/
|
||||
D1Database.prototype.deleteIndexOperation = function(operationId) {
|
||||
var stmt = this.db.prepare('DELETE FROM index_operations WHERE id = ?');
|
||||
return stmt.bind(operationId).run();
|
||||
};
|
||||
|
||||
/**
|
||||
* 列出索引操作记录
|
||||
*/
|
||||
D1Database.prototype.listIndexOperations = function(options) {
|
||||
options = options || {};
|
||||
var limit = options.limit || 1000;
|
||||
var processed = options.processed;
|
||||
|
||||
var query = 'SELECT * FROM index_operations';
|
||||
var params = [];
|
||||
|
||||
if (processed !== null && processed !== undefined) {
|
||||
query += ' WHERE processed = ?';
|
||||
params.push(processed);
|
||||
}
|
||||
|
||||
query += ' ORDER BY timestamp LIMIT ?';
|
||||
params.push(limit);
|
||||
|
||||
var stmt = this.db.prepare(query);
|
||||
if (params.length > 0) {
|
||||
stmt = stmt.bind.apply(stmt, params);
|
||||
}
|
||||
return stmt.all().then(function(response) {
|
||||
var results = response.results || [];
|
||||
return results.map(function(row) {
|
||||
return {
|
||||
id: row.id,
|
||||
type: row.type,
|
||||
timestamp: row.timestamp,
|
||||
data: JSON.parse(row.data),
|
||||
processed: row.processed
|
||||
};
|
||||
});
|
||||
});
|
||||
};
|
||||
|
||||
// ==================== 工具方法 ====================
|
||||
|
||||
/**
|
||||
* 从metadata中提取字段用于索引
|
||||
*/
|
||||
D1Database.prototype.extractMetadataFields = function(metadata) {
|
||||
return {
|
||||
fileName: metadata.FileName || null,
|
||||
fileType: metadata.FileType || null,
|
||||
fileSize: metadata.FileSize || null,
|
||||
uploadIP: metadata.UploadIP || null,
|
||||
uploadAddress: metadata.UploadAddress || null,
|
||||
listType: metadata.ListType || null,
|
||||
timestamp: metadata.TimeStamp || null,
|
||||
label: metadata.Label || null,
|
||||
directory: metadata.Directory || null,
|
||||
channel: metadata.Channel || null,
|
||||
channelName: metadata.ChannelName || null,
|
||||
tgFileId: metadata.TgFileId || null,
|
||||
tgChatId: metadata.TgChatId || null,
|
||||
tgBotToken: metadata.TgBotToken || null,
|
||||
isChunked: metadata.IsChunked || false
|
||||
};
|
||||
};
|
||||
|
||||
// ==================== 通用方法 ====================
|
||||
|
||||
/**
|
||||
* 通用的put方法,根据key类型自动选择存储位置
|
||||
*/
|
||||
D1Database.prototype.put = function(key, value, options) {
|
||||
options = options || {};
|
||||
|
||||
if (key.startsWith('manage@sysConfig@') || key.startsWith('manage@')) {
|
||||
return this.putSetting(key, value);
|
||||
} else if (key.startsWith('manage@index@operation_')) {
|
||||
var operationId = key.replace('manage@index@operation_', '');
|
||||
var operation = JSON.parse(value);
|
||||
return this.putIndexOperation(operationId, operation);
|
||||
} else {
|
||||
return this.putFile(key, value, options);
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* 通用的get方法,根据key类型自动选择获取位置
|
||||
*/
|
||||
D1Database.prototype.get = function(key) {
|
||||
var self = this;
|
||||
|
||||
if (key.startsWith('manage@sysConfig@') || key.startsWith('manage@')) {
|
||||
return this.getSetting(key);
|
||||
} else if (key.startsWith('manage@index@operation_')) {
|
||||
var operationId = key.replace('manage@index@operation_', '');
|
||||
return this.getIndexOperation(operationId).then(function(operation) {
|
||||
return operation ? JSON.stringify(operation) : null;
|
||||
});
|
||||
} else {
|
||||
return this.getFile(key).then(function(file) {
|
||||
return file ? file.value : null;
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* 通用的getWithMetadata方法
|
||||
*/
|
||||
D1Database.prototype.getWithMetadata = function(key) {
|
||||
var self = this;
|
||||
|
||||
if (key.startsWith('manage@sysConfig@') || key.startsWith('manage@')) {
|
||||
return this.getSetting(key).then(function(value) {
|
||||
return value ? { value: value, metadata: {} } : null;
|
||||
});
|
||||
} else {
|
||||
return this.getFileWithMetadata(key);
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* 通用的delete方法
|
||||
*/
|
||||
D1Database.prototype.delete = function(key) {
|
||||
if (key.startsWith('manage@sysConfig@') || key.startsWith('manage@')) {
|
||||
return this.deleteSetting(key);
|
||||
} else if (key.startsWith('manage@index@operation_')) {
|
||||
var operationId = key.replace('manage@index@operation_', '');
|
||||
return this.deleteIndexOperation(operationId);
|
||||
} else {
|
||||
return this.deleteFile(key);
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* 通用的list方法
|
||||
*/
|
||||
D1Database.prototype.list = function(options) {
|
||||
options = options || {};
|
||||
var prefix = options.prefix || '';
|
||||
var self = this;
|
||||
|
||||
if (prefix.startsWith('manage@sysConfig@') || prefix.startsWith('manage@')) {
|
||||
return this.listSettings(options);
|
||||
} else if (prefix.startsWith('manage@index@operation_')) {
|
||||
return this.listIndexOperations(options).then(function(operations) {
|
||||
var keys = operations.map(function(op) {
|
||||
return {
|
||||
name: 'manage@index@operation_' + op.id
|
||||
};
|
||||
});
|
||||
return { keys: keys };
|
||||
});
|
||||
} else {
|
||||
return this.listFiles(options);
|
||||
}
|
||||
};
|
||||
|
||||
// 导出构造函数
|
||||
export { D1Database };
|
||||
@@ -0,0 +1,214 @@
|
||||
/**
|
||||
* 数据库适配器
|
||||
* 提供统一的接口,可以在KV和D1之间切换
|
||||
*/
|
||||
|
||||
import { D1Database } from './d1Database.js';
|
||||
|
||||
/**
|
||||
* 创建数据库适配器
|
||||
* @param {Object} env - 环境变量
|
||||
* @returns {Object} 数据库适配器实例
|
||||
*/
|
||||
export function createDatabaseAdapter(env) {
|
||||
// 检查是否配置了D1数据库
|
||||
if (env.DB && typeof env.DB.prepare === 'function') {
|
||||
// 使用D1数据库
|
||||
console.log('Using D1 Database');
|
||||
return new D1Database(env.DB);
|
||||
} else if (env.img_url && typeof env.img_url.get === 'function') {
|
||||
// 回退到KV存储
|
||||
console.log('Using KV Storage (fallback)');
|
||||
return new KVAdapter(env.img_url);
|
||||
} else {
|
||||
console.error('No database configured. Please configure either D1 (env.DB) or KV (env.img_url)');
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* KV适配器类
|
||||
* 保持与原有KV接口的兼容性
|
||||
*/
|
||||
class KVAdapter {
|
||||
constructor(kv) {
|
||||
this.kv = kv;
|
||||
}
|
||||
|
||||
// 直接代理到KV的方法
|
||||
async put(key, value, options) {
|
||||
options = options || {};
|
||||
return await this.kv.put(key, value, options);
|
||||
}
|
||||
|
||||
async get(key) {
|
||||
return await this.kv.get(key);
|
||||
}
|
||||
|
||||
async getWithMetadata(key) {
|
||||
return await this.kv.getWithMetadata(key);
|
||||
}
|
||||
|
||||
async delete(key) {
|
||||
return await this.kv.delete(key);
|
||||
}
|
||||
|
||||
async list(options) {
|
||||
options = options || {};
|
||||
return await this.kv.list(options);
|
||||
}
|
||||
|
||||
// 为了兼容性,添加一些别名方法
|
||||
async putFile(fileId, value, options) {
|
||||
return await this.put(fileId, value, options);
|
||||
}
|
||||
|
||||
async getFile(fileId) {
|
||||
const result = await this.getWithMetadata(fileId);
|
||||
return result;
|
||||
}
|
||||
|
||||
async getFileWithMetadata(fileId) {
|
||||
return await this.getWithMetadata(fileId);
|
||||
}
|
||||
|
||||
async deleteFile(fileId) {
|
||||
return await this.delete(fileId);
|
||||
}
|
||||
|
||||
async listFiles(options) {
|
||||
return await this.list(options);
|
||||
}
|
||||
|
||||
async putSetting(key, value) {
|
||||
return await this.put(key, value);
|
||||
}
|
||||
|
||||
async getSetting(key) {
|
||||
return await this.get(key);
|
||||
}
|
||||
|
||||
async deleteSetting(key) {
|
||||
return await this.delete(key);
|
||||
}
|
||||
|
||||
async listSettings(options) {
|
||||
return await this.list(options);
|
||||
}
|
||||
|
||||
async putIndexOperation(operationId, operation) {
|
||||
const key = 'manage@index@operation_' + operationId;
|
||||
return await this.put(key, JSON.stringify(operation));
|
||||
}
|
||||
|
||||
async getIndexOperation(operationId) {
|
||||
const key = 'manage@index@operation_' + operationId;
|
||||
const result = await this.get(key);
|
||||
return result ? JSON.parse(result) : null;
|
||||
}
|
||||
|
||||
async deleteIndexOperation(operationId) {
|
||||
const key = 'manage@index@operation_' + operationId;
|
||||
return await this.delete(key);
|
||||
}
|
||||
|
||||
async listIndexOperations(options) {
|
||||
const listOptions = Object.assign({}, options, {
|
||||
prefix: 'manage@index@operation_'
|
||||
});
|
||||
const result = await this.list(listOptions);
|
||||
|
||||
// 转换格式以匹配D1Database的返回格式
|
||||
const operations = [];
|
||||
for (const item of result.keys) {
|
||||
const operationData = await this.get(item.name);
|
||||
if (operationData) {
|
||||
const operation = JSON.parse(operationData);
|
||||
operations.push({
|
||||
id: item.name.replace('manage@index@operation_', ''),
|
||||
type: operation.type,
|
||||
timestamp: operation.timestamp,
|
||||
data: operation.data,
|
||||
processed: false // KV中没有这个字段,默认为false
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return operations;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 获取数据库实例的便捷函数
|
||||
* 这个函数可以在整个应用中使用,确保一致的数据库访问
|
||||
* @param {Object} env - 环境变量
|
||||
* @returns {Object} 数据库实例
|
||||
*/
|
||||
export function getDatabase(env) {
|
||||
var adapter = createDatabaseAdapter(env);
|
||||
if (!adapter) {
|
||||
throw new Error('Database not configured. Please configure D1 database (env.DB) or KV storage (env.img_url).');
|
||||
}
|
||||
return adapter;
|
||||
}
|
||||
|
||||
/**
|
||||
* 检查数据库配置
|
||||
* @param {Object} env - 环境变量
|
||||
* @returns {Object} 配置信息
|
||||
*/
|
||||
export function checkDatabaseConfig(env) {
|
||||
var hasD1 = env.DB && typeof env.DB.prepare === 'function';
|
||||
var hasKV = env.img_url && typeof env.img_url.get === 'function';
|
||||
|
||||
return {
|
||||
hasD1: hasD1,
|
||||
hasKV: hasKV,
|
||||
usingD1: hasD1,
|
||||
usingKV: !hasD1 && hasKV,
|
||||
configured: hasD1 || hasKV
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* 数据库健康检查
|
||||
* @param {Object} env - 环境变量
|
||||
* @returns {Promise<Object>} 健康检查结果
|
||||
*/
|
||||
export async function healthCheck(env) {
|
||||
var config = checkDatabaseConfig(env);
|
||||
|
||||
if (!config.configured) {
|
||||
return {
|
||||
healthy: false,
|
||||
error: 'No database configured',
|
||||
config: config
|
||||
};
|
||||
}
|
||||
|
||||
try {
|
||||
var db = getDatabase(env);
|
||||
|
||||
if (config.usingD1) {
|
||||
// D1健康检查 - 尝试查询一个简单的表
|
||||
var stmt = db.db.prepare('SELECT 1 as test');
|
||||
await stmt.first();
|
||||
} else {
|
||||
// KV健康检查 - 尝试列出键
|
||||
await db.list({ limit: 1 });
|
||||
}
|
||||
|
||||
return {
|
||||
healthy: true,
|
||||
config: config
|
||||
};
|
||||
} catch (error) {
|
||||
return {
|
||||
healthy: false,
|
||||
error: error.message,
|
||||
config: config
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+242
-272
@@ -1,38 +1,26 @@
|
||||
/* 索引管理器 */
|
||||
/* 索引管理器 - D1数据库版本 */
|
||||
|
||||
import { getDatabase } from './databaseAdapter.js';
|
||||
|
||||
/**
|
||||
* 文件索引结构(分块存储):
|
||||
*
|
||||
* 索引元数据:
|
||||
* - key: manage@index@meta
|
||||
* - value: JSON.stringify(metadata)
|
||||
* - metadata: {
|
||||
* lastUpdated: 1640995200000,
|
||||
* totalCount: 1000,
|
||||
* lastOperationId: "operation_timestamp_uuid",
|
||||
* chunkCount: 3,
|
||||
* chunkSize: 10000
|
||||
* }
|
||||
*
|
||||
* 索引分块:
|
||||
* - key: manage@index_${chunkId} (例如: manage@index_0, manage@index_1, ...)
|
||||
* - value: JSON.stringify(filesChunk)
|
||||
* - filesChunk: [
|
||||
* {
|
||||
* id: "file_unique_id",
|
||||
* metadata: {}
|
||||
* },
|
||||
* ...
|
||||
* ]
|
||||
*
|
||||
* 原子操作结构(保持不变):
|
||||
* - key: manage@index@operation_${timestamp}_${uuid}
|
||||
* - value: JSON.stringify(operation)
|
||||
* 文件索引结构(D1数据库存储):
|
||||
*
|
||||
* 文件表:
|
||||
* - 直接存储在 files 表中,包含所有文件信息和元数据
|
||||
*
|
||||
* 索引元数据表:
|
||||
* - 存储在 index_metadata 表中
|
||||
* - 包含 lastUpdated, totalCount, lastOperationId 等信息
|
||||
*
|
||||
* 原子操作表:
|
||||
* - 存储在 index_operations 表中
|
||||
* - 包含 id, type, timestamp, data, processed 等字段
|
||||
* - operation: {
|
||||
* type: "add" | "remove" | "move" | "batch_add" | "batch_remove" | "batch_move",
|
||||
* timestamp: 1640995200000,
|
||||
* data: {
|
||||
* // 根据操作类型包含不同的数据
|
||||
* fileId: "file_unique_id",
|
||||
* metadata: {}
|
||||
* }
|
||||
* }
|
||||
*/
|
||||
@@ -55,8 +43,9 @@ export async function addFileToIndex(context, fileId, metadata = null) {
|
||||
|
||||
try {
|
||||
if (metadata === null) {
|
||||
// 如果未传入metadata,尝试从KV中获取
|
||||
const fileData = await env.img_url.getWithMetadata(fileId);
|
||||
// 如果未传入metadata,尝试从数据库中获取
|
||||
const db = getDatabase(env);
|
||||
const fileData = await db.getWithMetadata(fileId);
|
||||
metadata = fileData.metadata || {};
|
||||
}
|
||||
|
||||
@@ -96,7 +85,7 @@ export async function batchAddFilesToIndex(context, files, options = {}) {
|
||||
// 如果没有提供metadata,尝试从KV中获取
|
||||
if (!finalMetadata) {
|
||||
try {
|
||||
const fileData = await env.img_url.getWithMetadata(fileId);
|
||||
const fileData = await getDatabase(env).getWithMetadata(fileId);
|
||||
finalMetadata = fileData.metadata || {};
|
||||
} catch (error) {
|
||||
console.warn(`Failed to get metadata for file ${fileId}:`, error);
|
||||
@@ -197,7 +186,7 @@ export async function moveFileInIndex(context, originalFileId, newFileId, newMet
|
||||
if (finalMetadata === null) {
|
||||
// 如果没有提供新metadata,尝试从KV中获取
|
||||
try {
|
||||
const fileData = await env.img_url.getWithMetadata(newFileId);
|
||||
const fileData = await getDatabase(env).getWithMetadata(newFileId);
|
||||
finalMetadata = fileData.metadata || {};
|
||||
} catch (error) {
|
||||
console.warn(`Failed to get metadata for new file ${newFileId}:`, error);
|
||||
@@ -240,7 +229,7 @@ export async function batchMoveFilesInIndex(context, moveOperations) {
|
||||
if (finalMetadata === null || finalMetadata === undefined) {
|
||||
// 如果没有提供新metadata,尝试从KV中获取
|
||||
try {
|
||||
const fileData = await env.img_url.getWithMetadata(newFileId);
|
||||
const fileData = await getDatabase(env).getWithMetadata(newFileId);
|
||||
finalMetadata = fileData.metadata || {};
|
||||
} catch (error) {
|
||||
console.warn(`Failed to get metadata for new file ${newFileId}:`, error);
|
||||
@@ -284,7 +273,7 @@ export async function batchMoveFilesInIndex(context, moveOperations) {
|
||||
* @returns {Object} 合并结果
|
||||
*/
|
||||
export async function mergeOperationsToIndex(context, options = {}) {
|
||||
const { request } = context;
|
||||
const { waitUntil } = context;
|
||||
const { cleanupAfterMerge = true } = options;
|
||||
|
||||
try {
|
||||
@@ -301,11 +290,8 @@ export async function mergeOperationsToIndex(context, options = {}) {
|
||||
}
|
||||
|
||||
// 获取所有待处理的操作
|
||||
const operationsResult = await getAllPendingOperations(context, currentIndex.lastOperationId);
|
||||
|
||||
const operations = operationsResult.operations;
|
||||
const isALLOperations = operationsResult.isAll;
|
||||
|
||||
const operations = await getAllPendingOperations(context, currentIndex.lastOperationId);
|
||||
|
||||
if (operations.length === 0) {
|
||||
console.log('No pending operations to merge');
|
||||
return {
|
||||
@@ -315,7 +301,7 @@ export async function mergeOperationsToIndex(context, options = {}) {
|
||||
};
|
||||
}
|
||||
|
||||
console.log(`Found ${operations.length} pending operations to merge. Is all operations: ${isALLOperations}, if there are remaining operations they will be processed in the next merge.`);
|
||||
console.log(`Found ${operations.length} pending operations to merge`);
|
||||
|
||||
// 按时间戳排序操作,确保按正确顺序应用
|
||||
operations.sort((a, b) => a.timestamp - b.timestamp);
|
||||
@@ -393,8 +379,8 @@ export async function mergeOperationsToIndex(context, options = {}) {
|
||||
workingIndex.lastOperationId = processedOperationIds[processedOperationIds.length - 1];
|
||||
}
|
||||
|
||||
// 保存更新后的索引(使用分块格式)
|
||||
const saveSuccess = await saveChunkedIndex(context, workingIndex);
|
||||
// 保存更新后的索引元数据
|
||||
const saveSuccess = await saveIndexMetadata(context, workingIndex);
|
||||
if (!saveSuccess) {
|
||||
console.error('Failed to save chunked index');
|
||||
return {
|
||||
@@ -408,23 +394,7 @@ export async function mergeOperationsToIndex(context, options = {}) {
|
||||
|
||||
// 清理已处理的操作记录
|
||||
if (cleanupAfterMerge && processedOperationIds.length > 0) {
|
||||
await cleanupOperations(context, processedOperationIds);
|
||||
}
|
||||
|
||||
// 如果未处理完所有操作,调用 merge-operations API 递归处理
|
||||
if (!isALLOperations) {
|
||||
console.log('There are remaining operations, will process them in subsequent calls.');
|
||||
|
||||
const headers = new Headers(request.headers);
|
||||
const originUrl = new URL(request.url);
|
||||
const mergeUrl = `${originUrl.protocol}//${originUrl.host}/api/manage/list?action=merge-operations`;
|
||||
|
||||
await fetch(mergeUrl, { method: 'GET', headers });
|
||||
|
||||
return {
|
||||
success: false,
|
||||
error: 'There are remaining operations, will process them in subsequent calls.'
|
||||
};
|
||||
waitUntil(cleanupOperations(context, processedOperationIds));
|
||||
}
|
||||
|
||||
const result = {
|
||||
@@ -477,19 +447,46 @@ export async function readIndex(context, options = {}) {
|
||||
// 处理目录满足无头有尾的格式,根目录为空
|
||||
const dirPrefix = directory === '' || directory.endsWith('/') ? directory : directory + '/';
|
||||
|
||||
// 处理挂起的操作
|
||||
const mergeResult = await mergeOperationsToIndex(context);
|
||||
if (!mergeResult.success) {
|
||||
throw new Error('Failed to merge operations: ' + mergeResult.error);
|
||||
// 直接从数据库读取文件,不依赖索引
|
||||
const { env } = context;
|
||||
const db = getDatabase(env);
|
||||
|
||||
let allFiles = [];
|
||||
let cursor = null;
|
||||
|
||||
// 分批获取所有文件
|
||||
while (true) {
|
||||
const response = await db.listFiles({
|
||||
limit: 1000,
|
||||
cursor: cursor
|
||||
});
|
||||
|
||||
if (!response || !response.keys || !Array.isArray(response.keys)) {
|
||||
break;
|
||||
}
|
||||
|
||||
for (const item of response.keys) {
|
||||
// 跳过管理相关的键和分块数据
|
||||
if (item.name.startsWith('manage@') || item.name.startsWith('chunk_')) {
|
||||
continue;
|
||||
}
|
||||
|
||||
// 跳过没有元数据的文件
|
||||
if (!item.metadata || !item.metadata.TimeStamp) {
|
||||
continue;
|
||||
}
|
||||
|
||||
allFiles.push({
|
||||
id: item.name,
|
||||
metadata: item.metadata
|
||||
});
|
||||
}
|
||||
|
||||
cursor = response.cursor;
|
||||
if (!cursor) break;
|
||||
}
|
||||
|
||||
// 获取当前索引
|
||||
const index = await getIndex(context);
|
||||
if (!index.success) {
|
||||
throw new Error('Failed to get index');
|
||||
}
|
||||
|
||||
let filteredFiles = index.files;
|
||||
let filteredFiles = allFiles;
|
||||
|
||||
// 目录过滤
|
||||
if (directory) {
|
||||
@@ -568,7 +565,7 @@ export async function readIndex(context, options = {}) {
|
||||
files: resultFiles,
|
||||
directories: Array.from(directories),
|
||||
totalCount: totalCount,
|
||||
indexLastUpdated: index.lastUpdated,
|
||||
indexLastUpdated: Date.now(),
|
||||
returnedCount: resultFiles.length,
|
||||
success: true
|
||||
};
|
||||
@@ -606,30 +603,24 @@ export async function rebuildIndex(context, progressCallback = null) {
|
||||
lastOperationId: null
|
||||
};
|
||||
|
||||
// 分批读取所有文件
|
||||
while (true) {
|
||||
const response = await env.img_url.list({
|
||||
limit: KV_LIST_LIMIT,
|
||||
cursor: cursor
|
||||
});
|
||||
// 从D1数据库读取所有文件
|
||||
const db = getDatabase(env);
|
||||
const filesStmt = db.db.prepare('SELECT id, metadata FROM files WHERE timestamp IS NOT NULL ORDER BY timestamp DESC');
|
||||
const fileResults = await filesStmt.all();
|
||||
|
||||
cursor = response.cursor;
|
||||
for (const row of fileResults) {
|
||||
try {
|
||||
const metadata = JSON.parse(row.metadata || '{}');
|
||||
|
||||
for (const item of response.keys) {
|
||||
// 跳过管理相关的键
|
||||
if (item.name.startsWith('manage@') || item.name.startsWith('chunk_')) {
|
||||
continue;
|
||||
}
|
||||
|
||||
// 跳过没有元数据的文件
|
||||
if (!item.metadata || !item.metadata.TimeStamp) {
|
||||
// 跳过没有时间戳的文件
|
||||
if (!metadata.TimeStamp) {
|
||||
continue;
|
||||
}
|
||||
|
||||
// 构建文件索引项
|
||||
const fileItem = {
|
||||
id: item.name,
|
||||
metadata: item.metadata || {}
|
||||
id: row.id,
|
||||
metadata: metadata
|
||||
};
|
||||
|
||||
newIndex.files.push(fileItem);
|
||||
@@ -639,12 +630,9 @@ export async function rebuildIndex(context, progressCallback = null) {
|
||||
if (progressCallback && processedCount % 100 === 0) {
|
||||
progressCallback(processedCount);
|
||||
}
|
||||
} catch (error) {
|
||||
console.warn(`Failed to parse metadata for file ${row.id}:`, error);
|
||||
}
|
||||
|
||||
if (!cursor) break;
|
||||
|
||||
// 添加协作点
|
||||
await new Promise(resolve => setTimeout(resolve, 10));
|
||||
}
|
||||
|
||||
// 按时间戳倒序排序
|
||||
@@ -652,8 +640,8 @@ export async function rebuildIndex(context, progressCallback = null) {
|
||||
|
||||
newIndex.totalCount = newIndex.files.length;
|
||||
|
||||
// 保存新索引(使用分块格式)
|
||||
const saveSuccess = await saveChunkedIndex(context, newIndex);
|
||||
// 保存新索引元数据
|
||||
const saveSuccess = await saveIndexMetadata(context, newIndex);
|
||||
if (!saveSuccess) {
|
||||
console.error('Failed to save chunked index during rebuild');
|
||||
return {
|
||||
@@ -689,23 +677,65 @@ export async function rebuildIndex(context, progressCallback = null) {
|
||||
*/
|
||||
export async function getIndexInfo(context) {
|
||||
try {
|
||||
const index = await getIndex(context);
|
||||
// 直接从数据库读取文件信息,不依赖索引
|
||||
const { env } = context;
|
||||
const db = getDatabase(env);
|
||||
|
||||
// 检查索引是否成功获取
|
||||
if (index.success === false) {
|
||||
return {
|
||||
success: false,
|
||||
error: 'Failed to retrieve index',
|
||||
message: 'Index is not available or corrupted'
|
||||
let allFiles = [];
|
||||
let cursor = null;
|
||||
|
||||
// 分批获取所有文件
|
||||
while (true) {
|
||||
const response = await db.listFiles({
|
||||
limit: 1000,
|
||||
cursor: cursor
|
||||
});
|
||||
|
||||
if (!response || !response.keys || !Array.isArray(response.keys)) {
|
||||
break;
|
||||
}
|
||||
|
||||
for (const item of response.keys) {
|
||||
// 跳过管理相关的键和分块数据
|
||||
if (item.name.startsWith('manage@') || item.name.startsWith('chunk_')) {
|
||||
continue;
|
||||
}
|
||||
|
||||
// 跳过没有元数据的文件
|
||||
if (!item.metadata || !item.metadata.TimeStamp) {
|
||||
continue;
|
||||
}
|
||||
|
||||
allFiles.push({
|
||||
id: item.name,
|
||||
metadata: item.metadata
|
||||
});
|
||||
}
|
||||
|
||||
cursor = response.cursor;
|
||||
if (!cursor) break;
|
||||
}
|
||||
|
||||
// 如果没有文件,返回空结果
|
||||
if (allFiles.length === 0) {
|
||||
return {
|
||||
success: true,
|
||||
totalFiles: 0,
|
||||
lastUpdated: Date.now(),
|
||||
channelStats: {},
|
||||
directoryStats: {},
|
||||
typeStats: {},
|
||||
oldestFile: null,
|
||||
newestFile: null
|
||||
};
|
||||
}
|
||||
|
||||
// 统计各渠道文件数量
|
||||
const channelStats = {};
|
||||
const directoryStats = {};
|
||||
const typeStats = {};
|
||||
|
||||
index.files.forEach(file => {
|
||||
|
||||
allFiles.forEach(file => {
|
||||
// 渠道统计
|
||||
let channel = file.metadata.Channel || 'Telegraph';
|
||||
if (channel === 'TelegramNew') {
|
||||
@@ -726,15 +756,31 @@ export async function getIndexInfo(context) {
|
||||
typeStats[listType] = (typeStats[listType] || 0) + 1;
|
||||
});
|
||||
|
||||
// 找到最新和最旧的文件
|
||||
let oldestFile = null;
|
||||
let newestFile = null;
|
||||
|
||||
if (allFiles.length > 0) {
|
||||
// 按时间戳排序
|
||||
const sortedFiles = [...allFiles].sort((a, b) => {
|
||||
const timeA = a.metadata.TimeStamp || 0;
|
||||
const timeB = b.metadata.TimeStamp || 0;
|
||||
return timeA - timeB;
|
||||
});
|
||||
|
||||
oldestFile = sortedFiles[0];
|
||||
newestFile = sortedFiles[sortedFiles.length - 1];
|
||||
}
|
||||
|
||||
return {
|
||||
success: true,
|
||||
totalFiles: index.totalCount,
|
||||
lastUpdated: index.lastUpdated,
|
||||
totalFiles: allFiles.length,
|
||||
lastUpdated: Date.now(),
|
||||
channelStats,
|
||||
directoryStats,
|
||||
typeStats,
|
||||
oldestFile: index.files[index.files.length - 1],
|
||||
newestFile: index.files[0]
|
||||
oldestFile,
|
||||
newestFile
|
||||
};
|
||||
} catch (error) {
|
||||
console.error('Error getting index info:', error);
|
||||
@@ -768,9 +814,9 @@ async function recordOperation(context, type, data) {
|
||||
timestamp: Date.now(),
|
||||
data
|
||||
};
|
||||
|
||||
const operationKey = OPERATION_KEY_PREFIX + operationId;
|
||||
await env.img_url.put(operationKey, JSON.stringify(operation));
|
||||
|
||||
const db = getDatabase(env);
|
||||
await db.putIndexOperation(operationId, operation);
|
||||
|
||||
return operationId;
|
||||
}
|
||||
@@ -784,56 +830,27 @@ async function getAllPendingOperations(context, lastOperationId = null) {
|
||||
const { env } = context;
|
||||
|
||||
const operations = [];
|
||||
|
||||
let cursor = null;
|
||||
const MAX_OPERATION_COUNT = 30; // 单次获取的最大操作数量
|
||||
let isALL = true; // 是否获取了所有操作
|
||||
let operationCount = 0;
|
||||
|
||||
|
||||
try {
|
||||
while (true) {
|
||||
const response = await env.img_url.list({
|
||||
prefix: OPERATION_KEY_PREFIX,
|
||||
limit: KV_LIST_LIMIT,
|
||||
cursor: cursor
|
||||
});
|
||||
|
||||
for (const item of response.keys) {
|
||||
// 如果指定了lastOperationId,跳过已处理的操作
|
||||
if (lastOperationId && item.name <= OPERATION_KEY_PREFIX + lastOperationId) {
|
||||
continue;
|
||||
}
|
||||
|
||||
if (operationCount >= MAX_OPERATION_COUNT) {
|
||||
isALL = false; // 达到最大操作数量,停止获取
|
||||
break;
|
||||
}
|
||||
const db = getDatabase(env);
|
||||
const allOperations = await db.listIndexOperations({
|
||||
processed: false,
|
||||
limit: 10000 // 获取所有未处理的操作
|
||||
});
|
||||
|
||||
try {
|
||||
const operationData = await env.img_url.get(item.name);
|
||||
if (operationData) {
|
||||
const operation = JSON.parse(operationData);
|
||||
operation.id = item.name.substring(OPERATION_KEY_PREFIX.length);
|
||||
operations.push(operation);
|
||||
operationCount++;
|
||||
}
|
||||
} catch (error) {
|
||||
isALL = false;
|
||||
console.warn(`Failed to parse operation ${item.name}:`, error);
|
||||
}
|
||||
// 如果指定了lastOperationId,过滤已处理的操作
|
||||
for (const operation of allOperations) {
|
||||
if (lastOperationId && operation.id <= lastOperationId) {
|
||||
continue;
|
||||
}
|
||||
|
||||
cursor = response.cursor;
|
||||
if (!cursor || operationCount >= MAX_OPERATION_COUNT) break;
|
||||
operations.push(operation);
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('Error getting pending operations:', error);
|
||||
}
|
||||
|
||||
return {
|
||||
operations,
|
||||
isAll: isALL,
|
||||
}
|
||||
return operations;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -998,7 +1015,7 @@ function applyBatchMoveOperation(index, data) {
|
||||
}
|
||||
|
||||
/**
|
||||
* 并发清理指定的原子操作记录
|
||||
* 清理已处理的操作记录
|
||||
* @param {Object} context - 上下文对象
|
||||
* @param {Array} operationIds - 要清理的操作ID数组
|
||||
* @param {number} concurrency - 并发数量,默认为10
|
||||
@@ -1009,33 +1026,21 @@ async function cleanupOperations(context, operationIds, concurrency = 10) {
|
||||
try {
|
||||
console.log(`Cleaning up ${operationIds.length} processed operations with concurrency ${concurrency}...`);
|
||||
|
||||
let deletedCount = 0;
|
||||
let errorCount = 0;
|
||||
|
||||
// 创建删除任务数组
|
||||
const deleteTasks = operationIds.map(operationId => {
|
||||
const operationKey = OPERATION_KEY_PREFIX + operationId;
|
||||
return async () => {
|
||||
try {
|
||||
await env.img_url.delete(operationKey);
|
||||
deletedCount++;
|
||||
await getDatabase(env).delete(operationKey);
|
||||
} catch (error) {
|
||||
console.error(`Error deleting operation ${operationId}:`, error);
|
||||
errorCount++;
|
||||
}
|
||||
};
|
||||
});
|
||||
|
||||
// 使用并发控制执行删除操作
|
||||
await promiseLimit(deleteTasks, concurrency);
|
||||
|
||||
console.log(`Successfully cleaned up ${deletedCount} operations, ${errorCount} operations failed.`);
|
||||
return {
|
||||
success: true,
|
||||
deletedCount: deletedCount,
|
||||
errorCount: errorCount,
|
||||
};
|
||||
|
||||
console.log(`Successfully cleaned up ${operationIds.length} operations`);
|
||||
} catch (error) {
|
||||
console.error('Error cleaning up operations:', error);
|
||||
}
|
||||
@@ -1047,7 +1052,7 @@ async function cleanupOperations(context, operationIds, concurrency = 10) {
|
||||
* @returns {Object} 删除结果 { success, deletedCount, errors?, totalFound? }
|
||||
*/
|
||||
export async function deleteAllOperations(context) {
|
||||
const { request, env } = context;
|
||||
const { env } = context;
|
||||
|
||||
try {
|
||||
console.log('Starting to delete all atomic operations...');
|
||||
@@ -1059,12 +1064,18 @@ export async function deleteAllOperations(context) {
|
||||
|
||||
// 首先收集所有操作键
|
||||
while (true) {
|
||||
const response = await env.img_url.list({
|
||||
const response = await getDatabase(env).list({
|
||||
prefix: OPERATION_KEY_PREFIX,
|
||||
limit: KV_LIST_LIMIT,
|
||||
cursor: cursor
|
||||
});
|
||||
|
||||
|
||||
// 检查响应格式
|
||||
if (!response || !response.keys || !Array.isArray(response.keys)) {
|
||||
console.error('Invalid response from database list in cleanupProcessedOperations:', response);
|
||||
break;
|
||||
}
|
||||
|
||||
for (const item of response.keys) {
|
||||
allOperationIds.push(item.name.substring(OPERATION_KEY_PREFIX.length));
|
||||
totalFound++;
|
||||
@@ -1085,31 +1096,11 @@ export async function deleteAllOperations(context) {
|
||||
}
|
||||
|
||||
console.log(`Found ${totalFound} atomic operations to delete`);
|
||||
|
||||
// 限制单次删除的数量
|
||||
const MAX_DELETE_BATCH = 40;
|
||||
const toDeleteOperationIds = allOperationIds.slice(0, MAX_DELETE_BATCH);
|
||||
|
||||
|
||||
// 批量删除原子操作
|
||||
const cleanupResult = await cleanupOperations(context, toDeleteOperationIds);
|
||||
await cleanupOperations(context, allOperationIds);
|
||||
|
||||
// 剩余未删除的操作,调用 delete-operations API 进行递归删除
|
||||
if (allOperationIds.length > MAX_DELETE_BATCH || cleanupResult.errorCount > 0) {
|
||||
console.warn(`Too many operations (${allOperationIds.length}), only deleting first ${cleanupResult.deletedCount}. The remaining operations will be deleted in subsequent calls.`);
|
||||
// 复制请求头,用于鉴权
|
||||
const headers = new Headers(request.headers);
|
||||
|
||||
const originUrl = new URL(request.url);
|
||||
const deleteUrl = `${originUrl.protocol}//${originUrl.host}/api/manage/list?action=delete-operations`
|
||||
|
||||
await fetch(deleteUrl, {
|
||||
method: 'GET',
|
||||
headers: headers
|
||||
});
|
||||
|
||||
} else {
|
||||
console.log(`Delete all operations completed`);
|
||||
}
|
||||
console.log(`Delete all operations completed`);
|
||||
|
||||
} catch (error) {
|
||||
console.error('Error deleting all operations:', error);
|
||||
@@ -1125,8 +1116,8 @@ export async function deleteAllOperations(context) {
|
||||
async function getIndex(context) {
|
||||
const { waitUntil } = context;
|
||||
try {
|
||||
// 首先尝试加载分块索引
|
||||
const index = await loadChunkedIndex(context);
|
||||
// 首先尝试加载索引
|
||||
const index = await loadIndexFromDatabase(context);
|
||||
if (index.success) {
|
||||
return index;
|
||||
} else {
|
||||
@@ -1239,93 +1230,66 @@ async function promiseLimit(tasks, concurrency = BATCH_SIZE) {
|
||||
* @param {Object} index - 完整的索引对象
|
||||
* @returns {Promise<boolean>} 是否保存成功
|
||||
*/
|
||||
async function saveChunkedIndex(context, index) {
|
||||
async function saveIndexMetadata(context, index) {
|
||||
const { env } = context;
|
||||
|
||||
|
||||
try {
|
||||
const files = index.files || [];
|
||||
const chunks = [];
|
||||
|
||||
// 将文件数组分块
|
||||
for (let i = 0; i < files.length; i += INDEX_CHUNK_SIZE) {
|
||||
const chunk = files.slice(i, i + INDEX_CHUNK_SIZE);
|
||||
chunks.push(chunk);
|
||||
}
|
||||
|
||||
// 保存索引元数据
|
||||
const metadata = {
|
||||
lastUpdated: index.lastUpdated,
|
||||
totalCount: index.totalCount,
|
||||
lastOperationId: index.lastOperationId,
|
||||
chunkCount: chunks.length,
|
||||
chunkSize: INDEX_CHUNK_SIZE
|
||||
};
|
||||
|
||||
await env.img_url.put(INDEX_META_KEY, JSON.stringify(metadata));
|
||||
|
||||
// 保存各个分块
|
||||
const savePromises = chunks.map((chunk, chunkId) => {
|
||||
const chunkKey = `${INDEX_KEY}_${chunkId}`;
|
||||
return env.img_url.put(chunkKey, JSON.stringify(chunk));
|
||||
});
|
||||
|
||||
await Promise.all(savePromises);
|
||||
|
||||
console.log(`Saved chunked index: ${chunks.length} chunks, ${files.length} total files`);
|
||||
const db = getDatabase(env);
|
||||
|
||||
// 保存索引元数据到index_metadata表
|
||||
const stmt = db.db.prepare(`
|
||||
INSERT OR REPLACE INTO index_metadata (key, last_updated, total_count, last_operation_id)
|
||||
VALUES (?, ?, ?, ?)
|
||||
`);
|
||||
|
||||
await stmt.bind(
|
||||
'main_index',
|
||||
index.lastUpdated,
|
||||
index.totalCount,
|
||||
index.lastOperationId
|
||||
).run();
|
||||
|
||||
console.log(`Saved index metadata: ${index.totalCount} total files, last updated: ${index.lastUpdated}`);
|
||||
return true;
|
||||
|
||||
|
||||
} catch (error) {
|
||||
console.error('Error saving chunked index:', error);
|
||||
console.error('Error saving index metadata:', error);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 从KV存储加载分块索引
|
||||
* 从D1数据库加载索引
|
||||
* @param {Object} context - 上下文对象,包含 env
|
||||
* @returns {Promise<Object>} 完整的索引对象
|
||||
*/
|
||||
async function loadChunkedIndex(context) {
|
||||
async function loadIndexFromDatabase(context) {
|
||||
const { env } = context;
|
||||
|
||||
|
||||
try {
|
||||
const db = getDatabase(env);
|
||||
|
||||
// 首先获取元数据
|
||||
const metadataStr = await env.img_url.get(INDEX_META_KEY);
|
||||
if (!metadataStr) {
|
||||
const metadataStmt = db.db.prepare('SELECT * FROM index_metadata WHERE key = ?');
|
||||
const metadata = await metadataStmt.bind('main_index').first();
|
||||
|
||||
if (!metadata) {
|
||||
throw new Error('Index metadata not found');
|
||||
}
|
||||
|
||||
const metadata = JSON.parse(metadataStr);
|
||||
const files = [];
|
||||
|
||||
// 并行加载所有分块
|
||||
const loadPromises = [];
|
||||
for (let chunkId = 0; chunkId < metadata.chunkCount; chunkId++) {
|
||||
const chunkKey = `${INDEX_KEY}_${chunkId}`;
|
||||
loadPromises.push(
|
||||
env.img_url.get(chunkKey).then(chunkStr => {
|
||||
if (chunkStr) {
|
||||
return JSON.parse(chunkStr);
|
||||
}
|
||||
return [];
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
const chunks = await Promise.all(loadPromises);
|
||||
|
||||
// 合并所有分块
|
||||
chunks.forEach(chunk => {
|
||||
if (Array.isArray(chunk)) {
|
||||
files.push(...chunk);
|
||||
}
|
||||
});
|
||||
|
||||
// 从files表直接查询所有文件
|
||||
const filesStmt = db.db.prepare('SELECT id, metadata FROM files ORDER BY timestamp DESC');
|
||||
const fileResults = await filesStmt.all();
|
||||
|
||||
const files = fileResults.map(row => ({
|
||||
id: row.id,
|
||||
metadata: JSON.parse(row.metadata || '{}')
|
||||
}));
|
||||
|
||||
const index = {
|
||||
files,
|
||||
lastUpdated: metadata.lastUpdated,
|
||||
totalCount: metadata.totalCount,
|
||||
lastOperationId: metadata.lastOperationId,
|
||||
lastUpdated: metadata.last_updated,
|
||||
totalCount: metadata.total_count,
|
||||
lastOperationId: metadata.last_operation_id,
|
||||
success: true
|
||||
};
|
||||
|
||||
@@ -1358,7 +1322,7 @@ export async function clearChunkedIndex(context, onlyNonUsed = false) {
|
||||
console.log('Starting chunked index cleanup...');
|
||||
|
||||
// 获取元数据
|
||||
const metadataStr = await env.img_url.get(INDEX_META_KEY);
|
||||
const metadataStr = await getDatabase(env).get(INDEX_META_KEY);
|
||||
let chunkCount = 0;
|
||||
|
||||
if (metadataStr) {
|
||||
@@ -1367,7 +1331,7 @@ export async function clearChunkedIndex(context, onlyNonUsed = false) {
|
||||
|
||||
if (!onlyNonUsed) {
|
||||
// 删除元数据
|
||||
await env.img_url.delete(INDEX_META_KEY).catch(() => {});
|
||||
await getDatabase(env).delete(INDEX_META_KEY).catch(() => {});
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1375,12 +1339,18 @@ export async function clearChunkedIndex(context, onlyNonUsed = false) {
|
||||
const recordedChunks = []; // 现有的索引分块键
|
||||
let cursor = null;
|
||||
while (true) {
|
||||
const response = await env.img_url.list({
|
||||
const response = await getDatabase(env).list({
|
||||
prefix: INDEX_KEY,
|
||||
limit: KV_LIST_LIMIT,
|
||||
cursor: cursor
|
||||
});
|
||||
|
||||
|
||||
// 检查响应格式
|
||||
if (!response || !response.keys || !Array.isArray(response.keys)) {
|
||||
console.error('Invalid response from database list in getIndexStorageStats:', response);
|
||||
break;
|
||||
}
|
||||
|
||||
for (const item of response.keys) {
|
||||
recordedChunks.push(item.name);
|
||||
}
|
||||
@@ -1405,13 +1375,13 @@ export async function clearChunkedIndex(context, onlyNonUsed = false) {
|
||||
}
|
||||
|
||||
deletePromises.push(
|
||||
env.img_url.delete(chunkKey).catch(() => {})
|
||||
getDatabase(env).delete(chunkKey).catch(() => {})
|
||||
);
|
||||
}
|
||||
|
||||
if (recordedChunks.includes(INDEX_KEY)) {
|
||||
deletePromises.push(
|
||||
env.img_url.delete(INDEX_KEY).catch(() => {})
|
||||
getDatabase(env).delete(INDEX_KEY).catch(() => {})
|
||||
);
|
||||
}
|
||||
|
||||
@@ -1436,7 +1406,7 @@ export async function getIndexStorageStats(context) {
|
||||
|
||||
try {
|
||||
// 获取元数据
|
||||
const metadataStr = await env.img_url.get(INDEX_META_KEY);
|
||||
const metadataStr = await getDatabase(env).get(INDEX_META_KEY);
|
||||
if (!metadataStr) {
|
||||
return {
|
||||
success: false,
|
||||
@@ -1452,7 +1422,7 @@ export async function getIndexStorageStats(context) {
|
||||
for (let chunkId = 0; chunkId < metadata.chunkCount; chunkId++) {
|
||||
const chunkKey = `${INDEX_KEY}_${chunkId}`;
|
||||
chunkChecks.push(
|
||||
env.img_url.get(chunkKey).then(data => ({
|
||||
getDatabase(env).get(chunkKey).then(data => ({
|
||||
chunkId,
|
||||
exists: !!data,
|
||||
size: data ? data.length : 0
|
||||
|
||||
@@ -111,18 +111,22 @@ async function fetchSampleRate(context) {
|
||||
}
|
||||
}
|
||||
|
||||
// 检查 KV 是否配置,文件索引是否存在
|
||||
export async function checkKVConfig(context) {
|
||||
const { env, waitUntil } = context;
|
||||
import { checkDatabaseConfig as checkDbConfig } from './databaseAdapter.js';
|
||||
|
||||
// 检查 img_url KV 绑定是否存在
|
||||
if (typeof env.img_url == "undefined" || env.img_url == null) {
|
||||
// 检查数据库是否配置,文件索引是否存在
|
||||
async function checkDatabaseConfigMiddleware(context) {
|
||||
var env = context.env;
|
||||
var waitUntil = context.waitUntil;
|
||||
|
||||
var dbConfig = checkDbConfig(env);
|
||||
|
||||
if (!dbConfig.configured) {
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
success: false,
|
||||
error: "KV 数据库未配置 / KV not configured",
|
||||
message: "img_url KV 绑定未找到,请检查您的 KV 配置。 / img_url KV binding not found, please check your KV configuration."
|
||||
}),
|
||||
error: "数据库未配置 / Database not configured",
|
||||
message: "请配置 D1 数据库 (env.DB) 或 KV 存储 (env.img_url)。 / Please configure D1 database (env.DB) or KV storage (env.img_url)."
|
||||
}),
|
||||
{
|
||||
status: 500,
|
||||
headers: {
|
||||
@@ -133,5 +137,9 @@ export async function checkKVConfig(context) {
|
||||
}
|
||||
|
||||
// 继续执行
|
||||
return context.next();
|
||||
}
|
||||
return await context.next();
|
||||
}
|
||||
|
||||
// 保持向后兼容性的别名
|
||||
export const checkKVConfig = checkDatabaseConfigMiddleware;
|
||||
export const checkDatabaseConfig = checkDatabaseConfigMiddleware;
|
||||
@@ -2,32 +2,72 @@ import { getUploadConfig } from '../api/manage/sysConfig/upload';
|
||||
import { getSecurityConfig } from '../api/manage/sysConfig/security';
|
||||
import { getPageConfig } from '../api/manage/sysConfig/page';
|
||||
import { getOthersConfig } from '../api/manage/sysConfig/others';
|
||||
import { getDatabase } from './databaseAdapter.js';
|
||||
|
||||
export async function fetchUploadConfig(env) {
|
||||
const kv = env.img_url;
|
||||
const settings = await getUploadConfig(kv, env);
|
||||
// 去除 已禁用 的渠道
|
||||
settings.telegram.channels = settings.telegram.channels.filter((channel) => channel.enabled);
|
||||
settings.cfr2.channels = settings.cfr2.channels.filter((channel) => channel.enabled);
|
||||
settings.s3.channels = settings.s3.channels.filter((channel) => channel.enabled);
|
||||
try {
|
||||
const db = getDatabase(env);
|
||||
const settings = await getUploadConfig(db, env);
|
||||
// 去除 已禁用 的渠道
|
||||
settings.telegram.channels = settings.telegram.channels.filter((channel) => channel.enabled);
|
||||
settings.cfr2.channels = settings.cfr2.channels.filter((channel) => channel.enabled);
|
||||
settings.s3.channels = settings.s3.channels.filter((channel) => channel.enabled);
|
||||
|
||||
return settings;
|
||||
return settings;
|
||||
} catch (error) {
|
||||
console.error('Failed to fetch upload config:', error);
|
||||
// 返回默认配置
|
||||
return {
|
||||
telegram: { channels: [] },
|
||||
cfr2: { channels: [] },
|
||||
s3: { channels: [] }
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
export async function fetchSecurityConfig(env) {
|
||||
const kv = env.img_url;
|
||||
const settings = await getSecurityConfig(kv, env);
|
||||
return settings;
|
||||
try {
|
||||
const db = getDatabase(env);
|
||||
const settings = await getSecurityConfig(db, env);
|
||||
return settings;
|
||||
} catch (error) {
|
||||
console.error('Failed to fetch security config:', error);
|
||||
// 返回默认配置
|
||||
return {
|
||||
auth: {
|
||||
user: { authCode: "" },
|
||||
admin: { adminUsername: "", adminPassword: "" }
|
||||
},
|
||||
upload: {
|
||||
moderate: { enabled: false, channel: "default", moderateContentApiKey: "", nsfwApiPath: "" }
|
||||
},
|
||||
access: { allowedDomains: "", whiteListMode: false }
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
export async function fetchPageConfig(env) {
|
||||
const kv = env.img_url;
|
||||
const settings = await getPageConfig(kv, env);
|
||||
return settings;
|
||||
try {
|
||||
const db = getDatabase(env);
|
||||
const settings = await getPageConfig(db, env);
|
||||
return settings;
|
||||
} catch (error) {
|
||||
console.error('Failed to fetch page config:', error);
|
||||
// 返回默认配置
|
||||
return { config: [] };
|
||||
}
|
||||
}
|
||||
|
||||
export async function fetchOthersConfig(env) {
|
||||
const kv = env.img_url;
|
||||
const settings = await getOthersConfig(kv, env);
|
||||
return settings;
|
||||
try {
|
||||
const db = getDatabase(env);
|
||||
const settings = await getOthersConfig(db, env);
|
||||
return settings;
|
||||
} catch (error) {
|
||||
console.error('Failed to fetch others config:', error);
|
||||
// 返回默认配置
|
||||
return {
|
||||
telemetry: { enabled: false }
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -4,11 +4,11 @@ import { getTokenPermissions } from '../api/manage/apiTokens.js';
|
||||
/**
|
||||
* 验证API Token权限
|
||||
* @param {Request} request - 请求对象
|
||||
* @param {KVNamespace} kv - KV存储
|
||||
* @param {Object} db - 数据库适配器
|
||||
* @param {string} requiredPermission - 需要的权限 ('upload', 'delete', 'list')
|
||||
* @returns {Promise<{valid: boolean, error?: string}>}
|
||||
*/
|
||||
export async function validateApiToken(request, kv, requiredPermission) {
|
||||
export async function validateApiToken(request, db, requiredPermission) {
|
||||
const authHeader = request.headers.get('Authorization');
|
||||
|
||||
if (!authHeader) {
|
||||
@@ -29,14 +29,15 @@ export async function validateApiToken(request, kv, requiredPermission) {
|
||||
}
|
||||
|
||||
// 获取Token权限
|
||||
const permissions = await getTokenPermissions(kv, token);
|
||||
const permissions = await getTokenPermissions(db, token);
|
||||
|
||||
if (!permissions) {
|
||||
return { valid: false, error: '无效的Token' };
|
||||
}
|
||||
|
||||
// 检查权限
|
||||
if (!permissions.includes(requiredPermission)) {
|
||||
// 如果不需要特定权限(requiredPermission为null),则只要token有效就通过
|
||||
if (requiredPermission !== null && !permissions.includes(requiredPermission)) {
|
||||
return { valid: false, error: `缺少${requiredPermission}权限` };
|
||||
}
|
||||
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { fetchSecurityConfig } from './sysConfig';
|
||||
import { validateApiToken } from './tokenValidator';
|
||||
import { getDatabase } from './databaseAdapter.js';
|
||||
|
||||
/**
|
||||
* 客户端用户认证
|
||||
@@ -11,7 +12,7 @@ import { validateApiToken } from './tokenValidator';
|
||||
*/
|
||||
export async function userAuthCheck(env, url, request, requiredPermission = null) {
|
||||
// 首先使用Token验证
|
||||
const tokenValidation = await validateApiToken(request, env.img_url, requiredPermission);
|
||||
const tokenValidation = await validateApiToken(request, getDatabase(env), requiredPermission);
|
||||
if (tokenValidation.valid) {
|
||||
return true;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,4 @@
|
||||
[[d1_databases]]
|
||||
binding = "DB"
|
||||
database_name = "imgbed-database"
|
||||
database_id = "your-database-id"
|
||||
Reference in New Issue
Block a user