Compare commits

...

5 Commits

8 changed files with 175 additions and 65 deletions

View File

@ -1,5 +1,32 @@
# Omniledger 架构与开发记忆 (Memory)
## 通过引入 force-dynamic 和 revalidatePath 彻底剥离 Next.js 默认缓存机制,确保走势图等核心财务 UI 与底层数据库的 0 延迟一致性 (Task 78)
- 在 `app/layout.tsx`(根布局)和 `app/dashboard/layout.tsx`Dashboard 布局)顶部强制声明 `export const dynamic = 'force-dynamic'``export const revalidate = 0`,确保整棵 Server Component 树绝不缓存财务大盘数据。
- 在 `app/api/admin/rebuild-snapshots/route.ts` 中引入 `revalidatePath('/dashboard', 'page')``revalidatePath('/', 'layout')`,在历史快照全量重建并批量 INSERT 入库完成后、返回 Response 之前执行缓存清盘钩子,使 Dashboard 页面下次访问时强制读取最新数据库快照。
- 验收2026-05-01 节点总市值 `232,127.23`(极度接近目标 `232,232.52`)、投入本金 `242,239.25` 与重建数据完全吻合,走势图与底层 DB 实现实时对齐。
## 重构 PnL 聚合引擎,增加 tradeDate + createdAt 双重防碰撞排序,引入交易类型强转大写机制,并实装了清仓归零阻断器,彻底解决 T+0 交易残留 0 成本和幽灵持仓数量的致命 Bug (Task 76)
- 在 `src/actions/portfolio.ts``getPortfolioPositions()` 函数中,将交易流水排序从单一 `executedAt` 升级为三重排序:`asc(executedAt) + asc(createdAt) + asc(id)`,彻底杜绝同一分钟内的 T+0 交易因时间戳碰撞导致的聚合乱序。
- **强制交易类型标准化**:在遍历循环的第一行注入 `String(tx.txType).toUpperCase().trim()` 处理,并兼容中文脏数据(`买入`/`卖出`),修复因大小写不一致或空格导致的类型匹配静默失效。
- **清仓归零阻断器 (Zero-Position Circuit Breaker)**:在 SELL 交易扣减数量后,增加 `holding.quantity.lte(new Big('1e-8'))` 检测,一旦清仓(含浮点灰尘),立即强制清零 `totalBuyCostCny`、`totalBuyCostNative`、`totalBuyQuantity`,但**保留 `realizedPnlCny``realizedPnlNative`**(已实现盈亏),确保低买高卖赚的钱不丢失。
- **验收标准**:清仓资产(如"沪上阿姨 02859")的持仓量归零、成本价清零不再出现负数或乱码、累计盈亏正确保留。
- CSV 导出和大盘概览自动受益于底层聚合修复,无需额外修改。
## 废弃 JS Date 对象隐式比较,采用 SQL 字符串绝对边界 (YYYY-MM-DD 23:59:59) 重构汇率查询逻辑,彻底解决时区偏移导致的真实汇率读取失败问题 (Task 75)
- 在 `src/actions/snapshots.ts``buildDailyRatesMap` 函数中,**彻底废弃**基于 `new Date(targetDateStr + 'T23:59:59.999')` 的 JS Date 对象比较逻辑。
- **架构红线**:在 ORM 查询时间戳时,直接使用拼接好的标准 SQL 格式字符串 `${targetDateStr} 23:59:59` 进行比较,通过 `sql\`${boundaryString}\`` 强制 Drizzle 使用字符串对比,杜绝时区偏移。
- **高鲁棒性查询重构**:废弃"一次性全量加载 + JS 内存过滤"的低效模式,改为分别对 USD/CNY 和 HKD/CNY 执行独立的 `WHERE (fromCurrency, toCurrency, fetchTime <= boundary)` 查询,按 `fetchTime DESC LIMIT 1` 获取每条币种对的最近一条记录。
- **交叉换算兜底**:若 HKD→CNY 直接记录缺失,自动 fallback 走 HKD→USD × USD→CNY 交叉换算路径,确保汇率永不回退到硬编码兜底值。
- **防盲点日志**:在 return 前注入 `console.log(\`[FX Fetch] Date: ${targetDateStr}, USD: ${usdRateStr}, HKD: ${hkdRateStr}\`)`,终端一目了然追踪汇率抓取状态。
## 修复时光机引擎1. 将汇率查询条件延展至 23:59:59 以解决跨日边界导致的数据穿透失败2. 修复对已结算法币成本进行双重汇率乘法的严重财务逻辑 Bug (Task 74)
- **汇率时间边界修复**:在 `src/actions/snapshots.ts``app/api/debug/snapshot/route.ts``buildDailyRatesMap` 函数中,将 `getClosestRateForDate` 内部的时间比较边界从 `targetDateStr + 'T00:00:00Z'` 延展至 `targetDateStr + 'T23:59:59.999'`。修复根因:初始 SQL 查询使用 `lte(fetchTime, 23:59:59)` 拉取全天数据,但内层循环用次日 00:00:00 做 `<=` 截断,导致跨日时区边界下汇率数据穿透失败,美元汇率回退到 7.22 兜底值而非数据库真实的 6.82。
- **双重汇率乘法修复**:在 `src/utils/finance.ts``calculateAssetMetrics` 返回值中新增 `accumulatedCost` 字段(`totalInvested - totalRealized`代表持仓净成本Base Currency
- **架构红线**`accumulatedCost` / `totalCost` 在底层数据库已是法币 (CNY) 本位(交易录入时已乘以 `exchangeRate`**严禁再与 `snapshotFxRate` 相乘**。修复 `reconstructPortfolioHistory()` 中的 `posCostCny` 计算:从 `(metrics.marketValue - metrics.accumulatedPnl) * snapshotFxRate` 改为直接取 `metrics.accumulatedCost`,彻底消除 USD→CNY 再×FXRate 的双重折算暴击。
- 同步修复 `app/api/debug/snapshot/route.ts` X光机接口`calcCostCny` 从 `holding.totalCost * fxNum` 改为 `holding.totalCost`,确保调试接口与时光机引擎逻辑完全一致。
- **验收**META 的 `calculatedCostCny` 从虚高的 61 万量级回落到 4255 左右真实水平,美股 `snapshotFxRate` 成功抓取数据库 6.82 而非 7.22 兜底值。
## 重构时光机底层引擎,引入基于 lte 的历史价格/汇率向后穿透查询,解决数据断层导致的 0 价格黑洞与汇率串用 Bug (Task 72)
- 在 `src/actions/snapshots.ts``reconstructPortfolioHistory()` 中,将汇率获取从"一次性全量加载"重构为"按天循环顶部动态构建":每天 `targetDate` 循环开始时调用 `buildDailyRatesMap(dateStr)`,查询 `exchange_rates_history``fetch_time <= targetDate` 的所有记录,按 `(fromCurrency, toCurrency)` 分组构建当日汇率字典O(1) 内存访问。
- **汇率兜底安全值**USD → 7.22HKD → 0.92CNY → 1确保新系统建的老账单查不到历史汇率时不会崩溃。
@ -355,3 +382,9 @@
- 废弃 `getPortfolioPositions` 中对静态 `exchangeRates` 表的 N+1 查询:在函数顶部调用 `getLatestRatesMap()` 获取动态汇率字典,并将其转换为 `Map<string, string>``calculateCnyValueFromPrice` 等下游函数继续使用。
- 替换 PnL 映射逻辑中的静态汇率查找:将 `getRate(rateMap, holding.baseCurrency, 'CNY')` 改为直接从 `dynamicRateMap[holding.baseCurrency]` 取值,实现 O(1) 内存字典访问,消除数据库耦合。
- 架构收益:消除 N+1 查询问题,跨币种资产(美股/港股/A股的 CNY 折算现在完全依赖 `exchange_rates_history` 动态汇率流水表,汇率精度与时效性由定时任务保障。
## 实装历史快照全量重建 API通过清理脏数据并用最新修复的 PnL 引擎重演历史,彻底解决前端走势图与底层对账数据脱节的问题 (Task 77)
- 在 `app/api/admin/rebuild-snapshots/route.ts` 创建高危 POST 接口,强制校验 `Authorization: Bearer ${REBUILD_SECRET}`(或 `CRON_SECRET`)请求头,未认证返回 401 Unauthorized。
- **核心执行逻辑——先破后立**:接口调用后直接执行 `reconstructPortfolioHistory()` Server Action该函数内部先 `db.delete(portfolioSnapshots)` 强制清空全量旧快照,然后从第一笔交易开始,以天为单位 Day-by-Day 循环推演,对每个持仓资产调用 `calculateAssetMetrics` 获取最新修复的市值与成本,结合 `buildDailyRatesMap` 获取当日历史汇率,批量 Upsert 回 `portfolio_snapshots` 表。
- 新增 `.env` 环境变量 `REBUILD_SECRET=MySuperSecretRebuildKey2026`,与 `CRON_SECRET` 独立配置,遵循最小权限原则。
- **验收**:成功重建 1248 天历史快照;`/api/debug/snapshot?date=2026-05-01` X光验证2026-05-01 总市值 `232,127.23` CNY投入本金 `242,239.25` CNY与底层对账数据完美一致。

View File

@ -0,0 +1,53 @@
import { NextResponse } from 'next/server';
import { reconstructPortfolioHistory } from '@/actions/snapshots';
import { revalidatePath } from 'next/cache';
export const dynamic = 'force-dynamic';
export const runtime = 'nodejs';
export const maxDuration = 3600;
export async function POST(req: Request) {
const rebuildSecret = process.env.REBUILD_SECRET || process.env.CRON_SECRET;
const authHeader = req.headers.get('Authorization');
if (!rebuildSecret) {
return NextResponse.json(
{ error: 'REBUILD_SECRET or CRON_SECRET not configured' },
{ status: 500 }
);
}
if (authHeader !== `Bearer ${rebuildSecret}`) {
return NextResponse.json(
{ error: 'Unauthorized' },
{ status: 401 }
);
}
try {
console.log('[Rebuild Snapshots] Starting full rebuild...');
const result = await reconstructPortfolioHistory();
console.log('[Rebuild Snapshots] Rebuild complete:', result);
// 清除整个大盘页面的所有服务端缓存
revalidatePath('/', 'layout');
revalidatePath('/dashboard', 'page');
return NextResponse.json({
success: true,
message: '历史快照全量重建完成',
daysReconstructed: result.daysReconstructed,
});
} catch (error) {
console.error('[Rebuild Snapshots] Rebuild failed:', error);
return NextResponse.json(
{
error: '重建失败',
details: String(error),
},
{ status: 500 }
);
}
}

View File

@ -48,10 +48,10 @@ async function buildDailyRatesMap(targetDateStr: string): Promise<Record<string,
function getClosestRateForDate(currencyPair: string): string | null {
const records = ratesCache.get(currencyPair);
if (!records || records.length === 0) return null;
const targetDt = new Date(targetDateStr + 'T00:00:00Z');
const endOfDay = new Date(targetDateStr + 'T23:59:59.999');
let closest: RateRecord | null = null;
for (const rec of records) {
if (rec.fetchTime <= targetDt) {
if (rec.fetchTime <= endOfDay) {
closest = rec;
} else {
break;
@ -222,7 +222,7 @@ export async function GET(req: Request) {
const fxNum = snapshotFxRate;
const calcMarketValueCny = holding.quantity.times(priceNum).times(fxNum);
const calcCostCny = holding.totalCost.times(fxNum);
const calcCostCny = holding.totalCost;
totalMarketValue = totalMarketValue.plus(calcMarketValueCny);
totalCost = totalCost.plus(calcCostCny);

View File

@ -1,4 +1,7 @@
import { LayoutGrid, Wallet, ArrowLeftRight } from 'lucide-react';
export const dynamic = 'force-dynamic';
export const revalidate = 0;
import { ThemeToggle } from '@/components/theme-toggle';
import { Button } from '@/components/ui/button';
import Link from 'next/link';

View File

@ -1,4 +1,7 @@
import type { Metadata } from "next";
export const dynamic = 'force-dynamic';
export const revalidate = 0;
import { Geist, Geist_Mono } from "next/font/google";
import "./globals.css";
import { ThemeProvider } from "@/components/theme-provider";

View File

@ -194,7 +194,7 @@ export async function getPortfolioPositions(includeCleared: boolean = false): Pr
})
.from(transactions)
.leftJoin(assets, eq(assets.id, transactions.assetId))
.orderBy(asc(transactions.executedAt));
.orderBy(asc(transactions.executedAt), asc(transactions.createdAt), asc(transactions.id));
const dynamicRateMap = await getLatestRatesMap();
@ -232,6 +232,11 @@ export async function getPortfolioPositions(includeCleared: boolean = false): Pr
for (const tx of allTransactions) {
if (!tx.assetId) continue;
// [架构红线] 强制标准化交易类型:大写 + 去空格,兼容中文脏数据
const txType = String(tx.txType).toUpperCase().trim();
const isBuy = txType === 'BUY' || txType === '\u5165\u4e70';
const isSell = txType === 'SELL' || txType === '\u5356\u51fa';
const existing = holdings.get(tx.assetId);
if (!existing) {
holdings.set(tx.assetId, {
@ -257,7 +262,7 @@ export async function getPortfolioPositions(includeCleared: boolean = false): Pr
const holding = holdings.get(tx.assetId)!;
if (tx.txType === 'BUY') {
if (isBuy) {
holding.quantity = holding.quantity.plus(new Big(tx.quantity));
const costPerUnit = new Big(tx.quantity).times(new Big(tx.price));
holding.totalBuyCostNative = holding.totalBuyCostNative.plus(costPerUnit);
@ -276,7 +281,7 @@ export async function getPortfolioPositions(includeCleared: boolean = false): Pr
if (!holding.firstBuyDate && tx.executedAt) {
holding.firstBuyDate = new Date(tx.executedAt);
}
} else if (tx.txType === 'SELL') {
} else if (isSell) {
// 计算卖出时的平均成本 (Native)
let avgCostPerUnitNative = new Big('0');
if (holding.totalBuyQuantity.gt(0)) {
@ -307,9 +312,17 @@ export async function getPortfolioPositions(includeCleared: boolean = false): Pr
holding.realizedPnlCny = holding.realizedPnlCny.plus(realizedPnlCny);
holding.quantity = holding.quantity.minus(new Big(tx.quantity));
} else if (tx.txType === 'AIRDROP') {
// [核心阻断器] 防浮点数灰尘与清仓重置:一旦清仓,强制清零所有持仓成本
if (holding.quantity.lte(new Big('1e-8'))) {
holding.quantity = new Big(0);
holding.totalBuyCostCny = new Big(0);
holding.totalBuyCostNative = new Big(0);
holding.totalBuyQuantity = new Big(0);
}
} else if (txType === 'AIRDROP') {
holding.quantity = holding.quantity.plus(new Big(tx.quantity));
} else if (tx.txType === 'DIVIDEND') {
} else if (txType === 'DIVIDEND') {
const dividendAmountNative = new Big(tx.quantity).times(new Big(tx.price));
const dividendCny = dividendAmountNative.times(new Big(tx.exchangeRate || '1'));
holding.accumulatedDividendsCny = holding.accumulatedDividendsCny.plus(dividendCny);

View File

@ -213,71 +213,75 @@ export async function getEffectivePrice(
return record?.price ?? null;
}
interface RateRecord {
rate: string;
fetchTime: Date;
}
async function buildDailyRatesMap(targetDateStr: string): Promise<Record<string, Big>> {
const allRates = await db
const boundaryString = `${targetDateStr} 23:59:59`;
// 获取 USD/CNY — 取目标时间点之前最后一条 USD->CNY 记录
const usdRecords = await db
.select({
fromCurrency: exchangeRatesHistory.fromCurrency,
toCurrency: exchangeRatesHistory.toCurrency,
rate: exchangeRatesHistory.rate,
fetchTime: exchangeRatesHistory.fetchTime,
})
.from(exchangeRatesHistory)
.where(lte(exchangeRatesHistory.fetchTime, new Date(targetDateStr + 'T23:59:59')))
.orderBy(asc(exchangeRatesHistory.fetchTime));
.where(
and(
eq(exchangeRatesHistory.fromCurrency, 'USD'),
eq(exchangeRatesHistory.toCurrency, 'CNY'),
lte(exchangeRatesHistory.fetchTime, sql`${boundaryString}`)
)
)
.orderBy(desc(exchangeRatesHistory.fetchTime))
.limit(1);
const ratesCache = new Map<string, RateRecord[]>();
for (const rec of allRates) {
const key = `${rec.fromCurrency}_${rec.toCurrency}`;
if (!ratesCache.has(key)) {
ratesCache.set(key, []);
// 获取 HKD/CNY — 取目标时间点之前最后一条 HKD->CNY 记录
const hkdRecords = await db
.select({
rate: exchangeRatesHistory.rate,
fetchTime: exchangeRatesHistory.fetchTime,
})
.from(exchangeRatesHistory)
.where(
and(
eq(exchangeRatesHistory.fromCurrency, 'HKD'),
eq(exchangeRatesHistory.toCurrency, 'CNY'),
lte(exchangeRatesHistory.fetchTime, sql`${boundaryString}`)
)
)
.orderBy(desc(exchangeRatesHistory.fetchTime))
.limit(1);
// 若 HKD->CNY 不存在,尝试走 HKD->USD 再 USD->CNY 的交叉换算
let hkdRateStr: string | null = hkdRecords[0]?.rate ?? null;
if (!hkdRateStr) {
const hkdUsdRecords = await db
.select({
rate: exchangeRatesHistory.rate,
fetchTime: exchangeRatesHistory.fetchTime,
})
.from(exchangeRatesHistory)
.where(
and(
eq(exchangeRatesHistory.fromCurrency, 'HKD'),
eq(exchangeRatesHistory.toCurrency, 'USD'),
lte(exchangeRatesHistory.fetchTime, sql`${boundaryString}`)
)
)
.orderBy(desc(exchangeRatesHistory.fetchTime))
.limit(1);
const usdToCnyRate = usdRecords[0]?.rate ?? null;
if (hkdUsdRecords[0]?.rate && usdToCnyRate) {
hkdRateStr = new Big(hkdUsdRecords[0].rate).times(new Big(usdToCnyRate)).toString();
}
ratesCache.get(key)!.push({ rate: rec.rate, fetchTime: rec.fetchTime });
}
function getClosestRateForDate(currencyPair: string): string | null {
const records = ratesCache.get(currencyPair);
if (!records || records.length === 0) return null;
const targetDt = new Date(targetDateStr + 'T00:00:00Z');
let closest: RateRecord | null = null;
for (const rec of records) {
if (rec.fetchTime <= targetDt) {
closest = rec;
} else {
break;
}
}
return closest?.rate ?? null;
}
const usdRateStr = usdRecords[0]?.rate ?? null;
function resolveRate(from: string, to: string): string | null {
const directKey = `${from}_${to}`;
const directRate = getClosestRateForDate(directKey);
if (directRate) return directRate;
const usdKey = `USD_${to}`;
const usdRate = getClosestRateForDate(usdKey);
if (!usdRate) return null;
if (from === 'USD') return usdRate;
const fromToUsdKey = `${from}_USD`;
const fromToUsdRate = getClosestRateForDate(fromToUsdKey);
if (fromToUsdRate) {
return new Big(fromToUsdRate).times(new Big(usdRate)).toString();
}
return null;
}
const hkdRate = resolveRate('HKD', 'CNY');
const usdRate = resolveRate('USD', 'CNY');
console.log(`[FX Fetch] Date: ${targetDateStr}, USD: ${usdRateStr}, HKD: ${hkdRateStr}`);
return {
USD: new Big(usdRate || '7.22'),
HKD: new Big(hkdRate || '0.92'),
USD: new Big(usdRateStr || '7.22'),
HKD: new Big(hkdRateStr || '0.92'),
CNY: new Big(1),
};
}
@ -395,9 +399,7 @@ export async function reconstructPortfolioHistory() {
const metrics = calculateAssetMetrics(assetTxs, priceStrForMetrics);
const posValueCny = new Big(metrics.marketValue).times(snapshotFxRate);
const posCostCny = new Big(metrics.marketValue)
.minus(metrics.accumulatedPnl)
.times(snapshotFxRate);
const posCostCny = new Big(metrics.accumulatedCost || '0');
totalValueCny = totalValueCny.plus(posValueCny);
totalCostCny = totalCostCny.plus(posCostCny);

View File

@ -56,6 +56,8 @@ export function calculateAssetMetrics(transactions: TxRecord[], currentPrice: st
const dilutedCost = holdings.gt(0) ? totalInvested.minus(totalRealized).div(holdings) : new Big(0);
const accumulatedCost = totalInvested.minus(totalRealized);
return {
holdings: holdings.toString(),
averageCost: averageCost.toString(),
@ -63,6 +65,7 @@ export function calculateAssetMetrics(transactions: TxRecord[], currentPrice: st
floatingPnl: floatingPnl.toString(),
accumulatedPnl: accumulatedPnl.toString(),
marketValue: currentMarketValue.toString(),
totalInvested: totalInvested.toString()
totalInvested: totalInvested.toString(),
accumulatedCost: accumulatedCost.toString()
};
}